CN112637529B - Video processing method and device, storage medium and electronic equipment - Google Patents
Video processing method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN112637529B CN112637529B CN202011506301.3A CN202011506301A CN112637529B CN 112637529 B CN112637529 B CN 112637529B CN 202011506301 A CN202011506301 A CN 202011506301A CN 112637529 B CN112637529 B CN 112637529B
- Authority
- CN
- China
- Prior art keywords
- sound
- video
- target
- source
- different
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
- H04N5/60—Receiver circuitry for the reception of television signals according to analogue transmission standards for the sound signals
- H04N5/607—Receiver circuitry for the reception of television signals according to analogue transmission standards for the sound signals for more than one sound signal, e.g. stereo, multilanguages
Abstract
The embodiment of the application discloses a video processing method, a video processing device, an electronic device and a storage medium, wherein the video processing method is applied to the electronic device, the electronic device is provided with a camera, and the video processing method comprises the following steps: starting the camera to record the video to obtain a video source; in the video recording process, collecting sound signals of target sound sources in different directions, and recording the collected sound signals in different sound channels according to different target sound sources; and synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video. In this embodiment of the application, directional collection has been carried out to the direction that different sound sources are located to with the sound signal record of different sound sources at different sound channels, thereby avoided the sound signal stack of different directions, make the crosstalk between the sound signal of gathering littleer, effectively improved the recording effect when electronic equipment video recording.
Description
Technical Field
The present disclosure relates to the field of audio and video processing technologies, and in particular, to a video processing method, a video processing device, a storage medium, and an electronic device.
Background
With the development of technology, functions of electronic devices are becoming more and more abundant, and users can implement various functions through application programs previously installed in the electronic devices. For example, a user may implement a video recording function through an electronic device.
When video is recorded through the electronic equipment, video signals are collected through a camera of the electronic equipment on one hand, and sound signals are collected through a microphone of the electronic equipment on the other hand. However, various sounds are mixed together during video recording, and influence and interfere with each other, so that larger crosstalk is brought, and the recording effect is poor.
Disclosure of Invention
The embodiment of the application provides a video processing method, a video processing device, a storage medium and electronic equipment, which can improve the recording effect during video recording.
The embodiment of the application provides a video processing method, which is applied to electronic equipment, wherein the electronic equipment is provided with a camera, and the video processing method comprises the following steps:
starting the camera to record the video to obtain a video source;
in the video recording process, collecting sound signals of target sound sources in different directions, and recording the collected sound signals in different sound channels according to different target sound sources;
And synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video.
The embodiment of the application also provides a video processing device, which is applied to electronic equipment, wherein, the electronic equipment is provided with a camera, and the video processing device comprises:
the recording module is used for starting the camera to record the video to obtain a video source;
the acquisition module is used for acquiring sound signals of target sound sources in different directions in the video recording process and recording the acquired sound signals in different sound channels according to different target sound sources;
and the synthesis module is used for synthesizing the sound signals recorded in different sound channels with the recorded video source to obtain the target video.
The embodiment of the application also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program runs on a computer, the computer program causes the computer to execute the steps in any video recording processing method provided by the embodiment of the application.
The embodiment of the application also provides electronic equipment, wherein the electronic equipment is provided with a camera, the electronic equipment comprises a processor and a memory, the memory stores a computer program, and the processor executes steps in any video processing method provided by the embodiment of the application by calling the computer program stored in the memory.
In the embodiment of the application, the electronic equipment is provided with the camera, and the camera is started to record video to obtain a video source; meanwhile, in the video recording process, sound signals of target sound sources in different directions are collected, and the collected sound signals are recorded in different sound channels according to different target sound sources; and then synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video. In this embodiment of the application, directional collection has been carried out to the direction that different sound sources are located to with the sound signal record of different sound sources at different sound channels, thereby avoided the sound signal stack of different directions, make the crosstalk between the sound signal of gathering littleer, effectively improved the recording effect when electronic equipment video recording.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a video processing method according to an embodiment of the present application.
Fig. 2 is a schematic diagram of a first video recording according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a second video recording according to an embodiment of the present application.
Fig. 4 is a schematic diagram of a microphone position according to an embodiment of the present application.
Fig. 5 is a second flowchart of a video processing method according to an embodiment of the present application.
Fig. 6 is a first playback schematic diagram of a target video according to an embodiment of the present application.
Fig. 7 is a second playback schematic diagram of a target video according to an embodiment of the present application.
Fig. 8 is a third playback schematic diagram of a target video according to an embodiment of the present application.
Fig. 9 is a schematic diagram of a first configuration of a video processing apparatus according to an embodiment of the present application.
Fig. 10 is a schematic diagram of a second structure of a video processing apparatus according to an embodiment of the present application.
Fig. 11 is a schematic diagram of a first structure of an electronic device according to an embodiment of the present application.
Fig. 12 is a schematic diagram of a second structure of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All embodiments obtained by a person skilled in the art without any inventive effort on the basis of the embodiments in this application fall within the scope of protection of the present invention.
The terms first, second, third and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the objects so described may be interchanged where appropriate. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, or apparatus, terminal, system comprising a series of steps is not necessarily limited to those steps or modules and units explicitly listed and may include steps or modules or units not explicitly listed or may include other steps or modules or units inherent to such process, method, apparatus, terminal, or system.
The embodiment of the application firstly provides a video processing method. The main execution body of the video processing method may be a video processing device provided in the embodiment of the present application, or an electronic device integrated with the video processing device, where the video processing device may be implemented in a hardware or software manner, and the electronic device may be a device with a processing capability and configured with a processor, such as a smart phone, a tablet computer, a palm computer, a notebook computer, or a desktop computer, and a camera is provided on the electronic device.
Referring to fig. 1, fig. 1 is a flow chart of a video processing method according to an embodiment of the present application. The video processing method is applied to the electronic equipment provided by the embodiment of the application, and the electronic equipment is provided with the camera. The video processing method provided by the embodiment of the application can comprise the following steps:
101, starting a camera to record video, and obtaining a video source.
The electronic equipment is provided with cameras, and the number of the cameras can be one, two or more. The camera may include a front camera disposed on a front side (a side on which the display screen is disposed) of the electronic device and/or a rear camera disposed on a rear side (a side opposite to the front side) of the electronic device. For video recording, the camera that is turned on may be one camera in the electronic device, or two or more cameras in different positions.
Referring to fig. 2, fig. 2 is a schematic diagram of a first video recording according to an embodiment of the present application. The electronic device may use one camera to record video, for example, only use the front camera or the rear camera to record video, and a preview image obtained by recording is displayed on the screen.
With continued reference to fig. 3, fig. 2 is a schematic diagram of a second video recording method according to an embodiment of the present application. The electronic device may also use more than one camera for synchronized video recording. When more than one camera is used for synchronous video recording, more than one preview picture obtained during recording can be displayed on a screen at the same time in a split-screen display mode. For example, the front camera and the rear camera may be used simultaneously for video recording.
That is, the step of starting the camera to record the video to obtain the video source may include: and starting the front camera and the rear camera to record video, so as to obtain a first video source recorded by the front camera and a second video source recorded by the rear camera. The preview picture a may be a preview picture of a first video source recorded by a front camera, and the preview picture B may be a preview picture of a second video source recorded by a rear camera.
It should be noted that, the number and positions of cameras for recording video are not limited in the embodiments of the present application, and other video recording modes realized by changing the number and/or positions of the cameras except the above-mentioned number and positions are all within the protection scope of the embodiments of the present application.
102, in the video recording process, collecting sound signals of target sound sources in different directions, and recording the collected sound signals in different channels according to different target sound sources.
In the video recording process, the electronic equipment receives sound signals from all directions, sound sources in different directions are positioned according to the received sound signals, and a target sound source is determined from the positioned sound sources in different directions.
In an embodiment, the electronic device is provided with a plurality of microphones, and the step of locating sound sources in different directions according to the received sound signals may include:
acquiring time differences of arrival of each sound signal at a plurality of microphones;
and locating a sound source corresponding to each sound signal according to the time difference of arrival of each sound signal at the plurality of microphones.
Referring to fig. 4, fig. 4 is a schematic diagram of a microphone position according to an embodiment of the present application. Fig. 4 illustrates the locations where microphones may be located from a number of angles. An omnidirectional microphone can be arranged on the top, the bottom, the front and the back of the electronic equipment, a Beam Forming (BF) microphone array is formed by three or more microphones, after sound signals from all directions are received, the time difference of each sound signal reaching a plurality of microphones is obtained, so that the time difference of arrival (Time Difference of Arrival, TODA) of each sound signal is obtained, and the azimuth information of a corresponding sound source relative to a recording origin is obtained through the time difference of arrival, so that the specific position of the sound source is located.
In the embodiment of the present application, the arrangement positions of the plurality of microphones are not limited to the manner shown in fig. 4. By changing the setting position of the microphone, sound source localization in any direction can be realized, and the number of the sound sources in different directions can be two or more.
When the electronic equipment localizes two sound sources in different directions, the localized two sound sources in different directions are directly used as target sound sources; when the electronic device locates sound sources in different directions, the located sound sources in different directions can be used as target sound sources, and the target sound sources can be determined from the located sound sources in different directions.
In an embodiment, the step of determining the target sound source from the localized sound sources of the plurality of different directions may include:
determining a preset recording range;
acquiring a horizontal angle and a pitching angle of each sound source relative to the electronic equipment;
judging whether each sound source is positioned in a preset recording range according to the horizontal angle and the pitching angle of each sound source relative to the electronic equipment;
and determining the sound source in the preset recording range as a target sound source.
The preset recording range comprises a preset horizontal angle range and a preset pitching angle range. And determining the sound source as a target sound source when the horizontal angle of the sound source relative to the electronic equipment is within a preset horizontal angle range and the pitching angle of the sound source relative to the electronic equipment is within a preset pitching angle range. For example, the preset recording range may be horizontal angle-30 deg., pitch angle-15 deg. + -20 deg.. The user can manually adjust the angle of the electronic equipment according to the needs, so that the sound source needing to be subjected to key recording is located in a preset recording range.
After the target sound source is determined, sound signals of the target sound source in different directions are collected by microphones at different positions. For example, the plurality of microphones of the electronic device may include a front microphone and a rear microphone, and the determined target sound sources in different directions include a front sound source and a rear sound source. And collecting the sound signals of the front sound source by using the front microphone, and collecting the sound signals of the rear sound source by using the rear microphone.
In an embodiment, when collecting the sound signals of the target sound sources in different directions, blind source separation is performed on the sound signals of other sound sources except the target sound source, and the sound signals of the target sound source to be collected are amplified. For example, when the front microphone is used to collect the sound signals of the front sound source, the sound signals of the other sound sources except the front sound source are blind-source separated, and the sound signals of the front sound source are amplified. Therefore, the directional collection of the sound signals of the target sound source is realized, so that the collected sound signals are clean and free from noise. And directional collection is carried out on sound in a single direction, sounds in other directions are not overlapped, and other editing of recorded audio and corresponding audio by a subsequent user is facilitated.
After the sound signals of the target sound sources in different directions are collected, the collected sound signals are recorded in different sound channels according to the different target sound sources. For example, when the target sound source is a front sound source and a rear sound source, the collected sound signals of the front sound source may be recorded in the first channel, and the collected sound signals of the rear sound source may be recorded in the second channel.
The electronic device may have two channels or may have multiple channels. For electronic devices with dual channels, the first and second channels may be referred to as left and right channels.
The sound signals collected from the target sound sources in different directions are recorded in different sound channels, so that superposition of the sound signals in different directions is avoided. Different sound channels play an isolated role on sound signals, so that crosstalk among collected sound signals is smaller, and recording effect is clearer.
And 103, synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video.
The recorded video sources can be one, two or more, and when the video is recorded, the collected sound signals are recorded in at least two channels. That is, the target video is synthesized from sound signals of at least two channels and at least one video source.
At least two speakers are provided on the electronic device. After the target video is obtained, the target video is played through at least two speakers. When the target video is played, the same loudspeaker only plays the sound of one sound channel, and the sound playing effect of the target video is clear and does not contain noise because the recording effect of the sound signal of each sound channel is clear.
In an embodiment, when the synthesis of the sound signal and the video source is performed, each video source and the sound signal of the corresponding channel may be separately encoded to obtain at least two target videos. For example, synthesizing a sound signal of a front sound source recorded in a first sound channel with a first video source recorded by a front camera to obtain a first target video; and synthesizing the sound signal of the rear sound source recorded in the second channel with a second video source recorded by a rear camera to obtain a second target video.
And respectively playing at least two target videos obtained by adopting an independent coding mode. Wherein, when each target video is played, all speakers play the same sound. For example, when playing a first target video, all speakers play sound signals of a first channel; when playing the second target video, all speakers play the sound signal of the second sound. For the first target video, all the speakers play sound signals collected in a front sound source direction, and for the second target video, all the speakers play sound signals collected in a rear sound source direction.
Therefore, the sounds of the first target video and the second target video, which are obtained by adopting the independent coding mode, are the results of directional acquisition of sound sources at corresponding positions, and the played sounds are clean, clear and free of noise, and do not overlap with sounds in other directions.
In an embodiment, when the synthesis of the sound signal and the video source is performed, all the video sources and all the sound signals recorded in different channels can be encoded together to obtain a target video. Wherein, when the common encoding is performed, the number of the video sources participating in the synthesis can be one, two or more. When more than one video source is used for synthesis, all the video sources are combined into video pictures of the target video, and sound signals of different channels are respectively stored as different channels of the target video.
For example, the number of video sources involved in the composition may be two, including a first video source and a second video source. And combining the sound signal of the front sound source recorded in the first sound channel, the sound signal of the rear sound source recorded in the second sound channel, the first video source recorded by the front camera and the second video source recorded by the rear camera into a third target video.
For a target video obtained by adopting a common coding mode, if a video picture is synthesized by at least two video sources, when the target video is played, the pictures of the at least two video sources are displayed on a screen in a split screen mode, each loudspeaker only plays sound signals of one sound channel, and different loudspeakers play sound signals of different sound channels.
Therefore, in the third target video obtained by adopting the common coding mode, the sound played by each loudspeaker is the result of directional acquisition of sound sources at different positions, the sound played by each loudspeaker is clean, clear and free from noise, and the sounds in other directions are not overlapped. The sound signals recorded by different channels are played by different loudspeakers, so that a stereoscopic sound field is realized together, and the immersion experience is strong when a user watches videos. In addition, the synchronous recording, splicing and playing of different sound channels of front and rear videos are realized by combining split screen display, the synchronous shooting and sharing of front and rear cameras are realized, and the playing audio and video recording, live broadcast explanation, interactive video, personal travel and the like of the home and pets can be realized in another self-timer mode without second ginseng and.
For another example, the video source involved in the composition may be one. When the recorded video source is one (for example, the recorded video source can be recorded by a front camera or a rear camera), the sound signal of the front sound source recorded in the first channel, the sound signal of the rear sound source recorded in the second channel and the recorded video source are combined together into a fourth target video when video synthesis is performed.
For a target video obtained by adopting a common coding mode, when the target video is played, each loudspeaker only plays the sound signal of one sound channel, and different loudspeakers play the sound signals of different sound channels.
Therefore, in the fourth target video obtained by adopting the common coding mode, the sound played by each loudspeaker is the result of directional acquisition of sound sources at different positions, the sound played by each loudspeaker is clean, clear and free from noise, and the sounds in other directions are not overlapped. The sound signals recorded by different channels are played by different loudspeakers, so that a stereoscopic sound field is realized together, and the immersion experience is strong when a user watches videos.
The method described in the previous embodiment is described in further detail below by way of example.
In this embodiment, an example will be described in which two speakers are provided in an electronic device, a front camera and a rear camera are turned on to perform synchronous recording, and a collected target sound source is a front sound source and a rear sound source. It should be noted that, in the embodiment of the present application, the left channel and the right channel are merely used to distinguish different channels in the dual channels, and in a specific implementation, the implementation manner of the technical features related to the left channel may be changed to the right channel, and meanwhile, the implementation manner of the technical features related to the right channel may be changed to the left channel.
With continued reference to fig. 5, fig. 5 is a schematic diagram of a second flow chart of a video processing method according to an embodiment of the present application. The video processing method is applied to the electronic equipment provided by the embodiment of the application, and the electronic equipment is provided with the camera. The video processing method can comprise the following steps:
201. and starting the front camera and the rear camera to record video, so as to obtain a first video source recorded by the front camera and a second video source recorded by the rear camera.
The camera can include the front camera that sets up in electronic equipment front (the face that the display screen is located) and/or set up in the rear camera of electronic equipment back (the face opposite with the front), and electronic equipment can open front camera and rear camera and carry out synchronous video recording. When the front camera and the rear camera are used for synchronous video recording, more than one preview picture obtained during recording can be simultaneously displayed on a screen in a split-screen display mode.
202. In the video recording process, a front microphone is used for collecting sound signals of a front sound source, and the collected sound signals of the front sound source are recorded in a left channel; and acquiring the sound signal of the rear sound source by using the rear microphone, and recording the acquired sound signal of the rear sound source in a right channel.
During video recording, the electronic device receives sound signals from all directions, and positions a front sound source and a rear sound source according to the received sound signals.
Referring to fig. 4, fig. 4 is a schematic diagram of a microphone position according to an embodiment of the present application. Fig. 4 illustrates the locations where microphones may be located from a number of angles. An omnidirectional microphone can be arranged on the top, the bottom, the front and the back of the electronic equipment, a Beam Forming (BF) microphone array is formed by three or more microphones, after sound signals from all directions are received, the time difference of each sound signal reaching a plurality of microphones is obtained, the arrival time difference (Time Difference of Arrival, TODA) of each sound signal is obtained, and the azimuth information of a corresponding sound source relative to a recording origin is obtained through the arrival time difference, so that a front sound source and a rear sound source are determined.
The microphones of the electronic equipment comprise a front microphone and a rear microphone, after a front sound source and a rear sound source are determined, the front microphone is utilized to collect sound signals of the front sound source, and the rear microphone is utilized to collect sound signals of the rear sound source.
In one embodiment, when the front microphone is used to collect the sound signals of the front sound source, the sound signals of the other sound sources except the front sound source are subjected to blind source separation, and the sound signals of the front sound source are amplified. When the sound signals of the rear sound source are collected by the rear microphone, the sound signals of the other sound sources except the rear sound source are blind-source separated, and the sound signals of the rear sound source are amplified.
Therefore, the directional collection of the sound signals of the target sound source is realized, so that the collected sound signals are clean and free from noise. And directional collection is carried out on sound in a single direction, sounds in other directions are not overlapped, and other editing of recorded audio and corresponding audio by a subsequent user is facilitated.
After the sound signals of the front sound source and the sound signals of the rear sound source are collected, the collected sound signals of the front sound source are recorded in the left channel, and the collected sound signals of the rear sound source are recorded in the right channel.
The sound signals collected from the front sound source and the rear sound source are recorded in different sound channels, so that superposition of sound signals in different directions is avoided. Different sound channels play an isolated role on sound signals, so that crosstalk among collected sound signals is smaller, and recording effect is clearer.
After recording the collected sound signals of the front sound source in the left channel and the collected sound signals of the rear sound source in the right channel, the synthesis of the sound signals and the video source is performed based on the sound signals of different directions recorded in different channels.
When synthesizing the sound signal and the video sources, each video source can be independently encoded with the sound signal of the corresponding sound channel respectively to obtain two target videos; all video sources and all sound signals recorded in different channels can be encoded together to obtain a target video. For at least two target videos obtained by adopting an independent coding mode, respectively playing the at least two target videos;
in the following steps, steps 203 to 206 are independent codes, and steps 207 to 208 are common codes.
At least two speakers are provided on the electronic device. After the target video is obtained, the target video is played through at least two speakers.
203. And synthesizing the sound signal of the left channel and the first video source to obtain a first target video.
204. Playing a first target video, wherein when the first target video is played, both speakers play sound signals of a left channel;
referring to fig. 6, fig. 6 is a first playback schematic diagram of a target video according to an embodiment of the present application. In one embodiment, the audio signal recorded in the left channel and the first video source are independently encoded to obtain a first target video.
When the first target video is played, the two loudspeakers play the same sound, and the sound signals recorded in the left channel after the front sound source is directionally collected.
Therefore, in the first target video obtained by adopting the independent coding mode, the played picture is a front video picture recorded by the front camera, and the played sound is collected by orienting the front sound source. The picture is synchronous with the sound, and the played sound is clean, clear and free from noise, and the sounds in other directions are not overlapped.
205. And synthesizing the sound signal of the right channel and the second video source to obtain a second target video.
206. And playing the second target video, wherein when the second target video is played, the two speakers play the sound signals of the right channel.
Referring to fig. 7, fig. 7 is a second playback schematic diagram of a target video according to an embodiment of the present application. In one embodiment, the audio signal recorded in the right channel and the second video source are encoded independently to obtain the second target video.
When the second target video is played, the two loudspeakers play the same sound, and the sound signals recorded in the right channel after the rear sound source is directionally collected.
Therefore, in the second target video obtained by adopting the independent coding mode, the played picture is a rear video picture recorded by the rear camera, and the played sound is collected by orienting the rear sound source. The picture is synchronous with the sound, and the played sound is clean, clear and free from noise, and the sounds in other directions are not overlapped.
207. And synthesizing the sound signals of the left channel, the sound signals of the right channel, the first video source and the second video source into a third target video.
208. And playing the third target video, wherein when the third target video is played, the pictures of the first video source and the second video source are displayed on the screen in a split mode, and the two speakers play the sound signals of the left channel and the sound signals of the right channel respectively.
Referring to fig. 8, fig. 8 is a third playback schematic diagram of a target video according to an embodiment of the present application. In an embodiment, the sound signal of the front sound source recorded in the left channel, the sound signal of the rear sound source recorded in the right channel, the first video source recorded by the front camera, and the second video source recorded by the rear camera are combined together to form a third target video.
And for the third target video obtained by adopting the common coding mode, when the third target video is played, the pictures of the first video source and the second video source are displayed on a screen in a split screen mode, and the two loudspeakers respectively play the sound signals of the left channel and the sound signals of the right channel.
Therefore, in the third target video obtained by adopting the common coding mode, the sound played by one loudspeaker is the result of directional acquisition of the front sound source, the sound played by the other loudspeaker is the result of directional acquisition of the rear sound source, and the sounds played by the two loudspeakers are clean, clear and free from noise and do not overlap with the sounds in other directions. The sound signals collected at the front and the rear are played by the two loudspeakers respectively, so that a stereoscopic sound field is realized together, and the immersion experience is strong when a user watches videos. In addition, the synchronous recording, splicing and playing of different sound channels of front and rear videos are realized by combining split screen display, the synchronous shooting and sharing of front and rear cameras are realized, and the playing audio and video recording, live broadcast explanation, interactive video, personal travel and the like of the home and pets can be realized in another self-timer mode without second ginseng and.
In an embodiment, during split-screen display, a first video source is distributed to one side, closer to a left channel, of a screen for display, and a second video source is distributed to one side, closer to a right channel, of the screen for display, so that a video recorded in front and a sound collected in front are played on the same side, a video recorded in rear and a sound collected in rear are played on the same side, two video pictures displayed by the split-screen are respectively close to the sound corresponding to the shooting direction, and a stereoscopic sound field perceived by a user is more real and immersion experience is stronger.
As can be seen from the above, the video processing method provided by the embodiment of the present application is applied to an electronic device, where a camera is provided on the electronic device, and the video processing method provided by the embodiment of the present application starts the camera to record video first, so as to obtain a video source; meanwhile, in the video recording process, sound signals of target sound sources in different directions are collected, and the collected sound signals are recorded in different sound channels according to different target sound sources; and then synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video. In this embodiment of the application, directional collection has been carried out to the direction that different sound sources are located to with the sound signal record of different sound sources at different sound channels, thereby avoided the sound signal stack of different directions, make the crosstalk between the sound signal of gathering littleer, effectively improved the recording effect when electronic equipment video recording.
The embodiment of the application also provides a video processing device. Referring to fig. 9, fig. 9 is a schematic diagram of a first configuration of a video processing apparatus according to an embodiment of the present application. The video processing device 300 is applied to an electronic device, on which a camera is disposed, and includes a recording module 301, an acquisition module 302, and a synthesis module 303, as follows:
the recording module 301 is configured to start the camera to record video, so as to obtain a video source;
the acquisition module 302 is configured to acquire sound signals of target sound sources in different directions during a video recording process, and record the acquired sound signals in different channels according to different target sound sources;
and the synthesizing module 303 is configured to synthesize the sound signals recorded in different channels with the video source recorded to obtain the target video.
Referring to fig. 10, fig. 10 is a schematic diagram illustrating a second configuration of a video processing apparatus 300 according to an embodiment of the present disclosure. In one embodiment, the video processing apparatus 300 further includes a receiving module 304, a positioning module 305, and a determining module 306:
a receiving module 304 for receiving sound signals from various directions;
a positioning module 305, configured to position sound sources in different directions according to the received sound signals;
And the determining module 306 is used for determining the target sound source from the positioned sound sources in different directions.
In an embodiment, the electronic device is provided with a plurality of microphones, and when positioning sound sources in different directions according to the received sound signals, the positioning module 305 may be configured to:
acquiring time differences of arrival of each sound signal at a plurality of microphones;
and locating a sound source corresponding to each sound signal according to the time difference of arrival of each sound signal at the plurality of microphones.
In an embodiment, the plurality of microphones includes a front microphone and a rear microphone, the target sound sources in different directions include a front sound source and a rear sound source, when collecting the sound signals of the target sound sources in different directions, and recording the collected sound signals in different channels according to the differences of the target sound sources, the collecting module 302 may be configured to:
collecting the sound signal of the front sound source by using the front microphone, and recording the collected sound signal of the front sound source in a first sound channel;
and collecting the sound signals of the rear sound source by using the rear microphone, and recording the collected sound signals of the rear sound source in a second channel.
In an embodiment, the camera includes a front camera and a rear camera, and when the camera is turned on to record video, the recording module 301 may be configured to:
and starting the front camera and the rear camera to record video, so as to obtain a first video source recorded by the front camera and a second video source recorded by the rear camera.
In an embodiment, at least two speakers are disposed on the electronic device, and when synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video, the synthesizing module 303 may be configured to:
synthesizing the sound signal of the first sound channel and the first video source to obtain a first target video;
and synthesizing the sound signal of the second channel and the second video source to obtain a second target video.
With continued reference to fig. 10, in one embodiment, the video processing apparatus 300 further includes a playing apparatus 307. The playing means 307 may be configured to:
playing the first target video, wherein when the first target video is played, all speakers play the sound signals of the first sound channel;
and playing the second target video, wherein when the second target video is played, all speakers play the sound signals of the second channel.
In an embodiment, at least two speakers are disposed on the electronic device, and when synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video, the synthesizing module 303 may be configured to:
and combining the sound signal of the first channel, the sound signal of the second channel, the first video source and the second video source together into a third target video.
The playing means 307 may be configured to:
and playing the third target video, wherein when the third target video is played, the pictures of the first video source and the second video source are displayed on the screen in a split screen mode, each loudspeaker only plays the sound signal of one sound channel, and different loudspeakers play the sound signals of different sound channels.
In an embodiment, at least two speakers are disposed on the electronic device, and when synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video, the synthesizing module 303 may be configured to:
combining the sound signal of the first channel, the sound signal of the second channel and the recorded video source together into a fourth target video;
the playing means 307 may be configured to:
And playing the fourth target video, wherein when the fourth target video is played, each loudspeaker only plays the sound signal of one sound channel, and different loudspeakers play the sound signals of different sound channels.
As can be seen from the above, the video recording processing device provided in the embodiment of the present application is applied to an electronic device, where a camera is provided on the electronic device, and first, the recording module 301 starts the camera to record video, so as to obtain a video source; meanwhile, in the video recording process, the acquisition module 302 acquires sound signals of target sound sources in different directions, and records the acquired sound signals in different sound channels according to different target sound sources; the synthesizing module 303 then synthesizes the sound signals recorded in the different channels with the recorded video source to obtain the target video. In this embodiment of the application, directional collection has been carried out to the direction that different sound sources are located to with the sound signal record of different sound sources at different sound channels, thereby avoided the sound signal stack of different directions, make the crosstalk between the sound signal of gathering littleer, effectively improved the recording effect when electronic equipment video recording.
The embodiment of the application also provides electronic equipment. The electronic device may be a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality ) device, an automobile, a vehicle surrounding obstacle detection device, an audio playing device, a video playing device, a notebook, a desktop computing device, a wearable device such as a wristwatch, glasses, a helmet, an electronic bracelet, an electronic necklace, an electronic article of clothing, or the like.
Referring to fig. 11, fig. 11 is a schematic diagram of a first structure of an electronic device 400 according to an embodiment of the present application. Wherein the electronic device 400 comprises a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.
The processor 401 is a control center of the electronic device 400, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or calling computer programs stored in the memory 402, and calling data stored in the memory 402, thereby performing overall monitoring of the electronic device.
In this embodiment, the processor 401 in the electronic device 400 may load instructions corresponding to the processes of one or more computer programs into the memory 402 according to the steps in the video processing method, and the processor 401 executes the computer programs stored in the memory 402, so as to implement the steps:
starting the camera to record the video to obtain a video source;
In the video recording process, collecting sound signals of target sound sources in different directions, and recording the collected sound signals in different sound channels according to different target sound sources;
and synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video.
With continued reference to fig. 12, fig. 12 is a schematic diagram of a second structure of the electronic device 400 according to the embodiment of the present application. Wherein the electronic device 400 further comprises: a display 403, a control circuit 404, an input unit 405, a sensor 406, and a power supply 407. The processor 401 is electrically connected to the display 403, the control circuit 404, the input unit 405, the sensor 406, and the power supply 407.
The display 403 may be used to display information entered by a user or provided to a user as well as various graphical user interfaces of the electronic device, which may be composed of images, text, icons, video, and any combination thereof.
The control circuit 404 is electrically connected to the display screen 403, and is used for controlling the display screen 403 to display information.
The input unit 405 may be used to receive input numbers, character information, or user characteristic information (e.g., a fingerprint), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. The input unit 405 may include a fingerprint recognition module.
The sensor 406 is used to collect information of the electronic device itself or information of a user or external environment information. For example, the sensor 406 may include a plurality of sensors such as a distance sensor, a magnetic field sensor, a light sensor, an acceleration sensor, a fingerprint sensor, a hall sensor, a position sensor, a gyroscope, an inertial sensor, a gesture sensor, a barometer, a heart rate sensor, and the like.
The power supply 407 is used to power the various components of the electronic device 400. In some embodiments, the power supply 407 may be logically connected to the processor 401 through a power management system, so as to perform functions of managing charging, discharging, and power consumption management through the power management system.
Although not shown in fig. 11 and 12, the electronic device 400 further includes a camera, and may further include a bluetooth module, etc., which will not be described herein.
In this embodiment, the processor 401 in the electronic device 400 may load instructions corresponding to the processes of one or more computer programs into the memory 402 according to the steps in the video processing method, and the processor 401 executes the computer programs stored in the memory 402, so as to implement the steps:
starting the camera to record the video to obtain a video source;
In the video recording process, collecting sound signals of target sound sources in different directions, and recording the collected sound signals in different sound channels according to different target sound sources;
and synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video.
In one embodiment, before collecting sound signals of target sound sources in different directions, the processor 401 performs the following steps:
receiving sound signals from various directions;
positioning sound sources in different directions according to the received sound signals;
and determining the target sound source from the positioned sound sources in different directions.
In one embodiment, the electronic device is provided with a plurality of microphones, and when sound sources in different directions are located according to the received sound signals, the processor 401 performs the following steps:
acquiring time differences of arrival of each sound signal at a plurality of microphones;
and locating a sound source corresponding to each sound signal according to the time difference of arrival of each sound signal at the plurality of microphones.
In an embodiment, the plurality of microphones includes a front microphone and a rear microphone, the target sound sources in different directions include a front sound source and a rear sound source, and when collecting sound signals of the target sound sources in different directions and recording the collected sound signals in different channels according to the differences of the target sound sources, the processor 401 performs the following steps:
Collecting the sound signal of the front sound source by using the front microphone, and recording the collected sound signal of the front sound source in a first sound channel;
and collecting the sound signals of the rear sound source by using the rear microphone, and recording the collected sound signals of the rear sound source in a second channel.
In an embodiment, the camera includes a front camera and a rear camera, and when the camera is turned on to record video, the processor 401 performs the following steps:
and starting the front camera and the rear camera to record video, so as to obtain a first video source recorded by the front camera and a second video source recorded by the rear camera.
In an embodiment, at least two speakers are provided on the electronic device, and when synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video, the processor 401 performs the following steps:
synthesizing the sound signal of the first sound channel and the first video source to obtain a first target video;
and synthesizing the sound signal of the second channel and the second video source to obtain a second target video.
In one embodiment, the processor 401 further performs the steps of:
playing the first target video, wherein when the first target video is played, all speakers play the sound signals of the first sound channel;
and playing the second target video, wherein when the second target video is played, all speakers play the sound signals of the second channel.
In an embodiment, at least two speakers are provided on the electronic device, and when synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video, the processor 401 performs the following steps:
and combining the sound signal of the first channel, the sound signal of the second channel, the first video source and the second video source together into a third target video.
In one embodiment, the processor 401 further performs the steps of:
and playing the third target video, wherein when the third target video is played, the pictures of the first video source and the second video source are displayed on the screen in a split screen mode, each loudspeaker only plays the sound signal of one sound channel, and different loudspeakers play the sound signals of different sound channels.
In an embodiment, at least two speakers are provided on the electronic device, and when synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video, the processor 401 performs the following steps:
Combining the sound signal of the first channel, the sound signal of the second channel and the recorded video source together into a fourth target video;
in one embodiment, the processor 401 further performs the steps of:
and playing the fourth target video, wherein when the fourth target video is played, each loudspeaker only plays the sound signal of one sound channel, and different loudspeakers play the sound signals of different sound channels.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and the portions of an embodiment that are not described in detail in the foregoing embodiments may be referred to the detailed description of the video processing method, which is not repeated herein.
As can be seen from the foregoing, the embodiment of the present application provides an electronic device, on which a camera is disposed, and a processor in the electronic device executes the following steps: firstly, starting the camera to record video to obtain a video source; meanwhile, in the video recording process, sound signals of target sound sources in different directions are collected, and the collected sound signals are recorded in different sound channels according to different target sound sources; and then synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video. In this embodiment of the application, directional collection has been carried out to the direction that different sound sources are located to with the sound signal record of different sound sources at different sound channels, thereby avoided the sound signal stack of different directions, make the crosstalk between the sound signal of gathering littleer, effectively improved the recording effect when electronic equipment video recording.
The embodiment of the application also provides a storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer executes the video recording processing method of any embodiment.
For example, in some embodiments, when the computer program runs on a computer, the computer performs the steps of:
starting the camera to record the video to obtain a video source;
in the video recording process, collecting sound signals of target sound sources in different directions, and recording the collected sound signals in different sound channels according to different target sound sources;
and synthesizing the sound signals recorded in different channels with the recorded video source to obtain the target video.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
It should be noted that, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments may be implemented by a computer program, which may be stored in a computer readable storage medium, and the storage medium may include, but is not limited to: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The video processing method, the video processing device, the electronic equipment and the storage medium provided by the embodiment of the application are described in detail. The principles and embodiments of the present application are described herein with specific examples, the above examples being provided only to assist in understanding the methods of the present application and their core ideas; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present application, the contents of the present specification should not be construed as limiting the present application in summary.
Claims (11)
1. The video processing method is applied to electronic equipment and is characterized in that at least two cameras, at least two microphones and at least two loudspeakers in different directions are arranged on the electronic equipment, and the video processing method comprises the following steps:
starting the at least two cameras to record videos respectively to obtain at least two video sources;
in the video recording process, responding to the adjustment of the recording angle of the electronic equipment by a user, and acquiring the horizontal angle and the pitching angle of each sound source relative to the electronic equipment;
judging whether each sound source is positioned in a preset recording range of each microphone according to the horizontal angle and the pitching angle;
Determining a sound source in the preset recording range of each microphone as a target sound source, respectively acquiring sound signals of the target sound source in the preset recording range of each microphone through the at least two microphones, and recording the acquired sound signals in different sound channels according to different target sound sources;
respectively synthesizing sound signals recorded in different channels and the at least two video sources obtained by recording to obtain a target video;
and when the target video is played, respectively playing sound signals of sound channels corresponding to different video sources in the target video in corresponding speakers.
2. The video processing method according to claim 1, wherein before the capturing, by the at least two microphones, of the audio signals within the preset recording range of each microphone, respectively, further comprises:
receiving sound signals from various directions;
positioning sound sources in different directions according to the received sound signals;
and determining the target sound source from the positioned sound sources in different directions.
3. The recording processing method according to claim 2, wherein said locating sound sources of different directions based on the received sound signals comprises:
Acquiring a time difference of arrival of each sound signal at the at least two microphones;
and locating the sound source corresponding to each sound signal according to the time difference of arrival of each sound signal at least two.
4. The recording processing method according to claim 3, wherein the at least two microphones include a front microphone and a rear microphone, the target sound sources in different directions include a front sound source within a preset recording range of the front microphone and a rear sound source within a preset recording range of the rear microphone, the capturing, by the at least two microphones, sound signals of the target sound source within the preset recording range of each microphone, respectively, and recording the captured sound signals in different channels according to different target sound sources includes:
collecting the sound signal of the front sound source by using the front microphone, and recording the collected sound signal of the front sound source in a first sound channel;
and collecting the sound signals of the rear sound source by using the rear microphone, and recording the collected sound signals of the rear sound source in a second channel.
5. The method of claim 4, wherein the at least two cameras comprise a front camera and a rear camera, the starting the at least two cameras to record video respectively, and obtaining at least two video sources comprises:
And starting the front camera and the rear camera to record video, so as to obtain a first video source recorded by the front camera and a second video source recorded by the rear camera.
6. The video processing method according to claim 5, wherein at least two speakers are provided on the electronic device, and the synthesizing the sound signals recorded in different channels and the at least two video sources recorded respectively to obtain the target video includes:
synthesizing the sound signal of the first sound channel and the first video source to obtain a first target video;
synthesizing the sound signal of the second channel and the second video source to obtain a second target video;
the method further comprises the steps of:
playing the first target video, wherein when the first target video is played, all speakers play the sound signals of the first sound channel;
and playing the second target video, wherein when the second target video is played, all speakers play the sound signals of the second channel.
7. The video processing method according to claim 5, wherein at least two speakers are provided on the electronic device, and the synthesizing the sound signals recorded in different channels and the at least two video sources recorded respectively to obtain the target video includes:
Combining the sound signal of the first channel, the sound signal of the second channel, the first video source and the second video source together into a third target video;
the method further comprises the steps of:
and playing the third target video, wherein when the third target video is played, the pictures of the first video source and the second video source are displayed on the screen in a split screen mode, each loudspeaker only plays the sound signal of one sound channel, and different loudspeakers play the sound signals of different sound channels.
8. The video processing method according to claim 4, wherein at least two speakers are provided on the electronic device, and the synthesizing the sound signals recorded in different channels and the at least two video sources recorded respectively to obtain the target video includes:
combining the sound signal of the first channel, the sound signal of the second channel and the at least two recorded video sources together into a fourth target video;
the method further comprises the steps of:
and playing the fourth target video, wherein when the fourth target video is played, each loudspeaker only plays the sound signal of one sound channel, and different loudspeakers play the sound signals of different sound channels.
9. A video processing device applied to an electronic device, wherein at least two cameras and at least two microphones in different directions are arranged on the electronic device, the video processing device comprises:
the recording module is used for starting the at least two cameras to record videos respectively to obtain at least two video sources;
the acquisition module is used for responding to the adjustment of the recording angle of the electronic equipment by a user in the video recording process, acquiring the horizontal angle and the pitching angle of each sound source relative to the electronic equipment, judging whether each sound source is positioned in the preset recording range of each microphone according to the horizontal angle and the pitching angle, determining the sound positioned in the recording range of each microphone as a target sound source, respectively acquiring the sound signals of the target sound source positioned in the preset recording range of each microphone through the at least two microphones, and recording the acquired sound signals in different sound channels according to different target sound sources;
the synthesizing module is used for respectively synthesizing the sound signals recorded in different sound channels and the at least two video sources obtained by recording to obtain a target video;
And the playing device is used for respectively playing the sound signals of the sound channels corresponding to different video sources in the target video in the corresponding loudspeakers when the target video is played.
10. A storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the steps of the video recording method as claimed in any one of claims 1 to 8.
11. An electronic device having a camera provided thereon, the electronic device comprising a processor and a memory, the memory having stored therein a computer program, the processor executing the steps in the video processing method according to any one of claims 1 to 8 by invoking the computer program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011506301.3A CN112637529B (en) | 2020-12-18 | 2020-12-18 | Video processing method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011506301.3A CN112637529B (en) | 2020-12-18 | 2020-12-18 | Video processing method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112637529A CN112637529A (en) | 2021-04-09 |
| CN112637529B true CN112637529B (en) | 2023-06-02 |
Family
ID=75317319
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011506301.3A Active CN112637529B (en) | 2020-12-18 | 2020-12-18 | Video processing method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112637529B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021237565A1 (en) * | 2020-05-28 | 2021-12-02 | 深圳市大疆创新科技有限公司 | Audio processing method, electronic device and computer-readable storage medium |
| CN113473057B (en) * | 2021-05-20 | 2023-03-03 | 华为技术有限公司 | Video recording method and electronic equipment |
| CN113395451B (en) * | 2021-06-22 | 2023-04-18 | Oppo广东移动通信有限公司 | Video shooting method and device, electronic equipment and storage medium |
| CN113472943B (en) * | 2021-06-30 | 2022-12-09 | 维沃移动通信有限公司 | Audio processing method, device, equipment and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104699445A (en) * | 2013-12-06 | 2015-06-10 | 华为技术有限公司 | Audio information processing method and device |
| CN103888703B (en) * | 2014-03-28 | 2015-11-25 | 努比亚技术有限公司 | Strengthen image pickup method and the camera head of recording |
| CN108076300B (en) * | 2017-12-15 | 2020-07-07 | Oppo广东移动通信有限公司 | Video processing method, video processing device and mobile terminal |
| CN110389597A (en) * | 2018-04-17 | 2019-10-29 | 北京京东尚科信息技术有限公司 | Camera method of adjustment, device and system based on auditory localization |
| CN110572600A (en) * | 2019-08-20 | 2019-12-13 | 维沃移动通信有限公司 | video processing method and electronic equipment |
-
2020
- 2020-12-18 CN CN202011506301.3A patent/CN112637529B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112637529A (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112637529B (en) | Video processing method and device, storage medium and electronic equipment | |
| CN112165590B (en) | Video recording implementation method and device and electronic equipment | |
| CN108401124B (en) | Video recording method and device | |
| CN105791958A (en) | Method and device for live broadcasting game | |
| US10798518B2 (en) | Apparatus and associated methods | |
| CN113365013A (en) | Audio processing method and device | |
| EP3264222B1 (en) | An apparatus and associated methods | |
| EP3422744B1 (en) | An apparatus and associated methods | |
| CN112261481B (en) | Interactive video creating method, device and equipment and readable storage medium | |
| CN112929654B (en) | Method, device and equipment for detecting sound and picture synchronization and storage medium | |
| CN111246300A (en) | Method, device and equipment for generating clip template and storage medium | |
| CN108574904B (en) | Audio acquisition device, audio acquisition system, and audio acquisition method | |
| US9756421B2 (en) | Audio refocusing methods and electronic devices utilizing the same | |
| US20220225049A1 (en) | An apparatus and associated methods for capture of spatial audio | |
| CN113014844A (en) | Audio processing method and device, storage medium and electronic equipment | |
| CN114422935B (en) | Audio processing method, terminal and computer readable storage medium | |
| CN110493635B (en) | Video playing method and device and terminal | |
| WO2023231787A1 (en) | Audio processing method and apparatus | |
| Pingali et al. | Audio-visual tracking for natural interactivity | |
| EP3503579B1 (en) | Multi-camera device | |
| CN113329138A (en) | Video shooting method, video playing method and electronic equipment | |
| CN114245218A (en) | Audio and video playing method and device, computer equipment and storage medium | |
| CN110277105B (en) | Method, device and system for eliminating background audio data | |
| CN113707165A (en) | Audio processing method and device, electronic equipment and storage medium | |
| WO2022161146A1 (en) | Video recording method and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |