WO2023016214A1

WO2023016214A1 - Video processing method and apparatus, and mobile terminal

Info

Publication number: WO2023016214A1
Application number: PCT/CN2022/106791
Authority: WO
Inventors: 许玉新; 曾剑青; 张永兴; 张刘哲
Original assignee: 惠州Tcl云创科技有限公司
Priority date: 2021-08-12
Filing date: 2022-07-20
Publication date: 2023-02-16
Also published as: CN113810627A; CN113810627B

Abstract

The present application discloses a video processing method and apparatus, and a mobile terminal. The method comprises: in first video data captured by a first camera, identifying a view finding frame area conforming to a frame forming condition; processing second video data captured by a second camera and then superimposing said data on the view finding frame area of the first video data, so that the display size of the second video data matches the size of the view finding frame area; and, according to the video after superimposition, generating a synthesized video file carrying an auxiliary video image of the viewing frame. The present application can provide convenience for a user to perform video synthesis.

Description

Video processing method, device, mobile terminal

This application claims the priority of a Chinese patent application with an application date of August 12, 2021, an application number of 2021109330952, and an invention title of "video processing method, device, and mobile terminal", the entire contents of which are incorporated by reference in this application

technical field

The present application relates to the technical field of mobile terminal video processing, in particular to a video processing method, device, and mobile terminal.

Background technique

With the development of science and technology and the continuous improvement of people's living standards, the use of various mobile terminals such as mobile phones has become more and more popular, and mobile phones have become an indispensable communication tool in people's lives.

With more and more smartphones, cameras have become a must-have feature for every phone. Along with the promotion of user's demand, the function of camera is also more and more, for example is provided with front and back camera, but the video that each camera of the mobile terminal of prior art shoots is independent video, and the video shooting function is single, sometimes inconvenient for the user to use .

Therefore, the prior art still needs to be improved and improved.

technical problem

The video shot by each camera of the mobile terminal is an independent video, and the video shooting function is single, which is inconvenient for users to use.

technical solution

In order to solve the above technical problems, the technical scheme adopted by the application is as follows:

A video processing method, wherein the method includes:

Obtain the first video data captured by the first camera;

Analyzing the first video data captured by the first camera, identifying a viewfinder frame area meeting the frame forming condition from the first video data;

Obtain the second video data captured by the second camera, process the second video data and superimpose it on the viewfinder frame area of the first video data, so that the display size of the second video data matches the size of the viewfinder frame area;

The superimposed video is generated into a composite video file with an auxiliary video image with a viewfinder frame.

The video processing method, wherein, before the step of obtaining the first video data captured by the first camera, includes:

The dual-camera dual-shooting function is pre-set on the mobile terminal, which is used to control the simultaneous shooting of the first camera and the second camera in different directions when the dual-camera dual-shooting function is turned on, and the data captured by the first camera and the second camera respectively Superimposed to an interface display.

The video processing method, wherein the step of acquiring the first video data captured by the first camera includes:

The mobile terminal detects that the dual-camera dual-shooting function is enabled, and then controls the first camera and the second camera in different directions of the mobile terminal to start shooting at the same time;

The video data previewed by the first camera is displayed as the first video data, and the video data captured by the second camera is used as the second video data;

Obtain the first video data captured by the first camera;

And acquire the second video data captured by the second camera, and use the second video data captured by the second camera as auxiliary preview data.

The video processing method described above, wherein the first video data captured by the first camera is analyzed, and the step of identifying a viewfinder frame area that meets the conditions for forming a frame from the first video data includes:

Analyze the first video data captured by the first camera, and automatically identify whether there is a quadrilateral or circular window that meets the conditions for forming a frame from the main preview interface generated by the first video data according to a preset AI algorithm, so as to confirm the Whether the main preview interface generated by the first video data includes a viewfinder frame area;

When there is a qualified quadrilateral or circular window in the main preview interface formed by the first video data, then use the quadrilateral or circular window area as the viewfinder frame area in the main preview interface generated by the first video data .

The video processing method, wherein the second video data captured by the second camera is acquired, and the second video data is processed and superimposed on the viewfinder frame area of the first video data, so that the second video data is displayed The step of matching the size with the size of the viewfinder frame area includes:

Obtain the second video data taken by the second camera as the auxiliary preview data; the first camera is the rear camera of the mobile terminal, and the second camera is the front camera of the mobile terminal;

Based on the viewfinder frame area in the main preview interface generated by the first video data, the second video data is automatically edited to be video data that automatically matches the size of the viewfinder frame area;

Controlling to superimpose the edited second video data on the viewfinder frame area and display it together.

The video processing method, wherein, the second video data is automatically edited for the viewfinder frame area in the main preview interface generated based on the first video data, and edited to automatically match the size of the viewfinder frame area The steps for video data include:

The second video data taken by the second camera is used as the auxiliary preview data, and the second video data is cut or rotated, and the second video data display size is automatically matched with the size of the viewfinder frame area, and edited to be the same as the second video data. Video data that automatically matches the size of the frame area described above.

The video processing method, wherein the step of acquiring the second video data captured by the second camera further includes:

The operation instruction is received from the existing video file as the second video data.

The video processing method, wherein the step of processing the second video data and superimposing it on the viewfinder frame area of the first video data includes:

Extracting a portrait video ignoring the background from the second video;

and superimposing the extracted portrait video on the viewfinder frame area of the first video data.

A video processing device, wherein the device includes:

The first obtaining module is used to obtain the first video data captured by the first camera;

The viewfinder frame area identification module is used to analyze the first video data captured by the first camera, and identify the viewfinder frame area that meets the frame forming conditions from the first video data;

The video synthesis processing module is used to obtain the second video data taken by the second camera, and superimpose the second video data on the viewfinder frame area of the first video data after processing, so that the display size of the second video data is the same as that of the first video data. The size of the viewing frame area is matched; the superimposed video is generated into a composite video file with a secondary video image of the viewing frame.

A mobile terminal, wherein the mobile terminal includes a memory, a processor, and a video processing program stored in the memory and operable on the processor, and when the processor executes the video processing program, the The step of the video processing method described in any one.

A computer-readable storage medium, wherein a video processing program is stored on the computer-readable storage medium, and when the video processing program is executed by a processor, the steps of any one of the video processing methods described above are realized.

Beneficial effect

Compared with the prior art, the present application provides a video processing method. The present application analyzes the image data in the video data captured by the main camera, and finds a regular quadrilateral or circular viewfinder, such as a parallelogram, or an approximate parallelogram. The quadrilateral area divides this area into the preview display area of another video file; this application adds a new function to the mobile terminal: it can realize the viewfinder frame that meets the predetermined rules from the camera preview data of the main camera, and another The video data is automatically matched to the viewfinder frame to perform dual-camera video synthesis. This application is to automatically identify the viewfinder frame of the video data captured by the main camera. The position of the viewfinder frame of each different shooting subject is different, which can be automatically identified and synthesized. The application is simple and convenient for shooting, which provides convenience for users.

Description of drawings

FIG. 1 is a flow chart of a specific implementation of a video processing method provided in an embodiment of the present application.

FIG. 2 is a schematic diagram of identifying a viewfinder frame area by quadrilaterals and circles provided by the embodiment of the present application.

FIG. 3 is an effect diagram of using an image quality assimilation function for auxiliary video images according to an embodiment of the present application.

FIG. 4 is a schematic flowchart of synthesizing videos captured by front and rear cameras according to the second embodiment of the present application.

FIG. 5 is a schematic flowchart of synthesizing a rear camera and a pre-stored video file according to a third embodiment of the present application.

Fig. 6 is a schematic flow chart of post-processing the synthesized video provided by the third embodiment of the present application.

FIG. 7 is a functional block diagram of a video processing device provided by an embodiment of the present application.

FIG. 8 is a schematic diagram of an internal structure of a terminal device provided in an embodiment of the present application.

Embodiment of this application

In order to make the purpose, technical solution and effect of the present application more clear and definite, the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

It should be noted that if there are directional indications (such as up, down, left, right, front, back...) in the embodiment of the present application, the directional indications are only used to explain the position in a certain posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes, and cannot be interpreted as indications or hints Its relative importance or implicitly indicates the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In addition, the technical solutions of the various embodiments can be combined with each other, but it must be based on the realization of those skilled in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist , nor within the scope of protection required by the present application.

With the improvement of people's living standards, people are more and more fond of recording and sharing their interesting lives. Therefore, according to people's needs, mobile terminal manufacturers such as mobile phone manufacturers have increased the camera pixels in mobile phones developed year by year. With the rise of video applications, more and more users are using mobile terminals such as mobile phones to take pictures and upload them to the Internet. Video special effects have also changed from the initial beauty to the current AI face change, decoration, blue line and so on. Special effects and applications for mobile phone cameras have been developed.

But in the prior art, when the user wants to take pictures of the scenery of the rear camera and the content of the user of the front camera at the same time, the front camera and the rear camera can only be connected in real time by rotating the mobile phone and shooting separately or by software algorithms. However, no matter which method is used to shoot the video, it seems monotonous, stiff and inconsistent, and it cannot skillfully integrate the scenery of the rear camera with the content of the front camera to achieve natural , Interesting shooting effects.

In order to solve the above problems, an embodiment of the present application provides a video processing method. According to the video processing method of this embodiment, when the user turns on the preset dual-camera dual-shooting function, the mobile terminal intelligently analyzes the first video data of the first camera. , and search for and automatically generate a viewfinder frame area in the first video data, automatically match and superimpose the content captured by the second camera on the viewfinder frame area, and finally generate a natural and interesting front and rear camera composition through video synthesis video. It enables users to get a natural and interesting synthetic video that combines the two cleverly when they need to shoot the content of the front camera and the rear camera at the same time.

exemplary method

first embodiment

As shown in FIG. 1 , an embodiment of the present application provides a video processing method, and the video processing method can be used in mobile devices such as mobile phones and tablet computers. The method described in the embodiment of this application includes the following steps:

Step S100, acquiring the first video data captured by the first camera;

Wherein, before the step of obtaining the first video data captured by the first camera includes:

The dual-camera dual-shooting function is pre-set on the mobile terminal, which is used to control the simultaneous shooting of the first camera and the second camera in different directions when the dual-camera dual-shooting function is turned on, and the data captured by the first camera and the second camera respectively Superimposed to an interface display, as described below.

In this embodiment, the dual-camera dual-shooting function is pre-set in the mobile terminal used by the user. When the user wakes up the dual-camera dual-shooting function by clicking or other operations, the mobile terminal opens the different orientations set on the mobile terminal. The two cameras of the mobile terminal shoot at the same time. When more than 2 cameras are set in the mobile terminal, a selection operation interface for the user to manually select the first camera and the second camera is also set in the application interface of the dual-camera dual-shooting function. , it is convenient for the user to use any camera as the first camera and the second camera, and it is convenient for the user to shoot more diverse videos according to his own ideas, and superimpose the data captured by the first camera and the second camera into one Display interface for users to record.

Therefore, when the user turns on the dual-camera dual-shooting function, the mobile terminal automatically obtains the data captured by the default first camera as the first video data, or obtains the data captured by the user-defined first camera as the first video data. data, and use the second video data captured by the second camera as auxiliary preview data.

For example, in addition to a front camera and a rear camera on the mobile phone of user A, the mobile phone A is also provided with a telescoping and rotating periscope camera placed on the upper edge of the mobile phone A. After user A turns on the dual-camera double-shooting function preset in the mobile phone, the mobile phone A defaults to enabling the front camera and the rear camera to collect camera images, and uses the rear camera as the second camera by default. A camera, for acquiring first video data captured by the first camera. When user A wants to use the periscope camera as the first camera and the rear camera as the second camera, the first camera can be changed to Periscope camera, changing the second camera to a rear camera, the periscope camera rises and shoots after receiving the command to start capturing images, and mobile phone A obtains the first video collected by the periscope camera data, and simultaneously obtain the second video data collected by the rear camera as auxiliary preview data. By freely setting the first camera and the second camera, the user A can customize the shooting angle when using the dual-camera dual-shooting function.

Further, step S200, analyzing the first video data captured by the first camera, and automatically identifying the viewfinder frame area meeting the frame forming condition from the main preview interface generated by the first video data;

In this embodiment, the selected shooting data of the first camera, i.e. the first video data, is analyzed, and the mobile terminal controls the first video data to generate a main preview interface, and in the main preview interface through AI and other intelligent algorithms to automatically identify the viewfinder frame area that meets the conditions for forming a frame. The conditions for forming a frame include the closed area composed of lines that appear in the video, for example, when the subject is glasses, the edge lines of the glasses lenses on each side The frame can form a closed area, that is, the shooting glasses can get two viewfinder frame areas; or through AI and other intelligent algorithms to automatically identify objects that can be used as the viewfinder frame area, such as roadside billboards, mirrors and Spread out white paper, etc. When the mobile terminal recognizes such objects, it can automatically use them as the viewfinder frame area; or recognize whether there are quadrilateral or circular windows in the main preview interface, for example, there are different sizes and colors on a certain wall. If there is a dot wallpaper, each dot in the wallpaper can be regarded as a circular window, and the frame area generated by it can be controlled. Its advantage is that when the object or composition taken by the user is too complex, the viewfinder area of the image in the main preview interface can be selected in various ways, and when multiple viewfinder areas are recognized in the image taken by the user, The user can switch the selection of the viewfinder area by clicking, providing users with more intelligent and diversified composite videos, and the composite videos not only have no sense of violation, but also increase the interest of the video and enhance the user's fun of shooting and sticky.

For example, when the user A is indoors, and opens the dual-camera double-shooting function preset in the mobile phone A, and at this time, the rear camera of the mobile phone A is the first camera, and the front camera is the first camera. Describe the second camera, the main preview interface captured by the first camera includes sunglasses, a circular window, and hanging pictures. Then the mobile phone A is obtained through AI algorithm analysis, and the borders on both sides of the sunglasses form a frame. Closed area, so it is judged that the main preview interface contains the first viewfinder frame area related to sunglasses; the analysis shows that the window is a circular window, and if the pre-set conditions are met, it is judged that the circular window is the second viewfinder frame Area: The hanging picture in the video is recognized as the subject of the viewfinder frame area by AI algorithm, and then the hanging picture is judged to be the third viewfinder frame area. Finally, the mobile phone A uses the hanging picture as the default viewfinder area and processes the video to generate its viewfinder area.

If user A is dissatisfied with the viewfinder frame area of the hanging picture selected by the mobile phone A at this time, the viewfinder frame area can be directly modified through the main preview interface, such as double-clicking the circular window in the main preview screen, the mobile phone A controls to cancel the original video processing operation, and uses the circular window as a new viewfinder area for video processing and generates its viewfinder area.

In addition, as shown in Figure 2, in the application where regular graphics are used as the viewfinder frame area, the image captured by the user A through the rear camera of the mobile phone A is a billboard, and the mobile phone A finds that the image contains a quadrilateral , that is, the shape of the billboard, and it is judged that its side length or diameter is greater than 1/4 of the screen width, then it is judged that the quadrilateral area of the billboard can be used as the viewfinder frame area, and the generation of the viewfinder frame area is controlled.

Further, in step S300, obtain the second video data captured by the second camera, process the second video data and superimpose it on the viewfinder frame area of the first video data, so that the display size of the second video data is the same as that of the viewfinder The size of the frame area is matched, and the superimposed video is generated into a composite video file with an auxiliary video image of the viewfinder frame.

In this embodiment, the mobile terminal acquires the second video data captured by the second camera that needs to be superimposed on the viewfinder frame area, that is, auxiliary preview data. The mobile terminal controls to superimpose the second video data as auxiliary preview data on the viewfinder frame area, and automatically matches the display size and direction of the auxiliary preview data according to the size of the main preview interface, and automatically matches the display size and direction of the auxiliary preview data according to the shooting instruction. Generate a composite file of secondary video images with frames.

In the embodiment of the present application, regarding the implementation of superimposing the second video data as auxiliary preview data on the viewfinder area, the second video data can be cropped to the same size as the viewfinder area, and then the cropped The second video data is superimposed on the viewfinder frame area to generate a composite video file with auxiliary video images of the viewfinder frame, which is simple, convenient and easy to implement.

Of course, the second video data can also be superimposed on the viewfinder area, and then the second video data exceeding the viewfinder area can be clipped to generate a synthetic video file with auxiliary video images of the viewfinder, which is more convenient for cutting like this, and can directly Use the viewfinder frame area as the cropping edge.

The automatic matching method includes performing one or more of the above-mentioned processing methods such as rotating, zooming, translating, cropping or deforming the image of the auxiliary video data. When the mobile terminal cannot automatically place the main part of the auxiliary video image in the In the center of the viewfinder area or the position that the user wants to display, it can be translated by dragging, which is convenient for the user.

Further, in order to make the superimposed synthetic video more natural, before processing the auxiliary video image, an AI algorithm is used to identify whether the viewfinder area is tilted relative to the mobile terminal camera, that is, through the images in the viewfinder area and the / or the light and shadow information of its surrounding environment to identify whether the image in the viewfinder is facing the camera of the mobile terminal, and when it is judged that the image is not facing the camera of the mobile terminal, further analyze its inclination angle relative to the camera of the mobile terminal, and The auxiliary video image is deformed so that it shows the same tilt angle. The video spliced by this method is not a simple superimposition of two videos, but a simulated effect of attaching the auxiliary video image of the second camera to the real object in the viewfinder area and then shooting, which can achieve a more natural video effect , improve user experience.

In addition, when the user moves the position of the first camera when using the dual-camera dual-shooting function, causing the viewfinder frame area to be deformed, this method can be used to transform the auxiliary video image in real time with the same tilt angle and size, further eliminating the need for synthesizing. The incongruity of the video.

Meanwhile, the image quality assimilation function can be set according to the needs of the user, that is, analyze the first video data of the first camera including various parameters such as resolution, frame number, blur degree, filter effect, etc., and do the same for the auxiliary video image The image quality processing makes it further blend into the picture. As shown in Figure 3, after the wide-angle lens is turned on, the first camera is aimed at the rearview mirror of the car, and the second camera is aimed at a frame in the video shot of the rear seat of the car. It can be seen that because the first video data captured by the first camera has a high degree of blur, the mobile terminal automatically blurs the edges of the auxiliary video image, so that the synthesized image is more natural and artistic.

As a further embodiment, in the video processing method, the step of superimposing the second video data on the viewfinder frame area of the first video data after processing may further include: extracting the ignored video from the second video. Background portrait video; superimposing the extracted portrait video on the viewfinder frame area of the first video data. That is, in the embodiment of the present application, the second video data captured by the second camera may also be selected to be processed as required, to extract the portrait video (ignoring the background) in the second video data, and then superimpose the extracted portrait video on the first video. The viewfinder frame area of the video data, and then the superimposed video is generated into a composite video file with a viewfinder frame auxiliary video image. This synthesis technology is more in line with user needs, and does not require users to additionally process auxiliary video, which provides convenience for users.

Further, the auxiliary video data is not only provided by the second camera, but also includes the historical video files stored in the mobile terminal, that is, the historical video files can be imported into the viewfinder area through the selection of the user, for The user is provided with more operating space for synthesizing video, which improves user experience.

second embodiment

The method of the present application is described in further detail below through a specific application example:

In this specific application embodiment, the mobile terminal takes a mobile phone as an example, as shown in Figure 4, the video processing method in this specific application embodiment includes the following steps:

Step S10, start, enter step S11;

Step S11, open the rear camera preview, enter step S12;

Step S12, control and start the AI algorithm, and enter step S13;

Step S13, performing quadrilateral and/or circular detection on the preview data of the rear camera, and proceeding to step S14;

Step S14, analyze whether there is a quadrilateral and/or circular area in the preview data of the rear camera, if there is, then go to step S15, if not, then go to step S14;

Step S15, calculate the position information of the identified quadrilateral and/or circular area including information such as vertices, dots and radii, and proceed to step S16;

Step S16, transmit the position information to the front camera for preview, build a preview window, and enter step S17;

Step S17, the preview data of the front camera is superimposed on the top of the window of the rear camera that meets the area conditions, and enters step S18;

Step S18, start video recording, enter step S19;

Step S19, close the video recording, enter step S20;

Step S20, end.

It can be seen from the above that in the second specific application embodiment of the present application, the user completes the simultaneous opening of the front camera and the rear camera through a mobile phone and performs the splicing of the double-camera double-shooting video. Achieve the effect of making the synthesized video more natural and beautiful, and improve user experience.

As shown in Figure 4, when the user B turns on the dual-camera dual-shooting function of the mobile phone B, the mobile phone B controls to turn on the rear camera and previews the video captured by it, and at the same time starts the AI algorithm to analyze the video in the rear camera. The captured preview data is analyzed to analyze whether there are quadrilateral and/or circular images in the video data; when it is detected that the video data contains quadrilateral and/or circular image areas, calculate its area position information including the Information such as the vertex coordinates, dot coordinates and radius of the above-mentioned area; and the location information of the area is sent to the preview screen of the front camera, and the qualified area is formed in the preview screen of the front camera , so that the user can preview whether the content captured by the front camera is in the eligible area; when the user selects the position of the front camera image, the mobile phone B controls to superimpose the preview data of the front camera on the rear camera At the top of the eligible area window, and start the recording, and after the user clicks the function button to terminate the recording, the recording will end, and a synthetic video without any sense of violation will be generated.

third embodiment

In this specific application embodiment, the mobile terminal takes a tablet computer as an example, as shown in Figure 5, the video processing method in this specific application embodiment includes the following steps:

Step S30, start, enter step S31;

Step S31, open the rear camera preview, enter step S32;

Step S32, control and start the AI algorithm, and enter step S33;

Step S33, performing quadrilateral and/or circular detection on the preview data of the rear camera, and proceeding to step S34;

Step S34, analyze whether there is a quadrilateral and/or circular area in the preview data of the rear camera, if there is, then go to step S35, if not, then go to step S34;

Step S35, calculate the position information of the identified quadrilateral and/or circular area including information such as vertices, dots and radii, and proceed to step S36;

Step S36, click on the area, the control jumps to the file system of the tablet computer and controls to open the selected video file, and enters step S37;

Step S37, superimposing the opened video playback floating frame above the detection window of the rear camera, and proceeding to step S38;

Step S38, start video recording, enter step S39;

Step S39, close the video recording, enter step S40;

Step S40, end.

It can be seen from the above that in the third specific application embodiment of the present application, the purpose of video splicing through the rear camera and the existing video is realized through a tablet computer, which increases the fun of the video and enhances the user's enjoyment and interest in taking pictures. viscosity.

As shown in FIG. 5 , the main flow from step S30 to step S35 is the same as that of the second embodiment, and will not be repeated here. After the tablet computer recognizes the position information of the quadrilateral and/or circular area, the file system of the tablet computer, such as the system gallery, is opened by receiving the operation of clicking the quadrilateral and/or circular area by the user, and receiving the user's selection of video files operation, and display the selected video file in a floating frame. When the user turns on the video recording function, the front camera will be the picture recorded on the spot, and the floating frame of the video file will be the pre-stored video. It plays at the original speed, and when the user presses the function button of closing the recording, the recording ends.

After the recording is finished, as shown in FIG. 6 , it is further processing of the recorded video. It includes the following steps:

Step S50, start video post-processing, enter step S51 and step S61;

Step S51, load the first video file captured by the rear camera, and enter step S52;

Step S52, using a video editing algorithm to analyze each frame of the first video file, and proceed to step S53;

Step S53, the tablet computer starts the AI algorithm, and enters step S54;

Step S54, the tablet computer controls the quadrilateral and/or circular detection of each frame in the first video file, and enters step S55;

Step S55, judging whether a quadrilateral and/or circular area is detected, if yes, proceed to step S56, otherwise proceed to step S54;

Step S56, calculate the position information of the region detected by it, including apex, dot, radius, etc., and enter step S63;

Step S61, load the second video file pre-stored in the file system of the tablet computer, and enter step S62;

Step S62, using a video editing algorithm to analyze each frame of the second video file, and proceed to step S63;

Step S63, the tablet computer processes the first video file and the second video file by cutting and/or rotating according to the position information and image data, and proceeds to step S64;

Step S64, control to re-encode the superimposed single-frame image into a new video file, and store it, and enter step S70;

Step S70, end.

It can be seen from the above that, through the video post-processing method in this embodiment, the video picture previewed by the user is further finely processed, which improves the image quality of the synthesized video and improves the user experience.

As shown in FIG. 6 , after the user finishes recording, the tablet computer performs post-processing on the first video file captured by the rear camera of the tablet computer and the original second video file in the file system respectively. Analyze each frame of the first video file, control and start the AI algorithm to detect the quadrilateral and/or circle of each frame image, and calculate its regional position information when detecting a qualified region, including vertices, parameters such as dots and radius; on the other hand, analyze each frame of the loaded second video file, combine the obtained apex, dots and radius parameter information, and use editing tools to cut or rotate the The first video file and the second video file are superimposed together, and finally the superimposed frame image is re-encoded into a new video file, and the encoded video format includes but is not limited to H.264, MP4, MKV and other video formats.

exemplary device

As shown in FIG. 7 , an embodiment of the present application provides a video processing device, which includes: a first acquisition module 710 , a viewfinder frame area identification module 720 , and a video synthesis module 730 . Specifically, the first acquisition module 710 is configured to acquire the first video data captured by the first camera; the viewfinder frame area identification module 720 is configured to analyze the first video data captured by the first camera, and obtain In the main preview interface generated by the first video data, the viewfinder frame area that meets the frame forming conditions is automatically identified; the video synthesis processing module 730 is used to obtain the second video data captured by the second camera, and control the second video The data is superimposed on the viewfinder frame area, and the display size of the second video data is automatically matched with the size of the viewfinder frame area, and a composite video file of an auxiliary video image with a viewfinder frame is generated according to the shooting instruction.

Based on the foregoing embodiments, the present application further provides a terminal device, the functional block diagram of which may be shown in FIG. 8 . The terminal equipment includes a processor, a memory, a network interface, and a display screen connected through a system bus. Wherein, the processor of the terminal device is used to provide calculation and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the terminal device is used to communicate with external terminals through a network connection. The computer program implements a kind of video processing when executed by the processor. The display screen of the terminal device may be a liquid crystal display screen or an electronic ink display screen.

Those skilled in the art can understand that the functional block diagram in Figure 8 is only a block diagram of a partial structure related to the solution of this application, and does not constitute a limitation on the terminal equipment on which the solution of this application is applied. The specific terminal equipment may include comparison diagrams more or fewer components, or combine certain components or have a different arrangement of components.

In one embodiment, a terminal device is provided. The terminal device includes a memory, a processor, and a video processing program stored on the processor and operable on the processor. The following steps are performed for processing:

Obtain the first video data captured by the first camera;

Analyzing the first video data captured by the first camera, automatically identifying the viewfinder frame area that meets the frame forming conditions from the main preview interface generated by the first video data;

Obtain the second video data shot by the second camera, control to superimpose the second video data on the viewfinder frame area, and automatically match the display size of the second video data with the size of the viewfinder frame area, and generate a tape according to the shooting instruction Composite video file of the frame secondary video image.

Wherein, the step of obtaining the first video data taken by the first camera includes:

Obtain the first video data captured by the first camera;

And acquire the second video data captured by the second camera, and use the auxiliary preview data captured by the second camera as the second video data.

Wherein, the step of analyzing the first video data captured by the first camera, and automatically identifying the viewfinder frame area that meets the frame forming conditions from the main preview interface generated by the first video data includes:

Analyze the first video data captured by the first camera, and identify from the first video data according to a preset AI algorithm whether there is a quadrilateral or circular window that meets the conditions for forming a frame, so as to confirm the shape generated by the first video data Whether the viewfinder area is included in the main preview interface;

Wherein, the acquisition of the second video data captured by the second camera controls the superimposition of the second video data on the viewfinder frame area, and automatically matches the display size of the second video data with the size of the viewfinder frame area, according to The steps of generating a synthetic video file with auxiliary video images of the viewfinder frame by the shooting instruction include:

Obtain the auxiliary preview data taken by the second camera as the second video data; the first camera is the rear camera of the mobile terminal, and the second camera is the front camera of the mobile terminal;

Controlling the edited second video data to be superimposed on the viewfinder frame area to be displayed together; and generating a composite video file with auxiliary video images of the viewfinder frame according to the shooting instruction.

Wherein, the second video data is automatically edited based on the framing area in the main preview interface generated by the first video data, and the step of editing the video data automatically matching the size of the framing area includes:

The auxiliary preview data shot by the second camera is used as the second video data, and the second video data is cut or rotated, and the display size of the second video data is automatically matched with the size of the viewfinder frame area, and edited to be the same as the second video data. The size of the frame area automatically matches the video data.

Wherein, the step of obtaining the second video data taken by the second camera also includes:

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

To sum up, the present application discloses a video processing method, device, and mobile terminal. The method includes: acquiring the first video data captured by the first camera, and acquiring the second video data captured by the second camera; Analyze the data, find out the viewfinder frame area in the main preview interface generated by the first video data; The auxiliary preview data is superimposed on the viewfinder area, and the display size of the auxiliary preview data is automatically matched with the size of the viewfinder area. This application adds a new function to the mobile terminal: it can find out the viewfinder frame that meets the predetermined rules from the camera preview data of the main camera, automatically match the shooting data of another camera to the viewfinder frame, and perform video synthesis for the user's use. Convenience is provided.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

A video processing method, wherein the method includes:

Obtain the first video data captured by the first camera;

Analyzing the first video data captured by the first camera, identifying a viewfinder frame area that meets the frame forming condition from the first video data;

Acquiring the second video data shot by the second camera, processing the second video data and superimposing it on the viewfinder frame area of the first video data, so that the display size of the second video data matches the size of the viewfinder frame area ;

The superimposed video is generated into a composite video file with an auxiliary video image with a viewfinder frame.
The video processing method according to claim 1, wherein, before the step of obtaining the first video data captured by the first camera, the step comprises:

The dual-camera dual-shooting function is pre-set on the mobile terminal, which is used to control the simultaneous shooting of the first camera and the second camera in different directions when the dual-camera dual-shooting function is turned on, and the data captured by the first camera and the second camera respectively Superimposed to an interface display.
The video processing method according to claim 2, wherein the step of obtaining the first video data captured by the first camera comprises:

When it is detected that the dual-camera dual-shooting function is turned on, the first camera and the second camera in different directions of the mobile terminal are controlled to start shooting simultaneously;

Using the video data captured by the first camera as the first video data;

Obtain the second video data captured by the second camera, including:

The video data captured by the second camera is used as the second video data.
The video processing method according to claim 3, wherein the mobile terminal includes more than two cameras,

Before the first camera and the second camera in different directions of the mobile terminal are controlled to start shooting at the same time when it is detected that the dual-camera dual-shooting function is enabled, it also includes:

displaying an operation interface for selecting the first camera and the second camera;

In response to a selection operation on the selection operation interface, the cameras corresponding to the selection operation are used as the first camera and the second camera.
The video processing method according to claim 1, wherein the first video data captured by the first camera is analyzed, and the step of identifying a viewfinder frame area that meets the frame forming conditions from the first video data includes:

Analyze the first video data captured by the first camera, and automatically identify whether there is a quadrilateral or circular window that meets the conditions for forming a frame from the main preview interface generated by the first video data according to a preset AI algorithm, so as to confirm the Whether the main preview interface generated by the first video data includes a viewfinder frame area;

When there is a qualified quadrilateral or circular window in the main preview interface formed by the first video data, use the quadrilateral or circular window area as the viewfinder frame area in the main preview interface generated by the first video data .
The video processing method according to claim 1, wherein the step of analyzing the first video data captured by the first camera, and identifying a viewfinder frame area that meets the conditions for forming a frame from the first video data includes:

When an object used as a viewfinder frame area is recognized from the first video data by an intelligent algorithm such as AI, the area where the object is located is used as a viewfinder frame area.
The video processing method according to claim 1, wherein the step of analyzing the first video data captured by the first camera, and identifying a viewfinder frame area that meets the conditions for forming a frame from the first video data includes:

Analyzing the first video data captured by the first camera to obtain a closed area composed of lines in the first video data;

The closed area is taken as a viewfinder frame area meeting the frame forming condition.
The video processing method according to claim 1, wherein, analyzing the first video data captured by the first camera, identifying a viewfinder frame area that meets a frame forming condition from the first video data, include:

Analyzing the first video data captured by the first camera, identifying a plurality of viewfinder frame areas meeting the frame forming conditions from the first video data;

Filter out the default viewfinder area from multiple viewfinder areas;

The second video data is superimposed on the viewfinder frame area of the first video data after processing, so that the display size of the second video data matches the size of the viewfinder frame area, including:

The second video data is processed and superimposed on the default viewfinder frame area of the first video data, so that the display size of the second video data matches the size of the default viewfinder frame area.
The video processing method according to claim 1, wherein the first camera is a mobile terminal rear camera, the second camera is a mobile terminal front camera, and the second video data is processed and superimposed on The step of making the display size of the second video data match the size of the viewfinder frame area of the first video data includes:

Automatically edit the second video data based on the viewfinder frame area in the main preview interface generated by the first video data, and edit it into video data that automatically matches the size of the viewfinder frame area;

Controlling to superimpose the edited second video data on the viewfinder frame area and display it together.
The video processing method according to claim 9, wherein the second video data is automatically edited based on the viewfinder frame area in the main preview interface generated based on the first video data, and edited to match the viewfinder frame area The steps for automatically matching the size of the video data include:

Crop or rotate the second video data based on the viewfinder frame area in the main preview interface generated by the first video data, so as to edit the second video data to automatically match the size of the viewfinder frame area video data.
The video processing method according to claim 1, wherein, after the processing, the second video data is superimposed on the viewfinder frame area of the first video data, so that the display size of the second video data is the same as that of the viewfinder Matching box area sizes, including:

superimposing the second video data on the viewfinder frame area;

Clipping the data beyond the viewfinder frame area in the second video data, so that the display size of the second video data matches the size of the viewfinder frame area.
The video processing method according to claim 1, wherein the step of obtaining the second video data captured by the second camera comprises:

receive operating instructions;

According to the operation instruction, the second video data is obtained from the existing video file.
The video processing method according to claim 1, wherein the step of superimposing the processed second video data on the viewfinder frame area of the first video data comprises:

Extracting a portrait video ignoring the background from the second video;

and superimposing the extracted portrait video on the viewfinder frame area of the first video data.
The video processing method according to claim 1, wherein said processing the second video data and superimposing it on the viewfinder frame area of the first video data comprises:

determining an inclination angle of the viewfinder frame area relative to the first camera;

rotating the tilt angle for the second video data;

Superimposing the second video data rotated by the tilt angle on the viewfinder frame area of the first video data.
The video processing method according to claim 1, wherein said processing the second video data and superimposing it on the viewfinder frame area of the first video data comprises:

Obtaining the resolution, number of frames, degree of blur and filter effect of the first video data;

performing image quality processing on the second video data according to the resolution, frame number, blur and filter effect of the first video data;

Superimposing the image-quality-processed second video data into the viewfinder frame area of the first video data.
A video processing device, wherein the device includes:

The first obtaining module is used to obtain the first video data captured by the first camera;

A viewfinder frame area identification module, configured to analyze the first video data captured by the first camera, and identify a viewfinder frame area that meets the frame forming conditions from the first video data;

The video synthesis processing module is used to obtain the second video data taken by the second camera, and superimpose the second video data on the viewfinder frame area of the first video data after processing, so that the display size of the second video data is the same as that of the first video data. The size of the viewfinder frame area is matched; the superimposed video is generated into a composite video file with auxiliary video images of the viewfinder frame.
A mobile terminal, wherein the mobile terminal includes a memory, a processor, and a video processing program stored in the memory and operable on the processor, and when the processor executes the video processing program, the :

Obtain the first video data captured by the first camera;

Analyzing the first video data captured by the first camera, identifying a viewfinder frame area that meets the frame forming condition from the first video data;

Acquiring the second video data shot by the second camera, processing the second video data and superimposing it on the viewfinder frame area of the first video data, so that the display size of the second video data matches the size of the viewfinder frame area ;

The superimposed video is generated into a composite video file with an auxiliary video image with a viewfinder frame.
The mobile terminal according to claim 17, wherein, when the processor executes the video processing program, it further implements:

When it is detected that the dual-camera dual-shooting function is turned on, the first camera and the second camera in different directions of the mobile terminal are controlled to start shooting simultaneously;

Using the video data captured by the first camera as the first video data;

Obtain the second video data captured by the second camera, including:

The video data captured by the second camera is used as the second video data.
The mobile terminal according to claim 17, wherein, when the processor executes the video processing program, it further implements:

displaying an operation interface for selecting the first camera and the second camera;

In response to a selection operation on the selection operation interface, the cameras corresponding to the selection operation are used as the first camera and the second camera.
A computer-readable storage medium, wherein a video processing program is stored on the computer-readable storage medium, and when the video processing program is executed by a processor, the video processing according to any one of claims 1-16 is realized method steps.