WO2023029547A1

WO2023029547A1 - Video processing method, and electronic device

Info

Publication number: WO2023029547A1
Application number: PCT/CN2022/091447
Authority: WO
Inventors: 付庆涛; 陈斌
Original assignee: 荣耀终端有限公司
Priority date: 2021-08-31
Filing date: 2022-05-07
Publication date: 2023-03-09
Also published as: CN115633255A; CN115633255B

Abstract

Provided in the present application are a video processing method, and an electronic device. The video processing method is applied to an electronic device, and comprises: displaying a first image frame, wherein the first image frame is an image frame of a target object at a first position; when the target object moves to a second position, acquiring a second image frame, wherein the second position is a different position from the first position, and the second image frame refers to an image frame which is collected by an electronic device when the target object moves to the second position; performing facial detection according to the second image frame, so as to obtain coordinate information of a first detection box; obtaining coordinate information of a cropping box according to the first detection box; cropping the second image frame according to the cropping box, so as to obtain display content comprising the target object; and displaying a third image frame according to the display content, wherein there is an intersection between a first area in the first image frame and a second area in the third image frame. On the basis of the technical solution of the present application, where the power consumption is reduced, the accuracy of video processing can be improved.

Description

Video processing method and electronic device

This application claims the priority of a Chinese patent application filed with the State Intellectual Property Office on August 31, 2021, with application number 202111016638.0 and application name "Video Processing Method and Electronic Equipment", the entire contents of which are incorporated by reference in this application .

technical field

The present application relates to the field of image processing, in particular to a video processing method and electronic equipment.

Background technique

With the rapid development of image technology, users' demand for video recording function is increasing. For example, recording video through the camera application, recording video during a video call, and recording video in a surveillance scene, etc. Taking a video call as an example, the user can use an electronic device to shoot a video call; if the viewing range of the electronic device is not adjusted when the subject moves, the position of the subject in the video screen will deviate from the edge of the screen, making the video display The user sensory experience is not good. In order to obtain a better video experience and ensure that the subject can always be kept in the middle of the video display screen, the function of "moving with people" is proposed.

At present, the user tracking in the video display screen is usually realized by detecting the user's human body, so as to realize the function of "following the person's movement". Higher, moreover, the human body detection of the user is also likely to cause misidentification.

Therefore, in the case of reducing power consumption, how to accurately process video and realize the function of "moving with people" has become an urgent problem to be solved.

Contents of the invention

The present application provides a video processing method and electronic equipment. Through the video processing method of the embodiment of the present application, the video can be accurately processed while reducing power consumption, and the accuracy of the "follow people's movement" function can be improved.

In the first aspect, a video processing method is provided, and the video processing method is applied to electronic equipment, including:

Displaying a first image frame, the first image frame is an image frame of the target object at a first position; when the target object moves to a second position, acquiring a second image frame, the second position is the same as the second position The first position is a different position, and the second image frame refers to an image frame collected by the electronic device when the target object moves to the second position; face detection is performed according to the second image frame to obtain Coordinate information of the first detection frame, the first detection frame is used to indicate the position information of the face of the target object in the second image frame; obtain the coordinate information of the cropping frame according to the first detection frame; Perform cropping processing on the second image frame according to the cropping frame to obtain display content including the target object; display a third image frame according to the display content, where the first region in the first image frame is consistent with the target object. The second area in the third image frame has an intersection, the first area refers to the area where the target object is located in the first image frame, and the second area refers to the area of the target object in the third image frame. The area where the target object is located.

It should be understood that the second image frame may refer to an image frame captured by the camera in real time after the target object moves; the target object may refer to a part or all of the subject; Refers to the owner user; in the case of not receiving an instruction to enable the identification of the owner, the target object may refer to all shooting objects.

It should also be understood that the intersection between the first area of the first image and the second area of the third image may mean that the first area and the second area completely overlap; or, it may also mean that there is a partial intersection between the first area and the second area .

In a possible implementation, before the target object moves, the electronic device displays the first image frame of the target object; after the target object moves, the camera of the electronic device can collect the second image frame in real time, and the second image frame Perform face detection to obtain the coordinate information of the face frame corresponding to the target object; obtain the coordinate information of the cropping frame according to the coordinate information of the face frame; perform cropping processing on the second image frame according to the cropping frame to obtain the display content including the target object ; Displaying the third image frame according to the display content; the third image frame refers to the image frame of the target object displayed by the electronic device after the target object moves.

In the embodiment of the present application, by performing face detection on the acquired second image frame, the coordinate information of the human face frame of the target object is determined, and the coordinate information of the cropping frame is obtained according to the coordinate information of the human face frame; further according to the cropping frame Carry out cropping processing to the second image frame to obtain the display content including the target object; display the third image frame according to the display content; in the embodiment of the application, since the coordinate information of the cropping frame is determined by the coordinate information of the face frame, therefore Compared with the scheme of directly detecting the key points of the human body of the target object to determine the cropping frame, the video processing method of the present application can reduce the calculation amount of the electronic device and reduce the power consumption of the electronic device; in addition, because the video processing method of the present application is Determining the coordinate information of the cropping frame according to the face frame can prevent the target object from performing video tracking and display on the target object when the target object is facing away from the electronic device in the second image frame; therefore, the solution of the present application can reduce power consumption. , and can also improve the accuracy of video tracking display.

With reference to the first aspect, in some implementation manners of the first aspect, the first area coincides with the second area.

In a possible implementation manner, the first area and the second area overlap, and the first area and the second area are located in a middle area of the display interface.

With reference to the first aspect, in some implementation manners of the first aspect, when the first image frame and the third image frame are displayed, the positions of the electronic device are the same.

In the embodiment of the present application, the electronic device can keep its position unchanged, and after the target object is moved, the target object can always be displayed in the middle position or the middle area of the video display screen; to realize the tracking of the target object Display, that is, to realize the function of "following people".

In combination with the first aspect, in some implementation manners of the first aspect, it also includes:

An action is detected indicating that the camera application is running; or,

An action was detected indicating to run a video calling application.

In the embodiment of the present application, the video processing method may be applied in the process of shooting video by the camera application program; or, the video processing method may also be applied in the video call application program.

With reference to the first aspect, in some implementation manners of the first aspect, the obtaining the coordinate information of the cropping frame according to the first detection frame includes:

performing a first extension process on the first detection frame to obtain a second detection frame;

performing a second extension process on the second detection frame to obtain the cropping frame;

Wherein, the first expansion process refers to expanding the boundary of the first detection frame with the first detection frame as the center, and the second detection frame is used to indicate that the body of the target object is within the first detection frame. For the location information in the second image frame, the second extension process refers to extending the boundary of the second detection frame with the second detection frame as the center.

In the embodiment of the present application, in order to avoid local shaking of the first detection frame in multiple image frames in the video image, it is ensured that the cropping frame can remain unchanged when the target object is moving in a small range; therefore, through the second detection The frame is subjected to the second expansion process to obtain the cropping frame, which can ensure the stability of the image frame after the cropping process to a certain extent.

With reference to the first aspect, in some implementation manners of the first aspect, performing the first extension process on the first detection frame to obtain the second detection frame includes:

The first expansion process is performed on the first detection frame according to a first threshold to obtain the second detection frame, and the first threshold is used to indicate body proportion data.

With reference to the first aspect, in some implementation manners of the first aspect, performing cropping processing on the second image frame according to the cropping frame to obtain display content including the target object includes:

determining whether the second detection frame and the cropping frame meet a preset condition, the preset condition means that the second detection frame and the cropping frame satisfy a preset proportional relationship;

When the second detection frame and the cropping frame meet the preset condition, the second image frame is cropped according to the cropping frame to obtain the display content.

In a possible implementation manner, the preset condition may refer to that the second detection frame and the cropping frame satisfy a certain proportional relationship, and the second detection frame is located inside the cropping frame.

With reference to the first aspect, in some implementations of the first aspect, the coordinate information of the first detection frame refers to the coordinate information corresponding to the first detection frame when the second image frame is at the second resolution ,Also includes:

receiving a request instruction, where the request instruction is used to request a first resolution;

The second resolution is determined according to the first resolution, and the second resolution is greater than the first resolution.

In the embodiment of the present application, the resolution can be extended from the first resolution to the second resolution, which can solve the problem of lowering the definition of the second image frame caused by the subsequent cropping process; by performing the resolution expansion process, it is possible to To a certain extent, the clarity of the third image frame displayed after the trimming process is improved.

In a possible implementation manner, a request instruction requesting the first resolution is received; the first resolution is expanded to determine the second resolution; the coordinate information of the first detection frame is detected in the second image frame; The coordinate information of the first detection frame is converted to the corresponding coordinate information when the second image frame is of the second resolution.

With reference to the first aspect, in some implementation manners of the first aspect, the displaying the third image frame according to the display content includes:

performing scaling processing on the display content according to the first resolution to obtain the processed display content;

displaying the third image frame according to the processed display content.

With reference to the first aspect, in some implementation manners of the first aspect, the target object is the owner user, and further includes:

Receive an owner identification instruction, the owner identification instruction is used to instruct to identify the owner user;

Face recognition is performed according to the first detection frame to determine the owner user, and the owner user is a pre-configured user.

It should be understood that the owner of the device may refer to the management user of the electronic device; or, the owner may also be any user with a higher priority configured in advance; the identification of the owner refers to identifying the target through face detection during tracking and display. The owner user in the object, and track and display the owner user.

With reference to the first aspect, in some implementation manners of the first aspect, the first detection frame refers to a face frame of the owner user of the device.

With reference to the first aspect, in some implementation manners of the first aspect, the target object includes at least one user.

With reference to the first aspect, in some implementation manners of the first aspect, the target object includes a first user and a second user, and the first detection frame refers to a face frame of the first user and the first user The union frame of the face frames of the two users.

In a second aspect, an electronic device is provided, and the electronic device includes: one or more processors, a memory, and a display screen; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code comprising computer instructions that are invoked by the one or more processors to cause the electronic device to perform:

Displaying a first image frame, the first image frame is an image frame of the target object at a first position; when the target object moves to a second position, acquiring a second image frame, the second position is the same as the second position The first position is a different position, and the second image frame refers to an image frame collected by the electronic device when the target object moves to the second position; face detection is performed according to the second image frame to obtain Coordinate information of the first detection frame, the first detection frame is used to indicate the position information of the face of the target object in the second image frame; obtain the coordinate information of the cropping frame according to the first detection frame;

Perform cropping processing on the second image frame according to the cropping frame to obtain display content including the target object; display a third image frame according to the display content, and the first area in the first image frame is consistent with the third image There is an intersection in the second area in the frame, the first area refers to the area where the target object in the first image frame is located, and the second area refers to the area of the target object in the third image frame your region.

With reference to the second aspect, in some implementation manners of the second aspect, when the first image frame and the third image frame are displayed, the positions of the electronic device are the same.

With reference to the second aspect, in some implementation manners of the second aspect, the one or more processors invoke the computer instructions so that the electronic device further executes:

An action is detected indicating that the camera application is running; or,

An action was detected indicating to run a video calling application.

The second detection frame is obtained by performing the first expansion process on the first detection frame according to a first threshold, where the first threshold is used to indicate body proportion data.

With reference to the second aspect, in some implementations of the second aspect, the coordinate information of the first detection frame refers to the coordinate information corresponding to the first detection frame when the second image frame is at the second resolution , the one or more processors invoke the computer instructions so that the electronic device also performs:

displaying the third image frame according to the processed display content.

With reference to the second aspect, in some implementation manners of the second aspect, the first detection frame refers to a face frame of the owner user of the device.

With reference to the second aspect, in some implementation manners of the second aspect, the target object includes at least one user.

With reference to the second aspect, in some implementation manners of the second aspect, the target object includes a first user and a second user, and the first detection frame refers to a face frame of the first user and the first user The union frame of the face frames of the two users.

With reference to the second aspect, in some implementation manners of the second aspect, the first area coincides with the second area.

It should be understood that the extensions, limitations, explanations and descriptions of relevant content in the first aspect above are also applicable to the same content in the second aspect.

In a third aspect, a video processing device is provided, including a unit for executing any video processing method in the first aspect.

In a possible implementation manner, when the video processing device is an electronic device, the processing unit may be a processor, and the input unit may be a communication interface; the electronic device may also include a memory, and the memory is used to store computer programs A code for causing the electronic device to execute any method in the first aspect when the processor executes the computer program code stored in the memory.

In a fourth aspect, a chip system is provided, the chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call a computer instruction so that the electronic device executes the first aspect Any of the video processing methods in .

In a fifth aspect, a computer-readable storage medium is provided, the computer-readable storage medium stores computer program codes, and when the computer program codes are executed by an electronic device, the electronic device executes any of the items in the first aspect. A video processing method.

In a sixth aspect, a computer program product is provided, the computer program product comprising: computer program code, when the computer program code is run by an electronic device, the electronic device is made to perform any video processing in the first aspect method.

Description of drawings

FIG. 1 is a schematic diagram of a hardware system applicable to an electronic device of the present application;

Fig. 2 is a schematic diagram of a software system applicable to the electronic device of the present application;

FIG. 3 is a schematic diagram of an application scenario applicable to this application;

Fig. 4 is a schematic diagram of the intersection of the first area and the second area provided by the implementation of the present application;

Fig. 5 is a schematic flowchart of a video processing method provided by the present application;

FIG. 6 is a schematic diagram of a video processing display interface provided by the present application;

FIG. 7 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 8 is a schematic diagram of a video processing display interface provided by the present application;

FIG. 9 is a schematic diagram of a video processing display interface provided by the present application;

FIG. 10 is a schematic diagram of a video processing display interface provided by the present application;

FIG. 11 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 12 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 13 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 14 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 15 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 16 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 17 is a schematic diagram of a video processing display interface provided by the present application;

Fig. 18 is a schematic diagram of a video processing display interface provided by the present application;

FIG. 19 is a schematic structural diagram of a video processing device provided by the present application;

FIG. 20 is a schematic structural diagram of an electronic device provided by the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 1 shows a hardware system applicable to the electronic equipment of this application.

The electronic device 100 may be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, a vehicle electronic device, an augmented reality (augmented reality, AR) device, a virtual reality (virtual reality, VR) device, a notebook computer, a super mobile personal computer ( ultra-mobile personal computer (UMPC), netbook, personal digital assistant (personal digital assistant, PDA), projector, etc., the embodiment of the present application does not impose any limitation on the specific type of the electronic device 100.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It should be noted that the structure shown in FIG. 1 does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in FIG. 1 , or the electronic device 100 may include a combination of some of the components shown in FIG. 1 , or , the electronic device 100 may include subcomponents of some of the components shown in FIG. 1 . The components shown in FIG. 1 can be realized in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor) , ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, neural network processor (neural-network processing unit, NPU). Wherein, different processing units may be independent devices or integrated devices.

The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (inter-integrated circuit, I2C) interface, an inter-integrated circuit sound (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receiver/transmitter (UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, SIM interface, USB interface.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). The I2S interface can be used for audio communication. The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc.

In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 . The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal interface or as a data signal interface.

In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 and the sensor module 180 . The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface or MIPI interface.

The USB interface 130 is an interface conforming to the USB standard specification, for example, it can be a mini (Mini) USB interface, a micro (Micro) USB interface or a C-type USB (USB Type C) interface. The USB interface 130 can be used to connect a charger to charge the electronic device 100 , can also be used to transmit data between the electronic device 100 and peripheral devices, and can also be used to connect an earphone to play audio through the earphone. The USB interface 130 can also be used to connect other electronic devices 100, such as AR devices.

The connection relationship between the modules shown in FIG. 1 is only a schematic illustration, and does not constitute a limitation on the connection relationship between the modules of the electronic device 100 . Optionally, each module of the electronic device 100 may also adopt a combination of various connection modes in the foregoing embodiments.

The charging management module 140 is used to receive power from the charger. While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device 100 through the power management module 141 . The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (eg, leakage, impedance). Optionally, the power management module 141 may be set in the processor 110, or the power management module 141 and the charge management module 140 may be set in the same device.

The wireless communication function of the electronic device 100 may be realized by components such as the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, and a baseband processor. Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.

The mobile communication module 150 may provide a wireless communication solution applied to the electronic device 100, such as at least one of the following solutions: a second generation (2 ^th generation, 2G) mobile communication solution, a third generation (3 ^th generation, 3G) mobile communication solutions, fourth generation ( ^4th generation, 4G) mobile communication solutions, fifth generation ( ^5th generation, 5G) mobile communication solutions.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs a sound signal through an audio device (for example, a speaker 170A, a receiver 170B), or displays an image or video through a display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

Similar to the mobile communication module 150, the wireless communication module 160 can also provide a wireless communication solution applied to the electronic device 100, such as at least one of the following solutions: wireless local area networks (wireless local area networks, WLAN), Bluetooth (bluetooth, BT ), Bluetooth low energy (bluetooth low energy, BLE), ultra wide band (ultra wide band, UWB), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication (near field communication, NFC), infrared (infrared, IR) technology.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 of the electronic device 100 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other electronic devices through wireless communication technology.

The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

Display 194 may be used to display images or video. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible Light-emitting diode (flex light-emitting diode, FLED), mini light-emitting diode (mini light-emitting diode, Mini LED), micro light-emitting diode (micro light-emitting diode, Micro LED), micro OLED (Micro OLED) or quantum dot light emitting Diodes (quantum dot light emitting diodes, QLED). In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 , and the application processor.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can optimize the algorithm of image noise, brightness and color, and ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard red green blue (red green blue, RGB), YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Exemplarily, in the embodiment of the present application, the camera 193 can acquire a video image frame, and the video image frame can refer to a full-size image frame collected; the camera 193 can transmit the acquired video image frame to the ISP, and the ISP is used for Process the video image frame obtained by the camera 193; for example, the ISP can obtain the parameters of the target resolution and cropping processing and scaling processing from the processor 110; the ISP can adjust the full-size video image frame to the target resolution size according to the target resolution ; According to the parameters of the cropping processing and zooming processing, the video image frame of the target resolution is clipped and zoomed to obtain the processed video image frame, and the processed video image frame meets the requested resolution size issued by the application program; The processed video image frames are transmitted to the application program, and the display screen 194 displays the processed video image frames.

Exemplarily, in the embodiment of the present application, calculation of video stream target resolution, face detection, cropping and scaling parameter calculation may be performed in the processor 110 . It should be understood that the relevant steps of determining parameters in the video processing method of the present application may be executed in the processor 110; the ISP is used to obtain relevant parameters for processing video image frames, and process the video image frames according to the relevant parameters to obtain suitable electronic parameters. The display screen 194 of the device displays the output image frame of the specification.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3 and MPEG4.

The external memory interface 120 can be used to connect an external memory card, such as a secure digital (secure digital, SD) card, so as to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The internal memory 121 may include an area for storing programs and an area for storing data.

The electronic device 100 can implement audio functions, such as music playing and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

The audio module 170 is used to convert digital audio information into analog audio signal output, and can also be used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals.

Speaker 170A, also known as a horn, is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make a hands-free call through the speaker 170A. Receiver 170B, also known as an earpiece, is used to convert audio electrical signals into audio signals.

In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensor 180A, for example, it may be a resistive pressure sensor, an inductive pressure sensor or a capacitive pressure sensor. The capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes, and the electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when the touch operation with the touch operation intensity less than the first pressure threshold acts on the short message application icon, execute the instruction of viewing the short message; when the touch operation with the intensity greater than or equal to the first pressure threshold acts on the short message application icon , to execute the instruction of creating a new short message.

The gyro sensor 180B can be used to determine the motion posture of the electronic device 100 . In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x-axis, y-axis and z-axis) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used in scenarios such as navigation and somatosensory games.

The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case.

The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally x-axis, y-axis and z-axis). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 180E can also be used to identify the posture of the electronic device 100 as an input parameter for application programs such as horizontal and vertical screen switching and pedometer.

The distance sensor 180F is used to measure distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, for example, in a shooting scene, the electronic device 100 can use the distance sensor 180F for distance measurement to achieve fast focusing.

The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LEDs may be infrared LEDs. The electronic device 100 emits infrared light through the LED. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When the reflected light is detected, the electronic device 100 may determine that there is an object nearby. When no reflected light is detected, the electronic device 100 may determine that there is no object nearby. The electronic device 100 can use the proximity light sensor 180G to detect whether the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used for automatic unlocking and automatic screen locking in leather case mode or pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement functions such as unlocking, accessing the application lock, taking pictures, and answering incoming calls.

The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from being shut down abnormally due to the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

The touch sensor 180K is also referred to as a touch device. The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a touch screen. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor 180K may transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through the display screen 194 . In some other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 and disposed at a different position from the display screen 194 .

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal.

Keys 190 include a power key and a volume key. The key 190 can be a mechanical key or a touch key. The electronic device 100 may receive a key input signal, and implement a function related to the key input signal.

The motor 191 can generate vibrations. The motor 191 can be used for notification of incoming calls, and can also be used for touch feedback. The motor 191 can generate different vibration feedback effects for touch operations on different application programs. For touch operations acting on different areas of the display screen 194, the motor 191 can also generate different vibration feedback effects. Different application scenarios (for example, time reminder, receiving information, alarm clock and games) may correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging status and the change of the battery capacity, and can also be used to indicate messages, missed calls and notifications.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be inserted into the SIM card interface 195 to realize contact with the electronic device 100 , and can also be pulled out from the SIM card interface 195 to realize separation from the electronic device 100 .

The hardware system of the electronic device 100 is described in detail above, and the software system of the electronic device 100 is introduced below. The software system may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application uses a layered architecture as an example to exemplarily describe the software system of the electronic device 100 .

As shown in Figure 2, a software system adopting a layered architecture is divided into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the software system can be divided into four layers, which are application program layer, application program framework layer, Android Runtime (Android Runtime) and system library, and kernel layer respectively from top to bottom.

The application layer can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

The video processing method of the embodiment of the present application can be applied to camera applications or video applications; for example, the "follow people" function can be enabled in the settings of the electronic device, and after the electronic device detects that the video application program requests to open the camera , you can turn on the "Shadow Follower" function; or, you can enable the "Shadow Follower" function in the camera application. " function; the function of "following people's movement" can refer to the description in the follow-up Figure 3.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer can include some predefined functions.

For example, the application framework layer includes window managers, content providers, view systems, telephony managers, resource managers, and notification managers.

A window manager is used to manage window programs. The window manager can get the size of the display, determine whether there is a status bar, lock the screen, and capture the screen.

Content providers are used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, and phonebook.

The view system includes visual controls, such as those that display text and those that display pictures. The view system can be used to build applications. The display interface may be composed of one or more views, for example, a display interface including an SMS notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (connected or hung up).

The resource manager provides various resources to the application, such as localized strings, icons, pictures, layout files, and video files.

The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction.

Android Runtime includes core library and virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application program layer and the application program framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

The system library can include multiple functional modules, such as: surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (for example: open graphics library for embedded systems (open graphics library for embedded systems, OpenGL ES) and 2D graphics engine (for example: skia graphics library (skia graphics library, SGL)).

The surface manager is used to manage the display subsystem and provides the fusion of 2D layers and 3D layers for multiple applications.

The media library supports playback and recording of multiple audio formats, playback and recording of multiple video formats, and still image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, moving picture experts group audio layer III (MP3), advanced audio coding (AAC), auto Adaptive multi-rate (adaptive multi-rate, AMR), joint photographic experts group (joint photographic experts group, JPG) and portable network graphics (portable network graphics, PNG).

The 3D graphics processing library can be used to implement 3D graphics drawing, image rendering, compositing and layer processing.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer may include driver modules such as display driver, camera driver, audio driver and sensor driver.

In the following, the workflow of the software system and the hardware system of the electronic device 100 will be exemplarily described in combination with displaying a photographing scene.

When the user performs a touch operation on the touch sensor 180K, a corresponding hardware interrupt is sent to the kernel layer, and the kernel layer processes the touch operation into an original input event. The original input event includes information such as touch coordinates and a time stamp of the touch operation. The original input event is stored in the kernel layer, and the application framework layer obtains the original input event from the kernel layer, identifies the control corresponding to the original input event, and notifies the corresponding application (application, APP) of the control. For example, the above-mentioned touch operation is a single-click operation, and the APP corresponding to the above-mentioned control is a camera APP. After the camera APP is awakened by the single-click operation, it can call the camera driver of the kernel layer through the API, and control the camera 193 to take pictures through the camera driver.

FIG. 3 is a schematic diagram of an application scenario applicable to the present application, that is, the schematic diagram shown in FIG. 3 may refer to a scenario of “moving with shadows”.

For example, the principle of "following people's movements" may mean that the camera of the electronic device performs large-resolution acquisition according to a fixed field of view, detects and tracks the user on the captured video image frames, and locates the user's position in real time; When moving, the large-resolution video image frame can be cropped and scaled according to the real-time positioning of the user's position, and a small-resolution image that adapts to the display specifications and the user is located in a specific area of the image can be obtained, thereby realizing real-time monitoring according to the user's position. Adjust the display screen to achieve the effect of "moving with people".

In one example, the electronic device is a tablet device as an example, as shown in (a) in FIG. 211 and a control 212 for instructing video recording; before detecting that the user clicks on the control 212, a preview image may be displayed in the viewfinder frame 211.

It is detected that the user clicks on the operation of the control 212, and in response to the operation of the user, the tablet device can perform video shooting; when the first subject is at the first position, the first image frame as shown in (a) in Figure 3 is displayed; during video shooting During the process, the first photographed object moves; for example, the first photographed object moves from the first position to the second position, and the third image frame as shown in (b) in FIG. 3 is displayed after the first photographed object moves the position ; After the first shooting object moves, the first shooting object can always remain in the middle position in the viewfinder frame 211, and this shooting function is the "shadow follower" function; After the "manual movement" function, the position of the tablet device can be kept unchanged, and after the subject moves, the subject can always be displayed in the middle position or the middle area of the video display screen.

In the embodiment of the present application, when the first photographed object is at the first position, the first photographed object is located in the first area in the first image frame; when the first photographed object moves to the second position, the first photographed object is located in the first area The second area in the three image frames; wherein, there is an intersection between the first area and the second area.

In an example, the intersection between the first area and the second area may mean that the first area and the second area partly overlap, as shown in (a) in FIG. 4 and (b) in FIG. 4 .

In an example, the intersection between the first area and the second area may mean that the first area and the second area completely overlap, as shown in (c) in FIG. 4 .

Optionally, the first area and the second area may be located in a middle area of the display screen, and there is an intersection between the first area and the second area.

The scenario shown in Figure 3 above is described as an example, and the video processing method provided in the embodiment of the present application can be applied but not limited to the following scenarios:

Video calls, video conferencing applications, long-term and short-term video applications, live video applications, online video course applications, application scenarios of intelligent portrait movement, system camera recording function recording video, video surveillance, and portrait shooting scenarios such as smart peepholes, etc.

At present, the user tracking in the video display screen is usually realized by performing human body detection on the user, so as to realize the function of "following people's movement". Points may include but are not limited to: head, shoulders, arms, hands, legs, feet, eyes, nose, mouth, and clothes; High performance requirements.

In view of this, the embodiment of the present application provides a video processing method. In the embodiment of the present application, the video image frame of the target object is obtained after the target object moves, and the face detection is performed on the video image frame to determine the target object According to the coordinate information of the face frame, the coordinate information of the crop frame is obtained according to the coordinate information of the face frame; further, the video image frame is trimmed according to the crop frame to obtain the display content; in the embodiments of the application, due to the The coordinate information of the frame is used to obtain the coordinate information of the cropping frame, so compared with the scheme of directly detecting the key points of the human body of the target object to determine the cropping frame, the video processing method of the present application can reduce the amount of computation of the electronic device and reduce the power of the electronic device. In addition, because the video processing method of the present application is to determine the coordinate information of the cropping frame according to the face frame, it can avoid the video tracking display of the target object when the target object is facing away from the electronic device in the video image frame; therefore, the present application The solution can also improve the accuracy of video tracking and display while reducing power consumption.

The video processing method provided by the embodiment of the present application will be described in detail below with reference to FIG. 5 to FIG. 18 .

The video processing method provided in the embodiment of the present application may be used in a video mode, where the video mode may refer to the electronic device performing video shooting; or, the video mode may also refer to the electronic device performing video calls.

In a possible implementation, the function of "following people" can be set in the setting interface of the electronic device. " function executes the video processing method of the embodiment of the present application.

In a possible implementation, the camera of the electronic device can be set to enable the "follow people" function, and according to the settings, the "follow people" function can be turned on when recording a video, and the implementation of the embodiment of the present application can be performed. Video processing method.

Fig. 5 is a schematic flowchart of a video processing method provided by an embodiment of the present application. The video processing method 300 shown in FIG. 5 includes steps S301 to S316, and these steps will be described in detail below.

Step S301, request to turn on the camera.

For example, an application program in an electronic device sends an instruction requesting to turn on the camera; the application program may include but not limited to: WeChat video call application, video conferencing application, live video application, video recording application, camera application wait.

In one example, when the camera application program of the electronic device is recording a video, it may request to open the camera.

For example, as shown in FIG. 6 , it may be a request to turn on the camera when the user clicks the icon 411 of the camera application to shoot a video.

In an example, when the WeChat video call application in the electronic device initiates or receives a video invitation, it may request to turn on the camera.

For example, as shown in FIG. 6 , it may refer to a request to turn on the camera when the user clicks the icon 412 of the video application program to make a video call.

Step S302, the camera sensor detects an instruction requesting to turn on the camera, and the camera sensor acquires a video image frame (an example of a second image frame).

For example, the aforementioned camera sensor may refer to the image sensor in the camera module; the video image frame may refer to the image frame acquired by the image sensor in real time when the user's position changes.

Exemplarily, the resolution size of the video image frame acquired by the camera sensor may be a full size (full size).

For example, if the maximum resolution supported by the camera in the camera module is 4096*2160, then the resolution of the acquired full-size video image frame may be 4096*2160. Step S303, the application program issues a request resolution command.

Exemplarily, the application program may issue a request resolution instruction requesting that the video resolution is w1*h1 (an example of the first resolution); the request video resolution may refer to the processed video saved in the electronic device The resolution of the image frame.

Step S304, calculating the target resolution of the video image frame (an example of the second resolution).

For example, the resolution size requested by the application can be expanded to obtain the target resolution; for example, the requested w1*h1 resolution can be expanded by a certain factor to the resolution w2*h2 (w2>w1, h2>h1); where , the resolution w2*h2 may be the target resolution.

In the embodiment of the present application, the resolution is extended from w1*h1 to w2*h2, which can solve the problem of the video image frame definition degradation caused by subsequent cropping processing; by performing resolution extension processing, it is possible to To a certain extent, the clarity of the video image frame after the cropping process is improved.

Step S305, the ISP processes the video image frame according to the target resolution to obtain the video image frame of the target resolution.

Step S306, performing face detection on the video image frame to obtain the coordinate information of the face frame (an example of the first detection frame).

For example, an existing face detection algorithm may be used to perform face detection on video image frames acquired by a camera sensor to obtain coordinate information of the face frame.

In one example, since the processing of the full-size video image frame requires a large amount of calculation, in order to reduce the calculation amount during the processing of the video image frame, the full-size video image frame can be down-sampled; for example, the The full-size video image frame is down-sampled to obtain a video image frame with a resolution of w3*h3; the face detection is performed on a video image frame with a resolution of w3*h3 (w3<w1, h3<h1), and the obtained The coordinate information of the face frame.

Step S307 , judging whether to enable the identification of the owner of the device; if the identification of the owner of the device is enabled, execute step S308 ; if the identification of the owner of the device is not enabled, execute step S310 .

It should be understood that after the owner identification is turned on, only the owner user in the video image frame can be tracked and displayed; when the owner identification is not turned on, all users in the video image frame can be tracked and displayed; the owner can refer to the The administrative user of the tablet device; alternatively, the owner can also be any pre-configured user with higher priority.

Case 1: In the scene where the device owner identification is enabled

Step S308, performing face recognition according to the face frame.

Exemplarily, the image information in the face frame can be determined according to the coordinate information of the face frame; the face recognition is performed on the image information in the face frame; when the face recognition is performed on the image information in the face frame, the Matching is performed according to the face information database pre-stored in the electronic device, so as to determine the identity of the user corresponding to the image information in the face frame.

In one example, when the identification of the device owner is turned on, the face information database includes the face information of the user of the device, and the user of the device can be determined by matching the face information database with the image information in the face frame.

It should be noted that the face detection in step S306 is used to detect the coordinate information of the face frame in the image, that is, the face detection is used to detect the face area in the image; the face recognition is used to identify the face area corresponding to user identity information.

Step S309, acquiring the coordinate information of the face frame of the owner user.

For example, the owner user can be determined through step S308, so that the coordinate information of the face frame corresponding to the owner user can be determined.

Exemplarily, as shown in FIG. 18 , the image frame may include the first user and the second user; if the owner identification is enabled, the acquired coordinate information of the face frame may refer to the coordinates of the face frame of the owner user 711 information, such as the coordinate information of the rectangular frame 710 .

Step S310, performing coordinate transformation on the coordinate information of the face frame.

For example, perform down-sampling processing on the full-size video image frame to obtain a video image frame with a resolution of w3*h3; perform face detection on a w3*h3 video image frame to obtain the coordinate information of the face frame of the owner user ; Convert the coordinate information of the face frame of the owner user to the resolution coordinates of w2*h2, where w2>w3, h2>h3.

Exemplarily, as shown in FIG. 18, the coordinate information of the four vertices of the rectangular frame 710 is converted to obtain the corresponding vertex coordinate information at the w2*h2 resolution, and then determine the w2*h2 resolution image of the rectangular frame 720 location information in .

Case 2: The scene where the owner identification is not enabled

If the owner identification is not enabled, step S310 is executed after step S307 is executed; step S310, performing coordinate transformation on the coordinate information of the face frame detected in step S306.

In one example, as shown in FIG. 9 , the video image frame includes a single user, and the coordinate transformation of the face frame may refer to transforming the coordinate information of the four vertices of the rectangular frame 430 to obtain Corresponding vertex coordinate information, and then determine the position information of the rectangular frame 440 in the w2*h2 resolution image.

In one example, as shown in FIG. 12 , two users are included in the video image frame, and the coordinate transformation of the face frame may refer to transforming the coordinate information of the four vertices of the rectangular frame 510 to obtain The corresponding vertex coordinate information is determined, and then the position information of the rectangular frame 510 in the w2*h2 resolution image is determined, that is, the rectangular frame 520 is obtained.

Step S311 , calculating the coordinate information of the human body frame (an example of the second detection frame) according to the coordinate information of the human face frame after the coordinate transformation.

For example, according to step S310, the coordinate information of the human face frame after the coordinate conversion can be determined; according to the human body proportion data, the human face frame after the coordinate conversion is subjected to boundary extension processing (an example of the first extension processing), to obtain the coordinates of the human body frame information; wherein, the human body proportion data may be a preset value.

In an example, the upper boundary can be expanded outward by 0.5 times, the lower boundary can be expanded outward by 1.0 times, and the left boundary and right boundary can be expanded outward by 0.75 times, with the rectangular frame of the face as the center and the rectangular frame of the human face as the reference.

Exemplarily, as shown in FIG. 9, the single-user face frame as shown in the rectangular frame 440 can be subjected to boundary extension processing (an example of the first boundary extension), and the coordinates of the human body frame as shown in the rectangular frame 450 can be obtained. information.

Exemplarily, as shown in FIG. 12 , the multi-user human face frames shown in the rectangular frame 520 can be subjected to boundary extension processing (an example of the first boundary extension), and the two-user human body frames shown in the rectangular frame 530 can be obtained. coordinate information.

Step S312, calculating the coordinate information of the cropping frame according to the coordinate information of the human body frame.

For example, boundary extension processing (an example of the second boundary extension) may be performed according to the body frame to obtain coordinate information of the cropping frame.

In an example, the body frame can be taken as the center, and the upper boundary and the lower boundary can be expanded outward by 0.025 times, and the left boundary and the right boundary can be expanded outward by 0.025 times respectively based on the human body frame, to obtain a cropping frame.

Exemplarily, as shown in FIG. 9, a single-user human body frame (an example of the second detection frame) shown in the rectangular frame 450 may be subjected to boundary extension processing (an example of the second boundary extension), and a rectangular frame such as Coordinate information of the cropping frame shown in 460 .

Exemplarily, as shown in FIG. 12, the multi-user human body frame (an example of the second detection frame) shown in the rectangular frame 530 may be subjected to boundary extension processing (an example of the second boundary extension), to obtain a rectangular frame such as Coordinate information of the cropping frame shown in 540 .

In the embodiment of the present application, since the face detection algorithm detects based on each image frame in the video image frame, the output face frame may have local jumps in the time domain; in order to avoid The local jitter of the face frame ensures that the cropping frame remains unchanged when the user performs small movements; the cropping frame obtained by expanding the boundary of the human body frame can ensure the stability of the image frame after cropping to a certain extent sex.

Step S313 , judging the conditions of cropping processing and zooming processing.

For example, it may be determined whether the obtained human body frame and cropping frame satisfy a preset condition, so as to determine whether to perform subsequent cropping and scaling processing.

Exemplarily, the preset condition may refer to that the human body frame and the cropping frame meet a certain proportional relationship, and the human body frame is located inside the cropping frame.

In a possible implementation manner, when the body frame and the cropping frame do not meet the preset conditions, the above step S306 to step S312 are repeatedly executed to recalculate the coordinate information of the cropping frame.

Step S314 , parameter calculation for cropping and scaling.

For example, based on the coordinate information of the cropping frame and the screen coordinate information of the video image frame, according to the adjustment strategy of N frames of video image frames (for example, according to the smoothness requirement), the parameters for the clipping and scaling processing of the video image frame by the ISP are calculated, And deliver the parameters to the ISP.

In one example, after the cropping frame is determined, if the user is far away from the camera, that is, the display area of the user in the screen is small, the cropping frame can be enlarged to a certain extent; for example, the cropping frame can be zoomed in as the center, The maximum can be enlarged to 2 times the size of the crop frame to crop the video image frame.

Step S315 , the ISP receives the parameters of the cropping and scaling processing, and the ISP performs cropping and scaling processing on the video image frame.

For example, the ISP cuts the video image frame according to the coordinate information of the cropping frame to obtain the display content; it can scale the display content according to the requested resolution, so that the processed video image frame meets the requested resolution.

Step S316, displaying the video image frame (an example of the third image frame) in the application program.

For example, the video image frame that has been cropped and scaled by the ISP is transmitted to the application program, and the video image frame is displayed in the application program.

Exemplarily, the resolution size of the video image frame after the ISP cropping and scaling processing is the resolution size requested in step S303; the video image frame processed by the ISP is transmitted to the application program, and is resolved according to the display screen of the electronic device The video image frame is displayed at a rate size suitable for the display specifications of the electronic device.

In the embodiment of the present application, the video image frame of the target object is obtained after the target object moves, the face detection is performed on the video image frame, the coordinate information of the face frame of the target object is determined, and the coordinate information of the face frame is obtained according to the coordinate information of the face frame Coordinate information of the cropping frame; further according to the clipping frame, the video image frame is clipped to obtain the display content; in the embodiment of the application, since the coordinate information of the cropping frame is determined by the coordinate information of the face frame, it is directly related to the target Compared with the scheme of detecting the key points of the human body of the object to determine the cropping frame, the video processing method of the present application can reduce the amount of computation of the electronic device and reduce the power consumption of the electronic device; in addition, since the video processing method of the present application is based on the face frame Determining the coordinate information of the cropping frame can prevent the target object from performing video tracking and display on the target object when its back faces the electronic device in the video image frame; therefore, the solution of the present application can also improve the video frequency while reducing power consumption. The accuracy of the tracking display.

The process of video processing is described in detail below in conjunction with FIG. 6 to FIG. 18 , respectively, when the target object is a single user, multi-users do not enable the device owner identification, and multiple users enable the device owner identification.

Exemplarily, the electronic device is illustrated as a tablet device; FIG. 6 shows a graphical user interface (graphical user interface, GUI) of the tablet device, and the GUI is a desktop 410 of the tablet device; the desktop 410 may include a camera An icon 411 of the application program and an icon 412 of the video application program.

Case 1: Single-user automatic camera movement

In one example, a single user may be included in the video preview screen, and the video screen will automatically track this user at this time.

FIG. 7 is a display interface of a user using a tablet device to conduct a video call; as shown in FIG. box, a control to indicate Cancel, and a control to indicate Go to Speech. After the user initiates a video invitation to the other party through the tablet device, the camera of the tablet device collects a preview image with a fixed field of view and displays the display interface as shown in Figure 7; after the other party connects to the video call, the display as shown in Figure 8 can be displayed interface.

It should be understood that, in Figures 7 and 8, the electronic device may enable the function of "moving with the shadow", and the preview image collected by the camera may be cropped and zoomed through the video processing method provided by the embodiment of the present application, and the processing is suitable for the tablet device. A video image of the specification is displayed. When the camera is turned on in the tablet device, the video processing method provided in the embodiment of the present application will be executed.

The process of obtaining the video image shown in FIG. 7 will be described in detail with reference to FIG. 9 .

It should be understood that the processing shown in FIG. 9 is executed by a processor inside the tablet device or a chip configured in the tablet device, and the processing process will not be displayed on the display interface.

Exemplarily, for a single-user scenario, step S306 shown in FIG. 5 above may be to obtain a rectangular frame 430 as shown in FIG. 430 is converted to a rectangular frame 440, which represents the face frame after coordinate transformation. For example, the coordinate information of the four vertices of the rectangular frame 430 is converted to obtain the corresponding vertex coordinate information at the w2*h2 resolution, and then the position information of the rectangular frame 440 in the w2*h2 resolution image is determined. In step S311 , as shown in FIG. 9 , the coordinate information of the rectangular frame 450 can be obtained by performing boundary extension processing according to the rectangular frame 440 , and the rectangular frame 450 represents a single user's body frame. In step S312 , as shown in FIG. 9 , the coordinate information of the rectangular frame 460 can be obtained by performing boundary extension processing according to the rectangular frame 450 , and the rectangular frame 460 represents a single user's cropping frame.

Further, when the rectangular frame 450 and the rectangular frame 460 meet the preset conditions, determine the parameters of the cropping process and the scaling process according to the coordinate information of the rectangular frame 460 and the coordinate information of the video image frame; The image frame is cropped and scaled to obtain an output video image frame suitable for the display specification of the tablet device.

For example, as shown in Figure 9, the cropped display content can be obtained according to the cropping frame 460; the displayed content can be scaled according to the requested resolution size to obtain the processed video image frame; the processed video The image frame is sent to the video call application program, and the video image frame suitable for the display specification of the tablet device is obtained according to the resolution of the display screen of the tablet device.

It should be noted that, for the specific steps above, refer to the relevant description shown in FIG. 5 , and details are not repeated here.

Case 2: Multi-user automatic camera movement

In an example, multiple users may be included in the video preview screen, and the video screen may be automatically adjusted according to the positions of all users to ensure that all users are displayed in the video screen.

Fig. 10 is a display interface of a user using a tablet device for video calling; as shown in Fig. There are controls to indicate cancel and controls to indicate turn to speech. During the video call, the tablet device may display a display interface as shown in FIG. 11 .

It should be understood that in Figures 10 and 11, the function of "moving with the shadow" is enabled on the electronic device, and the preview image captured by the camera is processed and scaled through the video processing method provided by the embodiment of the present application, and processed into a display suitable for the tablet device specifications of the video image. When the camera is turned on in the tablet device, the video processing method provided in the embodiment of the present application will be executed.

The process of obtaining the video image frame shown in FIG. 11 will be described in detail with reference to FIG. 12 . It should be understood that the processing shown in FIG. 12 is executed by a processor inside the tablet device or a chip configured in the tablet device, and the processing process will not be displayed on the display interface.

Exemplarily, for a scenario where multiple users do not enable the owner identification, step S306 shown in FIG. 5 can determine the minimum union set including all multi-user face frames according to the coordinate information of each user's face frame as shown in FIG. 12 Coordinate information of the frame, a multi-user face frame such as the rectangular frame 510 . Step S310 may transform the matrix frame 510 into a rectangular frame 520 as shown in FIG. 12 , and the rectangular frame 520 represents the face frame after coordinate transformation. For example, the coordinate information of the four vertices of the rectangular frame 510 is converted to obtain the corresponding vertex coordinate information at the w2*h2 resolution, and then the position information of the rectangular frame 520 in the w2*h2 resolution image is determined. In step S311 , as shown in FIG. 12 , the coordinate information of the rectangular frame 530 can be obtained by performing boundary extension processing according to the rectangular frame 520 , and the rectangular frame 530 represents the body frames of multiple users. Step S312 , as shown in FIG. 12 , performs boundary extension processing according to the rectangular frame 530 to obtain the coordinate information 540 , and the rectangular frame 540 represents the cropping frame of multiple users.

Further, when the rectangular frame 540 and the rectangular frame 550 meet the preset conditions, determine the parameters of the cropping process and the scaling process according to the coordinate information of the rectangular frame 550 and the coordinate information of the video image frame; The image frame is cropped and scaled to obtain an output video image frame suitable for the display specification of the tablet device.

For example, as shown in Figure 12, the cropped display content can be obtained according to the cropping frame 540; the displayed content can be scaled according to the requested resolution size to obtain the processed video image frame; the processed video The image frame is sent to the video call application program, and the video image frame suitable for the display specification of the tablet device is obtained according to the resolution of the display screen of the tablet device.

It should be understood that the multi-user automatic mirror movement is compared with the single-user automatic mirror movement; when the multi-user automatic mirror movement determines the coordinate information of the multi-user face frame, the multi-user is determined according to the minimum union of the face frames of each user in the multi-user. The coordinate information of the user's face frame; the rest of the steps are the same as the single-user mirror operation, please refer to the relevant content description of the single-user mirror operation, and will not be repeated here.

In one example, as shown in FIG. 13 , the shooting scene corresponding to the video call interface 503 may include a first shooting object 504, a second shooting object 505, and a third shooting object 506; The object 505 is the user's face facing the camera, and the third object 506 is the user's face facing the camera; therefore, the video processing method provided according to the embodiment of the present application cannot detect the third object when performing face detection. The coordinate information of the face frame of the subject 506 can be tracked and displayed on the first subject 504 and the second subject 505 during user tracking, and the third subject 506 cannot be tracked and displayed; that is, the first subject 504 After moving with the second object 505, the first object 504 and the second object 505 can be tracked and displayed, so that the first object 504 and the second object 505 can always remain in the middle area of the video display screen; for example The display interface shown in Figure 14.

Situation 3: The owner automatically moves the mirror

In one example, the owner tracking mode can be turned on, and multiple users can be included in the video preview screen, face detection and face recognition are performed on multiple users to determine the target user is the owner user, and the video screen can realize the tracking of the owner user. Do automatic tracking.

FIG. 15 is a setting display interface for video passing; as shown in FIG. 15 , the main character mode can be turned on in the setting display interface 601 , and the main character mode can refer to turning on the owner identification as shown in FIG. 5 . 16 is a display interface of a user using a tablet device for a video call; the display interface may include a video call interface 602, and the video call interface 602 may include the first subject, a control for indicating cancellation, and a control for indicating to switch to Voice controls. During the video call, a display interface as shown in Figure 17 can be displayed.

It should be understood that in Figure 16 and Figure 17, the electronic device turns on the function of "moving with the shadow", and the video processing method provided by the embodiment of the present application performs cropping and scaling processing on the preview image captured by the camera, and processes it into a display suitable for a tablet device specifications of the video image. When the camera is turned on in the tablet device, the video processing method provided in the embodiment of the present application will be executed.

The process of obtaining the video image frame shown in FIG. 17 will be described in detail with reference to FIG. 18 . It should be understood that the processing shown in FIG. 18 is executed by a processor inside the tablet device or a chip configured in the tablet device, and the processing process will not be displayed on the display interface.

Exemplarily, for a scenario where multiple users enable the identification of the owner of the device, step S306 shown in FIG. 5 may acquire the coordinate information of the rectangular frame 710 as shown in FIG. 18 . Step S310 may transform the matrix frame 710 into a rectangular frame 720 as shown in FIG. 18 , and the rectangular frame 720 represents the face frame of the owner user after coordinate transformation. For example, the coordinate information of the four vertices of the rectangular frame 710 is converted to obtain the corresponding vertex coordinate information at the w2*h2 resolution, and then the position information of the rectangular frame 720 in the w2*h2 resolution image is determined. In step S311 , as shown in FIG. 18 , the coordinate information of the rectangular frame 730 can be obtained by performing boundary expansion processing according to the rectangular frame 720 , and the rectangular frame 730 represents the body frame of the owner user. In step S312 , as shown in FIG. 18 , the coordinate information of 740 can be obtained by performing border extension processing according to the rectangular frame 730 , and the rectangular frame 740 can represent the cropping frame of the owner user.

Further, when the rectangular frame 730 and the rectangular frame 740 meet the preset conditions, the parameters of the cropping process and the scaling process can be determined according to the coordinate information of the rectangular frame 740 and the coordinate information of the video image frame; The video image frame is clipped and scaled to obtain an output video image frame suitable for the display specification of the tablet device.

For example, as shown in Figure 18, the cropped display content can be obtained according to the cropping frame 740; the displayed content can be scaled according to the issued request resolution to obtain the processed video image frame; the processed video The image frame is sent to the video call application program, and the video image frame suitable for the display specification of the tablet device is obtained according to the resolution of the display screen.

It should be understood that the automatic mirror movement of the owner user is compared with the automatic mirror movement of a single user; the automatic mirror movement of the owner user will carry out the face frame of each user after determining the coordinate information of the face frame of each user among the multi-users. Face recognition, so as to determine the coordinate information of the face frame of the owner user; the rest of the steps are the same as single-user mirror operation, please refer to the relevant content description of single-user mirror operation, and will not repeat them here.

In the embodiment of the present application, by performing face detection on the acquired video image frame, the coordinate information of the face frame of the target object is determined, and the coordinate information of the cropping frame is obtained according to the coordinate information of the human face frame; The image frame is processed to display the output video image frame; in the embodiment of the present application, since the coordinate information of the cropping frame is determined by the coordinate information of the face frame, it is different from directly detecting the key points of the human body of the target object to determine the cropping frame. Compared with other solutions, the video processing method of the present application can reduce the computation load of the electronic device and reduce the power consumption of the electronic device; in addition, since the video processing method of the present application determines the coordinate information of the cropping frame according to the face frame, it can avoid the When facing back to the electronic device in the second image frame, video tracking and display is performed on the target object; therefore, the solution of the present application can also improve the accuracy of video tracking and display while reducing power consumption.

It should be understood that the above illustrations are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the illustrated specific values or specific scenarios. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples given, and such modifications or changes also fall within the scope of the embodiments of the present application.

The video processing method provided by the embodiment of the present application is described in detail above with reference to FIG. 1 to FIG. 18 ; the device embodiment of the present application will be described in detail below in conjunction with FIG. 19 and FIG. 20 . It should be understood that the devices in the embodiments of the present application can execute the various methods in the foregoing embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the foregoing method embodiments.

FIG. 19 is a schematic structural diagram of a video processing device provided by an embodiment of the present application. The video processing device 800 includes a display unit 810 and a processing unit 820 .

Wherein, the display unit 810 is configured to display a first image frame, and the first image frame is an image frame of the target object at the first position; the processing unit 820 is configured to display the target object when the target object moves to the second position , acquiring a second image frame, the second position is different from the first position, and the second image frame refers to an image frame collected by the electronic device when the target object moves to the second position ; Perform face detection according to the second image frame to obtain coordinate information of a first detection frame, the first detection frame is used to indicate the position information of the face of the target object in the second image frame; Obtain the coordinate information of the cropping frame according to the first detection frame; perform cropping processing on the second image frame according to the cropping frame to obtain the display content including the target object; the display unit 810 is also configured to obtain the display content according to the The above display content displays a third image frame, wherein there is an intersection between the first area in the first image frame and the second area in the third image frame, and the first area refers to the first image frame The area where the target object is located in, the second area refers to the area where the target object is located in the third image frame.

Optionally, as an embodiment, when the first image frame and the third image frame are displayed, the video processing device is located at the same position.

Optionally, as an embodiment, the processing unit 820 is further configured to:

An action is detected indicating that the camera application is running; or,

An action was detected indicating to run a video calling application.

Optionally, as an embodiment, the processing unit 820 is specifically configured to:

Optionally, as an embodiment, the coordinate information of the first detection frame refers to the coordinate information corresponding to the first detection frame when the second image frame is of the second resolution, and the processing unit 820 specifically Used for:

determining the second resolution based on the first resolution, the second resolution being greater than the first resolution;

The display unit 810 is used for:

displaying the third image frame according to the processed display content.

Optionally, as an embodiment, the target object is the owner user, and the processing unit 820 is specifically configured to:

Optionally, as an embodiment, the first detection frame refers to a face frame of the owner user of the device.

Optionally, as an embodiment, the target object includes at least one user.

Optionally, as an embodiment, the target object includes a first user and a second user, and the first detection frame refers to the difference between the face frame of the first user and the face frame of the second user. Union box.

Optionally, as an embodiment, the first area coincides with the second area.

It should be noted that the video processing apparatus 800 is embodied in the form of functional units. The term "unit" here may be implemented in the form of software and/or hardware, which is not specifically limited.

For example, a "unit" may be a software program, a hardware circuit or a combination of both to realize the above functions. The hardware circuitry may include application specific integrated circuits (ASICs), electronic circuits, processors (such as shared processors, dedicated processors, or group processors) for executing one or more software or firmware programs. etc.) and memory, incorporating logic, and/or other suitable components to support the described functionality.

Therefore, the units of each example described in the embodiments of the present application can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

FIG. 20 shows a schematic structural diagram of an electronic device provided by the present application. The dotted line in FIG. 20 indicates that this unit or this module is optional, and the electronic device 900 can be used to implement the video processing method described in the foregoing method embodiments.

The electronic device 900 includes one or more processors 901, and the one or more processors 901 can support the electronic device 900 to implement the method in the method embodiment. The processor 901 may be a general purpose processor or a special purpose processor. For example, the processor 901 may be a central processing unit (central processing unit, CPU), a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices such as discrete gates, transistor logic devices, or discrete hardware components.

The processor 901 can be used to control the electronic device 900, execute software programs, and process data of the software programs. The electronic device 900 may further include a communication unit 905, configured to implement input (reception) and output (send) of signals.

For example, the electronic device 900 can be a chip, and the communication unit 905 can be an input and/or output circuit of the chip, or the communication unit 905 can be a communication interface of the chip, and the chip can be used as a component of a terminal device or other electronic devices .

For another example, the electronic device 900 may be a terminal device, and the communication unit 905 may be a transceiver of the terminal device, or the communication unit 905 may be a transceiver circuit of the terminal device.

The electronic device 900 may include one or more memories 902, on which a program 904 is stored, and the program 904 may be run by the processor 901 to generate an instruction 903, so that the processor 901 executes the video processing described in the above method embodiment according to the instruction 903 method.

Optionally, data may also be stored in the memory 902 . Optionally, the processor 901 may also read data stored in the memory 902, the data may be stored in the same storage address as the program 904, and the data may also be stored in a different storage address from the program 904.

The processor 901 and the memory 902 may be set separately, or may be integrated together; for example, integrated on a system-on-chip (system on chip, SOC) of a terminal device.

Exemplarily, the memory 902 can be used to store the related program 904 of the video processing method provided in the embodiment of the present application, and the processor 901 can be used to call the related program 904 of the video processing method stored in the memory 902 during video processing, and execute The video processing method of the embodiment of the present application; for example, displaying a first image frame, where the first image frame is an image frame of a target object at a first position; when the target object moves to a second position, acquiring a second image frame; The second position is different from the first position, and the second image frame refers to the image frame collected by the electronic device when the target object moves to the second position; the face detection is performed according to the second image frame, and the coordinate information of the first detection frame is obtained , the first detection frame is used to indicate the position information of the face of the target object in the second image frame; the coordinate information of the cropping frame is obtained according to the first detection frame; the second image frame is cropped according to the cropping frame to obtain the The display content of the object; the third image frame is displayed according to the display content, the first area in the first image frame overlaps with the second area in the third image frame, and the first area refers to the location of the target object in the first image frame The second area refers to the area where the target object is located in the third image frame.

The present application also provides a computer program product, which implements the video processing method described in any method embodiment in the present application when the computer program product is executed by the processor 901 .

The computer program product can be stored in the memory 902 , such as a program 904 , and the program 904 is finally converted into an executable object file executable by the processor 901 through processes such as preprocessing, compiling, assembling and linking.

The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a computer, the video processing method described in any method embodiment in the present application is implemented. The computer program may be a high-level language program or an executable object program.

Optionally, the computer-readable storage medium is, for example, the memory 902 . The memory 902 may be a volatile memory or a nonvolatile memory, or, the memory 902 may include both a volatile memory and a nonvolatile memory. Among them, the non-volatile memory can be read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erases programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory can be random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, many forms of RAM are available such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (synchlink DRAM, SLDRAM ) and direct memory bus random access memory (direct rambus RAM, DR RAM).

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and technical effects of the devices and equipment described above can refer to the corresponding processes and technical effects in the foregoing method embodiments, here No longer.

In several embodiments provided in this application, the disclosed systems, devices and methods may be implemented in other ways. For example, some features of the method embodiments described above may be omitted, or not implemented. The device embodiments described above are only illustrative, and the division of units is only a logical function division. In actual implementation, there may be other division methods, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the various units or the coupling between the various components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

It should be understood that in various embodiments of the present application, the sequence numbers of the processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, rather than by the embodiments of the present application. The implementation process constitutes any limitation.

Additionally, the terms "system" and "network" are often used herein interchangeably. The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and A and B exist alone. There are three cases of B. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

In a word, the above descriptions are only preferred embodiments of the technical solutions of the present application, and are not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

A video processing method, characterized in that the video processing method is applied to electronic equipment, including:

displaying a first image frame, where the first image frame is an image frame of a target object at a first position;

When the target object moves to a second position, acquire a second image frame, the second position is a different position from the first position, and the second image frame means that the target object moves to the An image frame collected by the electronic device at the second position;

Perform face detection according to the second image frame to obtain coordinate information of a first detection frame, the first detection frame is used to indicate the position information of the face of the target object in the second image frame;

Obtaining the coordinate information of the cropping frame according to the first detection frame;

performing cropping processing on the second image frame according to the cropping frame to obtain display content including the target object;

Displaying a third image frame according to the display content, where the first area in the first image frame overlaps with the second area in the third image frame, and the first area refers to the first image frame The area where the target object is located in, the second area refers to the area where the target object is located in the third image frame.
The video processing method according to claim 1, wherein when displaying the first image frame and the third image frame, the electronic device is located at the same position.
The video processing method according to claim 1 or 2, further comprising:

An action is detected indicating that the camera application is running; or,

An action was detected indicating to run a video calling application.
The video processing method according to any one of claims 1 to 3, wherein said obtaining the coordinate information of the cropping frame according to the first detection frame includes:

performing a first extension process on the first detection frame to obtain a second detection frame;

performing a second extension process on the second detection frame to obtain the cropping frame;

Wherein, the first expansion process refers to expanding the boundary of the first detection frame with the first detection frame as the center, and the second detection frame is used to indicate that the body of the target object is within the first detection frame. For the location information in the second image frame, the second extension process refers to extending the boundary of the second detection frame with the second detection frame as the center.
The video processing method according to claim 4, wherein said performing first extension processing on said first detection frame to obtain a second detection frame comprises:

The first expansion process is performed on the first detection frame according to a first threshold to obtain the second detection frame, and the first threshold is used to indicate body proportion data.
The video processing method according to claim 4 or 5, characterized in that, performing cropping processing on the second image frame according to the cropping frame to obtain display content including the target object, comprising:

determining whether the second detection frame and the cropping frame meet a preset condition, the preset condition means that the second detection frame and the cropping frame satisfy a preset proportional relationship;

When the second detection frame and the cropping frame meet the preset condition, the second image frame is cropped according to the cropping frame to obtain the display content.
The video processing method according to any one of claims 1 to 6, wherein the coordinate information of the first detection frame refers to the first detection frame when the second image frame is at the second resolution. The coordinate information corresponding to the frame also includes:

receiving a request instruction, where the request instruction is used to request a first resolution;

The second resolution is determined according to the first resolution, and the second resolution is greater than the first resolution.
The video processing method according to claim 7, wherein said displaying a third image frame according to said display content comprises:

performing scaling processing on the display content according to the first resolution to obtain the processed display content;

displaying the third image frame according to the processed display content.
The video processing method according to any one of claims 1 to 8, wherein the target object is the owner user, further comprising:

Receive an owner identification instruction, the owner identification instruction is used to instruct to identify the owner user;

Face recognition is performed according to the first detection frame to determine the owner user, and the owner user is a pre-configured user.
The video processing method according to claim 9, wherein the first detection frame refers to a face frame of the owner user.
The video processing method according to any one of claims 1 to 8, wherein the target object includes at least one user.
The video processing method according to claim 11, wherein the target object includes a first user and a second user, and the first detection frame refers to the face frame of the first user and the second user. The union frame of the user's face frames.
The video processing method according to any one of claims 1 to 12, wherein the first area overlaps with the second area.
An electronic device, characterized in that the electronic device includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic The device executes the video processing method according to any one of claims 1 to 13.
A chip, characterized by comprising a processor, and when the processor executes instructions, the processor executes the video processing method according to any one of claims 1 to 13.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the The video processing method described above.
A computer program product, characterized in that the computer program product includes computer program code, and when the computer program code is executed by a processor, the processor is made to execute the video recording according to any one of claims 1 to 13. Approach.