WO2016165598A1

WO2016165598A1 - Video preprocessing method and device

Info

Publication number: WO2016165598A1
Application number: PCT/CN2016/078984
Authority: WO
Inventors: 郝春贺; 刘国喜; 李均荣
Original assignee: 中兴通讯股份有限公司
Priority date: 2015-04-17
Filing date: 2016-04-11
Publication date: 2016-10-20
Also published as: CN106161872A

Abstract

A video preprocessing method comprises: acquiring a video input source signal; checking, via a field programmable gate array (FPGA), the video input source signal, and obtaining a check result; and encoding, according to the check result, the video input source signal.

Description

Video preprocessing method and device

Technical field

This application relates to, but is not limited to, the field of communications.

Background technique

In the video conferencing system, the analog to digital (Analog to Digital, abbreviated as A/D) chip is generally used to directly send the captured video to the video encoding module, so once the image output by the A/D chip is abnormal, Will directly affect the encoding of the video.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

This paper provides a video preprocessing method and apparatus to avoid abnormality of the video image output by the A/D chip.

A video preprocessing method includes:

Obtaining a video input source signal;

The video input source signal is detected by a field programmable gate array FPGA, and the detection result is obtained;

And encoding the video input source signal according to the detection result.

Optionally, the detection result includes at least one of the following:

a determination result of whether the format of the video input source signal is abnormal, a determination result of whether the image quality of the video input source signal is abnormal, or a determination result of whether the video input source signal has a large amount of motion.

Optionally, the determination result that the format of the video input source signal is abnormal includes at least one of the following:

Anomaly of synchronization information of the video input source signal, aspect ratio of the video input source signal Anomaly of the example; and/or,

The determination of the image quality abnormality of the video input source signal includes:

The brightness of the video input source signal exceeds a first predetermined threshold range and/or the chrominance of the video input source signal exceeds a second predetermined threshold range.

Optionally, it also includes:

Before the encoding process of the video input source signal according to the detection result, if the format of the video input source signal is abnormal, the abnormal video input source signal is adjusted to a normal video. Input source signal.

Optionally, it also includes:

Adjusting the brightness and/or color of the video input source signal in a case where it is determined that the image quality of the video input source signal is abnormal before the encoding of the video input source signal according to the detection result degree.

Optionally, the encoding process of the video input source signal according to the detection result includes:

In a case where the video input source signal has a determination result of a large amount of motion, the video input source signal is encoded according to a motion complexity and/or a motion vector of the video input source signal.

Optionally, the video input source signal is video data that is modulo A/D converted and prior to encoding.

A computer readable storage medium storing computer executable instructions that, when executed, implement the video preprocessing method described above.

A video preprocessing apparatus includes:

a first acquiring module, configured to: acquire a video input source signal;

The second obtaining module is configured to: detect the video input source signal by using a field programmable gate array FPGA, and obtain a detection result;

The encoding module is configured to: perform encoding processing on the video input source signal according to the detection result.

Optionally, the detection result includes at least one of the following:

An abnormality of the synchronization information of the video input source signal, an abnormality of the aspect ratio of the video input source signal; and/or,

Through the embodiment of the invention, the video input source signal is acquired; the video input source signal is detected by the field programmable gate array FPGA, and the detection result is obtained; the video input source signal is encoded according to the detection result, thereby avoiding A/ The video image output by the D chip is abnormal. Thus, the process of video pre-processing is improved by the embodiment of the present invention.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

BRIEF abstract

1 is a flowchart of a video pre-processing method according to an embodiment of the present invention;

2 is a block diagram showing the structure of a video pre-processing apparatus according to an embodiment of the present invention;

3 is a block diagram of access of a pre-processing module at a front end of a video conference system according to an embodiment of the present invention;

FIG. 4 is a flowchart of accessing a pre-processing module at a front end of a video conference system according to an embodiment of the present invention.

Preferred embodiment of the invention

Embodiments of the present invention will be described below with reference to the accompanying drawings. It should be noted that there is no conflict In the case, the embodiments of the present application and the various aspects of the embodiments may be combined with each other.

A video preprocessing method is provided in this embodiment. FIG. 1 is a flowchart of a video preprocessing method according to an embodiment of the present invention. As shown in FIG. 1, the process includes the following steps:

Step S102, acquiring a video input source signal;

Step S104, detecting a video input source signal through a field programmable gate array FPGA, and obtaining a detection result;

Step S106, encoding the video input source signal according to the detection result.

Through the above steps, the analog input A/D converted and unencoded video input source signal is detected by using FPGA technology, and the detected result is obtained, and the video input source signal is subjected to subsequent encoding processing according to the information indicated by the detection result. Therefore, the video image outputted by the A/D chip is prevented from being abnormal. Thus, through the above steps, the process of video preprocessing is improved. In addition, compared with the related art, the use of other processors to complete the video preprocessing has poor real-time performance and the processing is not smart enough. The above steps solve the problem that the video image output by the A/D chip in the related art may be abnormal, and then perfect. The process of video preprocessing.

There are many kinds of detection results obtained by detecting the video input source signal by the FPGA in the above step S104, which will be exemplified below.

In an optional embodiment, the detection result may be a determination result of whether the format of the video input source signal is abnormal, a determination result of whether the image quality of the video input source signal is abnormal, or a determination result of whether the video input source signal has a large motion amount. .

In an optional embodiment, the determination result that the format of the video input source signal is abnormal may be an abnormality of the synchronization information of the video input source signal or an abnormality of the aspect ratio of the video input source signal.

In another alternative embodiment, the determination of the image quality abnormality of the video input source signal may be that the brightness of the video input source signal exceeds a first predetermined threshold range and/or the chrominance of the video input source signal exceeds a second predetermined threshold range .

The purpose of pre-processing the video input source signal is to make the video signal before encoding the correct signal. In an alternative embodiment, the video input source is determined before the video input source signal is encoded according to the detection result. In the case where the format of the signal is abnormal, the view of the exception will be The frequency input source signal is adjusted to a normal video input source signal. In another optional embodiment, before determining the image input source signal according to the detection result, adjusting the brightness of the video input source signal and/or determining that the image quality of the video input source signal is abnormal. Chroma. This ensures the correctness of the video signal before encoding.

In an alternative embodiment, in the case where the video input source signal has a determination result of a large amount of motion, the video input source signal is encoded according to the motion complexity and/or motion vector of the video input source signal. Further, encoding processing is performed according to the video input source signal.

The embodiment of the invention further provides a computer readable storage medium storing computer executable instructions, which are implemented when the computer executable instructions are executed.

In the embodiment, a video pre-processing device is provided, which is configured to implement the foregoing embodiments and optional implementations, and details are not described herein. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the devices described in the following embodiments are optionally implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

2 is a structural block diagram of a video pre-processing apparatus according to an embodiment of the present invention. As shown in FIG. 2, the apparatus includes:

The first obtaining module 22 is configured to: acquire a video input source signal;

The second obtaining module 24 is configured to: detect the video input source signal through the field programmable gate array FPGA, and obtain the detection result;

The encoding module 26 is configured to perform encoding processing on the video input source signal according to the detection result.

Through the above module, the analog input A/D converted and unencoded video input source signal is detected by using FPGA technology, and the detection result is obtained, and the video input source signal is subjected to subsequent encoding processing according to the information indicated by the detection result. Therefore, the video image outputted by the A/D chip is prevented from being abnormal. Thus, through the above modules, the process of video preprocessing is improved. In addition, compared with the related art, the use of other processors to complete the video preprocessing is not good in real-time and the processing is not smart enough. The above module solves the problem that the video image output by the A/D chip in the related art may be abnormal, and then perfects. The process of video preprocessing.

Optionally, the foregoing detection result includes at least one of the following:

Whether the format of the video input source signal is abnormal or not, whether the image quality of the video input source signal is abnormal, or whether the video input source signal has a large motion amount determination result.

The determination of the image quality anomaly of the video input source signal includes:

It should be noted that the above modules can be implemented by software or hardware. The first acquisition module 22, the second acquisition module 24, and the encoding module 26 are all located in the same processor; or the first acquisition module 22 and the second acquisition module 24 are the same. And encoding module 26 are located in the first processor, the second processor, and the third processor, respectively.

The output of the A/D chip is affected by its parameter configuration and hardware circuitry. The configuration of the current A/D parameters depends on the subjective judgment of the person, and the hardware circuit causes the abnormality of the chip output to be lack of detection means. At the same time, the video coding consumes a central processing unit (CPU) resource, especially when the video content has a large amount of motion. If a co-processing unit can be provided to complete the detection of large amounts of motion in the video, it will be able to save the encoder resources and improve its performance. The video image outputted by the A/D chip in the related art may have an abnormal problem, and an effective solution has not been proposed.

For the above problems in the related art, the following description will be made in conjunction with the detailed embodiments, which are combined with the above-described alternative embodiments and alternative embodiments thereof.

The current application of Field Programable Gate Array (FPGA) for video image processing has also become popular, but generally it is the application of FPGA for video scaling, coupling, synthesis, matrix switching or codec. The FPGA is rarely combined with the video capture front end and the video processing back end to detect image content.

The present invention is directed to the shortcomings of the above-mentioned background art, and an FPGA-based video pre-processing, which utilizes the advantages of FPGA parallel processing, can detect and process A/D-converted data in real time. Pre-processing of video pre-processing through FPGA, and feedback of pre-processing results For the relevant unit, the video anomaly dynamic detection of the front end, the adaptive adjustment of the A/D parameters, and the detection of the large motion amount in the video are realized.

The main purpose of this alternative embodiment is to provide a pre-processing mechanism based on FPGA front-end video. To achieve the above objective, the optional embodiment uses the FPGA to perform pre-processing on the front-end video signal, so that the entire video conference system completes the detection of the video anomaly at the front end, the adaptive adjustment of the A/D parameters, and the large motion detection of the video. .

The above preprocessing unit includes abnormality detection of video data, quality detection of video, and large motion amount determination.

The video format abnormality detection determines whether there is an abnormality in the format of the input video source, and the specific abnormality includes an abnormality of the synchronization information of the video, and an abnormality of the width and height of the video.

The video image quality detection described above uses the correlation algorithm of the image processing to complete the luminance and chrominance extraction in the video, and can determine whether the video has abnormalities in luminance and chrominance according to the actually set threshold value;

The large motion amount determination of the above video can use the motion detection algorithm to determine whether the video has a large amount of motion.

FIG. 3 is a block diagram of accessing a pre-processing module at a front end of a video conferencing system according to an embodiment of the present invention. As shown in FIG. 3, the entire pre-processing unit includes a video format abnormality detecting module 301, a video image quality detecting module 302, a video large motion amount detecting module 303, and a reporting module 304.

The format abnormality detecting module 301 of the above video determines whether the video format after the A/D conversion is abnormal. The detected content completed by the module includes whether the synchronization information of the video is correct, and whether there is a problem with the width and height of the video.

The image quality detecting module 302 of the above video analyzes the luminance and chrominance components in the video, and determines whether the video has an abnormality in luminance and chrominance according to the actually set threshold value;

The video large motion detection module 303 uses a motion detection algorithm of the video to determine whether the video has a large amount of motion;

The reporting module 304 shown feeds back the detection results of each of the modules mentioned above to the relevant unit.

4 is an access of a preprocessing module to a front end of a video conference system according to an embodiment of the present invention. flow chart. The specific implementation process of the front end of the video conference system in the embodiment of the present invention is described below with reference to FIG. 4:

Step 401, reading the A/D converted video signal to obtain a video input source;

Step 402: Preprocessing of the video, performing format abnormality determination, image quality detection processing, and large motion amount determination processing on the video by using the FPGA;

Step 403, the FPGA reports the pre-processed result to the main control unit;

Step 404: The main control unit receives the pre-processing result.

Step 405: The main control unit determines whether there is an abnormality in the format of the input video source, and if there is such an abnormality, step 408 is executed to schedule the exception processing unit.

Step 406: The main control unit determines whether there is abnormality of the brightness and chrominance of the video in the pre-processing result. If the type is abnormal, step 409 is performed, and the relevant parameter under the abnormality is fed back to the A/D parameter control unit to complete the pair. Automatic adjustment of brightness and chromaticity of the video;

Step 407, the main control unit determines whether there is a large amount of motion for the pre-processing result, if it is a large-volume video, step 410 is performed, and the detected motion complexity or motion vector is sent to the video codec unit;

Step 408, scheduling an exception processing unit to complete related processing of the abnormal video;

Step 409, scheduling an A/D parameter control unit to complete automatic adjustment of video brightness and chrominance;

Step 410: The video codec unit implements encoding of the video.

In summary, the preprocessing mechanism based on the FPGA front-end video provided by the embodiment of the present invention preprocesses the video in real time. The advantage of using FPGA to process data in parallel solves the problem of video preprocessing that other processors can perform, but the problem of poor real-time performance. At the same time, after the video is preprocessed by the FPGA, all the preprocessing results are fed back to the relevant unit in real time, so that the whole system can intelligently complete the processing of the front end video anomaly, the adaptive adjustment of the A/D parameters and the large motion amount in the video. determination.

In another embodiment, a software is also provided, which is applied to perform the technical solutions described in the above embodiments and alternative embodiments.

In another embodiment, a storage medium is further provided, wherein the storage medium has an upper storage Software, including but not limited to: optical disks, floppy disks, hard disks, rewritable memories, and the like.

One of ordinary skill in the art will appreciate that all or a portion of the steps of the above-described embodiments can be implemented using a computer program flow, which can be stored in a computer readable storage medium, such as on a corresponding hardware platform (eg, The system, device, device, device, etc. are executed, and when executed, include one or a combination of the steps of the method embodiments.

Alternatively, all or part of the steps of the above embodiments may also be implemented by using an integrated circuit. These steps may be separately fabricated into individual integrated circuit modules, or multiple modules or steps may be fabricated into a single integrated circuit module. achieve.

The devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.

When the device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium. The above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Industrial applicability

Claims

A video preprocessing method includes:

Obtaining a video input source signal;

The video input source signal is detected by a field programmable gate array FPGA, and the detection result is obtained;

And encoding the video input source signal according to the detection result.
The method of claim 1, wherein the detection result comprises at least one of the following:

a determination result of whether the format of the video input source signal is abnormal, a determination result of whether the image quality of the video input source signal is abnormal, or a determination result of whether the video input source signal has a large amount of motion.
The method according to claim 2, wherein the determination result that the format of the video input source signal is abnormal includes at least one of the following:

An abnormality of the synchronization information of the video input source signal, an abnormality of the aspect ratio of the video input source signal; and/or,

The determination of the image quality abnormality of the video input source signal includes:

The brightness of the video input source signal exceeds a first predetermined threshold range and/or the chrominance of the video input source signal exceeds a second predetermined threshold range.
The method of claim 2 further comprising:

Before the encoding process of the video input source signal according to the detection result, if the format of the video input source signal is abnormal, the abnormal video input source signal is adjusted to a normal video. Input source signal.
The method of claim 2 further comprising:

Adjusting the brightness and/or color of the video input source signal in a case where it is determined that the image quality of the video input source signal is abnormal before the encoding of the video input source signal according to the detection result degree.
The method according to claim 2, wherein said encoding processing said video input source signal according to said detection result comprises:

In a case where the video input source signal has a determination result of a large amount of motion, the video input source signal is encoded according to a motion complexity and/or a motion vector of the video input source signal.
The method according to any one of claims 1 to 6, wherein the video input source signal is modulo A/D converted and video data prior to encoding.
A video preprocessing apparatus includes:

a first acquiring module, configured to: acquire a video input source signal;

The second obtaining module is configured to: detect the video input source signal by using a field programmable gate array FPGA, and obtain a detection result;

The encoding module is configured to: perform encoding processing on the video input source signal according to the detection result.
The apparatus of claim 8, wherein the detection result comprises at least one of the following:

a determination result of whether the format of the video input source signal is abnormal, a determination result of whether the image quality of the video input source signal is abnormal, or a determination result of whether the video input source signal has a large amount of motion.
The apparatus according to claim 9, wherein the determination result that the format of the video input source signal is abnormal includes at least one of the following:

An abnormality of the synchronization information of the video input source signal, an abnormality of the aspect ratio of the video input source signal; and/or,

The determination of the image quality abnormality of the video input source signal includes:

The brightness of the video input source signal exceeds a first predetermined threshold range and/or the chrominance of the video input source signal exceeds a second predetermined threshold range.