KR101942371B1 - Apparatus and method for video frame scene change detection and encoding in mobile wireless environment - Google Patents

Abstract

The present invention provides a video encoding apparatus comprising: a first compressed data generation unit for compressing a previous video frame before a current video frame to generate first compressed data; A first encoding unit encoding the current video frame; A second compressed data generation unit for compressing the encoded current video frame to generate second compressed data; And a scene change detection unit for detecting a transition of a video frame by comparing the first compressed data and the second compressed data, and a scene change detection unit for detecting and encoding a video frame scene change in the mobile radio environment, ≪ / RTI >

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a video frame scene change detection and encoding apparatus in a mobile wireless environment,

The present invention relates to an apparatus and method for detecting and encoding a video frame scene transition in a mobile wireless environment, and more particularly, to an apparatus and method for detecting and encoding a video frame scene transition in a mobile wireless environment, And more particularly, to a video frame scene change detection and encoding apparatus and method in a mobile wireless environment.

Conventionally, advanced applications such as multimedia have been implemented by real-time encoding and transmitting low-resolution video contents in a PC-class wired environment with high processor specifications. However, as the application to video encoding / decoding has become more and more widespread, the ASIC of video codec has progressed and higher resolution applications have become possible.

This phenomenon proceeded not only in the PC-class wired environment but also in the mobile device environment providing the user's mobility. As a result, it is now possible to encode and play back high-resolution video content on devices such as smartphones and mobile pads (iPad, Samsung Tab).

However, the mobile device environment is based on mobility, so it is necessary to transmit compressed data for multimedia applications in a wireless environment. However, the wireless environment exhibits a packet transmission rate which is considerably lower than that in the wired environment, and the packet transmission rate is lowered as the number of packets increases compared with the bandwidth of the network to be provided.

Such a technique of detecting a scene change of a video and transmitting a video compressed stream generally proceeds to video encoding after scene change detection.

That is, raw data such as a camera and a screen capture are input, and a video preprocessing is performed to detect a scene change. The scene change is detected by mutual comparison of the input images after the video preprocessing. Based on the converted image information, video encoding is performed according to an implementation application, and a compressed bitstream is generated and transmitted. Finally, if the end command is lost during the video call, all program execution ends.

The scene change detection technique described above is performed in a stage before video encoding. The technique does not drop a video frame or lower a bit rate but predicts a flow of a video screen to provide a large bit rate to a scene change frame, Is transmitted.

 However, most of the multimedia applications in the mobile device environment are those in which video is mostly captured and stored locally, or images are reproduced and watched, and Korean Patent Application No. 10-2008-000075308 (filed on July 31, 2008) It should be considered that applications such as 'real-time scene change detection method for video coding data rate control, video call quality improvement method using video call system and video call system', transmission using wireless network are limited to video call. In this case, it is preferable not to transmit unnecessary redundant information in the video call, that is, a screen that does not change. In addition, video encoding in a mobile device environment must use a hardware accelerator, no matter how good the processor performance of a mobile device is. This has the problem that preprocessing the video and adding the scene change detection function in the mobile device environment deteriorates the performance against the video encoding.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a mobile wireless communication system capable of effectively using network bandwidth by eliminating non- Video scene change detection and encoding apparatus and method therefor.

Also, the present invention provides a video frame scene change detection and encoding apparatus and a method thereof in a mobile wireless environment, which can support packet drop according to priorities when network congestion is provided by effectively providing scene change information to a wireless router It has its purpose.

According to an aspect of the present invention, there is provided a video frame transition detection and encoding apparatus comprising: a first compressed data generation unit generating a first compressed data by compressing a previous video frame before a current video frame; A first encoding unit encoding the current video frame; A second compressed data generation unit for compressing the encoded current video frame to generate second compressed data; And a scene change detection unit for comparing the first compressed data and the second compressed data to detect a scene change of a video frame.

And the second compressed data generation unit compresses the encoded current video frame into a P frame type.

Wherein the scene change detection unit comprises: a bit calculation unit for calculating a bit amount from the first compressed data and the second compressed data, respectively; A comparison unit for comparing the sum of the calculated bit amounts with a predetermined reference bit amount; And a detection unit detecting that a scene change has occurred if the sum of the bit amounts is greater than or equal to the reference bit amount according to the comparison result.

And the comparator compares the sum of the calculated bit amounts with a preset reference bit amount based on Equation (1) below.

[Equation 1]

If i = 1,

| R _prev (i) - R _curr | > α * R _th

Else

(1 / (N-1))?? (I) * | R _prev (i) - R _curr | > R _th , i = 2 to N

Here, R _prev is the amount of bits generated from the first compressed data, R _curr is the amount of bits generated from the second compressed data,? Is the weighting factor of the image frame,? * R _th is the normal P frame Means that the amount of bits generated in?

Wherein the scene change is detected based on Equation (1) using the number of Intra modes of the first compressed data and the second compressed data to detect a scene change according to Equation (2).

[Equation 2]

I _th ≪ I (N)

Here, I (N) is the number of Intra Mode in the current frame, and I _th <I (N) means scene change detection.

The scene change is detected using the following equation (3) for detecting a scene change using the MV (motion vector) of the encoded second compressed data based on Equation (1).

[Equation 3]

D> D _th

Here, D is the standard deviation of the MV of the current frame, and MV (xi) ^ 2 + MV (yi) ^ 2, where D> D _th means scene change detection.

And a second encoding unit for re-encoding the current video frame in which the scene change is detected.

In this case, the second encoding unit may include: a setting unit for setting the current video frame detected as the scene change to an I frame type; A re-encoding unit for re-encoding a current video frame set to the I frame type; And a bit re-calculation unit for initializing a bit amount calculated from the second compressed data and restoring a bit amount from the re-encoded current video frame.

According to an aspect of the present invention, there is provided a method for detecting and encoding a video frame transition, the method comprising: generating a first compressed data by compressing a previous video frame before a current video frame; Encoding the current video frame, compressing the encoded current video frame to generate second compressed data, And comparing the first compressed data and the second compressed data to detect video frame transitions.

Wherein the step of detecting the transition of the video frame includes a step of comparing a sum of a bit amount calculated from the first compressed data and the second compressed data and a predetermined reference bit amount, If it is detected that the scene change has occurred.

And re-encoding the current video frame in which the scene change is detected, after detecting the video frame scene change.

And re-encoding the current video frame in which the scene change has been detected, re-encoding a bit amount from the re-encoded current video frame.

The video frame scene change detection and encoding apparatus and method according to the present invention having the above-described structure removes the non-useful information other than the video encoding scene change in the mobile environment and outputs a compressed bit stream By minimizing. The packet transmission rate can be improved by using the network bandwidth efficiently.

In addition, by providing scene change information to the wireless router as described above, it is possible to support packet drop according to the priority when the network is congested.

FIG. 1 is a view for explaining a technique of transmitting a video compressed stream by detecting a transition of a video frame according to an embodiment of the present invention,
2 is a diagram illustrating a configuration of a video frame scene change encoding apparatus according to an embodiment of the present invention,
3 is a diagram illustrating a detailed configuration of a scene change detection unit employed in a video frame scene change encoding apparatus according to an embodiment of the present invention,
4 is a diagram illustrating a detailed configuration of a second encoding unit employed in a video frame scene change encoding apparatus according to an embodiment of the present invention,
5 is a diagram illustrating a video frame scene transition encoding method according to an embodiment of the present invention,
6 is a diagram illustrating an example of re-encoding a current frame in which a scene change is detected in a video frame scene transition encoding method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to facilitate a person skilled in the art to easily carry out the technical idea of the present invention. . First, in adding reference numerals to the constituents of the drawings, it is to be noted that the same constituents are denoted by the same reference symbols as possible even if they are displayed on different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, a video frame scene change encoding apparatus and a method thereof according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

Figure 1 is a diagram illustrating a technique for detecting a video frame transition and transmitting a video compressed stream according to an embodiment of the present invention.

Referring to FIG. 1, a technique for transmitting a video compressed stream by detecting a transition of a video frame according to an exemplary embodiment of the present invention may implement a video call in a mobile terminal by detecting a scene change after video encoding. In more detail, the video source data is input from a camera or a screen capture application of the mobile terminal (S100), and the preprocessing is performed (S200). Based on the image information on which the pre-processing has been performed, video encoding is performed in accordance with the implementation application (S300). Then, the scene change is detected based on the encoded image information using the hardware accelerator provided by the mobile processor (S400).

To this end, the present invention provides a video frame scene change encoding apparatus 100 for detecting a scene change and a method thereof, which will be described in detail with reference to the accompanying drawings. Next, when a compressed bitstream is generated after the scene change is detected (S500), the compressed bitstream is transmitted. When the termination command during the video call (S600) falls, execution of all application programs is terminated.

FIG. 2 is a diagram illustrating a configuration of a video frame scene change encoding apparatus according to an embodiment of the present invention, and FIG. 3 is a block diagram illustrating a detailed configuration of a scene change detection unit employed in a video frame scene change encoding apparatus according to an embodiment of the present invention. And FIG. 4 is a diagram illustrating a detailed configuration of a second encoding unit employed in a video frame scene change encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 2, a video frame scene change encoding apparatus 100 according to an embodiment of the present invention includes a first compressed data generating unit 110 and a first encoding unit 120. A second compression data generation unit 130, a scene change detection unit 140, and a second encoding unit 150.

The first compressed data generation unit 110 compresses the previous video frame before the current video frame to generate the first compressed data.

The first encoding unit 120 encodes the current video frame.

The second compressed data generation unit 130 compresses the encoded current video frame to generate second compressed data. At this time, the second compressed data generation unit 130 compresses the encoded current video frame into a P frame type.

Here, the P frame type is one of the frame types of a frame collection from a GOP (Group of Pictures), that is, a key frame to a next key frame, and is called Previous or Predicted Frame. The P frame type is a frame composed of previously- And medium capacity. To describe the frame type of the GOP in more detail, there are three types of frame types of GOP, I, P and B. The characteristics of the I frame type and the B frame type are as follows. The I frame type is abbreviation of Infra Frame. The B frame type is an abbreviation of Bidirectional Frame. The B frame type is based on the information of I / P frames before and after. The B frame type having the minimum image quality and capacity is not applied in the present invention.

The scene change detection unit 140 compares the first compressed data with the second compressed data to detect a scene change of a video frame.

3, the scene change detection unit 140 may include a bit calculation unit 141, a comparison unit 143, and a sensing unit 145. [

The bit calculator 141 calculates the bit amount from the first compressed data and the second compressed data, respectively.

The comparing unit 143 compares the sum of the calculated bit amounts with the predetermined reference bit amount. The comparing unit 143 compares the sum of the calculated bit amounts based on the following equation (1) with the preset reference bit amount.

[Equation 1]

If i = 1,

| R _prev (i) - R _curr | > α * R _th

Else

(1 / (N-1))?? (I) * | R _prev (i) - R _curr | > R _th * i, i = 2 to N,

Here, R _prev is the amount of bits generated from the first compressed data, R _curr is the amount of bits generated from the second compressed data, and? * R _th is the amount of bits generated in the P frame, It is a formula that takes into account the belief. That is, | R _prev (i) - R _curr | > α * R _th means scene change detection. Also, δ denotes the weighting factor of the image frame. As δ (i) increases, the value of δ (i) decreases. Therefore, it places a lot of weight on recent frames.

In addition, in the scene change detection, the number of macroblocks encoded in the Intra mode can be known through the header data of the first compressed data and the second compressed data as well as the bit amount as described above, Can be used to detect a scene change. This is based on the following equation (2).

[Equation 2]

I _th < I (N)

Here, I (N) is the number of Intra Mode in the current frame, and I _th <I (N) means scene change detection.

Meanwhile, in the scene change detection, the scene change detection can be performed using the MV (motion vector) of the encoded second compressed data. If MV is taken and the absolute value of each MV is taken, the standard deviation is obtained. If it exceeds the predetermined value (D _th ), the re-encoding is performed as an I frame for the current video frame as in the method by the number of bits and Intra mode Which is based on the following equation (3).

[Equation 3]

D> D _th

Here, D is the standard deviation of the MV of the current frame. MV (xi) ^ 2 + MV (yi) ^ 2, where D> Dth denotes scene change detection.

If the sum of the bit amounts is greater than or equal to the reference bit amount, the detection unit 145 detects that a scene change has occurred.

The second encoding unit 150 re-encodes the current frame in which the scene change is detected.

For this, the second encoding unit 150 may include a setting unit 151, a re-encoding unit 153, and a bit re-calculation unit 155, as shown in FIG.

The setting unit 151 sets the current video frame in which the scene change is detected as the I frame type.

The re-encoding unit 153 re-encodes the current video frame set to the I frame type.

The bit re-calculation unit 155 initializes the bit amount calculated from the second compressed data, and recalculates the bit amount from the re-encoded current video frame.

FIG. 5 is a diagram illustrating a video frame scene transition encoding method according to an embodiment of the present invention, FIG. 6 is a diagram illustrating a video frame scene transition encoding method according to an embodiment of the present invention, Fig.

Referring to FIG. 5, a video frame scene change detection method (S400) according to the present invention uses the video frame scene change encoding apparatus 100 described above.

First, a previous video frame before a current video frame is compressed to generate first compressed data (S410)

Next, the current video frame is encoded (S420)

Next, the encoded current video frame is compressed to generate second compressed data (S430). The second compressed data generation unit 130 compresses the encoded current video frame into a P frame type.

Next, the first compression data and the second compressed data are compared to detect a video frame scene change (S440). This is because the scene change detection unit 140 detects the first compressed data and the bits calculated from the second compressed data When the sum of the positive and the predetermined reference bit amount is larger than the reference bit amount, the detection of the occurrence of the scene change is detected.

Finally, the current video frame in which the scene change is detected is re-encoded (S445). This is performed by the second encoding unit 150 to re-calculate the bit amount from the re-encoded current video frame.

This is an example in which a scene change is detected in the 7th P frame type after the start I frame type is first encoded as described with reference to FIG.

First, the start I frame type is encoded and the bit amount generated here is stored. The encoding is then performed for the P frame type, and compared to the bit amount with the immediately preceding I frame type. After comparison, the next P frame type is encoded without exceeding a certain value, and after that, it is compared with the bit amount of I, P ₁ which has been performed previously. In this case, the amount of bits generated in the I frame type is normally multiplied by α times the amount of bits generated in the P frame type, so that the threshold value is set in consideration of this. Up to P ₆ , the scene transition was not detected because it was smaller than the R _th value set during encoding. However, the P ₇ frame, that is, A is a case where scene change is detected because the generated bit amount is larger than the set value R _th . The P ₇ frame type is re-encoded as an I frame type, and the previously stored bit amount information is reset . That is, when a new I frame type is set, the same process as the start I frame type is performed again.

The result of calculating the reduction of the bit amount based on the example shown in FIG. 4 is as follows. R _I = 3A, R _P = Speaking of A, if not a transition has _{R WSC = R I + 13R P} = 16A, the case of transmitting only the I-frame type by using the scene change, the R _SC = 2R _I = 6A . That is, the bit rate reduction effect of 16/6 = 2.7 times is shown.

Meanwhile, the present invention has another effect of this technique in addition to the bit amount reduction effect as described above. That is, in the past, if a compressed sequence of a video sequence is compressed and transmitted, and a compressed packet of a frame (P ₇ ) before a significant frame in the middle of the sequence (here, P ₇ transition frame) is lost, do. However, in the case of C, which includes information on the scene change, since the image is generated independently by the screen switching, the deterioration of the image quality is not affected. In this case, if the amount of generated packets exceeds the transmission capability of the router in the case of going through a router to which a packet is to be transmitted, a sequence having information on scene change, that is, C, By removing the data, it is possible to transmit without deterioration of image quality or loss of important data.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art without departing from the scope of the appended claims. As will be understood by those skilled in the art.

100: Video frame scene transition detection and encoding device
110: first compressed data generation unit
120: a first encoding unit
130: second compressed data generation unit
140: scene change detection unit
150: second encoding section

Claims (12)

A first compressed data generation unit for compressing a previous video frame before a current video frame to generate first compressed data;
A first encoding unit encoding the current video frame;
A second compressed data generation unit for compressing the encoded current video frame to generate second compressed data; And
A scene change detection unit for detecting a scene change of a video frame by comparing the first compressed data and the second compressed data; And
A second encoding unit for re-encoding the current video frame in which the scene change is detected;
Lt; / RTI &gt;
The second encoding unit
And a bit re-calculation unit for initializing a bit amount calculated from the second compressed data and restoring a bit amount from the re-encoded current video frame. Device. The method according to claim 1,
Wherein the second compressed data generation unit comprises:
And compresses the encoded current video frame into a P frame type. The method according to claim 1,
Wherein the scene change detection unit comprises:
A bit calculating unit for calculating a bit amount from the first compressed data and the second compressed data, respectively;
A comparison unit for comparing the sum of the calculated bit amounts with a predetermined reference bit amount; And
A detecting unit detecting that a scene change occurs if the sum of the bit amounts is greater than or equal to the reference bit amount according to the comparison result;
Wherein the video frame transition detection and encoding apparatus comprises: The method of claim 3,
Wherein the comparison unit compares the difference between the calculated bit amount and the preset reference bit amount based on Equation (1) below.
[Equation 1]
If i = 1,
| R _prev (i) - R _curr | > α * R _th
Else
(1 / (N-1))?? (I) * | R _prev (i) - R _curr | > R _th , i = 2 to N
Here, R _prev is the amount of bits generated from the first compressed data, R _curr is the amount of bits generated from the second compressed data,? Is the weighting factor of the image frame,? * R _th is the normal P frame Is the number of previous frames, N is the total number of previous video frames, i is the more recent frame, i is the weighting factor of the video frame, and δ decreases as i increases in δ , We mean that we will put a lot of weight on the recent frame, and Σ means that δ (i) * | Rprev (i) - Rcurr | Which means that it is performing the summation to the user. 5. The method of claim 4,
Detecting a scene change based on Equation 1 and detecting a scene change using the number of Intra modes of the first compressed data and the second compressed data, Frame transition detection and encoding device.
[Equation 2]
I _th < I (N)
Here, I (N) is the number of Intra Mode in the current frame, I _th is the number of predetermined Intra modes, and I _th <I (N) means scene change detection. 6. The method of claim 5,
Wherein the scene change is detected through the following equation (3) for detecting a scene change using an MV (Motion Vector) of the encoded second compressed data based on Equation (1) A scene change detection and encoding device.
[Equation 3]
D> D _th
Here, D is a standard deviation of the MV of the current frame, MV (xi) ^ 2 + MV (yi) ^ 2, and D _th is MV Value, and D > _Dth means scene change detection. delete The method according to claim 1,
Wherein the second encoding unit comprises:
A setting unit configured to set a current video frame in which the scene change is detected to an I frame type;
A re-encoding unit for re-encoding a current video frame set to the I frame type;
Wherein the video frame transition detection and encoding apparatus further comprises: Compressing a previous video frame before a current video frame to generate first compressed data;
Encoding the current video frame;
Compressing the encoded current video frame to generate second compressed data;
Comparing the first compressed data with the second compressed data to detect a video frame transition; And
Re-encoding the current video frame in which the scene change has been detected;
Lt; / RTI &gt;
In re-encoding the current video frame in which the scene change has been detected,
And re-encoding a bit amount from the re-encoded current video frame. 10. The method of claim 9,
Wherein detecting the video frame transition comprises:
When a sum of the bits calculated from the first compressed data and the second compressed data is compared with a predetermined reference bit amount and the sum of the bit amounts is greater than or equal to the reference bit amount, To a video frame transition detection and encoding method in a mobile wireless environment.

delete delete

Images