JP4878052B2 - Video code amount control method, video encoding device, video code amount control program, and recording medium therefor - Google Patents

Description

  The present invention provides video code amount control for controlling the video code amount in accordance with the generated code amount of audio so that the sum of the bit rate of the audio stream and the bit rate of the video stream falls within a predetermined bit rate range. The present invention relates to a method, a video encoding device, a video code amount control program, and a recording medium thereof.

  In general, in a transmission system in which a video stream and other streams such as audio and data are multiplexed and transmitted, the video is subjected to bit rate control independently of other streams (Non-Patent Documents 1 and 2). reference).

  In general, in video / audio coding, the amount of code increases for an input signal that is complex and changes a lot, and the amount of code decreases for an input signal that is monotonous and changes little.

Since the lossy coded stream is transmitted at a constant bit rate (CBR), the generated code amount is controlled to fall within a predetermined range by changing the degree of signal degradation. On the other hand, in a lossless encoded stream, since the change in the amount of generated code cannot be controlled, it is generally transmitted at a variable bit rate (VBR).
ISO / IEC 13818-2 Annex C, ITU-T Recomendation H.264 Annex C The Institute of Image Information and Television Engineers, “Multi-media selection MPEG”, The Institute of Image Information and Television Engineers, published by Ohm Corporation, 1996.4.20

  When a lossless audio encoded stream such as MPEG-4 ALS and a video encoded stream (hereinafter, the encoded stream is also simply referred to as “stream”) are multiplexed, the fluctuation of the audio bit rate is larger than the video bit rate. Often too big to ignore.

  Therefore, in the multiplexed stream of video and audio, out of the fixed bit rate of audio, if the surplus code amount with small generated code amount of audio encoded data can be used for video encoding, The rate is not wasted, and it is thought that it contributes to the improvement of image quality.

  However, if the video bit rate is changed in accordance with the audio bit rate in order to effectively use the fluctuation of the generated code amount of the audio encoded data for video encoding, Although the image quality is improved, the image quality may be unnecessarily sharp in scenes where the amount of generated code of the speech encoded data is small, and the variation in image quality may be conspicuous depending on the variation in the speech encoded data. New problems arise.

  The present invention solves the above-described problem, effectively uses the surplus bit rate in the transmission of audio encoded data for transmission of video encoded data, and makes the fluctuation of image quality as a whole inconspicuous. The purpose is to improve the quality of the converted video.

  In order to solve the above-described problem, the present invention provides the following when multiplexing a lossy audio encoded stream, a data stream, etc., which cannot control the encoding bit rate, and a lossy video encoded stream into CBR as follows: The amount of video code is controlled according to the surplus bit rate when the amount of generated audio code is small.

  (1) The audio code amount is pre-read, and if it is found that there is a part where the video transmission possible bit rate is lower than the current video coding bit rate, the video coding bit rate is lowered to the transmission possible rate.

  For example, when video rate control is performed in units of GOP (Group Of Pictures), a value obtained by converting a GOP generated code amount into a bit rate is a video encoding bit rate. Also, a value obtained by subtracting the bit rate converted value of the audio code amount reproduced during the same GOP period from the TS (Transport Stream) bit rate or the transmission bit rate is the video transmission possible bit rate.

  (2) When the video encoding bit rate is lowered, it is lowered at the time of scene switching as much as possible. If that is not possible, gradually lower it. Conventional methods can be used for scene change detection.

  (3) When the video encoding bit rate is increased, the video transmission bit rate is increased to the lower limit value of the video transmission possible bit rate within a certain past period at the time of scene switching. If this period is long, the risk of image quality fluctuation is reduced, but the bit rate range to be increased is small.

  (4) As a means of prefetching the amount of audio code, a method of encoding audio for several seconds ahead of video, a method of encoding audio in advance and recording the amount of code, analyzing audio in advance, For example, a method of recording an estimated value of the amount can be used.

  (5) By pre-reading for 1 GOP (for example, about 0.5 seconds), the usable bandwidth up to the end of the GOP is normally determined at the start of GOP encoding, so that the rate control is stabilized. Also, if the pre-reading period of the voice code amount is set to several seconds or more, the probability that the bit rate can be changed at the time of a scene change is increased, and fluctuations in image quality during the scene can be suppressed.

  Based on the above points, the present invention controls the video code amount as follows. First, from the audio encoding result encoded prior to video encoding, from the transmission bit rate (TS bit rate) to the same period in the video encoding control unit (GOP or frame) period before video encoding. A surplus bit rate corresponding to a value obtained by subtracting a value obtained by converting a reproduced audio code amount into a bit rate and a predetermined reference video bit rate is calculated. A value obtained by adding the surplus bit rate to the reference video bit rate is a video transmission possible bit rate.

  Next, an additional video bit rate to be added to the reference video bit rate is determined according to a minimum surplus bit rate in a predetermined period in the past or in the future than the period of the current video encoding target video encoding control unit. Then, the target code amount in the period of the video encoding control unit to be encoded is determined based on the value obtained by adding the additional video bit rate to the reference video bit rate, and the video is determined according to the determined target code amount. The video signal in the encoding control unit is encoded.

  In the above invention, when determining the additional video bit rate, the lowest surplus bit rate in a predetermined period is lower than the additional video bit rate in the video encoding control unit immediately before the video encoding control unit to be encoded. In addition, it is also preferable to determine the additional video bit rate so that the additional video bit rate monotonously decreases by the period of the video coding control unit that provides the lowest surplus bit rate. This is because image quality fluctuations are less noticeable by gradually reducing the additional video bit rate rather than rapidly.

  Also, if there is a scene change, the image quality fluctuation before and after the scene change is less noticeable. Therefore, it is also preferable to increase or decrease the additional video bit rate at the time of scene switching as much as possible.

  Therefore, the present invention can be further implemented as follows. First, when a scene change is detected prior to video encoding and an additional video bit rate is determined, if a scene change is included in the period of the video encoding control unit to be encoded, the past a (A is a predetermined integer greater than or equal to 1) video encoding control unit period minimum surplus bit rate, preceding minimum audio bit encoding period surplus bit rate, or scene end after scene change If the surplus bit rate up to the point is known, the additional video bit rate after the scene change is determined according to the minimum surplus bit rate in the period of the video encoding control unit up to the scene end point, and the video code The minimum surplus in the previously encoded audio encoding period when no scene change is included in the period of the video encoding control unit to be encoded If the surplus bit rate up to the scene end point of the current scene is known, the additional video bit rate is determined according to the minimum surplus bit rate in the period of the video encoding control unit up to the scene end point. To do.

  Furthermore, the present invention provides a video coding control unit for video coding in order to effectively use it for video coding when a surplus bit rate increases during a long scene period. The minimum surplus bit rate in a period of a number of video coding control units in the past (a is a predetermined integer greater than or equal to 1) than the period is a certain value or a certain ratio or more than the additional video bit rate in the immediately preceding video coding control unit. In such a case, it is also preferable to set the additional video bit rate to a value larger than the additional video bit rate in the immediately preceding video coding control unit by a certain value or a certain ratio.

  According to the present invention, when the video code amount is controlled so that the encoded stream obtained by multiplexing the video encoded data and the audio encoded data falls within the predetermined transmission bit rate, the audio surplus is controlled. By using the bit rate for video transmission, the image quality can be improved as a whole, and the variation in image quality can be made inconspicuous even when the variation in the amount of audio code is large.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration example of an apparatus according to an embodiment of the present invention. The video encoding device 10 inputs a video signal from among the video / audio superimposed signals, encodes the video signal, and outputs a video stream. The audio encoding device 20 inputs an audio signal among the video / audio superimposed signals, encodes the audio signal, and outputs an audio stream. The multiplexing unit 30 multiplexes and outputs the audio stream and the video stream so as to have a predetermined bit rate.

  In the present embodiment, the audio encoding device 20 is the same as the prior art except that it has a function of transmitting the audio generation code amount to the video encoding device 10 for each frame or sequentially. Is omitted.

  The video encoding device 10 includes a video input unit 11 for inputting a video signal, a video input buffer 12 for storing the input video signal, a video GOP target code amount determination unit 13 for determining a target code amount of a video for each GOP, and a video Video frame target code amount determination unit 14 that determines a target code amount for each frame, video frame encoding processing unit 15 that encodes a video signal according to the frame target code amount, video stream output unit 16 that outputs a video stream, and video stream A video stream output buffer 17 for temporarily storing the video stream.

  The scene change detection unit 18 detects the presence or absence of a scene change for scene change detection target video data prior to encoding of the input video signal.

  The video GOP target code amount determination unit 13 includes a surplus bit rate calculation unit 131 that calculates a surplus bit rate that can be used for video encoding from the audio generation code amount encoded by the audio encoding device 20, and the calculated surplus. An additional video bit rate calculating unit 132 that calculates an additional video bit rate that is actually used for video encoding among the bit rates; a GOP target code amount calculating unit 133 that calculates a GOP target code amount using the additional video bit rate; Is provided.

  2 and 3 are flowcharts of the video encoding process according to the embodiment of the present invention. In this embodiment, video and audio are encoded in parallel. However, audio coding and video scene change detection are performed in advance of video coding by 1 GOP or more ((m−n + 1) GOP in the example of FIG. 2). Further, in this embodiment, the reference video bit rate refers to a video bit rate that can be secured when the generated code amount of an audio frame is always the maximum. That is, the reference video bit rate is a predetermined bit rate that is at least guaranteed for transmission of the video stream. The nth GOP is represented as GOP (n).

First, the video GOP target code amount determination unit 13 receives the information of the sound generation code amount from the audio encoding device 20, and at the start of encoding of GOP (n), the GOP (m) period (where m> n). From the preceding speech coding result, the surplus bit rate R _r (m) in the GOP (m) period is obtained by the following equation and stored (step S101).

R _r (m) = Σx _{= j} ^{j + M−1} {S _amax −S _a (x)} / (N / P)
However, a speech frame in the GOP (m) period is a frame j to j + M−1 (M is the number of speech frames in the GOP (m) period), S _a (x) is the generated code amount of the speech frame x, and S _amax is each This is the maximum generated code amount of a voice frame. N is the number of video frames in the GOP, and P is the video frame rate.

Next, the minimum surplus bit rate Ra of _a past G (a is a predetermined constant) GOP is updated from GOP (n) (step S102). min () is a function that returns the minimum value among the elements of the argument.

R _a = min (R _r (n−a + 1), R _r (n−a + 2),..., R _r (n))
Subsequently, according to the detection result by the scene change detection unit 18, it is determined whether or not the scene start point is included in GOP (n) (step S103). When the scene start point is included in GOP (n), step S112 and subsequent steps in FIG. 3 are executed.

When the scene start point is not included in GOP (n), the process proceeds to step S104, and it is determined whether or not the frame in which the scene being encoded ends and its GOP (s) have been determined (step S104). If it is already determined, the minimum surplus bit rate R _r (t) in the current scene and its period t are obtained (step S105). That is, the minimum surplus bit rate R _r (t) is obtained according to the following equation.

R _r (t) = min (R _r (n), R _r (n + 1),..., R _r (s))
On the other hand, if it has not been determined, the minimum surplus bit rate R _r (t) within the preceding encoding period and the period t are obtained (step S106). In this case, the minimum surplus bit rate R _r (t) is as follows.

R _r (t) = min (R _r (n), R _r (n + 1),..., R _r (m))
Next, it is determined whether or not the minimum surplus bit rate R _r (t) is lower than the additional video bit rate R _d (n−1) of the immediately preceding GOP (step S107). If lower, the additional video bit rate R _d (n) is set so as to monotonously decrease by the period t when the additional video bit rate is reached (step S108). For example, set as follows.

R _d (n) = R _d (n−1) − {R _d (n−1) −R _d (t)} / (t−n)
On the other hand, if it is not lower, the same additional video bit rate as the previous GOP is set (step S109). That is, R _d (n) = R _d (n−1).

Next, the GOP target code amount G (n) is calculated from the reference video bit rate R and the additional video bit rate R _d (n) (step S110). The GOP target code amount G (n) is obtained by the following equation.

G (n) = N · (R + R _d (n)) / P + carry-over code amount of previous GOP If GOP target code amount G (n) is determined, the video frame target code amount determination unit 14 The amount G (n) is allocated to each frame in GOP (n) to determine the target code amount of each frame, and the video frame encoding processing unit 15 determines GOP (n according to the target code amount of each frame. ) Are encoded (step S111). As a method for calculating the target code amount of each frame in GOP (n) from the GOP target code amount G (n), for example, there is a method disclosed in Reference Document 1 below.
[Reference 1] International Organization for Standardization, Test Model Editing Committee, 1993, Test Model 5, April, ISO-IEC / JTC1 / SC29 / WG11 / N0400
Thereafter, the process returns to step S101, and the encoding process is continued for the next GOP (n + 1) in the same manner.

If it is determined in step S103 that the scene start point is included in GOP (n), the process proceeds to step S112 in FIG. 3, and the frame where the scene started in GOP (n) ends and its GOP (s) are found. It is determined whether or not it has been completed (step S112). If it has already been determined, an additional video bit rate R _d ^* (n) after the scene change in the GOP is set to the smaller one of the minimum surplus bit rate in the scene period and the upper limit value TH _R of the video encoding bit rate. (Step S113).

R _d ^* (n) = min (TH _R , min (R _r (n),..., R _r (s)))
On the other hand, if not found already has a minimum excess bit rate R _r in the speech prior coding period, either smaller value of the minimum surplus bit rate R _a past a number of GOP, after GOP within the scene change The additional video bit rate R _d ^* (n) is set (step S114).

R _d ^* (n) = min (R _a , min (R _r (n),..., R _r (m)))
Next, it is determined whether the surplus bit rate R _r (n) of GOP (n) is lower than the additional video bit rate R _d (n−1) of the immediately preceding GOP (step S115). If it is lower, the surplus bit rate R _r (n) of GOP (n) is set to the additional video bit rate R _d ′ (n) before the scene change in GOP (step S116). That is, R _d ′ (n) = R _r (t).

On the other hand, if not lower, the same value as the additional video bit rate of the previous GOP is set as the additional video bit rate R _d ′ (n) before the scene change in the GOP (step S117). That is, R _d ′ (n) = R _d (n−1).

Next, from the reference video bit rate R, the additional video bit rate R _d ′ (n) before the scene change in the GOP, and the carry-over code amount of the previous GOP, the GOP target code amount G ′ (n) before the scene change is Calculation is performed according to the following equation (step S118).

G ′ (n) = (R + R _d ′ (n)) × α ′ + carry-over code amount of previous GOP where α ′ is an allocation determined by the number of frames, scene type, and complexity index before scene change in GOP Is the ratio.

  The video frame target code amount determination unit 14 determines the target code amount of each frame before the scene change of GOP (n) according to the GOP target code amount G ′ (n), and the video frame encoding processing unit 15 Each frame before the change is encoded (step S119).

Subsequently, the GOP target code amount G ^* (n) after the scene change is calculated from the reference video bit rate R, the additional video bit rate R _d ^* (n) after the GOP scene change, and the carry-over code amount from the first half of the GOP. Calculation is performed according to the equation (step S120).

G ^* (n) = (R + _Rd ^* (n)) × α ^* + carry code amount from the first half of the GOP where α ^* is a distribution ratio determined by the number of frames after a scene change in GOP, picture type, and complexity index It is.

The video frame target code amount determination unit 14 determines the target code amount of each frame after the scene change of GOP (n) according to the GOP target code amount G ^* (n), and the video frame encoding processing unit 15 Each frame after the change is encoded (step S121). Thereafter, the process returns to step S101, and the encoding process is continued for the next GOP (n + 1) in the same manner.

  FIG. 4 shows an example of bit rate allocation when the scene change start point is not included in GOP (n) in the embodiment of the present invention.

  In FIG. 4, R is a reference video bit rate, which is a video bit rate that can be secured when the generated code amount of the audio frame is always maximum. In other words, R is a bit rate that is at least guaranteed as the video bit rate.

Also, if the bit rate converted value of the audio code amount reproduced during the GOP (n) period is subtracted from the TS bit rate, the surplus bit rate R _r (n) can be transmitted. This is a value obtained by subtracting the reference video bit rate R from the bit rate.

The minimum surplus bit rate R _r (t) is the smallest value among the surplus bit rates in units of GOPs within a period in which audio is encoded prior to video encoding.

In principle, when trying to use the audio surplus code amount for video coding, video transmission is possible by adding the target code amount of GOP (n) to the reference video bit rate R by the surplus bit rate R _r (n). It can be determined according to the bit rate. However, in this case, if the amount of generated code of speech is large, the image quality may be noticeable.

Therefore, in the present invention, for example, as shown in FIG. 4, not all of the surplus bit rate R _r (n) is allocated to video coding, but a part thereof is defined as the additional video bit rate R _d (n), The additional video bit rate is monotonously decreased and gradually decreased by the minimum surplus bit rate period so that the additional video bit rate R _d (n) does not vary greatly. As a result, it is possible to avoid an extremely large variation in image quality.

  FIG. 5 shows an example of bit rate assignment when the scene change start point is included in GOP (n) and the end point of the scene being encoded is known in the embodiment of the present invention.

If there is a scene change in the video, the change in image quality is less noticeable before and after the scene change. Therefore, when a scene change is detected by prefetching the video signal, the additional video bit rate R _d ^* (n) is not gradually decreased monotonously, but, for example, as shown in FIG. 5, the additional video bit rate R _d ^* (N) may be determined to be the minimum surplus bit rate R _r (t) in the period up to GOP (m) including the end point of the scene. Even in this way, it is possible to make the change in image quality due to the change in the audio bit rate inconspicuous.

  FIG. 6 is a flowchart of video encoding processing according to another embodiment of the present invention. In the case of the present embodiment, even if the surplus bit rate is continuously sufficient even in the middle of the scene, the additional video bit rate is not monotonously decreased, but rather than the additional video bit rate of the immediately preceding GOP. By increasing only the effective bit rate, the effective use of surplus bit rate is promoted.

  In this embodiment, in the flowcharts shown in FIGS. 2 and 3, the step S110 is replaced with steps S1101 to S1103 shown in FIG. 6, and the other portions are the same as in the previous embodiment. Only the processing of the different parts will be described.

After performing steps S101 to S109 of the previous embodiment, is the minimum surplus bit rate R _{a of} the past a GOPs exceeding the additional video bit rate R _d (n−1) of the previous GOP by a certain value R _add or more? It is determined whether or not (step 1101). If the value exceeds or exceeds a certain value R _add , the additional video bit rate R _d (n) is increased by a certain value R _add from the immediately preceding GOP (step S1102). That is, R _d (n) = R _d (n−1) + R _add .

On the other hand, if the value does not exceed a certain value R _add or more, the additional video bit rate R (n) is made the same as R _d (n−1) of the immediately preceding GOP (step S1103). That is, R _d (n) = R _d (n−1). Thereafter, the same processing as steps S111 to S121 in the previous embodiment is performed.

In this example, past a number of minimum surplus bit rate R _a of GOP additional immediately preceding GOP video bit rate R _d (n-1) a constant value R _{the add} more, depending on whether above, additional image bit rate R _d ( Whether or not n) is to be increased is determined, but it may be determined whether it is to be increased by determining whether or not it is higher than a certain value rather than a certain value.

  The above video encoding processing can be realized by a computer and a software program, and the program can be recorded on a computer-readable recording medium or provided through a network.

It is a figure which shows the structural example of the apparatus which concerns on the Example of this invention. It is a flowchart of the video encoding process of the Example of this invention. It is a flowchart of the video encoding process of the Example of this invention. It is a figure which shows the example of allocation of a bit rate when a scene change start point is not included in GOP (n). It is a figure which shows the example of allocation of the bit rate when the scene change start point is included in GOP (n) and the end point of the scene being encoded is known. It is a flowchart of the video encoding process of the other Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video coding apparatus 11 Video input part 12 Video input buffer 13 Video GOP target code amount determination part 131 Surplus bit rate calculation part 132 Additional video bit rate calculation part 133 GOP target code amount calculation part 14 Video frame target code amount determination part 15 Video frame encoding processing unit 16 Video stream output unit 17 Video stream output buffer 18 Scene change detection unit 20 Audio encoding device 30 Multiplexing unit

Claims (7)

A video code amount control method for controlling a video code amount so that an encoded stream obtained by multiplexing video encoded data and audio encoded data falls within a predetermined transmission bit rate range,
Prior to encoding the video in the period of the video encoding control unit, from the result of encoding the audio reproduced during the synchronization in advance, a value obtained by converting the audio generation code amount to a bit rate is obtained, Calculating a value obtained by subtracting the converted value and a predetermined reference video bit rate from the transmission bit rate as a surplus bit rate during synchronization for each period of the video encoding control unit ;
An additional video bit rate to be added to the reference video bit rate is determined according to the minimum surplus bit rate in a plurality of consecutive video coding control unit periods including a current video coding control unit period. The process of
Determining a target code amount in a period of a video encoding control unit to be encoded based on a value obtained by adding the additional video bit rate to the reference video bit rate;
A video code amount control method comprising: encoding a video signal in the video encoding control unit according to the determined target code amount. In the video code amount control method according to claim 1,
In the process of determining the additional video bit rate, a minimum surplus bit rate in a period of the plurality of continuous video coding control units is a future video coding control unit than the video coding control unit to be coded. When the surplus bit rate is less than the additional video bit rate in the immediately preceding video coding control unit, the additional video bit rate is less than the video coding control unit period that is the lowest surplus bit rate. A video code amount control method characterized by determining an additional video bit rate so as to monotonously decrease. In the video code amount control method according to claim 1 or 2,
It has a process of detecting scene changes prior to video coding.
In the process of determining the additional video bit rate, when a scene change is included in a period of the video encoding control unit to be encoded, the past a number (a is a predetermined integer of 1 or more) of video encoding control. The lowest surplus bit rate in the unit period, or the lowest surplus bit rate in the previously encoded speech encoding period, or the surplus bit rate until the scene end point after the scene change is known. In this case, the additional video bit rate after the scene change is determined according to the minimum surplus bit rate in the period of the video encoding control unit up to the scene end point, and the video encoding control unit of the video encoding target to be encoded is determined. If the period does not include a scene change, the minimum surplus bit rate in the previously encoded speech encoding period, If the surplus bit rate up to the scene end point of the current scene is known, the additional video bit rate is determined according to the minimum surplus bit rate in the period of the video encoding control unit up to the scene end point. A video code amount control method. In the video code amount control method according to claim 1, claim 2 or claim 3,
Video encoding control with the lowest surplus bit rate in the period of the past a number of video encoding control units (a is a predetermined integer equal to or greater than 1) than the period of the video encoding control unit to be encoded. When the additional video bit rate in the unit is a certain value or a certain ratio or more, the additional video bit rate is set to a value that is a constant value or a constant rate larger than the additional video bit rate in the previous video coding control unit. A video code amount control method. A video encoding device that controls the amount of video code and encodes a video signal so that an encoded stream obtained by multiplexing video encoded data and audio encoded data falls within a predetermined bit rate range. And
Prior to encoding the video in the period of the video encoding control unit, from the result of encoding the audio reproduced during the synchronization in advance, a value obtained by converting the audio generation code amount to a bit rate is obtained, Means for calculating a value obtained by subtracting the converted value and a predetermined reference video bit rate from the transmission bit rate as a surplus bit rate during synchronization for each period of the video encoding control unit ;
An additional video bit rate to be added to the reference video bit rate is determined according to the minimum surplus bit rate in a plurality of consecutive video coding control unit periods including a current video coding control unit period. Means to do,
Means for determining a target code amount in a period of the video encoding control unit to be encoded based on a value obtained by adding the additional video bit rate to the reference video bit rate;
Means for encoding a video signal in the video encoding control unit according to the determined target code amount.   A video code amount control program for causing a computer to execute the video code amount control method according to any one of claims 1 to 4.   A computer-readable recording medium having recorded thereon a video code amount control program for causing a computer to execute the video code amount control method according to any one of claims 1 to 4.

Images