JP2010124486A - Image control apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve encoding efficiency and image quality of an image signal. <P>SOLUTION: On the basis of an inter-frame differential average value obtained by averaging an inter-frame differential for each block or for each pixel between images of the same image type, a scene change detecting means 21 detects occurrence of a scene change. In the case where the occurrence of the scene change is detected, an image insertion control means 22 inserts an I image into a stream. Furthermore, the scene change detecting means 21 determines a differential between a block average value obtained by blocking an image and averaging pixel data for each block and the inter-frame differential average value and in the case where blocks each having the differential greater than a fixed value are existent more than a fixed number, the scene change detecting means regards such a case as the occurrence of a scene change. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image control apparatus, and more particularly to an image control apparatus that performs encoding control of an image signal.

  MPEG that is internationally standardized by ISO / IEC and H.264 that is internationally standardized by ITU-T. In image coding schemes such as H.262, bi-predictive coded images (B pictures) are introduced in addition to intra-frame predicted images (I pictures) and inter-frame forward predicted images (P pictures). .

  Data in each picture is divided into small blocks and processed in units of blocks. Bidirectional prediction is prediction using past and future image frames, and I and P pictures are used for prediction.

  Therefore, as a result, in the encoded data stream, I picture is the head, and some B pictures are inserted between I picture and P picture or between P pictures (IBBPBBPB...). A set of pictures starting from the I picture is called a GOP (Group of Pictures).

  However, in the conventional GOP as described above, a B picture repetition number of 2 is generally used, but this is an appropriate value for an average picture, and other than 2 may be suitable depending on the picture. is there. For this reason, there has been a problem that optimum encoding has not been performed conventionally.

  For example, the closer the input image is to a still image, the greater the appropriate value of the B picture. Therefore, the coding efficiency increases when the motion of the input image is detected and the number of repetitions of the B picture is changed depending on the degree of motion.

  On the other hand, when a scene change occurs, it is often the case that image quality is better when intra-frame coding is performed. Conventionally, the timing of intra-frame coding is a fixed timing regardless of the scene change. It was done. As described above, conventionally, intra-frame encoding according to the state of the input image has not been appropriately performed, and there has been a problem that encoding efficiency is poor.

  The present invention has been made in view of these points, and an object of the present invention is to provide an image control apparatus that improves the encoding efficiency and image quality of an image signal.

In order to solve the above problems, an image control apparatus is provided. This image control device includes a scene change detection means for detecting the occurrence of a scene change based on an average value of inter-frame differences obtained by averaging the inter-frame differences for each block or pixel between pictures of the same picture type . And a picture insertion control means for inserting an I picture in the stream when the occurrence of the scene change is detected.

Here, the scene change detection means obtains a difference between a block average value obtained by blocking a picture and averaging pixel data for each block and an inter-frame difference average value, and blocks whose difference is greater than a certain value are constant. If there are more than the number, it is considered that a scene change has occurred .

Intraframe encoding can be performed appropriately according to the state of the input image, and encoding efficiency can be improved.

It is a principle diagram of an image control apparatus. It is a figure which shows the concept of the adaptive control of the B picture repetition number. It is a principle diagram of an image control apparatus. It is a figure which shows the structure of an image control apparatus. It is a figure which shows the structure of an encoding control part. It is a figure which shows the structure of a B value control part. It is a figure which shows the structure of a memory. It is a figure which shows the matrix table which a matrix process part manages. It is a figure which shows the structure of a scene change detection part. It is a figure which shows the structure of a scene change detection part. It is a figure which shows the structure of a scene change detection part. It is a figure which shows the structure of a scene change detection part. It is a figure which shows the structure of a scene change detection part. It is a state transition diagram on the writing side. It is a state transition diagram on the writing side. It is a state transition diagram on the reading side. It is a state transition diagram on the reading side. FIG. 6 is a sequence diagram of writing / reading when B increases. FIG. 6 is a sequence diagram of writing / reading when B decreases. It is a write / read sequence diagram at the time of a scene change. It is a write / read sequence diagram at the time of a scene change. It is a write / read sequence diagram at the time of a scene change.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a principle diagram of an image control apparatus. The image control apparatus 10 performs encoding control of an image signal.
The comparison processing means 11 performs first to third comparison processes. In the first comparison process, a motion vector (forward prediction vector with respect to the previous frame) is compared with a threshold value. The second comparison process compares the motion compensation prediction error with a threshold value. In the third comparison process, the subtraction value (= interframe difference−motion compensation prediction error) is compared with a threshold value. Then, information of these comparison results is output to the repetition number control means 12.

  The repetition number control means 12 adaptively controls the repetition number of the B picture in the GOP based on the information of the comparison result in the comparison processing means 11. In the figure, the number of repetitions = 3. In the following description, I picture, B picture, and P picture are abbreviated as I, B, and P, respectively.

Next, adaptive control of the B repetition number will be described. FIG. 2 is a diagram showing the concept of adaptive control of the B picture repetition number. The B repetition rate control direction F is determined by the sum of the B repetition rate control direction F (v) determined by the relationship diagram Z1 and the B repetition rate control direction F (e) determined by the inequality Z2.

  First, regarding the relationship diagram Z1, the relationship diagram Z1 has a motion vector on the horizontal axis and a motion compensation prediction error on the vertical axis. In addition, F (v) determined from the motion vector and the motion compensation prediction error increases in the B repetition rate control direction toward the upper left, and decreases in the B repetition rate control direction toward the lower right.

Here, the smaller the value of the motion vector, the closer the encoding target image is to a still image, so F (v) is controlled in a direction to increase the number of repetitions of B.
Further, when the motion compensation prediction error is small, the encoding target image is close to a still image or the motion vector detection is correctly performed, so that F (v) is controlled so as to increase the number of repetitions of B. Is done.

  In this case, when the number of repetitions of B is increased, the time difference between the reference picture and the current picture is increased. For this reason, the motion vector detection tends to deviate, and the motion compensation prediction error tends to increase. That is, if the number of repetitions of B is increased, the ratio of B in the GOP increases and the coding efficiency increases, but the effect of motion compensation tends to decrease. Therefore, the positive effect due to the increase in B and the negative due to the deviation of the vector. Considering a trade-off with the effect, F (v) is set to a well-balanced value.

  Next, for the inequality Z2, the subtraction value obtained by subtracting the motion compensation prediction error from the interframe difference is compared with the threshold value. If the subtraction value is larger than the threshold value, the B repetition rate control direction F (e) increases. If the subtraction value is smaller than the threshold value in the direction of B, the B repetition rate control direction F (e) is a decreasing direction.

  Here, when comparing the motion compensation prediction error and the inter-frame difference, if the former value is not smaller than the latter value, motion detection does not function well, so the number of repetitions of B Is controlled in a decreasing direction. In the opposite case, control is performed so that the number of repetitions of B is increased.

Next, an image control apparatus that inserts I when a scene change occurs will be described. FIG. 3 is a principle diagram of the image control apparatus. The image control device 20 performs encoding control of the image signal.
The scene change detection means 21 detects the occurrence of a scene change based on the average value of interframe differences obtained by averaging the interframe differences between the same pictures.

  The average difference between frames is the difference between every block or every pixel in the (t-1) frame and t frame, which are the same picture (for example, P), and the average of the difference values is calculated. It is the value taken.

When the occurrence of a scene change is detected, the picture insertion control means 22 inserts an I picture in the stream (performs intra-frame coding).
Next, scene change detection will be described. When a scene change is detected, any of the following five states is considered to be a scene change.
(1) The average difference between frames is greater than the threshold value. In such a case, it can be considered as a full screen scene change.
(2) A case where there are many blocks having a large difference between a block average value obtained by blocking a picture and calculating an average of pixel data for each block and an average difference value between frames. In such a case, it is considered that the scene has been changed on the partial screen (scenery is the same, a person suddenly appeared).
(3) When (1) and (2) are combined. That is, the average difference between frames is larger than the threshold value, and the difference between the block average obtained by blocking the picture and averaging the pixel data for each block and the average difference between frames is large. When there are many blocks.
(4) When the change of the inter-frame difference average value is larger than a certain value and shows a sudden change.
(5) When the average difference value between frames is lower than the threshold value and the change is larger than a certain value, indicating a sudden change.

Next, an image control apparatus having both of the image control apparatuses 10 and 20 will be described in detail. FIG. 4 is a diagram showing the configuration of the image control apparatus.
The image control device 30 is a device that embodies both the above-described B repetition number adaptive control and I insertion control when a scene change occurs.

  The fixed delay unit 31 delays the input image data by the time required for scene change detection for time adjustment. The motion vector detection unit 32 detects the motion vector of the (t−1) frame and the t frame, and transmits the value of the motion vector to the encoding control unit 4. Although motion compensation (hereinafter referred to as MC) is also performed by the encoder 37, the motion vector detection unit 32 detects a motion vector in advance for B repetition rate determination control.

  The MC prediction error detection unit 33 shifts the stored previous frame according to the motion vector, detects an MC prediction error that is a deviation from the current frame, and transmits the value of the MC prediction error to the encoding control unit 4 To do.

  The inter-frame difference detection unit 34 detects an inter-frame difference between the (t−1) frame and the t frame. The fixed delay unit 35 delays the received inter-frame difference and transmits the delayed inter-frame difference value to the encoding control unit 4 in order to adjust the time of the process performed by the scene change detection unit 100.

  The scene change detection unit 100 detects the occurrence of a scene change based on the interframe difference. Then, a 1-bit signal indicating whether or not a scene change has occurred is transmitted to the encoding control unit 4.

  The encoding control unit 4 processes the input data based on the frame CLK, generates a memory control signal for controlling the memory 36, and transmits the memory control signal to the memory 36. Further, a prediction mode signal for setting any one of I, P, and B is transmitted to the encoder 37.

The memory 36 performs writing / reading processing of the image data transmitted from the fixed delay unit 31 based on the memory control signal.
The encoder 37 encodes the image signal transmitted from the memory 36 based on the prediction mode signal. For example, when the prediction mode signal indicates I, the image signal transmitted from the memory 36 is subjected to intraframe prediction encoding processing.

FIG. 5 is a diagram showing the configuration of the encoding control unit 4. The encoding control unit 4 includes a B value control unit 40 and a state transition block 400.
The B value control unit 40 determines Inc / H / Dec (increase / hold / decrease) of the number of repetitions of B based on the motion vector, the MC prediction error, and the interframe difference.

  The state transition block 400 displays I, P, B when a scene change is detected and when the B value control unit 40 determines that it is necessary to increase or decrease the number of repetitions of B (when the determination result is Inc or Dec). Control the transition state.

  FIG. 6 is a diagram showing a configuration of the B value control unit 40. The comparison unit 41 compares the motion vector with a threshold value. The accumulating units 42a and 42b receive the reset signal in block units (basic units for image coding) and accumulate MC prediction errors and interframe differences in block units, respectively.

  The comparison unit 43 compares the accumulated MC prediction error with a threshold value. The subtracter 44 subtracts the accumulated MC prediction error from the accumulated interframe difference. The comparison unit 45 compares the subtraction value with a threshold value.

  The matrix processing unit 46 receives the comparison results from the comparison units 41, 43, and 45, and calculates and outputs Inc / H / Dec in units of blocks based on the determination criteria set internally.

  The counters 47a to 47c count the number of Inc / H / Dec blocks per frame (one screen) based on the block CLK. The weighting unit 48 weights the count values output from the counters 47a to 47c and outputs signals W1 to W3. The selection unit 49 selects the largest signal from the signals W1 to W3 and uses it as the final Inc / H / Dec determination result in the frame.

  FIG. 7 is a diagram showing the configuration of the memory 36. The memory 36 includes three FIFOs 36a to 36c for I, B, and P, an OR element 36d, and a frame memory 36e for increasing P.

  The memory 36 separates the I / P / B based on the memory control signal from the encoding control unit 4 and stores it in the FIFOs 36a to 36c. Further, when the output data is transferred to the encoder 37, it is necessary to rearrange the image frames in accordance with I / B / P. This is done by changing the selection destination of the FIFOs 36a to 36c to be read. When B is increased, P repeats once. Further, when reducing B, one B is discarded.

In the figure, WE-I, B, P and RE-I, B, P, a discard flag, and RE-P (increase) are included in the memory control signal.
FIG. 8 is a diagram illustrating a matrix table managed by the matrix processing unit 46.
The matrix processing unit 46 uses the matrix table T to correspond to the information of 3 bits in total, which is a comparison result of the motion vector, the MC prediction error, and the threshold value for the inter-frame difference value, Inc / H / Dec. Are determined as shown in the figure. In the table, 1 is larger than the threshold value, and 0 is smaller than the threshold value.

  Next, the scene change detection unit 100 will be described. The configuration of the scene change detection unit shown in FIGS. 9 to 13 corresponds to the states (1) to (5) related to the occurrence of the scene change described above.

  FIG. 9 is a diagram showing the configuration of the scene change detection unit. For the scene change detecting unit 100-1, the accumulating unit 111 receives a frame reset and accumulates inter-frame differences in units of frames. The average value processing unit 112 takes the average value of accumulated inter-frame differences according to the frame CLK, and generates an inter-frame difference average value.

  The comparison unit 113 compares the inter-frame difference average value with the threshold value. If the inter-frame difference average value is larger than the threshold value, it is considered that a scene change has occurred. In this case, for example, the output signal OUT is “1”.

  FIG. 10 is a diagram showing the configuration of the scene change detection unit. For the scene change detecting unit 100-2, the accumulating unit 121 receives a frame reset and accumulates inter-frame differences in units of frames. The average value processing unit 122 takes the average value of the accumulated inter-frame differences according to the frame CLK, and generates an inter-frame difference average value.

The accumulating unit 124 receives the block reset, accumulates the inter-frame differences in units of blocks, and generates a block average value.
Here, in order to determine whether or not the difference between the interframe difference average value and the block average value is large, an adder 123 adds an offset to the interframe difference average value. Then, the comparison unit 125 compares the inter-frame difference average value after the offset addition and the block average value, and outputs a 1-bit signal indicating whether or not the difference is large in units of blocks.

The counter 126 receives the frame reset, counts the output signal from the comparison unit 125 according to the block CLK, and outputs the count value in units of frames.
The capturing unit 127 captures the count value according to the frame CLK. The comparison unit 128 compares the captured count value with a threshold value. If the difference value is larger than the threshold value, it is considered that a scene change has occurred. In this case, for example, the output signal OUT is “1”.

FIG. 11 is a diagram showing the configuration of the scene change detection unit. The same components as those of the scene change detection unit 100-2 in FIG. 10 are denoted by the same reference numerals, and the description thereof is omitted.
The comparison unit 131 compares the inter-frame difference average value output from the average value processing unit 122 with a threshold value. When the inter-frame difference average value is larger than the threshold value, for example, the output signal OUT2 is “1”.

The AND element takes the logical product of the outputs of the comparison unit 128 and the comparison unit 131. The output when a scene change occurs is “1”.
FIG. 12 is a diagram showing the configuration of the scene change detection unit. For the scene change detecting unit 100-4, the accumulating unit 141 receives the frame reset and accumulates the inter-frame differences in units of frames. The average value processing unit 142 takes the average value of the accumulated inter-frame differences according to the frame CLK, and generates an inter-frame difference average value.

  The capturing unit 143 captures the inter-frame difference average value according to the frame CLK. The capturing unit 144 captures the inter-frame difference average value output from the capturing unit 143 according to the frame CLK.

  The subtracter 145 obtains a difference between the inter-frame difference average value (previous frame side) output from the capturing unit 143 and the inter-frame difference average value (current frame side) output from the capturing unit 144.

  The comparison unit 146 compares the difference value output from the subtractor 145 with a threshold value. If the difference value is larger than the threshold value, it is considered that a scene change has occurred. In this case, for example, the output signal OUT is “1”.

FIG. 13 is a diagram showing the configuration of the scene change detection unit. It should be noted that the same components as those of the scene change detection unit 100-4 in FIG.
The capturing unit 151 captures the difference value output from the subtracter 145. The comparing unit 152 compares the difference value IN output from the capturing unit 151 with a threshold value. If IN <threshold, it is regarded as a stable state, and the comparator 152 outputs “1”.

  The four-stage flip-flop FF154 holds “1” in accordance with the frame CLK and outputs it to the AND element 155. When all the outputs of the flip-flops FF are 1, the frames with the same pattern are continuous. The comparison unit 153 compares the difference value IN output from the capturing unit 151 with a threshold value. If IN> threshold, the comparator 152 outputs “1”.

The AND element 155 outputs “1” indicating that a scene change has occurred when all the inputs are “1” (when the input changes from a stable state to a sudden change state).
Next, the state transition operation performed in the state transition block 400 will be described in detail. 14 and 15 are state transition diagrams on the writing side. 16 and 17 are state transition diagrams on the reading side. FIG. 18 is a write / read sequence diagram when B increases. FIG. 19 is a write / read sequence diagram when B decreases.

  First, the operation on the writing side will be described. In addition, as a common matter of subsequent state transition operations, state transition is performed in units of frames, and state variable setting by increasing / decreasing B and scene change is performed asynchronously.

  First, in the INIT state, the B repetition cycle = M0, the number of I or P in the GOP = N0, and the variable are reset. The state immediately after initialization is the B state. Here, M0 and N0 are set in temporal variables N and M. In the case of a closed GOP, the BB flag is set to prohibit forward prediction (corresponding to the write sequence B1 in FIG. 18).

  In this state, since N = 3, M = 2, and BB = set, the condition “M ≠ 1 and INC1 = DEC = 0” is satisfied, and the next transition destination is the B state from FIGS. (This corresponds to the write sequence B2 in FIG. 18).

  As shown in the transition diagrams of FIGS. 14 and 15, since the value of M is reduced by 1 when returning to B, in this state (B2), N = 3, M = 1, and BB = set. Accordingly, since “M = 1 and N = 3” holds, the next transition destination is the I state from FIGS.

  Also, as shown in FIGS. 14 and 15, BB is reset when entering the I state. As a result, forward prediction of B pictures at the GOP boundary is prohibited, and a closed GOP can be realized. After the I state, the state is unconditionally transited to the B state (corresponding to the write sequence B3 in FIG. 18).

  When entering the B state, the value of N is decremented by 1, the value of M is returned to the initial value, and DEC2 and INC2 are reset. Since DEC2 and INC2 are not originally set, N = M = 2 and all other flags are reset. Therefore, the condition of “M ≠ 1 and INC1 = DEC = 0” is satisfied, and the next transition destination becomes the B state again from FIGS. 14 and 15 (corresponding to the write sequence B4 in FIG. 18).

  Further, as shown in FIGS. 14 and 15, since the value of M is reduced by 1 when returning to B, in this state, N = 2 and M = 1. In this case, since the condition “M = 1 and N ≠ 3” is satisfied, the next transition destination is the P state (corresponding to the write sequence P1 in FIG. 18).

In this state, since N = 2, M = 0, and CHG ≠ 1, the next transition destination is B. When entering the B state, the value of N is decremented by 1, the value of M is reset, and DEC2 and INC2 are set. Reset.
When the increase or decrease of B and the scene change do not occur, in the same manner, transition is made to B, B, P, B, B, P, B, B, P, B. When the value of N is 0 when entering the P state, the next state is the B state as in the other cases, but unlike the other cases, the value of N is also returned to the initial value. In this way, the transition of the GOP cycle (BBI BBP BBP BBP B ·· BP BBP) (BBI BBP...) Can be performed.

  Next, the operation on the reading side will be described. The reading side starts by sending a fixed delay compared to the writing side. The first state is the I state (see FIGS. 16 and 17), and M0 and N0 are substituted into temporal variables N and M. The next transition destination is unconditionally in the B state as shown in FIGS. 16 and 17 (corresponding to the read sequence B1 in FIG. 18). Since the value of M is decremented by 1 when transitioning to the B state, M = 1 and N = 3 in this state. In this state, since the condition “M ≠ 0 and DecSet ≠ 1” is satisfied, the next transition destination is also B (corresponding to the read sequence B2 in FIG. 18).

  Since the value of M is decremented by 1 when transitioning to the B state, M = 0 and N = 3 in this state. In this state, since the condition “M = 0 and N ≠ 0” is satisfied, the next transition destination is the P state (corresponding to the read sequence P1 in FIG. 18).

  At the time of transition to P, the value of M is reset and the value of N is decremented by 1. Therefore, in this state, M = 2 and N = 2. Then, if no increase in B or no scene change has occurred, the condition “IncSet = 0” is satisfied, and the next transition destination is B. Similarly, B, B, P, B, B, P, B, B, P, B,.

  When entering the B state, the value of N is 0, and when M = 0 while looping through the B state, the state transitions to the I state as shown in FIGS. 16 and 17, and the initial state (GOP Return to the top of In this way, transition of the GOP cycle (IBB PBB PBB PBB PB... PBB) (IBB PBB...) Can be performed.

  Next, the operation when B increases will be described. The increase in B is monitored in the B state on the writing side (see FIGS. 14 and 15). When in the B state on the writing side, the processing of the previous increase request has been completed (INC2 ≠ 1), and if the value of M does not exceed the maximum value and an increase determination is made, an increase request The flag INC1 / 2 is set asynchronously, and the values of M and M0 are incremented by one. Then, the B state is looped until the value of M becomes 1.

  In FIG. 18, when an increase determination is made at the write sequence B5, INC1 and INC2 are set near the end of B5, and the value of M0 is increased by 1 and changed to 3, as shown in FIG. At the same time, the value of M is also added to the current value of 2 to become 3. Since the next transition destination of B5 is INC1 = 1, it becomes B again from FIGS.

  When transitioning to the write sequence B6 in FIG. 18, the value of M is decremented by 1 and INC1 is reset, as shown in FIGS. Therefore, in B6, M = 2, INC1 = 0, and INC2 = 1. Then, according to the transition diagrams of FIGS. 14 and 15, the B state is looped until the value of M becomes 1.

  Since the condition “M = 1 and N ≠ 3” is satisfied at the write sequence B7 in FIG. 18, the next transition destination is the P state. Since INC2 = 1 at the write sequence P2 in FIG. 18, IncSet, which is the B increase flag on the read side, is set to 1.

  The next transition destination on the writing side is in the B state because CHG ≠ 1, and when transitioning to the writing sequence B8 in FIG. 18, N is subtracted, M is reset, and INC2 is reset. Similarly, until the M becomes 1, the B state is looped, transitioned to the P state, and then returned to the B state. When “N = 0 and M = 0” holds (for example, the write sequence P3 in FIG. 18) when transitioning to the P state, initialization of GOP is performed by initializing N when transitioning to the B state. Do.

  The reason why INC2 is included in the increase determination condition for B is to limit the increase unit of B between I or P to 1. As shown in FIG. 18, normally, there is a time difference of about 1 picture between I and P for writing and I and P for reading. Since the increase amount of B between I and P or between P and P is 1 at the maximum, when B is increased on the writing side, writing and reading of I and P are substantially in phase ( Reading is slightly slower).

  Therefore, as shown in FIG. 18, when IncSet is set on the writing side, the phase at which IncSet is output matches the phase of the P state on the reading side. On the reading side, if IncSet is set when the state transitions to the P state (reading sequence P2 in FIG. 18), the state transitions to the P Inc state as shown in the state transition diagrams of FIGS. When transitioning to the P Inc state, the value of M is incremented by 1, the IncSet is reset, and the P increase flag is output while in this state.

  The P increase flag is connected to the REN-P increase in the memory block diagram, and outputs the data on the FM side to re-output the data in the P2 state. The transition after the P Inc state is the same as the normal P state, but since the value of M has increased by 1, B increases.

  Next, the operation when B decreases will be described. The decrease in B is monitored in the B state on the writing side (see FIGS. 14 and 15). When the write side is in the B state, the process of the previous decrease request is completed, and if the value of M exceeds 1 and a decrease determination is made, the decrease request flag DEC1 / 2 is set asynchronously and the values of M and M0 are decremented by one. Then, the B state is looped until the value of M becomes 1.

  In FIG. 19, when the decrease determination is performed at the write sequence B5, as shown in FIG. 19, DEC1 and DEC2 are set near the end of B5, and the value of M0 is decreased by 1 and changed to 1. At the same time, the value of M is also subtracted from the current value of 1 to become 1. 14 and 15, the next transition destination of B5 is P state because “M = 1 and N ≠ 3” holds.

  At the time of transition to the write sequence P2 in FIG. 19, as shown in FIG. 14, the value of M is decreased by 1, DEC1 is reset, and DecSet is set. Therefore, in the write sequence P2 of FIG. 19, M = 0, DEC1 = 0, and DEC2 = 1. Then, according to the transition diagrams of FIGS. 14 and 15, the B state is looped until the value of M becomes 1. In this case, since the value of M is 1 from the beginning, the transition is alternately made between B and P. When “N = 0 and M = 0” is satisfied when the state transitions to the P state (for example, the write sequence P3 in FIG. 19), when the state transitions to the B state, the GOP is initialized by initializing N. .

  The reason why DEC2 is included in the increase determination condition for B is to limit the decrease unit of B between I and P to 1. If M is not 2 or more, the B decrease mode is not performed. Therefore, as is apparent from FIG. 19, when DecSet becomes 1, the read sequence side is in the B state. Therefore, in the state transition diagrams (READ) of FIGS. 16 and 17, DecSet is monitored by B STATE.

  In the example of FIG. 19, DecSet is set when in the read sequence B3. In this case, since N = 2, M = 1, and DecSet = 1, the condition “M ≦ 1 and DecSet = 1 N ≠ 0” is satisfied, and the next transition destination in FIGS. 16 and 17 is P Dec STATE. At the time of transition to this state, the value of N and the value of M0 are decremented by 1, M is initialized, and DecSet is reset. While in this state, since it is in the P state, data in the P frame is read out, but by outputting a discard flag, data in the B frame is also read out at the same time. However, this data (B4) is discarded without being used.

  As shown in FIGS. 16 and 17, the next transition destination of P Dec STATE is in the B state unconditionally. After that, when the condition of “N = 0 and M = 0” is met alternately between B and P (for example, the read sequence B6 in FIG. 19), the transition is made to the I state at the next timing, and the GOP Perform unit initialization.

Next, the operation at the time of a scene change will be described. 20 to 22 are write / read sequence diagrams at the time of a scene change. Scene changes are monitored in the B state and P state on the writing side. Since scene change detection is two frames ahead of other processes, the process is performed two frames before the scene change screen.

  For example, when a scene change occurs at P2 → I2 in FIG. 20, scene change processing is performed at B5, B6-P, and P2 → I2. The process on the writing side holds the value of M as MC, and transits to the I state via the P state to forcibly change the frame of the scene change to the I picture and the last GOP on the writing side. Is fixed to P.

  The reading side controls the subsequent transition according to the ChgSet flag and the value of MC in the P state. When ChgSet = 1 and MC = 0, the transition is made to the I state, the P state, and then the normal transition. When ChgSet = 1 and MC ≠ 0, the B CHG state is transitioned M0 + 1-MC times, then the transition is made to I and P, and then the normal transition is performed.

  Next, the operation will be described with reference to FIGS. If the picture in the write sequence P2 → I2 in FIG. 20 is a scene change picture, a scene change is detected at B5. Then, MC = 2 and CHG1 = 1 are set in the B state of FIGS. The next transition destination is the P state because the condition “(M = 1 and N ≠ 3) or CHG1 = 1” is satisfied in the B state in FIGS. 14 and 15 (B6-P in FIG. 20). When entering this state, the value of M is decremented by 1 and M = 1.

  The next transition destination is I STATE because “CHG1 = 1” is established in P STATE of FIGS. 14 and 15 (P2 → I2 of FIG. 20). When entering this state, CHG1, M, and N are initialized, and ChgSet that is a scene change detection flag on the reading side is set to 1.

  The next transition destination is unconditionally set to B STATE, and the subsequent transition is normal operation. Since the reading side starts to operate with repetition of B = 2, it transitions from I1 → B1 → B2 → P1 → B3 → B4 → P (B6-P). 16 and 17, the condition “ChgSet = 1 MC ≠ 0” is satisfied, so that the next transition destination is B CHG STATE (read sequence B5 in FIG. 21).

  Further, since CNT = 2-2-1 = 1, the condition “CNT ≦ 1” is satisfied, and the next transition destination is I CHG STATE (read sequence I2 in FIG. 21). Here, N is initialized, and transition is made unconditionally to P STATE (read sequence P3 in FIG. 21) at the next timing. At the transition to P STATE, N is subtracted, M is initialized, and ChgSet is reset, and the subsequent operation is a normal operation.

Therefore, as shown in FIG. 20, it is possible to make the scene change screen an I picture and prohibit the reference of the predicted value across the scene change screen.
If the picture in the write sequence B7 → I2 in FIG. 21 is a scene change picture, a scene change is detected at B6. Then, MC = 1 and CHG1 = 1 are set in the B state of FIGS.

  The next transition destination is the P state because the condition “(M = 1 and N ≠ 3) or CHG1 = 1” is satisfied in the B state in FIGS. 14 and 15 (P2Chg in FIG. 21). When entering this state, the value of M is decremented by 1 and M = 0.

  The next transition destination is I STATE because “CHG1 = 1” is established in P STATE of FIGS. 14 and 15 (B7 → I2 in FIG. 21). When entering this state, CHG1, M, and N are initialized, and ChgSet that is a scene change detection flag on the reading side is set to 1.

  The next transition destination is unconditionally set to B STATE, and the subsequent transition is normal operation. Since the reading side starts to operate with repetition of B = 2, transition is made from I1 → B1 → B2 → P1 → B3 → B4 → P2. In this state, the condition “ChgSet = 1 MC ≠ 0” is satisfied, so that the next transition destination is B CHG STATE (read sequence B5 in FIG. 21). Also, since CNT = 2-1-1 = 2, the condition “CNT> 1” is satisfied, and after transitioning to B CHG STATE once again (FIG. 21 read sequence B6), I CHG STATE (read sequence in FIG. 21). Transition to I2).

  Here, N is initialized, and at the next timing, transition is made unconditionally to P STATE (read sequence P3 in FIG. 21). At the transition to P STATE, N is subtracted, M is initialized, and ChgSet is reset, and the subsequent operation is a normal operation. Therefore, as shown in FIG. 21, it is possible to make the scene change screen an I picture and prohibit the reference of the predicted value across the scene change screen.

  If the picture at the write sequence B8 → I2 in FIG. 22 is a scene change picture, a scene change is detected at P2. Then, MC = 0 and CHG2 = 1 are set in the P state of FIGS.

  The next transition destination is the P state (B7-P in FIG. 22) because the condition “CHG2 = 1” is satisfied in the P state in FIGS. When returning to this state, CHG2 is reset and CHG1 is set, so that the next transition destination is I STATE (B8 → I2 in FIG. 22). When entering this state, CHG1, M, and N are initialized, and ChgSet that is a scene change detection flag on the reading side is set to 1.

  The next transition destination is unconditionally set to B STATE, and the subsequent transition is normal operation. Since the read side starts to operate at B repetition = 2, the transition is made as I1 → B1 → B2 → P1 → B3 → B4 → P2 → B5 → B6 → B7−P.

  16 and 17, the condition “ChgSet = 1 MC = 0” is satisfied, so that the next transition destination is I CHG STATE (read sequence I2 in FIG. 22). Here, initialization of N is performed, and at the next timing, transition is made unconditionally to P STATE (read sequence P3 in FIG. 22).

  At the transition to P STATE, N is subtracted, M is initialized, and ChgSet is reset, and the subsequent operation is a normal operation. Therefore, as shown in FIG. 22, it is possible to make the scene change screen an I picture and prohibit the reference of predicted values across the scene change screen.

  As described above, since it is possible to automatically converge to the optimum B value from the relationship between the magnitude of the motion vector, the MC prediction error, and the inter-frame difference, it is possible to improve encoding efficiency and image quality. It becomes possible. Also, by inserting I when a scene change occurs, an increase in the amount of information due to a scene change can be suppressed as compared with the conventional method in which the period of I is fixed, so that it is possible to improve encoding efficiency and image quality. It becomes possible.

(Supplementary Note 1) In an image control apparatus that performs encoding control of an image signal,
A first comparison process for comparing a motion vector and a threshold; a second comparison process for comparing a motion compensation prediction error and a threshold; a subtraction value obtained by subtracting a motion compensation prediction error from an interframe difference; A comparison processing means for performing at least one comparison process of a third comparison process for comparing with
Repetition number control means for adaptively controlling the number of repetitions of the B picture to be inserted into the stream based on the information of the comparison result;
An image control apparatus comprising:

  (Supplementary Note 2) When it is determined in the first comparison process that the motion vector is smaller than the threshold value, the repetition number control means increases in a direction to increase the repetition number of the B picture. The image control apparatus according to appendix 1, wherein the control is performed in a direction to decrease the number of repetitions.

  (Supplementary Note 3) When the second comparison process determines that the motion compensation prediction error is smaller than the threshold value, the repetition number control unit increases the repetition number of the B picture. The image control apparatus according to appendix 1, wherein when it is determined that the number of repetitions is large, the number of repetitions is controlled to decrease.

  (Supplementary Note 4) In the third comparison process, when the subtraction value is larger than the threshold value, the repetition number control unit increases the repetition number of the B picture. The image control apparatus according to appendix 1, wherein the number of repetitions is controlled in a decreasing direction.

  (Supplementary Note 5) The repetition number control means increases the number of repetitions of the B picture by associating the comparison results of the first comparison process, the second comparison process, and the third comparison process. The image control apparatus according to appendix 1, wherein control is performed in either a holding direction or a decreasing direction.

(Additional remark 6) In the image control apparatus which performs encoding control of an image signal,
Scene change detection means for detecting the occurrence of a scene change based on an average difference between frames obtained by averaging the differences between frames between the same pictures;
When the occurrence of the scene change is detected, picture insertion control means for inserting an I picture in the stream;
An image control apparatus comprising:

  (Supplementary note 7) The image control apparatus according to supplementary note 6, wherein the scene change detection means regards the occurrence of a scene change when the inter-frame difference average value is greater than a threshold value.

  (Supplementary Note 8) The scene change detection means obtains a difference between a block average value obtained by blocking a picture and averaging pixel data for each block, and the inter-frame difference average value, and the difference is determined from a constant value. The image control apparatus according to appendix 6, wherein when there are more than a certain number of large blocks, it is considered that a scene change has occurred.

  (Supplementary Note 9) The scene change detection means includes a block average value in which the inter-frame difference average value is larger than a threshold value and blocks the picture and averages pixel data for each block, and the inter-frame difference average 7. The image control apparatus according to appendix 6, wherein a difference from the value is obtained, and when there are more than a certain number of blocks in which the difference is greater than a certain value, it is regarded that a scene change has occurred.

  (Supplementary note 10) The supplementary note 6, wherein the scene change detection means considers that a scene change has occurred when a change in the inter-frame difference average value is larger than a certain value and shows a sudden change. Image control device.

  (Supplementary Note 11) The scene change detection means considers that a scene change has occurred when the inter-frame difference average value is lower than a threshold value and the change is greater than a certain value and indicates a sudden change. The image control device according to appendix 6, characterized by:

(Additional remark 12) In the image control apparatus which performs encoding control of an image signal,
A first comparison process for comparing a motion vector and a threshold; a second comparison process for comparing a motion compensation prediction error and a threshold; a subtraction value obtained by subtracting a motion compensation prediction error from an interframe difference; A comparison processing means for performing at least one comparison process of a third comparison process for comparing with
Repetition number control means for adaptively controlling the number of repetitions of the B picture to be inserted into the stream based on the information of the comparison result;
Scene change detection means for detecting the occurrence of a scene change based on an average difference between frames that is an average of differences between frames between the same pictures;
When the occurrence of the scene change is detected, picture insertion control means for inserting an I picture in the stream;
An image control apparatus comprising:

DESCRIPTION OF SYMBOLS 10 Image control apparatus 11 Comparison processing means 12 Repeat number control means

Claims (4)

In an image control apparatus that performs encoding control of an image signal,
A scene change detection means for detecting the occurrence of a scene change based on an average difference between frames obtained by averaging the difference between frames for each block or pixel between pictures of the same picture type ;
When the occurrence of the scene change is detected, picture insertion control means for inserting an I picture in the stream;
With
The scene change detection means calculates a difference between a block average value obtained by blocking a picture and averaging pixel data for each block, and the inter-frame difference average value, and blocks where the difference is greater than a certain value are constant. If there are more than the number of
An image control apparatus characterized by that.   In an image control apparatus that performs encoding control of an image signal,
  A scene change detection means for detecting the occurrence of a scene change based on an average difference between frames obtained by averaging the difference between frames for each block or pixel between pictures of the same picture type;
  When the occurrence of the scene change is detected, picture insertion control means for inserting an I picture in the stream;
  With
  The scene change detecting means has a difference between a block average value in which the average difference between frames is larger than a threshold value and a picture is blocked and pixel data is averaged for each block, and the average difference between frames. If there are more than a certain number of blocks in which the difference is greater than a certain value, it is considered that the scene change has occurred.
  An image control apparatus characterized by that.   In an image control apparatus that performs encoding control of an image signal,
  A scene change detection means for detecting the occurrence of a scene change based on an average difference between frames obtained by averaging the difference between frames for each block or pixel between pictures of the same picture type;
  When the occurrence of the scene change is detected, picture insertion control means for inserting an I picture in the stream;
  With
  The scene change detection means, when the change amount of the inter-frame difference average value is larger than a certain value and shows a sudden change, it is regarded as the occurrence of the scene change,
  An image control apparatus characterized by that.   In an image control apparatus that performs encoding control of an image signal,
  A scene change detection means for detecting the occurrence of a scene change based on an average difference between frames obtained by averaging the difference between frames for each block or pixel between pictures of the same picture type;
  When the occurrence of the scene change is detected, picture insertion control means for inserting an I picture in the stream;
  With
  The scene change detection means considers the occurrence of the scene change when the inter-frame difference average value is lower than a threshold value and the change is larger than a certain value and indicates a sudden change.
  An image control apparatus characterized by that.

Images