JP2011217085A - Image encoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image encoding device which performs code amount control corresponding to a system which transmits videos in real time.SOLUTION: Uncompressed image data 31a, 31b are input to an image encoding device 2. An encoding unit 22 encodes macroblocks of the uncompressed image data 31a, 31b on the basis of sequence information 41 indicating a sequence of encoding the macroblocks of the uncompressed image data 31a, 31b. An initial-value determining unit 232 determines an initial value of a quantization parameter QP of MBs to be encoded. A generated code amount integrating unit 231 integrates generated code amounts of a predetermined number of macroblocks encoded immediately before encoding the MBs to be encoded. A correction-value determining unit 233 determines a correction value V1 used for correcting the initial value on the basis of a target code amount per predetermined number of macroblocks and the integrated value of the generated code amounts of the predetermined number of macroblocks. The quantization parameter QP of the MBs to be encoded is determined by summing the initial value and the correction value V1.

Description

  The present invention relates to an image encoding apparatus that encodes a plurality of image data and outputs one compressed image data.

  When monitoring an ATM (automated teller machine) corner provided at a store of a financial institution, a plurality of cameras may be installed. By monitoring from different angles using a plurality of cameras, it is possible to prevent the generation of blind spots.

  The video signal generated by each camera is encoded and transmitted because the bandwidth of the transmission path is limited. Since it is necessary to simultaneously display and monitor the video of each camera, it is necessary to multiplex and transmit a plurality of video signals corresponding to each camera.

  Patent Document 1 below discloses a video signal encoding device that converts a plurality of frame videos input from each camera into a single system of compressed data.

  The video signal encoding apparatus according to Patent Literature 1 extracts a frame video to be encoded by thinning out a frame video that is not encoded from a frame video input from each camera. The extracted frame images are arranged for each camera and encoded in GOP (Group of Pictures) units. Thereby, a plurality of frame images input from each camera are converted into one system of compressed data.

JP 2005-151485 A

  As described above, the video signal encoding apparatus according to Patent Document 1 extracts a frame video to be encoded. The video signal encoding device cannot generate compressed data until the number of encoding target frame videos that can generate a GOP is extracted. As a result, since the time (delay) from when the frame video is input to the video signal encoding device to when it is output as one system of compressed data becomes long, it is difficult to transmit the video of each camera in real time. .

  In Patent Document 1, code amount control is performed in GOP units. When the code amount control is performed in GOP units, the generated code amount may momentarily exceed the target code amount. Since the target code amount is determined by the bandwidth of the transmission path and the like, there is a possibility that a transmission delay or the like may occur when the generated code amount greatly exceeds the target code amount.

  In view of the above problems, an object of the present invention is to provide an image encoding apparatus that performs code amount control corresponding to a system for transmitting video in real time.

  In order to solve the above problem, the invention described in claim 1 is an image encoding device that encodes first to Nth (N is a natural number of 1 or more) image data, and k is a natural number from 1 to N. The coding unit for coding the k-th macroblock based on the order information indicating the coding order of the k-th macroblock included in the k-th image data, and the quantum of the macroblock to be coded A quantization parameter determining unit that determines an encoding parameter, wherein the quantization parameter determining unit is a group of encoded macroblocks encoded by the encoding unit and immediately before the encoding target macroblock. A first number of macroblocks encoded in the first number, a code amount integrating unit that calculates an integrated value of the generated code amount for each of the first number of macroblocks, the integrated value, and the first number Based on the target code amount per macroblock, determines a correction value of the quantization parameter, and a correction value determining unit for correcting the quantization parameter based on the correction value.

  According to a second aspect of the present invention, in the image encoding device according to the first aspect, the quantization parameter determining unit, when the encoding target macroblock is included in the k-th image data, An initial value determination unit that identifies the second number k-th macroblock coded immediately before the block and calculates an average value of the quantization parameter for each of the second number k-th macroblock as an initial value The quantization parameter determination unit determines the quantization parameter based on the initial value and the correction value.

  According to a third aspect of the present invention, in the image encoding device according to the first or second aspect, the correction value determining unit includes the code when the encoding target macroblock is included in the k-th image data. The correction value is determined based on the generated code amount of one k-th macroblock encoded immediately before the conversion target macroblock.

  According to a fourth aspect of the present invention, in the image encoding device according to the first or second aspect, the correction value determining unit is configured to determine the correction value based on an activity indicating the complexity of the encoding target macroblock. To decide.

  According to a fifth aspect of the present invention, in the image encoding device according to the fourth aspect, the correction value determining unit includes activities in four regions each including an upper side, a lower side, a left side, and a right side of the encoding target macroblock. And the correction value is determined based on the minimum activity among the activities in each region.

  According to a sixth aspect of the present invention, in the image encoding device according to any one of the first to fifth aspects, the first number is a number of macroblocks constituting a line of macroblocks arranged in a line in the horizontal direction. Is a number.

  According to a seventh aspect of the present invention, in the image encoding device according to any one of the first to sixth aspects, the code amount integrating unit is configured to select the encoding target macro from the encoded macroblock group. The third number of macroblocks encoded immediately before the block are specified, the generated code amount for each of the third number of macroblocks is integrated, the upper limit comparison code amount is calculated, and the quantization parameter determination When the encoding target macroblock is included in the k-th image data and the upper limit comparison code amount is larger than a preset upper limit value, the unit is encoded immediately before the encoding target macroblock. The quantization parameter of one k-th macroblock is determined as a forced initial value, and the correction value determination unit corrects the forced initial value using a preset forced correction value.

  According to an eighth aspect of the present invention, in the image encoding device according to any one of the first to seventh aspects, the encoding unit encodes the first to Nth image data, respectively. N individual encoding units, and an output control unit that controls the output timing of the k-th encoded data generated by the k-th encoding unit based on the order information.

  The image coding apparatus according to the present invention performs code amount control in units of the first number of macroblocks. For this reason, it is possible to prevent the bit rate of the compressed data generated by encoding the k-th macroblock from instantaneously greatly exceeding the target bit rate. Therefore, the present invention can suppress transmission delay and the like.

  When the encoding target macroblock is included in the kth image data, the first number of macroblocks includes encoded macroblocks of other image data excluding the kth image data. For this reason, even when the generated code amount of a coded macroblock of other image data has changed significantly, the image coding apparatus sets the correction value of the quantization parameter of the encoding target macroblock as the generated code amount. It can be changed according to the change of. Therefore, the present invention can flexibly control the bit rate of the compressed data.

It is a figure which shows the structure of the image transmission system which concerns on embodiment of this invention. It is a block diagram which shows the functional structure of the image coding apparatus shown in FIG. It is a figure which shows the structure of the macroblock of uncompressed image data. It is a figure which shows the encoding order of a macroblock group. It is a figure which shows the encoding order of a macroblock group. It is a flowchart which shows the flow of encoding of a macroblock. It is the schematic which shows the structure of multiplexed compression data. It is a figure which shows the calculation object of a line generation code amount. It is a figure which shows a correction value determination table. It is a figure which shows the calculation object of a line generation code amount. It is a figure which shows a correction value determination table. It is a figure which shows a correction value determination table. It is a figure which shows the calculation object of upper limit comparison code amount. It is a block diagram which shows the functional structure of the image coding apparatus which concerns on the modification of embodiment of this invention.

{1. Overall configuration of image transmission system}
FIG. 1 is a diagram showing a configuration of an image transmission system according to an embodiment of the present invention. An image transmission system 100 shown in FIG. 1 includes cameras 1a and 1b, an image encoding device 2, and monitoring devices 4a and 4b. The image transmission system 100 is a system that monitors an ATM corner provided in a financial institution store in real time.

  The cameras 1a and 1b are attached so that the ATM corner can be photographed from different angles. The cameras 1a and 1b take pictures of ATM corners and generate uncompressed image data 31a and 31b. The image encoding device 2 is installed at the ATM corner together with the cameras 1a and 1b. The frame of uncompressed image data 31a and 31b is input to the image encoding device 2 in synchronization. The image encoding device 2 encodes and multiplexes the uncompressed image data 31a and 31b to generate multiplexed compressed data 33. The multiplexed compressed data 33 is transmitted to a monitor room that monitors an ATM corner via a wireless communication network or the like.

  The monitoring devices 4a and 4b are installed in the monitor room and receive the multiplexed compressed data 33. The monitoring devices 4a and 4b perform demultiplexing and decoding processing on the multiplexed compressed data 33, and display images corresponding to the cameras 1a and 1b. The cameras 1a and 1b photograph the same place (ATM corner) from different angles. For this reason, the monitoring devices 4a and 4b display the images of the cameras 1a and 1b in synchronization so that the timing for displaying the images of the cameras 1a and 1b is not shifted. A monitor resident in the monitor room remotely monitors the ATM corner by viewing the images displayed on the monitoring devices 4a and 4b.

  In the present embodiment, the description will be focused on the point that the image encoding device 2 encodes the uncompressed image data 31a and 31b to generate the multiplexed compressed data 33, respectively.

  The macroblocks of the uncompressed image data 31a and 31b are each divided into macroblock groups (hereinafter referred to as “MB groups”) composed of one or more macroblocks based on preset division units. The The multiplexed compressed data 33 is generated by alternately encoding the MB group of the uncompressed image data 31a and the MB group of the uncompressed image data 31b.

  The image encoding device 2 determines a quantization parameter of an encoding target macroblock (hereinafter referred to as an encoding target MB). The quantization parameter is determined based on a target code amount per predetermined number of macroblocks and a total sum of generated code amounts of a predetermined number of macroblocks encoded immediately before encoding of the encoding target MB. That is, the image encoding device 2 executes code amount control in units of a predetermined number of macroblocks when the uncompressed image data 31a and 31b are encoded to generate the multiplexed compressed data 33. Since the code amount control is performed in a unit time shorter than the GOP unit, it is possible to prevent the bit rate of the multiplexed compressed data 33 from exceeding a preset target bit rate.

  When the macroblock of the uncompressed image data 31a is the encoding target MB, the predetermined number of macroblocks encoded immediately before include the macroblock of the uncompressed image data 31b. When determining the quantization parameter of the macroblock of the uncompressed image data 31a, the image encoding device 2 uses the code amount of the encoded macroblock of the uncompressed image data 31b. Therefore, the bit rate of the multiplexed compressed data 33 can be stably controlled even if the code amount of the encoded data of the uncompressed image data 31b changes greatly.

{2. Configuration of Image Encoding Device 2}
FIG. 2 is a block diagram illustrating a functional configuration of the image encoding device 2. As illustrated in FIG. 2, the image encoding device 2 includes an order determination unit 21, an encoding unit 22, a quantization parameter determination unit 23, a multiplexing unit 24, and a communication unit 25.

  The order determination unit 21 determines the order in which the macroblocks of the uncompressed image data 31a and 31b are encoded. Order information 41 is created based on the determined order. The order determination unit 21 outputs the order information 41 to the encoding unit 22 and the multiplexing unit 24.

  Based on the order information 41, the encoding unit 22 encodes the macroblocks of the uncompressed image data 31a and 31b to generate encoded data 32a and 32b. For encoding of the uncompressed image data 31a and 31b, for example, H.264 is used. An encoding method such as H.264 or MPEG2 is used. The encoding unit 22 outputs the generated code amount Ea of the encoded MB to be encoded to the quantization parameter determining unit 23. The generated code amount Ea is used when determining the quantization parameter QP of the next encoding target MB.

  The quantization parameter determination unit 23 determines a quantization parameter QP used for quantization of each macroblock of the uncompressed image data 31a and 31b. The quantization parameter determination unit 23 includes a generated code amount integration unit 231, an initial value determination unit 232, and a correction value determination unit 233.

  The generated code amount integrating unit 231 integrates the generated code amounts of a predetermined number of macroblocks encoded immediately before the encoding target MB. The initial value determination unit 232 determines the initial value of the quantization parameter QP of the encoding target MB. The correction value determination unit 233 calculates a correction value used for correcting the initial value based on the target code amount per predetermined number of macroblocks and the integrated value of the predetermined number of macroblocks encoded immediately before. The quantization parameter determination unit 23 outputs the sum of the initial value and the correction value to the encoding unit 22 as the quantization parameter QP.

  The multiplexing unit 24 generates the multiplexed compressed data 33 by arranging the macroblocks of the encoded uncompressed image data 31a and 31b in the encoded order.

  The communication unit 25 adds an error correction signal to the multiplexed compressed data 33 and performs digital modulation. The communication unit 25 transmits the digitally modulated multiplexed compressed data 33 to the monitoring devices 4a and 4b using wireless communication.

{3. Operation of Image Encoding Device 2}
Hereinafter, the operation of the image encoding device 2 will be described in detail.

{3.1. Determination of encoding order}
First, the order determination unit 21 determines the order (encoding order) for encoding the macroblocks of the uncompressed image data 31a and 31b. In determining the encoding order, the order determining unit 21 generates a first MB group from the macroblock of the uncompressed image data 31a and a second MB group from the macroblock of the uncompressed image data 31b.

  FIG. 3 is a diagram illustrating a configuration of macroblocks of uncompressed image data 31a and 31b. As shown in FIG. 3, each frame of the uncompressed image data 31a and 31b is composed of 36 macro blocks. The frame rates of the uncompressed image data 31a and 31b are the same.

  The uncompressed image data 31a and 31b are configured by 6 horizontal × 6 vertical macroblocks. The macro blocks of the uncompressed image data 31a are assigned numbers Ma0 to Ma35 from the upper left to the lower right. The numbers Mb0 to Mb35 are assigned to the macro blocks of the uncompressed image data 31b from the upper left to the lower right.

  In the uncompressed image data 31a and 31b, a set of six macroblocks arranged in a line in the horizontal direction is referred to as a macroblock line (MB line). The uncompressed image data 31a has MB lines La1, La2, La3, La4, La5 and La6. The uncompressed image data 31b has MB lines Lb1, Lb2, Lb3, Lb4, Lb5, and Lb6.

  The order determination unit 21 generates the first MB group and the second MB group by partitioning the macroblocks of the uncompressed image data 31a and 31b based on a preset partition unit. Macroblocks, MB lines, and the like can be used as partition units.

  When a macroblock is set as a partition unit, each of the macroblocks Ma0 to Ma35 is partitioned as the first MB group. Each of the macro blocks Mb0 to Mb35 is divided as the second MB group. When the MB line is set as the division unit, the MB lines La1 to La6 are each divided as the first MB group. The MB lines Lb1 to Lb6 are each divided as the second MB group.

  The partition unit may be set so that the MB group is configured by one or more macroblocks. For example, when four macroblocks are set as a division unit, the macroblocks Ma0 to Ma3 and Ma4 to Ma7 of the uncompressed image data 31a are classified as the first MB group.

  After generating the MB group, the order determination unit 21 determines the encoding order of the MB groups so that the first MB group and the second MB group are alternately encoded.

  FIG. 4 is a diagram illustrating an encoding order when an MB group is generated in units of macroblocks. In this case, the order determination unit 21 determines the encoding order so that the macroblocks Ma0, Mb0, Ma1, Mb1, Ma2, Mb2,..., Mb34, Ma35, and Mb35 are encoded in this order. That is, the encoding order is determined so that the macroblock of the uncompressed image data 31a and the macroblock of the uncompressed image data 31b are encoded alternately.

  FIG. 5 is a diagram illustrating an encoding order when MB groups are generated in units of MB lines. In this case, the order determination unit 21 determines the encoding order such that the MB lines La1, Lb1, La2, Lb2, La3, Lb3,..., La6, Lb6 are encoded in this order. That is, the encoding order is determined so that the MB line of the uncompressed image data 31a and the MB line of the uncompressed image data 31b are encoded alternately. The macroblocks of each MB line are encoded in order from the left (in order of increasing macroblock number).

{3.2. Outline of encoding process}
The order determination unit 21 outputs order information 41 indicating the encoding order of the MB group to the encoding unit 22. Based on the order information 41, the encoding unit 22 encodes the macroblocks Ma0 to Ma35 and Mb0 to Mb35 for each MB group.

  FIG. 6 is a flowchart showing the flow of encoding a macroblock. Assume that the macroblock Ma16 is the encoding target MB. Referring to FIG. 6, encoding unit 22 specifies an encoding target MB based on order information 41 (step S1).

  The quantization parameter determination unit 23 determines whether or not an exception condition is met when determining the quantization parameter QP of the encoding target MB (step S2). If the exception condition is met (Yes in step S2), the quantization parameter QP is determined by executing the exception process (step S10). The exception process determines the quantization parameter QP without using the target code amount per predetermined number of macroblocks and the integrated value of the generated code amount of the macroblock encoded immediately before encoding of the encoding target MB. It is processing to do.

  If the exception condition is not met (No in step S2), the quantization parameter determination unit 23 determines the quantization parameter QP by the processing of steps S3 to S7.

  The initial value determination unit 232 calculates an initial value of the quantization parameter QP of the encoding target MB (step S3). The correction value determining unit 233 determines the correction value V1 (step S4). The correction value V1 is calculated based on the target code amount of the macroblocks per predetermined number and the integrated value of the generated code amounts of the predetermined number of macroblocks encoded immediately before the encoding target MB.

  The correction value determination unit 233 determines the correction value V2 based on the activity indicating the complexity of the image of the encoding target MB (step S5). The correction value determination unit 233 determines the correction value V3 based on the generated code amount of the macroblock encoded immediately before among the encoded macroblocks of the uncompressed image data to which the encoding target MB belongs (step). S6). The quantization parameter determination unit 23 determines the quantization parameter QP of the encoding target MB by summing the initial value and the correction values V1, V2, and V3 (step S7).

  The encoding unit 22 encodes the encoding target MB using the determined quantization parameter QP (step S8). When the encoding target MB is one of the macroblocks Ma0 to Ma35, the encoded macroblock is output to the multiplexing unit 24 as encoded data 32a. When the encoding target MB is one of the macroblocks Mb0 to Mb35, the encoded macroblock is output to the multiplexing unit 24 as encoded data 32b.

  If encoding of all the macroblocks of the uncompressed image data 31a and 31b has not been completed (No in step S9), the image encoding device 2 returns to the process of step S1. If encoding of all the macroblocks has been completed (Yes in step S9), the image encoding device 2 ends the processing shown in FIG.

  The multiplexing unit 24 generates the multiplexed compressed data 33 by arranging the encoded data 32a and 32b in the input order. FIG. 7 is a schematic diagram showing the configuration of the multiplexed compressed data 33. The multiplexed compressed data 33 includes a stream header 34 and stream data Sa and Sb. The stream header 34 includes control information used when reproducing the multiplexed compressed data 33.

  The stream data Sa corresponds to the uncompressed image data 31a, and includes a group header 35a and encoded data 32a. The group header 35a includes identification information of the uncompressed image data 31a and information for specifying the macroblocks constituting the first MB group.

  The stream data Sb corresponds to the uncompressed image data 31b and includes a group header 35b and encoded data 32b. The group header 35b includes identification information for the uncompressed image data 31b and information for specifying the macroblocks constituting the second MB group.

  The multiplexing unit 24 generates stream data Sa and Sb each time the encoded data 32a and 32b are input. The group headers 35a and 35b are created based on the order information 41. The stream data Sa and Sb are sequentially transmitted to the monitoring devices 4a and 4b as multiplexed compressed data 33.

  The image encoding device 2 can start encoding of the uncompressed image data 31a and 31b if data corresponding to the MB group is input from the uncompressed image data 31a and 31b. Therefore, it is possible to reduce the delay that occurs with the encoding of the uncompressed image data 31a and 31b.

  The monitoring devices 4a and 4b continuously receive the multiplexed compressed data 33. The monitoring devices 4a and 4b demultiplex the multiplexed compressed data 33 to obtain encoded data 32a and 32b. Which of the cameras 1a and 1b corresponds to the encoded data 32a and 32b can be confirmed based on the group headers 35a and 35b.

  The monitoring devices 4a and 4b decode the encoded data 32a and 32b in the order of acquisition, and generate decoded data. When the monitoring device 4a is set to display the video of the camera 1a, the monitoring device 4a displays the decoded data corresponding to the camera 1a. When the monitoring device 4b is set to display the video of the camera 1b, the monitoring device 4b displays the decoded data corresponding to the camera 1b.

  Hereinafter, the process of determining the quantization parameter QP (steps S3 to S7, see FIG. 6) will be described in detail for each step. Unless otherwise specified, it is assumed that the encoding target MB is the macroblock Ma16.

{3.3. Determination of initial value QPi (step S3)}
The initial value determination unit 232 calculates the initial value QPi of the quantization parameter QP of the macroblock Ma16 (Step S3). The macro blocks Ma0 to Ma15 are already encoded regardless of the MB group partition unit (see FIGS. 4 and 5). The initial value determination unit 232 identifies a predetermined number (six) of macroblocks Ma10 to Ma15 encoded immediately before the macroblock Ma16 among the encoded macroblocks of the uncompressed image data 31a to which the macroblock Ma16 belongs. To do. The predetermined number (six) is the number of macroblocks of 1 MB line. The initial value determination unit 232 calculates the average of the quantization parameters QP of the specified macroblocks Ma10 to Ma15, and determines the calculated average value as the initial value QPi.

  The initial value QPi is a value that serves as a reference for the quantization parameter QP of the macroblock Ma16. By adding the correction values V1, V2, and V3 to the initial value QPi, the quantization parameter QP of the macroblock Ma16 is determined.

  As described above, the initial value determination unit 232 is the average value of a predetermined number of macroblocks encoded immediately before the encoding target MB among the encoded macroblocks of the uncompressed image data to which the encoding target MB belongs. Is calculated as the initial value QPi of the encoding target MB. The initial value QPi of the macroblock Ma16 does not change significantly from the quantization parameter QP of the encoded macroblock of the uncompressed image data 31a. Accordingly, it is possible to prevent a sudden change in the image quality of the encoded data 32a generated from the uncompressed image data 31a.

{3.4. Determination of correction value V1 (step S4)}
Next, the correction value determination unit 233 determines the correction value V1 (step S4). The correction value V1 is calculated based on the generated code amount of the encoded macroblock of the uncompressed image data 31a and 31b. The process in step S4 will be described separately when the MB group is configured in units of macroblocks and when configured in units of MB lines.

(When MB group is configured in macroblock units)
The correction value V1 is an integrated value (line generation code) of the target code amount set per predetermined number (6) of macroblocks and the generated code amount of six macroblocks encoded immediately before the macroblock Ma16. It is determined on the basis of the difference from the amount. Hereinafter, the target code amount of macroblocks per predetermined number (six) is referred to as a line target code amount. This is because the predetermined number is set to the number of macro blocks (6) per 1 MB line.

  The calculation of the line target code amount will be described. The quantization parameter determining unit 23 calculates a target code amount (MB target code amount) per macroblock in advance. The MB target code amount includes the target bit rate TR (36 Mbps, see FIG. 2) of the multiplexed compressed data 33, the frame rate (30 fps) of the uncompressed image data 31a and 31b, and the macroblocks of the uncompressed image data 31a and 31b. Is calculated on the basis of the total number (72). The line target code amount is calculated by multiplying the MB target code amount by the number of macroblocks per MB line (six).

  The correction value determination unit 233 selects a predetermined number (6 blocks) encoded immediately before the macroblock Ma16 from among the already encoded macroblocks Ma0 to Ma15 and Mb0 to Mb15 (encoded macroblock group). ) Macroblock. FIG. 8 is a diagram illustrating a calculation target of the line generation code amount. As shown in FIG. 8, the six macroblocks encoded immediately before the encoding of the macroblock Ma16 are the macroblocks Ma13, Mb13, Ma14, Mb14, Ma15, and Mb15 in the order of encoding. This is because the macroblock of the uncompressed image data 31a and the macroblock of the uncompressed image data 31b are encoded alternately (see FIG. 3).

  As the encoding target MB changes, the line generation code amount calculation target changes. As shown in FIG. 8, when the macroblock Mb16 becomes the encoding target MB, the macroblocks Ma14 to Ma16 and Mb13 to Mb15 are the targets for calculating the line generation code amount. When the macroblock Ma17 becomes the encoding target MB, the macroblocks Ma14 to Ma16 and Mb14 to Mb16 are targets for calculating the line generation code amount. When an MB group is configured in units of macroblocks, the ratio between the macroblocks of the uncompressed image data 31a and the macroblocks of the uncompressed image data 31b, which are targets for calculation of the line generation code amount, is 1: 1.

  Returning to the description when the macroblock Ma16 is the encoding target MB. The generated code amount integrating unit 231 calculates the line generated code amount by integrating the generated code amounts of the macroblocks Ma13 to Ma15 and Mb13 to Mb15. The correction value determination unit 233 calculates a difference value ΔB1 obtained by subtracting the line target code amount from the line generation code amount.

  The correction value determination unit 233 determines the correction value V1 based on the calculated difference value ΔB1 and the preset variation value determination table 51.

  FIG. 9 is a diagram showing the correction value determination table 51. As shown in FIG. 9, if the difference value ΔB1 is smaller than the threshold value Th00 (−1000 bits), the correction value V1 is set to −4. If the difference value ΔB1 is equal to or greater than the threshold value Th00 and smaller than the threshold value Th01 (−500 bits), the correction value V1 is set to −2. If the difference value ΔB1 is equal to or larger than the threshold value Th01 and smaller than the threshold value Th02 (0 bit), the correction value V1 is set to -1. If the difference value ΔB1 is equal to or greater than the threshold value Th02 and smaller than the threshold value Th03 (500 bits), the correction value V1 is set to +1. If the difference value ΔB1 is greater than or equal to the threshold value Th03 and smaller than the threshold value Th04 (1000 bits), the correction value V1 is set to +2. If the difference value ΔB1 is equal to or greater than the threshold value Th04, the correction value V1 is set to +4.

  As described above, the correction value determination unit 233 determines the correction value V1 of the macroblock Ma16 using the generated code amount of the encoded macroblock of the non-compressed image data 31b.

(When MB group is composed of MB lines)
The determination of the correction value V1 when the MB group is configured in units of MB lines will be described. The macro block for which the line generation code amount is calculated is different from the case where the MB group is configured in units of macro blocks. FIG. 10 is a diagram illustrating a calculation target of the line generation code amount when the MB group is configured in units of MB lines.

  As illustrated in FIG. 10, the correction value determination unit 233 identifies macroblocks Mb10, Mb11, Ma12 to Ma15 as six macroblocks encoded immediately before the macroblock Ma16 becomes the encoding target MB. When an MB group is configured in units of MB lines, macroblocks belonging to the same MB line are encoded in order from the smallest number. In the MB line La3, Ma12 to Ma15 correspond to four of the six macroblocks encoded immediately before the macroblock Ma16 is encoded. As shown in FIG. 5, the MB line encoded immediately before the MB line La3 is the MB line Lb2. For this reason, the macroblocks Mb10 and Mb11 of the MB line Lb2 correspond to two of the six macroblocks encoded immediately before the macroblock Ma16.

  The generated code amount integrating unit 231 calculates the line generated code amount by integrating the generated code amounts of the macroblocks Mb10, Mb11, Ma12 to Ma15 specified by the correction value determining unit 233. The calculation method of the correction value V1 after the line generation code amount is calculated is the same as described above. As described above, the correction value V1 of the macroblock Ma16 is determined based on the generated code amount of the encoded macroblock of the uncompressed image data 31a even when the MB group is configured in units of MB lines.

  As the encoding target MB changes, the line generation code amount calculation target changes. After the macroblock Ma16 is encoded, the macroblocks Ma17, Mb12, and Mb13 are specified as encoding target MBs. When the macroblock Ma17 is the encoding target MB, the line generation code amount calculation targets are the macroblocks Mb11, Ma12 to Ma16. When the macroblock Mb12 is the encoding target MB, the calculation target of the line generation code amount is the macroblocks Ma12 to Ma17. When the macroblock Mb13 is the encoding target MB, the line generation code amount calculation targets are the macroblocks Ma13 to Ma17 and Mb12. When the MB group is configured in units of MB lines, the ratio between the macro block of the uncompressed image data 31a and the macro block of the uncompressed image data 31b that is the calculation target of the line generation code amount changes.

  As described above, the correction value V1 generates the target code amount (line target code amount) per predetermined number of macroblocks and the occurrence of the predetermined number of macroblocks encoded immediately before the encoding target MB is encoded. It is determined based on the integrated value of the code amount (line generation code amount). In the present embodiment, correction is performed based on the amount of generated code generated in units of time shorter than GOP units or frame units. Therefore, since the bit rate of the multiplexed compressed data 33 can be prevented from instantaneously exceeding the target bit rate TR instantaneously, occurrence of transmission delay can be prevented.

  The correction value V1 of the encoding target MB is determined based on the generated code amount of the encoded macroblock of other uncompressed image data. When the bit rate of encoded data generated from one non-compressed image data is low, the bit rate of encoded data generated from the other uncompressed image data can be set high, and the flexible code amount Control becomes possible.

{3.5. Determination of correction value V2 (step S5)}
Next, the correction value V2 determination process (step S5) will be described. The correction value V2 is determined based on the activity of the encoding target MB.

  The correction value determination unit 233 calculates activities in the upper, lower, left, and right areas of the encoding target MB. Specifically, the activity of an area (upper area) of 4 vertical pixels × 16 horizontal pixels including the upper side of the encoding target MB is calculated. Similarly, a vertical 4 pixel × horizontal 16 pixel region including the lower side (lower region), a vertical 16 pixel × horizontal 4 pixel region including the left side (left region), and a vertical 16 pixel × horizontal 4 pixel including the right side. Activity in the area (right area) is calculated. The activity is a numerical value indicating the degree of variation in pixel values of pixels existing in each region. For example, the absolute difference value between the average luminance value of all the pixels in a certain region and the luminance value of each pixel can be calculated as an activity.

  The correction value determination unit 233 selects the minimum activity ACT having the minimum value among the activities in the upper, lower, left, and right areas. Based on the minimum activity ACT and a preset correction value determination table 52, the correction value V2 is determined. FIG. 11 is a diagram showing the correction value determination table 52.

  As shown in FIG. 11, if the minimum activity ACT is smaller than the threshold value Th10 (2), the correction value V2 is set to −4. If the minimum activity ACT is equal to or greater than the threshold value Th10 and smaller than the threshold value Th11 (5), the correction value V2 is set to −2. If minimum activity ACT is equal to or greater than threshold value Th11 and is smaller than threshold value Th12 (10), correction value V2 is set to zero. If the minimum activity ACT is equal to or greater than the threshold value Th12 and smaller than the threshold value Th13 (30), the correction value V2 is set to +2. If the minimum activity ACT is equal to or greater than the threshold value Th03, the correction value V2 is set to +4.

  As described above, the correction value determining unit 233 determines the correction value V2 used for correcting the quantization parameter QP based on the minimum activity ACT among the activities of the upper, lower, left, and right regions of the encoding target MB. By correcting the quantization parameter QP according to the complexity of the image of the encoding target MB, it is possible to execute the code amount control without extremely degrading the image quality.

  In determining the correction value V2, the activity of the encoding target MB may be used instead of the minimum activity ACT. In this case, the correction value determination unit 233 calculates the activity of the entire region of the encoding target MB (vertical 16 pixels × horizontal 16 pixels) and determines the correction value V2.

{3.6. Determination of correction value V3 (step S6)}
Next, the correction value V3 determination process (step S6) will be described. The correction value V3 is determined based on the generated code amount of one macro block encoded immediately before the encoding target MB. The determination of the correction value V3 when the MB group is configured in units of macro blocks will be described.

  Please refer to FIG. Assume that the encoding target MB is the macroblock Ma16. When determining the correction value V3, the correction value determination unit 233 uses the generated code amount of the macroblock Ma15 encoded immediately before the macroblock Ma16 among the encoded macroblocks of the uncompressed image data 31a. . That is, the correction value determination unit 233 acquires the generated code amount of the encoded macroblock encoded immediately before among the encoded macroblocks belonging to the same uncompressed image data as the encoding target MB. The correction value determination unit 233 determines the correction value V3 based on the acquired generated code amount (reference generated code amount RA).

  The correction value determination unit 233 determines the correction value V3 based on the generated code amount (reference generated code amount RA) of the macroblock Ma15 and the preset correction value value determination table 53. FIG. 12 is a diagram showing the variation value determination table 53.

  If the reference generated code amount RA is smaller than the threshold value Th20 (1/2 of the MB target code amount), the correction value V3 is set to -2. The MB target code amount is the target code amount per macroblock as described above. If reference generated code amount RA is equal to or greater than threshold value Th20 and is smaller than threshold value Th21 (MB target code amount), correction value V3 is set to -1. If reference generated code amount RA is equal to or greater than threshold value Th21 and smaller than threshold value Th22 (3/2 of the MB target code amount), correction value V3 is set to +1. If the reference generated code amount RA is equal to or greater than the threshold value Th22, the correction value V3 is set to +2.

  In this way, the correction value determination unit 233 sets the correction value V3 so that the quantization parameter QP increases if the reference generated code amount RA is larger than the MB target code amount. Thereby, the bit rate of the encoded data corresponding to the encoding target MB can be suppressed. If the reference generated code amount RA is smaller than the MB target code amount, the correction value V3 is set so that the quantization parameter QP becomes smaller. Thereby, the image quality of the encoding target MB can be improved.

{3.7. Calculation of quantization parameter QP (step S7)}
After determining the correction values V1, V2, and V3 (steps S4 to S6, see FIG. 6), the quantization parameter determination unit 23 adds the correction values V1, V2, and V3 to the initial value QPi calculated by the initial value determination unit 232. Is added as the quantization parameter QP of the encoding target MB (macroblock Ma16) (step S7).

  The quantization parameter determination unit 23 outputs the quantization parameter QP of the macroblock Ma16 to the encoding unit 22. The encoding unit 22 encodes the macroblock Ma16 using the input quantization parameter QP (step S8). The encoding unit 22 outputs the encoded macroblock Ma16 to the multiplexing unit 24 as encoded data 32a.

{3.8. Exception Processing (Step S10)}
The exception condition determination (step S2) and exception processing (step S10) shown in FIG. 6 will be described in detail below. The exceptional process (step S10) is a process executed when the upper limit comparison code amount may exceed the upper limit set based on the target bit rate. In the exception process, the quantization parameter QP of the encoding target MB is forcibly set without considering the line generation code amount and the minimum activity of the encoding target MB.

  The upper limit comparison code amount is an integrated value of generated code amounts of 12 encoded macroblocks encoded immediately before the encoding target MB. The twelve encoded macroblocks correspond to one third of the number of macroblocks (36) of the uncompressed image data 31a and 31b.

  The process of step S2 will be described by taking as an example a case where the MB group is configured in units of macroblocks. Even when the MB group is configured in units of MB lines, the processes in steps S2 and S10 are similarly executed.

  The quantization parameter determination unit 23 specifies a macro block that is a target of calculation of the upper limit comparison code amount based on the order information 41 and the position of the encoding target MB. FIG. 13 is a diagram illustrating a calculation target of the upper limit comparison code amount. When the MB group is configured in units of macro blocks, the macro blocks of the uncompressed image data 31a and 31b are encoded alternately (see FIG. 4). For this reason, when the encoding target MB is the macroblock Ma8, the macroblocks Ma2 to Ma7 and Mb2 to Mb7 are to be calculated. The generated code amount integrating unit 231 integrates the generated code amounts of the identified encoded macro blocks Ma2 to Ma7 and Mb2 to Mb7. Thereby, the upper limit comparison code amount is calculated.

  When the macroblock Mb20 is the encoding target MB, the calculation targets of the upper limit comparison code amount are the macroblocks Ma15 to Ma20 and the macroblocks Mb14 to Mb19. When the macroblock Ma33 is the encoding target MB, the calculation targets of the upper limit comparison code amount are the macroblocks Ma27 to Ma32 and Mb27 to Mb32. As described above, when an MB group is configured in units of macroblocks, the ratio between the macroblocks of the uncompressed image data 31a and the macroblocks of the uncompressed image data 31b, which are the targets for calculating the upper limit comparison code amount, is 1: 1. It becomes. When the MB group is configured in units of MB lines, the ratio of the macroblock of the non-compressed image data 31a and the macroblock of the non-compressed image data 31b, which is the calculation target of the upper limit comparison code amount, similarly to the line generation code amount Changes according to the position of the encoding target MB.

  The quantization parameter determination unit 23 determines whether or not the encoding process of the uncompressed image data 31a and 31b satisfies the exception condition based on the upper limit comparison code amount and the upper limit value (step S2). A method for calculating the upper limit value will be described. The quantization parameter determination unit 23 calculates, as an allowable code amount, a value obtained by multiplying the MB target code amount by the number of macroblocks (12) for which the upper limit comparison code amount is to be calculated. A value obtained by multiplying the allowable code amount by a predetermined coefficient is set in the quantization parameter determination unit 23 as an upper limit value. For example, 0.98 is set as the value of the predetermined coefficient.

  When the upper limit comparison code amount is equal to or less than the upper limit value, the quantization parameter determination unit 23 determines that the encoding process does not satisfy the exception condition (No in step S2). In this case, the quantization parameter QP is determined by executing steps S3 to S7 described above.

  When the upper limit comparison code amount is larger than the upper limit value, the quantization parameter determination unit 23 determines that the encoding process satisfies the exception condition (Yes in Step S2), and executes the exception process (Step S10). This is because the bit rate of the multiplexed compressed data 33 may exceed the target bit rate TR.

  In the process of step S2, the number of encoded macroblocks encoded immediately before the encoding target MB may be less than twelve. In this case, the determination as to whether or not the exception condition is satisfied (step S2) is performed by comparing the upper limit comparison code amount calculated by the following calculation method with the allowable generated code amount. If the number of encoded macroblocks is four, the upper limit comparison code amount is the sum of the generated code amounts of the four encoded macroblocks. The allowable generated code amount is a value obtained by multiplying the MB target code amount by the number of encoded macroblocks (four). Further, when there is no encoded macroblock (when the macroblock Ma0 is the encoding target MB), the process of step S2 is omitted. This is because the upper limit comparison code amount cannot be calculated.

  The exception process (step S10) will be described in detail. As described above, the encoding target MB is the macroblock Ma8. For this reason, the quantization parameter determination unit 23 obtains the quantization parameter QP of the macroblock Ma7 encoded immediately before the macroblock Ma8 among the encoded macroblocks of the uncompressed image data 31a. The quantization parameter determination unit sets the obtained quantization parameter QP of the macroblock Ma7 to the initial value (forced initial value) of the macroblock Ma8. The quantization parameter determination unit 23 determines a value obtained by adding a preset forced correction value to the forced initial value as the quantization parameter QP of the macroblock Ma8.

  As described above, when the encoding process satisfies the exceptional condition, the quantization parameter determination unit 23 identifies the macroblock encoded immediately before among the macroblocks of the uncompressed image data to which the encoding target MB belongs. To do. A value larger than the quantization parameter QP of the identified macroblock is determined as the quantization parameter QP of the encoding target MB. Thereby, since the amount of codes generated by encoding the encoding target MB can be suppressed, it is possible to prevent the bit rate of the multiplexed compressed data 33 from exceeding the target bit rate.

{Modification 1}
In the above-described embodiment, an example in which two pieces of image data (uncompressed image data 31a and 31b) are input to the image encoding device 2 has been described. However, the number of uncompressed image data input to the image encoding device 2 may be three or more. In this case, an MB group is generated for each input uncompressed image data. The order determination unit 21 determines the encoding order so that the MB groups of the respective uncompressed image data are encoded in order.

  Specifically, the image encoding device 2 inputs first, second,..., Nth image data. The encoding unit 22 encodes the kth macroblock based on the order information 41 indicating the encoding order of the kth macroblock included in the kth image data. N is a natural number of 1 or more. k is a natural number from 1 to N. The generated code amount accumulating unit 231 calculates the six generated code immediately before the encoding target MB when calculating the line generated code amount used for determining the correction value V1 of the encoding target MB (kth macroblock). The generated code amount of the encoded macroblock is integrated to calculate the line generated code amount. The line generation code amount calculation target includes encoded macroblocks of other image data excluding the k-th image data. The quantization parameter QP of the encoding target MB (kth macroblock) is corrected according to the generated code amount of the six macroblocks encoded immediately before. Therefore, the image encoding device 2 can stably control the bit rate of the multiplexed compressed data 33.

{Modification 2}
In the above embodiment, an example in which one encoding unit 22 encodes the macroblocks of the uncompressed image data 31a and 31b has been described. However, the image encoding device 2 may include two encoding units that encode the uncompressed image data 31a and 31b, respectively.

  FIG. 14 is a block diagram illustrating a functional configuration of the image encoding device 2 according to the second modification of the present embodiment. The image encoding device 2 illustrated in FIG. 14 includes an encoding unit 22a and an encoding unit 22b instead of the encoding unit 22. The encoding units 22a and 22b encode the macroblocks of the uncompressed image data 31a and 31b, respectively, to generate encoded data 32a and 32b.

  The image encoding device 2 newly includes an output control unit 26 in addition to the encoding units 22a and 22b. Based on the order information 41, the output control unit 26 controls the timing at which the encoding unit 22a outputs the encoded data 32a and the timing at which the encoding unit 22b outputs the encoded data 22b. The multiplexing unit 24 generates the multiplexed compressed data 33 by arranging the encoded data 32a and 32b in the order input from the encoding units 22a and 22b. Since the encoding of the uncompressed image data 31a and 31b is performed in parallel, the delay associated with the encoding of the uncompressed image data 31a and 31b can be shortened.

  The encoding units 22a and 22b may each encode two macro blocks of uncompressed image data, similarly to the encoding unit 22 (see FIG. 2). In this case, the image encoding device 2 shown in FIG. 14 can generate the multiplexed compressed data 33 from the four uncompressed image data.

  In the above-described embodiment, an example in which uncompressed image data 31a and 31b with synchronized frames is input from the cameras 1a and 1b to the image encoding device 2 has been described. However, the image encoding device 2 may generate uncompressed image data 31a and 31b in which frames are synchronized inside the device itself.

1a, 1b Camera 2 Image encoding device 4a, 4b Monitoring device 21 Order determining unit 22 Encoding unit 23 Quantization parameter determining unit 24 Multiplexing unit 25 Communication unit 26 Output control unit 231 Generated code amount integrating unit 232 Initial value determining unit 233 Correction value determination unit

Claims (8)

An image encoding device that encodes first to Nth (N is a natural number of 1 or more) image data,
an encoding unit for encoding the k-th macroblock based on order information indicating the encoding order of the k-th macroblock included in the k-th image data when k is a natural number from 1 to N; ,
A quantization parameter determining unit that determines a quantization parameter of a macroblock to be encoded;
With
The quantization parameter determination unit
A first number of macroblocks encoded immediately before the encoding target macroblock are identified from the group of encoded macroblocks encoded by the encoding unit, and the first number of macroblocks is specified. A code amount integration unit for calculating an integrated value of the generated code amount for each block;
A correction value determination for determining a correction value for the quantization parameter based on the integrated value and a target code amount per the first number of macroblocks and correcting the quantization parameter based on the correction value And
An image encoding device comprising: The image encoding device according to claim 1,
The quantization parameter determination unit identifies a second number of kth macroblocks encoded immediately before the encoding target macroblock when the encoding target macroblock is included in the kth image data. , An initial value determination unit that calculates an average value of quantization parameters for each of the second number k-th macroblocks as an initial value;
Including
The quantization parameter determination unit is an image encoding device that determines the quantization parameter based on the initial value and the correction value. In the image coding device according to claim 1 or 2,
When the encoding target macroblock is included in the k-th image data, the correction value determining unit is configured to generate a code amount of one k-th macroblock encoded immediately before the encoding target macroblock. An image encoding device for determining the correction value. In the image coding device according to claim 1 or 2,
The correction value determination unit is an image encoding device that determines the correction value based on an activity indicating the complexity of the encoding target macroblock. The image encoding device according to claim 4, wherein
The correction value determination unit calculates activities in four regions each including an upper side, a lower side, a left side, and a right side of the encoding target macroblock, and the correction value is determined based on a minimum activity among the activities in each region. An image encoding device for determining The image encoding device according to any one of claims 1 to 5,
The image encoding apparatus according to claim 1, wherein the first number is the number of macroblocks constituting a line of macroblocks arranged in a line in a horizontal direction. The image encoding device according to any one of claims 1 to 6,
The code amount integrating unit identifies a third number of macroblocks encoded immediately before the encoding target macroblock from the encoded macroblock group, and the third number of macroblocks The amount of generated code for each is integrated to calculate the upper limit comparison code amount,
When the encoding target macroblock is included in the k-th image data and the upper limit comparison code amount is larger than a preset upper limit value, the quantization parameter determination unit immediately precedes the encoding target macroblock. A quantization parameter of one encoded k-th macroblock is determined as a forced initial value;
The correction value determination unit is an image encoding device that corrects the forced initial value using a preset forced correction value. The image encoding device according to any one of claims 1 to 7,
The encoding unit includes:
First to Nth individual encoding units for encoding the first to Nth image data respectively;
An output control unit that controls the output timing of the k-th encoded data generated by the k-th encoding unit based on the order information;
An image encoding device including:

Images