JP2010166489A - Image compression/expansion apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that, in the conventional configuration, even when any event occurs and a monitor video image is modified, it is difficult for a receiving side to confirm detection of the change before a compressed encoded stream is decoded. <P>SOLUTION: In an image compression/expansion apparatus which generates a reference predictive image by adaptively switching an intra-prediction section and a motion compensation prediction section, an identification flag is accumulated which indicates which prediction section is used to generate the reference prediction image to be generated within a predetermined unit time and increase/decrease of the identification flag is transmitted while being added to a header of an encoded stream as prediction selection information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image compression / decompression apparatus, and more particularly to an image compression / decompression apparatus that adaptively switches between an intra prediction unit and a motion compensation prediction unit to generate a reference prediction image.

  In the video monitoring field, systems that remotely monitor monitoring video for 24 hours regardless of daytime or nighttime are widespread.

  For example, when monitoring video is sent from a monitoring camera installed at an intersection or traffic sparsely located to a monitoring center, the monitoring video is a relay transmission device such as an FPU (Field Pickup Unit). Alternatively, it is transmitted to the center using a transmission line such as a network.

  In such a relay transmission device or a transmission path of a network or the like, the bandwidth of the transmission path that can be used in the channel to be transmitted is limited. In general, the data is compressed to a transmission bit rate allowed by the

  MPEG-2 and H.264 are used to compress these surveillance videos. The H.264 encoding method is employed.

  MPEG-2 and H.264 In the H.264 encoding method, a reference prediction image is generated using an intra prediction unit and a motion compensation prediction unit at the time of encoding. Intra prediction is a prediction method that generates a predicted image signal based on adjacent pixels on the upper and left sides of a macroblock that does not use inter-frame prediction, and encodes the difference from the predicted image signal. is there.

  Also, motion compensated prediction is a method of generating a necessary predicted image signal by searching for frames going back and forth in the time axis direction, and encoding a difference between the input image and the predicted image signal.

  A picture coded by intra prediction is an I picture, a picture coded using motion compensated prediction is a P picture, a B picture (a picture coded based on reference prediction pictures before and after in the time axis direction is a B picture, The other encoding method is called a P picture) and has an encoded picture structure called IPB. The encoding method is suitable for moving images.

  Switching control between the intra prediction unit that determines the coded picture structure and the motion compensated prediction unit periodically selects the intra prediction unit so that intra pictures are inserted at a constant cycle based on the set bit rate. Or a motion vector based on the input image, and a long prediction interval (motion compensation prediction unit) that uses the previous and next frames preferentially in a scene with little motion, while a frame in a scene with heavy motion Among them, a method of preferentially selecting a short prediction interval (intra prediction unit) and efficiently changing the coded picture structure according to the scene of the input image is employed.

  In decoding on the reception side, when generating a reference prediction image, header analysis is performed on the received encoded stream so that the selected prediction unit matches the transmission side, and the intra prediction unit and motion compensation prediction are performed. The part is switched adaptively.

  As described above, MPEG-2 and H.264 are used. In a remote monitoring system using the H.264 encoding method, in order to detect image changes during monitoring, decoding processing is performed once on the receiving side. When monitoring video is stored, data is checked after visual confirmation. In order to reduce the amount, the compressed stream is compressed again and stored in the data server.

JP 2002-26273 A

  For example, as an example of such a surveillance camera system, Patent Document 1 discloses a detection unit that detects an intruder or an entering object in a data compression unit that compresses a video signal in a digital surveillance system using digital data compression. And variable data compression means that can change the frame rate and picture configuration at the time of data compression of the video signal, and control means that changes the encoding format of the variable data compression means according to the signal of the detection means, When no intruder or intruder is detected by the detecting means, the data compression frame rate of the variable data compressing means is lowered, and the data compression is controlled so as to perform the data compression by the picture configuration optimum for the frame rate, and the variable data compression The compressed video data output by the means is supplied to the recording means for recording. Digital monitoring system constructed in so that is proposed in Patent Document 1.

  In the configuration of the conventional example described above, even if some event occurs and the monitoring video changes, in order to detect it on the receiving side, it is necessary to confirm that the compressed encoded stream is not decoded. There was a problem that was difficult. Also in Patent Document 1, when reproducing a video from a recording unit that records compressed data, it is difficult to confirm without decoding, and it cannot be confirmed immediately on the receiving side.

  Accordingly, an object of the present invention is to decode a received encoded stream in a system for remotely monitoring a monitoring video using an encoding method in which a reference prediction image is generated by adaptively switching between an intra prediction unit and a motion compensation prediction unit. Provides an image compression / decompression device that can detect whether or not there is a change (motion) in the time axis direction of a monitoring image without processing, and if the change is recognized, an encoded stream to be received thereafter can be recorded as an alarm image There is to do.

  In order to achieve the above object, an image compression / decompression apparatus according to the present invention is an image compression / decompression apparatus that generates a reference prediction image by adaptively switching between an intra prediction unit and a motion compensation prediction unit. The identification flag indicating which prediction unit is used to generate the reference prediction image is accumulated, and the increase / decrease is added to the header of the encoded stream as prediction selection information and transmitted. This is a feature and has the following configuration.

  In the image compression / decompression apparatus according to the present invention, an encoding unit on a transmission side includes a macroblock division unit that divides video data into macroblocks, and a difference image between the video data divided into the macroblocks and a predicted image signal. An orthogonal transform unit that performs orthogonal transform, an encoding unit that encodes the difference image that has undergone orthogonal transform, an inverse orthogonal transform unit that performs inverse orthogonal transform on the difference image that has undergone orthogonal transform, and an output of the inverse orthogonal transform unit. An adder that adds the predicted image signal to generate a decoded image; and an intra prediction unit that generates the predicted image signal from the output of the adder. A prediction mode selection unit that generates a prediction image signal of the image and selects a predetermined prediction image signal based on the video data divided into the macroblocks, and the decoded image data It is configured to have a switching unit that controls whether to supply to any of the pixel block dividing unit of the plurality of pixel block division unit.

  As shown in FIG. 1, the encoding unit 100 includes a motion compensation prediction unit 13 and a motion detection unit 16, and the motion detection unit 16 is divided into the macroblocks as shown in FIG. Movement for searching for a predetermined prediction image signal by comparing the pixel block divided into the plurality of pixel blocks and the decoded image data with a plurality of pixel block division units 131 to 137 for dividing data into a plurality of pixel blocks A vector search unit 138 and a motion optimal prediction image signal selection unit 139 that selects an output of the motion vector search unit are included. The motion compensation prediction unit 13 is configured to output the decoded image based on the output of the prediction image signal selection unit. The predetermined prediction image signal is output from the data.

  Further, the encoding unit 100 accumulates an identification flag indicating which prediction unit is used to generate the reference prediction image output from the prediction mode selection unit 106 within a predetermined unit time, and increases or decreases the identification flag. The prediction mode accumulating unit 104 outputs the information as prediction selection information, and is configured to output the prediction selection information to the variable length encoding unit 7 having a function of transmitting the prediction selection information by adding it to the header of the compressed encoded stream. .

  On the other hand, the decoding unit on the reception side of the image compression / decompression apparatus according to the present invention is buffered by the reception buffer 20 for buffering the compressed encoded stream received at the input terminal 120 and the reception buffer as shown in FIG. A header extraction analysis unit 105 that extracts and analyzes a stream header from the compressed encoded stream, a variable length decoding unit 22 that performs a decoding process on the encoded stream from which the header is removed, and the variable length decoding process. Output from the inverse quantization unit 8 that inversely quantizes the prediction difference image signal PΔV1, the inverse orthogonal transform unit 9 that performs inverse orthogonal transform on the result, and the output of the inverse orthogonal transform unit 9 and a prediction mode selection unit described later. An addition unit 10 that adds the predicted image signals to generate a decoded image signal, and an intra prediction unit 14 that generates the prediction image signal from the output of the addition unit, The decoded image signal in units of macroblocks filtered by the deblocking filter unit is buffered in the reference picture buffer of the storage unit, and a motion vector is searched for each predetermined block size defined by the standard, and the predicted image It has a motion compensated prediction unit for generating. The switching between the intra prediction unit 14 and the motion compensation prediction unit 13 is performed according to the prediction mode output from the header extraction analysis unit 105.

  Further, in addition to the above-described configuration, the reference prediction image generated using the intra prediction unit and the motion compensation among the reference prediction images generated by switching the prediction mode selection unit 15 within a predetermined unit time are generated. For example, the prediction mode information storage unit 106 that calculates the generation ratio ratio of the reference prediction image has a ratio of the generation reference prediction image generated by the intra prediction section 14 rapidly increasing. If exceeded, an alarm signal is issued from the prediction mode information storage unit to the alarm detection unit, and the alarm detection unit receiving the alarm signal records an alarm image with an encoded image received thereafter as an alarm image. The unit 25 is configured to start recording.

  According to the present invention, it is possible to immediately detect a change in the monitoring image on the input side on the receiving side while maintaining the compressed and encoded stream.

H in the present invention. The block diagram of the Example of a H.264 encoding part. H. In the compression / decompression apparatus of the present invention. The block diagram for demonstrating a H.264 decoding part. Conventional H.264. The block diagram for demonstrating a H.264 encoding part. Conventional H.264. The block diagram for demonstrating a H.264 encoding part. H. The block diagram of the motion detection part of a H.264 decoding system.

  Before describing the present invention, H.C. A conventional example of the H.264 encoding method will be described with reference to FIGS. 3 and FIG. It is a block diagram for demonstrating a H.264 encoding part.

  First, H. The H.264 encoding unit will be described. In FIG. 3, reference numeral 102 denotes an encoding unit, and 1 denotes an input terminal. A monitoring video imaged by a camera or the like is input to this input terminal, and video data of a luminance signal component Y and two types of color difference signal components Cb and Cr are separately input. The video data of the input luminance signal component Y and the two types of color difference signal components Cb and Cr is converted into an MB (macroblock) 16 × 16 pixel block luminance signal Y and an 8 × 8 pixel block color difference signal by the MB dividing unit 3. Cb and Cr are respectively converted into encoding target signals MBS in units of macroblocks. The video data of the luminance signal component Y and the two types of color difference signal components Cb and Cr are converted into an MB16 × 16 pixel block luminance signal Y and an encoding target signal MBS in units of macroblocks by the MB dividing unit 3. These encoding target signals MBS in units of macroblocks are subtracted from a prediction image signal PV (to be described later) by the subtraction unit 4, and the prediction difference image signal PΔV 1 is supplied to the orthogonal transformation unit 5.

  For the input video data, the 16 × 16 pixel block luminance signal Y and the 8 × 8 pixel block color difference signals Cb and Cr MBS in units of macroblocks are described in the following description in order to simplify the description. Only the luminance signal Y of the pixel block will be described as an encoding target.

  The orthogonal transform unit 5 converts horizontal and vertical image luminance data into horizontal and vertical frequency spectra, and the quantizing unit 6 quantizes the video data converted into the spatial frequency spectrum. The quantization unit 6 uses the characteristic that human visual characteristics are insensitive to high-frequency components, and changes the roughness of quantization between the high-frequency component side and the low-frequency component side to reduce the amount of data. The video data quantized by the quantization unit 6 is supplied to the variable length coding unit 7.

  The variable length encoding unit 7 encodes the data quantized by the quantization unit 6, the encoding target signal MBS in units of macroblocks can reduce the code amount, and the image data is compressed. The signal is transmitted from the output terminal 2 to the next-stage signal processing device.

  On the other hand, the video data converted into the spatial frequency spectrum from the quantization unit 6 is subjected to inverse quantization processing on the transform coefficients of the luminance signal Y and the color difference signals Cb and Cr by the inverse quantization unit 8, and further the inverse of the subsequent stage. The orthogonal prediction unit 9 restores the reproduction prediction difference signal PΔV2 and outputs it to the addition unit 10. In the addition unit 10, the reproduction prediction signal PΔV 2 and a prediction image signal PV described later are added by the addition unit 10 to reproduce the decoded image signal RV and supplied to the storage unit 12 via the deblocking filter unit 11. . The decoded image signal RV is supplied to the subtraction unit 4 via the intra prediction unit 14 and the prediction mode selection unit 15, and the subtraction unit 4 outputs the above-described prediction difference image signal PΔV1.

  Next, the motion compensation prediction unit 13 and the motion detection unit 16 will be described. The decoded image signal RV from the adding unit 10 is stored in the storage unit 12 by the deblocking filter unit 11 that removes block distortion in order to perform motion compensation. The decoded image signal RV stored in the storage unit 12 is supplied to the motion compensation prediction unit 13 and the motion detection unit 16 as a reference image.

  The motion detection unit 16 divides the encoding target signal MBS into 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4 pixel blocks to be subjected to motion search. The search is performed for each block. FIG. 5 is a block diagram illustrating a configuration example of the motion detection unit 16. 130 is an input terminal, 140 is an output terminal, 131 is a 16 × 16 block dividing unit, 132 is a 16 × 8 block dividing unit, 133 is an 8 × 16 block dividing unit, 134 is an 8 × 8 block dividing unit, and 135 is 8 × A 4-block dividing unit, 136 is a 4 × 8 block dividing unit, 137 is a 4 × 4 lock dividing unit, and 138 is a motion vector search unit. For example, the 16 × 16 encoding target signal MBS is divided into four blocks in the 8 × 8 pixel block unit 134, and the four blocks are compared with the reference image to search for a matching block or a similar block. . For example, if the search range is a 16 × 16 pixel range centered on the 16 × 16 encoding target signal MBS, the block search range of 8 × 8 pixels is sequentially moved in pixel units. Thus, the matching block or the most similar block is searched. That is, the difference value between the luminance value of each pixel of the divided 8 × 8 pixel block and the luminance value of each pixel in the 8 × 8 pixel block in the search range is calculated, and each pixel of the 8 × 8 block is calculated. Select the one with the smallest difference value. In this way, the search is performed for all four blocks. In the above-described method, the motion vector search unit 138 calculates the minimum luminance difference value for each pixel block in each of the pixel block division units 131 to 137. Part 139. That is, the seven blocks respectively searched by the motion vector search unit 138 are supplied to the motion optimal image selection unit 139, and the motion optimal image selection unit 139 selects the divided block having the smallest difference value from these seven blocks. Are selected as the optimal prediction image signals.

  The output of the motion optimal image selection unit 139 is supplied to the motion compensation prediction unit 13 and the prediction mode selection unit 15. In the motion compensation prediction unit 13, an optimal prediction image signal is selected based on the detection result of the motion optimal image selection unit 139, and is supplied to the subtraction unit 4 and the addition unit 10 via the prediction mode selection unit 15.

  In the prediction mode selection unit 15, if the difference value in the motion vector search unit 138 of the motion detection unit 16 is larger than the threshold set experimentally in advance, as described above, the target code within the search range. It is determined that there is no block that matches or is similar to the conversion target signal MBS, and in this case, no output signal is output from the motion detection unit 16. That is, when an image having no correlation between frames is input as a surveillance video due to intrusion of a suspicious object or the like, the intra prediction unit 14 tends to be selected. In this case, the output of the intra prediction unit 14 described above is predicted. It is selected by the mode selection unit 15 and supplied to the subtraction unit 4 and the addition unit 10.

  In addition, there is no suspicious object intrusion, there is little movement including the background, or even if a partial image is input as a surveillance video, the motion compensation unit 15 tends to be easily selected. In this case, the motion vector search unit 138 of the motion detection unit 16 selects the divided block having the smallest difference value, the prediction mode selection unit 15 selects the output of the motion compensation prediction unit 13, and the subtraction unit 4 and the addition unit 10 is supplied. In the above description, only the luminance signal Y has been described, but the color difference signal is processed in the same manner as the luminance signal except when it is divided into 4 × 4 fixed pixels, and thus detailed description thereof is omitted. To do.

  Next, an embodiment of the encoding unit in the image compression / decompression apparatus according to the present invention will be described.

  FIG. It is a block diagram of the Example of a H.264 encoding part. In FIG. 1, reference numeral 100 indicates an encoding unit, 106 is a prediction mode selection unit, and 104 is a prediction mode accumulation unit. Blocks common to FIG. 3 are given the same reference numerals.

  Here, the basic principle of the present invention shown in FIG. 1 will be described. H. The operation of the prediction mode selection unit 15 in the H.264 encoding method is as described above with reference to FIG. 3, and the output of the prediction mode selection unit 15 is adaptively switched depending on the input image. If the difference value in the motion vector search unit 138 is experimentally set in advance and is larger than the threshold value, it is determined that there is no block that matches or is similar to the target encoding target signal MBS within the search range. The output of the intra prediction unit 14 is selected and supplied to the subtraction unit 4 and the addition unit 10. Conversely, when the motion vector search unit 138 of the motion detection unit 16 selects the divided block with the smallest difference value, the output of the motion compensation prediction unit 13 is selected, and the subtraction unit 4 and the addition unit 10 output the selected block. Supply.

  However, the standard determines that the output of the intra prediction unit must be selected periodically so that error propagation can be stopped and restored even if an error occurs due to transmission trouble such as video interruption. This operation is called refresh.

  Therefore, in the present invention, for example, when the monitoring subject is an entry-inhibited area that is normally prohibited from entering or leaving humans or living things, there is no intruding object, and there is no significant image change between the previous and next frames. Since the difference value in the motion vector search unit 138 of the motion detection unit 16 falls within the threshold set experimentally in advance, the mode selection unit 15 matches the target encoding target signal MBS within the search range. When it is determined that there is a block or a similar block, an output signal is output from the motion detection unit 16, and the output of the motion compensation prediction unit 13 is selected. The prediction mode selection unit 15 in the encoding device is a threshold that the difference value in the motion vector search unit 138 of the motion detection unit 16 is experimentally set in advance. It is determined that there is no block that matches or is similar to the target encoding target signal MBS within the search range, the output signal from the motion detection unit 16 is stopped, and the output of the intra prediction unit 14 is selected. In the image compression / decompression apparatus according to the present invention, the refresh rate is set to a predetermined unit time, and the intra prediction unit and the motion compensation prediction unit selected by the prediction mode selection unit 15 during this period are focused on. The generation ratio ratio of the predicted reference image generated by using the prediction mode accumulating unit 104 is calculated, and the ratio ratio of the generated reference image using the intra prediction unit is rapidly increased and decreased. If the ratio exceeds the ratio, it is determined that there has been some change in the monitored subject, and the prediction selection information is sent to the variable length coding unit 7 so that the user can use the code In addition to the user area of the stream header, to send the recipient the decryption unit.

  Next, referring to FIG. After describing the H.264 decoding unit 103 (conventional example), an embodiment of the receiving side decoding unit according to the present invention will be described with reference to FIG.

  Reference numeral 120 denotes an input terminal for receiving a compressed encoded stream on the receiving side. A reception buffer 20 for buffering the received compressed encoded stream, a header extraction analysis unit 21 for extracting and analyzing a stream header from the compressed encoded stream buffered by the reception buffer 20, and an encoding with the header removed A variable length decoding unit 22 that performs decoding processing on the stream, an inverse quantization unit 8 that performs inverse quantization on the prediction difference image signal subjected to the variable length decoding processing, and an inverse orthogonal transform unit that performs inverse orthogonal transformation on the result 9 and the output of the inverse orthogonal transform unit and the prediction image signal PV output from the prediction mode selection unit 15 (to be described later) are added to generate the decoded image signal RV. The above decoding in units of macroblocks filtered by the intra prediction unit 14 that generates the image signal PV and the deblocking filter unit 11 A motion detection unit 16 that buffers an image signal in a reference picture buffer of a storage unit and searches for a motion vector for each predetermined block size in the same manner as the motion detection unit 16 in the encoding unit, and a motion detection unit The motion compensation prediction unit 13 generates a predicted image according to the 16 outputs.

  The decoded image signal RV is also output to the scan conversion unit 23. After the macroblock is changed to a line scan, a decoded stream returned to a predetermined format is output from the output terminal 27 through the monitor output unit 24. . When a conventional remote monitor visually checks the decoded stream and finds any problem, the alarm detector 26 issues an alarm, records it in the alarm image recording unit, and stores the monitoring image. In FIG. 4, the same reference numerals are assigned to the same functions as those of the encoding unit 102 (FIG. 3) on the transmission side. Next, a block which is a feature of the decoding unit of the present invention will be described with reference to FIG. 2 which is an embodiment of the present invention.

  The corresponding blocks are the header extraction analysis unit 105, the prediction mode information storage unit 106, the alarm detection unit 26, and the alarm image recording unit 25. (The same reference numerals are assigned to the same components as those in FIG. 4) As described above, the compressed encoded stream sent from the transmission side is received at the input terminal 120 and then passed through the reception buffer 20, Input to the header extraction analysis unit 105. The header extraction analysis unit 105 extracts the prediction selection information added by the transmission-side variable length encoding unit 7 from the user area of the encoded stream header, performs analysis, and extracts the extracted prediction selection information as prediction mode information. The data is sent to the storage unit 106. The prediction mode information accumulating unit 106 accumulates the prediction selection information, determines whether or not the increase / decrease for each refresh interval exceeds a preset threshold level, and if the threshold level is exceeded, that is, the intra prediction unit 14 When the used generation reference image continues and occupies the majority, it is determined that some event has occurred in the monitoring monitor image on the transmission side, and an alarm signal is issued to the alarm detection unit 107.

When receiving an alarm signal, the alarm detection unit 107 is configured to output an alarm trigger signal for starting recording in the alarm image recording unit 108 using an encoded stream received thereafter as an alarm image according to the reception frequency. The alarm image recording unit 108 starts recording the encoded stream output from the header extraction analysis unit 105 without decoding in accordance with the alarm trigger signal.
As described above, in the encoding apparatus according to the present invention, the refresh reference is generated by using the intra prediction unit and the motion compensation prediction unit selected by the prediction mode selection unit 106 during the refresh rate as a predetermined unit time. If the ratio exceeds the closing ratio, it is determined that there has been some change in the monitored subject, the prediction selection information is sent to the variable length coding unit 7, and the user area of the coded stream header that can be used by the user And sent to the decoding unit on the receiving side.

  On the other hand, the decoding unit on the receiving side analyzes header information of the compressed stream, and sends the extracted prediction selection information to the prediction mode information storage unit 106. The prediction mode information accumulating unit 106 accumulates the prediction selection information and determines whether or not the increase or decrease of the refresh interval exceeds a preset threshold level. When the threshold level is exceeded, that is, the intra prediction unit 14 is used. When the generated reference image continues and occupies the majority, it is determined that some event has occurred in the monitoring monitor image on the transmission side, and an alarm signal is issued to the alarm detection unit 107.

The present invention can be used in an image compression / decompression apparatus.

DESCRIPTION OF SYMBOLS 1,120 ... Input terminal, 2 ... Output terminal, 3 ... MB division part, 4 ... Subtraction part, 5 ... Orthogonal transformation part, 6 ... Quantization part, 7 ... Variable length coding part, 8 ... Dequantization part, DESCRIPTION OF SYMBOLS 9 ... Inverse orthogonal transformation part, 10 ... Addition part, 11 ... Deblocking filter part, 12 ... Reference picture buffer, 13 ... Motion compensation prediction part, 14 ... Intra prediction part, 15, 106 ... Prediction mode selection part, 16 ... Motion Detection unit, 20 ... reception buffer, 22 ... variable length decoding unit, 23 ... scan conversion unit, 24 ... monitor output unit,
108 ... alarm image recording unit, 26 ... alarm detection unit, 104 ... prediction mode accumulation unit,
100, 102 ... Coding unit, 101, 103 ... Decoding unit, PV ... Predicted image signal,
PΔV1 ... prediction difference image signal, PΔV2 ... reproduction prediction difference signal,
RV: Decoded image signal, 105: Header extraction analysis unit.

Claims (2)

In an image compression / decompression apparatus that adaptively switches between an intra prediction unit and a motion compensation prediction unit to generate a reference prediction image, the reference prediction image generated within a predetermined unit time is generated using which prediction unit An image compression / decompression apparatus characterized by accumulating identification flags indicating whether or not the information has been added, and adding the increase / decrease of the identification flag to the header of the encoded stream as prediction selection information. In a remote monitoring system using an encoding method that adaptively switches between an intra prediction unit and a motion compensation prediction unit to generate a reference prediction image, a change in the time axis direction of the monitoring image without decoding the received encoded stream An image compression / decompression apparatus comprising means for detecting the presence or absence of the image.
Compression and decompression device.

Images