JP2005101720A - Partial image coding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a partial image coding device capable of reducing a processing load of coding processing when segmenting a partial image from a base image, coding the images and transmitting them. <P>SOLUTION: A base image coding preprocessor 22 divides a base image into blocks, obtains preprocessing data including an optimum motion vector and an orthogonal converting coefficient for each block according to rules described in a reference block description table, and stores the obtained data. A partial image position input 23 inputs the initial position and movement speed of the partial image segmented from the base image. A partial original image generator 24 determines the size and position of the partial image. In addition, this position is determined so as to be spaced from the reference position of the base image per block or macroblock. A partial image coder 25 obtains the preprocessing data concerning the partial image from the preprocessor 22, applies quantization and variable length processing to the obtained data, and delivers the processed data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a partial image encoding apparatus, and more particularly to a partial image encoding apparatus that can encode a partial image or a cut-out image from a base image with a low load.

  2. Description of the Related Art Conventionally, a system that cuts out a partial image from a base image such as camera photography and performs transmission or the like has been proposed as described in, for example, JP-A-9-298469.

In the system described in this publication, a part of an image photographed by a television camera is cut out by designating it from the monitor (monitor) unit or CPU of the image, and the cut out image is MPEG / H261 format. The technology of compressing and transmitting to the monitoring unit, etc., and when compressing, compare the neighboring image with the current image in the time axis direction, and transmit only the difference of the image information, thereby reducing the amount of data after compression It is disclosed to reduce.
JP-A-9-298469

  However, the background art described above has the following problems. That is, when a plurality of partial images, particularly partial images whose positions move, are cut out from the base image and transmitted, the encoding process is performed for each cut-out image. There was a problem that time would become large.

  The present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to cut out a partial image or a partial image whose position and size change with time from a base image, and encode and transmit the extracted partial image. In such a case, the present invention is to provide a partial image encoding device that can reduce the processing load of the encoding process.

  In order to achieve the above-described object, the present invention provides a partial image coding apparatus that cuts out and encodes a partial image having a smaller shape from a base image having a predetermined shape, and a reference block for each block of the base image. Pre-processing means for obtaining and storing pre-processed data including an optimal motion vector and orthogonal transformation coefficient according to a predetermined rule for the pre-processing, and the pre-processing corresponding to the block included in the partial image cut out from the base image A first feature is that it includes a partial image encoding means for acquiring and encoding data.

  In addition, the present invention has a second feature in that the reference position of the partial image is separated in units of blocks or macroblocks from the reference position of the base image.

  Furthermore, the present invention has a third feature in that the position or the size of the partial image cut out from the base image can be changed with time.

  According to the first feature, since the partial image encoding means can encode the partial image using the preprocess data, when performing a plurality of partial image encoding, the partial image encoding means The processing load can be reduced and the processing speed can be greatly improved.

  According to the second feature, the preprocessed data can be used for encoding regardless of the cut-out position of the partial image. In other words, the partial image and the preprocess data can be matched.

  According to the feature 3, panning, tilting, zooming, and the like of a partial image to be encoded can be executed.

  Further, according to the present invention, for each block of the base image, preprocessing data is obtained according to a rule including no restriction of a reference block and restrictions on an upper, lower, left, or right block sequence. Since the processing data is used for encoding, the encoding efficiency is improved.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 8 is a conceptual diagram of an omnidirectional camera, and FIG. 9 shows an example of a screen in which image data captured by the omnidirectional camera is displayed on a display such as a personal computer.

  Although the omnidirectional camera is known and has various types, the omnidirectional camera 1 of FIG. 8 has a hemispherical or bowl-shaped omnidirectional mirror provided on the upper surface inside the transparent acrylic cylinder 2. 3, a center needle 4 extending downward from the center of the omnidirectional mirror 3, a CCD camera 5 and a cable connector 6 provided below the center needle 4. Since this camera is well known, detailed description is omitted.

  When the omnidirectional photographing camera 1 is connected to a personal computer or the like via the cable connector 6 and an image photographed by the omnidirectional photographing camera 1 is displayed on a display, a screen 11 as shown in FIG. 9 is obtained. On the screen 11, a donut-shaped 360 ° omnidirectional image 11a that is a captured image of the omnidirectional camera 1 itself, an image 11b obtained by developing the donut-shaped 360 ° omnidirectional image 11a in a belt shape, that is, a panoramic developed image, A normal one-frame-sized image 11c obtained by cutting out a part of the image 11b is displayed. 11d is a cut-out image of the image 11c. The image 11c can be distributed to, for example, a mobile phone.

  The image obtained from the omnidirectional camera 1 is a moving image of 15 frames / second, for example, in the same manner as a normal PC camera. Since the developed image 11b is many times larger than a normal one frame size such as a PC camera image, for example, when distributing to a mobile phone or the like, a partial image whose position is moved may be cut out from the base image, encoded, and transmitted. Done. When a plurality of different partial images are generated at the same time, if the conventional technique is used, the processing load of the encoding process increases in proportion to the number of partial images.

  The present invention has been made to solve such problems, and an embodiment thereof will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of an embodiment of the present invention.

  From the base image input unit 21, the developed image (hereinafter referred to as a base image) 11b is input. The base image encoding pre-processing unit 22 performs pre-encoding processing on all small regions, for example, blocks (8 × 8 pixels) or macroblocks (16 × 16 pixels) of the expanded image 11b. For example, the motion estimation (ME) / motion compensation (MC) + orthogonal transformation (eg, DCT) processing in the case of MPEG is performed using the correlation with the adjacent block, or the use of the correlation with the adjacent block is restricted. Pre-processing is performed. Data obtained as a result of the preprocessing is stored in a storage device (not shown) in the base image encoding preprocessing unit 22. The details will be described later with reference to FIG.

  The partial image position input unit 23 inputs an initial position of a partial image cut out from the developed image 11b, a speed of movement of the partial image, and the like. Note that the initial position of the partial image is determined at a position separated from the reference position of the base image in units of blocks or macroblocks. The partial original image generation unit 24 determines the size of the partial image and the position of the partial image. The partial original image is cut out based on the input signal from the partial image position input unit 23 and output. In this case, as will be apparent from the following description, the reference position of the partial image is cut out so as to be a position separated from the reference position of the base image in units of blocks or macroblocks.

  The partial image encoding unit 25 encodes the partial original image input from the partial original image generation unit 24 using the data created and stored by the base image encoding preprocessing unit 22. In this case, the peripheral part of the partial original image uses data obtained by processing that restricts the use of correlation with adjacent blocks, and the partial original image contains data obtained by processing using correlation with adjacent blocks. Is used.

  FIG. 2 is a block diagram showing the detailed functions of the base image coding preprocessing unit 22, which is composed of an image block dividing unit 22a, an ME / MC and DCT unit 22b, a reference block description table 22c, and a memory 22d. Has been.

As shown in FIG. 3, the image block division unit 22 a converts the developed image 11 b that is a base image into blocks b ₁₁ , b ₂₁ , b _31, ... Having 8 × 8 pixels or 16 × 16 pixels, for example. , B _{nn ′} . The ME / MC and DCT unit 22b obtains an optimal motion vector for each rule in accordance with the reference block selection rule described in the reference block description table 22c, and obtains a DCT coefficient corresponding thereto. The motion vector and DCT coefficient are stored in the memory 22d. The loop S1 is circulated by the number of reference block descriptions.

  In the reference block description table 22c, for example, 0: no reference block restriction, 1: no reference to the left block row, 2: no reference to the upper block row, 3: right Does not refer to block sequence, 4: Does not refer to lower block sequence, 5: Refers to only 1 block sequence on the left, 6: Refers to only 1 block sequence, 7: Refers to only 1 block sequence on the right 8: Below, a rule is described in which only one block row is referred to below. Further, another rule may be described in this table, or a description to refer to the block of the previous frame may be added.

Therefore, the ME / MC and DCT unit 22b corresponds to the optimal motion vector MV for each of the blocks b ₁₁ , b ₂₁ , b ₃₁ ,..., B _{nn ′} of the base image and for each rule. DCTs are obtained and stored in the memory 22d.

Now, assuming that the blocks b ₁₁ , b ₂₁ , b _31, ..., B _{nn ′} are 16 × 16 pixel macro blocks, each of the rules 0 to 8 and combinations thereof are provided for each macro block. For the rule (for example, rule 1 + 2), optimum motion vectors MV _{0 to} MV _k (k ≧ 8) are obtained. As for DCT, four DCTs (DCT1 to DCT4) related to luminance and two DCTs (DCT5 to 6) related to chromaticity are obtained for four 8 × 8 pixels. These motion vectors MV and DCT (preprocessing data group) are stored in the memory 22d.

  As described above, the base image encoding preprocessing unit 22 outputs the base image and the preprocess data group.

  Next, details of the function of the partial original image generation unit 24 will be described with reference to FIG. The partial original image generation unit 24 includes a partial image size / position determination unit 24a. The partial image size / position determination unit 24a receives an initial position of the partial image from the partial image position input unit 23, a movement speed of the partial image, and the like. And the size and position of the partial image are determined. The loop S2 is circulated by the number of cut out partial images. With this function, panning, tilting, zooming, and the like of the frame to be encoded are executed. That is, changing the position of the partial image results in pan, tilt, etc., and changing the position and size of the partial image results in zoom.

  In FIG. 6, 31 is an example of pan, 32 is a tilt, and 33 is an example of zoom. In the figure, (X0, Y0) indicates the reference position of the base image 11b, and the partial image size is a rectangle of x1 × y1. In the pan 31, the tilt 32, and the zoom 33, the initial positions P1, P11, and P21 of the first cut-out partial image are 8 pixels or 16 pixels in the X and Y directions from the reference position (X0, Y0) of the development screen 11b, respectively. It is set at a position separated by the unit of.

  Next, when panning 31 is performed in the direction of arrow a, the reference positions P1, P2, P3,... Of each screen are set to the partial image position input unit 23 in the direction of arrow a in units of 8 pixels or 16 pixels. Shifted at the speed (pan speed) specified in. When the tilt 32 is performed in the arrow b direction, the reference positions P11, P12, P13,... Of each screen are shifted in the arrow b direction in units of 8 pixels or 16 pixels in both the X and Y directions. . Further, when zoom 33 is performed, the screen size changes as x1 × y1, x2 × y2, x3 × y3,... And the reference positions P21, P22, P23,. Both the X and Y directions are shifted in the direction of arrow c in units of 8 pixels or 16 pixels. As described above, the reference position of each screen is shifted in units of 8 pixels or 16 pixels in both the X and Y directions when the partial image encoding unit 25 encodes the base image encoding preprocessing unit 22. This is to make it possible to use the preprocessed data obtained in step 1 without any processing.

When the partial image size and position are determined by the partial image size / position determination unit 24a as described above, each partial image such as the pan 31, tilt 32, or zoom 33 is determined by the partial image encoding unit 25. Is encoded. In the encoding process, the rule that does not refer to the left and upper block rows of the rule 1 + 2 is applied to the peripheral block of each partial image, for example, the block b _{00 in} the upper left corner of the partial image 40 in FIG. In this case, preprocessed data (motion vectors MV and DCT) are read from the memory 22d and applied. In addition, for each of the upper blocks b _{01 to} b _{0, n−1} , preprocessed data in the case where the above-described rule 2 is not referred to is used. For the block b _{0n in the} upper right corner, pre-processing data in the case of applying the above rule 2 + 3 and not referring to the right block column is used. Also, preprocessing data when the rule 1 is applied to the left peripheral blocks b _{10 to} b _m−1,0 , and the rule 1 + 4 is applied to the block b _{m0 at the} lower left corner The pre-processing data is used. Further, for the blocks b _{m ′ and n ′} in the central part, the preprocess data in the case where the restriction of the reference block of Rule 0 is applied is used. In addition, preprocessing data when a block of the previous frame is referred to may be used.

  As described above, in the partial image (cutout image), the peripheral block uses a mode that restricts the use of the inter-adjacent correlation of the base image or a unique processing mode, and the other blocks use the inter-adjacent correlation of the base image. Use the mode that uses.

  The partial image encoding unit 25 then quantizes (Q) the preprocessed data read from the memory 22d by a known method, and then performs variable length encoding (VLC) and outputs the result. The output data can be delivered to a mobile phone or the Internet.

It is a block diagram which shows the structure of the outline of this invention. It is a block diagram which shows the detail of the base image encoding pre-processing part of FIG. It is explanatory drawing of the block division of a base image. It is explanatory drawing of a reference block description table. It is a block diagram which shows the detail of the partial original image generation part of FIG. It is a figure which shows the example of the partial image from which a position or a size changes. It is explanatory drawing of operation | movement of the partial image encoding part of FIG. It is explanatory drawing of a structure of the omnidirectional camera. It is a figure which shows the example of a display when the image image | photographed with the omnidirectional camera is displayed on a display.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 21 ... Base image input part, 22 ... Base image encoding pre-processing part, 23 ... Partial image position input part, 24 ... Partial original image generation part, 25 ... Partial image encoding part .

Claims (6)

In a partial image encoding device that cuts out and encodes a partial image having a smaller shape from a base image having a predetermined shape,
For each block of the base image, preprocessing means for obtaining and storing preprocessing data including an optimal motion vector and an orthogonal transformation coefficient according to a predetermined rule regarding a reference block;
A partial image encoding apparatus comprising: a partial image encoding unit that acquires and encodes the preprocess data corresponding to a block included in a partial image cut out from the base image. The partial image encoding device according to claim 1,
The partial image encoding apparatus, wherein the reference position of the partial image is separated in units of blocks or macroblocks from the reference position of the base image. The partial image encoding device according to claim 1 or 2,
The partial image encoding apparatus characterized in that the position or size of a partial image cut out from the base image can be changed with time. The partial image encoding device according to any one of claims 1 to 3,
The partial image coding apparatus, wherein the rule includes no restriction of a reference block and restriction on an upper, lower, left, or right block sequence. The partial image encoding device according to claim 4,
The partial image encoding means acquires pre-processed data related to the restriction for the peripheral blocks of the partial image, and acquires pre-processed data related to the non-restricted for other blocks. The partial image coding apparatus characterized by the above-mentioned. The partial image encoding device according to claim 1,
2. The partial image encoding apparatus according to claim 1, wherein the base image is an image obtained by developing a 360 ° omnidirectional image in a band shape.

Images