JP2008167267A - Stereo image encoding apparatus, stereo image encoding method and stereo image encoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a stereo image encoding apparatus, stereo image encoding method and stereo image encoding program in which motion compensation prediction encoding close to actual motion is achieved to improve precision in encoding a plurality of images. <P>SOLUTION: The stereo image encoding apparatus comprises: a grouping unit 4 for implementing grouping of regions within a screen in accordance with a parallax calculated by a parallax calculation unit 3 and for extracting characteristics of the grouped regions; and a motion prediction unit 7 for predicting motions of the respective regions grouped by the grouping unit 4, in consideration of characteristics extracted by the grouping unit 4, then an encoding unit 8 uses motion prediction results of the regions by the motion prediction unit 7 to encode images A, B. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a stereo image encoding device, a stereo image encoding method, and a stereo image encoding program for encoding a plurality of images (stereo images) taken at the same time.

A conventional stereo image encoding apparatus inputs a background image without parallax and a three-dimensional component image with parallax.
However, the background image without parallax and the 3D component image with parallax are both moving images, and the 3D component image is a gaze from a stereo image taken by arranging two cameras in parallel. Only the object is cut out, and the parallax of the image has only a horizontal component.

When the stereo image encoding device receives the background image and the three-dimensional component image, the stereo image encoding device encodes the background image and the three-dimensional component image.
The background image is encoded by performing motion compensation predictive encoding that performs block matching using a generally known block of 16 × 16 pixels or the like as an encoding unit.
In the encoding of the three-dimensional component image, the component image for the left eye is encoded by performing motion compensation prediction encoding using generally known block matching.
The right-eye component image is encoded by performing parallax compensation using the left-eye image at the same time.
Note that, as described above, motion compensation prediction coding in which block matching is performed is performed in motion compensation in a conventional stereo image coding device, but motion compensation prediction coding is controlled according to the characteristics and characteristics of component images. It is not a thing (for example, refer patent document 1).

JP 09-275578 A (paragraph numbers [0021] to [0037], FIG. 1)

  Since the conventional stereo image encoding apparatus is configured as described above, the component image for the left eye is encoded by performing motion compensation prediction encoding using block matching, but depending on the characteristics and characteristics of the component image Since motion compensation predictive coding is not controlled, problems such as motion compensated predictive coding away from the actual motion of the component image may be performed, and the coding accuracy of the component image may deteriorate. was there.

  The present invention has been made in order to solve the above-described problems, and realizes a motion compensated predictive coding close to an actual motion and can improve the coding accuracy of a plurality of images. An object is to obtain a stereo image encoding method and a stereo image encoding program.

  The stereo image encoding apparatus according to the present invention includes a grouping unit that groups areas in a screen according to the parallax calculated by the parallax calculating unit, and a characteristic extraction that extracts characteristics of each region grouped by the grouping unit And a motion prediction means for predicting the motion of each area grouped by the grouping means in consideration of the characteristics extracted by the characteristic extraction means, and the encoding means predicts the motion of each area by the motion prediction means. A plurality of images are encoded using the result.

  According to this invention, the grouping means for grouping the areas in the screen according to the parallax calculated by the parallax calculation means, the characteristic extraction means for extracting the characteristics of each area grouped by the grouping means, and the characteristic extraction In consideration of the characteristics extracted by the means, a motion prediction means for predicting the motion of each area grouped by the grouping means is provided, and the encoding means uses the motion prediction result of each area by the motion prediction means, Since the configuration is such that a plurality of images are encoded, motion compensated predictive encoding close to the actual motion can be realized, and as a result, there is an effect that the encoding accuracy of the plurality of images can be improved. .

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a stereo image coding apparatus according to Embodiment 1 of the present invention. In FIG. 1, a camera 1 captures an object in a three-dimensional space and outputs an image A that is the capturing result. .
The camera 2 is installed at a position different from that of the camera 1, images the same object at the same time as the camera 1, and outputs an image B that is a result of the imaging.

The parallax calculation unit 3 inputs the image A output from the camera 1 and the image B output from the camera 2 and performs a process of calculating the in-screen parallax between the images A and B. The parallax calculation unit 3 constitutes parallax calculation means.
The grouping unit 4 groups the regions in the screen according to the parallax calculated by the parallax calculation unit 3 and outputs an image of each grouped region (hereinafter referred to as a “group region image”). Then, a characteristic extraction process for extracting characteristics (for example, distribution status, size, shape, pattern, color) of each group area image and outputting group area characteristic information indicating the characteristics of each group area image is performed.
The grouping unit 4 constitutes grouping means and characteristic extraction means.

The identifier information adding unit 5 outputs an identifier corresponding to the group area characteristic information output from the grouping unit 4 to the motion prediction unit 7.
The prediction reference image memory 6 is a memory that stores an image for prediction reference (hereinafter referred to as “prediction reference image”) taken at a different time from the images A and B.
The motion prediction unit 7 considers the characteristics of each group region image from the prediction reference image memory 6 (ascertains the characteristics of the group region image from the identifier output from the identifier information adding unit 5), and each group region image Is searched for, the motion of each group region image is predicted using the predicted reference image, and a motion vector indicating the prediction result is output. The motion prediction unit 7 constitutes motion prediction means.

  The encoding unit 8 encodes the images A and B output from the cameras 1 and 2 using the motion vector output from the motion prediction unit 7 and outputs encoded data as the encoding result. On the other hand, a locally decoded image obtained by decoding encoded data by an encoder (not shown) is stored in the predicted reference image memory 6 as a predicted reference image. The encoding unit 8 constitutes an encoding unit.

  In the example of FIG. 1, the parallax calculation unit 3, the grouping unit 4, the identifier information addition unit 5, the motion prediction unit 7, and the encoding unit 8, which are components of the stereo image encoding device, each have dedicated hardware (for example, MPU In the case where the stereo image encoding device is configured by a computer, a parallax calculation unit that is a component of the stereo image encoding device is assumed. 3, the stereo image encoding program describing the processing contents of the grouping unit 4, the identifier information adding unit 5, the motion prediction unit 7 and the encoding unit 8 is stored in the memory of the computer, and the CPU of the computer stores it in the memory It is also possible to execute a stereo image encoding program.

  FIG. 2 is a flowchart showing a stereo image encoding method according to Embodiment 1 of the present invention, and FIG. 3 is an explanatory diagram showing images A and B taken by the cameras 1 and 2 and parallax in the screen.

Next, the operation will be described.
When the cameras 1 and 2 photograph the same object in the three-dimensional space at the same time, the cameras 1 and 2 output images A and B that are the photographing results.
Here, FIG. 3A shows an image A photographed by the camera 1, and in the first embodiment, the image A is used as a left-eye image.
FIG. 3B shows an image B taken by the camera 2. In the first embodiment, the image B is used as a right-eye image.

When the image A is input from the camera 1 and the image B is input from the camera 2, the parallax calculation unit 3 calculates the in-screen parallax between the image A and the image B (step ST1).
FIG. 3C shows the in-screen parallax in the images A and B. Two objects existing in the image A and the image B are represented by dotted lines, and are combined in the parallax screen for convenience. ing.
The parallax in the screens of the images A and B represents the displacement of the objects in the screens of the images A and B. For example, the displacement of the objects in the screen of the image B on the basis of the image A. Can be obtained for each pixel.

FIG. 3C illustrates the parallax of each pixel, and an arrow in the figure represents the parallax. Actually, the parallax is calculated for each pixel, but in FIG. 3C, some are displayed as an example for convenience.
In FIG. 3C, the parallax of the background is the smallest, the parallax of the elliptical object is the largest, and the rectangular object has the second largest parallax.

When the parallax calculation unit 3 calculates the in-screen parallax between the images A and B, the grouping unit 4 groups the areas in the screen according to the in-screen parallax between the images A and B (step ST2). .
That is, the grouping unit 4 classifies regions having the same parallax in the region within the screen into the same group, and assigns images of the region of the same group to the identifier information adding unit 5 and the motion prediction unit 7 as group region images. Output.
In the example of FIGS. 3A and 3B, the areas of the elliptical objects are classified into the same group, the areas of the rectangular objects are classified into the same group, and the background area is the same. As shown in FIG. 3A, the grouping unit 4 is classified into groups, and the grouping unit 4 includes a group region image related to the elliptical object included in the image A and a group region related to the rectangular object. An image and a group area image related to the background are output. Further, as shown in FIG. 3B, a group area image related to the elliptical object included in the image B, a group area image related to the rectangular object, and a group area image related to the background, Is output.
Here, the grouping unit 4 has been described for classifying areas having the same parallax into the same group. However, areas having substantially the same parallax (areas having an arbitrary range of parallax) are classified into the same group. May be.

When the grouping unit 4 groups the regions in the screen according to the parallax in the screen, the grouping unit 4 extracts the characteristics of each group region image and sends group region characteristic information indicating the characteristics of each group region image to the identifier information adding unit 5. Output (step ST3).
That is, the grouping unit 4 extracts, for example, the distribution status, size, shape, pattern, color, and the like as the characteristics of each group region image.
As for the distribution status of the group area images, for example, the pictures of images A and B (frames represented by dotted lines in FIG. 4) are each divided into four parts in the horizontal direction and the vertical direction (the picture is divided into 16 two-dimensional planes). This indicates in which divided area the group area image exists, and is represented by 4-bit information.

The size of the group region image is, for example, a group region image classified into four levels by a threshold value (threshold value related to the area), and is represented by 2-bit information.
The shape of the group area image is, for example, a group area image that is classified into four types of shapes (triangle, quadrangle, circle, and other shapes), and is represented by 2-bit information.
The pattern of the group area image is, for example, one in which the dispersion value of the group area image is classified into four types according to the threshold value, and is represented by 2-bit information.
The color of the group area image is, for example, a group area image classified into 8 colors, and is represented by 3-bit information.

Here, the grouping unit 4 has shown the case where, for example, the distribution status, size, shape, pattern, color, and the like are extracted as the characteristics of each group region image. However, the present invention is not limited to this. The maximum pixel value, the minimum pixel value, the pixel range, etc. may be extracted.
In addition, about the distribution condition of a group area | region image, other division numbers, such as 10 divisions and 20 divisions, may be sufficient, for example, and the two-dimensional coordinate position of a target object may be sufficient.
The number of levels of the size of the group area image may be other numbers, and the shape, pattern, and color may be other classification methods.

When the identifier information adding unit 5 receives the group region characteristic information indicating the characteristics of each group region image from the grouping unit 4, the identifier information adding unit 5 outputs an identifier corresponding to the group region characteristic information to the motion prediction unit 7 (step ST4).
That is, the identifier information adding unit 5 includes a group region image (see FIG. 4A) related to the object whose ellipse is the motion prediction unit 7 and a group region image related to the rectangular object (FIG. 4B). And the group area image related to the background, and the group area image can be classified into categories having the same characteristics or similar characteristics. “0” is set as the identifier of the object, “10” is set as the identifier of the group area image related to the rectangular object, and “20” is set as the identifier of the group area image related to the elliptical object. Output to unit 7.

When the motion prediction unit 7 receives an identifier corresponding to the group region characteristic information from the identifier information addition unit 5, the motion prediction unit 7 refers to the identifier, grasps the characteristics of each group region image output from the grouping unit 4, and performs prediction reference A prediction reference image corresponding to each group area image is searched from the image memory 6 in consideration of the characteristics of each group area image (step ST5).
For example, if the group region image output from the grouping unit 4 is a group region image related to a rectangular object, the shape of the group region image is recognized as a square from the identifier “10”, and the prediction reference is made. A rectangular prediction reference image is searched from the image memory 6.

When searching for a prediction reference image corresponding to each group region image, the motion prediction unit 7 predicts the motion of each group region image using the prediction reference image, and encodes a motion vector indicating the prediction result. (Step ST6).
Here, the motion vector calculates the absolute difference value of the pixels at the same position in the group region image and the predicted reference image, calculates the sum of the absolute difference values (difference absolute value sum) in the region image, and the difference absolute value This corresponds to a vector indicating the image region position in the predicted reference image when the value sum is minimized.
The motion vector corresponds to a displacement from the position of the group area image in the images A and B, and is located at the position indicated by the motion vector from the position of the group area image in the images A and B (for example, the upper left of the area). An image region in a picture (an image region in a picture (see the dotted frame in FIG. 4) stored in the predicted reference image memory 6) is close to a group region image (a predicted reference image corresponding to the group region image) ).

When the encoding unit 8 receives the motion vector from the motion prediction unit 7, the encoding unit 8 encodes the images A and B output from the cameras 1 and 2 using the motion vector, and the encoding result is a code. The converted data is output (step ST7).
Hereinafter, an example of the encoding process in the encoding unit 8 will be specifically described.
When the encoding unit 8 receives the motion vector from the motion prediction unit 7, the encoding unit 8 converts the image region in the picture (in the group region image) from the position of the group region image included in the image A of the camera 1 to the position indicated by the motion vector. A corresponding prediction reference image) is specified, and a difference image between the group region image and the prediction reference image is obtained.
When the encoding unit 8 obtains a difference image between the group region image and the prediction reference image included in the image A, the encoding unit 8 encodes the difference image and the motion vector.

Next, the encoding unit 8 specifies an image region (predicted reference image corresponding to the group region image) in the picture at the position indicated by the motion vector from the position of the group region image included in the image B of the camera 2. Then, a difference image between the group region image and the predicted reference image is obtained.
When the encoding unit 8 obtains a difference image between the group region image and the prediction reference image included in the image B, the encoding unit 8 encodes the difference image and the motion vector.
The encoding unit 8 encodes the images A and B output from the cameras 1 and 2 and outputs encoded data, so that an encoder (not shown) can cope with encoding of subsequent images. The local decoded image in which the encoded data is decoded is stored in the predicted reference image memory 6 as a predicted reference image.

  As is apparent from the above, according to the first embodiment, the areas in the screen are grouped according to the parallax calculated by the parallax calculation unit 3 and the characteristics of the grouped areas are extracted. The grouping unit 4 and a motion prediction unit 7 that predicts the motion of each region grouped by the grouping unit 4 in consideration of the characteristics extracted by the grouping unit 4 are provided. Since the image A and B are encoded using the motion prediction result of each region according to the above, motion compensated prediction encoding close to the actual motion can be realized. There exists an effect which can improve the encoding precision of B.

  Further, according to the first embodiment, the grouping unit 4 is configured to extract the distribution status, size, shape, pattern, or color of each region as the characteristics of each group region image. Can be divided into categories having the same characteristics or similar characteristics, and as a result, there is an effect that the search efficiency of motion vectors can be improved.

In the first embodiment, the image A is used as the left-eye image and the image B is used as the right-eye image. However, the image A is used as the right-eye image and the image B is used as the left-eye image. In addition, it is not necessary to fix the images A and B as the image for what number.
In the first embodiment, the images A and B photographed by the two cameras 1 and 2 are used. However, three or more kinds of images photographed by three or more cameras are used. Alternatively, any two types of images from among three or more types of images may be used.

  In the first embodiment, the parallax calculation unit 3 calculates the displacement of the target object in the screen of the image B with respect to the image A as the parallax in the screen. The reference image may be used as the reference image, and the shift of the object in the screen may be calculated not for each pixel but for each image region composed of a plurality of pixels.

  In the first embodiment, the identifier information adding unit 5 applies the identifier “0” of the group area image related to the background, the identifier “10” of the group area image related to the rectangular object, and the elliptical object. As shown in FIG. 5, there are a plurality of group region images related to a rectangular object and a plurality of group region images related to an elliptical object. If present, for example, “10” and “11” are used as identifiers for the group region image related to the rectangular object, and “20” and “21” are used as identifiers for the group region image related to the elliptical object. You may make it set.

  Further, as shown in FIG. 5, even when there are a plurality of group area images related to a rectangular object or a plurality of group area images related to an elliptical object, the group area images related to a plurality of square objects. The identifier of “10” may be set for the same group, and the identifier of “20” may be set for the group region images related to the plurality of elliptical objects as the same group.

Embodiment 2. FIG.
In the first embodiment, the motion prediction unit 7 searches for the prediction reference image corresponding to each group region image from the prediction reference image memory 6 in consideration of the characteristics of each group region image. When the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, the search range of the image may be controlled (restricted) to a range in which each region overlaps with a preset designated region.

That is, in the first embodiment, when the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, all of the pictures represented by the dotted frame in FIG. 4 are used as the image search range. In the second embodiment, as shown in FIG. 6, for example, a square or rhombus-shaped region (hereinafter referred to as “designated region”) is set in advance (see a one-dot broken line frame in FIG. 6). When the motion prediction unit 7 searches for a prediction reference image corresponding to each group area image, the search range of the image may be controlled to a range where the designated area and the group area image overlap.
When the motion prediction unit 7 controls the search range of the image to a range where the designated region and the group region image overlap, an area outside the range where the designated region and the group region image overlap is excluded from the search region, and only the overlapping range is included. Since it becomes a search area, the search efficiency (search efficiency of a motion vector) of the prediction reference picture corresponding to each group field picture can be raised.

Therefore, in the case of the second embodiment, for example, if the group region image is a single point pixel such as a point light source, the motion vector is searched for only one target pixel.
In the second embodiment, it is not necessary to limit to a generally known block unit vector search, and a one-point light source or an arbitrarily shaped object search may be used.

Embodiment 3 FIG.
In Embodiment 2 described above, when the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, the search range of the image is controlled to a range in which the designated region and the group region image overlap. When motion prediction (motion vector search) is performed in units of image blocks, as shown in FIG. 7, depending on the shape of the group area image and the position in the screen, the image block may straddle the boundary of the group area image. obtain.

Therefore, in the third embodiment, when the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, the search range of the image is controlled (limited) to a range including a portion where the designated region and the group region image overlap. )
That is, an image block that partially includes a portion where the designated area and the group area image overlap is also included in the motion vector search target.
The motion prediction unit 7 controls the image search range to a range including a portion where the designated region and the group region image overlap, and performs motion prediction (motion vector search) in units of image blocks. An evaluation value is obtained and a motion vector corresponding to the minimum evaluation value is output.
Here, the prediction evaluation value is obtained for each image block, and the motion vector corresponding to the minimum evaluation value is output, but the prediction evaluation value may be a generally used difference absolute value sum, You may make it output the motion vector corresponding to the evaluation value below a threshold value.

  As is apparent from the above, according to the third embodiment, when the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, a portion where the designated region and the group region image overlap the search range of the image. Therefore, even when motion prediction (motion vector search) is performed in units of image blocks, the search efficiency of the prediction reference image corresponding to each group region image (motion vector search efficiency) ) Can be enhanced.

Embodiment 4 FIG.
In Embodiment 2 described above, when the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, the search range of the image is controlled to a range in which the designated region and the group region image overlap. The motion prediction unit 7 may obtain a representative motion vector of each group region image, and control (limit) the image search range to a range based on the representative motion vector.

FIG. 8 is an explanatory diagram for explaining a representative motion vector.
When the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, the sum of absolute differences of each group region image is minimized among the prediction reference images stored in the prediction reference image memory 6. As a representative motion vector.
In the example of FIG. 8, the representative motion vector of the group region image related to the rectangular object and the representative motion vector of the group region image related to the elliptical object are obtained.

When the motion prediction unit 7 obtains the representative motion vector of each group region image, the motion prediction unit 7 controls the search range of the image to a range based on the representative motion vector (see the dashed line frame), and corresponds to each group region image. The predicted reference image to be searched is searched (motion vector is searched).
That is, the motion prediction unit 7 divides each group region image into a plurality of regions, and searches for a motion vector for each divided region.
At this time, instead of using the zero origin as a reference, the motion vector is searched in a search range using the representative motion vector as a reference (center). That is, the representative motion vector is added as an offset to the zero origin, and the motion vector is searched in the search range using the addition point as a reference point.

  As apparent from the above, according to the fourth embodiment, the motion prediction unit 7 obtains the representative motion vector of each group region image, and controls the image search range to a range based on the representative motion vector. Since it is configured, it is possible to search for a predicted reference image (search for a motion vector) in the vicinity of each group region image, and as a result, search efficiency (motion vector) of a predicted reference image corresponding to each group region image. The search efficiency is improved.

In the fourth embodiment, the motion prediction unit 7 uses a representative motion vector as a representative motion vector, among the prediction reference images stored in the prediction reference image memory 6, for which the sum of absolute differences of the group region images is minimized. However, the representative motion vector may be obtained in the predicted reference image for the group region image itself without including the surrounding image.
Further, as shown in FIG. 9A, for an area composed of several square blocks including a group area image (the area of dotted square blocks at the four corners may or may not be included) The representative motion vector may be obtained in the predicted reference image.
Further, as shown in FIG. 9B, a representative motion vector may be obtained in a predicted reference image with respect to a square region of a group region image, and a matching region for the group region image when the representative motion vector is obtained is Optional.

  Further, when obtaining the representative motion vector, when obtaining the representative motion vector using the pixels in the group area image or the pixels in the target area including the vicinity of the group area image, the representative motion vector is obtained using all the pixels in the target area. A motion vector search may be performed. Alternatively, subpixels in the target area may be subsampled, and the representative motion vector may be searched using the subsampled pixels, or the representative motion vector may be searched using only the luminance component of the target area. Also good. In addition, the representative motion vector may be searched using the luminance component and the color difference component of the target region, and the handling of the pixels of the target region may be arbitrary.

Embodiment 5. FIG.
FIG. 10 is a block diagram showing a stereo image coding apparatus according to Embodiment 5 of the present invention. In the figure, the same reference numerals as those in FIG.
The motion position information setting unit 11 inputs motion position information indicating the motion and position of the cameras 1 and 2, stores the motion position information for the necessary pictures, and the motion position information at the time when the predicted reference image is taken And the movement position information at the time when the images A and B were taken, capture the change of the cameras 1 and 2 from the time when the predicted reference image was taken to the time when the images A and B were taken. The camera change information indicating the change of 2 is output.

The search range control unit 12 performs a process of determining the search range of the predicted reference image based on the camera change information output from the motion position information setting unit 11 and the parallax calculated by the parallax calculation unit 3.
Similar to the motion prediction unit 7 in FIG. 1, the motion prediction unit 13 searches for a prediction reference image corresponding to each group region image and predicts the motion of each group region image using the prediction reference image. When searching for a predicted reference image corresponding to an image, the search range of the predicted reference image is controlled (limited) to the search range determined by the search range control unit 12.
The motion position information setting unit 11, the search range control unit 12, and the motion prediction unit 13 constitute a motion prediction unit.

Next, the operation will be described.
Except for the motion position information setting unit 11, the search range control unit 12, and the motion prediction unit 13, it is the same as in the first embodiment.
When the motion position information setting unit 11 receives motion position information indicating the motion and position of the cameras 1 and 2, the motion position information setting unit 11 stores the motion position information for the necessary pictures. That is, every time the movement position information is input, the latest movement position information is overwritten and recorded on the oldest movement position information.

  The motion position information setting unit 11 uses the motion position information at the time when the predicted reference image is captured and the motion position information at the time when the images A and B are captured, from the time when the predicted reference image is captured. The camera change information indicating the change of the cameras 1 and 2 is output to the search range control unit 12 by capturing the changes of the cameras 1 and 2 up to the present time when B is photographed.

When receiving the camera change information from the motion position information setting unit 11, the search range control unit 12 determines the search range of the predicted reference image based on the camera change information and the parallax calculated by the parallax calculation unit 3.
A specific method for determining the search range in the search range control unit 12 will be described later in the seventh embodiment and later.

  The motion prediction unit 13 searches for a prediction reference image corresponding to each group region image and predicts the motion of each group region image in the same manner as the motion prediction unit 7 in FIG. In order to increase efficiency, unlike the motion prediction unit 7 in FIG. 1, the search range of the predicted reference image is controlled to the search range determined by the search range control unit 12.

  As is apparent from the above, according to the fifth embodiment, the motion position information indicating the motion and position of the cameras 1 and 2 is input, and when the predicted reference image corresponding to each group region image is searched, the predicted reference Since the search range of the image is controlled according to the movement and position of the camera indicated by the movement position information, the prediction reference image can be searched in the search range corresponding to the actual movement of the object. As a result, there is an effect that the search efficiency (motion vector search efficiency) of the prediction reference image corresponding to each group region image can be increased.

Embodiment 6 FIG.
In the fifth embodiment, the movement position information setting unit 11 inputs the movement position information indicating the movement and position of the cameras 1 and 2. Information including two horizontal rotation angles or vertical rotation angles is input, and information including the movement distances of the cameras 1 and 2 in the three-dimensional direction (movement distance of at least one direction) is used as information indicating the positions of the cameras 1 and 2. May be input.
If the motion position information setting unit 11 inputs information including the horizontal rotation angle or vertical rotation angle of the cameras 1 and 2 and information including the movement distance of the cameras 1 and 2 in the three-dimensional direction, the predicted reference image is captured. There is an effect that the change of the cameras 1 and 2 from the time to the current time when the images A and B are taken can be accurately captured.

Here, the movement position information setting unit 11 has been shown to input information including the horizontal rotation angle or vertical rotation angle of the cameras 1 and 2 and information including the movement distance of the cameras 1 and 2 in the three-dimensional direction. The movement position information setting unit 11 may input information including a horizontal movement angle or a vertical movement angle with the focus as the origin, or three-dimensional movement information in the imaging direction of the cameras 1 and 2.
Further, the distance from the focal point, information indicating a relative position, or information indicating an absolute position may be input.

Embodiment 7 FIG.
In Embodiment 2 described above, when the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, the search range of the image is controlled to a range in which the designated region and the group region image overlap. Each group region image grouped by the grouping unit 4 when the cameras 1 and 2 are moved while rotating while the base point is located at the center of the image with the base point at infinity. When searching for a predicted reference image corresponding to, the search range of the predicted reference image may be controlled (limited) to a smaller range as the group region image has a smaller parallax.

FIG. 11 is an explanatory diagram showing the movement of the cameras 1 and 2, the search range, and the apparent motion vector.
FIG. 11 shows a state where the cameras 1 and 2 are moving from the right to the left on the horizontal plane while rotating while the base point is located at the center of the image with the base point at infinity.
In this case, the parallax of the group area image related to the elliptical object is larger than the parallax of the group area image related to the background, and the apparent amount of motion of the group area image related to the elliptical object having a large parallax is the parallax. Is larger than the apparent amount of motion of the group area image related to the small background.
Here, the apparent amount of movement is the amount of movement of each object before and after the movement of the cameras 1 and 2 when the cameras 1 and 2 are rotated while the object in the screen is stationary. Means.

Therefore, the motion prediction unit 7 uses an elliptical object with a large parallax when the cameras 1 and 2 are moved while rotating while the base point is located at the center of the image with the base point at infinity. The search range of the prediction reference image corresponding to the group region image related to the object is increased, and the search range of the prediction reference image corresponding to the group region image related to the background having a small parallax is reduced.
That is, the search range of the predicted reference image corresponding to the group region image with the largest parallax is set to the maximum, and the search range of the predicted reference image corresponding to the group region image with the smallest parallax is set to a smaller range.

  As is apparent from the above, according to the seventh embodiment, when the cameras 1 and 2 are moved while rotating while the base point is placed at the center of the image with the base point at infinity, the movement When the prediction unit 7 searches for a prediction reference image corresponding to each group region image grouped by the grouping unit 4, the search range of the prediction reference image is controlled to be smaller as the group region image has a smaller parallax. Therefore, even when the cameras 1 and 2 are moved while rotating with the base point at infinity as the base point, the search for the prediction reference image corresponding to each group region image is performed. There is an effect that efficiency (motion vector search efficiency) can be improved.

Normally, the cameras 1 and 2 have a base point at the center of the input image. However, depending on the imaging system, the base point may not be at the center of the input image. Even when it is not in the center, it can cope.
In the seventh embodiment, the cameras 1 and 2 rotate on the horizontal plane. However, the cameras 1 and 2 may rotate in any direction.
In addition, even if an upper limit of the motion vector search range is provided due to the size of the hardware, etc., if the search range can be controlled within the range below the upper limit, the control range may be the motion vector of the stereo image. Search efficiency can be increased.

Embodiment 8 FIG.
In Embodiment 7 described above, when the cameras 1 and 2 are moved while rotating while the base point is located at the center of the image with the base point at infinity, the motion predicting unit 7 applies to each group region image. In the case of searching for a corresponding predicted reference image, the search range of the predicted reference image is controlled so that the group region image with a smaller parallax has a smaller range. However, the cameras 1 and 2 are rotated without changing the position. When searching for a predicted reference image corresponding to each group region image grouped by the grouping unit 4, the search range of the predicted reference image may be controlled (limited) to a smaller range as the group region image has a larger parallax. Good.

FIG. 12 is an explanatory diagram showing rotation of the cameras 1 and 2 at a fixed position, a search range, and an apparent motion vector.
FIG. 12 shows a state where the cameras 1 and 2 are rotating on the horizontal plane without changing the position.
In this case, the parallax of the group area image related to the elliptical object is larger than the parallax of the group area image related to the background, and the apparent amount of motion of the group area image related to the elliptical object having a large parallax is the parallax. Is smaller than the apparent amount of motion of the group area image related to the background.
Here, the apparent amount of movement is the amount of each object before and after the rotation of the cameras 1 and 2 when the cameras 1 and 2 are rotated at a fixed position with the object in the screen stationary. It means the amount of movement.

Therefore, when the cameras 1 and 2 are rotated without changing the position, the motion prediction unit 7 reduces the search range of the prediction reference image corresponding to the group region image related to the elliptical object having a large parallax. Thus, the search range of the predicted reference image corresponding to the group region image related to the background with small parallax is increased.
That is, the search range of the prediction reference image corresponding to the group region image with the smallest parallax is set to the maximum, and the search range of the prediction reference image corresponding to the group region image with the large parallax is set to a smaller range.

  As is apparent from the above, according to the eighth embodiment, when the cameras 1 and 2 are rotated without changing the position, the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image. At this time, since the search range of the predicted reference image is configured to be controlled to a smaller range as the group region image has a larger parallax, even when the cameras 1 and 2 are rotated without changing the position, each group region image is supported. There is an effect that it is possible to improve the search efficiency (motion vector search efficiency) of the predicted reference image to be performed.

In the eighth embodiment, the cameras 1 and 2 rotate on the horizontal plane. However, the cameras 1 and 2 may rotate in an arbitrary direction.
In addition, even if an upper limit of the motion vector search range is provided due to the size of the hardware, etc., if the search range can be controlled within the range below the upper limit, the control range may be the motion vector of the stereo image. Search efficiency can be increased.

Embodiment 9 FIG.
In Embodiment 8 above, when the cameras 1 and 2 are rotated without changing the position, when the motion prediction unit 7 searches for a predicted reference image corresponding to each group region image, the search range of the predicted reference image is set. Although the group area image having a larger parallax has been controlled to be controlled to a smaller range, when the cameras 1 and 2 are zoomed in and out, the motion prediction unit 7 searches for a predicted reference image corresponding to each group area image. At this time, the search range of the predicted reference image may be controlled (limited) to a smaller range as the group region image has a smaller parallax.

FIG. 13 shows search ranges and apparent motion vectors when the cameras 1 and 2 are zoomed in and out when the positions of the cameras 1 and 2 are not changed and the cameras 1 and 2 are not rotated. It is explanatory drawing.
FIG. 13 shows a state in which the cameras 1 and 2 are zoomed in and out in a state where the positions of the cameras 1 and 2 are not changed and the cameras 1 and 2 are not rotating.
When the cameras 1 and 2 are zoomed in and out, the parallax of the group area image related to the elliptical object becomes larger than the parallax of the group area image related to the background, and the group related to the elliptical object having a large parallax The apparent amount of motion of the region image is larger than the apparent amount of motion of the group region image related to the background with a small parallax.
Here, the apparent amount of movement is determined when the position of the cameras 1 and 2 cannot be changed and the cameras 1 and 2 are not rotating when the object in the screen is stationary. This means the amount of movement of each object before zooming in (or zooming out) and after zooming in (or zooming out) of cameras 1 and 2 when zooming in and zooming out 2 are performed.

Therefore, the motion prediction unit 7 has a large parallax when the cameras 1 and 2 are zoomed in and out while the positions of the cameras 1 and 2 are not changed and the cameras 1 and 2 are not rotating. The search range of the prediction reference image corresponding to the group region image related to the background with a small parallax is reduced by increasing the search range of the prediction reference image corresponding to the group region image related to the elliptical object.
That is, the search range of the predicted reference image corresponding to the group region image with the largest parallax is set to the maximum, and the search range of the predicted reference image corresponding to the group region image with the smallest parallax is set to a smaller range.

  As is apparent from the above, according to the ninth embodiment, when the cameras 1 and 2 are zoomed in and out, when the motion prediction unit 7 searches for a prediction reference image corresponding to each group region image, Since the search range of the predicted reference image is configured to be controlled to a smaller range as the group region image having a smaller parallax, the positions of the cameras 1 and 2 cannot be changed, and the cameras 1 and 2 are not rotated. Even when the cameras 1 and 2 are zoomed in and out, the search reference image search efficiency (motion vector search efficiency) corresponding to each group region image can be improved.

  Even if an upper limit of the motion vector search range is provided due to the scale of the hardware, etc., if the search range can be controlled within the range below the upper limit, the control range is the motion vector of the stereo image. Search efficiency can be increased.

Embodiment 10 FIG.
FIG. 14 is a block diagram showing a stereo image encoding apparatus according to Embodiment 10 of the present invention. In the figure, the same reference numerals as those in FIG.
The reference search range setting unit 21 is a group region image (for example, a group region image related to an elliptical object included in the images A and B) among the group region images grouped by the grouping unit 4. The process of setting the search range of the predicted reference image corresponding to) as the standard search range is performed.
The search range coefficient setting unit 22 performs a process of setting a search range coefficient for each group area image.

Similar to the motion prediction unit 7 in FIG. 1, the motion prediction unit 23 searches for a prediction reference image corresponding to each group region image and predicts the motion of each group region image using the prediction reference image. When searching for a predicted reference image corresponding to an image, the search range of the predicted reference image corresponding to each group region image is determined from the setting contents of the standard search range setting unit 21 and the search range coefficient setting unit 22.
The reference search range setting unit 21, the search range coefficient setting unit 22, and the motion prediction unit 23 constitute a motion prediction unit.

Next, the operation will be described.
Except for the reference search range setting unit 21, the search range coefficient setting unit 22, and the motion prediction unit 23, it is the same as the first embodiment.
The reference search range setting unit 21 sets the search range of the predicted reference image corresponding to the reference group region image among the group region images grouped by the grouping unit 4 as the reference search range.
For example, when the group area image related to the elliptical object included in the images A and B is set to be the reference group area image, the elliptical shape included in the images A and B is set. The search range of the predicted reference image corresponding to the group region image related to the object is set as the standard search range (here, for convenience of explanation, the standard search range is “S”).

The search range coefficient setting unit 22 sets search range coefficients for group area images other than the reference group area image among the group area images grouped by the grouping unit 4.
For example, the search range coefficient of the group area image related to the rectangular object included in the images A and B is set to “1/2”, and the search range coefficient of the group area image related to the background is set to “1/4”. Set to "".

In the motion prediction unit 23, the reference search range setting unit 21 sets the reference search range of the prediction reference image corresponding to the reference group region image, and the search range coefficient setting unit 22 sets the group region image other than the reference group region image. When the search range coefficient is set, the search range of the group area image other than the reference group area image is determined as follows.
Search range of group region images other than the standard group region image = reference search range × search range coefficient of the group region image Accordingly, the search range of the predicted reference image corresponding to the group region image related to the rectangular object is “ The search range of the predicted reference image corresponding to the group region image related to S / 2 "and the background is determined to be" S / 4 ".
The prediction reference image search processing and motion prediction processing in the motion prediction unit 23 are the same as those in the motion prediction unit 7 in FIG.

  As apparent from the above, according to the tenth embodiment, the search range of the predicted reference image corresponding to the reference group region image among the group region images grouped by the grouping unit 4 is set as the reference search range. And a search range coefficient setting unit 22 for setting a search range coefficient for each group region image, and the motion prediction unit 23 searches for a prediction reference image corresponding to each group region image. At this time, since the search range of the prediction reference image corresponding to each group region image is determined from the setting contents of the reference search range setting unit 21 and the search range coefficient setting unit 22, an appropriate search range can be easily calculated. There is an effect that can be obtained.

In the tenth embodiment, the reference search range setting unit 21 and the search range coefficient setting unit 22 refer to the identifier output from the identifier information adding unit 5, and the reference group region image or the reference group region image In the above description, the group area image other than the above is identified. However, the reference search range and the search range coefficient may be set in consideration of the parallax calculated by the parallax calculation unit 3.
In the tenth embodiment, the search range coefficient setting unit 22 sets “1/2” or “1/4” as the search range coefficient of the group area image. Any number is acceptable.
In the tenth embodiment, the search range coefficient of the reference group region image is not set, but the search range coefficient setting unit 22 sets “1” as the search range coefficient of the reference group region image. You may make it output.

Embodiment 11 FIG.
In the tenth embodiment, for example, the group area image related to the elliptical object has been set in advance as the reference group area image. The group area image in which 1 and 2 are in focus may be determined as the reference group area image, and the search range of the predicted reference image corresponding to the group area image may be set as the reference search area.

In this way, when the reference search range setting unit 21 determines the group area image in which the cameras 1 and 2 are in focus as the reference group area image, the parallax of the in-focus group area image is minimized. The standard search range of the predicted reference image corresponding to the region image becomes the minimum search range.
Therefore, since the parallax becomes larger and the search range becomes larger as the group region image on the near side of the focus becomes larger than the group region image on the focus, the search range coefficient setting unit 22 sets the group region image on the near side of the focus. A large search range coefficient is set.
Further, since the parallax becomes larger and the search range becomes larger as the group area image on the deeper side of the focus, the search range coefficient setting unit 22 sets a larger search range coefficient for the group area image on the deeper side of the focus. .

  As is apparent from the above, according to the eleventh embodiment, the reference search range setting unit 21 determines the group region image in which the cameras 1 and 2 are in focus as the reference group region image, and the group region image Since the search range of the predicted reference image corresponding to is set as the standard search range, an appropriate search range can be set even if the cameras 1 and 2 move or zoom in and zoom out. Play.

Embodiment 12 FIG.
FIG. 15 is a block diagram showing a stereo image coding apparatus according to Embodiment 12 of the present invention. In the figure, the same reference numerals as those in FIG.
The apparent motion vector control unit 31 performs a process of calculating an apparent motion vector from the camera change information output from the motion position information setting unit 11 and the parallax calculated by the parallax calculation unit 3.
The motion prediction unit 32 searches for a predicted reference image corresponding to each group region image in consideration of the apparent motion vector calculated by the apparent motion vector control unit 31, and uses the predicted reference image to determine the motion of each group region image. Predict.
The motion position information setting unit 11, the apparent motion vector control unit 31, and the motion prediction unit 32 constitute a motion prediction unit.

Next, the operation will be described.
Except for the apparent motion vector control unit 31 and the motion prediction unit 32, the present embodiment is the same as the fifth embodiment.
When the motion position information setting unit 11 receives the motion position information indicating the motion and position of the cameras 1 and 2, the motion position information setting unit 11 stores the motion position information for the necessary pictures as in the fifth embodiment. That is, every time the movement position information is input, the latest movement position information is overwritten and recorded on the oldest movement position information.
The motion position information setting unit 11 uses the motion position information at the time when the predicted reference image is captured and the motion position information at the time when the images A and B are captured, from the time when the predicted reference image is captured. The change of the cameras 1 and 2 up to the present time when B is photographed is captured, and the camera change information indicating the change of the cameras 1 and 2 is output to the apparent motion vector control unit 31.

Upon receiving the camera change information from the motion position information setting unit 11, the apparent motion vector control unit 31 calculates an apparent motion vector from the camera change information and the parallax calculated by the parallax calculation unit 3.
The apparent motion vector corresponds to the apparent motion amount of the group area image in FIGS.

The motion prediction unit 32 searches for a prediction reference image corresponding to each group region image and predicts the motion of each group region image in the same manner as the motion prediction unit 7 in FIG. In order to increase efficiency, unlike the motion prediction unit 7 in FIG. 1, a predicted reference image corresponding to each group region image is searched in consideration of the apparent motion vector calculated by the apparent motion vector control unit 31.
That is, the motion prediction unit 32 searches for the motion vector centered on the apparent motion vector at the base point (center) of the motion vector search range for the apparent motion vector calculated by the apparent motion vector control unit 31.

  As is apparent from the above, according to the twelfth embodiment, the change in the cameras 1 and 2 from the time when the predicted reference image is taken to the time when the images A and B are taken is recognized, and the cameras 1 and 2 are recognized. Since the apparent motion vector is calculated from the change in the parallax and the parallax calculated by the parallax calculation unit 3 and the predicted reference image corresponding to each group region image is searched in consideration of the apparent motion vector, the actual target The prediction reference image can be searched in the search range corresponding to the movement of the object, and as a result, the search efficiency (search efficiency of the motion vector) of the prediction reference image corresponding to each group region image can be increased. There is an effect.

  In the twelfth embodiment, although the apparent motion vector is set at the base point (center) of the motion vector search range, the apparent motion vector is changed for other reasons or from the viewpoint of encoding efficiency. You may make it place in the arbitrary positions of the search range of a motion vector.

Embodiment 13 FIG.
In the twelfth embodiment, the search for the predicted reference image corresponding to each group region image in consideration of the apparent motion vector calculated by the apparent motion vector control unit 31 has been described, but as shown in FIG. When the cameras 1 and 2 are moved while rotating while the base point is located at the center of the image with the base point at infinity, the motion prediction unit 32 selects the prediction reference image corresponding to each group region image. When searching, the apparent motion vector control unit 31 may make the apparent motion vector smaller as the group region image has a smaller parallax.

FIG. 11 shows a state in which the cameras 1 and 2 are moving from right to left on the horizontal plane while rotating, with the base point being at the center of the image, as described above, with the base point being at the center of the image. Represents.
In this case, the parallax of the group area image related to the elliptical object is larger than the parallax of the group area image related to the background, and the apparent amount of motion of the group area image related to the elliptical object having a large parallax is the parallax. Is larger than the apparent amount of motion of the group area image related to the small background.
Here, the apparent amount of movement is the amount of movement of each object before and after the movement of the cameras 1 and 2 when the cameras 1 and 2 are rotated while the object in the screen is stationary. Means.

Thus, the apparent motion vector control unit 31 uses an elliptical shape with a large parallax when the cameras 1 and 2 are moved while rotating while the base point is located at the center of the image with the base point being at infinity. The apparent motion vector of the group area image related to the target object is increased, and the apparent motion vector of the group area image related to the background with a small parallax is reduced.
That is, the apparent motion vector is maximized for the group area image having the largest parallax, and the apparent motion vector is made smaller for the group area image having the smallest parallax.

  As is apparent from the above, according to the thirteenth embodiment, when the cameras 1 and 2 are moved while rotating while the base point is placed at the center of the image with the base point at infinity, When the prediction unit 32 searches for a prediction reference image corresponding to each group region image, the apparent motion vector control unit 31 is configured to decrease the apparent motion vector as the group region image has a smaller parallax. As a base point, even when the cameras 1 and 2 are moved while rotating while the base point is placed at the center of the image, the search efficiency (motion vector of the motion vector) corresponding to each group region image is moved. (Search efficiency) can be improved.

Normally, the cameras 1 and 2 have a base point at the center of the input image. However, depending on the imaging system, the base point may not be at the center of the input image. However, in Embodiment 13, the base point is the input image. Even when it is not in the center, it can cope.
In the thirteenth embodiment, the cameras 1 and 2 rotate on the horizontal plane. However, the cameras 1 and 2 may rotate in any direction.
In addition, even if an upper limit of the motion vector search range is provided due to the size of the hardware, etc., if the search range can be controlled within the range below the upper limit, the control range may be the motion vector of the stereo image. Search efficiency can be increased.

Embodiment 14 FIG.
In the above-described thirteenth embodiment, the apparent motion vector control unit 31 has shown that the group motion image having a smaller parallax reduces the apparent motion vector. However, as shown in FIG. When the rotation of 2 is performed, when the motion prediction unit 32 searches for a prediction reference image corresponding to each group region image, the apparent motion vector control unit 31 decreases the apparent motion vector as the group region image has a larger parallax. You may do it.

FIG. 12 shows a state where the cameras 1 and 2 are rotating on the horizontal plane without changing the position as described above.
In this case, the parallax of the group area image related to the elliptical object is larger than the parallax of the group area image related to the background, and the apparent amount of motion of the group area image related to the elliptical object having a large parallax is the parallax. Is smaller than the apparent amount of motion of the group area image related to the background.
Here, the apparent amount of movement is the amount of each object before and after the rotation of the cameras 1 and 2 when the cameras 1 and 2 are rotated at a fixed position with the object in the screen stationary. It means the amount of movement.

Therefore, when the cameras 1 and 2 are rotated without changing the position, the apparent motion vector control unit 31 reduces the apparent motion vector only for the group region image related to the elliptical object having a large parallax, The apparent motion vector is increased only for the group area image related to the background with small parallax.
That is, the apparent motion vector for the group area image with the smallest parallax is maximized, and the apparent motion vector for the group area image with the largest parallax is made smaller.

  As is apparent from the above, according to the fourteenth embodiment, when the cameras 1 and 2 are rotated without changing the position, the motion prediction unit 32 searches for a prediction reference image corresponding to each group region image. At this time, since the apparent motion vector control unit 31 is configured to reduce the apparent motion vector as the group region image has a larger parallax, each group region can be obtained even when the cameras 1 and 2 are rotated without changing the position. There is an effect that the search efficiency of the predicted reference image corresponding to the image (the search efficiency of the motion vector) can be increased.

In the fourteenth embodiment, the cameras 1 and 2 rotate on the horizontal plane. However, the cameras 1 and 2 may rotate in an arbitrary direction.
In addition, even if an upper limit of the motion vector search range is provided due to the size of the hardware, etc., if the search range can be controlled within the range below the upper limit, the control range may be the motion vector of the stereo image. Search efficiency can be increased.

Embodiment 15 FIG.
In the thirteenth embodiment, the apparent motion vector control unit 31 has shown that the group region image having a smaller parallax reduces the apparent motion vector. However, as shown in FIG. When out is performed, when the motion prediction unit 32 searches for a prediction reference image corresponding to each group region image, the apparent motion vector control unit 31 decreases the apparent motion vector as the group region image has a smaller parallax. May be.

FIG. 13 shows how the cameras 1 and 2 are zoomed in and out when the positions of the cameras 1 and 2 are not changed and the cameras 1 and 2 are not rotated as described above. Yes.
When the cameras 1 and 2 are zoomed in and out, the parallax of the group area image related to the elliptical object becomes larger than the parallax of the group area image related to the background, and the group related to the elliptical object having a large parallax The apparent amount of motion of the region image is larger than the apparent amount of motion of the group region image related to the background with a small parallax.
Here, the apparent amount of movement is determined when the position of the cameras 1 and 2 cannot be changed and the cameras 1 and 2 are not rotating when the object in the screen is stationary. This means the amount of movement of each object before zooming in (or zooming out) and after zooming in (or zooming out) of cameras 1 and 2 when zooming in and zooming out 2 are performed.

Therefore, the apparent motion vector control unit 31 performs parallax when the cameras 1 and 2 are zoomed in and out when the positions of the cameras 1 and 2 are not changed and the cameras 1 and 2 are not rotated. The apparent motion vector is increased for the group area image related to the elliptical object having a large and the apparent motion vector is decreased for the group area image related to the background having a small parallax.
That is, the apparent motion vector is maximized for the group area image having the largest parallax, and the apparent motion vector is made smaller for the group area image having the smallest parallax.

  As is apparent from the above, according to the fifteenth embodiment, when the cameras 1 and 2 are zoomed in and zoomed out, when the motion prediction unit 32 searches for a predicted reference image corresponding to each group region image, Since the apparent motion vector control unit 31 is configured to reduce the apparent motion vector as the group region image has a smaller parallax, the positions of the cameras 1 and 2 cannot be changed, and the cameras 1 and 2 are not rotated. In this state, even when the cameras 1 and 2 are zoomed in and out, the search reference image search efficiency (motion vector search efficiency) corresponding to each group region image can be improved.

  Even if an upper limit of the motion vector search range is provided due to the scale of the hardware, etc., if the search range can be controlled within the range below the upper limit, the control range is the motion vector of the stereo image. Search efficiency can be increased.

Embodiment 16 FIG.
FIG. 16 is a block diagram showing a stereo image coding apparatus according to Embodiment 16 of the present invention. In the figure, the same reference numerals as those in FIG.
The reference apparent motion vector setting unit 41 is a group region image (for example, a group region related to an elliptical object included in the images A and B) among the group region images grouped by the grouping unit 4. (Image) An apparent motion vector is set as a reference apparent motion vector.
The apparent motion vector coefficient setting unit 42 performs processing for setting an apparent motion vector coefficient for each group region image.

Similarly to the motion prediction unit 32 in FIG. 15, the motion prediction unit 43 searches for a prediction reference image corresponding to each group region image in consideration of the apparent motion vector, and uses the prediction reference image to move the motion of each group region image. However, when searching for a prediction reference image corresponding to each group region image, the apparent motion vector is determined for each group region image from the setting contents of the reference apparent motion vector setting unit 41 and the apparent motion vector coefficient setting unit 42. To do.
The reference apparent motion vector setting unit 41, the apparent motion vector coefficient setting unit 42, and the motion prediction unit 43 constitute a motion prediction unit.

Next, the operation will be described.
Except for the reference apparent motion vector setting unit 41, the apparent motion vector coefficient setting unit 42, and the motion prediction unit 43, the present embodiment is the same as the first embodiment.
The reference apparent motion vector setting unit 41 sets an apparent motion vector as a reference apparent motion vector among the group region images that are grouped by the grouping unit 4.
For example, when the group area image related to the elliptical object included in the images A and B is set to be the reference group area image, the elliptical shape included in the images A and B is set. The apparent motion vector of the group region image related to the object is set as a reference apparent motion vector (here, the reference apparent motion vector is “V” for convenience of explanation).

The apparent motion vector coefficient setting unit 42 sets apparent motion vector coefficients only for group area images other than the reference group area image among the group area images grouped by the grouping unit 4.
For example, the apparent motion vector coefficient for the group area image related to the rectangular object included in the images A and B is set to “1/2”, and the apparent motion vector coefficient for the group area image related to the background is set to “1”. Set to / 4 ".

In the motion prediction unit 43, the reference apparent motion vector setting unit 41 sets the reference apparent motion vector of the reference group region image, and the apparent motion vector coefficient setting unit 42 applies the apparent motion vector of the group region image other than the reference group region image. When the coefficient is set, the motion vector for the group area image other than the reference group area image is determined as follows.
The apparent motion vector of the group region image other than the reference group region image = reference apparent motion vector × the apparent motion vector coefficient of the group region image. Thus, the apparent motion vector of the group region image related to the rectangular object is “V / 2 ", the apparent motion vector of the group area image related to the background is determined to be" V / 4 ".
The prediction reference image search processing and motion prediction processing in the motion prediction unit 43 are the same as those in the motion prediction unit 32 in FIG.

  As is apparent from the above, according to the sixteenth embodiment, a reference for setting an apparent motion vector as a reference apparent motion vector in a reference group region image among group region images grouped by the grouping unit 4. When an apparent motion vector setting unit 41 and an apparent motion vector coefficient setting unit 42 for setting an apparent motion vector coefficient for each group region image are provided, the motion prediction unit 43 searches for a predicted reference image corresponding to each group region image. Since the configuration is such that the apparent motion vector is determined only for each group region image from the setting contents of the reference apparent motion vector setting unit 41 and the apparent motion vector coefficient setting unit 42, an appropriate apparent motion vector can be easily obtained by calculation. There is an effect that can.

In the sixteenth embodiment, the reference apparent motion vector setting unit 41 and the apparent motion vector coefficient setting unit 42 refer to the identifier output from the identifier information adding unit 5, and the reference group region image and the reference group In the above description, the group area image other than the area image is identified. However, the reference apparent motion vector and the apparent motion vector coefficient may be set in consideration of the parallax calculated by the parallax calculation unit 3.
In the sixteenth embodiment, the apparent motion vector coefficient setting unit 42 sets “1/2” or “1/4” as the apparent motion vector coefficient for the group region image. Any number may be used.
In the sixteenth embodiment, the apparent motion vector coefficient is not set for the reference group region image, but the apparent motion vector coefficient setting unit 42 sets “1” as the apparent motion vector coefficient for the reference group region image. It may be set and output.

Embodiment 17. FIG.
In Embodiment 16 described above, for example, the group area image related to the elliptical object has been set in advance as the reference group area image, but the reference apparent motion vector setting unit 41 The group area image in which the cameras 1 and 2 are in focus may be determined as the reference group area image, and the apparent motion vector of the group area image may be set as the reference apparent motion vector.

As described above, when the reference apparent motion vector setting unit 41 determines the group area image in which the cameras 1 and 2 are in focus as the reference group area image, the parallax of the in-focus group area image is minimized. The reference apparent motion vector of the group area image becomes the minimum apparent motion vector.
Therefore, since the parallax increases and the apparent motion vector increases as the group region image on the near side of the focus becomes larger than the group region image on the focus, the apparent motion vector coefficient setting unit 42 performs the group region on the near side of the focus. A larger apparent motion vector coefficient is set for an image.
Further, since the parallax increases and the apparent motion vector increases as the group area image on the far side of the focal point, the apparent motion vector coefficient setting unit 42 sets the larger apparent motion vector coefficient as the group area image on the far side of the focal point. It will be.

  As apparent from the above, according to the seventeenth embodiment, the reference apparent motion vector setting unit 41 determines the group region image in which the cameras 1 and 2 are in focus as the reference group region image, and the group region Since the apparent motion vector of the image is set as the reference apparent motion vector, an appropriate apparent motion vector can be set even if the cameras 1 and 2 move or zoom in and out. .

Embodiment 18 FIG.
FIG. 17 is a block diagram showing a stereo image encoding apparatus according to Embodiment 18 of the present invention. In the figure, the same reference numerals as those in FIG.
The operation information setting unit 51 inputs operation information indicating the operation of the cameras 1 and 2, stores the operation information for the necessary pictures, operation information at the time when the predicted reference image was captured, and images A and B Based on the operation information at the time when the predicted reference image was taken, the change in the operation of the cameras 1 and 2 from the time when the predicted reference image was taken to the current time when the images A and B were taken is captured. Operation change information indicating is output.

The image conversion unit 52 converts the image of each group area image grouped by the grouping unit 4 based on the operation change information output from the operation information setting unit 51 and the camera change information output from the motion position information setting unit 11. To implement.
The motion prediction unit 53 searches the prediction reference image memory 6 for a prediction reference image corresponding to each group region image after image conversion by the image conversion unit 52 in consideration of the characteristics of each group region image, and performs prediction reference. A motion of each group region image is predicted using the image, and a motion vector indicating the prediction result is output.
The motion position information setting unit 11, the operation information setting unit 51, the image conversion unit 52, and the motion prediction unit 53 constitute a motion prediction unit.

Next, the operation will be described.
Except for the motion position information setting unit 11, the operation information setting unit 51, the image conversion unit 52, and the motion prediction unit 53, it is the same as in the first embodiment.
When the motion position information setting unit 11 receives motion position information indicating the motion and position of the cameras 1 and 2, the motion position information setting unit 11 stores the motion position information for the necessary pictures. That is, every time the movement position information is input, the latest movement position information is overwritten and recorded on the oldest movement position information.
The motion position information setting unit 11 uses the motion position information at the time when the predicted reference image is captured and the motion position information at the time when the images A and B are captured, from the time when the predicted reference image is captured. The camera change information indicating the change of the cameras 1 and 2 is output to the image conversion unit 52 by capturing the changes of the cameras 1 and 2 up to the present time when B is photographed.

When the operation information setting unit 51 inputs operation information indicating the operation of the cameras 1 and 2, the operation information setting unit 51 stores the operation information for the necessary pictures. That is, each time operation information is input, the latest operation information is overwritten and recorded on the oldest operation information.
The operation information setting unit 51 captures images A and B from the time when the predicted reference image is captured from the operation information at the time when the predicted reference image is captured and the operation information at the time when images A and B are captured. The change in the operation of the cameras 1 and 2 up to the present time is captured, and operation change information indicating the change in the operation of the cameras 1 and 2 is output to the image conversion unit 52.

When the image conversion unit 52 receives the camera change information from the motion position information setting unit 11 and receives the operation change information from the operation information setting unit 51, the image conversion unit 52 outputs the camera change information based on the camera change information and the operation change information. Image conversion of each group area image is performed.
A specific example of the image conversion processing in the image conversion unit 52 is described in the following nineteenth embodiment.

The motion prediction unit 53 searches for a prediction reference image corresponding to each group region image by a method similar to that of the motion prediction unit 7 in FIG. 1, and predicts the motion of each group region image using the prediction reference image.
However, unlike the motion prediction unit 7 in FIG. 1, the motion prediction unit 53 searches for a prediction reference image corresponding to each group region image after image conversion by the image conversion unit 52.

  As is apparent from the above, according to the eighteenth embodiment, each group region image is based on the movement position information indicating the movement and position of the cameras 1 and 2 and the operation information indicating the operation of the cameras 1 and 2. Since the image conversion is performed and the movement of each group area image after the image conversion is predicted, even if the cameras 1 and 2 are operated, the search efficiency (motion of the prediction reference image corresponding to each group area image is determined. This has the effect of improving the vector search efficiency.

Embodiment 19. FIG.
In the eighteenth embodiment, the operation information setting unit 51 inputs operation information indicating the operation of the cameras 1 and 2. However, for example, operation information indicating zoom in, zoom out, pan, rotation, focus position, and the like. May be input.
For example, when the cameras 1 and 2 are zoomed in, the group area image is enlarged, so that the group area image becomes larger than the corresponding predicted reference image.
In view of this, the image conversion unit 52 matches the group area image after zooming in with the image size of the corresponding prediction reference image, and the reciprocal of the enlargement ratio from the prediction reference image with respect to the group area image output from the grouping unit 4. To reduce the group area image, and output the reduced group area image to the motion prediction unit 53.

On the contrary, when the cameras 1 and 2 are zoomed out, the group area image is reduced, so that the group area image becomes smaller than the corresponding predicted reference image.
Therefore, the image conversion unit 52 adjusts the reduction ratio from the predicted reference image to the group region image output from the grouping unit 4 in order to match the group region image after zoom-out with the image size of the corresponding predicted reference image. The group area image is enlarged by multiplying by the inverse, and the enlarged group area image is output to the motion prediction unit 53.
Note that, when the zoom-in or zoom-out is performed, the parallax changes, so that the identifier information of the group region image and the identifier information of the predicted reference image are associated according to the magnification.
Similarly, since the parallax changes by changing the focal position, the identifier information of the group region image and the identifier information of the predicted reference image are associated according to the focal position.

When the cameras 1 and 2 rotate, the image conversion of the group area image differs between when the cameras 1 and 2 rotate at a fixed position and when the cameras 1 and 2 rotate while moving.
Even when the cameras 1 and 2 rotate while moving, the image conversion of the group area image differs between when the focus is on the target and when the focus is at a different location from the target.
When the focus is on the object, the group area image that is the object is rotated in reverse by the rotation angle of the cameras 1 and 2 so that the predicted reference image corresponding to the group area image is oriented in the same direction. Become.
As for the other rotations of the cameras 1 and 2, the rotation angle of the group area image is determined based on the rotation angle of the cameras 1 and 2 and the position information.

  Further, when the cameras 1 and 2 are panned, if the cameras 1 and 2 are not rotated, image conversion is not particularly performed, and the group area image is regarded as a converted image and is output as it is. In panning, an assumed motion vector is obtained from the relationship between the movement distances of the cameras 1 and 2 and the moving pixels in the screen, so that the motion prediction unit 53 outputs the assumed motion vector to the motion prediction unit 53 from the motion position information. Then, an efficient motion vector search can be performed using the deemed motion vector as a reference (center) of the search range.

  As apparent from the above, according to the nineteenth embodiment, the operation information setting unit 51 is configured to input operation information indicating zoom-in, zoom-out, pan, rotation, focus position, etc. Even if the zoom-in / zoom-out / pan / rotation / focus position operations are performed, the search reference image search efficiency (motion vector search efficiency) corresponding to each group region image can be improved. .

In the nineteenth embodiment, the operation information setting unit 51 inputs the operation information indicating zoom-in, zoom-out, pan, rotation, focus position, etc., but the present invention is not limited to this. Information may be input.
In the nineteenth embodiment, the image conversion unit 52 performs the image conversion for rotating, enlarging, or reducing the group area image. However, the present invention is not limited to this, and other image conversion is performed. Also good.
Further, in the nineteenth embodiment, when the operation information is rotation, the image conversion unit 52 rotates the group region image. However, when the image conversion unit 52 rotates the group region image, the group region image is rotated. The group area image may be interpolated by using an image of a portion where the image is not visible (for example, the side surface of the group area image).
In this case, the group region image may be interpolated using an image captured by another camera (for example, the side surface of the group region image), or an image stored in the database in advance (for example, the group region image) The group area image may be interpolated using the side surface.

Embodiment 20. FIG.
In the above eighteenth and nineteenth embodiments, the image conversion unit 52 performs the image conversion of the group region image. However, the prediction reference image (prediction corresponding to the group region image) stored in the prediction reference image memory 6 has been described. The image conversion of the reference image) may be performed to match the image sizes of the group region image and the predicted reference image, and the same effects as in the above-described eighteenth and nineteenth embodiments can be achieved.

When the image conversion unit 52 rotates the predicted reference image, the predicted reference image may be interpolated using an image of a portion where the predicted reference image is not visible (for example, a side surface of the predicted reference image).
In this case, the prediction reference image may be interpolated using an image captured by another camera (for example, a side surface of the prediction reference image), or an image stored in the database in advance (for example, the prediction reference image of the prediction reference image). The prediction reference image may be interpolated using the side surface.

Embodiment 21. FIG.
FIG. 18 is a block diagram showing a stereo image coding apparatus according to Embodiment 21 of the present invention. In the figure, the same reference numerals as those in FIG.
The encoding unit 61 performs encoding processing similar to that of the encoding unit 8 of FIG. 1, and also includes camera change information (or camera motion position information) output from the motion position information setting unit 11 and an operation information setting unit 51. The operation change information (or camera operation information) output from the image conversion information and the image conversion information indicating the image conversion by the image conversion unit 52 are encoded. Note that the encoding unit 61 constitutes encoding means.

In Embodiments 1 to 20 described above, the encoding unit 8 obtains a difference image between the group region image and the prediction reference image included in the images A and B, and encodes the difference image and the motion vector. However, in the twenty-first embodiment, the encoding unit 61 obtains a difference image between the group region image after the image conversion by the image conversion unit 52 and the predicted reference image, and encodes the difference image and the motion vector.
The encoding unit 61 also includes camera change information (or camera motion position information) output from the motion position information setting unit 11, operation change information (or camera operation information) output from the operation information setting unit 51, The image conversion information indicating the image conversion by the image conversion unit 52 is also encoded.

  As is apparent from the above, according to the twenty-first embodiment, the camera change information (or camera movement position information) output from the movement position information setting unit 11 and the operation change information output from the operation information setting unit 51. (Or camera operation information) and image conversion information indicating image conversion by the image conversion unit 52 are encoded, so that when the cameras 1 and 2 are operated to perform image conversion of a group area image However, there is an effect that it is possible to realize motion compensation predictive coding close to actual motion.

  In Embodiment 21, the encoding unit 61 encodes the image conversion information. However, when the image conversion unit 52 does not perform the image conversion of the group area image, the image conversion information is displayed. There is no need to encode.

Embodiment 22. FIG.
Although not particularly mentioned in Embodiments 1 to 21, when encoding unit 8 (or encoding unit 61) encodes images A and B, group regions included in images A and B The image may be divided into blocks and encoding may be performed for each block.
At this time, for blocks in which a plurality of group area images included in the images A and B exist, encoding is performed in consideration of the attributes of the plurality of group area images.
Specifically, it is as follows.

In the twenty-second embodiment, the group area images included in the images A and B are divided into 8 × 8 pixel blocks, and encoding is performed for each block.
In the fourth embodiment, as shown in FIG. 9A, the representative motion vector is obtained including the block including the boundary of the group area image. However, in the twenty-second embodiment, a plurality of blocks are included in a certain block. When group area images exist, encoding is performed in consideration of attributes of a plurality of group area images.
In the example of FIG. 9A, for the block including the gray area and the white area in the drawing, the encoding is controlled based on the attribute of the gray area and the attribute of the white area.
This is because the coding control is individually performed for each of the gray area in the block, the white area in the block, and the gray and white area in the block.

  As is apparent from the above, according to the twenty-second embodiment, a block in which a plurality of group area images are included in the images A and B is coded in consideration of the attributes of the plurality of group area images. Therefore, it is possible to individually perform coding control for regions having different attributes, and as a result, it is possible to perform coding control according to the purpose. That is, with respect to the attention area, when all the images in the block are the attention area images, a large amount of encoding information is allocated, and when the attention area and the image that is not the attention area are mixed in the block, In comparison, the amount of encoded information to be allocated is reduced, and when all the images are not in the region of interest, it is possible to allocate a small amount of encoded information.

In the twenty-second embodiment, the group area image included in the images A and B is divided into 8 × 8 pixel blocks. However, the number of divided pixels may be arbitrary, and the motion prediction block The size and the block size for encoding may be different.
In the twenty-second embodiment, for the attention area, if the entire block is an image of the attention area, a large amount of encoding information is allocated, and if the attention area and the image that is not the attention area are mixed in the block, all The amount of encoding information to be allocated is reduced compared to the case of the image of the attention area, and if all the images of the attention area are not described, it is described that the amount of encoding information can be allocated. The assignment of may be arbitrary.

As an attribute of the group region image, for example, an area in the block can be used. Further, the attribute of the group area image may be parallax, characteristic information, the median value of the area of the group area image corresponding to the identifier in the block, or the median value of parallax. Further, it may be a positional relationship in the depth direction with the group area image that is in focus.
The attribute of the group region image determines the weighting coefficient based on the area of each group corresponding to the identifier in the block, and corresponds to the weighting coefficient closest to the average value of the weighting coefficient or the square root of the squared average value of the weighting coefficient. It may be an attribute of a group area image.

It is a block diagram which shows the stereo image coding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the stereo image coding method by Embodiment 1 of this invention. It is explanatory drawing which shows the image A and B image | photographed with the cameras 1 and 2, and the parallax in a screen. It is explanatory drawing which shows a group area | region image. It is explanatory drawing which shows a group area | region image. It is explanatory drawing explaining the search range of an image. It is explanatory drawing explaining the search range of an image. It is explanatory drawing explaining a representative motion vector. It is explanatory drawing explaining the search method of a representative motion vector. It is a block diagram which shows the stereo image coding apparatus by Embodiment 5 of this invention. It is explanatory drawing which shows the movement of the cameras 1 and 2, a search range, and an apparent motion vector. It is explanatory drawing which shows the rotation in the fixed position of the cameras 1 and 2, a search range, and an apparent motion vector. FIG. 4 is an explanatory diagram showing a search range and an apparent motion vector when the cameras 1 and 2 are zoomed in and out when the positions of the cameras 1 and 2 are not changed and the cameras 1 and 2 are not rotating. is there. It is a block diagram which shows the stereo image coding apparatus by Embodiment 10 of this invention. It is a block diagram which shows the stereo image coding apparatus by Embodiment 12 of this invention. It is a block diagram which shows the stereo image coding apparatus by Embodiment 16 of this invention. It is a block diagram which shows the stereo image coding apparatus by Embodiment 18 of this invention. It is a block diagram which shows the stereo image coding apparatus by Embodiment 21 of this invention.

Explanation of symbols

  1, 2 cameras, 3 parallax calculation unit (parallax calculation unit), 4 grouping unit (grouping unit, characteristic extraction unit), 5 identifier information addition unit, 6 prediction reference image memory, 7, 13, 23, 32, 43, 53 Motion prediction unit (motion prediction unit), 8, 61 encoding unit (encoding unit), 11 motion position information setting unit (motion prediction unit), 12 search range control unit (motion prediction unit), 21 reference search range setting unit (Motion prediction means), 22 search range coefficient setting section (motion prediction means), 31 apparent motion vector control section (motion prediction means), 41 reference apparent motion vector setting section (motion prediction means), 42 apparent motion vector coefficient setting section (Motion prediction means), 51 operation information setting section (motion prediction means), 52 image conversion section (motion prediction means).

Claims (26)

  A plurality of images taken at the same time are input, and a parallax calculation unit that calculates a parallax in the screen of the plurality of images, and a region in the screen are grouped according to the parallax calculated by the parallax calculation unit Grouping means, characteristic extracting means for extracting the characteristics of each area grouped by the grouping means, and movement of each area grouped by the grouping means in consideration of the characteristics extracted by the characteristic extracting means A stereo image encoding apparatus comprising: a motion prediction unit that predicts the image; and an encoding unit that encodes the plurality of images using a motion prediction result of each region by the motion prediction unit.   2. The stereo image encoding apparatus according to claim 1, wherein the characteristic extraction unit extracts a distribution state, a size, a shape, a pattern, or a color of each region as a characteristic of each region grouped by the grouping unit. .   The motion prediction unit is configured to group the group by the grouping unit in consideration of the characteristics extracted by the characteristic extraction unit from the prediction reference image memory storing the prediction reference images taken at different times from the plurality of images. The stereo image code according to claim 1 or 2, wherein a prediction reference image corresponding to each divided area is searched, and a motion of each area is predicted using the prediction reference image. Device.   When searching for prediction reference images corresponding to the respective areas grouped by the grouping means, the motion predicting means controls the search range of the image to a range in which each designated area overlaps with a preset designated area. The stereo image encoding device according to claim 3, wherein:   When searching for prediction reference images corresponding to the respective areas grouped by the grouping means, the motion predicting means controls the search range of the image to a range including a portion where each designated area overlaps with a preset designated area. The stereo image coding apparatus according to claim 3, wherein:   The motion prediction means obtains a representative motion vector of each area grouped by the grouping means, and when searching for a prediction reference image corresponding to each area, the search range of the image is based on the representative motion vector. The stereo image coding apparatus according to claim 3, wherein the range is controlled to a range.   The motion prediction means inputs motion position information indicating the motion and position of a camera taking a plurality of images, and when searching for prediction reference images corresponding to the respective areas grouped by the grouping means, 4. The stereo image encoding apparatus according to claim 3, wherein an image search range is controlled in accordance with a camera movement and position indicated by the movement position information.   8. The stereo according to claim 7, wherein the motion prediction means inputs motion position information indicating a horizontal rotation angle or a vertical rotation angle of the camera and indicating a moving distance of one or more directions in the three-dimensional direction of the camera. Image encoding device.   When the camera is moved while rotating while the base point is located at the center of the image with the base point at infinity as the base point, the motion prediction unit is for prediction reference corresponding to each area grouped by the grouping unit. 9. The stereo image encoding apparatus according to claim 7, wherein when the image is searched, the search range of the image is controlled to be a smaller range as the parallax is smaller.   When the camera is rotated without changing the position, the motion prediction unit searches for the prediction reference image corresponding to each region grouped by the grouping unit, and the search range of the image is a region with a large parallax. 9. The stereo image encoding apparatus according to claim 7, wherein the stereo image encoding device is controlled to be as small as possible.   When the camera predicts zoom-in and zoom-out, when searching for a prediction reference image corresponding to each area grouped by the grouping means, the motion prediction means has a smaller search range for the image as the parallax is smaller. 9. The stereo image encoding device according to claim 7, wherein the stereo image encoding device is controlled to a range.   When searching for prediction reference images corresponding to the respective areas grouped by the grouping means, the motion prediction means sets a search range of the prediction reference images corresponding to the reference area in each area. A reference search range setting unit; and a search range coefficient setting unit for setting search range coefficients for other regions, and corresponding to other regions from the setting contents of the reference search range setting unit and the search range coefficient setting unit. 4. The stereo image encoding apparatus according to claim 3, wherein a search range of an image for prediction reference is determined.   The reference search range setting unit sets the search reference image search range corresponding to the region in which the camera is focused among the regions grouped by the grouping unit to the prediction reference image corresponding to the reference region. The stereo image encoding device according to claim 12, wherein the stereo image encoding device is set as a search range.   The motion prediction means predicts reference from motion position information indicating the motion and position of the camera at the time when the image for prediction reference is captured, and motion position information indicating the motion and position at the time when the plurality of images are captured. Recognizing a change in the camera from the time when the image is taken to the time when a plurality of images are taken, calculating the apparent motion vector from the change in the camera and the parallax calculated by the parallax calculating means, and the apparent motion 4. The stereo image encoding apparatus according to claim 3, wherein an image for prediction reference corresponding to each area grouped by the grouping means is searched in consideration of the vector.   When the camera is moved while rotating while the base point is located at the center of the image with the base point at infinity as the base point, the motion prediction unit is for prediction reference corresponding to each area grouped by the grouping unit. 15. The stereo image encoding apparatus according to claim 14, wherein when the image is searched, the apparent motion vector is reduced in a region where the parallax is small.   When the camera is rotated without changing the position, the motion prediction unit searches for a prediction reference image corresponding to each region grouped by the grouping unit, and the apparent motion vector is calculated for a region having a larger parallax. 15. The stereo image encoding device according to claim 14, wherein the stereo image encoding device is made small.   When the camera is zoomed in and zoomed out, when searching for prediction reference images corresponding to the respective areas grouped by the grouping means, the motion prediction unit decreases the apparent motion vector in a region with a smaller parallax. The stereo image encoding device according to claim 14.   The motion prediction means, when searching for prediction reference images corresponding to the respective areas grouped by the grouping means, a reference apparent motion vector setting section for setting an apparent motion vector only as a reference area in each area; An apparent motion vector coefficient setting unit for setting an apparent motion vector coefficient for other regions, and an apparent motion vector for other regions is set based on the setting contents of the reference apparent motion vector setting unit and the apparent motion vector coefficient setting unit. The stereo image encoding device according to claim 14, wherein the stereo image encoding device is determined.   The reference apparent motion vector setting unit is configured to set an apparent motion vector as an apparent motion vector corresponding to a reference region, among the regions grouped by the grouping unit, in the region where the camera is focused. The stereo image encoding device according to claim 18.   The motion prediction means performs image conversion of each region grouped by the grouping means based on motion position information indicating the motion and position of a camera that has captured a plurality of images and operation information indicating the operation of the camera. The stereo image encoding apparatus according to claim 1, wherein the stereo image encoding device is implemented and predicts the motion of each region after image conversion.   The motion prediction means is a prediction corresponding to each area grouped by the grouping means based on motion position information indicating the motion and position of a camera taking a plurality of images and operation information indicating the operation of the camera. 4. The stereo image encoding apparatus according to claim 3, wherein image conversion of a reference image is performed, and motion of each region is predicted using a prediction reference image after image conversion.   The stereo image encoding device according to claim 20 or 21, wherein the encoding means encodes motion position information, camera operation information, or image conversion information indicating image conversion.   When encoding a plurality of images using the motion prediction results of each region by the motion prediction unit, the encoding unit divides the image into blocks and performs encoding for each block. The stereo image encoding device according to claim 1 or 2.   24. The stereo image code according to claim 23, wherein the encoding means performs encoding in consideration of attributes of a plurality of areas for a block in which a plurality of areas grouped by the grouping means exist. Device.   A parallax calculation unit that inputs a plurality of images taken at the same time by the parallax calculation unit and calculates a parallax within the screen in the plurality of images, and a grouping unit according to the parallax calculated by the parallax calculation unit Considering the grouping step for grouping the areas in the group, the characteristic extraction step for the characteristic extraction means to extract the characteristics of each area grouped by the grouping means, and the characteristics for the motion prediction means extracted by the characteristic extraction means Then, the motion prediction step for predicting the motion of each region grouped by the grouping unit, and the encoding unit encodes the plurality of images using the motion prediction result of each region by the motion prediction unit. A stereo image encoding method comprising: an encoding step to be performed.   A plurality of images taken at the same time are input, a parallax calculation processing procedure for calculating the parallax in the screen in the plurality of images, and a region in the screen according to the parallax calculated by the parallax calculation processing procedure are grouped Considering the characteristics extracted by the grouping process procedure, the characteristic extraction process procedure for extracting the characteristics of each area grouped by the grouping process procedure, and the characteristic extraction process procedure, the grouping process procedure is performed. The computer executes a motion prediction processing procedure for predicting the motion of each divided region and a coding processing procedure for encoding the plurality of images using the motion prediction result of each region by the motion prediction processing procedure. Stereo image encoding program for making

Images