JPH0937264A - Image coder, image coding method, image decoder, image decoding method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the image compression rate by multiplexing chain coding data and a motion vector. SOLUTION: A chain coding circuit 12 applies chain coding to a moving image and chain coded data obtained as a result are given to a RAM 14, in which the data are stored. A similar chain detection circuit 15 references the stored data in the RAM 14 to detect chains tying characteristic points on an image and chains forming a same contour in existence over plural frames are connected. Furthermore, the similar chain detection circuit 15 detects motion vectors among connected chains and the result is given to a select multiplexer circuit 16. The select multiplexer circuit 16 multiplexes chain coded data as to a chain among the chains connected by the similar chain detection circuit 15 onto a motion vector outputted from the similar chain detection circuit 15 and provides an output of the result.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding device and an image encoding method, an image decoding device and an image decoding method, and a recording medium. In particular, it is possible to reduce the generated code amount by multiplexing the data obtained by encoding the image data relating to the feature points of the moving image and the motion vector of the chain connecting the feature points. The present invention relates to an encoding device and an image encoding method, an image decoding device and an image decoding method, and a recording medium.

[0002]

2. Description of the Related Art Conventionally, in an apparatus for transmitting an image under a limited transmission band or an apparatus for recording on a recording medium having a limited storage capacity, the amount of image data is reduced. A high-efficiency coding method for efficiently compressing with code words is adopted. As a high-efficiency image coding method, for example, an input image is orthogonally transformed by DCT (discrete cosine transform) and adaptive quantization is performed for each frequency band according to human visual characteristics, and a wavelet basis ( A method is known in which an image is divided into subbands by wavelet transformation, and each band is weighted and encoded. According to these encoding methods, distortion is visually inconspicuous, and
A high compression rate can be realized.

However, in these encoding systems, if the compression ratio is further improved, the visually unfavorable effects such as so-called block distortion become remarkable.

Therefore, as an encoding method which does not cause visually harmful distortion even at a high compression rate, for example, characteristic points (characteristic points) of the structure of an image (for example, points (pixels) forming edges) are used. 2. Description of the Related Art There is known a structure extraction coding method by extracting a characteristic point of an image and efficiently encoding image data at the characteristic point.

FIG. 56 shows the structure of an example of a conventional image coding apparatus for compressing and coding an image by the structure extraction coding method. The image data is input to the quantizer 11 and the two-dimensional change point detection circuit 10. In the quantizer 11, the image data is quantized, and the quantized coefficient is obtained. This quantized coefficient is supplied to the chain (chain) encoding circuit 12. Further, the two-dimensional change point detection circuit 10 detects whether or not the input image data is a feature point, and if the input image data is a feature point, for example, the feature point data having a value of 1 is If it is not a feature point, feature point data having a value of 0 is output to the chain encoding circuit 12, respectively.

When the quantization process in the quantizer 11 and the feature point detection process in the two-dimensional change point detection circuit 10 are completed for one frame of image data,
In the chain encoding circuit 12, the position information (position (coordinates) of the feature point) about the point (pixel) having the feature point data of 1 is chain-encoded, and further the quantized coefficient of the feature point (detected as the feature point is detected. It is multiplexed with the quantized coefficient of the image data in the pixel) to form chain encoded data. The chain-encoded data is supplied to the buffer (buffer memory) 17, is temporarily stored, and then recorded on a recording medium (not shown) or output to the transmission path. Note that the chain-encoded data is temporarily stored in the buffer 17 so that the amount of data is smoothed,
As a result, the chain encoded data is output to the recording medium or the transmission path at a substantially constant data rate.

[0007]

By the way, in the conventional image coding apparatus which performs the coding by detecting the feature points as described above, since the coding target is the still image, the moving image is targeted. When the encoding is performed, there is a problem that the compression rate is worse than that of the transform encoding method including the motion compensation processing and the DCT processing for removing the redundant component in the time direction.

The present invention has been made in view of such a situation, and it is possible to improve the compression rate when the image is compression-encoded by the structure extraction encoding method.

[0009]

An image coding apparatus according to the present invention detects a chain connecting feature points and a feature point coding unit that encodes information about feature points of a moving image and outputs coded data. Chain detection means, motion vector detection means for detecting the motion vector of the chain detected by the chain detection means, coded data output from the feature point coding means, and motion vector detected by the motion vector detection means And multiplexing means for multiplexing.

The image coding apparatus further comprises a deciding means for connecting chains existing in different frames and deciding the coded data and the motion vector to be multiplexed by the multiplexing means based on the connection result. You can The determining means may calculate the similarity between chains existing in adjacent frames, and connect the chains based on the similarity. Also,
The deciding means multiplexes the coded data corresponding to a predetermined one of the chains connected over the plurality of frames and the motion vector between adjacent frames of the plurality of frames by the multiplexing means. You can Further, the determining means determines the similarity between a chain A existing in a predetermined frame and another chain B existing in an adjacent frame adjacent to the predetermined frame by using a motion vector from the predetermined frame to the adjacent frame. According to the above, the chain A is moved, and the pixel can be calculated based on the number of the pixels constituting the chain A after the movement, which are closest to the chain B among the chains existing in the adjacent frame. In addition, the determination means includes the similarity SA of the chain A to the chain B.
And the similarity SB of the chain B to the chain A are calculated, and the chains A and B can be connected when the similarity SA and SB are both equal to or greater than a predetermined threshold value.

When the chain C exists in one of the adjacent frames and the plurality of chains D1, D2, D3, ... C, multiple chains D
1, D2, D3, ... Similarity SC to each
1, SC2, SC3, ... and a plurality of chains D1,
The similarity SD1, SD2, SD3, ... For each chain D2, D3, ... Is calculated, and the similarity SD1, SD2, SD3 ,. And the similarity SC1, SC2, SC3
When the sum total of ... Is greater than or equal to a predetermined threshold, the chain C
, And the chains D1, D2, D3, ... Can be connected to each other.

Further, the image coding apparatus of the present invention moves a chain A existing in a predetermined frame according to a motion vector from the predetermined frame to an adjacent frame adjacent to the chain A, and after moving the chain. Of chain A,
When the importance calculated by the importance calculating means for calculating the importance indicating the visual importance in the adjacent frame and the importance calculated by the importance calculating means is equal to or more than a predetermined threshold, the chain in the adjacent frame is moved. It is possible to further include replacement means for replacing what is around the subsequent chain A with the moved chain A, in which case the multiplexing means includes the encoded data corresponding to the chain A and the replacement means. It is possible to multiplex the motion vector to the adjacent frame in which the chain is replaced by.

The importance calculating means can calculate the importance based on the characteristic points existing around the chain A after the movement in the adjacent frames. Further, the importance calculating means can calculate the importance based on the edge strength on the chain A after the movement in the adjacent frame.

The image encoding method of the present invention encodes data relating to feature points of a moving image, detects a motion vector of a chain connecting the feature points, and multiplexes the encoded data and the motion vector. It is characterized by doing.

The image decoding apparatus of the present invention comprises: an extraction unit for extracting, from the transmission data, encoded data obtained by encoding data relating to feature points of a moving image and a motion vector of a chain connecting the feature points; It is characterized by comprising a decoding means for decoding the data and a motion compensation means for compensating the output of the decoding means according to the motion vector.

According to the image decoding method of the present invention, the encoded data obtained by encoding the data relating to the characteristic points of the moving image and the motion vector of the chain connecting the characteristic points are extracted from the transmission data, and the encoded data is decoded. The decoded data is then subjected to motion compensation according to the motion vector.

The recording medium of the present invention is characterized in that the data recorded therein includes encoded data obtained by encoding data relating to characteristic points of a moving image and a motion vector of a chain connecting the characteristic points. To do.

In the image coding apparatus of the present invention, the characteristic point coding means codes the information relating to the characteristic points of the moving image and outputs the coded data. The chain detecting means detects the chain connecting the characteristic points, and the motion vector detecting means detects the motion vector of the chain detected by the chain detecting means. The multiplexing means is configured to multiplex the coded data output from the feature point coding means and the motion vector detected by the motion vector detecting means.

In the image coding method of the present invention, the data relating to the feature points of the moving image is coded, the motion vector of the chain connecting the feature points is detected, and the coded data and the motion vector are multiplexed. It is designed to change.

In the image decoding apparatus of the present invention, the extraction means extracts, from the transmission data, the encoded data obtained by encoding the data relating to the feature points of the moving image and the motion vector of the chain connecting the feature points, The decoding means is adapted to decode the encoded data. The motion compensation means is adapted to perform motion compensation on the output of the decoding means according to the motion vector.

In the image decoding method of the present invention, the encoded data obtained by encoding the data relating to the characteristic points of the moving image and the motion vector of the chain connecting the characteristic points are extracted from the transmission data, and the encoded data is extracted. Decoding is performed as decoded data, and the decoded data is motion-compensated according to the motion vector.

In the recording medium of the present invention, the data recorded therein includes encoded data obtained by encoding the data relating to the characteristic points of the moving image and the motion vector of the chain connecting the characteristic points.

[0023]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below, but before that, in order to clarify the correspondence between each means of the invention described in the claims and the following embodiments. The features of the present invention are described as follows by adding a corresponding embodiment (however, an example) in parentheses after each means.

That is, the image coding apparatus according to the first aspect is an image coding apparatus for coding a moving image, and the characteristic point for coding the information about the characteristic point of the moving image and outputting the coded data. Encoding means (for example, the chain encoding circuit 12 shown in FIG. 1), chain detection means for detecting a chain connecting characteristic points (for example, chain map circuit 51 shown in FIG. 18), and chain detection means Motion vector detecting means for detecting the motion vector of the generated chain (for example, the motion vector calculating circuit 54 shown in FIG. 18), the encoded data output from the feature point encoding means, and the motion vector detecting means. It is characterized by including a multiplexing means (for example, the select multiplexing circuit 16 shown in FIG. 1) for multiplexing the motion vector.

The image coding apparatus according to a second aspect of the present invention connects the chains existing in different frames, and determines the coded data and the motion vector to be multiplexed by the multiplexing means based on the connection result. Means (eg,
The chain connection circuit 43 shown in FIG. 17) is further provided.

The image coding apparatus according to the present invention moves a chain A existing in a predetermined frame according to a motion vector from the predetermined frame to an adjacent frame adjacent to the predetermined frame, and after the movement. Chain A
Of the importance degree calculating means for calculating the importance degree indicating the visual importance in the adjacent frame (for example, processing step S144 of the program shown in FIG. 52) and the importance degree calculated by the importance degree calculating means are predetermined. When it is equal to or larger than the threshold value, a replacement unit that replaces the chain around the moved chain A among the chains existing in the adjacent frames with the moved chain A (for example, processing step S146 of the program shown in FIG. 52). ) Is further provided, the multiplexing means multiplexes the encoded data corresponding to the chain A and the motion vector to the adjacent frame in which the chain has been replaced by the replacement means.

An image decoding apparatus according to a twelfth aspect is an image decoding apparatus for decoding a moving image, wherein encoded data obtained by encoding data relating to a characteristic point of the moving image from transmission data and the characteristic point. Extraction means (for example, the separation circuit 72 shown in FIG. 39) for extracting the motion vector of the chain connecting the two and decoding means (for example, the separation circuit 72 shown in FIG. 39)
Motion compensation means (for example, the motion compensation circuit 77 shown in FIG. 39) for compensating the outputs of the chain decoding circuit 73 and the like shown in FIG. 39 and the decoding means according to the motion vector.
Etc.) and are provided.

Of course, this description does not mean that each means is limited to the above.

FIG. 1 is a block diagram showing the configuration of a first embodiment of an image coding apparatus to which the present invention is applied. Note that, in the figure, parts corresponding to those in FIG. 56 are denoted by the same reference numerals.

This image coding apparatus is adapted to compress and code a moving image by the structure extraction coding method. That is, for example, input image data obtained by digitizing an image signal of a moving image is transmitted through the two-dimensional change point detection circuit 10 or the quantizer 11 as described with reference to FIG.
By being supplied to the chain encoding circuit 12, it becomes chain encoded data. This chain encoded data is
It is supplied to the selector 13. The selector 13 stores the chain-encoded data in units of frames in a RAM (Chain Code).
RAM) 14 _{1 to} 14 _N.

That is, the similar chain detection circuit 15 and the select multiplex circuit 16 in the subsequent stage are designed to perform processing in N frame units (for example, 15 frame units), and the RAM 14 ₁ stores them in N frame units. the first frame of the chain coded data (the first frame) of the ones stored, also the RAM 14 _2, the second
The chain-coded data of the frame is stored, and in the same manner, the chain-coded data of the nth frame is stored in the RAM 14 _n (where n = 1, 2, ...
., N).

When N frames of chain encoded data are stored in the RAMs 14 _{1 to} 14 _N , the similar chain detection circuit 15 reads the chain encoded data stored in the RAM 14 _n and refers to it. As a result, a chain that is a series of consecutive feature points existing in each of the first frame to the Nth frame is detected. Further, the similar chain detection circuit 15 detects chains having a high degree of similarity among the chains between the frames, connects them (corresponds them as representing the contour of the same object), and detects the motion vector between the connected chains. To detect.

Then, the similar chain detection circuit 15 determines a so-called basic chain (hereinafter appropriately referred to as a basic chain) from the connected chains, and determines the frame number (the book) of the frame in which the basic chain exists. In the embodiment, when the frame in which the chain exists is the nth frame, for example, n is the frame number), and the chain number of the basic chain (as described later,
Each chain existing in each frame is given a unique number for distinguishing it from other chains. In the present embodiment, for example, this number is used as a chain number) , And outputs the motion vector to the chain connected to it, which exists in another frame, to the select multiplexing circuit 16.

Upon receiving the frame number, the chain number, and the motion vector from the similar chain detection circuit 15, the select multiplexing circuit 16 determines the chain corresponding to the chain number among the chains existing in the frame corresponding to the frame number. Chain encoded data to RA
The data is read from any of M141 to 14N, the chain encoded data thereof and the motion vector from the similar chain detection circuit 15 are multiplexed and output as multiplexed chain encoded data. This multiplexed chain encoded data is
It is output to the transmission line via the buffer 17. Alternatively, it is supplied to and recorded on a recording medium 18 such as a magnetic tape, an optical disc, or a magneto-optical disc.

FIG. 2 shows a configuration example of the chain encoding circuit 12 of FIG. The quantized coefficient from the quantizer 11 is supplied to and stored in the coefficient frame buffer 21. In the coefficient frame buffer 21, the quantization coefficient Qij of the image data at the point (i, j) on the image is represented by i or j, respectively, as shown in FIG. The data is stored in the address (however, each i or j represents, for example, the horizontal (x-axis direction) or horizontal (y-axis direction) coordinate of a pixel (point) forming an image).

The feature point data from the two-dimensional change point detection circuit 10 is supplied to the mask frame buffer 22 and stored therein. Note that the mask frame buffer 22 also stores the feature point data in the same manner as in the coefficient frame buffer 21 described above.

When one frame of quantized coefficients or feature point data is stored in the coefficient frame buffer 21 or the mask frame buffer 22, respectively, the direction searcher 23 performs the processing according to the flowchart shown in FIG. Be seen.

That is, first, in step S1, the status register 24, the X coordinate register 25, and the Y register.
The coordinate register 26 is initialized to 0, and the process proceeds to step S2 to determine whether the stored value of the state register 24 is 0 or not. When it is determined in step S2 that the value stored in the state register 24 is 0, the process proceeds to step S3, and it is determined whether there are unprocessed coordinates. That is,
The direction searcher 23 determines X in step S4 described later.
In the coordinate register 25 or the Y coordinate register 26, x or y coordinates of pixels forming one frame image are set in, for example, a so-called line scan order (from left to right and from top to bottom). In step S3, it is determined whether the x or y coordinate of the pixel located at the end when the image is viewed in the line scan order is set in the X coordinate register 25 or the Y coordinate register 26 (for example, When the image of one frame has 640 × 400 pixels, it is determined whether 640 or 400 is set in the X coordinate register 25 or the Y coordinate register 26).

If it is determined in step S3 that there are unprocessed coordinates, the process proceeds to step S4, and the first unprocessed coordinate (x, y) that appears when the image is viewed in line scan order.
(Coordinates) are set in the X coordinate register 25 and the Y coordinate register 26. Therefore, immediately after the quantized coefficient for one frame or the feature point data is stored in the coefficient frame buffer 21 or the mask frame buffer 22, 0 is set in both the X coordinate register 25 and the Y coordinate register 26. It

Here, in the chain encoding circuit 12, X
The pixel of the coordinates corresponding to the stored values of the coordinate register 25 and the Y coordinate register 26 is the target of the process first, but the pixel which is (is) the target of the process is hereinafter referred to as the pixel of interest as appropriate. Say.

After that, the process proceeds to step S5, where the feature point data corresponding to the pixel of interest is the mask frame buffer 22.
Is read out (the feature point data is read out from the address having the x-coordinate or the y-coordinate of the pixel of interest as the upper address or the lower address, respectively), and it is determined whether it is 1. If it is determined in step S5 that the feature point data is not 1 (if it is determined to be 0), that is, if the pixel of interest is not a feature point, the process returns to step S2. If it is determined in step S5 that the feature point data is 1, that is, if the pixel of interest is a feature point, the process proceeds to step S6, and the search X coordinate register 27 or the search Y coordinate register 28 stores the X coordinate. The value stored in the register 25 or the Y coordinate register 26 is set. Further, in step S6, the 3-bit counter 29 is initialized to 0. The 3-bit counter 29 can count a number that can be represented by 3 bits, that is, 0 to 7 (2 ³ −1), for example.

Then, in the direction searching device 23, step S
At 7, the valid data selection signal whose value is 1 (H level) is output. The valid data selection signal is usually
It is set to 0 (L level). Further, the valid data selection signal is supplied to the latch circuit 34, the selector 36, and the multiplexer 37. further,
The valid data selection signal is set to 0 again when the time for one clock elapses after it is changed from 0 to 1.

Further, in step S7, the coordinates stored in the search X coordinate register 27 and the search Y coordinate register 28 are read out, and the selector 3 is set as the start point coordinate.
6 is output. In step S7, 1 is set in the status register 24, and the feature point data having a value of 1 and corresponding to the pixel of interest stored in the mask frame buffer 22 is rewritten to 0.
Return to 2.

Here, the selector 36 has a direction searching device 2
3, the valid data selection signal and the start point coordinates are supplied, the stored value of the state register 24, the ROM (Direction
The output of the ROM 33 and the output of the direction change signal generator 35 are supplied. When the selector 36 receives the valid data selection signal whose value is 1, the selector 36 refers to the value stored in the state register 24, and when it is 0, the start point coordinate from the direction searcher 23 and the output of the ROM 33. , Or the start point coordinate from the direction searching device 23 among the outputs of the direction change signal generator 35, and outputs it to the multiplexer 37.

Further, when the multiplexer 37 receives the valid data selection signal having the value 1, the multiplexer 37 reads the quantized coefficient of the pixel of interest from the coefficient frame buffer 21, multiplexes it with the output of the selector 36, and outputs it. There is.

Therefore, in step S7, when the value stored in the status register 24 is 0, the valid data selection signal becomes 1 (when the pixel of interest is a feature point).
Includes the chain encoding circuit 12 (multiplexer 37)
Outputs chain coded data in which the coordinates (start point coordinates) of the pixel of interest as the feature point and the quantized coefficient at the feature point are multiplexed.

When the value stored in the state register 24 is 0 (hereinafter, appropriately referred to as state 0), when the valid data selection signal becomes 1 is a characteristic feature of the chain. Then, when the starting point was found. Then, in this case, as described above, the stored value of the state register 24 is set to 1 in step S7.

On the other hand, if it is determined in step S2 that the stored value of the status register 24 is not 0, the process proceeds to step S8, and it is determined whether or not the stored value is 1. If it is determined in step S8 that the value stored in the status register 24 is 1, the process proceeds to step S11 in FIG. 5, where the feature point data stored at the address generated by the address generator 32 is the mask frame. It is read from the buffer 22.

Here, the address generator 32 is the adder 3
An address having a value output from 8 or 39 as an upper address or a lower address is output. The adder 38 has a search X coordinate register 2
X coordinate stored in 7 and ROM (Search X-ROM) 3
An output value of 0 is supplied, and both are added and output there. Further, the Y coordinate stored in the search Y coordinate register 28 and the output value of the ROM (Search Y-ROM) 31 are supplied to the adder 39. As in the case, both are also added and output.

The ROMs 30 and 31 have a 3-bit address space in which the values shown in FIG. 6 are stored. The ROMs 30 and 31 use the output value of the 3-bit counter 29 as an address and output the stored value stored at that address.

Therefore, the outputs of adders 38 and 39 are
(Output of adder 38, output of adder 39), and the value stored in search X coordinate register 27 or search Y coordinate register 28 is expressed as x or y, respectively, a 3-bit counter. When the output value of 29 is 0 to 7, the outputs of the adders 38 and 39 are
(X-1, y-1), (x-1, y), (x-, respectively)
1, y + 1), (x, y-1), (x, y + 1), (x
+1, y-1), (x + 1, y), (x + 1, y + 1)
Becomes

Therefore, when the output value of the 3-bit counter 29 changes from 0 to 7, the address generator 32 causes the search X coordinate register 27 and the search Y to be changed as shown in FIG.
Eight pixels P0 adjacent to the pixel of the coordinate represented by the stored value of the coordinate register 28 (the hatched portion in the drawing)
The address corresponding to the coordinates of P7 to P7 is output.

From the above, in step S11, the pixel adjacent to the pixel of the coordinates represented by the stored values of the search X coordinate register 27 and the search Y coordinate register 28 (the pixel corresponding to the count value output by the 3-bit counter 29). The feature point data in any of P0 to P7 will be read from the mask frame buffer 22. here,
The process described below is performed by the search X coordinate register 27 and the search Y.
Based on the pixel of the coordinate represented by the value stored in the coordinate register 28, the pixel adjacent to the pixel is searched (sequentially as a target pixel), so the search X coordinate register 27
The pixel at the coordinate represented by the stored value of the search Y coordinate register 28 will be hereinafter referred to as a search center pixel, as appropriate.

When the characteristic point data is read, the process proceeds to step S12, and it is determined whether or not the value is 1.
If it is determined in step S12 that the feature point data is not 1, that is, if the pixel of interest is not a feature point, the process proceeds to step S13, and it is determined whether the count value (output value) of the 3-bit counter 29 is 7. To be done. If it is determined in step S13 that the count value of the 3-bit counter 29 is not 7, that is, if processing has not been performed for all pixels adjacent to the search center pixel, the process proceeds to step S14. Is incremented by 1 and the process returns to step S2 in FIG. In this case, the value stored in the status register 24 is 1
As it is, the processing from step S11 onward is performed again through step S2 and step S8.

If it is determined in step S13 that the count value of the 3-bit counter 29 is 7, that is, all the pixels adjacent to the search center pixel are processed, and as a result, the search center pixel is adjacent to the search center pixel. When it is determined that the pixel has no feature point, the process proceeds to step S15, 0 is set in the state register 24, and step S2 is performed.
Return to

On the other hand, if it is determined in step S12 that the feature point data is 1, that is, if the target pixel is a feature point, the process proceeds to step S16, and the target pixel is newly set as the search center pixel. It That is, the stored value of the search X coordinate register 27 or the search Y coordinate register 28 is
Each is updated to the x or y coordinate of the pixel of interest. Then, in step S17, the valid data selection signal is set to 1 and the value stored in the status register 24 is set to 2. Further, in step S17, the 3-bit counter 29 is initialized to 0, and the feature point data having a value of 1 and stored in the mask frame buffer 22 corresponding to the pixel of interest is rewritten to 0. Return to S2.

Here, when the selector 36 receives the valid data selection signal having a value of 1, it refers to the value stored in the state register 24, and if it is 1, the start point coordinate from the direction searcher 23. , The output of the ROM 33 from the outputs of the ROM 33 or the output of the direction change signal generator 35, and outputs the selected output to the multiplexer 37.

The ROM 33 has a 3-bit address space, like the ROMs 30 and 31 described above, in which values as shown in FIG. 8 are stored. And R
The OM33, like the ROMs 30 and 31, again
The output value of the 3-bit counter 29 is used as an address, and the stored value stored at that address (hereinafter, this stored value is referred to as direction data) is output.

Therefore, when the output value of the 3-bit counter 29 is 0 to 7, the direction data C is read from the ROM 33.
0 to C7 are output respectively. This direction data C0
C7 to C7 represent eight directions in which pixels adjacent to the search center pixel are located, that is, as shown in FIG. 9, for example, upper left, left, lower left, upper,
The direction data C0 to C7 indicates the direction in which the adjacent pixels P0 to P7 are located with respect to the search center pixel, respectively.

Therefore, the ROM 33 outputs direction data indicating the direction of the pixel of interest with respect to the search center pixel. In addition to the selector 36, the direction data is
The latch circuit 34 and the direction change signal generator 35 are also supplied.

From the above, in step S17, when the value stored in the state register 24 is 1 and the valid data selection signal becomes 1 (when the pixel of interest is a feature point), the chain encoding circuit 12 (Multiplexer 3
From 7), direction data indicating the direction in which the pixel of interest adjacent to the search center pixel (pixel located at the coordinates of the start point) and serving as the feature point is located, and the quantization coefficient of the pixel of interest. Is output as chain encoded data.

When the value stored in the state register 24 is 1 (hereinafter, appropriately referred to as state 1), the fact that the valid data selection signal becomes 1 is a characteristic feature of the chain. Then, the one adjacent to the starting point is found. In this case, as described above, the value stored in the status register 24 is set to 2 in step S17.

Returning to FIG. 4, if it is determined in step S8 that the value stored in the status register 24 is not 1, the process proceeds to step S21 in FIG. Where status register 2
An integer from 0 to 2 is stored in 4. Therefore, the value stored in the status register 24 is
It is determined in step S2 in FIG.
If it is determined in step S8 that the value is not 1, it means that the value is 2.

In step S21 of FIG. 10, as in the case of step S11 of FIG. 5, feature point data in any pixel (pixel corresponding to the count value output by the 3-bit counter 29) adjacent to the search center pixel. But,
It is read from the mask frame buffer 22, and the process proceeds to step S22, where it is determined whether or not the value is 1.
If it is determined in step S22 that the feature point data is not 1, that is, if the pixel of interest is not the feature point, the process proceeds to step S23, and it is determined whether the count value of the 3-bit counter 29 is 7. If it is determined in step S23 that the count value of the 3-bit counter 29 is not 7, that is, if processing is not performed for all pixels adjacent to the search center pixel, step S2
4, the count value of the 3-bit counter 29 is incremented by 1, and the process returns to step S2 in FIG. In this case, the value stored in the status register 24 remains 2, so that the values stored in step S2 and step S8 are
The processes in and after step S21 will be performed again.

When it is determined in step S23 that the count value of the 3-bit counter 29 is 7, that is, all the pixels adjacent to the search center pixel are processed, and as a result, the search center pixel is adjacent to the search center pixel. When it is determined that the pixel has no feature point, the process proceeds to step S25, 0 is set in the state register 24, and step S2
Return to

On the other hand, if it is determined in step S22 that the characteristic point data is 1, that is, if the pixel of interest is the characteristic point, the process proceeds to step S26 and the search X coordinate register 27 or the search Y coordinate register 28 is searched. The stored value is
The target pixel is newly set as the search center pixel by updating to the x or y coordinate of the target pixel. Then, the process proceeds to step S27, and the valid data selection signal is set to 1. Further, in step S27, the 3-bit counter 29 is initialized to 0, and the feature point data, which has a value of 1 and is stored in the mask frame buffer 22, is rewritten to 0. Return to S2. In this case, the status register 24
Since the stored value of 2 is still 2, the processing from step S21 onward is performed again through steps S2 and S8. However, in this case, step S26
At, the pixel of interest that was the target of the previous processing is
Since the pixel is newly set as the search center pixel, the pixel adjacent to the new search center pixel is processed as the pixel of interest.

Here, when the selector 36 receives the valid data selection signal having a value of 1, it refers to the value stored in the state register 24, and if it is 2, the start point coordinate from the direction searcher 23. , The output of the direction change signal generator 35 from the output of the ROM 33 or the output of the direction change signal generator 35, and outputs the selected output to the multiplexer 37.

The direction change signal generator 35 includes a ROM 33.
And the output of the latch circuit 34 are supplied there, and based on these inputs, direction change data, which will be described later, is generated.

That is, as shown in FIG. 11A, when the valid data selection signal becomes 1, the latch circuit 34 is
The direction data (FIG. 11 (B)) output by 33 is latched, delayed by one clock, and then the direction data is changed to the direction change signal generator 35 until the valid data selection signal becomes 1. To continue to output (FIG. 11 (C)). That is,
When the feature point is found (when the effective data selection signal becomes 1), the latch circuit 34 continues to output direction data indicating the direction from the search center pixel to the feature point until the next feature point is found. . Here, the direction data output from the latch circuit 34 will be appropriately referred to as forward direction data hereinafter.

When the valid data selection signal becomes 1, that is, when the characteristic point is found, the direction change signal generator 35 outputs the direction indicated by the forward direction data output from the latch circuit 34 and R.
The direction indicated by the direction data output from the OM 33 is compared, and the direction change data indicating the change in the direction indicated by the direction data with respect to the direction indicated by the front direction data is output in accordance with the comparison result. .

That is, the direction change signal generator 35 outputs D0 as the direction change data when the direction indicated by the forward direction data and the direction indicated by the direction data are the same, as shown in FIG. 12A, for example. To do. In addition, the direction change signal generator 35 is, for example, as shown in FIGS.
When the direction indicated by the direction data differs from the direction indicated by the forward direction data by 45 degrees, 90 degrees, or 135 degrees counterclockwise, D1 to D3 are respectively output as the direction change data. Further, the direction change signal generator 35 has
As shown in (E) to (G), the direction indicated by the direction data is 90 degrees clockwise from the direction indicated by the forward direction data by 90 degrees.
In the case of a difference of 135 degrees or 135 degrees, D4 to D6 are output as the direction change data.

As for the relationship between the direction indicated by the direction data and the direction indicated by the forward direction data, in addition to the case described above, FIG.
As shown in FIG. 2 (H), the two directions may be different by 180 degrees. However, in the chain encoding circuit 12 of FIG. 2, once the feature point becomes a processing target, as described above, the corresponding feature Since the point data is rewritten to 0, the direction indicated by the direction data and the direction indicated by the forward direction data are 180
There will be no occasions where they differ. Therefore, as shown in FIG. 12H, when the direction indicated by the direction data and the direction indicated by the forward direction data are different by 180 degrees, no code is given as the direction change data (it is necessary to give the code). Absent).

In reality, the direction change signal generator 35 stores a correspondence table of the direction change data and the forward direction data and direction data as shown in FIGS. 13 and 14, and the latch circuit 34 stores the correspondence table. The forward direction data being output,
A line matching the combination with the direction data output from the ROM 33 is searched from the correspondence table, and the direction change data described in the right column of the line is output. Note that the direction change data D0 to D6 are assigned with code words as shown in FIG. 15, for example, and the code words are actually output.

From the above, in step S27, when the value stored in the state register 24 is 2 and the valid data selection signal becomes 1 (when the pixel of interest is a feature point), the chain encoding circuit 12 (Multiplexer 3
From 7), the direction about the feature point found last time (the direction from the feature point found two times before to the feature point found last time) and the direction about the feature point found this time (from the feature point found last time, found this time) The direction change data indicating the difference between the direction change data) and the quantized coefficient at the feature point (pixel of interest) found this time is multiplexed, and chain encoded data is output.

When the value stored in the state register 24 is 2 (hereinafter, appropriately referred to as the state 2), the valid data selection signal becomes 1 when the number of consecutive characteristic points is 3 or more. This is the case when a feature point that constitutes a chain that is other than the start point and ones adjacent thereto (third feature point and subsequent ones) is found. In this case, the value stored in the status register 24 remains 2.

Returning to FIG. 4, if it is determined in step S3 that there are no unprocessed coordinates, the process ends. Then, after waiting for the quantized coefficient or feature point data for the next frame to be stored in the coefficient frame buffer 21 or the mask frame buffer 22, the processing from step S1 is performed again.

The output of the selector 36 will be further described with reference to FIG. Now, for example, as shown in FIG. 16, when there is a chain consisting of five consecutive feature points (hatched portions in the figure), the coordinates (i, The feature point located at j) appears first. Since the state when the feature point is first found is state 0, the selector 36 outputs the coordinate (i, j) of the feature point output from the direction searcher 23.
Is output as the start point coordinates. After that, the state is state 1.
Then, the characteristic point (i + 1, j) adjacent to the characteristic point located at the coordinate (i, j) (hereinafter, appropriately described as the characteristic point (i, j)) is detected. This feature point (i + 1,
j) is located to the right of the previously found feature point (i, j), and the state is state 1. Therefore, from the selector 36, the direction data C6 indicating the right direction output from the ROM 33 (Fig. 9) is output. Then, in this case, the state is set to state 2.

Next, the feature point (i + 2, j + 1) adjacent to the feature point (i + 1, j) is detected. In this case, since the state is the state 2, the direction change data indicating the difference between the direction of the previously found feature point and the direction of the currently found feature point is output from the selector 36.
That is, the direction of the feature point (i + 1, j) found last time is the right direction represented by the direction data C6 as described above, and the feature point (i + 2, j +) found this time.
The direction of 1) is the characteristic point (i + 2, j + 1)
Is located in the lower right direction of the feature point (i + 1, j),
It is represented by the direction data C7. Therefore, the forward direction data or the direction data in the correspondence table shown in FIG. 14 is C6, respectively.
Alternatively, the direction change data D4 described in the right column of the row of C7 is output from the selector 36.

Further, the feature point (i + 2, j + 1) is
Since the feature points (i + 3, j + 2) are adjacent to each other, this feature point (i + 3, j + 2) is detected. The direction in which the feature point (i + 3, j + 2) is located is the direction data C for the previously found feature point (i + 2, j + 1).
It is the same lower right direction as the direction indicated by 7, and therefore the direction for the feature point (i + 3, j + 2) found this time is
It is represented by the direction data C7. In addition, since the state remains the state 2, in this case, from the selector 36, the forward direction data and the direction data in the correspondence table shown in FIG. Direction change data D0 being output.

The feature point (i + 2, j + 3) adjacent to the feature point (i + 3, j + 2) is also associated with the feature point (i
+3, j + 2), and as a result, the selector 36 outputs direction change data D6.

Therefore, when there is a chain on the image, the chain encoding circuit 12 first multiplexes the coordinates of the feature point, which is the starting point of the chain, and the quantized coefficient at the feature point. Stuff is output. Then, the direction data of the feature point adjacent to the start point and the quantized coefficient at the feature point are multiplexed and output. If there is a feature point (excluding the start point) adjacent to the feature point, that is, if there is a third feature point or later, the direction change data about the feature point and the feature point And the quantized coefficient in 1 is multiplexed and sequentially output.

As described above, the chain encoding circuit 1
As described above, the chain-encoded data output from No. 2 is supplied to the RAMs 14 _{1 to} 14 _N via the selector 13 (FIG. 1), so that the RAM 14 _n receives the n-th frame of the chain-encoded data. The data is stored. And
When the chain encoded data is stored in all the RAMs 14 _{1 to} 14 _N , that is, when the chain encoded data for N frames is stored, the chain encoded data is read out by the similar chain detection circuit 15 and will be described in detail below. Processing is performed.

FIG. 17 shows a similar chain detection circuit 1 of FIG.
5 shows a configuration example of No. 5. The similarity detection circuit 41 uses RA
When the chain encoded data is stored in all M14 _{1 to} 14 _N , that is, when the chain encoded data for N frames is stored, the RAM 14 _{1 to} 14 _N respectively
The chain-encoded data of each frame is read out, and information about chains having a high degree of similarity between adjacent frames is supplied to and stored in the RAMs 42 _{1 to} 42 _N. Information regarding the chain of the nth frame is stored in the RAM 4
It is designed to be stored in 2 _n .

When each of the RAMs 42 _{1 to} 42 _N stores the information about the chain for each of the first to Nth frames, the chain connection circuit 43 reads the information, and further, based on the information, the chain Chains existing in adjacent frames are connected while detecting a motion vector. Then, the chain connection circuit 43 stores information about the connected chains in the RAM.
(Connection RAM) Sequentially stores them in the connection RAM 44, and when the processing for one chain is completed, the chain number of a predetermined chain among the connected chains, the frame number of the frame in which the chain exists, and the detected motion vector are output. To do.

FIG. 18 shows a configuration example of the similarity detection circuit 41 of FIG. Chain encoded data of each frame stored in the RAM 14 ₁ to 14 _N is adapted to be supplied to the chain map circuit 51 and the similarity calculating circuit 53. Upon receiving the chain encoded data of each frame, the chain map circuit 51 refers to the start point coordinates, the direction data, and the direction change data (hereinafter collectively referred to as a chain direction component) contained therein. Then, the chain existing in each frame is expanded on the RAM (Map RAM) 52 _{1 to} 52 _{N by} assigning a chain number as a unique number to each chain.

That is, for example, as shown in FIG. 19, when the chains CH1 to CH5 exist in the nth frame,
The chain map circuit 51 includes chains CH1 to CH5.
Add a unique chain number to each. Then, as shown in FIG. 20, the chain numbers assigned to the respective chains CH1 to CH5 are stored in the addresses of the RAM 52 _n corresponding to the positions where the chains CH1 to CH5 exist. In the embodiment of FIG. 20, chains CH1 to CH5
Chain number 0, 10, 14, 43, 12 respectively
3 is attached. In addition, the chain map circuit 51
A value not used as the chain number (-1 in the embodiment of FIG. 20) is stored in the address of the RAM 52 _n corresponding to the position where the chain does not exist.

As described above, the chain map circuit 5
1 refers to the chain existing in the nth frame in the RAM 52
_It is expanded to _n , so to speak, as a bitmap by chain number.

The chain number has only to be unique for the chains existing in each frame. However, the chain number is similar to the chain detection circuit 15
All chains existing in the N frame, which is the processing unit of, may be unique.

[0089] each RAM 52 ₁ to 52 _N, the first through the chain of the N-th frame is expanded, the similarity calculating circuit 53, respectively stored in the RAM 52 ₁ to 52 _N, for a chain of each frame, FIG. The processing according to the flowchart shown in 21 is performed.

That is, first, in step S31, an unprocessed chain (a chain which is not a chain of interest to be described later) exists in the frame currently being processed (hereinafter referred to as a frame of interest as appropriate). It is determined whether to do. If it is determined in step S31 that there is an unprocessed chain, that chain (if there are multiple unprocessed chains, any one of them is used as the processing target) Is referred to as a chain of interest), and the processing from step S32 onward is performed. Note that, here, the n-th frame is assumed to be the frame of interest unless otherwise specified.

That is, in step S32, for the chain of interest, 1 of the frames in which the chain of interest exists.
The motion vector calculation circuit 54 detects (calculates) a motion vector for a frame before the frame (hereinafter, appropriately referred to as a previous frame) (here, the (n−1) th frame).
Note that in step S32, the similarity calculation circuit 53 causes the R
The chain of interest is recognized based on the chain-encoded data of the nth frame supplied from the AM 14 _n, and the chain bitmap of the _n− 1th frame, which is the previous frame, is read from the RAM 52 _n−1 .
By supplying these to the motion vector calculation circuit 54,
The motion vector detection circuit 54 is made to detect the motion vector of the chain of interest to the previous frame.

When the motion vector of the chain of interest to the previous frame is calculated, the motion vector is associated with the chain number of the chain of interest and stored in the RAM 42 _n (see FIG. 1).
It is output to 7) and stored. Then, step S33
Then, the target chain is motion-compensated according to the motion vector calculated in step S32. Then, the process proceeds to step S34, and using the motion-compensated chain, the chain having the highest similarity to the chain of interest among the chains existing in the previous frame is detected.

Here, in step S34, the similarity calculation circuit 53 calculates the similarity of the chain of interest to a certain chain CH existing in the previous frame as follows. That is, the similarity calculation circuit 53 determines that among the pixels (feature points) that form the target chain after motion compensation,
Of the chains existing in the (n-1) th frame which is the previous frame, the one having the shortest distance to the chain CH is stored in the RAM.
52 _n-1 is detected and the number is counted. Then, the count number is divided by the number of pixels forming the chain of interest, and the division result is taken as the similarity of the chain of interest to the chain CH.

Therefore, in step S34, the RAM 52
Of the chains stored in n-1, for each of the pixels that make up the chain of interest after motion compensation, the one closest to that pixel is selected, and the chain with the most selections is the chain of interest. Is detected as having the highest degree of similarity with. At this time, the degree of similarity is obtained by dividing the number of selections by the number of pixels forming the chain of interest.

When a chain existing in the previous frame (hereinafter, appropriately referred to as a high similarity chain) that maximizes the similarity of the chain of interest is detected, the process proceeds to step S35, and the similarity is equal to or higher than a predetermined threshold value T. Is determined. When it is determined in step S35 that the similarity of the target chain to the high similarity chain is equal to or larger than the predetermined threshold T, the process proceeds to step S36, and the chain number of the high similarity chain corresponds to the chain number of the target chain. Attached. Then, it is output to and stored in the RAM 42 _n , and the process proceeds to step S38. The chain number of the chain of interest is already stored in the RAM 42 _n in association with the motion vector (motion vector of the chain of interest to the previous frame) in step S32 described above. To the chain number of the high similarity chain,
It is stored in M42 _n .

On the other hand, in step S35, the similarity of the target chain to the high similarity chain is a predetermined threshold value T.
When it is determined that the above is not the case, the process proceeds to step S36,
In the previous frame, there is no chain that may be connected to the chain of interest (there is no chain representing the same (object) contour as the chain of interest), and an uncorresponding code (in this embodiment, As shown in FIG. 23 described later,
For example, -1) is associated with the chain number of the chain of interest, stored in the RAM 42 _n , and the process proceeds to step S38.

In step S38, the motion vector to the frame one frame after the frame in which the chain of interest exists (hereinafter, appropriately referred to as the subsequent frame) (here, the (n + 1) th frame) is set as the motion vector. The calculation circuit 54 is caused to detect (calculate) in the same manner as in the case of step S32 described above. Further, in step S38, the motion vector of the chain of interest to the subsequent frame is associated with the chain number of the chain of interest and output to and stored in the RAM 42 _n . afterwards,
In step S39, the target chain is motion-compensated according to the motion vector calculated in step S38. Then, the process proceeds to step S40, in which the motion-compensated chain of interest is used to maximize the similarity with the chain of interest among the chains existing in the subsequent frame (hereinafter, also referred to as a high-similarity chain as appropriate. ) Is detected in the same manner as in step S34 described above.

Then, the process proceeds to step S41, where the similarity of the target chain to the high similarity chain is a predetermined threshold value T.
It is determined whether or not this is the case. When it is determined in step S41 that the degree of similarity is equal to or greater than the predetermined threshold value T, the process proceeds to step S42, and the chain number of the high similarity chain is associated with the chain number of the chain of interest and output to the RAM 42 _n . Are stored. Also,
If it is determined in step S41 that the similarity of the target chain to the high-similarity chain is not greater than or equal to the predetermined threshold value T, the process proceeds to step S43, and there is no chain in the rear frame that may be connected to the target chain. As in the case of step S37 described above,
The uncorresponding code indicating that is associated with the chain number of the chain of interest and stored in the RAM 42 _n .

After the processing of steps S42 and S43,
In either case, the process returns to step S31, and the processes of steps S31 to S43 are repeated until it is determined in step S31 that there is no unprocessed chain. Then, in step S31, when it is determined that there is no unprocessed chain, the process ends. After that, instead of the nth frame, which has been regarded as the current frame of interest, the n + 1th frame is replaced.
The process shown in FIG. 21 is repeated with the frame as a new frame of interest. That is, the above processing is performed in the RAM 1
This is performed for the chain encoded data for N frames stored in 4 _{1 to} 14 _N.

Through the above processing, the following information about chains is stored in the RAM 42 _n . That is, for example, as shown in FIG. 22, a chain # 0 (the number after # represents the chain number) is present in the nth frame, and the similarity of this chain # 0 is equal to or greater than the threshold value T. When the high similarity chain # 12 or # 34 exists in the previous or the subsequent frame, the motion vector of the chain of interest to the previous frame or the subsequent frame is (1, 2) or (0 , -8), in the RAM 42 _n , as shown in FIG. 23 (the first row of FIG. 23), the chain number 0 of the chain # 0, the motion vector (0, -8) to the subsequent frame, the previous The motion vector (1, 2) to the frame, the chain number of the high similarity chain # 12 existing in the previous frame (“the chain number having the highest similarity in the previous frame”) 12, and the high class existing in the subsequent frame Degrees chains # 34 chain number ( "number of the highest chain similarity rear frame") 34 are stored in association with each other.

Similarly, in the RAM 42 _n , information on all chains existing in the nth frame is stored for each chain. It should be noted that the chain number of the high similarity chain existing in the previous frame ("the number of the chain having the highest similarity in the previous frame") or the chain number of the high similarity chain existing in the subsequent frame ("the similarity of the subsequent frame" The highest number is the chain number)) is -1, that is, the uncorresponding code is that the chain corresponding to the chain number of the row has the It indicates that there is no chain that may be connected to the chain.

Next, FIG. 24 is a flow chart for explaining the processing of the chain connection circuit 43 of FIG. In the chain connection circuit 43, when the RAMs 42 _{1 to} 42 _N store information about the chains of the 1st to Nth frames, respectively, the RAM 44 is initialized in step S51, and the process proceeds to step S52 to unprocess the chains. ,
It is determined whether or not the data is stored in the RAMs 42 _{1 to} 42 _N. Note that this determination is performed by searching for unprocessed chains in the order of chain number and frame number, for example.

If it is determined in step S52 that there is an unprocessed chain, then that chain is set as the target chain, and the flow advances to step S53 to proceed to the chain number of the target chain and the previous frame associated with the chain number. And the motion vector to the subsequent frame are read from the RAM 42 (RAM 42 _{1 to} 42 _N ). Further, in step S53, these are the frame numbers of the frames in which the chain of interest exists (this is the RA in which the chain of interest was stored).
(Equal to the suffix n of M42n),
The data is output to and stored in the RAM 44.

Then, in step S54, it is determined whether or not there is a chain that satisfies a predetermined connection condition with the chain of interest in the previous frame. That is, step S
In 54, for example, a high-similarity chain to the chain of interest exists in the previous frame, and the high-similarity chain is
A high-similarity chain that has not yet been regarded as a chain of interest and that exists in the rear frame of the high-similarity chain (the rear frame (hence, the frame of interest viewed from the high-similarity chain existing in the previous frame)) Is determined by referring to the RAM 42.

If a chain satisfying the connection condition with the chain of interest as described above exists in the previous frame, the process proceeds to step S55, the chain is newly set as the chain of interest, and the process returns to step S53. Therefore, in this case, the chain number of the new target chain, the motion vector to the previous frame and the motion vector to the rear frame associated with the chain number are read from the RAM 42, and the new target Corresponding to the frame number of the frame in which the chain exists, it is output to the RAM 44 and stored. As a result, the chain that was the attention chain last time and the new attention chain are connected. That is, the chain that was the attention chain last time and the new attention chain are associated with each other as showing the same contour.

The chain (high similarity chain) stored in the RAM 42 is as described in FIG.
Since the similarity to the chain of interest is equal to or greater than the threshold value T, according to the above processing, the chain A existing in a certain frame and the chain B existing in the preceding frame are similar to the chain B in the chain A. When SA and the similarity SB of the chain B to the chain A are both equal to or larger than a predetermined threshold value T, the connection is established.

On the other hand, if it is determined in step S54 that the chain satisfying the connection condition with the target chain does not exist in the previous frame, the process proceeds to step S56, immediately after it is determined in step S52 that there is an unprocessed chain. Then, the chain selected as the target chain is again set as the target chain, and the process proceeds to step S57. Step S5
At 7, it is determined whether or not there is a chain that satisfies a predetermined connection condition with the chain of interest in the rear frame.
That is, in step S57, for example, a high-similarity chain with respect to the chain of interest exists in the rear frame, the high-similarity chain has not yet been set as the chain of interest, and the previous frame of the high-similarity chain has been determined. Whether or not the high-similarity chain existing in the previous frame (hence the frame of interest viewed from the high-similarity chain existing in the rear frame) is the chain of interest is determined by the RAM 4
It is determined by referring to 2.

If a chain satisfying the above-described connection condition with the chain of interest exists in the rear frame, the process proceeds to step S58, the chain is newly set as the chain of interest, and the process proceeds to step S59. In step S59,
The frame number of the new chain of interest, the motion vector to the previous frame and the motion vector to the subsequent frame associated with the chain number are read from the RAM 42, and the frame in which the new chain of interest exists The frame number is associated with the frame number and is output to and stored in the RAM 44. As a result, the chain that was the attention chain last time and the new attention chain are connected.

On the other hand, if it is determined in step S57 that the chain satisfying the connection condition with the target chain does not exist in the subsequent frame, the process proceeds to step S60 and R
Of the frame numbers stored in the AM 44, for example, the smallest frame number (hereinafter, appropriately referred to as minimum frame number), the chain number associated with the minimum frame number (hereinafter, appropriately referred to as minimum chain number), and All motion vectors to the frame are output to the select multiplexing circuit 16 (FIG. 1).

That is, the chain associated with the frame having the smallest frame number is determined as the basic chain, and all the motion vectors to the subsequent frames are determined as the motion vectors to be multiplexed, and the frame number in which the basic chain exists. , Its chain number, and all the motion vectors to the subsequent frame are output to the select multiplexing circuit 16.

With the above processing, _if the storage contents of the RAM 42 _n are as shown in FIG. 25A, the RAM 44 stores the information as shown in FIG. From the chain connection circuit 43, see FIG.
The information as shown in FIG. 5 (C) is output. That is, for example, the chain # 56 (the chain whose chain number is 56) present in the sixth frame shown in FIG.
However, if it is assumed that the chain of interest is set in step S52, the frame number 6,
Chain number 56, motion vector to the next frame (-
3, −6), and the motion vector (0,
2) is stored in the RAM 44 (FIG. 25 (B)).

Then, in the column of "the number of the chain having the highest similarity in the previous frame" for the chain # 56,
Since the chain number 273 is described, the chain # 5 is set in the fifth frame, which is the previous frame of the sixth frame.
There is a chain # 273, which is a high similarity chain for 6. Further, the chain number # 56 is described in the column of "the chain number having the highest degree of similarity in the rear frame for this chain # 273" (see FIG. 25).
(A)) Therefore, assuming that this chain # 273 has never been the chain of interest, the chain # 273 is
The connection condition determined in step S54 is satisfied. Therefore, in this case, the frame number 5 of the chain # 273, the chain number 273, the motion vector (-1,3) to the subsequent frame, and the motion vector (3,6) of the previous frame are stored in the RAM 44 ( Figure 25
(B)).

The chain # 273 satisfying the connection condition is set as a new chain of interest in step S55,
Since the unsupported code (-1) is described in the column of "the number of the chain with the highest degree of similarity in the previous frame" for this chain # 273, the connection condition with the chain # 273 is written in the previous frame. There is no chain that satisfies.

Therefore, in step S56, the chain #
56 is again taken as the attention chain. And the chain #
Since the chain number # 24 is described in the column of “the chain number having the highest degree of similarity in the subsequent frame” for 56, the seventh frame, which is the subsequent frame of the sixth frame, corresponds to the chain # 56. There is a chain # 24 which is a high similarity chain. In addition, this chain #
The chain number # 56 is described in the column of “the chain number having the highest degree of similarity in the previous frame” for No. 24 (FIG. 25 (A)). Therefore, this chain # 24 is regarded as the chain of interest. If not, chain #
24 satisfies the connection condition determined in step S57. Therefore, in this case, the frame number 7 of the chain # 24, the chain number 24, the motion vector (0, -2) to the subsequent frame, and the motion vector (3, 9) of the previous frame are stored in the RAM 44 ( Figure 25
(B)).

The chain # 24 satisfying the connection condition is regarded as a new chain of interest in step S58, but the column of "the number of the chain having the highest similarity in the subsequent frame" for this chain # 24 is not supported. Code (-
Since 1) is described, there is no chain that satisfies the connection condition with the chain # 24 in the subsequent frame.

Therefore, of the information stored up to now in the RAM 44, the minimum frame number, the minimum chain number,
And all the motion vectors to the subsequent frame are read and output in step S60.

That is, the contents stored in the RAM 44 are as shown in FIG.
In the case shown in FIG. 25B, as shown in FIG. 25C, the minimum frame number is 5, and the chain number is 27, which is associated with the frame number.
3 and the motion vector to the subsequent frame (-1,-
3), (-3, -6) and (0, 2) are output from the similar chain detection circuit 15 (chain connection circuit 43) to the select multiplexing circuit 16.

Then, in the select multiplexing circuit 16, as described above, the chain-encoded data for the chain with the smallest chain number existing in the frame with the smallest frame number n is read from the RAM 14 _n and the chain-encoded data is obtained. The data and all the motion vectors from the similar chain detection circuit 15 are multiplexed and output as multiplexed chain encoded data.

That is, for example, as shown in FIG. 26, assuming that the chains CH _{n to} CH _N respectively existing in the _nth to Nth frames are connected, the select multiplexing circuit 16 indicates that the chains CH _n are Of the chain encoded data and all the motion vectors of the chain CH _n to the chain CH _N-1 to the subsequent frames are output.

Therefore, a chain connected over a plurality of frames (similar chains when viewed in the time direction) moves the chain encoded data for a certain chain (basic chain) and the chain. Since it is output as motion vectors for (motion compensation) to restore other chains, that is, by connecting similar chains across multiple frames, a tree with such chains as nodes Make up the structure,
Regarding the chain that is one of the nodes,
Since the chain encoded data for that chain is output as the motion vector for the other chains, it is compared with the conventional case where the chains existing in each frame are output as chain encoded data. As a result, the redundancy in the time direction can be reduced, and as a result, the image compression efficiency can be improved.

Further, as a result, it becomes possible to transmit a moving image through a transmission path having a limited transmission band or record it on a recording medium having a small capacity.

The basic chain is a chain existing in the frame having the smallest frame number among the connected chains, for example, a chain existing in the frame having the largest frame number, or any other chain. It is possible. If the basic chain is, for example, a chain existing in the frame with the largest frame number, the motion vector output from the similar chain detection circuit 15 (chain connection circuit 43) may be all motion vectors to the previous frame.

Next, FIG. 27 shows another configuration example of the similar chain detection circuit 15 of FIG. Similar to the similarity detection circuit 41 of FIG. 17, when the chain encoded data is stored in all the RAMs 14 _{1 to} 14 _N , the multi-chain similarity detection circuit 61 outputs each frame from each of the RAMs 14 _{1 to} 14 _N. Read the chain encoded data and set the information about the chains existing in the adjacent frames to R
It is supplied to the AM 62 and stored.

However, in the similarity detection circuit 41, a chain (high similarity chain) that maximizes the similarity of the chain of interest and has a similarity not less than a predetermined threshold value T is adjacent frames (previous frame and rear frame). Frame), the motion vector to the subsequent frame, the motion vector to the previous frame, the chain number of the high similarity chain existing in the previous frame, and the high similarity existing in the subsequent frame. The chain number of the degree chain is associated and stored in the RAM 42 (FIG. 23). However, in the multi-chain similarity detection circuit 61, a chain that makes the degree of similarity of the chain of interest larger than 0 (the chain of interest is If a chain having a similarity (hereinafter, appropriately referred to as a similarity chain) exists in an adjacent frame, the chain of interest The chain number of is associated with the motion vector to the subsequent frame, the motion vector to the previous frame, the chain number of all similarity chains in the previous frame, and the chain number of all similarity chains in the subsequent frame. The RAM 62 is also configured to be stored in the RAM 62. Further, in the RAM 62, the chain number of the similarity chain is stored in association with the similarity of the chain of interest to the chain. .

FIG. 28 shows the contents stored in the RAM 62n. In the row of the chain number # 0 in the embodiment of FIG. 28, the motion vector to the subsequent frame or the motion vector to the previous frame of the chain # 0 existing in the nth frame is (0, -8) or ( 1, 2), similarity chains # 12, # 8, and # 44 exist in the previous frame, and similarity chain # 10 in the subsequent frame.
9 and # 69 are present. Also, the similarity chains # 12, # 8, # 44 existing in the previous frame
, The similarity of the attention chain is 0.7,
It is 0.23 and 0.06, which means that the similarity of the chain of interest to the similarity chains # 109 and # 69 existing in the rear frame is 0.7 and 0.1, respectively.

If there is no similarity chain in the preceding frame or the following frame, the chain number column of the preceding frame or the chain number of the succeeding frame and its similarity column indicate that fact. For example, -1 is described.

In the row of chain number # 1, the motion vector to the subsequent frame or the motion vector to the previous frame of the chain # 1 existing in the nth frame is (7,
7) or (2, 1), and a similarity chain # 121 with a similarity of 0.92 exists in the previous frame, and a similarity chain with a similarity of 0.97 in the subsequent frame. # 39 is present. Also,
In the row of chain number # N2, the motion vector to the subsequent frame or the motion vector to the previous frame of the chain # N2 existing in the nth frame is (−5, −5), respectively.
Alternatively, it is (0, -1), which indicates that the similarity chain # 2 having a similarity of 0.2 exists in the previous frame and that the similarity chain does not exist in the subsequent frame.

When the RAMs 62 _{1 to} 62 _N store the above-described information about the chains of the first to Nth frames, respectively, the multi-chain connection circuit 63 performs the processing according to the flowchart shown in FIG. . That is, in the multi-chain connection circuit 63, first, the RAM (Connection RAM) 64 is initialized in step S61, and the process proceeds to step S62 to determine whether unprocessed chains are stored in the RAMs 621 to 62N. To be judged. Note that this determination is made in step S52 of FIG.
In the same manner as the case of, the unprocessed chains are searched, for example, in the order of the chain number and the frame number.

If it is determined in step S62 that there is an unprocessed chain, then that chain is set as the target chain, and the flow advances to step S63 to proceed to the chain number of the target chain and the previous frame associated with the chain number. And the motion vector to the subsequent frame are read from the RAM 62. further,
In step S63, these are associated with the frame number of the frame in which the chain of interest exists, and the RAM 64
Is output and stored.

After that, the process advances to step S64, and the connection process to the chain existing in the previous frame is performed. That is,
In step S64, the chains existing in the previous frame that should be connected to the chain of interest are detected and connected to the chain of interest.

Specifically, as shown in FIG. 30, first of all, in step S71, among the chains (similarity chains) existing in the previous frame and having the similarity of interest, the subsequent frame (previous frame) is selected. It is determined whether or not there is a chain having the highest similarity to the chain of the post-frame (hence, the frame of interest) viewed from the frame as the chain of interest. If it is determined in step S71 that the chain having the highest similarity to the chain in the subsequent frame is not the focused chain among the similarity chains in the previous frame, steps S72 to S74 are skipped, Step S7
Go to 5.

If it is determined in step S71 that, among the similarity chains of the previous frame, the chain with the highest similarity to the chain of the subsequent frame is the chain of interest, the process proceeds to step S72. A chain whose similarity to the target chain is equal to or higher than a predetermined threshold value T is a connection candidate chain as a chain that may be connected to the target chain. When there are a plurality of chains in the previous frame whose similarity to the chain of interest is equal to or higher than the predetermined threshold value T, all of them are considered as connection candidate chains. If a chain whose similarity to the target chain is equal to or higher than the predetermined threshold value T does not exist in the previous frame, although not shown in FIG. 30, steps S73 and S74 are skipped and the process proceeds to step S75. .

After obtaining the connection candidate chain, step S7
At 3, it is determined whether the connection candidate chain satisfies the connection condition for the chain of interest. That is, in step S73, for example, when the similarity of the target chain to the connection candidate chain (when there are a plurality of connection candidate chains, the sum of the similarities to the connection candidate chains) is equal to or more than a predetermined threshold T, the connection is performed. It is determined that the candidate chain satisfies the connection condition for the chain of interest.

If it is determined in step S73 that the connection candidate chain does not satisfy the connection condition for the chain of interest as described above, step S74 is skipped and the process proceeds to step S75. If it is determined in step S73 that the connection candidate chain satisfies the above-described connection condition for the chain of interest (hereinafter, a connection candidate chain satisfying the connection condition will be referred to as a connection chain), the process proceeds to step S74. Go to RAM6
Referring to 2, the frame number in which the connection chain exists is associated with the chain number of the connection chain, the motion vector to the previous frame, and the motion vector to the subsequent frame, and is output and stored in the RAM 64. When there are a plurality of connection chains, the information on all the connection chains is stored in the RAM 64. As a result, the target chain and the connection chain are connected. That is, the target chain and the connection chain are associated with each other as showing the same contour.

According to the processing of steps S71 to S74 described above, for example, as shown in FIG. 31, when the chain of interest C is divided into three chains D1, D2, and D3 in a previous frame, so to speak. , The sum of the similarities SC1, SC2, SC3 of the target chain C with respect to the chains D1, D2, D3 is equal to or greater than the threshold T, and the similarities SD1, SD2 of the chains D1, D2, D3 with respect to the chain C, respectively. When both SD3 are equal to or greater than the threshold value T, the chain of interest C and the chains D1, D2, D3 are connected.

Returning to FIG. 30, in step S75, the chain Z whose similarity to the target chain is equal to or greater than the threshold T is
It is determined whether it exists in the previous frame. In step S75, the similarity to the target chain is the threshold T
When it is determined that the above chain Z does not exist in the previous frame, steps S76 to S79 are skipped,
Proceed to step S80. If it is determined in step S75 that the chain Z whose similarity to the target chain is equal to or greater than the threshold value T is present in the previous frame, the process proceeds to step S76 and the rear frame (from the chain Z) having the similarity to that chain Z is detected. Whether or not there is a chain that has the highest degree of similarity to the chain of the previous frame among the chains existing in the after-frame (hence, the frame of interest), is RA.
It is determined with reference to M62.

If it is determined in step S76 that no chain among the chains existing in the rear frame as viewed from the chain Z has the highest degree of similarity to the chain in the previous frame, the chain Z is determined in step S77. Through S79 are skipped, and the process proceeds to step S80.

If it is determined in step S76 that the chain Z has the highest degree of similarity to the chain of the previous frame among the chains existing in the rear frame as viewed from the chain Z,
In step S77, a chain whose similarity to the chain Z is equal to or higher than a predetermined threshold value T among the chains is set as a connection candidate chain as a chain that may be connected to the chain Z. If there are a plurality of chains in the subsequent frame whose degree of similarity to the chain Z is equal to or higher than the predetermined threshold value T, all of them are connection candidate chains. If a chain whose similarity to the chain Z is equal to or higher than a predetermined threshold value T does not exist in the subsequent frame, steps S78 and S79 are skipped and the process proceeds to step S80, which is not shown in FIG. .

Then, in step S78, it is determined whether or not the connection candidate chain satisfies the connection condition for the chain Z. That is, in step S78, for example, when the similarity of the chain Z to the connection candidate chain (when there are a plurality of connection candidate chains, the total of the similarities to the connection candidate chains) is equal to or more than a predetermined threshold T, the connection is performed. It is determined that the candidate chain satisfies the connection condition for chain Z.

If it is determined in step S78 that the connection candidate chain does not satisfy the connection condition for the chain Z as described above, step S79 is skipped and the process proceeds to step S80. If it is determined in step S78 that the connection candidate chain satisfies the above-described connection condition for the chain Z (hereinafter, a connection candidate chain that satisfies this connection condition will also be referred to as a connection chain), step S79. Go to RAM62
With reference to, the frame number in which the connection chain exists is associated with the chain number of the connection chain, the motion vector to the previous frame, and the motion vector to the subsequent frame, and is output and stored in the RAM 64.

When there are a plurality of connection chains,
Information about all connection chains is stored in RAM 64. Further, the connection chain includes the chain of interest, and the information regarding the chain of interest is shown in FIG.
Since it has already been written in the RAM 64 in step S29, the writing is not performed in step S79.

Further, in step S79, the RAM 62 is
Referring to, the frame number in which the chain Z exists is associated with the chain number of the chain Z, the motion vector to the previous frame, and the motion vector to the subsequent frame, and is output and stored in the RAM 64.

As a result, the chain Z and the connection chain are connected. That is, the chain Z and the connection chain are associated with each other as showing the same contour.

According to the above-described processing of steps S75 to S79, for example, as shown in FIG.
Is one of a plurality of chains D1, D2, D3 which are obtained by dividing the chain Z existing in the previous frame, the chains D1, D2, D3 of the chain Z are
Chains D1, D2, in which the sum of the similarities SZ1, SZ2, SZ3 for each is greater than or equal to the threshold value T, and which includes the chain of interest (chain D1 in the embodiment of FIG. 32).
Similarity SD1, S of each D3 to the chain Z
When both D2 and SD3 are equal to or more than the threshold value T, the chain Z and the chains D1, D2 including the chain of interest are included.
Each of D3 is connected.

After the processing of step S79, step S8
In step S81, which will be described later, of the chains stored in the RAM 64 (excluding the chains existing in the frame of interest), it is determined whether or not there are chains that are not yet regarded as the chain of interest. Of the chains stored in the RAM 64 at step S80,
If it is determined that there is a chain that is not the chain of interest, the process proceeds to step S81, a chain that is not yet a chain of interest is set as a new chain of interest, and the process returns to step S71. Therefore, in this case, the process after step S71 is repeated for the new chain of interest,
As a result, the tree structure as described above is configured for the chains existing in the frame before (temporally before) the frame of the chain selected as the target chain in step S62 of FIG.

In step S80, the RAM 6
If it is determined that none of the chains stored in No. 4 is not the target chain, that is, if all the chains stored in the RAM 64 are the target chains, the process proceeds to step S82. When the process of connecting to the chain existing in the previous frame is started, the chain that has been the target chain is again set as the target chain. Then, the process of connecting to the chain existing in the previous frame is ended, and the process proceeds to step S65 of FIG.

In step S65, the process of connecting to the chain existing in the subsequent frame is performed. That is, step S
At 65, a chain existing in the rear frame that should be connected to the target chain is detected and connected to the target chain.

Here, the flowchart of FIG. 33 shows the details of the connection processing to the chain existing in the subsequent frame. As shown in the figure, in the process of connecting to the chain existing in the rear frame, the same processes as in steps S71 to S82 of FIG. 30 are performed in steps S91 to S102. However, in steps S71 to S82 of FIG. 30, the chain existing in the previous frame is connected to the target chain, but in FIG. 33, the chain existing in the rear frame is connected to the target chain. Is being done.

Therefore, according to the processing of steps S91 to S94, for example, as shown in FIG.
When there are three chains D1, D2, D3 that are obtained by dividing the chain of interest C, so to speak, the sum of the degrees of similarity SC1, SC2, SC3 of the chain of interest C with respect to the chains D1, D2, D3 is equal to or greater than the threshold value T. And the similarity SD1, SD2, SD3 of each of the chains D1, D2, D3 with respect to the chain C is equal to or greater than the threshold value T, the chain of interest C and the chains D1, D2, D3 are connected. It

Further, according to the processing of steps S95 to S99, for example, as shown in FIG.
Is one of a plurality of chains D1, D2, D3 which are obtained by dividing the chain Z existing in the rear frame, the chains D1, D2, D3 of the chain Z are
Chains D1, D2, in which the sum of the degrees of similarity SZ1, SZ2, SZ3 for each is equal to or greater than the threshold value T, and which includes the chain of interest (chain D1 in the embodiment of FIG. 35).
Similarity SD1, S of each D3 to the chain Z
When both D2 and SD3 are equal to or more than the threshold value T, the chain Z and the chains D1, D2 including the chain of interest are included.
Each of D3 is connected.

As a result, a tree structure having similar chains as nodes is constructed for a chain existing in a frame after (temporally after) the frame of the chain selected as the target chain in step S62 of FIG. It

Returning to FIG. 29, when the process of connecting to the chain existing in the subsequent frame in step S65 is completed, the process proceeds to step S66, and the frame in which the chain of interest exists among the chains stored in the RAM 64, that is, the frame of interest. It is determined whether or not there is a chain (excluding the chain of interest) existing in 1) that has not been made to be the chain of interest. Step S66
In the case where it is determined that there is no chain among the chains stored in the RAM 64 that is present in the frame of interest and is not the chain of interest, step S67 is skipped and the process proceeds to step S68. Also,
In step S66, if it is determined that, among the chains stored in the RAM 64, there is a chain that exists in the frame of interest and has not yet been set as the chain of interest, the process proceeds to step S67 and the chain of interest is selected. As a result, the connection processing to the chain existing in the rear frame of FIG. 33 is performed.

If there are a plurality of chains stored in the RAM 64 that are present in the frame of interest and are not yet regarded as the chain of interest, in step S67, the plurality of chains are The chain of interest is sequentially changed, and the connection processing to the chain existing in the subsequent frame is performed.

After that, the process proceeds to step S68, and similarly to the case in step S66, of the chains stored in the RAM 64, the frame in which the chain of interest exists, that is, the chain existing in the frame of interest (excluding the chain of interest). Therefore, it is determined whether or not there is a chain that is not regarded as a chain of interest.

If it is determined in step S68 that the chains stored in the RAM 64 exist in the frame of interest and are not regarded as the chain of interest, step S69 is skipped and step S70 is skipped. Proceed to. In step S68,
If it is determined that, among the chains stored in the RAM 64, there are chains that are present in the frame of interest and have not yet been set as the chain of interest, step S6.
Proceed to step 9 and select that chain as the chain of interest.
It is determined whether or not there is a chain stored in the RAM 64 by performing a process of connecting to the chain existing in the preceding frame.

In step S69, among the chains stored in the RAM 64, a chain existing in the frame of interest, which has not yet been set as the chain of interest, is set as a chain of interest, and the chain is connected to the chain existing in the previous frame. If it is determined that there is a chain stored in the RAM 64 by performing
Among the chains stored in the AM64, the chains existing in the frame of interest, which have not yet been set as the chain of interest, are set as the chain of interest, and step S64
Return to If there are a plurality of such chains, any one of them is the chain of interest.

On the other hand, in step S69, the RAM 6
Among the chains stored in 4, the chains existing in the frame of interest, which have not yet been marked as the chain of interest, are set as the chain of interest, and are connected to the chains existing in the previous frame. When it is determined that there is no stored chain, the process proceeds to step S70, and the information stored in the RAM 64 is output to the select multiplexing circuit 16 (FIG. 1).

That is, in step S70, for example, RA
Of the information stored in M64, it is determined that the chain existing in the frame with the smallest number of chains (hereinafter, appropriately referred to as the minimum frame) is the basic chain,
The frame number of the smallest frame, the chain number associated with that frame number, the number of frames from the smallest frame to the frame with the smallest frame number, the motion vector from that frame to the previous frame, the smallest frame to the largest frame The number of frames up to the numbered frame and the motion vector to the subsequent frame between the frames are read from the RAM 64 (however, since the number of frames described above is equal to the number of motion vectors, the number of motion vectors is counted. Is recognized) and is output to the select multiplexing circuit 16 (FIG. 1).

The number of frames from the smallest frame to the frame of the smallest frame number, all the motion vectors to the preceding frames between the frames, the number of frames from the smallest frame to the frame of the largest frame number,
Motion vectors for the subsequent frames between the frames are multiplexed and output.

By the above processing, _if the contents stored in the RAM 62 _n are as shown in FIG. 36, for example, R
Information as shown in FIG. 37 (A) is stored in the AM 64, and further, from the multi-chain connection circuit 63, FIG.
The information as shown in (B) is output. That is, for example,
Now, assuming that the chain # 56 existing in the eighth frame shown in FIG. 37 is set as the chain of interest in step S62, its frame number 8, chain number 56, and motion vector (5, 5) to the subsequent frame are calculated in step S63. 3), and the motion vector to the previous frame (-
2, 0) is stored in the RAM 44 (FIG. 37).
(A)).

Then, the chains # 36 and # of the previous frame (seventh frame) having the similarity with the chain # 56.
With respect to 151, the chain having the highest similarity to the chain of the rear frame is the chain # 56 which is the noticed chain (FIG. 36). Now, assuming that the threshold T is 0.8, for example, the chain # 36 or #
The similarity of 151 to chain # 56 is 0.97 or 0.91, respectively, and the chain # 56
Of the chain # 36 or # 151
Since they are 0.96 or 0.91, respectively, the degrees of similarity of the chains # 36 and # 151 to the chain # 56 are both equal to or greater than the threshold value T, and
The sum of the degrees of similarity of 56 to the chains # 36 or # 151, respectively, is also equal to or greater than the predetermined threshold T.

Therefore, the chain # 36 of the seventh frame,
# 151 frame number 7 and chain numbers 36 and 15
1, the motion vector (0, -2) to the subsequent frame, and the motion vectors (3,9) and (3,1) of the previous frame are
In step S79, it is stored in the RAM 64 (see FIG. 3).
7 (A)).

The chains # 36 and # 151 stored in the RAM 64 are set as new attention chains in step S81.
Since there is no chain of the previous frame (sixth frame) having the similarity (FIG. 36), the chain # 56 is set as the target chain again in step 82.

Then, the chain # 24 and the chains # 24 and # 9 of the rear frame (the ninth frame) having the similarity.
Regarding 25, the chain with the highest similarity to the chain of the previous frame is the chain # 56 which is the chain of interest (FIG. 36). Further, in this case, the chain # 56 of the chain # 24 or # 25
Is 0.95 or 0.90 respectively, and the similarity of chain # 56 to chains # 24 or # 25 is 0.99 or 0.
92, the degree of similarity between chains # 24 and # 25 with respect to chain # 56 is equal to or greater than threshold value T, and the total degree of similarity between chain # 56 with respect to chain # 24 or # 25 is also predetermined. Is greater than or equal to the threshold value T.

Therefore, the chain # 24 of the ninth frame,
The frame number 9 of # 25, the chain numbers 24 and 25, the motion vector (1, -1) to the subsequent frame, and the motion vectors (4,0) and (4,2) of the previous frame are stored in the RAM 64 in step S99. It is stored (Fig. 37
(A)).

The chains # 24 and # 25 stored in the RAM 64 are set as new chains of interest in step S101. However, the chain of the subsequent frame (the tenth frame) to which the chain # 24 or # 25 has a similarity is Since it does not exist (FIG. 36), in step S102,
Chain # 56 is once again regarded as the chain of interest.

At this stage, of the chains stored in the RAM 64, there is no chain that is not the chain of interest in the eighth frame, which is the frame of interest. Of these, the chain that exists in the minimum frame is the basic chain, and information that centers on that basic chain is,
In step S70, it is read and output. That is, the frame number of the smallest frame, the chain number of the basic chain, the number of frames from the smallest frame to the frame of the smallest frame number, the motion vector to the previous frame between the frames, from the smallest frame to the frame of the largest frame number Of frames and all motion vectors to the subsequent frames between the frames are read and output in step S70.

Specifically, for example, when the storage content of the RAM 64 is as shown in FIG. 37 (A), the frame with the smallest number of chains is the eighth frame, so this is the smallest frame. Therefore, FIG.
As shown in (B), the frame number of the smallest frame is 8, the chain number associated with the frame number is 56, and the number of frames from the smallest frame to the smallest frame number 7 is 1, It is a motion vector to the previous frame between the frames (-2,
0), 1 which is the number of frames from the smallest frame to the frame with the largest frame number 9, and the motion vector (5, 3) to the subsequent frame between the frames are similar chain detection circuit 15 (multi-chain connection circuit). 63) to the select multiplexing circuit 16.

Then, in the select multiplex circuit 16, as described above, the chain coded data for the chain existing in the frame of the frame number n received from the similar chain detection circuit 15 refers to the chain number to make the RAM 14 The chain encoded data read from _n and all the frame numbers and motion vectors from the similar chain detection circuit 15 are multiplexed to be output as multiplexed chain encoded data.

That is, for example, as shown in FIG. 38, one chain existing in the (n-3) th frame, the nth chain,
-Two chains existing in the -2 frame, three chains existing in the (n-1) th frame, one chain existing in the nth frame, one chain existing in the (n + 1) th frame, two chains existing in the (n + 2) th frame However, if it is assumed that the chain existing in the nth frame is connected as a basic chain, the select multiplexing circuit 16 outputs the chain encoded data for the basic chain of the nth frame, the nth frame to the nth-3rd frame. A multiplexed number of the number of frames and the motion vector to the preceding frame, and the number of frames from the nth frame to the (n + 2) th frame and the subsequent motion vector to the subsequent frame are output.

Therefore, the similar chain detection circuit 1 of FIG.
5 is also used, the similar chain detection circuit 15 of FIG.
As in the case of using a chain, a chain connected over multiple frames (similar chains when viewed in the time direction) is a chain encoded data for a chain (base chain) and the chain. Since it is output as a motion vector for moving (motion compensation) to restore another chain, redundancy in the time direction can be reduced, and as a result, image compression efficiency can be improved. it can.

Furthermore, the similar chain detection circuit 1 of FIG.
According to FIG. 5, as shown in FIGS. 31, 32, 34, and 35, one frame has one chain, but in other frames, for some reason,
Since the chain divided into a plurality of parts is connected, it is possible to further improve the compression rate as compared with the case in FIG.

In the above case, the RAM 64
Although the chain existing in the frame having the smallest number of chains stored in is set as the basic chain, it is also possible to set the chains of other frames as the basic chain. However, for the basic chain, since the chain encoded data is output as described above, it is preferable that the amount of chain encoded data is small from the viewpoint of improving the compression rate. Therefore, as described above, it is preferable that the chain existing in the frame having the smallest number of chains is the basic chain.

In the embodiment shown in FIG. 38, for example, three chains exist in the (n-1) th frame,
In such a case, the motion vector from each of the three chains to the previous frame (n-2th frame) or the subsequent frame may be different. In such a case,
It is possible to output the motion vector to the previous frame, that is, three motion vectors from each of the three chains existing in the (n-1) th frame, but from the viewpoint of improving the compression rate, the (n-1) th frame Of the motion vector to the previous frame from each of the three chains existing in the It is preferable to output as a representative value.

Next, FIG. 39 shows the configuration of an embodiment of an image decoding apparatus for decoding the multiplexed chain encoded data output from the image encoding apparatus of FIG. 1 into the original moving image. . The multiplexed chain encoded data output from the image encoding device of FIG. 1 is adapted to be supplied to the buffer 71 via the transmission line or reproduced from the recording medium 18, and the buffer 71 The multiplexed chain encoded data is once stored.
The separation circuit 72 reads the multiplexed chain encoded data stored in the buffer 71, and then extracts the chain encoded data and the motion vector. The chain encoded data or the motion vector is output to the chain decoding circuit 73 or the motion compensation circuit 77, respectively.

The chain decoding circuit 73 chain-decodes the chain-encoded data to obtain feature points,
The image forming circuit 7 is used as the quantized coefficient at the characteristic point.
4 is output. Image configuration circuit 75
Is configured to reconstruct the original image from the feature points and the quantized coefficients from the chain decoding circuit 73. That is, the image forming circuit 75 obtains image data by dequantizing the quantized coefficient, and arranges the image data at the feature points to form the image data forming the basic chain (hereinafter referred to as the basic chain as appropriate). (Referred to as data). This basic chain data is
It is adapted to be output to the selector 75 and the motion compensation circuit 77.

The selector 75 is adapted to output the basic chain data output from the image forming circuit 74 to _{one of the} RAMs 76 _{1 to} 76 _N configured as a frame memory for storage. Further, the selector 75 also outputs the image data output from the motion compensation circuit 77 to any _one of the RAMs 76 _{1 to} 76 _N for storage. As a result, the RAM 76 _n is configured to store the _n-th frame of the N-frame image data, which is the processing unit of the similar chain detection circuit 15 of FIG. The motion compensation circuit 77 performs motion compensation according to the motion vector output from the separation circuit 72 on the basic chain data output from the image configuration circuit 74, and outputs it to the selector 75. Read circuit 78, the image data of N frames are stored in the RAM 76 ₁ to 76 _N, the image data, RAM 76 ₁
In this order, the data is read out in the order of No. to 76 _N , and output to a monitor (not shown) for display.

Next, the operation will be described with reference to the flowchart in FIG. The multiplexed chain coded data input to the buffer 71 is temporarily stored therein. Then, in the separation circuit 72, the multiplexed chain encoded data stored in the buffer 71 is read out and separated into the chain encoded data and the motion vector in step S111. Then, the chain encoded data is output to the chain decoding circuit 73, and the motion vector is output to the motion compensation circuit 77.

In the chain decoding circuit 73, step S
At 112, the chain-encoded data is chain-decoded to form feature points and quantized coefficients, which are output to the image forming circuit 74. In step S113, the image forming circuit 74, based on the feature points and the quantization coefficient (decoded data) from the chain decoding circuit 73,
The basic chain data (image data forming the basic chain) is generated as described above. This basic chain data is supplied to the selector 75.

Here, in the case where the similar chain detection circuit 15 of the image encoding device is configured as shown in FIG. 17,
As mentioned above, the chain encoded data is for the chains present in the smallest frame. Therefore,
In this case, when the selector 75 receives the basic chain data from the image forming circuit 74, the selector 75 outputs the basic chain data to the RAM 76 _n having the frame number n of the minimum frame as a suffix for storage. In addition,
In the select multiplexing circuit 16 of FIG. 1, the frame number of the minimum frame is included in the multiplexed chain encoded data. Although not shown in FIG. 39, the frame number of this minimum frame is , Separation circuit 72
, Separated from the multiplexed chain encoded data,
It is adapted to be supplied to the selector 75. Then, in the selector 75, the basic chain data is transferred to the RAM 76 (R
Any of AM76 _{1 to} 76 _N ) is stored.

The basic chain data output from the image construction circuit 74 is supplied not only to the selector 75 but also to the motion compensation circuit 77. When the motion compensation circuit 77 receives the basic chain data from the image configuration circuit 74,
In step S114, the basic chain data is subjected to motion compensation according to the motion vector output from the separation circuit 72.

That is, since the motion vector output from the separation circuit 72 is the motion vector to the subsequent frame during the frame in which the chain connected to the basic chain exists, the motion compensation circuit 77 uses the basic chain data. But,
By performing motion compensation according to the first motion vector from the separation circuit 72, the image data of the chain existing in the frame (post-frame) next to the frame in which the basic chain exists (hereinafter referred to as the first chain data as appropriate). )
Is generated. Further, in the motion compensation circuit 77, according to the second motion vector from the separation circuit 72, the first motion vector is calculated.
Image data of the chain existing in the frame next to the frame in which the basic chain exists, and thus in the frame subsequent to the frame in which the basic chain exists (hence, the frame after the frame in which the first chain data exists). (Hereinafter, appropriately referred to as second chain data) is generated.

Thereafter, in the same manner, the motion compensation circuit 77 performs the motion compensation processing using all the motion vectors output from the separation circuit 72. Therefore, the separation circuit 72
When the number of motion vectors output from M is M (where M <N), the motion compensation circuit 77 generates the first to Mth chain data.

The first to Mth chain data generated by the motion compensation circuit 77 are sequentially output to the selector 75. Upon receipt of the first to Mth chain data, the selector 75 sequentially stores the chain data in the RAM 76 _n and the RAMs 76 _n and the RAMs in which the basic chain data is stored. That is, the selector 75 sequentially stores the _first to Mth chain data in the RAMs 76 _{n + 1 to} 76 _{n + M.}

As described above, when the data for all chains existing in N frames are stored in the RAMs 76 _{1 to} 76 _N , the read circuit 78 causes the RAM 7 to operate.
Data is sequentially read from 6 _{1 to} 76 _N and output to the monitor for display.

The RAMs 76 _{1 to} 76 _N are, for example, 1
It is composed of two banks capable of storing image data for frames. Then, for the image data of a certain N frames, when the image data is stored in one bank and is read out,
Regarding the image data for the next N frames, the image data is stored in the other bank, and when the image data stored in one bank is read out, the image data stored in the other bank is read out. Upon start, the image data for the next N frames is further stored in one bank. By the bank switching as described above, images are continuously displayed on the monitor without interruption.

In FIG. 41, as described in FIG. 26,
The decoded image obtained by decoding the multiplexed chain encoded data obtained by using the chain CH _n existing in the nth frame as a basic chain is shown. The chain CH _n of the n-th frame, which is the basic chain, can be obtained by chain-decoding the chain-coded data included in the multiplexed chain-coded data. Further, the chain CH _{n + 1} of the ( _{n + 1) th} frame corresponds to the decoded chain CH _n ,
Chain CH included in multiplexed chain encoded data
It can be obtained by motion compensation according to the motion vector from _n to the subsequent frame. Furthermore, the chain CH n _{+ 2} of the (n + 2) frame, the chain CH n _{+ 1} obtained by motion compensation chain CH _n, included in the multiplexed chain encoded data, the rear frame from the chain CH n _{+ 1} It can be obtained by performing motion compensation according to the motion vector to. After that, in the same way, basic chain C
It is possible to obtain up to the chain CH _N of the Nth frame connected to H _n .

Next, when the similar chain detection circuit 15 of the image coding apparatus is configured as shown in FIG. 27, as described above, the chain coded data is for the chains existing in the minimum frame. Is. Therefore, in this case, when the selector 75 receives the basic chain data from the image forming circuit 74, the selector 75 outputs the basic chain data to the RAM 76 _n having the frame number n of the minimum frame as a suffix for storage. The select multiplex circuit 16 of FIG. 1 includes the frame number of the minimum number of frames in the multiplex chain encoded data, which is not shown in FIG. 39.
The frame number of this minimum frame is separated from the multiplexed chain encoded data in the separation circuit 72 and supplied to the selector 75. Then, the selector 75 sends the basic chain data to the RAM 76 (RAM 7) based on the frame number from the separation circuit 72.
6 _{1 to} 76 _N ).

The basic chain data output from the image construction circuit 74 is supplied not only to the selector 75 but also to the motion compensation circuit 77. When the motion compensation circuit 77 receives the basic chain data from the image configuration circuit 74,
In step S114, the basic chain data is subjected to motion compensation according to the motion vector output from the separation circuit 72.

That is, from the separation circuit 72, the motion vector to the previous frame up to the temporally earliest frame in which the chain connected to the basic chain exists (hereinafter referred to as the forward motion vector, as appropriate). ), And a motion vector to the subsequent frame (hereinafter, appropriately referred to as a backward motion vector) up to the temporally latest frame in which a chain connected to the basic chain exists. Therefore, in the motion compensation circuit 77, the basic chain data is motion-compensated according to the first forward motion vector from the separation circuit 72, so that the image data of the chain existing in the frame preceding the frame in which the basic chain exists ( Hereinafter, the first previous frame chain data will be appropriately generated). Furthermore, in the motion compensation circuit 77,
The first previous frame chain data is motion-compensated according to the second forward motion vector from the separating circuit 72, so that the frame preceding the frame in which the basic chain exists (thus, the frame before the frame in which the basic chain is present). (1) The previous frame of the frame in which the previous frame chain data exists)
Image data of the chains existing in (hereinafter, appropriately referred to as second previous frame chain data) is generated.

Thereafter, in the same manner, the motion compensation circuit 77 performs the motion compensation processing using all the forward motion vectors output from the separation circuit 72. Therefore, when the number of forward motion vectors output from the separation circuit 72 is M1 (where M1 <N), the motion compensation circuit 77 generates first to M1th previous frame chain data. It will be.

Further, in the motion compensating circuit 77, the basic chain data is motion-compensated according to the first backward motion vector from the separating circuit 72, so that the basic chain data of the chain existing in the succeeding frame of the frame in which the basic chain exists. Image data (hereinafter, appropriately referred to as first post frame chain data) is generated. Furthermore, the motion compensation circuit 7
In 7, the first backward frame chain data is motion-compensated according to the second backward motion vector from the separation circuit 72, so that the frame after the frame in which the basic chain exists and the frame after the frame ( Therefore, the first
The image data of the chain existing in the frame subsequent to the frame in which the subsequent frame data is present (hereinafter, appropriately referred to as second subsequent frame chain data) is generated.

In the same manner, the motion compensation circuit 77 performs the motion compensation process using all backward motion vectors output from the separation circuit 72. Therefore, when the number of backward motion vectors output from the separation circuit 72 is M2 (where M2 <N), the motion compensation circuit 77 generates the first to M2th subsequent frame chain data. It will be.

When the similar chain detection circuit 15 of the image coding apparatus is configured as shown in FIG. 27, the multiplexed chain coded data is converted into the multiplexed chain coded data from the description in FIGS. 37 (B) and 38. Is the number of forward motion vectors M1
And the number M2 of backward motion vectors. The numbers M1 and M2 are separated from the multiplexed chain encoded data in the separation circuit 72 and supplied to the motion compensation circuit 77, whereby the motion compensation circuit 77 uses the forward motion vector. The motion compensation process is performed M1 times, and the motion compensation process using the backward motion vector is performed M2 times.

The first to M1th previous frame chain data generated by the motion compensation circuit 77 and the first to M2th data
The subsequent frame chain data is output to the selector 75. When the selector 75 receives the first to M1th previous frame chain data and the first to M2th subsequent frame chain data, the selector 75 sends the first to M1th previous frame chain data to the RAMs 76 _{n-1 to} 76 _n−. together they are respectively stored in _M1, the chain data of the frame after the first, second M2, and stores the respective RAM 76 n + ₁ to 76 _{n + M2.}

As a result, the RAMs 76 _{n -M1 to} 76 _{n + M2 are}
In, the chain data of consecutive M1 + M2 + 1 frames (each of the chain data of the (n−M1) th frame to the (n + M2) th frame) is stored.

[0197] As described above, data for all chains at the N frame, and stored in RAM 76 ₁ to 76 _N, as described above, by the readout circuit 78, the data from the RAM 76 ₁ to 76 _N It is read out sequentially, output to the monitor and displayed.

In FIG. 42, as described in FIG. 38,
The decoded image obtained by decoding the multiplexed chain encoded data obtained by using the chain CH _n existing in the nth frame as a basic chain is shown. The chain CH _n of the n-th frame, which is the basic chain, can be obtained by chain-decoding the chain-coded data included in the multiplexed chain-coded data. Further, the chain CH _{n- 1} of the n-11th frame is the decoded chain CH
_n can be obtained by motion compensation according to the motion vector from the chain CH _n to the previous frame included in the multiplexed chain encoded data. Furthermore, the n-th
Chain CH _n-2 two frames, a chain CH _n-1 obtained by motion compensation chain CH _n, included in the multiplexed chain encoded data, the motion from the chain CH _n-1 to the previous frame It can be obtained by motion compensation according to the vector. Similarly, the chain CH _n-3 of the n-3 frame, a chain CH _n-2 obtained by motion compensation of the chain CH _n-1, included in the multiplexed chain encoded data, the chain CH _n It can be obtained by motion compensation according to the motion vector from _-2 to the previous frame.

The chain CH of the (n + 1) th frame
_{n + 1} can be obtained by motion-compensating the decoded chain CH _n according to the motion vector from the chain CH _n to the subsequent frame included in the multiplexed chain encoded data. Furthermore, the chain C of the (n + 2) th frame
H n _{+ 2} is a chain CH n _{+ 1} obtained by motion compensation chain CH _n, included in the multiplexed chain encoded data, the motion according to the motion vector to the rear frame from the chain CH n _{+ 1} It can be obtained by compensation.

As shown in FIG. 38, the chains included in the original image are two for the (n-2) th frame, three for the (n-1) th frame, and two for the (n + 2) th frame. The image decoding apparatus of FIG. 39 performs the motion compensation processing based on the basic chain, so that, as shown in FIG. One chain, three chains of the (n-1) th frame, and two chains of the (n + 2) th frame will all be decoded into one chain. However, since the chains decoded in this way are connected in the image coding apparatus so as to form the same contour, there is a visual problem even if they are decoded as one chain. Rather, in this case, the chain that has been divided into two or more for some reason is displayed as one chain, so that the contour of the image can be made clear.

Next, FIG. 43 shows the structure of the second embodiment of the image coding apparatus to which the present invention is applied. In the figure, the same reference numerals are given to portions corresponding to the case in FIG. That is, the image coding apparatus is similar to the similar chain detection circuit 15 or the select multiplexing circuit 16
Instead of the above, a chain replacement circuit 115 or a select multiplexing circuit 116 is provided, and the configuration is similar to that of the image coding apparatus of FIG.

[0203] In the chain replacement circuit 115, present in the first frame, there chain (thus, stored in RAM 14 _1, there chain) is moved according to the motion vector to the rear frame from the chain, after the movement Chain (hereinafter referred to as moving chain as appropriate)
The importance degree (Visual Significant) indicating the visual importance in the subsequent frame is calculated. Further, the chain replacement circuit 115 replaces the chain around the moving chain among the chains existing in the rear frame with the moving chain when the importance is equal to or higher than the predetermined threshold T1, and uses the replaced chain. Then, the same processing is performed for the frames thereafter until the importance does not exceed the threshold T1 or until the Nth frame. Then, after that, the chain replacement circuit 115 moves the chain encoded data corresponding to the chain of the first frame used first and the motion to the subsequent frame from the first frame to the last frame where the chain replacement is performed. The vector is determined to be multiplexed in the select multiplexing circuit 116, and the chain number of the chain of the first frame used first and the number of frames in which the chain has been replaced (this is the movement to the subsequent frame described below). (Equal to the number of vectors) and the motion vector to the subsequent frame from the first frame to the last frame where the chain replacement is performed are output to the select multiplexing circuit 116.

[0203] select multiplexing circuit 116, the motion vector from the chain on coded data for the chain corresponding to the chain number of the replacement circuit 115, read from RAM 14 _1, the chain coded data, and chain substitution circuit 115 And the number thereof are multiplexed to form multiplexed chain encoded data, which is output to the buffer 17.

FIG. 44 shows the chain replacement circuit 11 of FIG.
5 shows a configuration example of No. 5. Chain map circuit 121
Is the RAM 14 ₁ to 14 _N, the N frames of the chain coded data is stored, reading the chain coded data of the first frame from the RAM 14 _1, RAM (Ch
ain RAM) 122 for storage. Further, the chain map circuit 121, similar to the case of the chain map circuit 51 of FIG. 18, stores the bitmaps of the chains existing in the second to Nth frames in the RAM.
(Map RAM) 123 _{1 to} 123 _N-1 .

In the chain map circuit 51 of FIG. 18, as described with reference to FIG. 20, the chain is expanded as a bit map by the chain number, and the address corresponding to the position (pixel) where the chain does not exist. However, in the chain map circuit 121 of FIG. 44, for example, 1 is stored in the address corresponding to the position where the chain exists, and 1 is stored in the address corresponding to the position where the chain does not exist. , For example, 0 is stored. That is, in the nth frame (however,
In this case, n ≠ 1), for example, when there are chains CH1 to CH5 as shown in FIG.
_A bitmap as shown in FIG. 45 is expanded in _n-1 .

The RAM 122 stores the chain encoded data of the first frame, and the RAMs 123 _{1 to} 123 _N-1.
Then, when the bitmaps for the chains of the 2nd to Nth frames are respectively developed, in the replacement circuit 124,
For each chain existing in the first frame stored in the RAM 122, the process according to the flowchart of FIG. 46 is performed.

That is, first, in step S121, the RAM (motion vector RAM) 126 is initialized, the process proceeds to step S122, the first frame (head frame) is set as the frame of interest, and the process proceeds to step S123. In step S123, a motion vector from a certain chain (a chain of interest) existing in the frame of interest, that is, the first frame to a subsequent frame is calculated. In the calculation of the motion vector in step S123, the replacement circuit 124 replaces the data necessary for the calculation of the motion vector with
The motion vector calculation circuit 125 is configured to read out from the RAMs 122 and 123 ( _{one of the} RAMs 123 _{1 to} 123 _N ) and output to the motion vector calculation circuit 125.

After calculating the motion vector, step S124.
Then, the importance (visual importance) of the chain of interest in the subsequent frame is calculated. That is, in step S124, for example, as shown in FIG. 47 (A), the chain of interest is moved according to the motion vector obtained in step S123, and in the subsequent frame, as shown in FIG. Later attention chain (moving chain)
The number of feature points existing in the range of L1 × L2 pixels (the range surrounded by the dotted line in the drawing) centered on each of the pixels (the part shaded in the drawing) that configures the above is calculated. Further, in step S124, after the number of characteristic points existing in the range of L1 × L2 pixels centered on each pixel forming the moving chain in the subsequent frame is calculated, the average value thereof (each of the forming points forming the moving chain is calculated). L1 centered on the pixel
The sum of the number of feature points existing in the range of × L2 pixels is divided by the number of pixels forming the moving chain), and this is the importance.

Here, in the RAM 123 _n , as shown in FIG.
As described in 5, the address corresponding to the pixel (feature point) forming the chain in the (n + 1) th frame is set to 1
Is stored in the other addresses, and 0 is stored in the other addresses. Therefore, when the frame of interest is the nth frame, the RAM 1 is stored in step S124.
23 _n by referring to the bitmap stored in _n , centering on each pixel forming the moving chain
By calculating the sum of the stored values of the addresses corresponding to the range of × L2 pixels (for example, L1 = L2 = 3), each pixel forming the moving chain is centered in the subsequent frame (in this case, the (n + 1) th frame). Then, the number of feature points existing in the range of L1 × L2 pixels is calculated.

The importance calculated as described above can be said to be the density of feature points existing in the range of L1 × L2 pixels centering on each pixel forming the moving chain in the subsequent frame. As the degree of importance, it is possible to adopt an amount other than the density of such feature points.

That is, for example, in the subsequent frame, among the pixels forming the moving chain, L1 having the pixel as the center is L1.
It is possible to divide the number of feature points existing in the range of × L2 pixels by the number of pixels forming the moving chain, and use the value obtained as a result as the importance. Also, for example,
RAM123 ₁ to 123 _N-1, second or so as to store the image data of the N frame, see it,
In the subsequent frame, the edge strength of each pixel corresponding to each pixel forming the moving chain (the edge strength of each pixel existing in the moving chain in the succeeding frame) is obtained, and the average value of the edge strengths is taken as the importance. It is also possible.

After the importance is calculated, the process proceeds to step S125, and it is determined whether the importance is equal to or higher than a predetermined threshold value T1. If it is determined in step S125 that the degree of importance is not equal to or higher than the predetermined threshold T1, it is determined that there is no chain connected to the target chain (a chain that (substantially) overlaps with the target chain after movement) in the rear frame. Steps S126 to S128 are skipped, and the process proceeds to step S129. If it is determined in step S125 that the degree of importance is equal to or higher than the predetermined threshold value T1, that is, if there is a chain connected to the chain of interest in the subsequent frame, the process proceeds to step S126, and the chain of the frame is changed thereafter. , Is replaced with the moving chain (this replacement means that the replacement circuit 124 recognizes the chain of the rear frame to be connected to the target chain by replacing it with the moving chain).

Further, in step S126, the motion vector to the subsequent frame calculated in step S123 is
It is stored in the RAM 126, and the process proceeds to step S127. In step S126, the RAM 123 is also updated. That is, assuming that there is a chain to be connected to the chain of interest in the rear frame, that chain is
When it is replaced with a moving chain, it is not necessary to process the chain thereafter. Therefore, step S1
26, for example, if the frame of interest is the nth frame, then the n + 1th frame stored in the RAM 123 _n is stored.
In the bit map of the frame, the stored values of the addresses corresponding to the range of L1 × L2 pixels centering on each pixel forming the movement chain are all set to 0. This allows
In the (n + 1) th frame, so-called feature points in the range of L1 × L2 pixels centered on each pixel forming the moving chain are deleted.

Then, in step S127, the subsequent frame is newly set as the frame of interest, and step S128.
To the Nth frame (final frame).
Is determined. If it is determined in step S128 that the frame of interest is not the Nth frame, the process returns to step S123, and the frame of interest is newly included in step S127.
The process after step S123 is performed using the chain (moving chain) replaced in 126 as the target chain.

If it is determined in step S128 that the new frame of interest is the Nth frame,
Proceeding to step S129, the chain of the frame (hence the first frame) selected as the frame of interest in step S122, which is the chain of interest first, is set as the basic chain, and its chain number (this is the chain code When the converted data is stored in the RAM 122, it is attached to each chain), the motion vector stored in the RAM 126 (motion vector to the subsequent frame), and the number of the motion vectors. , Are output to the select multiplexing circuit 116, and the processing is terminated.

As described above, the processing shown in FIG. 46 is performed for all chains existing in the first frame.

[0217] In the select multiplexing circuit 116, as described above, the chain coded data for basic chains present in the first frame is read from the RAM 14 _1, and the chain coded data, the chain substitution circuit 1
The motion vector from 15 and its number are multiplexed and output as multiplexed chain encoded data.

That is, for example, as shown in FIG. 48, assuming that the chains CH _{1 to} CH _N respectively existing in the _first to Nth frames are connected, the select multiplexing circuit 116 outputs the information about the chain CH ₁ Of the chain encoded data, the motion vectors of the chains CH ₁ to CH _N-1 to the subsequent frame and the number N-1 thereof are output.

Therefore, the chain that can be replaced (connected) with the target chain of the first frame is the chain encoded data of the target chain (basic chain),
Since it is output as a motion vector for moving (compensating for) that chain and restoring the other chain,
Again, the redundancy in the time direction can be reduced, and as a result, the image compression efficiency can be improved.

The RAM 122 shown in FIG.
As in the case of 123 _{1 to} 123 _N-1 , it is possible to expand the bitmap of the first frame. However, in this case, the RAM 122 needs to be a frame memory capable of storing a bit map for one frame, and therefore needs to have a large storage capacity. On the other hand, as described above, the RAM1
In the case where the chain encoded data existing in the first frame is stored in 22, the chain encoded data for one frame usually has a smaller data amount than the bitmap for one frame. , RAM122
Also, it is possible to use a memory having a small storage capacity.

By the way, according to the image coding apparatus of FIG. 43, it is possible to improve the compression efficiency of the image data as described above, but only the chains existing in the first frame are shown in FIG. Because the processing is performed, the second
When a chain first appears in a frame subsequent to the frame (for example, when the scene changes in the second frame and subsequent frames), information about the chain is not output from the select multiplexing circuit 116, and It becomes difficult for the decoding side to decode the chain.

Therefore, FIG. 49 shows the configuration of the third embodiment of the image coding apparatus to which the present invention is applied. Note that, in the figure, parts corresponding to the case in FIG. 43 are denoted by the same reference numerals. That is, this image encoding device
Chain replacement circuit 115 or select multiplexing circuit 11
43, except that a chain replacement circuit 215 or a select multiplexing circuit 216 is provided in place of 6 respectively.
The image coding apparatus is configured similarly to the image coding apparatus.

The chain replacement circuit 215 performs the same processing as the chain replacement circuit 115 of FIG. 43 on not only the chains existing in the first frame but also the chains existing in the second and subsequent frames, which will be described later. As described above, the frame number, the chain number, the motion vector, and the number of the motion vectors are selected by the select multiplexing circuit 216.
Output to

The select multiplexing circuit 216 reads out the chain-encoded data of the chain corresponding to the chain number from the chain permutation circuit 215 from the RAM 14 _n similarly having the frame number n from the chain permutation circuit 215 as a suffix, The chain encoded data, the motion vector from the chain replacement circuit 215 and the number thereof are multiplexed to form multiplexed chain encoded data, which is output to the buffer 17.

FIG. 50 is a circuit diagram of the chain replacement circuit 21 of FIG.
5 shows a configuration example of No. 5. Note that, in the figure, portions corresponding to those in the chain replacement circuit 115 of FIG. 44 are designated by the same reference numerals. That is, the chain replacement circuit 215 is replaced by the replacement circuit 13 instead of the replacement circuit 124.
2 is provided, and further, a new chain encoding circuit 131
_The configuration is the same as that of the chain replacement circuit 115 in FIG. 44, except that ₁ to 131 _N-1 and the chain decoding circuits 133 _{1 to} 133 _N-1 are newly provided.

In the chain replacement circuit 215 having the above-described structure, the RAM is replaced by the RAM as in the case of FIG.
To 122, the chain coded data of the first frame is stored, in the RAM 123 ₁ through 123 _N-1, the bit map for the second to the chain of the N-th frame is deployed, respectively, in a substitution circuit 132, FIG. 51 The process according to the flowchart of FIG.

That is, first, in step S131, a variable n for counting the number of frames is set to 1 as an initial value, and the process proceeds to step S132 to perform a replacement process based on the nth frame as described later. Be seen. Then, when the replacement process based on the nth frame is completed, the variable n is incremented by 1 in step S133, the process proceeds to step S134, and the variable n
Is less than or equal to N, which is the number of frames in a processing unit. In step S134, the variable n is N
When it is determined that the following is true, the process returns to step S132, and the replacement process based on the nth frame is performed again.
If it is determined in step S134 that the variable n is not equal to or less than N, the process ends.

Next, FIG. 52 is a flow chart showing details of the replacement process based on the nth frame of FIG. In the replacement process based on the nth frame, step S14
In 1 to S148, the same processes as in steps S121 to S128 of FIG. 46 are performed, respectively, except that the nth frame is set as the frame of interest instead of the first frame in step S142.

Here, since the chain coded data for the chains existing in the first frame is stored in the RAM 122, the chain coded data stored in the RAM 122 is used for the chains existing in the first frame. , The chain existing in the first frame can be recognized, and the motion vector from that chain to the subsequent frame can be calculated, for example, in step S143. Since only the bit map in which the stored value of the address corresponding to the position of the feature point forming the is set to 1 is stored in the RAMs 1231 to 123N-1, the chains existing in the second and subsequent frames can be recognized. A bit map of the frame of interest in the replacement circuit 132 It is necessary to scan the storage contents of the RAM 123 in which is expanded and reconfigure a portion in which 1s are continuous as a chain, which requires time for processing.

[0230] Further, the bit map stored in RAM 123 ₁ through 123 _N-1, in step S146, but is updated as in step S126 described in FIG. 46, the substitution circuit 132, after the update Need to recognize the chain of.

Therefore, in step S142, the second
When the frame after the frame is set as the frame of interest,
The replacement circuit 132 recognizes a chain existing in the frame of interest as follows.

That is, for example, if it is assumed that the n-th frame is the frame of interest in step S142 (here, n is an integer of 2 or more and N or less),
The control signal is output to the new chain encoding circuit 131 _n-1 configured similarly to the chain encoding circuit 12 described in FIG. The chain encoding circuit 131 _n-1 is the replacement circuit 1
When the control signal is received from 32, the chain decoding circuit 1
33 _n-1 is controlled, whereby the chain encoded data of the nth frame is read from the RAM 14 _n and the chain is decoded. The image data of the n-th frame obtained by being chain-decoded by the chain decoding circuit 133 _n-1 is supplied to the new chain encoding circuit 131 _n-1 , where the contents stored in the RAM 123 _n-1 are stored. The chain is coded again with reference.

That is, from the chain decoding circuit 133 _n−1 , in addition to the image data of the nth frame, the nth frame obtained by decoding the chain encoded data of the nth frame is decoded.
The feature point of the frame is also the new chain encoding circuit 131.
_In the new chain encoding circuit 131 _n-1 , the chain decoding circuit 133 _n-1 is supplied to the _n-1.
The point (pixel) which is the feature point of the n-th frame from No. ₁ and whose stored value of the address of the RAM 123 _n-1 corresponding to the feature point is 1 is, so to speak, a true feature point, and is referred to as a chain decoding circuit. The image data of the n-th frame supplied from 133 _n-1 is chain-coded, and the chain-coded data obtained as a result is output to the replacement circuit 132.

Therefore, the new chain encoding circuit 131
_{The n-} th frame chain encoded data output from _n-1 reflects the updated contents of the RAM 131 _n-1 .

In the replacement circuit 132, as described above,
R supplied from the new chain encoding circuit 131 _n-1
The chain existing in the n-th frame is recognized based on the chain-encoded data of the n-th frame that reflects the updated contents of AM131 _n-1 .

When the replacement circuit 132 recognizes a chain existing in the nth frame, the chain is given a unique chain number.

Then, in the replacement circuit 132, step S
In 149, as in the case of step S129 in FIG. 46, the chain number of the chain initially set as the target chain, the motion vector (motion vector to the subsequent frame) stored in the RAM 126, and the number of the motion vectors are The frame number of the frame that is output to the select multiplex circuit 216 and in which the chain first set as the target chain exists (the frame set as the target frame in step S142) is also selected.
Is output to

The process shown in FIG. 52 is performed in step S
This is performed for all chains existing in the nth frame which is the frame of interest at 142.

In the select multiplex circuit 216, as described above, from the RAM 14n having the frame number n output from the chain replacing circuit 215 as a suffix, the chain corresponding to the chain number output from the chain replacing circuit 215 (basic Chain-encoded data for a chain) is read, and the chain-encoded data is multiplexed with the motion vector and the number thereof from the chain replacement circuit 215 to be output as multiplexed chain-encoded data. To be done.

That is, for example, as shown in FIG. 53, assuming that the chains CH _{1 to} CH _N respectively existing in the _first to Nth frames are connected, the case shown in FIG. 48 from the select multiplexing circuit 216. Similarly, the chain coded data for the chain CH _1, obtained by multiplexing the motion vector and the number N-1 thereof to the frame after the chain CH ₁ to the chain CH _N-1 is output. Furthermore, as shown in the figure, in the second frame,
Even when the chain CH ₂ ′ appears for the first time, the “replacement process based on the second frame” is performed in step S132 of FIG. 51, so that the chain CH ₂ ′ is connected to the chain (the embodiment of FIG. 53). Is connected to the chain CH ₃ ') of the third frame, and as a result, the chain from the select multiplexing circuit 216 to the chain C
Chain encoded data for H ₂ 'and chain C
A multiplexed motion vector for the subsequent frame of H ₂ 'and its number 1 is also output.

Therefore, in this case, even if the chain appears for the first time in the second and subsequent frames, the decoding side can decode the chain.

The multiplexed chain coded data output from the image coding apparatus shown in FIGS. 43 and 49 is shown in FIG.
The image decoding apparatus described in 9 can perform decoding in the same manner as in the above case.

However, the decoded image is as shown in FIG. 54 and FIG.
As shown in FIG. That is, for example, in FIG.
As shown in (A), in the original image, when a certain chain is present in the nth frame and three chains which are obtained by dividing the chain are present in the rear frame, the nth frame The chain is the basic chain,
When connected (replaced) with the three chains of the subsequent frame, the image decoding apparatus decodes the chains of the subsequent frame by motion-compensating the basic chain. As shown in FIG. 6B, the data is decoded as one chain.

Also, for example, as shown in FIG. 55 (A), in the original image, there are three chains in the nth frame, and in the rear frame there is one chain which is a combination of these chains. If present, each of the three chains in the nth frame is a base chain,
When connected to (replaced by) one chain of the subsequent frame, the image decoding apparatus decodes the chain of the subsequent frame by motion-compensating the basic chain. As shown in (B) of the same figure, it is decoded as three chains.

Although the image coding apparatus and the image decoding apparatus to which the present invention is applied have been described above, such an image coding apparatus and an image decoding apparatus are incorporated in, for example, a video tape recorder or a video conference system. It is possible.

In this embodiment, the image data is chain-encoded. However, the present invention is not limited to this.
It is applicable to the case where the image data is subjected to structure extraction coding other than chain coding.

[0247]

As described above, according to the image coding apparatus and the image coding method of the present invention, the data relating to the characteristic points of the moving image are coded, and the data obtained as a result and the chain connecting the characteristic points are formed. Since it is multiplexed with the motion vector, it is possible to reduce the amount of generated code.

According to the image decoding apparatus and the image decoding method of the present invention, the encoded data obtained by encoding the data relating to the feature points of the moving image and the motion vector of the chain connecting the feature points are extracted from the transmission data. Then, the encoded data is decoded. And the resulting decrypted data is
Motion compensation is performed according to the motion vector. Therefore, it is possible to obtain a decoded image from the transmission data including the encoded data obtained by encoding the data regarding the feature points of the moving image and the motion vector of the chain connecting the feature points.

According to the recording medium of the present invention, there is recorded therein the data including the coded data obtained by coding the data relating to the characteristic points of the moving image and the motion vector of the chain connecting the characteristic points. It becomes possible to record a long-time moving image.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of an image encoding device to which the present invention has been applied.

2 is a block diagram showing a configuration example of a chain encoding circuit 12 in FIG.

FIG. 3 is a diagram for explaining storage contents of a coefficient frame buffer 21 of FIG.

FIG. 4 is a flowchart illustrating an operation of the direction searcher 23 of FIG.

FIG. 5 is a flowchart that follows the flowchart of FIG.

FIG. 6 is a diagram showing storage contents of ROMs 30 and 31 of FIG.

FIG. 7 is a diagram for explaining an address output by the address generator 32 of FIG.

FIG. 8 is a diagram showing stored contents of a ROM 33 of FIG.

9 is a diagram showing a direction represented by direction data output from a ROM 33 of FIG.

10 is a flowchart following the flowchart of FIG.

11 is a timing chart showing the output timing of the valid data selection signal from the direction searcher 23, the direction data from the ROM 33, and the forward direction data from the latch circuit 34 in FIG.

12 is a diagram for explaining direction change data output from the direction change signal generator 35 of FIG.

13 is a diagram showing a correspondence table of forward direction data and direction data stored in a direction change signal generator 35 of FIG. 2 and direction change data.

14 is a diagram showing a correspondence table of forward direction data and direction data stored in the direction change signal generator 35 of FIG. 2 and direction change data.

15 is a diagram showing code words assigned to the direction change data output from the direction change signal of FIG.

16 is a diagram for explaining the data output from the selector 36 of FIG.

17 is a block diagram showing a configuration example of a similar chain detection circuit 15 of FIG.

18 is a block diagram showing a configuration example of a similarity detection circuit 41 of FIG.

FIG. 19 is a diagram showing an example of chains existing on an image.

20 is a diagram showing the stored contents of the RAM 52 _n of FIG. 18 when the chain as shown in FIG. 19 exists on the image of the nth frame.

21 is a flowchart illustrating the operation of the similarity calculation circuit 53 of FIG.

22 is a diagram for explaining the process shown in the flowchart of FIG.

23 is a diagram showing stored contents of a RAM 42 _n shown in FIG.

24 is a flowchart illustrating the operation of the chain connection circuit 43 of FIG.

FIG. 25 is a diagram for explaining the process shown in the flowchart of FIG. 24.

26 is a diagram for explaining multiplexed chain encoded data output from the select multiplex circuit 16 of FIG.

27 is a block diagram showing another configuration example of the similar chain detection circuit 15 of FIG. 1. FIG.

FIG. 28 is a diagram showing stored contents of a RAM 62 _n shown in FIG. 27.

29 is a flowchart explaining the operation of the chain connection circuit 63 of FIG.

FIG. 30 is a flowchart illustrating details of the process of step S64 of FIG. 29.

31 is a diagram for explaining the processing of steps S71 to S74 of FIG. 30. FIG.

32 is a diagram for explaining the processing of steps S75 to S79 in FIG. 30. FIG.

FIG. 33 is a flowchart illustrating the details of the process of step S65 of FIG.

34 is a diagram for explaining the processing of steps S91 to S94 of FIG. 33. FIG.

FIG. 35 is a diagram for explaining the processing of steps S95 to S99 of FIG. 33.

36 is a diagram for explaining the process shown in the flowchart of FIG. 29. FIG.

FIG. 37 is a diagram for explaining the process shown in the flowchart of FIG. 29.

38 is a diagram for explaining multiplexed chain encoded data output from the select multiplex circuit 16 of FIG. 1. FIG.

[Fig. 39] Fig. 39 is a block diagram illustrating the configuration of an embodiment of an image decoding device to which the present invention has been applied.

40 is a flowchart for explaining the operation of the image decoding device in FIG. 39.

41 is a diagram for explaining a decoded image output from the image decoding device in FIG. 39. FIG.

[Fig. 42] Fig. 42 is a diagram for describing a decoded image output from the image decoding device in Fig. 39.

[Fig. 43] Fig. 43 is a block diagram illustrating a configuration of a second embodiment of an image encoding device to which the present invention has been applied.

44 is a block diagram showing a configuration example of the chain replacement circuit 115 of FIG. 43.

45 is a diagram showing stored contents of a RAM 123 _n shown in FIG. 44.

FIG. 46 is a flowchart illustrating an operation of replacement circuit 124 of FIG. 44.

FIG. 47 is a diagram for explaining the process of step S124 of FIG. 46.

48 is a diagram for explaining multiplexed chain encoded data output from the select multiplex circuit 116 of FIG. 43.

[Fig. 49] Fig. 49 is a block diagram illustrating a configuration of a third embodiment of an image encoding device to which the present invention has been applied.

50 is a block diagram showing a configuration example of the chain replacement circuit 215 of FIG. 49.

51 is a flowchart illustrating the operation of the replacement circuit 132 of FIG. 50.

52 is a flowchart illustrating details of the process of step S132 of FIG. 51.

53 is a diagram for explaining multiplexed chain encoded data output from the select multiplex circuit 216 of FIG. 49.

54 is a diagram for explaining a decoded image obtained by decoding the multiplexed chain encoded data output from the image encoding device in FIG. 43 (FIG. 49).

[Fig. 55] Fig. 55 is a diagram for describing a decoded image obtained by decoding the multiplexed chain encoded data output from the image encoding device in Fig. 43 (Fig. 49).

[Fig. 56] Fig. 56 is a block diagram illustrating a configuration of an example of a conventional image encoding device.

[Explanation of symbols]

10 Two-dimensional change point detection circuit 11 Quantizer 12 Chain coding circuit 13 Selector 14 _{1 to} 14 _N RAM 15 Similar chain detection circuit 16 Select multiplex circuit 17 Buffer 41 Similarity detection circuit 42 _{1 to} 42 _N RAM 43 Chain connection Circuit 44 RAM 51 Chain map circuit 52 _{1 to} 52 _N RAM 53 Similarity calculation circuit 54 Motion vector calculation circuit 61 Multiple chain similarity detection circuit 62 _{1 to} 62 _N RAM 63 Multiple chain connection circuit 64 RAM 71 Buffer 72 Separation circuit 73 Chain decoding circuit 74 the image configuration circuit 75 selector 76 ₁ to 76 _N RAM 77 motion compensation circuit 78 read circuit 115 chain substitution circuit 116 select multiplexing circuit 121 chain map circuit 122 ₁ through 123 _N RAM 124 substituted Road 125 motion vector calculation circuit 126 RAM 131 ₁ through 131 _N-1 new chain coding circuit 132 replacing circuit 133 ₁ through 133 _N-1 chain decoding circuit 215 chain permutation circuit 216 select multiplexing circuit

Claims (14)

[Claims] 1. An image encoding apparatus for encoding a moving image, comprising: a feature point encoding means for encoding information about a feature point of the moving image and outputting encoded data; and a chain connecting the feature points. A chain detecting means for detecting, a motion vector detecting means for detecting a motion vector of the chain detected by the chain detecting means, and encoded data output from the feature point encoding means,
An image coding apparatus comprising: a multiplexing unit that multiplexes the motion vector detected by the motion vector detecting unit. 2. The system further comprises a determining means for connecting the chains existing in different frames and determining the encoded data and the motion vector to be multiplexed by the multiplexing means based on the connection result. The image encoding device according to claim 1. 3. The method according to claim 2, wherein the determining means calculates the degree of similarity between the chains existing in adjacent frames and connects the chains based on the degree of similarity. Image coding device. 4. The determination means includes the encoded data corresponding to a predetermined one of the chains connected over a plurality of frames, and the motion vector between adjacent frames of the plurality of frames. The image encoding device according to claim 2, wherein the image encoding device is multiplexed by the multiplexing means. 5. The determining means determines, from the predetermined frame, the similarity between the chain A existing in a predetermined frame and another chain B existing in an adjacent frame adjacent to the predetermined frame. According to the motion vector to the adjacent frame, the chain A is moved, and among the pixels forming the chain A after the movement, the distance to the chain B is the largest among the chains existing in the adjacent frame. The image coding apparatus according to claim 3, wherein the image coding apparatus calculates the number based on the number of close ones. 6. The determining means calculates the similarity SA of the chain A with respect to the chain B and the similarity SB of the chain B with respect to the chain A, and the similarity SA and SB are The image coding apparatus according to claim 5, wherein the chains A and B are connected when both are equal to or more than a predetermined threshold value. 7. When the chain C is present in one of the adjacent frames and a plurality of chains D1, D2, D3, ... Is present in the other frame, the determining means is Of the chain C, the similarity SC1, SC2, SC3, ... With respect to each of the plurality of chains D1, D2, D3, ..., and each of the plurality of chains D1, D2, D3 ,. The similarity SD1, SD2, SD3, ...
.And are calculated, and the similarity SD1, SD2, SD3 ,.
.. are both equal to or greater than a predetermined threshold value, and the sum of the similarities SC1, SC2, SC3, ... Is also equal to or greater than a predetermined threshold value, the chain C and the chain D1,
The image coding apparatus according to claim 5, wherein the image coding apparatus is connected to each of D2, D3, ... 8. A chain A existing in a predetermined frame is moved according to the motion vector from the predetermined frame to an adjacent frame adjacent thereto, and the adjacent frame of the chain A after the movement. In the chain that exists in the adjacent frame when the importance calculated by the importance calculating unit that calculates the importance indicating the visual importance in the, and the importance calculated by the importance calculating unit is equal to or more than a predetermined threshold value. Further comprising: a replacement unit that replaces the one around the chain A after the movement with the chain A after the movement, wherein the multiplexing unit includes the encoded data corresponding to the chain A and the replacement unit. The method according to claim 1, wherein the motion vector to the adjacent frame in which the chains have been replaced by the above is multiplexed. The image coding apparatus. 9. The degree of importance calculation means calculates the degree of importance based on the characteristic points existing around the chain A after the movement in the adjacent frame. The image encoding device described. 10. The image code according to claim 8, wherein the importance degree calculating means calculates the importance degree based on the edge strength on the chain A after the movement in the adjacent frame. Device. 11. An image coding method for coding a moving image, wherein data on a characteristic point of the moving image is coded, and a motion vector of a chain connecting the characteristic points is detected and coded. An image encoding method, characterized in that the data and the motion vector are multiplexed. 12. An image decoding apparatus for decoding a moving image, wherein encoded data obtained by encoding data relating to a characteristic point of the moving image from transmission data and a motion vector of a chain connecting the characteristic points are transmitted. An image decoding apparatus comprising: an extracting unit for extracting, a decoding unit for decoding the encoded data, and a motion compensating unit for compensating the output of the decoding unit according to the motion vector. 13. An image decoding method for decoding a moving image, wherein encoded data obtained by encoding data relating to characteristic points of the moving image from transmission data and a motion vector of a chain connecting the characteristic points are transmitted. An image decoding method comprising: extracting, decoding the encoded data to obtain decoded data, and performing motion compensation on the decoded data according to the motion vector. 14. A recording medium on which is recorded data obtained by encoding a moving image, wherein the data is a chain connecting the encoded data obtained by encoding data relating to the characteristic points of the moving image and the characteristic points. And a motion vector of the recording medium.