JPH01500236A - Bandwidth compressed video signal reception and playback device - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] [Name of invention] Video signal processing device for in-band compression It relates to a method that allows pressure 1i1 (reduction) in the band of the signal. Regarding the present invention European standard 625 line, 50 fields/5 (European 625 1ine, 50 field/s) interlace standard” in detail Although described, the invention is not limited to any particular standard; -Can also be used for both interlaced and non-interlaced (sequential) formats. Ru. Whenever the terms “transmitter” and “receiver” (and similar terms) are used, However, the terms "recording device" and "playback device" may be substituted even if they are not expressed as such. It can be understood as . The present invention is applicable to HDTV (high-definition television) transmission. This is a specific utility.

(References) 1, L, Chiariglione, L, Corgnier and M. Guglie-1mo, “Preprocessing in Video Terminals Using Motion Vectors” and Post-processing method” (1, B, C, BrightLon, 1981+) 2 , “Vertical resolution line interlacing” by B. Girod and R. Thoma. ・Lossless motion compensation conversion method J for television systems (November 1986, L' Eurasip Wo on “Aquila, rHDTVHDTV Method” rkshop) 3, R, 5torey, HDTV method using rDATV Motion adaptive bandwidth compression method J (BBCRe5search Departa+ent Report 198615゜UK Patent Application No. 8531777) 4, R, 5to "625 Line - Compatible Transmission Method for Channel Niokel HDTV" written by Ray ( British Patent Specification No. 8620110) 5, G., A., Thomas, “Adaptive Subsampling and Motion Compensation D Medium-band compression using ATV method J (1986, October) (128th SMPT E Technical Conference % 1986 lθ Month 24-2 (9th, New York) British Patent Application Nos. 8606809 and 86 No. 17320.

6, Y., Nihomiya et al., “Single Channel HDTV Broadcasting System, T he MUSEJ (N)I Research Institute, Journal No. 304) Subsampling was carried out. not a periodic sequence of fields, e.g. a sequence of four fields. Each cycle of the field sends a different set of sub-sampling points at A still image is generated by forming an image at the receiver by accumulating the points of It is possible to perform in-band compression of the image. This in-band compression is effectively This is achieved using a filter technique.

Further compression can be achieved by transmitting only a certain point in this way and using interpolation on the receiver side. This can be done by forming other points.

Temporal filtering methods are recommended for images with movement, as they may cause image blurring. It cannot be applied to areas. Already switched to the low spatial detail filter and started working. It has been proposed to provide compressed mid-band signals for the area (see document 3).

However, the motion areas that maintain good association between fields with high detail It is desirable to be able to transmit the area. Solution when low spatial detail filter is used The loss of image resolution is quite obvious.

It is an object of the present invention to The purpose is to enable compression.

Although the present invention is particularly defined in the claims of Bunun, the present invention is specifically defined in the claims of Bunun. I will now describe the aspects in detail.

(Brief explanation of the drawing) FIG. 1 is a diagram showing the basic sample configuration used in the in-band compression method; Figure 2 is a diagram that reproduces a fine image in a static area, and Figure i3a is a diagram showing low spatial detail. ・The effective sampling grid used in the motion domain to act as a filter diagram showing, Figure 3b shows a graph showing the required prefilter characteristics. Figure 4 shows how the velocity vector is interpolated when the vector is measured over the image period. diagram showing, Figure 5 shows the effect of block size on the number of samples in one block. Figure, Figure 6 shows a simple two-dimensional linear interpolator, Figure 7 uses motion vector information. Diagram showing reproduction of a fine image; FIG. 8 is a diagram showing the regeneration process shown in the one-dimensional case; Figure 9 shows the problem of unclear background and how the “follow back” method works. Figure 1θ shows the entire transmission system incorporating the present invention. Block Diagram, FIG. 11 is a block diagram of a second embodiment of the present invention, and FIGS. 12 and 13 are a block diagram of a second embodiment of the present invention. Characteristic diagram of the prefilter of the system in Figure 1, Figures 14a and 14b show the sampling grid for the embodiment of Figure 11. Figure, Figure 15 is a diagram showing the operation of a simple interlaced/sequential converter; Figures 16 to 19 show spectra of sampled objects moving at various speeds. FIG.

Some hardware was recently built to study motion-adaptive in-band compression schemes. (Refer to Reference 1) This device has an input monochrome 625-line television screen. contains two digital filters that filter the signal in two different ways. , one is best suited for the moving region of the image (spatial filter), and one is best suited for the static image region. (temporal filter) that is optimal for the area. Both types of filters Compress by a factor of 4, so that the filtered signal is a repeating sequence of four fields. allows resampling on a relatively coarse sampling grid with Ru. In FIG. 1, one point represents one location that is not sampled. numbers 1 to 4 are the fields for a repeating sequence of four fields, respectively. It represents the locations sampled in modes 1 to 4. This method creates a fine image. The samples obtained over one 4-field period are accumulated at the receiver to This makes it possible to transmit extremely fine images in static areas. do.

This is shown in Figure 2, where the upper part is a continuous field n- 3, n-2, n-1 and '', the lower part shows the accumulated samples in the output image. There is. Points are locations that are not sampled (see Figure 1), and these intervening points are It is interpolated in a manner known per se in the receiver.

In the motion domain, this method cannot be used (the samples are cumulative over a period of time) (to allow for blurring of moving objects), resulting in low altitude The output from the intermediate detail filter produces a relatively good representation of the image. This fi The signal from the router is completely represented by samples taken in one field. I can do it. Figure 3b shows the required characteristics for a certain prefilter. Ru. The dotted lines represent the vertical and horizontal frequencies of a completely sequential scheme. It shows the KIST limit. The hatched triangle indicates the passband of the prefilter. . The hardware determines which of the two transmission methods is most suitable for the block method. , but one block consists of one diamond spanning six pixels. form a shape. Information about which transmission mode was selected for each block is provided in the digital signal to the receiver, so the receiver knows how to perform the playback. . This system is a television system assisted by digital methods. DATV in the meaning of lly-assisted tevevision) and generally Called.

One major advantage of this in-band compression method is that it reduces the bandwidth required to produce recognizable images. A ``simple'' receiver (no frame store, simple spatial interpolator) The only thing that can be seen is above. The only thing that is surprising is that This is a folding operation in the area sent out in "Fine" mode, but this is a signal pre-filled in a way that allows the field to be represented by only one sample field. This is because it was not recorded. Therefore, the zither between the lines horizontally and vertically is A similar effect occurs at a quarter of the field rate.

Furthermore, if an image containing e.g. 1249 lines is encoded in this way For example, send out samples in such a way that the signal looks ``like'' a 625 line signal. It becomes possible to

Although this in-band compression method produces surprisingly good results, it suffers from one major drawback. areas of motion that are not visible or well-correlated (that the human eye can easily follow) The loss of resolution in is particularly obvious. in well-associated motor areas. In order to maintain a high spatial resolution, according to the present invention, the speed of such motion is Moving objects as far as measuring degrees and directions and sampling structures are concerned Moving the subsampling grid so that things appear stationary It is necessary that this is possible.

Two problems are thus solved. That is, the first is the superiority of measuring motion vectors. Second, we need to find new methods to improve the transmission method. This means that the details of how the information will be used must be considered.

The problem of finding a good kinematics measurement method is to develop computer simulations of promising methods. was the subject of research including The results of these studies are described elsewhere. (see reference 5) The present invention addresses the second problem, namely the in-band compression scheme using a motion sampling structure. Relating to adapting motion vector information. Studied algorithms and obtained The results will be described below.

The motion vector measurement method described in Ref. Correlate the phase between large blocks and aim for the peak value in the related plane. This includes extracting the dominant vectors that exist. Then the image Each pixel (i.e., small block) in the block is assigned one of these vectors. , based on this, the pixels (i.e. blocks) in one image are When compared with the corresponding region in The file provides the best match. How has this method achieved the bandwidth pressure mentioned in the main text? The details of how it is applied to the reduction problem are shown below.

Each odd field of the input image is divided into measurement blocks of 64 pixels x 32 lines. (other similar sizes would probably give good results, but the fast Fourier transform Since the correlation operation is performed using a conversion, it is convenient if this size is a power of 2) . The phase correlation (see reference 5) is the correlation between corresponding blocks in consecutive odd fields. It is done between the three best Find the peak value of large. The area of search is the “recognizable” range of speeds and corresponding correlations. However, if the width of the measurement block is only 64 pixels and the width of the measurement block is only 64 pixels, If the determination is done over one image period, ±32 pixels per image period. (corresponding to a maximum speed of about 1 second per image width) ).

Vector measurements and assignments are difficult to perform in one field period for several reasons: This is done over one image period. That is, (a) This method uses samples spatially obtained from the same point in the correlation and matching process. , but this is a stationary object with a great deal of vertical detail. This means that the object does not appear to be moving.

(b) All motion is twice the size, so it is as if the motion is one field. It appears as if it were performed over a period of time, which actually makes the exercise more precise ( (absolutely) and even easier to measure vectors of similar velocities. allows for identification.

(C) Vectors in intermediate fields are interpolated from "image-based" vectors. Therefore, only half of the vector information is sent to the receiver.

(d) The amount of vector measurement processing at the transmitter is halved.

The only penalty for such an attempt is that it accelerates significantly in the image period The object is not accurately tracked. However, this occurs relatively rarely. stomach. FIG. 4 shows how velocity vector interpolation is performed. two flaps If we assume that the sum of the motions over the field is equal to the motion in one image, then Below formula % formula % Furthermore, the vector in field 2 is the field of field 1 on either side. As a result, vn,2 = l/2(V,,”Vn+1.l) The interpolation method is useful when the input signal is e.g. Cannot be used when inputting from a camera. In this case, the field The vector of motion relative to d2 is always zero.

Each small diamond-shaped transmitting block is measured in a correlation process One of the vectors is assigned. ``Menu Vector'' where one vector is selected The list of vectors measured in immediately adjacent measurement blocks is is measured in the measurement block containing the transmit block (the “center” block). Formed from (maximum) three vectors.

Therefore, a maximum of 27 vectors can be tried.

However, in the trial calculations of some very similar vectors, there are few points (large The velocity of an object is probably measured in several adjacent blocks. ), so if the vectors of adjacent blocks are more than a certain amount vectors in these blocks if different from vectors in the center block It only includes. The minimum difference of 0.2 pixels per image period is a passive value. I found out something. In addition, the accuracy of vector assignment in a certain area of the image can be determined by To measure the sum of modulus pixel differences over an area slightly larger than the block It is also possible to improve the image size by increasing the area size (calculations were made for an area that is 1 pixel larger).

As discussed in Ref. 5, the accuracy of the vector assignment process depends on the transmission block. The modal error added over the vector is multiplied by a weighting factor that increases with the vector size. It can be improved by looking at it. This means that two vectors are almost one area. If they are equally fit, a bias will result in the selection of relatively small vectors. It means to hold. Therefore, areas lacking important detail may be of various sizes. The tendency for the same small vector to be assigned throughout rather than a mixture of vectors have Future work will be useful if a block has different vectors from all its neighbors. A vector allocation procedure that performs a reallocation of one vector if it has a Other possible improvements to the process will be considered.

The overall accuracy required in the motion vector measurement process is It can be evaluated by considering the method of reconstruction. four streams of information Subsequent fields are combined to create a single output image, so if this period In two pixel periods, if there is a measurement error of one pixel between them, There will be a significant decrease in the amplitude of frequencies that have one cycle (na Iquist limit).

Therefore, the motion vector is approximately -1/4 of a pixel per field period or must be measured to an accuracy of better than half a pixel per image period.

The phase correlation method described earlier uses a more sophisticated method than quadrant interpolation between points on the correlation surface. An accuracy of approximately 0.2 pixels per image period is possible. Therefore, this method The law should certainly be sufficient for this usage.

The motion vector measurement method in Reference 5 was found to be very effective, but the performance was not as high as above. Other methods can be applied to the in-band compression method of the present invention as long as they reach the content shown in the figure. There is no reason why it cannot be used.

As mentioned earlier, human images are processed using a gentle diagonal spatial filter prior to sampling. and must be passed through a vertical temporal filter. In fact, the result is This filter can be omitted for testing purposes without significant impact. There are very small frequencies in real images with diagonal components. this is.

Processed by the hardware listed above, and also by the computers mentioned in the text. This has also been done in data simulation.

The temporal vertical filter means that the image material moving with the assigned velocity vector remains stationary. I have to think that it is.

This means that the location of the input samples to the filter is that the sum of the movements between the field and the "center" field must be moved. means. This is because the motion vector is known as a sub-vixel product. pixel interpolation is required. But actually this interpolation operation This can be done by slightly changing the values of the filter coefficients. each transmission Since a block has its own motion vector and its own motion history, Must be filtered individually.

In order to transmit moving objects in "high definition" mode, a sampling structure is required. It is necessary to move the structure together with the object. The reason is that today's Europe Easy to understand in the context of standard 625 line interlaced TV formats. I can do it. Interlacing is best for objects that do not move vertically In such a case, it is necessary to sequentially reconstruct a complete 625-line image of the object. It is possible to (If horizontal motion exists, the motion compensation method is It will still be necessary. ) At the same time that the object begins to move in the vertical direction, this pair objects are no longer properly sampled.

At a motion rate of one image line per field period, the same location on the object is Each field is being sampled, and scanning is effectively 312 line sequential. It becomes. However, if the scanning raster is fixed relative to the object, then It will always be possible to reconstruct images with high definition. Medium-band pressure discussed in the text The situation in the compression method is that the sampling structure is also effectively interlaced horizontally. are very similar, except that

Each small diamond-shaped transmission block has a motion vector relative to this block. must be considered to have its own sample structure moving according to the

This is immediately fixed per block to maintain a constant data rate. The size of the transmission block and This imposes certain constraints on the shape and shape. As shown in Figure 5, the diamond-shaped block The 6-pixel width of block A is: 1. Can hold 2 or 4 samples. “Fixed number of samples” criterion The only reasonable block size is a 4 or 8 pixel wide rectangle, i.e. It is diamond-shaped. The 8-pixel width of Diamond B was selected for this study. This is because this size roughly matches the pore size of the spatial filter. . The advantages of using diamond shapes instead of square blocks are many, perhaps from many blocks, since most objects have horizontal or vertical edges. There is less of a problem with the edges of objects that appear at the same time and make the block structure even clearer. That's true.

If even smaller blocks are used (in terms of image quality) perhaps While this would be an advantage, this disadvantageously reduces the bit rate of the digital part of the transmission link. This will likely increase the number of people. In fact, even larger blocks can be used to increase the bit rate. Further restrictions may be needed.

The first set of samples picked up by the transmission block is field 1 in Figure 1. It is placed in the location as shown in. The following set of samples shows the field This block is removed at the position shown relative to the sample location in block 2. is shifted by the motion vector assigned to the track. The sample is the input sample. A simple linear interpolation procedure was performed when required between pull locations (Figure 6).

This is the frequency (especially vertically if the source is interlaced) The use of more complex interpolators may be helpful, although it will limit the response somewhat. essential Here are the improvements made by improving the interpolator based on the cubic spline fitting method. was extreme, but the required processing time increased significantly.

Early research made it possible to continue this process “indefinitely,” resulting in The sample structure can grow infinitely. This attempt has several drawbacks as described below.

That is, (a) For a moving object, the sample structure is one frame from its “natural resting place.” It may come to rest so that the image line is shifted. This means that the sample is The vertical resolution is degraded as a result of continuous interpolation between two field lines. means. This problem is much smaller than with sequential sources.

(b) The sample structures of two adjacent blocks can move in opposite directions, resulting in A permanent gap will be left within the sample structure as a whole.

(e) Transmission errors in digital motion vector information prevent the receiver from properly sampling This may cause you to lose track of the location of the pull structure, but this is because each sample's location is This is because it relies on the complete motion history of the lock.

To avoid these problems, the location of the sample structure should be changed every four field periods. There is a way to reset it to its "Field 1" position at the starting point of Ta. For this reason, the position of a structure is never more than three motion vectors from where it started. I'll never leave you.

However, this means that each set of four fields is processed in the interpolation operation. The reconstruction process must also look “forward” as well as “backward” in time. means that it must be done. This will be discussed in more detail below. .

Of course, when the sample structure moves more than 4 pixels horizontally or vertically, When another sample “falls” from the edge, the “new” sample location is placed inside the block. It is necessary to move it to. As mentioned before, the size of the block depends on the other When a sample location leaves, a new sample location always enters and a fixed number of sample locations are added per block. selected to maintain the sample.

One complete sequential image is then reconstructed from the four fields worth of subsamples. Consider the processes necessary to achieve this.

Each transmission block is individually reconstructed. The reconfiguration process is the end of this process. In the sample, half of the frame store locations form a five-mouth pattern. 4 files at appropriate locations in one "frame store" to be filled with files. It can be thought of as taking a sample from each of the sub-sample phases.

The other half of the location is now two-dimensional, as in the case of the static image in FIG. Interpolated using a spatial interpolator.

Each output image is formed from 4 fields of samples containing the current "phase" (See Figure 7). Number 1.2.3.41.11 is the subsample sequence represents a field. The first field 1 is always sampled at a fixed location. will be sent. The field unit motion vectors v, 2, V23 and VS2 are indicated by arrows. shown. Vector ■4X is not used. At this time, for example, set field 3 to The shifts applied for the reconstruction are shown as in the figure. For example, field 2 -V2. Shifted a lot. In this way, instruct at the same time as the output image at this time. Therefore, a set of samples does not need to be moved by any motion vector. do not have.

The other three sets have an operational history between the time the sample is taken and the time corresponding to the output image. It is necessary to be able to move by an amount equal to the sum of the motion vectors. sampled The location of the sample after the reconstruction process has been shifted by the same sum of motion vectors. The final configuration of the grid is guaranteed to be five-sided, similar to the grid shown in Figure 1. Ru. The reconstruction process consists of sample locations in fields 1 to 4 in the absence of motion. The one-dimensional case is shown in FIG. 8 shown at the top. exercise pair The objects are represented by bars R in fields 1-4, and the shifted sampling points is written as 1.2.3.4 in each bar. The vector shift in Figure 7 is 4 The reconstructed image RR for field 3 containing all sampling points is Shown added to form.

The exact location where the sample ends is determined by the position of these locations in the corresponding input field. depends on the position of the subsample relative to the It doesn't seem to match. Therefore, the last configured frame store in the receiver Each block in the image is shifted out of position by half the gap between the quincunx locations. It is possible that This determines where the sample “requires” movement and how these frame struts to compensate for the distance between the sample and the available fixed location. This means that sub-pixel interpolation is required when reading from a. need For sub-vixel shifts, the length of the shift is the position of the sample structure at this point. Because it depends on the location, it is different for each block.

If each block at each phase is allowed to contribute to the output image, then Various problems can arise. Visible in one phase of the sample structure but later The image area that becomes blurred in subsequent phases that reappears when the image is reconstructed is Undesirable. Therefore, you can find out which areas of the image have become "unsharp" and Samples from these areas should not be used in the reconstruction process. be. This problem involves "tracing" vectors for each region of the output image, and also using a reconstruction process. The solution is to use image information found "along the way" of the process.

This means that each region of the output image has more than four samples contributing to this This means that the image information that becomes blurry will not be used. Figure 9 shows the potential problem and how this solution works. call this special processing An example of the type of image information is a foreground moving somewhat quickly over the background. This is the edge of the object in the code. Without this “tracking” method, the object The front edge cannot be properly reconstructed due to the "high definition" mode, and therefore the "low We will return to the "accuracy" transmission method. In order to realize this algorithm, The problem of extra convergence required is not warranted by the modest improvement in quality obtained. It will probably not be there.

In FIG. 9, vertical dashed lines represent block boundaries. Block SO is related to a stationary object with respect to a moving background MV; The motion vector of is a diagonal arrow. Simple reconstruction in field 4 reconstruction Using the construction method, the block denoted by a ends up at location a, and block b finally becomes location b°. This means that a blurred background is a static target. It means to break into something. However, if each arrow in the last phase is in the opposite direction , and if only blocks found "in the middle" are used, blocks a and and b are not used for generating the output image. The same idea applies to "future" and This is valid when reconstructing using "past" fields.

In this example, for simplicity, the motion vector is exactly one block per field period. It was selected to be a tsuku width.

If the motion vectors of each block point in directions that move away from each other, then Problems can also arise at the junction of the Sample structures are often not filled The sample structure is moved to leave less space in the frame store. If If the frame store is filled with zero samples prior to the reconstruction process, this This state results in almost no gray spot appearing in the output image. This problem, First frame scan, using a spatial interpolator to fill in the missing 75% of the samples. avoided by filling the tor with images obtained only from the current input sample phase. be able to. In this way, the place placed in the five-mouth shape in the frame store If the image is not filled in by the reconstruction process, the image visible at these locations The data will be a kind of "low precision" version of the field at that time. this thing In this case, the proposed method can solve the problem of buffers that appear without switching the entire transmission block to low-precision mode. It is possible to revert to a lower precision mode in pixels in one region of the background. It definitely means that. "Low Precision" used for frame store filling field is not a true low-resolution version of the input image, but this is a suitable empty obtained from a set of samples of images that were not previously filtered using an interval filter. This is for the purpose of This indicates that there is a certain folding state in the fine region. This is because this information is only needed for filling a few pixels. It's not that important.

Next, we will discuss the same “return” low-rust thin transmission mode used in Reference 3. Describe it as follows. If the motion measurement and motion subsampling algorithms are If working well, this mode will eliminate turbulent motion or uncovered background. Required only in the region of the image that contains it.

The input image is prefiltered and subsampled as shown in FIG. sub The sample grid follows the 4-field sequence in Figure 1, and This means that it does not move to follow the object, similar to the sampling grid of It is "fixed" in The output image is reconstructed using a special interpolator, Fields comparable to those obtained by "High Definition" mode as described below. occurs.

This is done to select which transmission mode is the most appropriate for each block. The method used in the study is similar to that used in the previously mentioned hardware. The method is the same as the method (see Reference 3). For completeness, this method is described here I will briefly explain this.

For each transmission block, both the original signal and the “low” and “high” spatial definition modes are transmitted. The absolute value of the difference between the signal after the signal is calculated. The sum of errors across the block area is , are compared after being multiplied by a weighting factor and block using the method that yields the smallest error. to transmit. The weighting factor used for these considerations is the error in low-detail mode. It was 0.55 for the error in the high definition mode, and 0.45 for the error in the high definition mode.

This has been found to give the best native results and is therefore Mode selection is slightly biased.

The area of the transmission block containing the area of the image where the errors are summed also becomes slightly larger. A number of experiments were attempted. This is modest in certain areas of the image. However, it remains to be seen whether this method provides a more reliable mode selection method. Further investigation is required to find out whether this is the case.

Once a mode is selected for a transmission block, the A subsample is transmitted. However, if low definition mode is selected, the transmission The frame store samples in the "fine" part of the system are unchanged.

This means that if a mode change is made in the 4 fields of this set. In this mode, the output of the high-definition part of the system on the transmitter side is transmitted to the receiver in this mode. This means that it is not exactly the same as the signal that is decoded. do not change the sample The reason is that high-definition mode is "consistent" at this time, and one field is simply This is not the case simply because the signal is not displayed well. child This protects the system from becoming "stuck" in low detail mode. There must be. Also, because the reconstruction method looks both forward and backward in time, Therefore, the sample cannot be updated in an acceptable way at the transmitter, but this Indicates which fields were transmitted in which direction before the configuration process started This is because it becomes necessary.

The simulation work performed on the data was based on the basic adaptive subsample System performance can be greatly enhanced by adding motion vector information. We showed what we can do. For example, for the image sequence obtained in Figure 3 of Reference 3, As a result of our study, we found that a car can operate in high spatial definition mode with minimal degradation. transmission, but without motion vector information it is impossible to return to low-detail mode. and showed. However, the motion gate in this sequence is conveniently However, apart from this gate, there is almost no spatial detail. Because it moves fast enough (approximately 2.4 pixels/field period) that there is no It is.

There is still some small degradation in the output image.

For example, the edges of the car look slightly different, which is probably a corresponding This is probably because the transmission blower includes two types of motion. In early research, “detail” Do not use a temporal vertical prefilter in the path and include such a filter (thermal The movement of the image information is not properly tracked by the motion measurement system. It is possible to reduce the width. For this reason, in a well-specified manner, one movement It may also reduce the amplitude of vertical lines that are about to appear in areas that do not exist. like this The appearance of subalternae (sp old, 1ae) also increases the size of various filter pores. and by allowing these to have sharper cutoffs. You can also

the pore size used in the vector assignment and mode assignment steps; ``high speed J weighting factor, ``mode selection'' weighting factor, and color of vector measuring stage There is scope for optimizing parameters such as various details. Measurement and transmission block the dimensions of the vector, the number of “menu” vectors required, and the It is also necessary to consider the resolution. Many of these parameters are It will depend on the available bandwidth in the digital part of the network.

For this reason, motion vector measurements are based on motion adaptive subsampling in the video band. We have described a method that is applicable to medium compression methods. Figure 10 shows the entire transmission system. A block diagram is shown. A computer simulation of this method shows that the original (Non-vector method) yielded promising results showing a significant improvement in the performance of the system. .

The method is already sufficient to form the basis of high-definition television medium-band compression systems. It appears that the performance is good, and further research should further improve its performance. It is something.

Since this system is composed of well-known components such as filters and interpolators, A detailed description of the hardware is not necessary. The motion vector generator 10 It is configured as shown in Figure 5. The motion vector MV is sub-sampling device 1 This sampling device is used to manage the position of sampling points in 2. There is a pre-filter 13 after the position, regardless of whether there is movement (vector zero). It provides a high definition mode and also provides high definition sub-sample image data SSH.

The sampling points are moved by changing the sampling time points as seeds.

The high-definition reconstruction operation 14 is realized as described above to connect SSH to high-definition reconstruction video. Convert to signal RVH.

Pre-filter 16 and fixed grid sub-sampling device 18 provide low-definition sub-sampling. (spatially filtered) image data SSL that you want to be filtered. spatial interpolator The device 20 performs low-definition reconstruction and then reconnects the SSL to the low-definition reconstructed video signal. Convert to No. RVL.

Mode selection device 22 selects RVH and RVL from IN video (with appropriate delay). An analog signal with a reduced bandwidth to be transmitted (recorded) compared to an analog signal. and controls a switch 24 that selects between SSH and SSL.

The transmitted data includes, in addition to the analog signal, the motion vector MV and the block S A digital signal containing both a mode signal indicating whether it is sent as SH or SSL. Consists of digital signals.

On the receiver (recycle bag@) side, the decoder includes blocks 14 and 20 configured as a high-definition reconstruction device 14'' and a low-definition reconstruction device 20'. It has become. The high-definition device 14' uses the received motion vectors as described above to The switch 26 is controlled by the mode signal received and sent to the display. A selection is made between RVH' and RVL' as the video output signal to be output.

The mid-band compression system described in this paper uses coefficients due to interlaced input signals. 4 compression. The system is provided by sequential sources and displays. can support the full resolution of Performs area compression.

The transmitted signal is packaged so that it looks like a signal with a relatively small number of lines. can be easily edited to produce the desired image with a minimum amount of processing. Can be displayed simply on the receiver.

In the case of poor implementation described in the main text, the behavior of the sample structure is It is "reset" every 4 fields relative to its original starting position, and any movement The opening movement of the subsequent three fields is made to follow. For this reason, if the object Sample the same point on a moving object to be sampled even if it is at rest. It has the effect of

If all samples are taken during the first image in the four field sequence (assuming perfect motion measurements and allowing for differences in the prefilter), or If the other three fields are not sampled at all, you will get virtually the same samples. be able to. Information placed in other fields is also stored in the first interlaced frame. The process is used to generate sequential images from a field, and the process is based on the motion vector Use the information.

The advantage of this approach is that any error in the motion vector measurement process no longer This is not per se obvious by the appearance of subsample structures in the image. .

Instead, the error appears as a slight discontinuity at the edge of the block, which is likely Raku is probably not much of a problem in terms of wood quality.

Using this approach, the delivery system appears to be a 12.5Hz sequential system. For compatible transmission purposes, shirtful samples, and digital assistants Frequency at the receiver assisted by motion vector information transmitted in the channel. This involves rounding up the system speed. The low-displacement “folding” mode still exists. Ru.

A transparent interconversion method between source and display standards using motion compensation (Reference 1) is , suggests a method of bandwidth compression. This method is suitable for HDTV transmission. There is. However, this requires the use of digital supplementary channels. this The system was found to be very similar to the methods suggested in Refs. 3.4 and 5. Ru. By looking at this method from a different perspective, NHK's MUSE system (text) The considerable difference with reference 6) becomes even clearer. Furthermore, this point of view The proposed method suggests possible improvements.

The basis of this method is subsampling with temporal motion compensation. this reduces signal redundancy due to temporal frequency caused by moving objects exclusively do. The quality of the image is determined by the resolution of the data in a compressed band in the form of motion vectors. This information is saved by transmitting it as a digital assistant channel.

The following description concentrates on the "fine" image channel. This channel is stationary It possesses information from well-correlated motor regions of the body. In Reference 3 and Figure 10 The low-definition channel will also be available to allow for better resolution. This is poorly correlated. may be used when the fine image channel is not successful due to the image content.

This method can be modified to be used with thermal, HDTV sources. .

The first part of the channel (shown in FIG. 1) is a motion compensated standard converter 30. be. This is 625150/2:1 (video) or 625/25/2:1 (film) will produce an output of 625/121/2/1:1. Yo If the input is from an interlaced video, which is often the case, the converter must convert sequentially from suggestive interlace. There will be no such thing.

This converter is followed by a spatial (diagonal) prefilter 32, which is a sub-sampler. Enables the use of 2:1 spatial subsampling in the pull device 34. suitable As a result of filter operations, image quality is reduced as described in references 3.4 and 5 above. Resulting in a slight decrease in quality.

The actual configuration of this system consists of combining these first two blocks. Become. This combined filter/standard converter is more hardware than a prefilter. No hardware is required.

The spatio-temporal filter used is shown in FIGS. 12 and 13. No. Figure 12 shows the pre-filter used in the fine channel. illustrated The above filter is for stationary objects, and this filter is for moving objects. Appropriate modifications will be made to the artifacts. In low definition channel The fixed prefilter used is shown in FIG.

Figure 14a shows a sample used in a high-resolution channel and repeated at 12.5 Hz. A pulling lattice is shown. FIG. 14b shows a sample used at 50Hz in the low definition channel. A four-field sequence of sampling points is shown.

The temporally subsampled output from the standard converter is then (313150 /2:1) Shuffled in the shuffler 36 to resemble a pseudo five-head signal Of course, in HDTV systems, subsampled signals are compatible. can be shuffled to resemble the obtained 625150/2:1 signal (Reference 4 ). This shuffling operation uses motion vectors to ensure that the motion vectors are in the appropriate position at the right time. This is done by moving the object so that it appears in the same position. sub-vixel compensation No intermittent operations will be performed. The movement vector used to shift the object is vertically and horizontally.

The pseudo-five-head signal is then transmitted to the digital assistant channel with associated movement It can be sent along with the information.

The received signal is unshuffled to yield a 625/121/21:1 signal. 40 and interpolated in an interpolator 42. This signal is Up-conversion is performed using a motion compensated standards converter 44 to the desired display standard.

Again in the actual system, the interpolation filter 42 and the standard converter 44 are They will be combined.

The final display standard has little to do with the actual transmission of the signal. this thing The clear display standard is the same as the source, but other display standards are equally valid. .

For example, an image can be displayed at 525/10 po/2:1, and the up-translation All the necessary information is present in the digital auxiliary data. The image is conceptually It is also possible to display based on the 60Hz standard. A film source movement Knowing that contours are automatically improved through the use of this mid-band compression system is interesting.

The system achieves bandwidth compression by a factor of 4:1. This for bandwidth compression The trial is relatively immune to small errors in the measurement of motion vectors. pseudo five The transmission method via a mouth-shaped signal is compatible with the transmission of low-definition signals. the latter Transmission mode is used in radii of motion where correlation operations are poor or random. It will probably happen.

The hardware requirements for a simple version of this system are shown in Figure 5 and above. and the requirements for the system of FIG. Actually a lot of hardware are the same.

You can also build a simple version of the sender's standard converter first. child This uses the motion vector to convert field 2 (4 field sequence ) would simply interpolate the "missing" line in field 1. centre Fields 3 and 4 would be discarded. But these fields are , will be used for the measurement of movement. More hardware in the future More complex converters could be constructed using fields 3 and 4. . This should improve the performance of the system.

Simple versions of output standard converters can also be built. This is an exercise information to move a moving object to its proper position in the output field. Ru. A sub-pixel interpolation operation will yield the best output image.

In the future, more complex standard converters may be used to improve performance. cormorant.

The use of a good standard converter provides two main improvements. First, this The improvement should reduce noise in the output image. This is (motion compensated ) This is because the temporal band rjJ is even narrower. Second, the This should reduce time lapses. This is suitable (motion compensated) This is achieved through pre- and post-filtering.

The interlace-to-sequential conversion method is an interpolation process. in one field Interpolated intermediate lines in time and space are interpolated from adjacent lines in time and space. must be done. A straightforward attempt is to select one from adjacent fields (front and back). interpolating missing lines in the field of to allow for object motion. It is. This is shown in Figure 15, where the vector represents the speed of the object. Unfortunately, this method produces spatial folding components.

These components are shifted after the adjacent (sampled) fields are interpolated It occurs because it is done. Ideally, adjacent fields are shifted before being sampled. However, this is not possible. Motion-compensated transformations are fixed in time. This could be an improvement on the vertical filter operation, but this is due to the spatial/temporal This is because spatial folding occurs instead.

The problem inherent in interlaced to sequential conversion is that it is difficult to consider it in the frequency domain. According to this, it is understandable.

Ideally, the interlaced image is padded with zeros and with a suitable low frequency gap. Let's be filtered. Figures 16, 17, 18 and 19 are samples. stationary object, object moving at 25s/ph, object moving at 12s/ph and the spectrum of an object moving at 6.25s/ph (625/25/1:l) It shows. The dotted arrows in Figure 16 move at various speeds (25s/ph target). It corresponds to the movement of objects. The dotted line area in Figure 16 indicates the The pores of the filter are shown. Stiff areas are shown as “squares”; This does not affect the issue.

Figure 17 shows that only half of the bandwidth can be obtained even at low speeds at appropriately low interpolation holes. It shows that it cannot be occupied. This means that when the vertical velocity of the object increases, , a motion-compensated interpolator can handle changes in the shape of an object that moves more or less. It means being able to respond.

By the time the object speed reaches the slow speed of 12 s/ph (Figure 18), the motion compensation The interpolator created by the interpolator causes unavoidable spatial folding.

These arguments apply to sequential variations other than motion images, which are very slow even with motion-compensated interpolators. This suggests that it is not possible to perform transparent interlacing for conversion.

Motion compensated image processing than interlaced sequential sources of equal bandwidth It can be considered that it is more suitable for If the interlaced saw If a source is used, the results are likely to be significantly worse than a sequential source.

The discussion in the text considered the ``ideal'' source.

It contains as much signal content as possible.

Such sources are a serious target in HDTV. However, today's source provides as little signal content as possible. This means that interlaced vs. The failure caused by the sequential transformation is greater in the current source than in the ideal source above. This means that it is not difficult for some people.

Jo 1F (no content) Ftcy, 2 Engraving (no changes to the content) Horizontal Pj 4-value number Sakirl! '1 □ Input p fee I redo bu 21212 no 2 1 2)-1-v-no -1,,-'shi)-ノ-+,ia,s% R7z-! n n+1 n-1 2n + 3r total r (2mm; 232 women without g) Ftcy, 6 Jyo7I5 (inner strawberry branch, noodles) FIo, 9 (No changes to 1f1 contents) Jyo 3 Koyo, 22 Shishi) Pure IF (no change in content) ×　×　×× Engraving (no changes to the content) Pool←Ple V Procedural amendment (jj) October 21, 1986 1.Display of the incident PCT/GB86100795 2. Name of the invention Video signal processing device 3 for band medium voltage line, person performing correction Relationship to the incident: Applicant address Name: British Broadcasting Corporation 4. Agent Address: Shin-Otemachi Building, 206-ku, 2-2-1 Otemachi, Chiyoda-ku, Tokyo 5. Date of amendment order: September 27, 1988 (shipment date) 6. Subject of amendment international search report ◇ to XTo IKE INTERNAT engineering 0NAL 5IEA, RCHREP OrtT ON

Claims (25)

[Claims] 1. A digital signal containing motion vector information about a block of pixels In a receiving or reproducing device for a compressed in-band video signal, each means for accumulating subsample points from a field of field iteration cycles; Shift the sub-sample points in response to the motion vector information so that the resulting accumulated High definition in both the static domain and the motion domain where the points are correlated field by field. A communication or reproduction device characterized in that it is provided with means for producing an image. 2. Claims characterized in that means are provided for interpolating intermediate points between accumulated points. The receiving or reproducing device according to item 1. 3. The cumulative points for each field are calculated both before and after the same field cycle. According to claim 1 or 2, the method is obtained from a field. receiving or reproducing equipment. 4. The cumulative points for each field are derived only from that field and the previous field. The reception or reproduction according to claim 1 or 2, characterized in that Device. 5. For spatial interpolation operations of said subsample points received in only one field. means for reconstructing an image of a point in response to a mode signal accompanying said video signal; Is it between an image reconstructed by accumulation and an image reconstructed by spatial interpolation? Claims 1 to 3 further include means for selecting an output from the 4. The receiving or reproducing device according to any one of Item 4. 6. The selection means selects a junction between two blocks sent in different modes. The reception according to claim 5, characterized in that the switching is performed rapidly at a point. Or playback equipment. 7. The selection means selects a junction between two blocks sent in different modes. The receiver according to claim 5, characterized in that the switching is performed gradually at a point. transmission or reproduction equipment. 8. A frame store where high-definition images are reconstructed, and a frame store where point accumulation reconstruction is performed. The frame store can be simplified from subsample points of one field before means for filling the image with an image generated by a simple interpolation operation. A receiving or reproducing device according to any one of claims 1 to 7. 9. The motion vectors are back tracked from the field being reconstructed and the sub-sample The field before or after the sample point and the addition from the field being reconstructed. When related to a motion vector that was not reached by the trace, the subsample point is included in the accumulation. Claims 1 to 8, characterized in that means are provided to prevent A receiving or reproducing device according to any of the above. 10. In a device that produces a compressed in-band video signal from an input video signal , A motion vector that describes the motion of each of multiple pixel blocks that make up one image. and means for sub-sampling said input video signal to produce a signal of the same cycle. of the corresponding subset in proper spatial relationship to other field points in the field. The image points are divided into one repeating field cycle, with the same hand occurring in each field. and conveying the video signal and motion vector produced by the sub-sampling means. and means for generating a composite signal consisting of digital signals. . 11. The sub-sampling means samples the one sample from the first field of one cycle. Claim 10, characterized in that taking a subset of all image points during a cycle. Apparatus described in section. 12. The sub-sampling means follow a motion vector describing the movement of each block. each subset from the corresponding field using a sampling grid displaced by 11. The apparatus according to claim 10, wherein the apparatus takes two image points. 13. The sampling grid collects data for the first field of each cycle. 13. Device according to claim 12, characterized in that it is restored to position. 14. By field-by-field subsample operation that realizes spatial filter operation, means for producing a signal in another compressed band; and a block between the two compressed band signals; means for selecting each signal and using the signal that best matches the input video signal. The apparatus according to any one of claims 10 to 13, characterized in that: 15. The selection means selects the features of each cycle that are shifted according to the motion vector. By accumulating subsample points from the first compressed band medium signal, means for reconstructing the fine signal and said another compressed in-band signal by a spatial interpolation operation; means for reconstructing low definition signals from said input video signal; and means for reconstructing each reconstructed signal from said input video signal. and means for determining a relatively good match compared to the 15. The apparatus according to claim 14. 16. said comparison means performs a weighted comparison operation resulting in a high definition signal; 16. The device of claim 15, characterized in that: 17. a spatial prefix in front of said field-by-field subsampling means; Claim 14, Claim 15, or Claim 15, characterized in that a router is provided. 17. Apparatus according to any one of paragraphs 16 to 16. 18. The composite signal includes a mode signal indicating which compressed band medium signal is selected. The device according to any one of claims 14 to 17, characterized in that it comprises: . 19. In front of the sub-sampling means according to claim 10, The input samples are filtered as if each motion block could be stationary. Vertical spatial filter or spatial filter displaced by a motion block such that as claimed in any one of claims 10 to 18, characterized in that the equipment. 20. The sample structure assumes a quincunx shape and reacts over four field cycles. The device according to any one of claims 10 to 19, characterized in that Place. 21. the input video signal is an interlaced signal, and the motion vector The motion vectors are interpolated to produce field-by-field measurements. Any one of claims 10 to 20, characterized in that it generates a motion vector. The device described in Crab. 22. Said means for generating a motion vector determines the position of a block question of two successive images. Realize the phase correlation and extract the dominant vector as a peak in one corresponding surface. and each pixel or pixel block results in image-wise alignment. Claims 10 to 21, characterized in that a motion vector of Apparatus according to any of paragraphs. 23. The phase correlation is applied to a large block and the vector assignment is 23. Device according to claim 22, characterized in that it is applied to small blocks. 24. Standard converter with preliminary motion compensation responsive to said motion vector The method according to any one of claims 10 to 23, characterized in that Device. 25. A standard converter is provided with subsequent motion compensation in response to the motion vector. An apparatus according to any one of claims 1 to 9, characterized in that: