JPH1028270A - Image transmitting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image transmitting device in which the encoding amounts of picture data to be transmitted can be decreased even when the shake of the image data is generated at the time of compressing plural image data by inter-frame correlating, and successively transmitting them. SOLUTION: Mutually corresponding constituting blocks are detected by a detecting means 13 from objective image data to be compressed held in a first storing means 11 and reference image data being an image in a frame previous to the objective image data held in a second storing means 12, and a position deviation value between the both constituting blocks is detected by a deviation value detecting means 14. Then, the objective image data held in the first storing means 11 are corrected based on the detected position deviation value by a correcting means 15, so that the position deviation value for the reference image data can be decreased. Then, the objective image data corrected by the correcting means are compressed based on the reference image data by an encoding means, and the compressed objective image data are transmission-outputted by a transmitting means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image transmitting apparatus for sequentially transmitting a plurality of image data compressed using an inter-frame correlation. The present invention relates to an image transmission apparatus capable of reducing the amount of code of image data and efficiently performing image transmission.

[0002]

2. Description of the Related Art For example, as image transmission systems for transmitting image data, there are still image transmission systems and moving image transmission systems, and intermittent moving images (so-called quasi-moving images).
Is known. For example,
Generally, in a moving image transmission system, image data is compressed by using a compression method using inter-frame correlation, and then image transmission is performed on the compressed image data. Will be described with reference to the drawings.

FIG. 6 shows the overall configuration of a moving picture transmission system. First, in the transmitting device, analog image data is input to the image input device 40 via image input means such as a video camera, and various corrections such as A / D conversion and shading are performed on the input analog image data. After processing such as color component system conversion is performed, the digital image data obtained by the processing is encoded by the encoding device 4.
1 is output.

[0004] Next, in the encoding device 41, the digital image data input from the image input device 40 is converted into H.264 data. 261,
The image data is encoded by an image compression method using inter-frame correlation such as MPEG1 and MPEG2, and the encoded compressed image data is output to the transmission device 42. The details of image compression are described in "Latest MPEG Textbook, edited by Hiroshi Fujiwara ASCII".

[0005] Next, the transmission device 42 performs speed conversion on the compressed image data input from the encoding device 41,
After processing such as code conversion is performed, the compressed image data converted by the processing is transmitted and output to the transmission path 43 whose transmission is controlled. Here, in general, the compressed image data transmitted and output from the transmitting device 42 is transmitted to the receiving side device of another image transmission system via the transmission line 43.

On the other hand, in the receiving device, the compressed image data transmitted and output from the transmitting device of another image transmission system to the transmission path 44 is received by the receiving device 45, and the received compressed image data is transmitted to the transmitting device. After performing speed conversion, code conversion, and the like corresponding to the device, the compressed image data converted by those processes is output to the decoding device 46.

Next, in the decoding device 46, the receiving device 45
Is compressed by a decompression method corresponding to the transmission-side device, and the decoded digital image data is output to the image output device 47. Next, after various processes such as D / A conversion and color component system conversion are performed on the digital image data input from the decoding device 46 in the image output device 47, the analog image obtained by those processes is processed. The data is displayed and output by image output means such as a CRT or a liquid crystal display.

The control device 48 includes a CPU for performing various operations such as calculations, a ROM for storing a predetermined program, and an R for use as an operation process and an image data storage area.
The image input device 40, the encoding device 41, the transmitting device 42, the receiving device 45, the decoding device 4
6 and various controls on the image output device 47.

[0009]

However, in the image transmission apparatus in the above-described moving image transmission system, when a video camera or the like as an image input means captures an image of a subject, the camera or the like shakes or the subject shakes. For example, the captured image may be blurred. FIG. 7 shows an example of image data in a case where continuously captured images are blurred. In the case where image blurring occurs as in images 50 to 53 in the figure, even if image data is compressed using inter-frame correlation, the degree of correlation is low. Has a problem that the code amount of the compressed image data is large and the amount of data to be transmitted is large as compared with the case where is always at the same position in the frame.

That is, in the compression processing of image data using inter-frame correlation, since image data to be compressed is compressed based on the image data of the previous frame, when image data is blurred, Since the difference between the two image data becomes large, the code amount of the compressed image data becomes large.

SUMMARY OF THE INVENTION The present invention has been made to solve such a conventional problem. In an image transmission system for sequentially transmitting a plurality of image data compressed by using an inter-frame correlation, the image data is blurred. It is an object of the present invention to provide an image transmission device that can reduce the amount of code of image data to be transmitted even when it occurs.

[0012]

In order to achieve the above object, in the image transmission apparatus according to the present invention, when performing compression processing using inter-frame correlation, the first storage means stores the target image data to be subjected to compression processing. And the detecting means stores the second
Using the image data of the frame preceding the target image data stored and held as the reference image data in the storage means, the target image data held in the first storage means and the second
And the corresponding block is detected from the reference image data held in the storage means, and the shift amount detecting means detects the positional shift amount between the two constituent blocks.

Then, the correcting means corrects the target image data held in the first storage means based on the detected positional shift amount to reduce the positional shift amount with respect to the reference image data, and the encoding means sets the correcting means. The target image data corrected by the above is subjected to compression processing based on the reference image data, and the transmission unit transmits and outputs the compressed target image data.

That is, in the image transmission apparatus of the present invention, after the detecting means detects the corresponding constituent blocks from the target image data and the reference image data, the displacement detecting means detects the positional displacement between the two constituent blocks. Then, the correcting means corrects the target image data based on the detected positional shift amount to reduce the positional shift amount with respect to the reference image data. The degree of correlation between data can be restored to its original level, and the amount of code after compression can be reduced.

Here, the image data to be transmitted includes:
Image data input via various cameras, interfaces, etc., image data stored in a RAM, or the like is used. However, as long as image data is compressed using inter-frame correlation, it is not limited to moving images. The present invention can be applied to transmission of quasi-moving images and still images.

The size and number of corresponding constituent blocks detected from the target image data and the reference image data, and where the two constituent blocks are located in the image data are not particularly limited. Any configuration block may be detected as long as the configuration block can detect the amount of displacement of the target image data with respect to the reference image data.

Further, the positional deviation amount of the target image data with respect to the reference image data may be detected by detecting a pair of constituent blocks and comparing the pixel data in both constituent blocks. It may be detected by detecting and averaging the positional shift amounts of the constituent blocks, or may be detected by detecting a plurality of pixel points and comparing the positional relationship formed by the pixel points. In short, it is only necessary that the amount of displacement be detected based on the element parts constituting the image, that is, the constituent blocks.

The method for compressing the target image data is not particularly limited, and any method may be used as long as the method performs a compression process using inter-frame correlation. Further, the application field of the image transmission apparatus according to the present invention is not limited to the field of wired transmission, but is also applied to the field of wireless transmission.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An example of an image transmission system to which an image transmission device according to the present invention is applied will be described with reference to the drawings. FIG. 1 shows the overall configuration of an image transmission system. This image transmission system includes an image input device 1 and a shift correction device 2 as transmission-side devices that transmit and output image data via a transmission path 5. , Encoding device 3 and transmitting device 4
And a receiving device that receives image data via the transmission path 6 includes a receiving device 7, a decoding device 8, and an image output device 9. In addition, the image transmission system includes a control device 10 that performs overall control of the above devices.

The image input device 1 includes image input means such as a video camera. When analog image data is input, various corrections such as A / D conversion and shading are performed on the input analog image data. , And performs processing such as system conversion of color components, and outputs the digital image data obtained by those processings to the shift correction device 2. When digital image data is input to the image input device 1, the image input device 1 does not need to have a function of performing A / D conversion processing. After performing processing such as system conversion of components, it is output to the deviation correction device 2.

Here, the image input device 1 may be a device for directly inputting image data through various interfaces, in addition to the means for capturing and inputting an image.
Further, the size of the image input by the image input device 1 is not particularly limited, but it is preferable that the size is such that the subject can be reliably contained in the image even if the image is blurred. Now, an example in which such sufficiently large image data is input from the image input device 1 will be described.

For example, as shown in FIG. 2, a first memory means 11 for storing target image data to be subjected to a compression process, and image data of a frame preceding the target image data are stored in a reference image data. A second storage means 12 for storing the target image data stored in the first storage means 11 and a detecting means for detecting corresponding constituent blocks from the reference image data stored in the second storage means 12 13, a shift amount detecting means 14 for detecting the detected positional shift amount between the two constituent blocks, and correcting the target image data held in the first storage means 11 based on the detected positional shift amount. Correction means 15 for reducing the amount of displacement with respect to the reference image data.

First storage means 11 and second storage means 12
Is a frame memory that can store and hold at least one frame of image data input from the image input device 1. The first storage unit 11 uses the image data input from the image input device 1 as target image data. The second storage means 12 stores the image data input from the first storage means 11 as reference image data.

This reference image data is image data of a frame which is input one time earlier than the target image data, and the first storage means 11 and the second storage means 12 are constituted by separate memories. In the case, the first storage unit 11
Is transferred to the second storage means 12 as reference image data at the next processing timing. Note that the first storage unit 11 and the second storage unit 12 may be configured by one memory, and in such a case,
The target image data may be handled as reference image data in accordance with the progress of the processing timing.
The distinction from the storage means 12 is logical.

The detection means 13 detects constituent blocks corresponding to each other from the target image data held in the first storage means 11 and the reference image data held in the second storage means 12. The displacement detecting means 14 detects the displacement between the two constituent blocks detected by the detecting means 13. The correction unit 15 corrects the target image data based on the position shift amount detected by the shift amount detection unit 14 to reduce the position shift amount with respect to the reference image data.

With the above-described means, the shift correction device 2 corrects the image data input from the image input device 1 and outputs the corrected target image data to the encoding device 3. In this example, as will be described later with reference to FIG. 5, the target image data 30 shown in FIG.
An example in which an image obtained by cutting out a part of the image data and reducing the number of pixels is output to the encoding device 3 as the corrected target image data 32 is shown.

The encoding device 3 is means for compressing the target image data input from the shift correcting device 2 by inter-frame correlation based on the reference image data. 26
1, target image data encoded by an image compression method such as MPEG1 and MPEG2 is output to the transmission device 4.
Here, as a method of compressing image data, any method may be used as long as the method compresses image data using inter-frame correlation, other than the method described above.

The transmitting device 4 performs processing such as speed conversion and code conversion on the compressed image data input from the encoding device 3, and transmits and outputs the compressed image data converted by those processes to the transmission line 5. , The transmission of the transmission path 5 is controlled.

The receiving device 7 controls the reception of the transmission line 6,
Receiving the compressed image data transmitted via the transmission path 6,
After performing speed conversion, code conversion, and the like corresponding to the transmission-side device on the received compressed image data, the compressed image data converted by the processing is output to the decoding device 8. The decoding device 8 decodes the compressed image data input from the receiving device 7 by a decompression method corresponding to the transmitting device, and then outputs the decoded target image data to the image output device 9.

The image output device 9 includes image output means such as a CRT or a liquid crystal display, for example.
Performs various processes such as D / A conversion and color component system conversion on the digital image data input from the PC, and displays and outputs analog image data obtained by the processes. Here, when the image output device 9 includes a device for displaying and outputting digital image data, the image output device 9 is used.
May not have the function of performing the D / A conversion process,
After performing various processes such as color component system conversion on the digital image data input from the decoding device 8, the processed digital image data is displayed and output.

The control device 10 performs processing such as various calculations.
A PU, a ROM storing a predetermined program, a RAM used as an arithmetic processing and an image data storage area, and the like. The image input device 1, the shift correction device 2, the encoding device 3, the transmission device 4, the reception device 7, Various controls are performed on the decoding device 8 and the image output device 9.

An image transmission of moving image data performed by the image transmission system shown in FIG. 1 will be described. Processing such as detection, calculation, and correction in each component device is performed by the control device 1.
Controlled by 0. First, the transmitting side device sequentially receives analog image data from the image input device 1 and performs various corrections such as A / D conversion and shading, and system conversion of color components on the input analog image data. After that, the digital image data obtained by those processes is output to the deviation correcting device 2.

Next, in the shift correction device 2, the target image data input from the image input device 1 is stored in the first storage unit 11.
Using the image data already stored in the second storage unit 12 as the reference image data, the detection unit 13 detects corresponding constituent blocks from the target image data and the reference image data. Then, the amount of displacement between the two constituent blocks is detected by the amount of displacement detection means 14, and the correcting means 1 is set based on the detected amount of displacement.
After the target image data is corrected by 5 and the amount of displacement with respect to the reference image data is reduced, the corrected target image data is output to the encoding device 3.

Next, in the encoding device 3, the target image data input from the displacement correcting device 2 is subjected to compression processing by inter-frame correlation based on the reference image data, and the transmission device 4 performs processing such as speed conversion and code conversion. After that, the compressed image data converted by those processes is transmitted to the transmission line 5.
Is output to Therefore, the target image data is restored to the original degree of inter-frame correlation with respect to the reference image data, and compression is efficiently performed by the inter-frame correlation.

On the other hand, in the receiving apparatus, the compressed image data transmitted via the transmission line 6 is received by the receiving apparatus 7, and the received compressed image data is subjected to speed conversion and code conversion corresponding to the transmitting apparatus. After the processes such as are performed, the compressed image data converted by those processes is output to the decoding device 8.

Next, in the decoding device 8, after the compressed image data input from the receiving device 7 is decoded by the decompression method corresponding to the transmitting device, the decoded target image data is converted to the image output device 9. Output to Next, after various processes such as D / A conversion and color component system conversion are performed on the digital image data input from the decoding device 8 in the image output device 9, the analog image obtained by those processes is processed. The data is displayed and output.

Here, the flow of the positional deviation correction processing performed by the deviation correction device 2 will be described with reference to the flowchart shown in FIG. In this example, as shown in FIG. 4, a plurality of constituent blocks are extracted as sample blocks 21, 22, and 23 from the center axis of the frame of the reference image data 20, and the sample blocks 21 are extracted from the frame of the target image data 24. , 22, and 23 corresponding to blocks,
A method of correcting the target image data based on the average amount of the positional shift amount between the corresponding blocks (21 and 25, 22 and 26, 23 and 27) after detecting 26 and 27 will be described.

First, in step s1, the target image data 24 input from the image input device 1 is stored in the first storage means 11, as shown in FIG. It is assumed that the reference image data 20 is already stored and held in the second storage unit 12. Here, in the reference image data 20, the subject is located at the center of the frame, but in the target image data 24, the position of the subject is shifted from the center of the frame by a certain amount.

Next, in step s2, it is determined whether or not the block search performed in step s6, which will be described later, has been performed a predetermined number of times. If is not detected, the search process is stopped in step s3, and the target image data 24 is output without correction. On the other hand, if the block search has not been performed a predetermined number of times, in step s4, for example, a sample block 21 is extracted from the center axis of the frame with respect to the reference image data 20.

Next, in step s6, a search for the same or similar block as the extracted sample block 21 is performed on the target image data 24. This block search is performed on the entire target image data 24. Since the search process takes a long time, the target image data 2
4. A predetermined search range is determined for step 4, and step s
In block 6, a block search is performed within this search range.

The predetermined search range is, for example, when the sample block 21 is extracted from the reference image data 20, the sample block 2 is extracted from the target image data 24.
1 is a range determined as a range in the vicinity of the position corresponding to 1, and within a frame of the target image data 24, a certain size square or circle centered on the position corresponding to the sample block 21 is set as a search range. May be appropriately determined according to the position corresponding to the sample block 21 in the frame of the target image data 24.

Here, if the block search is not successful in step s6, the process returns to step s2, and the process is terminated or another sample block 2
2 and 23 are extracted. On the other hand, the sample block 21
Is detected, and if the block search is successful, the amount of displacement between the two constituent blocks is calculated in step s7.

Here, in this example, when correcting the target image data 24, an amount obtained by averaging the displacement amounts of a plurality of constituent blocks is used. Therefore, in step s8, a predetermined number of constituent blocks required for the correction processing are used. It is determined whether or not is detected. Here, for example, in the case of the present example, the predetermined number is three, and if three constituent blocks have not been detected yet by the search in step s6, the process returns to step s2, and the process ends or Extract another sample block.

On the other hand, for example, the sample blocks 22 and 23
Are detected, and if a predetermined number of constituent blocks are detected, the process proceeds to step s
In step 9, the detected positional deviation amounts of the constituent blocks are averaged to obtain a positional deviation correction amount for performing correction. Here, the displacement correction amount is expressed as a vector amount. For example, as shown in FIG. 4, the X′-Y ′ orthogonal axis in the target image data 24 is set to the XY orthogonal axis in the reference image data 20. The target image data 24 can be corrected by performing movement correction in a direction in which the vector amount becomes zero.

Next, in step s10, for example, FIG.
As shown in (1), an output pixel range 31 surrounded by a dotted line for outputting the target image data 30 as the corrected target image data 32 is determined. Then, in step s11, the target image data 30 is corrected by cutting out the determined output pixel range 31, and the corrected target image data 32 is output to the encoding device 3. here,
FIG. 5 shows a case where the target image data 30 before correction has a larger number of pixels than the target image data 32 after correction.

In the above embodiment, the target image data is corrected based on a plurality of constituent blocks. However, it is not always necessary to detect and correct a plurality of constituent blocks. In this case, there is no need to provide the processing means in step s8 in the above processing configuration. Further, since the positional deviation between the two constituent blocks is detected in step s7, It is not necessary that the processing means in s9 be provided.

In the above embodiment, the sample block is extracted from the center axis of the frame of the reference image data. However, the sample block does not necessarily need to be extracted from the center axis of the frame. A sample block may be extracted from the image part. In addition, any coordinate system may be used in various arithmetic processing in correction processing such as block extraction and block detection, and an orthogonal coordinate system or a polar coordinate system may be used.

Also, in the above embodiment, when detecting the corresponding constituent blocks from the target image data and the reference image data, after extracting the sample blocks from the reference image data, the same or the same as the sample blocks are extracted from the target image data. A method of detecting a similar block was used. On the contrary, a method of extracting a sample block from target image data and then detecting a block identical or similar to the sample block from the reference image data was used. You may.

In the above embodiment, the search range is determined in advance in step s5 when the block search is performed on the target image data. However, the entire target image data is determined without determining the search range as described above. A block search may be performed, and in this case, the processing configuration need not include step s5.

Also, in the above embodiment, a block identical to or similar to a sample block is detected from the target image data in the block search processing. In this processing, when a certain block is similar to the sample block, It is necessary to determine in advance the similarity detection range of whether to detect it as a similar block,
This range may be determined according to the image quality of image data used for transmission, the desired degree of frame correlation after correction processing, and the like.

In the above embodiment, when correcting the target image data in steps s10 and s11, the correction is performed by cutting out a part of the target image data. In this way, the number of pixels of the target image data is reduced. Instead of the correction method of outputting the image data, a correction method of outputting the corrected image data with the same number of pixels or a larger number of pixels as the image data before correction may be used. In this case, step s10 And s11 may be configured by means for performing such a correction method.

In the above embodiment, as a process when the block search is not successful, if the predetermined number of blocks are not detected even after the predetermined number of search processes in step s2, the process is stopped. Output without correcting the target image data was used.
Even if it is not necessarily such a method, a method of repeating the detection process only within a predetermined time, or if even one detected block could not be detected, stop the process and correct the target image data without correcting A method of outputting, a method of performing correction within a range of the number of detected blocks even if a predetermined number of blocks are not detected, and further, for example, searching within a predetermined number of times and within a predetermined time. For example, a method in which the above processing methods are combined or the like may be used.

In the above embodiment, the transmission line 5 is used as the transmission line for the transmission side device, and the transmission line 6 is used for the reception side device.
However, as the transmission path, the transmission side apparatus and the reception side apparatus may share one transmission path.

[0054]

As described above, according to the image transmission apparatus of the present invention, when compressing a plurality of image data by inter-frame correlation and transmitting the image data, the image data is not compressed before the image data is compressed. Since the correction is made to increase the inter-frame correlation, the code amount of the compressed image data can be reduced, and the image transmission can be performed efficiently.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an image transmission system to which an image transmission device according to an embodiment of the present invention is applied.

FIG. 2 is a diagram for explaining means provided in a displacement correction device according to one embodiment of the present invention.

FIG. 3 is a diagram for explaining a flow of a position shift amount correction process according to an embodiment of the present invention.

FIG. 4 is a diagram for explaining misregistration correction according to one embodiment of the present invention.

FIG. 5 is a diagram of target image data before and after a positional deviation amount correction according to an embodiment of the present invention.

FIG. 6 is a configuration diagram of an image transmission system to which a conventional image transmission device is applied.

FIG. 7 is a diagram for explaining blurring occurring in image data in a conventional image transmission device.

[Explanation of symbols]

1. Image input device 2. Misalignment correction device 3.
Encoding device, 4 ··· Transmission device, 5 ··· Transmission path, 6 ·
Transmission line, 7 receiving device, 8 decoding device, 9
..Image output devices 10 Control devices

Claims (1)

[Claims] 1. An image transmission apparatus for sequentially transmitting a plurality of image data subjected to compression processing using inter-frame correlation, wherein: first storage means for storing target image data to be subjected to compression processing; Second storage means for storing the image data of the frame as reference image data; and constituent blocks corresponding to the target image data stored in the first storage means and the reference image data stored in the second storage means. Detecting means for detecting the positional deviation, a deviation amount detecting means for detecting the positional deviation amount between the two constituent blocks, and correcting the target image data held in the first storage means based on the detected positional deviation amount Means for reducing the amount of displacement with respect to the reference image data, and a code for compressing the target image data corrected by the correction means based on the reference image data An image transmitting apparatus comprising: a converting unit; and a transmitting unit configured to transmit and output the compressed target image data.