JPH11146395A - Image transmission system and image coder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the quality of a desired area in one image pattern and to change a data quantity in a frame depending on a transmission rate. SOLUTION: An area to improve the quality in one image pattern is selected by a remote controller 2c and information (pattern selection signal) relating to the area is fed to a remote side image transmitter 1. An encoder 11 of the remote side image transmitter 1 encodes image data with a different selection of an internal quality table when image data supplied from a camera 1a denote an area whose quality is to be improved as designated by a pattern selection signal or other area, and the image data are sent to a local side image transmitter 2 via a transmission buffer memory 13. In this case, the encoder 11 selects the quantization table in response to a transmission rate of a transmission line 3 so that the quantity of the image data in one pattern is made equal to each other.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image transmission system for transmitting image data through a transmission line having a limited transmission rate, and an image encoding apparatus suitable for encoding an image transmitted by such a system. In particular, the present invention relates to a device capable of designating a region for improving image quality by remote control.

[0002]

2. Description of the Related Art In a video conference system, image data of a moving image is transmitted through a transmission line such as a telephone line. The image data of this moving image has a considerably large amount of data, but the amount of data (transmission rate) that can be transmitted through a transmission path is limited, so that the image data is usually compressed and transmitted. .

However, the image data is compressed and transmitted, so that the image quality of the image displayed on the receiving side of the image data is degraded. On the other hand, in a video conference system, a plurality of participants of a conference are usually displayed on a screen, but only a participant such as a speaker attracts attention on the display screen. For this reason, the region where the speaker is projected needs to be a high-quality image, but no problem occurs even if the other regions are low-quality images. That is, it is convenient if the compression ratio of the image data can be changed for each area of the screen.

As a technique capable of changing the compression ratio of image data for each area of the screen, for example, a compression recording / reproducing method of image data disclosed in Japanese Patent Application Laid-Open No. Hei 4-24689 (hereinafter referred to as a conventional example) 1) and JP-A-5-37
There is an image recording apparatus disclosed in Japanese Patent Application Laid-Open No. 901 (hereinafter, Conventional Example 2).

[0005]

In the prior art 1, the compression ratio of image data is determined for each divided area according to the contrast. However, in this case, the contrast of the region where the person of interest is displayed is not always high. Therefore, even if Conventional Example 1 is applied to a video conference system, a desired result is not always obtained.

On the other hand, in Conventional Example 2, one frame image is divided into a plurality of regions, and the image is compressed by changing the quantization step size for each region in accordance with an external input. However, prior art 2 does not consider a case where an image is transmitted in real time to a remote place via a transmission path, and does not disclose a technique for limiting the amount of data per frame due to the limitation of the transmission rate.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, and when transmitting image data in real time through a transmission path, it is possible to specify a region in which image quality is to be improved, Image transmission system that can change the amount of
And an image encoding apparatus applied to the image transmission system.

[0008]

In order to achieve the above-mentioned object, image data supplied from the outside is encoded by a transmitting-side image encoding device according to a first aspect of the present invention.
An image transmission system that transmits the encoded image data to the reception side via a transmission path, decodes the image data encoded by the decoding device on the reception side, and displays the image data on the display device, is provided on the reception side, A region selecting unit that selects a desired region from the image display regions of the display device and supplies region data indicating the selected region to the image encoding device on the transmission side via the transmission path; The image encoding device on the side identifies whether the image data supplied from the outside is of the selected area indicated by the area data supplied from the area selecting means via the transmission path. And a compression unit for compressing the image data supplied from the outside in accordance with the identification result of the region identification unit.

In the above-described image transmission system, for example, a desired area for which image quality is desired to be improved can be selected from the receiving side by the area selecting means. Information about the selected area is transmitted over a transmission path,
It is supplied to the area identification means. Then, by compressing the image data supplied from the outside by the compression unit according to the identification result of the area identification unit,
The compression ratio of image data can be different for each area. Moreover, with the above configuration, image data can be compressed in real time. As a result, on the display device on the receiving side, for example, a high quality image is displayed in the selected area, and a relatively low quality image is displayed in the other areas.

In the above-mentioned image transmission system, it is preferable that the compression means includes means for compressing the externally supplied image data so as to conform to a transmission rate of the transmission line.

In this case, the image data supplied from the outside is compressed according to the transmission rate of the transmission path. In particular, the image data outside the selected area is compressed according to the transmission rate. The amount of data
This is suitable for transmission through the transmission path.

According to a second aspect of the present invention, there is provided an image encoding apparatus for encoding externally supplied image data and transmitting the encoded image data through a transmission line. The image data supplied from is supplied from the outside according to the identification result of the area identification means for identifying whether or not the image data supplied from the area is a desired area indicated by the externally supplied area data. Compression means for compressing the compressed image data.

In the above-mentioned image coding apparatus, for example, a desired area where the image quality is desired to be improved can be selected from outside. The area identification means identifies whether the image data supplied from the outside is that of the selected area,
According to the identification result, the compression unit compresses the image data supplied from the outside. This makes it possible to make the compression ratio of image data different for each area. Moreover, with the above configuration, image data can be compressed in real time.

[0014] In the above-mentioned image encoding apparatus, it is preferable that the compression means includes means for compressing the image data supplied from the outside so as to follow the transmission rate of the transmission path.

In this case, the image data supplied from the outside is compressed according to the transmission rate of the transmission path. In particular, the image data outside the selected area is compressed according to the transmission rate. The amount of data
This is suitable for transmission through the transmission path.

In the above image encoding apparatus, the compression means may include, for example, a plurality of quantization tables having different stored quantization widths, and the plurality of quantization tables in accordance with an identification result of the area identification means. A selector for selecting any one of the following quantization tables; a subtractor for generating a difference signal between the image data supplied from the outside and the corresponding image data one frame before; A discrete cosine transformer for performing a discrete cosine transform of the difference signal generated by the device, and a quantizer for quantizing the difference signal that has been subjected to the discrete cosine transform by the discrete cosine transformer according to the quantization table selected by the selector. , Can be provided.

In the image encoding apparatus, the area data may be supplied to the area identification means through the transmission path.

[0018]

Embodiments of the present invention will be described below with reference to the accompanying drawings. In this embodiment, a case where the image transmission system of the present invention is applied to a video conference system will be described as an example.

FIG. 1 is a block diagram showing the configuration of a video conference system according to this embodiment. As shown in the figure, the video conference system includes a remote side image transmission device 1 connected via a transmission line 3 such as a telephone line,
And a local-side image transmission device 2. The camera 1a and the monitor 1b are connected to the remote-side image transmission device 1. A monitor 2a, a camera 2b, and a remote controller 2c are connected to the local-side image transmission device 2.

The cameras 1a and 2b each take an image, and add a synchronization signal to digital image data corresponding to the taken image and supply the digital image data to the encoders 11 and 21. The monitors 2a and 1b display moving images corresponding to the image data expanded by the decoders 16 and 26, respectively, as described later. The remote controller 2c is for selecting and inputting an area in which a high-quality image is to be displayed from a screen divided into a plurality of areas as described later.

The remote-side image transmission device 1 includes an encoder 11, a multiplexer (MUX) 12, a transmission buffer memory 13, a reception buffer memory 14, a demultiplexer (DMUX) 15, and a decoder 16. . The local image transmission device 2 includes an encoder 21, a multiplexer (MUX) 22, a transmission buffer memory 23, a reception buffer memory 24, a demultiplexer 25, and a decoder 26.

The encoders 11 and 12 compress and output the image data supplied from the cameras 1a and 2b, respectively. The configuration of the encoder 11 of the remote-side image transmission device 1 will be described in more detail. The multiplexers 12 and 22 are respectively connected to the encoders 11 and 12
Is superimposed on audio data encoded by an audio encoding circuit (not shown) on the image data output from. The multiplexer 22 further superimposes and outputs a later-described screen selection signal output from the remote controller receiver 27. The transmission buffer memories 13 and 23 store the output signals of the multiplexers 12 and 22, respectively.

The reception buffer memories 14 and 24 are stored in the transmission buffer memories 23 and 13, respectively,
Are temporarily stored. The demultiplexers 15 and 25 are respectively connected to the reception buffer memory 14,
The signals stored in H.24 are separated for each signal type (for image data, audio data, and screen selection signal). The demultiplexer 15 supplies image data to the decoder 16 and supplies a screen selection signal to the encoder 11. The demultiplexer 25 supplies the image data to the decoder 26.

The decoders 16 and 26 expand the image data supplied from the demultiplexers 15 and 25, respectively, and supply them to the monitors 1b and 2a. The decoder 26 has a quantization table similar to the quantization table of the encoder 11 to be described later, and a signal (screen selection signal) supplied from the remote controller receiver 27 and indicating a region where a selected high-quality image is to be displayed. , The received image data is expanded. The remote controller receiver 27
A signal from the remote controller 2c is received, and a signal (screen selection signal) indicating an area in which the selected high-quality image is to be displayed is supplied to the multiplexer 22 and the decoder 26.

FIG. 2 is a block diagram showing the configuration of the encoder 11 of the remote-side image transmission device 1. As shown, the encoder 11 includes a frame counter 110,
Decoder circuit 111, encoding control circuit 112, quantization table 113, selector 114, subtractor 115
, A DCT circuit 116, a quantizer 117, a variable length encoding circuit 118, an inverse quantizer 119, an IDCT circuit 120, an adder 121, a frame memory 122,
And a Tframe calculation circuit 123.

The frame counter 110 is a counter that counts up each time image data for one pixel is input and is cleared each time a frame pulse (vertical synchronization signal) is input. The count value is the address of the image data. It shows. The count value of the frame counter 110 is supplied to the decoder circuit 111. Decoder circuit 1
Reference numeral 11 decodes the screen selection signal supplied from the demultiplexer 15 and the count value supplied from the frame counter 110, and outputs an ON / OFF signal Sa and a selection area signal Sb.

The Tframe calculation circuit 123 calculates the following equation 1 according to the transmission rate of the transmission line 3 to calculate the limit value Tframe of the amount of generated information with respect to the transmission rate.
It is supplied to the encoding control circuit 112.

Tframe = (transmission rate) / number of frames per second If the transmission rate of the transmission path 3 is fixed, Tfr
The ame calculation circuit 123 may not be provided. However, also in this case, it is necessary to supply the limit value Tframe (fixed value) of the generated information amount to the encoding control circuit 112.

The encoding control circuit 112 includes a decoder 111
ON / OFF signal Sa and selection area signal S
b, the limit value Tframe of the generated information amount calculated by the Tframe calculation circuit 123, and the information amounts S1 and S2 supplied from the quantizer 117, which will be described later, to convert the quantization table 113 with the image data of the selected area. It outputs a control signal Sc for switching. The encoding control circuit 112 includes, as shown in FIG. 3, generated information amount calculation circuits 31, 32, virtual buffers 33, 34, and virtual buffer calculation circuits 35, 36.
, Quantization width calculation circuits 37 and 38, and a selector 39.

The generated information amount calculation circuit 31 includes a multiplier 31a for multiplying a limit value Tframe of the generated information amount by a gain value described later, and a divider 31b for dividing the multiplication result of the multiplier 31a by the number of screen divisions. And the target generated information amount T1 for the selected area by performing the calculation of Equation 2
Is calculated.

## EQU2 ## T1 = gain.times. (Tframe / number of screen divisions)

The generated information amount calculation circuit 32 is composed of a subtracter for subtracting the generated information amount T1 of the selected area from the limit value Tframe of the generated information amount, and calculates the equation (3) to generate a target generated area other than the selected area. The information amount T2 is calculated.

## EQU3 ## T2 = Tframe-T1

The virtual buffer 33 stores the output of the virtual buffer calculation circuit 35 for one frame period. The virtual buffer calculation circuit 35 calculates the actual information amount S from the quantizer 117.
Subtractor 35 for subtracting target generated information amount T1 from 1
a, an adder 35b that adds the information amount d1 ′ of the previous frame stored in the virtual buffer 33 and the output of the subtractor 35a
And calculates the value of the virtual buffer d1 by performing the calculation of Expression 4.

## EQU4 ## d1 = d1 '+ S1-T1

The virtual buffer 34 stores the output of the virtual buffer calculation circuit 36 for one frame period. The virtual buffer calculation circuit 36 calculates the actual information amount S from the quantizer 117.
Subtractor 36 for subtracting the target generated information amount T2 from 2
a, the information amount d2 'of the previous frame stored in the virtual buffer 34 and the output of the subtractor 36a.
The value of the virtual buffer d2 is obtained by performing the calculation of Expression 5.

## EQU4 ## d2 = d2 '+ S2-T2

The multiplier 37 multiplies the value d1 of the virtual buffer by a predetermined proportionality constant a1 to obtain a quantization width q in the selected area.
1 is determined. The multiplier 38 multiplies the value d2 of the virtual buffer by a predetermined proportionality constant a2 to determine the quantization width q2 outside the selected area. The selector 39 includes a decoder circuit 111
And outputs a control signal Sc in accordance with the selection area signal Sb supplied from.

The encoding control circuit 112 includes the multiplier 3
It has a gain setting circuit that sets the value of gain supplied to 1a. This gain setting circuit, as shown in FIG. 4, performs LSB and ON / O of a fixed gain setting value.
An OR circuit 4x for performing a logical sum operation with the FF signal Sa, and AND circuits 41 to 4n for performing a logical product operation of another fixed bit of the gain and the ON / OFF signal. As a result, the gain setting circuit turns ON / O
When the FF signal Sa is OFF, the gain value is 1
When ON, a fixed gain value is output.

The quantization table 113 shown in FIG. 2 has a plurality of quantization tables (1) to (n) each corresponding to the value of the control signal Sc. The selector 114 selects one of the quantization tables (1) to (n) according to the control signal Sc supplied from the encoding control circuit 112 and supplies the selected quantization table to the quantizer 117.

The subtracter 115 outputs difference data obtained by subtracting the image data stored in the frame memory 122 from the image data supplied from the camera 1a. The DCT circuit 116 converts the difference data from the subtracter 115 into a DCT (Di
The discrete cosine transform is performed by multiplying by the transform matrix of "screte Cosine Transform".

The quantizer 117 quantizes the differential data after the discrete cosine transform according to the quantization table supplied from the selector 114. The amounts of information S1 and S2 generated by the quantizer 117 are added to the adder 35 of the encoding control circuit 112.
a, 35b. The variable length encoding circuit 118
In the combination of the run and the level, run-length encoding is performed by assigning a short code end to a set having a high appearance frequency and a long code end to a set having a low appearance frequency. The image data run-length encoded by the variable length encoding circuit 118 is sent to the transmission buffer memory 13 via the multiplexer 12.

The inverse quantizer 119 inversely quantizes the difference data quantized by the quantizer 117. IDCT circuit 1
Reference numeral 20 performs inverse inverse DCT on the difference data inversely quantized by the inverse quantizer 119. The adder 121 adds the difference data subjected to the inverse IDCT conversion by the IDCT circuit 120 to the image data of the corresponding pixel one frame before stored in the frame memory 122. As a result, the content stored in the frame memory 122 is updated to the image data of the new frame. Note that the frame memory 122 stores the image data output from the adder 121 for one frame.

The selection of the area by the remote controller 2a
As shown in FIG. 4, when one screen is divided into nine regions by dividing the screen vertically and horizontally into three regions, and the respective regions are numbered 1 to 9, this can be performed by designating these numbers. FIG. 4 shows that the enhancement of the image quality of the sixth area is selected.

The operation of the video conference system according to this embodiment will be described below. Remote control 3
Upon receiving the signal according to the selection in c, the remote controller receiver 27 outputs a screen selection signal designating that the area No. 6 has high image quality. This screen selection signal is supplied to the decoder circuit 111 of the encoder 11 via the multiplexer 22, the transmission buffer memory 23, the transmission path 3, the reception buffer memory 14, and the demultiplexer 15. On the other hand, the digital image data from the camera 1a is supplied to the subtractor 115 of the encoder 11, and the image data and the frame pulse in the synchronization signal are supplied to the frame counter 110.

The decoder circuit 111 is turned on based on the count value of the frame counter 110 and the screen selection signal.
A / OFF signal Sa and a selection area signal Sb are generated and supplied to the encoding control circuit 112. The encoding control circuit 112 is also supplied with the limit value Tframe of the generated information amount from the Tframe calculation circuit 123 and the actual generated information amounts S1 and S2 from the quantizer 117.

In the encoding control circuit 112, ON / OFF
According to the control signal Sa, the limit value Tframe of the generated information amount, and the actual generated information amounts S1 and S2, the quantization width q1 in the selected region and the quantization width q2 outside the selected region are calculated.
One of the quantization width q1 in the selected area and the quantization width q2 outside the selected area is selected by the selector 39 according to the selected area signal Sb, and is output as the control signal Sc.

According to the control signal Sc output from the encoding control circuit 112, that is, according to the quantization width to be quantized, one of the quantization tables (1) to (n) in the quantization table 113 is selected by the selector 16. Selected.
The selected quantization table is supplied to the quantizer 117.

On the other hand, the difference between the image data supplied from the camera 2 b and the image data of the corresponding pixel one frame before stored in the frame memory 25 is subtracted by the subtracter 115. The difference signal output from the subtracter 115 is subjected to a discrete cosine transform by the DCT circuit 116 and further quantized according to one of the quantization tables (1) to (n) selected by the selector 16. Then, it is run-length encoded by the variable length encoding circuit 118 and stored in the transmission buffer memory 13 via the multiplexer 12.

The image data compressed as described above and stored in the transmission buffer memory 13 is transmitted via the transmission line 3 and stored in the reception buffer memory 24 of the local image transmission device 2.

The compressed image data stored in the reception buffer memory 24 is supplied to the decoder 26 via the demultiplexer 5. In the decoder 26, the compressed image data is decompressed by processing in the reverse order of the compression, and
It is supplied to the monitor 2a. Then, an image corresponding to the image data expanded by the decoder 26 is displayed on the monitor 2a.

The image data picked up by the local camera 2b is compressed by the encoder 21,
The data is input to the decoder 16 via the multiplexer 22, the transmission buffer memory 23, the transmission path 3, the reception buffer memory 14, and the demultiplexer 15. The decoder 16 decompresses the compressed image data in the reverse order and displays it on the monitor 1b. In this case, however, no area is specified, and the entire screen is displayed with the same image quality. The audio data is also compressed, transmitted between the remote side and the local side, and output after being subjected to decompression processing.

As described above, in the video conference system according to the present embodiment, a desired area on one screen where the image quality is to be improved can be selected by operating the local remote controller 2c. Information on the selected area is transmitted through the transmission path 3 and supplied to the encoder 11 of the remote-side image transmission device 1 as a screen selection signal. Then, in the encoder 11, based on the ON / OFF signal Sa and the selection area signal Sb output according to the screen selection signal, the encoding control circuit 112 causes the corresponding quantization tables (1) to (1) to
Since the selector 114 selects (n), the compression ratio of the image data can be changed in real time according to the area. As a result, the area selected by the operation of the remote controller 2c is displayed with high image quality, and the other areas are displayed with relatively low image quality. Moreover, since the encoding control circuit 112 switches the selection of the quantization tables (1) to (n) according to the transmission rate of the transmission path 3, for example,
This is suitable for transmitting image data to a remote place.

In the above embodiment, the decoder circuit 11
The ON / OFF signal Sa output from 1 has been used for setting the value of gain by the circuit shown in FIG. On the other hand, as shown in FIG. 6, the control of the selector 3X for selecting the information amount calculated by the generated information amount calculation circuit 31 of the encoding control circuit 112 and the limit value Tframe of the generated information amount is ON.
The / OFF signal Sa may be used.

In the above embodiment, one screen is divided into nine areas of the same size. However,
The number, size and shape of the divided areas are not limited to those described above. Further, the number, size and shape of the divided areas may be variably designated by operating the remote controller 2c. Further, instead of dividing one screen into a plurality of areas, an area for transmitting high-quality image data may be designated.

In the above embodiment, a case has been described in which the present invention is applied to a video conference system in which image data transmission is bidirectional. However, the present invention is not limited to this, and can be applied to transmission of unidirectional image data. Further, the present invention can be applied not only to a video conference system but also to a remote monitoring system, a remote education system, and the like. In particular, in a remote monitoring system that monitors a machine that moves rapidly, the entire machine operation is transmitted with image data with a large quantization width and not so high image quality at a limited transmission rate, and the entire machine operation is monitored. In addition to being able to monitor, it is also possible to transmit a subtle movement of an instrument attached to or near the machine by using image data having a small quantization width and high image quality and confirming it.

[0052]

As described above, according to the present invention, whether or not image data supplied from the outside belongs to the selected area is identified in accordance with the selection of the desired area where the image quality is desired to be improved. Then, according to this identification result,
Since the image data supplied from the outside is compressed, the compression ratio of the image data can be different for each area. In addition, the image data can be compressed in real time.

Further, by compressing the image in accordance with the transmission rate of the transmission path, the data amount of the compressed image data becomes suitable for transmission via the transmission path.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a video conference system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a circuit configuration of the encoder of FIG.

FIG. 3 is a block diagram illustrating a configuration of an encoding control circuit in FIG. 2;

FIG. 4 shows gain provided in the encoding control circuit of FIG.
FIG. 3 is a block diagram illustrating a configuration of a setting circuit.

FIG. 5 is a diagram schematically showing a selection screen.

FIG. 6 is a block diagram showing a configuration of an encoding control circuit according to a modification of the embodiment of the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 remote-side image transmission device 2 local-side image transmission device 3 transmission path 1a camera 1b monitor 2a monitor 2b camera 2c remote control 11 encoder 12 multiplexer 13 transmission buffer memory 14 reception buffer memory 15 demultiplexer 16 decoder 21 encoder 22 multiplexer 23 transmission buffer memory Reference Signs List 24 reception buffer memory 25 demultiplexer 26 decoder 27 remote controller receiver 31, 32 generated information amount calculation circuit 33, 34 virtual buffer 35, 36 virtual buffer calculation circuit 37, 38 quantization width calculation circuit 39 selector 110 frame counter 111 decoder circuit 112 Coding control circuit 113 Quantization table 114 Selector 115 Adder 116 DCT circuit 117 Quantizer 118 Variable length code Circuit 119 inverse quantizer 120 IDCT circuit 121 the adder 122 frame memory 123 Tframe calculator

Claims (6)

[Claims] An image data supplied from outside is encoded by an image encoding device on a transmission side, the encoded image data is transmitted to a reception side via a transmission path, and encoded by a decoding device on the reception side. An image transmission system which decodes the decoded image data and displays the decoded image data on a display device, wherein the image transmission system is provided on a receiving side, selects a desired region from an image display region of the display device, and indicates an area indicating the selected region An area selection unit that supplies data to the image encoding device on the transmission side via the transmission line, wherein the image encoding device on the transmission side transmits the image data supplied from the outside via the transmission line. Area identification means for identifying whether or not the area data supplied from the area selection means belongs to the selected area; and an externally supplied area according to the identification result of the area identification means. Image transmission system comprising a compression means, it is characterized in that compressed image data. 2. The apparatus according to claim 1, wherein the compression unit includes a unit configured to compress the image data supplied from the outside so as to be in accordance with a transmission rate of the transmission line. Image transmission system. 3. An image encoding apparatus for encoding image data supplied from outside, and transmitting the encoded image data through a transmission path, wherein the image data supplied from outside is supplied from outside. Area identification means for identifying whether or not the area data corresponds to a desired area indicated by the area data, and compression means for compressing the image data supplied from outside according to the identification result of the area identification means. An image encoding device characterized by the above-mentioned. 4. The apparatus according to claim 3, wherein the compression unit includes a unit that compresses the image data supplied from the outside so as to conform to a transmission rate of the transmission line. Image encoding device. 5. The compression means comprises: a plurality of quantization tables having different quantization widths stored therein; and any one of the plurality of quantization tables according to an identification result of the area identification means. A selector for selecting a conversion table; a subtractor for generating a difference signal between the image data supplied from the outside and the corresponding image data one frame before; and a difference signal generated by the subtractor. A discrete cosine transformer for performing discrete cosine transform, and a quantizer for quantizing a differential signal subjected to discrete cosine transform by the discrete cosine transformer according to a quantization table selected by the selector. The image encoding device according to claim 3, wherein 6. The image encoding apparatus according to claim 3, wherein the area data is supplied to the area identification unit via the transmission path.