JPH08214314A - Image transmission equipment - Google Patents

Abstract

PURPOSE: To transmit a still picture in real time similarly as a moving image by applying frame-encoding to the still picture and inserting it to the code of the moving image. CONSTITUTION: SIF-formatted still picture data is encoded by an intra-encoding system of MPEG system, and converted to one picture. A picture header is attached on such picture data, and code constitution of only one sheet/one picture is formed as shown in (B). Such encoded still picture data is inserted to just before one picture of real time encoded MPEG code as shown in (A). An insertion timing is set at a time just before one picture appearing in a MPEG bit stream first after the completion of intra-encoding of the still picture. In such a way. the MPEG bit stream including a still picture intra-frame is recognized by a code on a reception side, and the still picture is reproduced separately from the reproduction of the moving image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image transmission device for encoding and transmitting moving images and still images.

[0002]

2. Description of the Related Art Rapid advances in digital image processing technology have made it possible to realize not only black and white images handled in the G3 and G4 standards, but also color images and moving images in image communication. This is due to the fact that the coding techniques for various images have advanced and international standardization has been advanced. For example, JBIG, which is a hierarchical encoding method for binary images, JPEG, which is an encoding method for color natural images, ITU-T Recommendation H.264, which is a moving image encoding method for videophones and video conferences. Two
61, and MPEG, which is a storage transmission coding method, are standardized by the international standardization organizations ISO and ITU-T.

These encoding methods are becoming popular mainly for their respective applications. That is, the JBIG method is G
The next generation of 3 and G4 facsimiles, the JPEG method for color facsimiles, and the H.264 standard. 261 is 64 kbit /
For low bit rate video conferencing from s to 384 kbit / s, the MPEG system has a relatively high bit rate, C
It is being used for storage and broadcasting such as D-ROM.

However, these encoding methods are not compatible with each other and are therefore used separately. 16
The schematic block diagram of a video conference system is shown. In FIG.
Two computers 10A and 10B are connected via a communication line 12 such as ISDN. Computers 10A, 10
B includes video conference boards 14A and 14B, video cameras 16A and 16B for inputting moving images, and monitors 18A and 18B for displaying images. The computers 10A and 10B are H.264. In accordance with H.261, moving images can be mutually communicated in real time. For example, an image input by the video camera 16A (or 16B) is transmitted by the video conference board 14A (or 14B) to H.264. 261 is encoded in accordance with H.261, and the computer 10 is connected via the communication line 12.
B (or 10A). Computer 10B
(Or 10A) decodes the received image by the video conference board 14B (or 14A), and the monitor 18B
(Or 18A).

FIG. 17 shows a schematic block diagram of a facsimile communication. Facsimile machines 20A, 20
B connects through the communication line 22. The facsimile device 20A (or 20B) encodes the image read by the attached scanner by an encoding method such as MMR. The encoded data is transmitted at the rate of the communication line 22 (64 kbit / s in the ISDN line) to the facsimile device 20B (or 2).
0A). Receiving side facsimile device 20B
(Or 20A) decodes the received code and outputs a hard copy by the printer. The transmission time depends on the capabilities of the facsimile machines 20B and 20A and the line speed of the communication line 22. Usually, it takes several tens of seconds to several minutes.

[0006]

During a video conference, that is,
You may want to transmit materials, etc. while transmitting moving images to each other. Although another facsimile device can be used, a configuration has been proposed in which a document image camera as shown in FIG. 18 is provided in the computers 10A and 10B and a still image is transmitted on the same line as the moving image transmission.

However, in the conventional example, when a still image having a sufficient resolution is transmitted by the document camera, a long transmission time and / or a large transmission capacity are required. Therefore, in order to secure a sufficient image quality, moving image transmission is required. I had to suspend it temporarily.

According to the present invention, there is no interruption in moving image transmission.
An object of the present invention is to provide an image transmission device capable of transmitting a still image with sufficient image quality.

[0009]

An image transmitting apparatus according to the present invention is a moving image input unit for inputting moving image data, a still image input unit for inputting still image data, and an input by the moving image input unit. Encoding means for encoding the moving image data using intra-picture encoding and inter-picture encoding,
The still image data input by the still image input means is encoded by intra-picture encoding of the method of the encoding means, and is encoded in the moving image encoded data encoded by the encoding means. And an output unit for inserting and outputting still image data.

[0010]

By the above means, the encoded data of the still image is inserted into the code of the moving image and transmitted, so that the still image can be transmitted in real time like the moving image. At the same time, since a full size still image is encoded and transmitted, a still image with sufficient resolution can also be transmitted. With these, both real-time property and resolution can be realized.

[0011]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a schematic block diagram showing the entire system of an embodiment of the present invention. In FIG. 1, 30
Indicates various data (in this embodiment, JPEG and MPE
ATM networks for transferring image data, audio data, control data, etc. encoded by various methods such as G (Asynchronous transfer mode), 32 and 34 are configured by Ethernet (trademark of Xerox Corporation in the United States) A local area network (LAN) 36 is a facsimile machine connected to the ATM network 30 and having a compression / expansion function of image data, and 38 is a color printer having a page memory therein.

Reference numeral 40 denotes a color copying machine equipped with a color reader and a color printer, and a compression circuit for compressing image data of an original read by the color reader based on, for example, the JPEG system, and a compressed image. It includes a page memory in which data is written and a decompression circuit for reading and decompressing the compressed image data written in the page memory. The image data expanded by the expansion circuit is supplied to an internal printer or an external printer.

Reference numeral 42 is a file server for storing and storing image data input via the ATM network 30, 44 is a workstation for inputting / outputting data to / from the file server 42, and 46 is a connection with the ATM network 30. It is a personal computer (PC) used. The computer 46 exchanges MPEG data and JPEG with the local area networks 32 and 34.
It is possible to exchange data, perform encoding / decoding of data, and perform various processes such as editing of image data. Further, this computer 46 is connected to the network 34.
Alternatively, it is connected to the printer 38 via a dedicated line.

Reference numeral 48 is a file server having the same configuration as the server 42. The server 48 includes a color copying machine 40.
A color copying machine 50 having the same configuration as is directly connected.

Reference numeral 52 is a digital television connected to the ATM network 30. The digital television 52 decodes the MPEG data and JPEG data input via the ATM network 30 and displays it on the CRT display device as a visible image. The monitor display may be, for example, a display device using a ferroelectric liquid crystal.

Reference numeral 54 is a digital video tape recorder which receives MPEG data and JPEG data input via the ATM network 30 and records them on a magnetic tape in a compressed state or after a predetermined process. . The VTR 54 also includes an image compressor that compresses uncompressed data received from the outside based on the MPEG or JPEG method, and the compressed data can also be recorded on a magnetic tape.

Reference numeral 56 denotes MPE on the ATM network 30.
CAT for sending G image data and JPEG image data
It is station V. The CATV station 56 is an ATM network 3
Broadcast data is output to 0.

Reference numeral 58 designates another A in the ATM network 30.
A router for connecting the TM network, 60 is L
A router for connecting the AN 34 to another local area network.

The facsimile device 36 and the printer 3
8, and an ATM network switch (not shown) is provided between the color copying machine 40 and the ATM network 30.

JPEG data transmitted via the ATM network 30 will be described. JPEG data is an international standard method CC that aims to compress a color still image by utilizing the frequency characteristics of data and human visual characteristics.
The data is coded based on the ITT / ISO JPEG system, and the coded data and various marker codes are shown in an image / frame / frame as shown in FIG.
It is structured as a hierarchical structure of scans.

That is, the JPEG data is the SOI (Sta
rt Of Image) code, frame, EOI
(End Of Image) code.
A frame is composed of a plurality of frames for each layer in the case of hierarchically encoded data, and is composed of a single frame when not hierarchically encoded. The frame is S
It consists of OF (Start Of Frame) code, frame header and scan, and scan is SOS.
(Start Of Scan) code, scan header, and encoded data. The scan consists of a plurality of scans when the luminance data (Y) and the two color difference data (Cr, Cb) are encoded separately (when non-interleaved), and when each data is encoded without being divided ( Interleaving) consists of a single scan.

An encoding / decoding algorithm in a basic baseline system of the JPEG system will be described with reference to FIGS. 3 and 4. FIG. 3 shows a schematic block diagram of an encoding system, and FIG. 4 shows a schematic block diagram of a decoding system.

FIG. 3 will be described. Blocking circuit 110
Divides the input image data into blocks of 8 pixels × 8 pixels, and the DCT (discrete cosine transform) circuit 112 outputs the block output from the blocking circuit 110 to one direct current component (DC) by two-dimensional discrete cosine transform. And a horizontal / vertical spatial frequency component (DCT coefficient) composed of 63 AC components (AC).

The quantization circuit 114 quantizes each frequency component output from the DCT circuit 112 by dividing each frequency component by a predetermined coefficient (quantization coefficient). The quantized DC component and AC component are processed by different algorithms. Note that the quantized coefficient generally has a different value for each frequency component, and the quantized coefficient for the visually important low-frequency component is set smaller than the quantized coefficient for the high-frequency component. This increases the compression ratio of relatively unimportant high frequency components and reduces the effect on image quality,
We are trying to reduce the amount of data as a whole.

The DC component of the DCT transform coefficient has a high correlation with the adjacent block. By utilizing this, the difference circuit 116
Is the difference of the DC component from the preceding block,
The Huffman coding circuit 118 performs one-dimensional Huffman coding on the output difference value of the difference circuit 116 and outputs it to the addition circuit 130 as DC component coded data.

On the other hand, for the AC components, the scan circuit 120 converts the above-mentioned 63 AC components into a one-dimensional array by sequentially performing zigzag scanning from the visually important frequency components on the low frequency side to the high frequency side. To do. The coefficient determination circuit 122 determines whether the value of each component is “0 value” or a value other than 0 value (effective coefficient).
Is determined. For "0 value", the run length counter 124 counts 0 runs, and for the effective coefficient, the grouping circuit 126 groups by that value. The combination of the run length and the group value obtained by the above is used as the Huffman coding circuit 12
8 outputs the two-dimensional Huffman code to the additional circuit 130 as AC component coded data.

In the Huffman coding, a shorter code length is assigned to one having a higher occurrence probability (the above-mentioned difference value for the DC component, and the combination of the run length and the effective coefficient for the AC component). This reduces the amount of data as a whole. Furthermore, by assigning a predetermined code (ZRL code) to the one having a low occurrence probability, all patterns can be expressed with a finite number of codes.

The above processing is performed for each block,
The encoding of one color still image ends. Additional circuit 13
In the case of 0, the above-mentioned marker code and the like are added to each of the above encoded data, and JPEG data having the data structure shown in FIG. 2 is output.

Since the quantization coefficient in the quantization circuit 114 and the Huffman code in the Huffman coding circuits 118 and 128 can be set arbitrarily, the data representing the quantization coefficient and the Huffman code used in the coding are It is added after the SOI code.

The decoding system shown in FIG. 4 will be described. The decoding algorithm is basically the reverse of the encoding algorithm. The decoding circuit 132 converts the input coded data into
Decoding using the Huffman table input together,
DC component is added to adder 134 and AC component is rearranged circuit 1
Supply to 36. The adder 134 adds the DC component of the preceding block to the input DC component (difference value). The rearrangement circuit 136 rearranges each decoded frequency component into the original two-dimensional array.

The inverse quantization circuit 138 inversely quantizes the DC component output from the adder 134 and the AC component output from the rearrangement circuit 136, and the inverse DCT circuit 140 inverses the output from the inverse quantization circuit 138. Discrete cosine transform. The output of the inverse DCT circuit 140 is the restored image data.

The above processing is performed for each block to complete the decoding of one color still image.

The algorithm described above with reference to FIGS. 3 and 4 is the basic one of the JPEG system.
An extended system incorporating various hierarchical codings is also recognized as the JPEG system. When this hierarchical encoding is performed, the type is represented by the SOF code.

Next, MPEG data will be described. MPE
The G method is an international standard aimed at high-efficiency coding of moving images, and basically uses the frequency characteristics of data and human visual characteristics as in the JPEG method described above.
Furthermore, a higher compression rate is achieved by utilizing the redundancy in the time axis direction that is unique to moving images.

As the MPEG system, MPEG1 having a maximum transfer rate of 1.5 Mbps for a digital storage medium is used.
There is MPEG2 which is intended to be used in all transmission systems such as bidirectional digital multimedia equipment, digital VTRs, A-TVs and optical fiber networks by eliminating the upper limit of the transmission rate. Since the basic algorithm is almost the same, based on MPEG1,
The data structure and the encoding / decoding algorithm will be described. In MPEG2, usable coding methods are defined by a plurality of profiles (simple profile, main profile, scalable, spatial scalable, and high), but a typical main profile is basically MPEG1. The principle of a high-efficiency coding system based on MPEG, which is almost the same as the above, will be described. M
PEG reduces the redundancy in the time axis direction by taking the difference between frames. Then, the difference data obtained thereby is subjected to DCT and variable length coding processing to reduce the redundancy in the spatial direction. These processes realize a high compression rate as a whole.

Regarding the redundancy in the time axis direction, in the case of a moving image, paying attention to the fact that the correlation between consecutive frames is high, and the difference between the frame to be encoded and the frame preceding or succeeding in time is calculated. By doing so, the redundancy can be reduced. In MPEG, as shown in FIG. 5, in addition to an intra-coded image (I picture) obtained in a coding mode in which coding is performed exclusively in a frame, a difference value with respect to a temporally preceding frame is coded forward. Bidirectional prediction in which a predictive coded image (P picture) and a temporally preceding or following frame difference value or a difference value between an interpolated frame by both of them and having the smallest data amount are coded A coded image (B picture) is included, and frames in these coding modes are combined in a predetermined order. That is, in MPEG, I picture, P picture, and B picture are set as 1 unit (GOP) with 1 picture, 4 pictures, and 10 pictures respectively, and an I picture is arranged at the beginning, and 2 B pictures and 1 P picture are arranged. It is recommended that the combination of repeatedly arranging is used, and by placing an I picture at a constant cycle, special reproduction such as reverse reproduction or partial reproduction in units of GOP is possible and error propagation is prevented.

When a new object appears in a frame, the difference value may be smaller by taking the difference with the subsequent frame than by taking the difference with the temporally preceding frame. . Therefore, in MPEG, the bidirectional predictive coding as described above is performed to realize a high compression rate.

In MPEG, motion compensation is also used.
That is, the difference from the macroblock in the vicinity of the corresponding block of the preceding or succeeding frame is taken as a unit (macroblock) by collecting the previous 8 pixel × 8 pixel block for 4 blocks for luminance data and 2 blocks for chrominance data. Then, the motion vector is detected by searching the macro block having the smallest difference. Then, after temporally preceding a frame is once encoded, a frame that is decoded again is set as a preceding frame, and the difference between the macro block in this preceding frame and the macro block of the frame to be encoded is DCT-transformed and Huffman encoded. To do. This motion vector is also encoded and transmitted. At the time of decoding, the corresponding macroblock data of the preceding or following frame is extracted using this motion vector, and thereby the motion compensation encoded difference data is decoded. In addition, M
PEG1 uses motion compensation between frames, but MPE
G2 employs inter-field motion compensation.

As shown in FIG. 6, the data structure of the MPEG system comprises a video sequence layer, a GOP layer, a picture layer, a slice layer, a macroblock layer and a block layer. Hereinafter, each layer will be described in order from the lower layer.

Similar to JPEG, the block layer is composed of 8 pixels × 8 pixels for each of luminance data and color difference data, and DCT is performed in units of this.

The macroblock layer is composed of 8 pixels × 8 described above.
A block of pixels is grouped into four blocks for luminance data and one block for color difference data, and a macroblock header is added. In the MPEG system,
This macroblock is used as a unit for motion compensation and coding described later. The macroblock header includes each data of the motion compensation and quantization step for each macroblock, and the six DCT blocks (Y
0, Y1, Y2, Y3, Cr, Cb) includes data indicating whether or not it has data.

The slice layer is composed of one or more macroblocks and a slice header which are consecutive in the scanning order of the image. The quantization step in a series of macroblocks in the same slice layer can be constant. The slice header has data relating to the quantization step in each slice layer, and if there is no quantization step data unique to each macroblock, the quantization step in that slice layer is fixed. The top macroblock is reset with the difference value of the DC component.

The picture layer is a collection of a plurality of slice layers in units of one frame, and is composed of a header including a picture start code and the like, and one or more slice layers following the header. The header here includes a code indicating the image encoding mode and a code indicating the accuracy of motion detection (pixel unit or half pixel unit).

The GOP layer is composed of a group start code, a header such as a time code indicating the time from the beginning of the sequence, and a plurality of I frames, B frames or P frames following the header.

The video sequence layer starts with a sequence start code and ends with a sequence end code. In the meantime, a plurality of GOPs having the same control data and image size necessary for decoding such as image size and aspect ratio are arranged.

The MPEG system having such a data structure also defines a bit stream in its standard.

An MPEG type coding apparatus and decoding apparatus will be described with reference to FIGS. 7 and 8. FIG. FIG. 7 shows a schematic block diagram of the encoding device, and FIG. 8 shows a schematic block diagram of the decoding device.

In the encoding apparatus shown in FIG. 7, the image size to be encoded corresponds to a high level of 1,920 pixels × 1,080 pixels (corresponding to a high level in MPEG2, as shown in FIG. 9). ), 1,440 pixels x 1,08
High 1440 with 0 pixels (High 1 in MPEG2
Corresponds to 440 levels. ) 4: 2: 2 or 4: 2:
CCIR. 601-compatible images (corresponding to the main level in MPEG2), SIF format, C
Some are compatible with the IF format and the QCIF format. At the low level of MPEG1 and MPEG2, the image size of the SIF format is targeted.

The block forming circuit 210 divides the image data to be encoded into blocks of 8 pixels × 8 pixels, and the output thereof is applied to the a contact of the switch 214, the subtracter 212 and the motion vector detecting circuit 234. Switch 214
Is connected to the a contact when the input image data is an intra frame (I frame), and is connected to the b contact when it is another frame (P frame or B frame).

In the case of an intra frame, the output of the blocking circuit 210 is passed through the switch 214 to the DCT circuit 2
16 and the DCT circuit 216 outputs the input data to D
After CT conversion, the quantization circuit 218 quantizes the DCT coefficient output from the DCT circuit 216. The variable length coding circuit 220 performs variable length coding on the output of the quantization circuit 218. The output of the variable length coding circuit 220 is temporarily stored in the buffer 222, the rate is adjusted, and the result is applied to the adding circuit 224.

The inverse quantization circuit 228 is the quantization circuit 2
The output of 18 is inversely quantized, and the inverse DCT circuit 230 performs inverse DCT conversion of the output of the inverse quantization circuit 228. Switch 23
8 is open in the intra frame and closed in other than the intra frame. Therefore, the adder 232 outputs the output of the inverse DCT circuit 230 as it is in the intra frame, and outputs the output of the motion compensation circuit 236 (predicted value) to the output of the inverse DCT circuit 230 in the cases other than the intra frame.
Is added and output.

The motion vector detecting circuit 234 outputs the output of the blocking circuit 210 (image to be encoded) and the adder 232.
From the output (image already encoded) and the motion vector are detected, and the detected value is supplied to the motion compensation circuit 236. The motion compensation circuit 236, according to the motion vector from the motion vector detection circuit 234, predicts the motion-compensated image data of a predetermined frame (preceding frame, subsequent frame or their interpolation frame) obtained from the output of the adder 232. As output to the adder 232 via the switch 238 and to the subtractor 212. That is, the motion compensation circuit 236 is
According to the motion vector detected by the motion vector detection circuit 234, the corresponding macroblock in a predetermined frame (leading frame, trailing frame or their interpolation frame) obtained from the output of the adder 232 is output.

The subtractor 212 subtracts the output of the blocking circuit 210 from the output of the motion compensation circuit 236, and applies the result to the b contact of the switch 214. Therefore, in the case other than the intra frame, the DCT circuit 216 performs a discrete cosine transform on the difference value, and the transform coefficient is quantized by the quantizing circuit 218 and variable length encoded by the variable length encoding circuit 220. After that, it is supplied to the additional circuit 224 via the output buffer 222.

The rate control circuit 226 uses the output buffer 22.
2 so that the amount of stored data is constant.
Control quantization step size of 8.

The variable length coding circuit 220 basically has
It is similar to the Huffman coding circuit in JPEG described above, but differs in that a predetermined code (escape code) is uniformly assigned to the one having a low occurrence probability.

The adding circuit 224 adds the various headers described above to the encoded data output from the output buffer 222, and outputs it as MPEG data compatible with the MPEG system.

The decoding device basically performs the reverse processing of the above-mentioned encoding. That is, the encoded data is applied to the variable length decoding circuit 242 via the input buffer 240, and the variable length code is decoded here. Variable length decoding circuit 2
The output of 42 is inversely quantized by the inverse quantization circuit 244,
The inverse DCT circuit 246 performs an inverse discrete cosine transform and restores the data in the spatial domain.

The switch 248 connects to the a contact in the intra frame, and connects to the b contact in the other frames. The output of the inverse DCT circuit 246 is a of the switch 248.
It is applied to the contact and the adder 252, and the output of the adder 252 is applied to the b contact of the switch 248. The output of the switch 248 is the output buffer 254 and the motion compensation circuit 2
50, and the output of the motion compensation circuit 250 is the adder 2
52 is applied. As a result, in the intra frame, the output of the inverse DCT circuit 246 is output to the outside as the restored image data via the switch 248 and the output buffer 254, and is simultaneously applied to the motion compensation circuit 250. The motion compensation circuit 250 motion-compensates the restored image data from the switch 248 and applies it to the adder 252 as a reference frame of the next inter-frame encoded frame.
Except for intra frames, adder 252 uses inverse DCT
The motion compensation data from the motion compensation circuit 250 is added to the output of the circuit 246, and the output is applied as the restored image data to the output buffer 254 and the motion compensation circuit 250 via the switch 248.

An encoding identification circuit (not shown) identifies the encoding method of the encoded data input to the input buffer 240, and controls the switch 248 according to the identification result.

The output buffer 254 temporarily stores the restored image data, rearranges the original spatial arrangement, and outputs the rearranged image data to the outside.

Next, the ATM communication format will be described. In ATM communication, as shown in FIG. 10, a series of bit streams is divided into a plurality of fixed length packets, and each packet is composed of a plurality (for example, four) ATM cells. Each ATM cell consists of a packet header and a payload for data, and in general, the packet header is 5
Bytes and data are 48 bytes. ATM communication is independent of network bit rate (asynchronous)
The data can be transmitted by, and the transmission rate can be set arbitrarily by the number of transmission cells per unit time. Therefore, it is suitable for a transmission system that transmits various data in a mixed manner.

FIG. 11 shows a personal computer 46.
3 is a schematic configuration block diagram of FIG. The configuration of the computer 46 in this embodiment will be described with reference to FIG. The computer 46 has a plurality of data buses that can be selected according to the data amount of data to be handled and the transfer speed required for processing.
That is, in this embodiment, 16-bit data buses D1, 3
2-bit data bus D2, 64-bit data bus D
3. A 128-bit data bus D4 as an expansion bus and a system bus SB are provided. Like a normal personal computer, this computer 46 also has an expansion board interface that enables expansion of functions, and desired functions can be expanded by connecting various expansion boards to this interface.

Reference numeral 310 is a network interface, which has an ATM control circuit 312 inside. Through the network interface 310 and the internal ATM control circuit 312, various data can be exchanged with each of the above-mentioned transmission channels. ATM control circuit 312
Not only functions as an ATM switch, but also the above-mentioned A
It is provided with various communication control functions such as congestion control in the TM-LAN.

Reference numeral 314 is a CPU for controlling the whole. C
In addition to the PU 314, a bus control circuit 316 that controls the above bus system as a sub CPU and a bit converter 3
18 are provided. As described above, the bus of this embodiment
In the system, the required processing capacity is provided by appropriately using any one of the data buses D1 to D4, SB according to the capacity of data to be processed and the processing speed.

Reference numeral 320 is a ROM and 322 is a memory control circuit. The memory control circuit 322 uses the external storage device 324.
Hard disk drive 326 and CD-ROM drive 3
Data is exchanged with 28.

An edit control circuit 330 controls the phase of data when editing an image.

Reference numeral 332 is a display control circuit, and the image data output from the display control circuit 332 is stored in the memory 33.
CRT display 336 and / or FLC via 4
These are applied to the display 338, and a visible image is displayed by them. The display control circuit 332 also performs known processing according to the type of display device.

Reference numeral 340 is a printer control circuit. The printer control circuit 340 stores data to be printed in the memory 3
It is applied via 42 to a thermal transfer printer 344 or a hybrid printer 346 compatible with a plurality of methods such as an inkjet method and a thermal transfer method. The printer control circuit 340 selectively uses the printers 344 and 346 according to the image data to be printed. The memories 334 and 342 may be shared for display and printing.

Reference numeral 348 is a codec for coding and decoding image data. In this embodiment, as described above,
It supports the JPEG system and the MPEG system.

Reference numeral 350 is an expansion board interface, and various expansion boards 352, 354, 356 can be connected through this interface 350 to expand the function.

Reference numeral 358 denotes a mouse / keyboard control circuit, and the keyboard 360 and the mouse 362 are connected via this control circuit 358.

Reference numeral 364 is a voice processing circuit, and 366 is a speaker for outputting voice.

A handwriting input device 370, a voice microphone 372, a video camera 374 and an image scanner 376 are also connected to the computer via the system port 368.

The computer having the configuration shown in FIG. 11 has data buses D1, D2, D3, D4 and a bus control circuit 316.
In the bus system including the bit converter 318, the optimum data bus is selectively used according to the amount of data, the transfer / processing speed required for processing, and the like.

The desired functions can be expanded by the expansion boards 352, 354, 356 connected to the expansion board interface 350. For example, by connecting a codec expansion board that performs encoding / decoding corresponding to each profile described above, these processes can be performed.

The configuration of the codec 348 in FIG. 11 will be described with reference to FIG. As shown in FIG. 12, the codec 348 has a configuration in which various functional blocks are connected to the data bus 410 and the system bus 412. Various data is transmitted and received to and from the system bus SB of the computer main body and the respective data buses SD1 to D4 via the microcomputer interface 414 and the data interface 416.

Reference numeral 418 is C for controlling the operation of the entire codec.
It is PU. The CPU 418 causes each unit to execute processing for encoding and decoding based on a program stored in the RAM 420 in advance.

A code detector 422 detects a start code (time code) in the input coded data, control codes such as various headers, and the coded data. Each code detected by the code detector 422 is sent to the CPU 41 via the data bus 410 or the system bus 412.
8 and used for controlling each part. The detected code is also stored in the parameter memory 424 so that each functional block can refer to it as needed.

426 is a motion compensation unit, 428 is a rate control unit, 430 is an encoding buffer unit,
432 is a decoding buffer unit. The input image data transmitted through the encoding buffer unit 430, or the motion vector data and the difference value transmitted from the motion compensation unit 426 may be input to the DCT circuit and the inverse DCT.
A conversion unit 434 composed of a circuit, a quantization unit 436 composed of a quantization circuit and an inverse quantization circuit, and a variable length coding unit 438 composed of a variable length coding circuit and a variable length decoding circuit are sequentially processed to serve as an output buffer. CPU 4 stored in a functional decoding buffer 432
Data bus 4 at a predetermined timing designated by 18.
10 and data bus D via interface 416
1 to D4.

The quantization unit 436 and the variable length coding unit 438 respectively include the quantization table 44.
0 and Huffman table 442 are included. Various parameters such as quantization step and Huffman code necessary for executing the processing in these units 436 and 438 are appropriately transferred from the parameter memory 424 to these tables 440 and 442. .

The coded data to be decoded is processed through the decoding buffer 432, the variable length coding unit 438, the quantization unit 436 and the conversion unit 434, and the coding buffer 43 functioning as an output buffer.
0, and the data bus 410 and the interface 41 at a predetermined timing instructed by the CPU 418.
It is output to the data buses D1 to D4 via 6.

The motion compensation unit 426 uses the reference buffer 444 to perform motion compensation of P frames and B frames during encoding and decoding. In this embodiment, the motion compensation unit 426 is also used for the process of obtaining the difference value of the DC component in JPEG encoding.

A bus arbiter 446 arbitrates the right to use the data bus 410 in pipeline processing and the like.

The codec having such a configuration receives the instruction from the CPU 314 of the computer main body and receives the instruction from the CPU 4
18 operates each predetermined unit to encode image data or decode encoded image data.

Further, in this codec, when a plurality of systems of encoding and decoding are performed at the same time, when encoding and decoding are simultaneously processed in parallel, or various processing and encoding such as communication, display and printing in the computer main body are performed. In the case where the encryption / decryption processing is performed in parallel, the data transfer to each unit and the operation are controlled in an optimum sequence according to various processing modes. The operation program corresponding to this sequence is stored in the RAM 420 in advance.

The program stored in the RAM 420 can be updated appropriately.

In the above-described embodiment, real-time moving image communication as in a video conference is performed as follows. That is, it is assumed that the computer 46 having the configuration shown in FIG. 11 is connected to the ATM network 30 or the LANs 32 and 34 as shown in FIG. The moving image input by the video camera 374 (FIG. 11) is converted to the SIF format by the moving image input interface (not shown) in the system port 368. Converted digital data Y, Cr,
Cb is transferred to the codec 348 via the data bus D4 and encoded into the MPEG data described in FIG. 6 as described in FIG. In this embodiment, 30 frames per second of SIF data are encoded in real time.

The MPEG data thus obtained is sent from the network interface 310 to the LAN 32 via the data bus D3 and transferred to another computer. The computer that receives the MPEG data also has the configuration shown in FIG. 11 similarly to the computer 46, takes in the MPEG data via the network interface 310 and the data bus D3, and decodes it by the codec 348. The decoding process is performed as described with reference to FIG. The decoded SIF data Y, Cr, Cb are transferred to the display control circuit 332 via the data bus D4 and converted into RGB signals. The display control circuit 332 applies the RGB data to the CRT display 336 via the memory 334 to display an image.

The encoding, transmission, reception, decoding and display of the SIF format moving image as described above are executed at a rate of 30 frames per second. At the same time, the moving image is transmitted through the reverse route.

Audio is transmitted at the same time as the video. The voice is input from the microphone 372, and the system port 36
Converted to a digital signal by the A / D converter in 8.
It is transferred to the audio processing circuit 364 via the system 7 / bus D1. The voice processing circuit 364 encodes the input voice data. The encoded voice data is transmitted to the partner computer via the network interface 310. The computer which should receive the encoded voice data takes in the encoded voice data by the network interface 310, decodes it by the voice processing circuit 364, and outputs the voice from the speaker 366. Audio is similarly transmitted in the opposite direction.

As described above, when it is desired to quickly transmit a high-definition color still image to the other computer during real-time moving image transmission by the MPEG system, the present embodiment also performs the following processing. It That is, the still image is input by the image scanner 376. In this embodiment, it is assumed that an A4 size image is input with a resolution of 400 dpi. The input image data is transmitted to the codec 34 via the system port 368 and the system bus SB.
8 is transferred. The codec 348 encodes the input image data by the JPEG method. The encoded image data is output to the LAN via the system bus D2 and the network interface 310, and the L
It is transferred to another computer connected to the AN.

If the computer that receives the JPEG data also has the same configuration as shown in FIG.
Data on LAN is network interface 3
10 is taken in, transferred to the codec 348 via the data bus D2, and decoded. Codec 3
The image data restored in 48 has the format shown in FIG. 13, and the data bus D2, network
It is transferred to the color printer 38 via the interface 310 and the LAN 34 and printed out.

In the case of a high-definition color still image, the amount of data becomes enormous, so that one image transmission, that is, from encoding to decoding and output (display), is much more difficult than moving image transmission. Takes time. In this sense, high-definition color still images are usually impossible or extremely difficult to be transmitted in real time in the same way as moving image transmission. In order to solve this problem, in this embodiment, the following measures are taken to realize real-time transmission at substantially the same level as moving image transmission.

That is, FIG. 1 read by the scanner 376.
The image of the format shown in FIG. 3 is JPEG encoded and transmitted, and at the same time, converted into the SIF format shown in FIG. Specifically, the CPU 314 takes in the output image data of the system port 368 and sets the longitudinal direction to about 1 /
13. Reduce the widthwise direction to 1/7 and convert to SIF format. A known smoothing subsampling method or projection method may be used for the reduction technique.

In order to transmit a still image in SIF format at the same speed as a moving image, its code data is inserted into an MPEG code and transmitted together with the moving image as follows.

As described above, the video camera 37
The moving image that is always input from 4 is encoded in real time by the MPEG method. As described with reference to FIGS. 5 and 6, the MPEG data is composed of an intra-frame encoded I picture, a forward predictive encoded P picture, and a bidirectional predictive encoded B picture. A GOP layer is formed starting from the I picture. At the beginning of each picture, there is a picture header that specifies which of I picture, P picture, and B picture.

The SIF-formatted still image data is encoded by the MPEG intra-encoding system,
Converted to an I picture. A picture header is added to this picture data, and as shown in FIG.
The code configuration has only one picture. In the picture header, information indicating that there is only one still image is written in the user data area. As shown in FIG. 14A, this encoded still image data is inserted immediately before the I picture of the MPEG code (bit stream) which is encoded in real time. The insertion timing is immediately before the I picture that first appears in the MPEG bit stream after the intra-coding of the still image is completed.

As described above, the MPEG bit stream including the still image intra frame is recognized by the codec on the receiving side, and the still image is reproduced separately from the moving image reproduction. As shown in FIG. 15, the CPU 314 causes the display control circuit 33 to display the still image in a window different from the moving image.
Control 2

As described above, by converting the still image into the SIF format and transmitting it together with the moving image bit stream, the still image can be transmitted in real time and can be displayed on the monitor screen on the receiving side. As described above, the high-definition color still image before reduction conversion to SIF has a separate JP
Since it is EG-encoded and transmitted, the receiving side may perform printout or monitor output as necessary. For editing and filing, high definition images can be used.

In the above embodiment, the moving image and the still image were transmitted via the LAN via the network interface 310, but it is obvious that the same can be done via the ATM network. In this case, the ATM control circuit 312 transmits the MPEG bit stream and the JPEG bit stream by the packet shown in FIG. The communication line may be a public line such as ISDN. I on the extension board interface 350 of FIG.
Connect the SDN interface board.

Further, although the MPEG system and the JPEG system are adopted as the coding system for moving image transmission and the high-definition color still image coding system, other coding systems may be used. For example, for moving images, ITU-T Recommendation H.264.
261 and the JBIG system for still images.

As an application example of the present invention, a still image can be inserted in a broadcast moving image such as CATV. In this case, C
The ATV station 56 inserts the still image as described above into the moving image to be transmitted (broadcast), and transmits it together. The bit stream containing the still image is transmitted over the ATM network 30. A computer (for example, 46) connected to the ATM network 30 takes in and decodes this transmission data, and displays a moving image in the moving image window and a still image in the still image window, as shown in FIG. Various information can be added to the still image.

The still image does not necessarily have to be input from the image scanner, and may be an image input from a still video camera or an image or document created on a computer.

The present invention can also be used for recording a still image as additional information when recording a moving image with the digital VTR 54.

[0106]

As can be easily understood from the above description, according to the present invention, a still image can be encoded in a frame and inserted into the code of the moving image, so that the still image can be displayed in real time like the moving image. Can be transmitted.

For a still image having a large size such as a high-definition still image, a full size still image is encoded and transmitted, and at the same time, it is reduced to the same size as a moving image and encoded.
It is transmitted as the intra-frame code of the moving image code. As a result, although it is a reduced image, real-time transmission can be realized and a high-definition image can be transmitted.

[Brief description of drawings]

FIG. 1 is a schematic configuration block diagram of a system configuration using an embodiment of the present invention.

FIG. 2 is a data structure diagram of JPEG data.

FIG. 3 is a schematic configuration block diagram of a JPEG encoding device.

FIG. 4 is a schematic block diagram of a JPEG decoding device.

[Fig. 5] Fig. 5 is a diagram showing a frame structure of an MPEG encoding system.

FIG. 6 is a data structure diagram of MPEG data.

FIG. 7 is a schematic block diagram of an MPEG encoding device.

FIG. 8 is a schematic block diagram of an MPEG decoding device.

FIG. 9 is a diagram showing various formats of an image.

FIG. 10 is a diagram showing an ATM format.

FIG. 11 is a schematic block diagram of a computer 46.

FIG. 12 is a schematic block diagram of a codec 348.

FIG. 13 is a format of a high-definition color still image.

FIG. 14 is a video sequence according to the present embodiment.

FIG. 15 is an example of a display screen that separately displays a moving image and a still image.

FIG. 16 is a schematic configuration diagram of a video conference system.

FIG. 17 is a basic connection configuration diagram of facsimile communication.

FIG. 18 is a schematic view of the appearance of a document camera.

[Explanation of symbols]

10A, 10B: Computer 12: Communication line 14A, 14B: Video conference board 16A, 16B: Video camera 18A, 18B: Monitor 20A, 20B: Facsimile device 22: Communication line 30: ATM network 32, 34: Local area・ Network (LA
N) 36: Facsimile device 38: Color printer 40: Color copying machine 42: File server 44: Workstation 46: Personal computer (PC) 48: File server 50: Color copying machine 52: Digital television 54: Digital -Video tape recorder 56: CATV station 58: Router 60: Router 110: Blocking circuit 112: Discrete cosine transform circuit 114: Quantization circuit 116: Difference circuit 118: Huffman coding circuit 120: Scan circuit 122: Coefficient determination Circuit 124: Run length counter 126: Grouping circuit 128: Huffman coding circuit 130: Additional circuit 132: Decoding circuit 134: Adder 136: Rearrangement circuit 138: Inverse quantization circuit 140: Inverse DCT circuit 210: Block Circuit 212: Subtractor 214: Switch 216: DCT circuit 218: Quantization circuit 220: Variable length coding circuit 222: Buffer 224: Additional circuit 226: Rate control circuit 228: Inverse quantization circuit 230: Inverse DCT circuit 232: Addition 234: Motion vector detection circuit 236: Motion compensation circuit 238: Switch 240: Input buffer 242: Variable length decoding circuit 244: Inverse quantization circuit 246: Inverse DCT circuit 248: Switch 250: Motion compensation circuit 252: Adder 254 : Output buffer 310: Network interface 312: ATM control circuit 314: CPU 316: Bus control circuit 318: Bit converter 320: ROM 322: Memory control circuit 324: External storage device 326: Hard disk device 328: CD-ROM device 3 0: Edit control circuit 332: Display control circuit 334: Memory 336: CRT display 338: FLC display 340: Printer control circuit 342: Memory 344: Thermal transfer printer 346: Hybrid printer 348: Codec 350: Expansion board interface 352, 354 , 356: Expansion board 358: Mouse / keyboard control circuit 360: Keyboard 362: Mouse 364: Audio processing circuit 366: Speaker 368: System port 370: Handwriting input device 372: Audio microphone 374: Video camera 376: Image scanner 410: Data Bus 412: System Bus 414: Microcomputer Interface 416: Data Interface 418: CPU 420: RAM 422: Code Detector 24: Parameter memory 426: Motion compensation unit 428: Rate control unit 430: Encoding buffer unit 432: Decoding buffer unit 434: Transform unit 436: Quantization unit 438: Variable length coding unit 440: Quantization table 442: Huffman table 444: Reference buffer 446: Bus arbiter D1, D2, D3, D4: Data bus SB: System bus

Claims (7)

[Claims] 1. A moving image input unit for inputting moving image data, a still image input unit for inputting still image data, and moving image data input by the moving image input unit,
Coding means for coding using intra-picture coding and inter-picture coding, and still picture data inputted by the still picture input means is coded by intra-picture coding of the method of the coding means, An image transmission device comprising: an output unit that inserts the encoded still image data into the encoded moving image data encoded by the encoding unit and outputs the inserted still image data. 2. The image transmission apparatus according to claim 1, wherein the encoded still image data is resolution-converted into a frame size of the moving image data and encoded. 3. The image transmission device according to claim 2, further comprising second encoding means for encoding and outputting still image data input by said still image input means. 4. The image transmission device according to claim 3, wherein the second encoding means encodes image data by a JPEG system. 5. The image transmission device according to claim 3, wherein the second encoding means encodes image data by the JBIG method. 6. The image transmission device according to claim 1, 4 or 5, wherein said encoding means encodes image data by an MPEG system. 7. The encoding means is ITU-T Recommendation H.264. Two
The image transmission device according to claim 1, wherein the image data is encoded by the 61 system.