JPH10294953A - Moving image data receiver - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image data receiver which can preserve data transmitted while utilizing inter-frame correlative relation in a data format so as to reproduce them from any arbitrary frame position. SOLUTION: Image data compressed by a TIM method as a compressing method utilizing the inter-frame correlative relation are received. When receiving image data, the moving image data receiver successively compounds respective frames (b), (c), (d)... from a key frame a' through a compound part 231 and provides frames without the inter-frame correlative relation. At an encoding part 234, key frames e'... are prepared by performing intra-frame compression to the compounded frame for each prescribed frame and stored in a random access file storage part 25 while being exchanged with respective correspondent frames (e)... before compounding. Thus, reproduction from the middle of image data is enabled while designating any arbitrary prepared key frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image compression apparatus, and more particularly to an image compression apparatus for compressing color image data and compressing and transmitting a still image or a moving image of a color image.

[0002]

2. Description of the Related Art A moving image is captured by a video camera, an MPEG camera, or the like, and the moving image data is transmitted to an image data receiving apparatus using a personal computer or the like using a wireless communication line in wireless communication means such as a mobile phone. There is an image data transmission device. Since the amount of moving image data to be transmitted by such an image data transmitting apparatus has a very large data amount, when transmitting the image data using a wireless communication line, it is necessary to transmit the image data amount as small as possible. .
As a method for reducing the amount of data, there is a data compression method using a correlation between captured frames.

FIGS. 13 and 14 conceptually show a method of compressing image data using a correlation between frames. Here, as shown in FIG. 13 and FIG. 14, each frame of each captured image data is represented by A,
B, C, D, E, F,... FIG. 13 illustrates a case where the data amount is compressed by taking a difference from the previous frame. For the first frame A,
The image data A 'which has been compressed (intra-frame compression) so that it can be reproduced within the frame is transmitted. For the second and subsequent frames B, C,..., The difference from the immediately preceding frame is obtained, and only the difference data is transmitted. That is, for the B frame, difference data (BA) from the A frame is obtained, and only this difference data is transmitted. Hereinafter, the difference data (CB), (D-
C), (ED),... Are sequentially transmitted. When the moving image is reproduced in the image data receiving apparatus which has received the compressed image data, first, A ′ compressed in the frame, which is a key frame, is reproduced to display the A frame as an image and A + (B B-frame is reproduced and displayed by -A), and B + (CB) = C, C + (D-
C) = D, D + (ED) = E,...

[0004] On the other hand, FIG. 14 shows a compression method used in MPEG or the like using the discrete cosine transform (DCT), in which key frames are transmitted at predetermined frames, and between frames of each key frame. The compressed frame is transmitted by utilizing the correlation of (1).
For example, the first frame A and the frames D,... After a predetermined frame are transmitted with intra-frame compression A ', D',..., And the frames between them are compressed using the correlation with the immediately preceding frame. Images B (A), C (B), E
(D), F (E), ... are transmitted. That is, A ', B (A)
, C (B), D ', E (D), F (E),...

[0005]

However, as described with reference to FIG. 13, when a key frame is transmitted first, and thereafter compressed data is transmitted sequentially using the correlation between frames, image data reception is performed. The device does not have any problem if the received data is reproduced at the time of reception.However, if the received data is temporarily stored in the storage device and then reproduced, since the key frame exists only at the beginning, the first screen is displayed. However, there is a problem that the image can be reproduced only from the beginning and the image cannot be reproduced from the middle. On the other hand, as described with reference to FIG. 14, when a key frame is transmitted for each predetermined frame on the transmission side of image data, the image data receiving apparatus includes:
The temporarily stored received data can be reproduced from an arbitrary key frame position. However, in this case, since it is necessary to transmit a key frame every predetermined frame, there is a problem that the amount of data to be transmitted is increased by the amount of the key frame as compared with the method shown in FIG. .

Accordingly, the present invention has been made to solve the above-mentioned problems of the prior art, and it is possible to reproduce data transmitted using a correlation between frames from an arbitrary frame position. It is an object of the present invention to provide an image data receiving device capable of storing data in a possible data format.

[0007]

According to the first aspect of the present invention, there is provided a moving image data acquiring means for acquiring moving image data composed of a plurality of frames compressed by utilizing a correlation with a previous frame, and a method for acquiring the moving image data. Reproducing means for sequentially reproducing the moving image data obtained by the means in accordance with the correlation with the previous frame, at least into frames in a state where there is no correlation with the previous frame; and a frame for every predetermined number of frames reproduced by the reproducing means. A moving image data receiving device includes a key frame, and a storage unit for storing each key frame and a frame excluding at least a frame corresponding to the key frame in the moving image data acquired by the moving image data acquiring unit. To achieve the above object. According to a second aspect of the present invention, in the moving image data receiving apparatus according to the first aspect, the moving image data acquiring unit acquires moving image data transmitted by a wireless communication unit such as a mobile phone. According to a third aspect of the present invention, in the moving image data receiving apparatus according to the first or second aspect, the moving image data acquiring means for acquiring moving image data, and each frame of the moving image data acquired by the moving image data acquiring means. And a block generation unit that generates block data of n × m pixels from a plurality of pieces of color information representing the block from the block data generated by the block generation unit;
Encoding means for generating a block component consisting of an n × m bitmap indicating a component distribution of color information; and for the block component generated by the encoding means, a block in the frame and a corresponding block in the immediately preceding frame Encoding means for selecting a similar block whose block component is the same or most similar, and encoding using code data designating the selected similar block and differential data from the block component of the similar block. The moving image data obtaining means obtains the moving image data compressed by the data compression device. According to a fourth aspect of the present invention, in the moving image data receiving apparatus according to any one of the first to third aspects, the display means for displaying an image and the image data stored in the storage means are provided. Index icon display control means for displaying a plurality of images of the key frames as index icons on the display means, designating means for designating the index icons displayed on the display means, and corresponding to the index icons designated by the designating means. Random playback means for sequentially playing back the frames stored in the storage means after the key frame to be played, and moving image display control means for displaying on the display means an image of each frame sequentially played back by the random playback means. Are further provided.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the image data receiving apparatus of the present invention will be described below with reference to FIGS.
This will be described in detail with reference to FIG. (1) Overview of Embodiment In this embodiment, image data compressed by a TIM method described later is transmitted and received as a compression method using the correlation between frames. Then, in the image data receiving device, upon reception of the received compressed image data or after storing it in a predetermined storage device, the key data is reproduced sequentially from the first key frame once, a key frame is generated for each predetermined frame, and a random access file is created. save. As a result, it is possible to designate an arbitrary key frame that has been created and reproduce the image data from the middle. Further, since a plurality of key frames are created, images of the plurality of key frames are displayed on the screen as a list, so that the images can be used as index images for reproduction.

(2) Details of Embodiment FIG. 1 shows a system configuration of an image data communication system using the image data receiving device of the embodiment. As shown in the figure, the image data communication system includes a moving image data transmitting apparatus 1 for capturing and transmitting an image, and a moving image data receiving apparatus 2 for receiving and displaying image data captured by the moving image data transmitting apparatus 1. It is configured. The moving image data receiving device 2 is capable of remotely controlling the image capturing of the moving image data transmitting device 1. This image data communication system transmits and receives captured moving images by wireless communication such as a mobile phone. For example, monitoring and reporting of the progress of construction sites, remote monitoring of places where humans cannot always be present, fires, etc. It can be widely used as a survey tool for surveys of dams and dams, transmission of news videos taken in remote places, and directing tools for television programs and events.

The moving picture data transmitting apparatus 1 mainly includes a video camera 4, a control unit 5, an image transmitting apparatus 6, a mobile phone 7, and a modem 8. The video camera 4 can be connected to and disconnected from the image transmission device 6 with an NTSC cable. The video camera 4 constantly captures an image and outputs image data based on the NTSC signal, for example, at a rate of 3 per second.
Zero (frame) images are transmitted.
The control unit 5 can connect to and disconnect from the image transmission device 6 via an RS232C cable. The control unit 5 controls the imaging angle of the video camera (upward / downward angle = tilt, left / right angle = pan) according to a camera control command supplied from the moving image data receiving device 2 via the image transmitting device 6, an imaging range by zooming, Focus and the like are controlled.

Although not shown, the image transmission device 6 includes a CPU for performing various controls related to transmission and reception of image data,
R storing programs for various controls by PU
OM, a memory for storing image data and the like (flash memory, S-RAM, etc.), a conversion unit for converting an NTSC signal to an RGB signal, a compression unit for compressing image data, a video camera 4, a control unit 5, It is provided with various connection terminals for connecting the mobile phone 7, the modem 8, the power cable and the like, and other devices. The image transmitting device 6 receives the camera position change command from the moving image data receiving device 2 and supplies the command to the control unit 5, and converts the image data supplied from the video camera 4 from an NTSC signal to an RGB signal by a converting unit. The image data is converted by a compression unit into a TIM described later.
Compressed image data by compression. The image transmission device 6 adds the position information (pan, tilt, zoom value) of the camera and the time data relating to the date and time of imaging of the transmission frame to the header of the compressed image data, and transmits the header to the moving image data reception device 2. ing. The compressed image data is transmitted through the side of the cellular phone 7 and the modem 8 to which the communication line between the moving image data transmitting apparatus 2 is connected, and the cellular phone 7 is mainly used. ing.

The image data compression employed in the present embodiment is processed by an image compression board (printed board) disposed in the image transmission device 6, but may be compressed by software. . The image transmission device 6 has a function of encrypting the compressed image data with a predetermined encryption key in order to keep the communication contents secret, and the encrypted compressed image data is transferred to the moving image data receiving device 2.
May be transmitted.

On the other hand, the moving image data receiving apparatus 2 mainly comprises a mobile phone 9 and a personal computer 10 (hereinafter simply referred to as a personal computer 10) to which the mobile phone 9 is connected / disconnected. Mobile phone 9 and mobile phone 7
Is mainly 9600b via digital mobile communication network
Although connection is made by high-speed communication of ps, another communication method may be used. For example, ISDN (Integrat
(Ed Services Digital Network) or various data communication networks such as a normal telephone line network, the Internet, a FAX communication network, a PHS, a wireless satellite communication, and an optical communication. In addition, each mobile phone 7, 9 on the transmitting / receiving side
Also, various telephones can be used, and different types of telephones may be used. Further, regarding the communication speed between the transmitting and receiving apparatuses 1 and 2,
It may be set automatically according to the communication capabilities of the mobile phones 7, 9 and the modem 8.

FIG. 2 is a block diagram showing the TIM compression and decompression processing functions of the moving picture data transmitting apparatus 1 and the moving picture data receiving apparatus 2 of the image data communication system. The video data transmitting apparatus 1 includes a YUV acquisition unit 11 that acquires image data in which each pixel has a luminance signal Y and color difference signals U and V from the video camera 4, and a block encoding unit 12.
And a compressing unit 13 for judging and encoding the converted and intra-frame matches of the block components, and a transmitting unit 14 for transmitting the encoded data.

The block encoding unit 12 divides the image data acquired by the YUV acquiring unit 11 into blocks of a predetermined size of n × m, and a block component of the block from the YUV data of each pixel constituting the block. (A block luminance component ya, yb, a block color component u, v, and a bitmap bm). The compression unit 13 includes a current frame buffer 131 in which a block component of the current frame which is the target of the encoding process at this time is stored for one frame, and a current frame buffer 131 in which the block component of one frame before is stored for one frame. It includes a frame buffer 132 and an encoding unit 133 that performs data encoding by encoding according to the coincidence or non-coincidence of the block components.

On the other hand, the moving image data receiving device 2 includes a receiving unit 24 as moving image data acquiring means for receiving the encoded block data transmitted from the moving image data transmitting device 1.
A decompressing unit 23 for inversely converting encoded block data into block components and creating a key frame for each predetermined frame, a block decoding unit 22 for inversely converting block components into YUV pixel data, and an output unit for outputting YUV data 21. The moving image data receiving device 2 also stores a block data of each frame received by the moving image data receiving device 2 and a random access file in which a predetermined number of frames are updated to key frames created by the decompression unit 23. The storage unit 25 is provided.

The decompression unit 23 decodes the coded block data into the original block components.
1 and the current frame buffer 322 for one frame in which the decoded block components of the frame currently being decoded are sequentially stored, and one frame before supplied from the current frame buffer 322 for one frame , And a coding unit 234. The encoding unit 234
For each predetermined frame, a key frame is created by encoding a decoded block component of the current frame sequentially stored in the current frame buffer 232 within the frame.

The basic concept of image compression by the TIM method used in the image data communication system of the present embodiment is as follows. That is, in the image data communication system, upon image compression, (a) the human eye is sensitive to the luminance change but insensitive to the color change, and (b) the pixels constituting the image are appropriately blocked. The assumption that there is only one color in one block that is generally established in many natural images, (c) the fact that in a general moving image, a portion that does not change between a certain frame and the immediately preceding frame occupies a large portion, Image compression is performed based on the premise of the above.

Next, the operation of the TIM method for transmitting and receiving moving image data according to the embodiment configured as described above will be described. On the transmitting side of the image data communication system, the YUV acquisition unit 11 converts each pixel into a luminance signal Y and color difference signals U and V.
The image data of each frame represented by
(Charge Coupled Device). In the YUV acquisition unit 11, R (red), G (green),
When color information of B (blue) is input, YUV data is calculated from the color information according to the following equation 1. Note that each color information of RGB is generally represented by 8 bits, and 28 × 28 × 28 = 16,777,216 colors are represented. For this RGB, each of the converted Y, U, and V data is also represented by 8 bits. In this way, YU
The Y, U, and V data acquired by the V acquiring unit 11 are sequentially supplied to the block encoding unit 12 in frame units.

[0020]

## EQU00001 ## Y = 0.299R + 0.587G + 0.114B U = 0.564 (BY) V = 0.713 (RY)

When the image data is supplied to the block encoding unit 12, the dividing unit 121 converts the image data into an arbitrary size such as 4 × 2, 2 × 4, 8 × 4, 8 × 8 for each frame. To divide. In the present embodiment, as shown in FIG.
It is divided into blocks of 4 pixels × 4 pixels. The resolution per frame of the original image is arbitrary as long as it is an integer multiple of the division size. In the present embodiment, 640 pixels × 480 pixels per frame = 307, 20
It has a resolution of 0 pixels. Therefore, the original picture 1
The frame is divided into 160 blocks × 120 by the dividing unit 12.
The block is divided into 19,200 blocks.

Then, as shown in FIG. 4, the encoding unit 122 calculates the block luminance component y from the YUV data for 16 pixels contained in each divided block.
A block component composed of a, yb, block color components u, v, and bm (bitmap) is generated. This block component is composed of 8 bits for ya, yb, u and v and 16 bits for bm. Thereby, the dividing unit 121
At the stage of the division by the above, data composed of 8 bits × 3 components (YUV) × 16 pixels = 384 bits per block is changed to 8 bits × 4 by the block components.
Type (ya, yb, u, v) +16 bits (bitmap) = 48 bits. That is, the data is compressed to 48/384 = 1/8 by the encoding unit 122.

Here, the calculation of each block component will be described. That is, the block color components u and v are generated by an average value of the pixel color components U1 to U16 and V1 to V16 in the block. That is, u = (ΣUi) / 16, v
= (ΣVi) / 16. Here, the range i of Σ is 1 to 16. Also, block luminance components ya, y
b calculates the threshold value H from the pixel luminance components Y1 to Y16 in the block, and calculates the average value of the pixels having the luminance component Yh larger than the threshold value ya as the average value of the pixels having the luminance component Yl smaller than the threshold value h. Is defined as yb.

The selection of the threshold value H is arbitrary and can be determined by various methods. In this embodiment, the maximum value Ymax of the luminance component Y and the minimum value Ym
The average value of the 14 luminance components Y excluding in is adopted as the threshold value H. Therefore, the threshold value H and the block luminance component y
a and yb are calculated by the following equation (2). In Equation 2, p is the number of Yh, q is the number of Yl, and p + q =
Sixteen.

[0025]

H = [(ΣYi) −Ymax−Ymin] / 14 ya = (ΣYh) / pyb = (ΣYl) / q

FIG. 5 shows a specific calculation of the threshold value H and the luminance components ya and yb. As shown in FIG. 4, the pixel luminance components Y1 to Y16 in the block are
It is assumed that the values are as shown in FIG. In this case, the maximum value Ymax of the luminance component is the value “10” of Y9 and the minimum value Ymax
min is the value “1” of Y16. Therefore, Y9 and Y1
When the average value of the luminance components other than 6 is calculated, (83−
10-1) /14=5.14. Therefore, the threshold value H =
5.14.

As shown by the shaded portion in FIG. 5B, Y1, Y5, Y5
6, 7, Y9, Y10, Y13, and Y14 correspond to each other, and from the average value, ya = 77.1. In addition, the luminance component Yl smaller than the threshold value H has Y
2, Y3, Y4, Y7, Y8, Y11, Y12, Y1
5 and 9 of Y16, and from the average value, yb =
3.22. Then, as shown in FIG.
The values of Yh are uniformly regarded as block luminance components ya and represented by code “1”, while the values of nine Yl are uniformly regarded as block luminance components yb and represented by code “0”, whereby 2
Create a valued bm. That is, bm = (“10
00110011001100 ").

The block encoder 12 supplies the block components (ya, yb, u, v, bm) of each block generated by the encoder 122 to the compressor 13. After moving the data (block component of one frame before) already stored in the current frame buffer 131 to the previous frame buffer 132, the compression unit 13 converts the block components supplied from the block encoding unit 12 into the current frame buffer 131. To be stored. And
The encoding unit 133 performs data encoding of the block component of the block to be encoded (the latest block) between the previous frame block (Pf) and the peripheral block of the current frame according to the block component matching condition. .

FIG. 6 is a diagram for explaining the concept of data encoding based on block component matching conditions. In this figure, a case will be described where the block in the third row and third column of the current frame, which is indicated by oblique lines, is encoded as a processing block Cb. For the processing block Cb, the previous frame block stored at the same position of the previous frame buffer 132 at the 3'-th row and the 3'-th column is defined as Pf. In the current frame buffer 131, the block in the second row and third column, which has been subjected to the encoding process immediately before the processing block, is set to Pb. Further, with respect to the two columns immediately before, the upper left (second row and second column) of the processing block Cb is Prl, the upper right (3 rows and 2 columns) is Pr, and the upper right (4 rows and 2 columns) is Prr.

The block components of the processing block Cb and the comparison blocks Pf, Pb, Prl, Pr, P
The rr block component is compared with the rr block component to find a matching block, and coding is performed by the matching block according to the coding rule shown in FIG. That is, when the processing block Cb matches the immediately preceding block Pb, the block components ya, yb, u,
v and bm (total 48 bits) are compressed into 3 bits of “100”.

On the other hand, if there is no matching block in the comparison blocks, the encoding unit 133 determines whether the block component of the processing block Cb is similar to the previous frame block Pf or the previous block Pb. Judgment is made, and coding is performed using similar blocks according to FIG. That is, a three-digit header indicating a similar block and a 5-bit map (“●●●●●”) indicating the match / mismatch of each block component ya, yb, u, v, bm.
● ”) and the component of the processing block Cb that does not match (★
★…). Here, “●●●●●” is a bit string of 0 or 1. Also, * is a numerical value indicating the actual component value of the processing block Cb, and each component value is represented by an 8-bit binary number.

For example, it is assumed that the values of both components of the immediately preceding block Pb similar to the processing block Cb are as follows. Cb; ya = 200, yb = 100, u = 128, v =
120, bm = x Pb; ya = 200, yb = 120, u = 110, v =
120, bm = x In this case, the code of the processing block Cb is as follows. That is, the header is represented by a bit string “111” indicating the immediately preceding block that is a similar block. The match / mismatch bitmap is represented by the bit string “01100” in the order of ya, yb, u, v, and bm because ya, v, and bm match and yb and u do not match. The mismatch component value of Cb is represented by a numerical value yb = 100 and a numerical value u = 128. Accordingly, the code of the processing block Cb is composed of 3 bits of header (“111”) + 5 bits of match / mismatch bitmap (“01100”) + mismatch component value (100,
128). Note that the mismatch component values 100 and 1
28 are each represented by 8 bits.

The coded data of the block component of the processing block Cb coded by the compression unit 13 has the minimum data amount when it matches the previous frame block Pf. In this case, since the data amount of one block at the time of division by the division unit 121 is 8 × 3 × 16 = 384 bits, the data is compressed to 1/384. On the other hand, the maximum value of the encoded data by the compression unit 13 is that there is no inter-frame match and all components of the processing block Cb are
This is the case where all components of Pf and Pb do not match. In this case, the amount of encoded data of the compression unit 13 is 3 + 5 +
48 = 56 bits, and the compression ratio in this case is 56/3
84 = 1 / 6.8.

With the above processing, the coded data of the block component coded by the coding unit 133 is added from the compression unit 13 to the transmission unit by adding time data indicating the imaging date and time of each frame to the header of the frame. 14. In the transmission unit 14, a telephone network, a dedicated line, wireless communication,
It is transmitted to the moving image data receiving apparatus 2 shown in FIG. 1 by various communication means such as SDN (Integrated Services Digital Network) and computer communication.

The case encoding unit 133 performs compression processing on the first frame of the captured moving image and supplies the key frame to the transmission unit 14 as a key frame. That is, in the encoding unit 133, other comparison blocks Pb, Prl, Pr, except for the previous frame block Pf from the previous frame buffer 132 shown in FIG.
By comparing the block component of Prr with the block component of the processing block Cb, coding is performed according to the coding rules shown in FIGS.

Next, the operation of receiving the block data encoded by the moving image data receiving apparatus 2 will be described with reference to FIGS. FIG. 8 shows a state of each frame in each unit of the moving image data receiving device 2. Now, as shown in FIG. 8, each frame a ', b,
It is assumed that the coded data of c, d, e,... is sequentially received. Here, it is assumed that the frame with the symbol “′” represents a key frame that has been compressed in the frame and can be decoded by itself. Receiver 24
Then, the received block data of the frames a ', b, c, d, e,... Is supplied to the decompression unit 23 and the random access file storage unit 25 in the order of reception.

The decompressing section 23 decodes the supplied block data as a processing block [Cb] and supplies it to the block decoding section 22. In addition, the decompressing section 23 creates a key frame for every predetermined number of frames and stores it in the random access file storage section. 25. In the present embodiment, 15 is adopted as the predetermined number of key frames to be generated, corresponding to the number of frames 30 per second transmitted from the moving image data transmitting apparatus 1. The random access file storage unit 25 stores the data a ', b, c, d, f,... Of each frame supplied from the receiving unit 24, and stores a frame corresponding to the key frame supplied from the decompressing unit 23. By updating, a file of the received moving image data that can be finally randomly accessed (reproduced from an arbitrary position) is stored.

FIG. 9 is for explaining the concept of data decoding based on the block component matching condition. As shown in this figure, a block (indicated by oblique lines) in the third row and third column of the current frame 232, which is to be decoded by the decoding unit 231 at present, is defined as a processing block Db. Also, the previous frame block stored in the 3′-row and 3′-column at the same position of the previous frame buffer 233 with respect to this processing block Db is set to Pf, and the current frame buffer 232 is set to a position immediately before the processing block. The block in the second row and the third column, which has been encoded in
For the second column one column before, the upper left (second row, second column) of the processing block Cb is set to the upper left block Prl, and the upper right (3 rows, 2
The column) is a block right above and the upper right (4 rows, 2 columns) is a upper right block Prr. The processing block before decoding by the decoding unit 231 is [Db], and the processing block after decoding is Db.

Now, it is assumed that the decoding of the frames a ', b, c, and d received by the receiving unit 24 has been completed, and the decoding process for the frame e and the creation process of the key frame e' have been performed. I do. In this case, the previous frame buffer 2
33 stores the immediately preceding frame d, and the current frame buffer 232 sequentially stores the decoded blocks among the blocks of the frame e. Then, the previous frame buffer 233 supplies the previous frame block Pf to the decoding unit 231, and the current frame buffer 2
32 supplies the immediately preceding block Pb, the upper left block Prl, the immediately upper block Pr, and the upper right block Prr.

The decoding section 231 converts the encoded data of the processing block [Db] supplied from the receiving section 24 into the data shown in FIG.
(A) According to the coding code allocation of (b), the previous frame block Pf, the immediately preceding block Pb, and the upper left block Pr
1, a block to be referred to is selected from among the right upper block Pr and the upper right block Prr. Then, the decoding unit 23
1 is a block component ya, y stored (already decoded) stored at a corresponding position (a position indicated by the code) if the encoded data is a code indicating inter-block coincidence.
b, u, v, bm are decoded as block components of the processing block Db. On the other hand, if the encoded data of the processing block [Db] is a code indicating inter-block mismatch, the block component of the similar block specified by the header part of the encoded data and the code in accordance with the encoding rule of FIG. From the conversion data, the block components ya, y of the processing block Db
Decoding is performed by generating b, u, v, and bm.

The decoding section 231 supplies the decoded processing block Db to the block decoding section 22 and the current frame buffer 232. Current frame buffer 2
In step 32, the processing block Db is stored at the corresponding position (the position of the third row and the third column shown by diagonal lines in FIG. 9), the processing block Db after decoding, the block Pb immediately before this processing block Db, and the upper left block. The Prl, the upper right block Pr, and the upper right block Prr are supplied to the encoding unit 234.

If the current frame being processed is a frame for which a key frame is to be created, the encoding unit 234 determines whether or not the block components of the processing block Db and the blocks of these comparison blocks Pb, Prl, Pr, and Prr are to be processed. The components are compared, and if there is a matching block, the coding rule shown in FIG. 7A is used.
Encoding is performed according to the encoding rule of (b). The encoding in the encoding unit 234 is performed by the previous frame block P
This is intra-frame compression that does not use f, and is performed in the same manner as the intra-frame compression process of creating a key frame a ′ from the first frame a in the encoding unit 133 of the moving image data transmission device 1. The block data encoded by the encoding unit 234 is supplied to the random access file storage unit 25 as block data of a key frame, and the corresponding frame (already supplied and stored from the reception unit 24) is used as the key data. Updated to the frame.

When the decoding is completed for all the blocks of the frame e, all the block data of the frame e sequentially stored in the current frame buffer 232 are supplied to the previous frame buffer 233, and the decoding of the next frame f is performed. Do.

The block decoding unit 22 converts the decoded block components ya, yb, u, v, and bm generated by the decoding unit 231 for the processing block Db into a 4 × 4 pixel YUV as shown in FIG. Generate data.
That is, assuming that the bm of the block component is the bitmap shown in FIG. 10A, as shown in FIG. 10B, the pixel whose bitmap data is “1” is represented by Y = y
a, U = u, V = v, and the pixel which is “0” is
Y = yb, U = u, and V = v. Thus, the decoded block data is one color composed of the color difference data U and V as a whole, and its luminance data Y is ya and y.
b.

The block decoding unit 22 outputs the Y, U, and V data of each pixel after decoding via the output unit 21.
The data is output as drawing data on the display screen of various display devices 150 such as a CRT, a plasma display, and a liquid crystal display device. Here, the output unit 21 is connected to the connected display device 1.
If 50 requests Y, U, V data, it is output as it is, and if it requests RGB data, R, G, B data is converted from Y, U, V data according to the following equation 3. Is calculated and output.

[0046]

## EQU3 ## B = 1.773U + Y R = 1.403V + Y G = Y− (0.564 / 0.587) U + (0.713
/0.587)V

As described above, according to the moving picture data receiving apparatus of the present embodiment, the decompressing unit performs processing on the data transmitted using the correlation between the frames for the second and subsequent frames. 23, a key frame is created from a predetermined number of received frames every 15 frames and stored in the random access file storage unit 25. Therefore, by using the moving image data receiving apparatus of the present embodiment, random access can be performed by specifying an arbitrary key frame created at one-second intervals. In addition, since key frames for each predetermined number of frames are created by the moving image data receiving device, the moving image data transmitting device 1 does not need to transmit key frames other than the first key frame a '. Therefore, when transmitting the moving image data receiving system using a wireless communication medium such as a mobile phone, the transmission data amount does not increase due to the plurality of key frames.

Further, according to the moving picture data transmitting apparatus of the moving picture data communication system using the present embodiment, the block encoding section 1 has already been encoded before being encoded by the compress section 13.
2, the data is compressed to 1/8 for each block, so that the current frame buffer 131, the previous frame buffer 1
32 buffer size can be reduced. In addition, since the coding process of the block component by the coding unit 133 and the decoding process of the coded data by the decoding unit 231 are only the detection of the inter-frame coincidence and the conversion according to the coding rule of FIG. Moving images can be transmitted and received by software processing without using special hardware that performs high-speed decoding.

FIG. 11 shows an example of a configuration in which the moving image data receiving device represented by the functional blocks in FIG. 2B is constituted by the personal computer 10 and its software. As shown in FIG. 11, the personal computer 10 includes a control unit 110 for controlling the entire apparatus. This control unit 1
10 via a bus line 120 such as a data bus.
A keyboard 130, a mouse 140, a display device 150, a storage device 160, a storage medium driving device 170, a communication control device 180, and an input / output IF 190 are connected.

The control unit 110 includes a CPU 111 functioning as a so-called computer, a ROM 112, a RAM 1
13 is provided. The ROM 112 is a read-only memory in which various data and programs are stored in advance. The RAM 113 is a random access memory used as a working memory by the CPU 111. When the personal computer 10 is used as the moving image data receiving apparatus of the present embodiment, the RAM 113 secures a current frame buffer 232, a previous frame buffer 233, a random access file area 25a, and other areas for storing data and programs. It is supposed to be.
In the random access file area 25a, when the moving image data is received, the received frame and the creation key frame are stored, and then the file is moved to the random access file storage unit 25. The access file is moved.

The keyboard 130 has various keys such as kana keys and numeric keys for inputting kana characters, function keys for executing various functions, and cursor keys. As the display device 150, for example, a CRT or a liquid crystal display is used. On this display device, a reproduced image of the received moving image is displayed, and, as described later, an index icon or the like for reproducing the received moving image data from an arbitrary key frame is displayed. . The mouse 140 is a pointing device,
This is an input device that specifies a corresponding function by left-clicking a key, icon, index icon, or the like displayed on the display device 150.

The storage device 160 is composed of a readable and writable recording medium and a drive device for reading and writing various information such as programs and data on the storage medium. As the storage medium used for the storage device 160, a hard disk is mainly used, but a readable / writable storage medium among the storage media used in the storage medium driving device 170 described later may be used. . The storage device 160 is connected to the moving image data transmitting device 1 and information on the types of the video cameras 4 that can be connected to the image transmitting device 6 and items that can be controlled by each video camera (zoom, pan, tilt range, etc.). Is stored. The random access file storage unit 25 stores a random access file supplied from the random access file area 25a.

The program storage area of the storage device 160 stores various application programs such as an image data reception related program according to the present embodiment, in addition to a basic program such as an OS for operating the personal computer 10. In the moving image data receiving device 2 of the present embodiment, the decoding unit 231 and the block decoding unit 22 reproduce a program for image data compressed by the TIM method, and generate a key frame by the encoding unit 234. And a program for reproducing a moving image of a random access file from a key frame corresponding to a display of an access frame designation screen described later and a designated index icon. When the compressed image data transmitted from the moving image data transmitting apparatus 1 is encrypted, a program for performing decryption using the corresponding key is also stored in the storage device 16.

The recording medium driving device 170 includes a CPU 111
Is a drive device for reading a computer program from an external recording medium. The computer programs recorded on the recording medium include image data reception-related application programs executed by the personal computer 10 of the present embodiment, and dictionaries and data used therein. Here, the recording medium refers to a recording medium on which a computer program is recorded, and specifically, a magnetic storage medium such as a floppy disk, a hard disk, and a magnetic tape; a semiconductor storage medium such as a memory chip and an IC card; Storage media such as ROM, MO, PD (phase change rewritable optical disk) etc. from which information can be read optically, paper cards and paper tapes, paper such as printed materials for reading programs using a character recognition device (and paper And a recording medium in which a computer program is recorded by various methods. As a recording medium used in the personal computer 10 of the present embodiment, a CD-ROM or a floppy disk is mainly used. Recording medium drive 17
In addition to reading a computer program from these various types of recording media, 0 can write data in the RAM 113 and the storage device 16 in the case of a writable recording medium such as a floppy disk.

In the personal computer 10 constituting the present moving image data receiving apparatus, the CPU 111 reads a computer program from an external recording medium set in the recording medium driving device 170 and stores (installs) it in the storage device 160. When the image data reception-related program according to the present embodiment is executed, the corresponding program is read from the storage device 160 into the RAM 113 and executed. However, not from the storage device 160,
The recording medium driving device 170 can also directly read and read the data from an external recording medium into the RAM 113 and execute the same.
Further, depending on the personal computer 10, the program related to the image data reception according to the present embodiment may be recorded in the ROM 112 in advance, and the program may be executed by the CPU 111.

The communication control device 180 has a connection terminal for the mobile phone 9 and transmits a camera position change command for remotely controlling the video camera 4.
It functions as the receiving unit 24 shown in (b) and receives compressed image data from the moving image data transmitting apparatus 1. Further, the communication control device 18 can transmit and receive various data such as text data and bitmap data to and from another personal computer or the like. The input / output IF 190 is an interface for connecting / disconnecting a printing device that prints images and characters displayed on the display device 150 and various other types, and a speaker that outputs audio and music. Printing devices include laser printers, dot printers, inkjet printers, page printers,
Various printing devices such as a thermal printer and a thermal transfer printer are used.

Next, a description will be given of a case where image data is reproduced from an arbitrary key frame using the random access file having key frames at predetermined intervals created as described above. When the random access file to be played and the random access file to be played are specified by operating the keyboard 130 or the mouse 140, the control unit 110 reads the corresponding random access file from the random access file storage unit 25, and In the random access file area 25a. Then, for each predetermined key frame,
The index icon is created by reproducing and reducing the YUV pixel data by the function of the block decoding unit 22. A plurality of index icons based on the created key frames are displayed on the display device 150 as an access frame designation screen for designating a reproduction position.

FIG. 12 shows an access frame designation screen displayed on the display device 150. This figure 1
As shown in FIG. 2, 12 index icons 151 are displayed on the access frame designation screen at a time,
At the bottom of each file, an imaging date and time 152 is displayed based on time information added to the header of each file. In the embodiment described above, a key frame is created every 15 frames and random access is possible in units of 1 second.
By displaying the index icon 151 created from the key frame every minute (every 900 frames), 12
The index icon can be displayed on one screen for the image data of the minute. Further, the frame interval or the time interval for displaying the index icon may be specified by the user at intervals of, for example, 1000 frames, 1500 frames, 2 minutes, 5 minutes, 10 minutes, and the like.

If there are 13 or more index icons 151 created from the key frames of the random access file, they are displayed on the next display screen. To display another index icon that is not displayed, the knob 153 in the vertical scroll bar displayed at the right end of the display screen is moved with the mouse 140.
Is operated downward, or the reverse display of the file icon is moved downward by operating the cursor movement key of the keyboard 130. The user moves the inverted display position of each index icon displayed on the display screen to the position of the desired index icon by operating the keyboard 130 or the mouse 140, and selects the position by left-clicking the mouse or operating the enter key / execute key. .
When an index icon is designated, the control unit 110
Reproduces each frame after the corresponding key frame by the respective functions of the encoding unit 231 and the block decoding unit 22 and sequentially displays them on the display device 150.

As described above, according to the moving image data receiving apparatus of the present embodiment, since the received image data is stored as a random access file replaced with a key frame for each predetermined frame, each key frame is converted into an index icon. The user can quickly search for and specify a reproduction start key frame while checking the image content. Further, as shown in FIG. 12, the shooting date and time 152 of the key frame is also displayed below each index icon 151, so that the user can search for the required reproduction start key frame from the shooting date and time. it can.

Although the moving image data receiving apparatus according to one embodiment of the present invention has been described above, the present invention is not limited to the described embodiment, and various modifications can be made within the scope of the invention described in each claim. It is possible to do. For example, in the above-described embodiment, the case where the encoding unit 234 creates a key frame every 15 frames has been described. However, since the key frame is an intra-frame compression, the compression ratio is low, and the key frame is stored in the random access file storage unit 25. Since the data amount increases, the user may be allowed to freely set the key frame creation interval. That is, the user may specify the number of frames or a time interval at which random access is possible. In the former case, the specified number of frames is Z
Thus, if the number of frames per second transmitted from the moving image data transmitting apparatus is W, a key frame is created every Z / W seconds, and random access at Z / W second intervals becomes possible.
In the latter case, if the specified time interval is T seconds and the number of frames per second transmitted from the moving image data transmission device is W, a key frame is created for each T × W frame.

In the embodiment described above, the receiving unit 24
Random access is created at the time of receiving moving image data, but the moving image data is temporarily stored in the random access file storage unit 25 without creating a key frame for each predetermined frame, and then the received moving image data is read out and randomized. An access moving image file may be created. In this case, the image data obtaining means obtains the image data from the random access file storage unit 25.

A key frame created by the moving image data receiving apparatus is obtained by compressing a frame in a frame (for example,
Frame e ') Although stored in the random access file storage unit 25, it may be stored without intra-frame compression.
That is, after the decoding in the decoding unit 231 is completed for all the blocks in one frame, the data in the current frame buffer 232 is stored in the previous frame buffer 233, and the data is stored as a key frame that has not been subjected to intra-frame compression. You may make it store in the random access file storage part 25 as it is. Further, the encoding unit 234 performs the intra-frame compression processing of the key frame based on the processing blocks Db, Pb, Prl, Pr, and Prr stored in the current frame buffer 232, but stores the key frame in the previous frame buffer 233. The intra-frame compression of a key frame may be performed based on each of the decoded block data.

In the moving picture data communication system described above, the block component bm is checked for coincidence or non-coincidence of peripheral blocks or the like with the 16-bit content, but the present invention is not limited to this. For example, all “1”, all “0”, upper half “1”, right half “1”, every other column “1”.
May be created in advance, and each table may be designated by a 6-bit code. As described above, since the bm of the block component is compared with each of the 64 tables represented by 6 bits, the degree of coincidence between the blocks is increased, and the data is further compressed. Even if they do not match, the 16-bit bitmap is
Since the data is represented by a bit code string, 10-bit data is compressed. In the present embodiment, each of the block components ya, yb, u, and v is represented by 8 bits. However, the present invention is not limited to this. For example, each block component is represented by 6 bits, and The number of bits may be represented by another number such as 4 bits. And y
The luminance signal and the chrominance signal may be represented by different numbers of bits, such as 8 bits or 6 bits for a and yb, and 6 bits or 5 bits for u and v.

Further, when calculating the block color components u and v from the color components U and V, in the above-described embodiment, U, v
Although the average value of V was used, the values of the most frequent color components U and V in the processing block were replaced with the block color components u and
It may be v. Also, two block luminance components ya,
As a threshold value H for distinguishing yb, a luminance component Ymax
Although the average value of the values excluding Ymin and Ymin is used, a simple average value of all luminance components Y may be used as the threshold value H, and an arbitrary value may be set according to the type of image data to be compressed. It may be.

In this embodiment, the processing block Cb
Are all block components of Pf, Pb, Prl, Pr, pr
In the case where none of r is coincident, as shown in FIG. 7B, Pf and Pb are used as similar block selection targets. However, the present invention is not limited to this, and an arbitrary block is determined in advance. A similar block may be selected between. For example, as selection targets, two of Pf and Pr
Case, three cases of Pf, Pr, and Pb may be used. Further, in the present embodiment, Pf, Pb, Prl, Pr, and prr are used as targets for determining whether or not all block components match the processing block Cb. Pf, P
b and Pr may be three targets. By reducing the number of comparison targets, processing by software can be further facilitated.

Further, in the moving picture data communication system described above, TIM compression has been described as an image compression method using the correlation between frames. However, other compression using the correlation between frames is also applicable. For example, the moving image data receiving apparatus according to the present embodiment may be used for the moving image data compressed by the compression using the difference data described in FIG. 13 or the FST compression described in Japanese Patent Application Laid-Open No. H5-222747. It may be. FST
The compression encodes the A / D converted video signal into a matrix block consisting of a plurality of pixels and a bitmap representing one of two types of reference values, and finds non-redundant information in the coded block. Identify and code and remove each block of redundancy and between frames by comparing each block to the corresponding block of the previous frame, and furthermore, the current color value is calculated as the difference between the previous color value and the previous color value. The amount of information is compressed by encoding in a form. In the FST compression, for RGB data, for a block composed of n × m bits, a bitmap (bm) indicating a luminance value and which of two colors A color or B color indicates each pixel. Generate a block component consisting of Then, the generated block component is compared with the previous block and the immediately preceding frame and encoded according to a predetermined encoding rule by a method similar to the encoding unit in the present embodiment.

[0068]

According to the present invention, moving image data composed of a plurality of frames compressed by utilizing the correlation with the previous frame is obtained, and the obtained moving image data is converted into at least the previous image in accordance with the correlation with the previous frame. The frames are sequentially reproduced in a frame having no correlation with the frames, and the key frames of the reproduced predetermined number of frames and the frames of the obtained moving image data excluding at least the frames corresponding to the key frames are stored in the storage unit. Therefore, data transmitted using the correlation between frames can be stored in a data format that can be reproduced from an arbitrary frame position.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of a moving image data communication system using a moving image data receiving device according to an embodiment of the present invention.

FIG. 2 is a functional block diagram showing the functions of TIM compression and decompression processing by the moving image data transmitting device and the moving image data receiving device of the moving image data communication system in blocks.

FIG. 3 is an explanatory diagram of a case where image data is divided into blocks of 4 × 4 pixels in the moving image data transmitting apparatus.

FIG. 4 shows a block component ya, yb,
FIG. 9 is an explanatory diagram when u, v, and bm are generated.

FIG. 5 is an explanatory diagram illustrating generation of luminance components ya and yb and a bitmap bm in a device encoding unit for transmitting moving image data.

FIG. 6 is a conceptual explanatory diagram of data encoding based on a block component matching condition in the encoding unit of the moving image data transmitting apparatus.

FIG. 7 is a diagram illustrating an encoding rule of a block component in an encoding unit of the moving image data transmission device.

FIG. 8 is an explanatory diagram showing a state of each frame in each functional block unit of the moving image data receiving device.

FIG. 9 is a conceptual explanatory diagram of decoding in the decoding unit of the moving image data receiving device.

FIG. 10 shows a block component ya, yb, u, v, bm in the block decoding unit of the moving image data receiving apparatus.
FIG. 4 is an explanatory diagram in the case of generating 4 × 4 pixel YUV data from.

FIG. 11 is a configuration diagram when the moving image data receiving apparatus is configured by a personal computer and its software.

FIG. 12 is an explanatory diagram showing an access frame designation screen displayed on the display device of the moving image data receiving device.

FIG. 13 conceptually illustrates a conventional method of compressing image data using a correlation between frames, and is an explanatory diagram in a case where a data amount is compressed by taking a difference from a previous frame. .

FIG. 14 conceptually illustrates a conventional method of compressing image data using a correlation between frames, in which a key frame is transmitted every predetermined frame, and a frame is transmitted for each frame between the key frames. FIG. 6 is an explanatory diagram of a case where compression is performed using a correlation between the two.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 moving image data transmitting device 2 moving image data receiving device 4 video camera 5 control unit 6 image transmitting device 7, 9 mobile phone 8 modem 10 personal computer 11 YUV acquisition unit 12 block encoding unit 121 division unit 122 encoding unit 13 compression unit 131 Frame buffer 132 Previous frame buffer 133 Encoding unit 14 Transmission unit 21 Output unit 22 Block decoding unit 23 Decompression unit 231 Decoding unit 232 Current frame buffer 233 Previous frame buffer 234 Encoding unit 24 Receiving unit 25 Random access file storage unit 110 Control Unit 111 CPU 112 ROM 113 RAM 130 keyboard 140 mouse 150 display device 160 storage device 170 storage medium drive device 180 communication control device 190 input / output IF

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Toshihiro Mori 2--19-12 Sotokanda, Chiyoda-ku, Tokyo Inside Equos Research Co., Ltd. (72) Inventor Hiroki Ishikawa 2--19 Sotokanda, Chiyoda-ku, Tokyo 12 Equos Research Co., Ltd.

Claims (4)

[Claims] 1. A moving image data acquiring means for acquiring moving image data composed of a plurality of frames compressed using a correlation with a previous frame, and moving image data acquired by the moving image data acquiring means
A reproducing means for sequentially reproducing at least frames having no correlation with the previous frame in accordance with the correlation with the previous frame; and a frame every predetermined number of frames reproduced by the reproducing means as a key frame; And a storage unit for storing at least a frame excluding a frame corresponding to the key frame in the moving image data acquired by the moving image data acquiring unit. 2. The moving image data receiving device according to claim 1, wherein the moving image data obtaining unit obtains the moving image data transmitted by a wireless communication unit such as a mobile phone. 3. A moving image data obtaining means for obtaining moving image data, a block generating unit for generating nxm pixel block data from each frame of the moving image data obtained by the moving image data obtaining means, From the block data generated in
Encoding means for generating a block component composed of a plurality of pieces of color information representing the block and an n × m bitmap indicating a component distribution of the color information; and for the block component generated by the encoding means, A similar block having the same or most similar block component is selected from the block and the corresponding block in the immediately preceding frame, and code is specified using code data specifying the selected similar block and difference data from the block component of the similar block. The moving image data receiving device according to claim 1 or 2, wherein the moving image data obtaining unit obtains moving image data compressed by a moving image data compression device having an encoding unit that converts the moving image data. 4. A display means for displaying an image, an index icon display control means for displaying a plurality of key frame images stored in the storage means as index icons on the display means, and an index displayed on the display means. Designation means for designating an icon; random reproduction means for sequentially reproducing frames stored in the storage means after the key frame corresponding to the index icon designated by the specification means; and sequential reproduction by the random reproduction means 4. A moving image display control means for displaying an image of each frame to be displayed on the display means.
The moving image data receiving device according to claim 1.