WO2004004331A1

WO2004004331A1 - Image processing apparatus

Info

Publication number: WO2004004331A1
Application number: PCT/JP2003/007880
Authority: WO
Inventors: Hiroshi Sakai; Keiichi Iwasaki
Original assignee: Ricoh Company, Ltd.
Priority date: 2002-06-27
Filing date: 2003-06-20
Publication date: 2004-01-08
Also published as: JP4272388B2; JP2004032495A

Abstract

An image processing apparatus is provided able to process input image data frame by frame at a specified timing without data loss. The image processing apparatus comprises a plurality of buffer memories each storing a portion of the image data, a data storing unit configured to sequentially store portions of the image data each equaling one frame into the respective buffer memories, and a data outputting unit configured to sequentially output the frames of image data stored in the buffer memories one by one at the predetermined timing. The data outputting unit outputs the image data equaling one frame at the predetermined timing from one buffer memory, while the data storing unit stores the input image data into another buffer memory.

Description

DESCRIPTION IMAGE PROCESSING APPARATUS

TECHNICAL FIELD The present invention relates to an image processing apparatus that processes image data in the unit of frames.

BACKGROUND ART

FIG. 4 shows a configuration of an image processing apparatus 200 of the related art. In FIG. 4, encoded video data equaling a number of frames, which are obtained by shooting an object, are stored in a memory (MEM) 3. To encode the video data, each image frame is divided into a number of blocks each including a specified number of pixels arranged in a matrix manner, and image data corresponding to each block are encoded. The memory 3 stores the encoded image data in the unit of blocks.

When reproducing the video data, a CPU 1 sends a reproduction signal to a DMA 2 for reproducing the first encoded data block of the first frame. According to the reproduction signal, the DMA 2 then by-passes the CPU 1 and outputs the first encoded data block of the first frame stored in the memory 3 to a decoder (DEC) 4 through a data bus (DBUS) 7. After outputting the first encoded data block, the DMA 2 releases the data bus 7 temporarily. Subsequently, the CPU 1 sends another reproduction signal to the DMA 2 for reproducing the second encoded data block of the first frame. According to the reproduction signal, the DMA 2 by-passes the CPU 1 again, and through the data bus 7, outputs to the decoder 4 the second encoded data block of the first frame stored in the memory 3. Again, the DMA 2 releases the data bus 7 after outputting the encoded data of the second block.

In this way, the CPU 1 sends reproduction signals to the DMA 2 sequentially for reproducing the third, the fourth, and all other encoded data blocks of the first frame, all blocks of the second frame, and furthermore, all other frames to be processed, respectively.

The decoder 4 decodes the encoded data blocks to the original image data, and outputs the image data to a subsequent buffer (BUFF) 5. The buffer 5 stores the image data blocks input from the decoder 4 one by one, and outputs the image data blocks to a display (DISP) 6 in synchronization with a frame synchronization signal SYNC that is input every 1/30 second. Based on the image data, the display 6 displays images continuously to reproduce the original video data.

FIG. 5 shows a timing chart of the encoded data blocks stored in the memory 3 , the encoded data blocks input to the decoder 4 through the data bus 7 , the decoded data blocks output from the decoder 4, the frame synchronization signal SYNC, and the decoded data output from the buffer 5 upon reception of the frame synchronization signal.

As described above, in response to the request from the CPU 1 , the DMA 2 accepts control of the data bus 7 from the CPU 1 each time an image data block is to be output to the decoder 4. When executing the control change, if the data bus 7 is being used by the CPU 1 , the DMA 2 has to wait until the CPU 1 releases the data bus 7.

In FIG. 5, this waiting time is expressed with intervals between consecutive encoded data blocks input to the decoder 4 and intervals of consecutive decoded data blocks output from the decoder 4 (the image data) . As illustrated in FIG. 5, when transferring data in the above way continuously, the waiting time can accumulate to such a degree that sometimes reading of all the blocks of one frame cannot be completed within the duration of 1/30 second, the period of the frame synchronization signal SYNC. In FIG. 5, two data blocks are lost when the decoded data (image data) are output from the buffer 5, resulting in quality degradation of the reproduced images.

DISCLOSURE OF THE INVENTION

Accordingly, it is a general object of the present invention to provide an image processing apparatus to solve the above problem of the related art. A more specific object of the present invention is to provide an image processing apparatus that processes image data at a specified timing in the unit of frames, in particular, the image processing apparatus is able to sequentially process the image data at the specified timing without data loss.

To attain the above object, according to a first aspect of the present invention, there is provided an image processing apparatus adapted to process image data having a plurality of frames and sequentially output the image data in the unit of frames at a predetermined timing, comprising a plurality of buffer memories each storing a portion of the input image data, a data storing unit configured to sequentially store portions of the input image data each equaling one frame into the respective buffer memories, and a data outputting unit configured to sequentially output the image data stored in the buffer memories one by one at the predetermined timing, wherein the data outputting unit outputs the image data equaling one frame at the predetermined timing from one of the buffer memories containing the image data, while the data storing unit is storing the input image data equaling one frame into one of the buffer memories.

Preferably, in the image processing apparatus, each of the buffer memories is configured to be able to store image data equaling one frame. Preferably, the image processing apparatus further comprises a decoder that receives encoded image data, decodes the encoded image data into the image data, and outputs the image data to the data storing unit, and an output device that receives the image data output from the data outputting unit at the predetermined timing in the unit of frames.

Alternatively, the image processing apparatus further comprises a decoder that receives encoded image data and decodes the encoded image data into decoded image data, and an output device that receives the decoded image data output from the decoder in the unit of frames , wherein the image data input to the data storing unit and the image data output from the data outputting unit are encoded image data, and the decoder receives the encoded image data output from the data outputting unit at the predetermined timing in the unit of frames, decodes the encoded image data into image data, and outputs the image data to the output device.

According to the above aspect of the present invention, a plurality of buffer memories are provided. The data storing unit sequentially stores the image data into the respective buffer memories in the unit of frames, and the data outputting unit sequentially outputs the frames of image data stored in the buffer memories one by one at the predetermined timing while the data storing unit is storing the image data. Because there are a number of buffer memories, the data outputting process and the data buffering process can be performed in parallel instead of in series as in the related art. Specifically, while the data storing unit is storing image data to one buffer memory, the data outputting unit can output the image data previously stored in another buffer memory. Therefore, during data transfer, even if the image data stored presently have a time length somewhat longer than the specified time period related to the specified timing, the time length of the previously stored image data is always made shorter than the specified time period, and hence the previously stored image can be totally transferred without data loss to the next stage, for example, an output device or a decoder . To attain the above object, according to a second aspect of the present invention, there is provided an image processing method for processing image data having a plurality of frames and sequentially outputting the image data in the unit of frames at a predetermined timing, comprising a data storing step of sequentially storing portions of the image data each equaling one frame into the respective buffer memories, and a data outputting step of sequentially outputting the image data stored in the buffer memories in the unit of frames at the predetermined timing, wherein the image data equaling one frame are output at the predetermined timing from one of the buffer memories containing the image data in the data outputting step, while the image data equaling one frame are being stored to another one of the buffer memories in the data storing step. Preferably, the image processing method further comprises, before the data storing step, a step of decoding encoded image data into the image data, and outputting the decoded image data for processing in the data storing step, and a step of outputting the image data output after the data outputting step to an output device at the predetermined timing in the unit of frames .

Alternatively, in the image processing method, the image data processed in the data storing step and the image data processed in the data outputting step are encoded image data. The image processing method further comprises a step of, after the data outputting step, receiving the encoded image data output after the data output step at the predetermined timing in the unit of frames , decoding the encoded image data into decoded image data, and outputting the image data to an output device.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features, and advantages of the present invention will become more apparent from the following detailed description of preferred embodiments given with reference to the accompanying drawings, in which:

FIG. 1 is a view of a configuration of an image processing apparatus 100 according to an embodiment of the present invention; FIG. 2 is a view of the buffer 8 in the image processing apparatus 100 according to the embodiment of the present invention;

FIG. 3 is a timing chart of signals in the image processing apparatus 100 according to the embodiment of the present invention;

FIG. 4 is a view of a configuration of an image processing apparatus 200 of the related art; and

FIG. 5 is a timing chart of signals in the image processing apparatus 200 of the related art.

BEST MODE FOR CARRYING OUT THE INVENTION

Below, preferred embodiments of the present invention will be explained with reference to the accompanying drawings . FIG. 1 is a view of a configuration of an image processing apparatus 100 according to an embodiment of the present invention. Note that like reference numerals are used for like components in FIG. 1 and FIG. 4 illustrating the image processing apparatus 200 of the related art. In FIG. 1, a data bus (DBUS) 7 is connected to a CPU 1 that controls the overall operation of the image processing apparatus 100, a memory (MEM) 3 and a decoder (DEC) 4.

Encoded video data equaling a number of frames obtained by shooting an object are stored in the memory (MEM) 3. To encode the video data, each image frame is divided into a number of blocks each including a specified number of pixels arranged in a matrix manner, and image data corresponding to each block are encoded. The memory 3 stores the encoded image data in the unit of blocks. When reproducing the video data, the CPU 1 sends a reproduction signal to a DMA 2 for reproducing the first encoded data block of the first frame. According to the reproduction signal, the DMA 2 then by-passes the CPU 1 and outputs the first encoded data block of the first frame stored in the memory 3 to the decoder (DEC) 4 through the data bus 7. After outputting the first encoded data block, the DMA 2 releases the data bus 7 temporarily.

Subsequently, the CPU 1 sends another reproduction signal to the DMA 2 for reproducing the second encoded data block of the first frame. According to the reproduction signal, the DMA 2 by-passes the CPU 1 again, and through the data bus 7, outputs to the decoder 4 the second encoded data block of the first frame stored in the memory 3. Again, the DMA 2 releases the data bus 7 after outputting the encoded data of the second block. In this way, the CPU 1 sends reproduction signals to the DMA 2 sequentially for reproducing the third, the fourth, and all other subsequent encoded data blocks of the first frame, furthermore, all encoded data blocks of the second frame and all other frames to be processed, respectively.

The decoder 4 decodes the encoded data blocks to the original image data , and outputs the image data to a buffer (BUFF) 8 sequentially.

As described below, the buffer 8 includes two buffer memories each having a storage capacity equivalent to one frame. In the buffer 8, first, one buffer memory is directed to store the decoded image data blocks ; once image data equaling one frame is stored in the buffer memory, the other buffer memory is directed to start to store the subsequent image data blocks. After the switching of the buffer memories, the image data equaling one frame stored in the former buffer memory is output to a display (DISP) 6 in synchronization with an input frame synchronization signal SYNC.

FIG. 2 is a view of the buffer 8 in the image processing apparatus 100 according to the embodiment of the present invention. In the buffer 8, the decoded data (image data) are input to a selector (SEL) 10 and a bit-counter (BIT- C) 13. The selector 10 selects the first buffer memory 12 or the second buffer memory 11 according to the level of a select signal B (SIGB) input to the selector 10. When the select signal B is at a low level ("0") , the selector 10 selects the first buffer memory 12 to store the input image data; when the select signal B is at a high level ("1") , the selector 10 selects the second buffer memory 11 to store the image data. The initial signal level of the signal B is 0.

As described below, the selector 10, the bit-counter 13, the comparator 14, the block counter 15, and the one-bit- counter 17 function as the data storing unit of the present invention for storing a series of image data frames into the first buffer memory 12 and the second buffer memory 11.

The bit-counter 13 counts the number of bits of the input image data, and outputs the count to the subsequent comparator 14.

The comparator 14 compares the input count with a preset number of bits in one block, for example, 128 x 128 bits, and when the input count becomes equal to the preset number of bits, that is, when one image data block has been input, the comparator 14 outputs a high level signal to the subsequent block counter 15. At the same time, the comparator 14 outputs the same signal to a reset terminal (R) of the bit- counter 13 to reset the count. After the bit-counter 13 is reset, the signal output from the comparator 14 is changed from the low level to the high level .

The block-counter (BLK-C) 15 counts the high level signal output from the comparator 14, and outputs the count to the subsequent comparator 16.

The comparator 16 compares the input count with a preset number of blocks in one frame and when the input count becomes equal to the preset number of blocks , that is , when image data equaling one frame have been input to the buffer memory, the comparer 16 outputs a high level signal A (SIGA) to the subsequent one-bit-counter 17 to indicate completion of input of one frame. At the same time, the comparator 16 outputs the same signal (SIGA) to a reset terminal (R) of the block-counter 15 to reset the count. After the block-counter

15 is reset, the signal output from the comparer 16 is changed from the low level to the high level .

The one-bit-counter 17 receives signals from the comparator 16 and counts the high level signals from the comparator 16. The one-bit-counter 17 outputs the count, as the select signal B (SIGB) , to the select signal input terminal of the selector 10. In other words, once image data equaling one frame is input to the first buffer memory 12 or the second buffer memory 11, the level of the signal output from the one-bit-counter 17 is switched from the high level to the low level, or from the low level to the high level, to start data input to the other buffer memory.

As described below, the one-bit-counter 17, the AND gate 18, the AND gate 19, and the delay circuit (DLY) 20 function as the data outputting unit of the present invention for outputting image data frame by frame to the subsequent display 6 at a specified timing from the first buffer memory 12 or the second buffer memory 11, in each of which a frame of image data is collected. The output signal from the one-bit-counter 17 is inverted and then input to the AND gate 18 having two input terminals, and at the same time, the same output signal from the one-bit-counter 17 is input to the AND gate 19 having two input terminals . The delay circuit 20 delays the frame synchronization signal SYNC by a specified time period (for example, half a cycle) , and outputs the delayed signal (signal C, SIGC) to the other input terminal of the AND gate 18 and the other input terminal of the AND gate 19. The frame synchronization signal SYNC is input every 1/30 second.

When the output signal from the one-bit-counter 17 is changed from the low level to the high level, that is, when image data equaling one frame have been stored in the first buffer memory 12, according to the signal C, a high level signal D (SIGD) is input to an input terminal (data reading request terminal) of the first buffer memory 12 from the AND gate 19 to request the first buffer memory 12 to read out the image data. Receiving the high level signal D, the first buffer memory 12 outputs the stored data equaling one frame to the display 6. On the other hand, when the output signal from the one-bit-counter 17 is changed from the high level to the low level, that is, when image data equaling one frame have been stored in the second buffer memory 11, according to the signal C, a high level signal E (SIGE) is input to an input terminal (data reading request terminal) of the second buffer memory 11 from the AND gate 18 to request the second buffer memory 11 to read out the image data. Receiving the high level signal E, the second buffer memory 11 outputs the stored image data equaling one frame to the display 6.

FIG. 3 is a timing chart of signals in the image processing apparatus 100 according to the embodiment of the present invention. The signals shown in FIG. 3 include the encoded data blocks stored in the memory 3 , the encoded data blocks input to the decoder 4 through the data bus 7, the decoded data blocks output from the decoder 4 (image data) , the signal A (SIGA) indicating completion of data input to one of the buffer memories, the select signal B (SIGB) , the delayed frame synchronization signal C (SIGC) , the signal D (SIGD) for requesting the first buffer memory 12 to read out the image data therein, the signal E (SIGE) for requesting the second buffer memory 11 to read out the image data therein, and the decoded data output from the buffer 8.

FIG. 3 illustrates the case in which when the data bus 7 is busy, so that storing all data blocks of one frame into the first buffer memory 12 cannot be completed within the duration of 1/30 second, the period of the frame synchronization signal SYNC.

In the buffer 8, because the two buffer memories 11 and 12 each store a frame of image data alternately, in the interval after all data blocks of one frame are completely stored in one of the buffer memories and before the frame of image data two frames later is input, the stored one frame of image data may be output to the display 6 according to the signal C that is the delayed frame synchronization signal SYNC. Therefore, as shown in FIG. 3, even though the data bus 7 is busy, and data transfer by the DMA 2 generates a time delay, loss of data blocks does not occur, and all image data blocks can be output to the display 6. In the above, a description is made of the image processing apparatus 100 including the buffer 8 between the decoder 4 and the display 6. The two buffer memories 11 and 12 may be provided inside the buffer 8 as illustrated in the above, or may be provided outside. Further, the buffer 8 may be provided between the data bus 7 and the decoder 4. In this case, in the buffer 8, the first buffer memory 12 and the second buffer memory 11 alternately store one frame of image data transmitted in the unit of blocks, and alternately output one frame of image data stored therein to the display 6 in synchronization with the delayed frame synchronization signal SYNC ( signal C) .

It is described above that the encoded data are read from the memory 3 via the data bus 7 block by block, but the present embodiment is not limited to this . The encoded data may also be read from other peripheral storage media via the data bus 7 block by block.

Further, it is not always necessary to divide the encoded data transmitted via the data bus 7 into blocks . The encoded data equaling one frame may be transmitted at once. The reason is explained below. In the present embodiment, the image processing apparatus 100 displays images frame by frame on the display 6 every 1/30 second in synchronization with the frame synchronization signal SYNC, and the buffer 8 is provided to eliminate the problem that at stages prior to the display 6 that process image data in the unit of frames, sometimes image data of a frame to be displayed cannot be collected during the 1/30 second because the data bus 7 is busy.

Furthermore, the buffer 8 may also be used in an image processing apparatus in which an encoder is used to encode image data transmitted from a camera in the unit of frames in synchronization with the frame synchronization signal SYNC, and the encoded data are stored in a storage medium such as a memory or a hard disk in real-time. In this case, the buffer 8 is provided before the storage medium and at a place where delay of the image data transmitted from the camera may occur during data transfer, for example, after the data bus .

For example, considering a case in which the image data transmitted from the camera are received by the encoder, the encoder divides an image frame into blocks each having a specified size, and the data encoded in the unit of blocks are stored in the storage medium via the data bus. In this case, the buffer 8 is provided between the data bus and the storage medium. When employing this configuration, it is possible to correctly store the encoded data of images obtained by the camera in the storage medium in real-time.

While the present invention has been described with reference to specific embodiments chosen for purpose of illustration, it should be apparent that the invention is not limited to these embodiments, but numerous modifications could be made thereto by those skilled in the art without departing from the basic concept and scope of the invention.

Summarizing the effect of the invention, according to the present invention, because the data outputting unit outputs the image data at a specified timing from one of the buffer memories containing the image data while the data storing unit is storing the input image data into another one of the buffer memories not containing the image data, during data transfer, even if transfer of the image data equaling one frame cannot be completed within the time period related to the specified timing, the image data can still be totally transferred to the output device without data loss.

The present application is based on Japanese priority application No. 2002-187550 filed on June 27, 2002, the entire contents of which are hereby incorporated by reference.

Claims

1. An image processing apparatus adapted to process image data having a plurality of frames and to sequentially output the image data in the unit of frames at a predetermined timing, comprising: a plurality of buffer memories each storing a portion of the image data; a data storing unit configured to sequentially store the portions of the image data each equaling one frame into the respective buffer memories; and a data outputting unit configured to sequentially output the frames of the image data stored in the buffer memories one by one at the predetermined timing, wherein the data outputting unit outputs the image data equaling one frame at the predetermined timing from one of the buffer memories containing the image data, while the data storing unit stores the image data equaling one frame to another one of the buffer memories.

2. The image processing apparatus as claimed in claim 1, wherein each of the buffer memories is configured to be able to store the image data equaling one frame.

3. The image processing apparatus as claimed in claim 1, further comprising: a decoder that receives encoded image data, decodes the encoded image data into the image data , and outputs the decoded image data to the data storing unit; and an output device that receives the image data output from the data outputting unit at the predetermined timing in the unit of frames .

4. The image processing apparatus as claimed in claim 1, further comprising: a decoder that receives encoded image data and decodes the encoded image data into decoded image data; and an output device that receives the decoded image data output from the decoder in the unit of frames, wherein the image data input to the data storing unit and the image data output from the data outputting unit are encoded image data; and the decoder receives the encoded image data output from the data outputting unit at the predetermined timing in the unit of frames, decodes the encoded image data into image data, and outputs the image data to the output device.

5. An image processing method for processing image data having a plurality of frames and for sequentially outputting the image data in the unit of frames at a predetermined timing, comprising: a data storing step of sequentially storing portions of the image data each equaling one frame into the respective buffer memories; and a data outputting step of sequentially outputting the image data stored in the buffer memories in the unit of frames at the predetermined timing, wherein the image data equaling one frame are output at the predetermined timing from one of the buffer memories containing the image data in the data outputting step, while the image data equaling one frame are being stored to another one of the buffer memories in the data storing step.

6. The image processing method as claimed in claim 5, further comprising: before the data storing step, a step of decoding encoded image data into the image data, and outputting the decoded image data for processing in the data storing step; and after the data outputting step, a step of outputting the image data to an output device at the predetermined timing in the unit of frames .

7. The image processing method as claimed in claim 5, wherein the image data processed in the data storing step and the image data processed in the data outputting step are encoded image data, the image processing method further comprising a step of, after the data outputting step, receiving the encoded image data output after the data output step at the predetermined timing in the unit of frames , decoding the encoded image data into image data, and outputting the image data to an output device .