JP2003150141A - Device and method for processing image, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accelerate the display processing speed of moving image data. SOLUTION: Processing for storing moving image data compressed by an MPEG 4 in an input buffer 36a is performed parallel with processing for converting moving image data converted into YUV signals stored in an intermediate buffer 36b into RGB signals and storing these data in an output buffer 36c and further, processing for converting the moving image data compressed by the MPEG 4 stored in the input buffer 36a into YUV signals and storing these data in the intermediate buffer 36b is performed parallel with processing for displaying the moving image data converted into the RGB signals stored in the output buffer 36c on an LCD.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, and a program, and more particularly, to an image processing apparatus and method capable of improving an image frame rate by increasing an image display speed. And about the program.

[0002]

2. Description of the Related Art A technique for reproducing and displaying moving image data is becoming popular. The moving image data is compressed (encoded) by a predetermined compression method, recorded on a recording medium, or distributed by a transmission medium.

Generally, in a so-called VCR that reproduces moving image data, a CPU (Central Processing Unit) is used.
The moving image data recorded on the recording medium by the RAM (R
once stored in the andom Access Memory), and then the DSP (Digital Si)
gnal Processor). Next, the DSP decodes the transferred moving image data into YUV signals (Y: luminance signal, U: difference signal between luminance signal and red component, V: difference signal between luminance signal and blue component), and further, YUV signal. RGB signal color space
(Red Green Blue) signal is converted to LCD (Liquid Crys
tal Display).

Conventionally, moving image data is displayed on an LCD by performing various processes as shown in the timing chart of FIG. That is, as shown in FIG. 1B, from time t0 to t1, the CPU copies (transfers) one frame of the moving image data recorded on the recording medium to the DSP. Next, as shown in FIG. 1A, at the timing from time t1 to t2, the DSP decodes the read one frame of moving image data into a YUV signal. Further, from time t2 to t3, the DSP performs color space conversion of the YUV signal into an RGB signal suitable for display on the display. After that, as shown in FIG. 1B, from time t3 to t4, the CPU copies (transfers) the RGB signal to the LCD and displays moving image data for one frame.

Further, at times t4 to t5, the CPU
Copies (transfers) the image data for the next one frame of the moving image data recorded on the recording medium to the DSP, and thereafter repeats the same processing as described above. That is, as shown in FIG. 1A, at the timing from time t5 to t6, the DSP
Decodes the read one frame of moving image data into a YUV signal. From time t6 to t7, DSP is YUV
Color space conversion of signals into RGB signals suitable for display on a display. After that, as shown in FIG. 1B, from time t7 to t8, the CPU copies (transfers) the RGB signal to the LCD and displays moving image data for one frame.

Here, FIG. 1A shows the processing timing of the DSP, and FIG. 1B shows the processing timing of the CPU.

As described above, the processing in which the CPU reads moving image data into the DSP in frame units, the DSP decodes the moving image data, performs RGB conversion, and further transfers to the LCD for display by the CPU is repeated. By doing so, the moving image data
Displayed on LCD.

[0008]

However, in the above-mentioned configuration, the DSP is not available while the CPU processing is being executed.
No processing is being executed, and conversely, while DSP processing is being executed, CPU processing is not being executed. In this way, each processing is performed alternately, so CPU and DSP
However, there is a problem that the processing efficiency is low.

The present invention has been made in view of such a situation, and controls the mutual processing timings of the CPU and DSP to perform parallel processing, thereby increasing the display speed of moving image data. The frame rate of moving image data is improved.

[0010]

A first image processing apparatus of the present invention comprises a first storage means for storing image data encoded by a predetermined method, and a first storage means for storing the image data. Decoding means for decoding image data encoded by a predetermined method, second storage means for storing image data decoded by the decoding means, and decoded image stored by the second storage means Conversion means for converting data corresponding to the display device, third storage means for storing the image data converted for the display device by the conversion means, and display device stored by the third storage means The image data converted in accordance with the above is transferred to a display device and displayed by a transfer means and a decoding means, which are encoded by a predetermined method stored by the first storage means. The image data stored in the third storage unit and converted in correspondence with the display device at the timing of decoding the stored image data,
And a timing control means for controlling the transfer means so that the data is transferred to the display device and displayed.

In the timing control means, the decoding means stores the image data stored in the first storage means and stored in the third storage means at the timing of decoding the image data encoded by the predetermined method. In addition to the fact that the image data of one frame before, which has been converted corresponding to the display device, can be transferred to the display device and the transfer means can be controlled so as to display the image data, the third storage means converts the data. The first storage unit may be controlled to store the image data of one frame after the image data converted by the unit is stored.

A first image processing method according to the present invention uses a first storing step for storing image data encoded by a predetermined method and a predetermined method stored by the processing of the first storing step. A decoding step of decoding the encoded image data, a second storage step of storing the image data decoded in the processing of the decoding step, and a decoded image stored in the processing of the second storage step A conversion step of converting the data corresponding to the display device; a third storage step of storing the image data converted corresponding to the display device in the processing of the conversion step; and a storage step of the third storage step. Further, the processing of the transfer step of transferring the image data converted corresponding to the display device to the display device for display and the decoding step is the first step. The image data converted in correspondence with the display device, which is stored in the process of the third storage step, is stored at the timing of decoding the image data encoded by the predetermined method stored in the process of the storage step. , A timing control step for controlling the processing of the transfer step so that the data is transferred to a display device and displayed.

According to a first program of the present invention, a first storage control step for controlling storage of image data encoded by a predetermined method, and storage control by the processing of the first storage control step, A decoding control step for controlling decoding of image data encoded by a predetermined method, a second storage control step for controlling storage of image data whose decoding is controlled by the processing of the decoding control step, and a second storage A conversion control step for controlling the conversion of the decoded image data corresponding to the display device whose storage is controlled by the processing of the control step, and an image whose conversion is controlled for the display device by the conversion control step. A third storage control step for controlling storage of data, and conversion corresponding to the display device whose storage is controlled by the processing of the third storage control step Transfer of the stored image data to the display device, the transfer control step of controlling the display, and the decoding control step are encoded by a predetermined method whose storage is controlled by the processing of the first storage control step. Transferring to the display device and display of the image data converted in correspondence with the display device whose storage is controlled by the process of the third storage control step at the timing of controlling the decoding of the image data being displayed. And a timing control step for controlling the processing of the transfer control step so as to control the computer.

The second image processing apparatus of the present invention stores the image data encoded by the predetermined method in the first area of the data buffer by the first storage means and the first storage means. Decoding means for decoding the image data encoded by the predetermined method, second storage means for storing the image data decoded by the decoding means in the second area of the data buffer, and the second storage means.
Converting means for converting the decoded image data stored by the storage means in correspondence with the display device, and the image data converted by the converting means in correspondence with the display device in the third area of the data buffer. A third storage unit for storing the image data, a transfer unit for transferring the image data stored in the third storage unit, which has been converted corresponding to the display device, to the display device, and displaying the image data, and a decoding unit. At the timing of decoding the image data encoded by the predetermined method stored by the first storage means, the image data converted by the display device, which is stored by the third storage means, is displayed. And timing control means for controlling the transfer means so as to transfer to the device and display.

In the timing control means, the decoding means stores the image data stored in the first storage means and stored in the third storage means at the timing of decoding the image data encoded by the predetermined method. In addition to the fact that the image data of one frame before, which has been converted corresponding to the display device, can be transferred to the display device and the transfer means can be controlled so as to display the image data, the third storage means converts the data. The first storage unit may be controlled to store the image data of one frame after the image data converted by the unit is stored.

According to a second image processing method of the present invention, a first storing step of storing image data encoded by a predetermined method in a first area of a data buffer, and a first storing step.
The decoding step of decoding the image data encoded by the predetermined method stored in the processing of the storage step, and the image data decoded in the processing of the decoding step are stored in the second area of the data buffer. Second storage step, a conversion step for converting the decoded image data stored in the processing of the second storage step corresponding to the display device, and a conversion step corresponding to the display device in the processing of the conversion step. A third storage step of storing the image data stored in the third area of the data buffer;
The image data converted in correspondence with the display device, which is stored in the process of the third storage step, is transferred to the display device and displayed, and the process of the decoding step is performed by the process of the first storage step. When the image data encoded by the predetermined method stored in the process is decoded, the image data stored in the process of the third storage step is stored.
And a timing control step for controlling the processing of the transfer step so that the image data converted corresponding to the display device is transferred to the display device and displayed.

A second program of the present invention comprises a first storage control step for controlling storage of image data encoded by a predetermined method in a first area of a data buffer, and a first storage control. A decoding control step of controlling decoding of image data encoded by a predetermined method, the storage of which is controlled by the processing of step, and a second data buffer of the image data of which decoding is controlled by the processing of the decoding control step. Storage control step for controlling the storage in the area of the display, and a conversion control step for controlling the conversion corresponding to the display device of the decoded image data whose storage is controlled by the processing of the second storage control step. And controlling the storage of the image data for which the conversion corresponding to the display device is controlled in the processing of the conversion control step in the third area of the data buffer. The third storage control step of, third
The processing of the transfer control step of controlling the display, the transfer of the image data converted in correspondence with the display device, the storage of which is controlled by the processing of the storage control step of The conversion corresponding to the display device stored in the process of the third storage step is controlled at the timing of decoding the image data encoded by the predetermined method stored in the process of the first storage step. It is characterized in that the computer is made to execute the transfer of the image data to the display device and the timing control step for controlling the processing of the transfer control step for controlling the display.

A first image processing apparatus and method of the present invention,
In the program, the image data encoded by a predetermined method is stored, the stored image data encoded by the predetermined method is decoded, the decoded image data is stored, and stored. The decoded image data is converted corresponding to the display device, the converted image data corresponding to the display device is stored, and the stored image data converted corresponding to the display device is displayed. The stored image data that has been converted in correspondence with the display device is transferred to the device, displayed, and stored at a timing at which the stored image data is decoded in the display device. Controlled to be transferred and displayed.

A second image processing apparatus and method of the present invention,
In the program, the image data encoded by the predetermined method is stored in the first area of the data buffer, the stored image data encoded by the predetermined method is decoded, and the decoded image is obtained. Data is stored in the second area of the data buffer, the stored decoded image data is converted corresponding to the display device, and the image data converted corresponding to the display device is converted into the data buffer. The image data stored in the third area and converted and stored in correspondence with the display device is transferred to the display device, displayed, and stored in a predetermined format. At the timing of decoding, the stored image data converted corresponding to the display device is transferred to the display device and displayed. It is controlled so as to.

[0020]

1 is a diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention. CPU (Central
Processing Unit 1 is, for example, a MC manufactured by Motorola
68 (trademark) series, SH (trademark) series manufactured by Hitachi, VR series manufactured by NEC Corporation, or Inte
It is a central processing unit such as the Pentium (registered trademark) series manufactured by l company, which controls the operation of the entire image processing device.
Application programs stored in the storage unit 12 via the U-bus 6 and a basic OS (Operating System)
To actually execute. Further, when starting reproduction of moving image data recorded in the semiconductor memory 9 and compressed (encoded) in the MPEG (Moving Picture Experts Group) 4 format, the CPU 1 passes through the semiconductor memory drive 8 and the semiconductor memory controller 7. Read continuously, RAM (R
andom-Access Memory) 3 and store it. Further, the CPU 1 supplies the image data stored in the RAM 3 to the DSP 4 frame by frame at a predetermined timing to perform a decoding process and an RGB conversion process, and then the moving image data converted into an RGB signal is displayed on the LCD controller. Output to 10 and display on LCD 11. Also, CPU1 is drive 1
Magnetic disk 111 and optical disk 11 mounted on 01
2. Magneto-optical disk 113 or semiconductor memory 11
The program and data recorded in 4 are read out, and a predetermined program and data are recorded.

A ROM (Read-Only Memory) 3 generally stores basically fixed data of a program used by the CPU 1 and parameters for calculation. RAM3 is
Generally, DRAM (Dynamic Random Access Memory) or SDRAM (Synchronous Dynamic Random Access Memor)
y) and stores a program used in the execution of the CPU 1 and parameters that change appropriately in the execution.

The DSP 4 is an MP which is read from the RAM 3 as needed.
The moving image data compressed (encoded) by the EG4 method is decompressed (decoded) and decoded into a YUV signal.
Further, the DSP 4 LC decodes the moving image data decoded into YUV signals.
Convert to RGB signal corresponding to D11 display. DSP4 is
For example, C54x (TM) from Texas Instruments
Series or C55x (trademark) series. The configuration of the DSP 4 will be described later with reference to FIG.

The audio processor 5 processes audio data corresponding to reproduced moving image data, generates an audio signal, and outputs it from a speaker (not shown). Note that audio data processing software may be installed in the storage unit 12 and an audio signal may be generated by the audio data processing software. In this case, the audio processor 5 may not be provided.

The semiconductor memory controller 7 reads data (for example, the above-described moving image data) recorded in a specific area of the semiconductor memory 9 mounted on the semiconductor memory drive 8 based on a command from the CPU 1, Or you can write it in. Semiconductor memory 9
Is a recording medium made of a non-volatile semiconductor, and is, for example, a memory stick (trademark) manufactured by the present applicant, Sony Corporation.

The LCD controller 10 temporarily stores the moving image data supplied from the DSP 4 by the CPU 1 in a built-in VRAM (Video RAM) (not shown) based on a command from the CPU 1 and outputs it to the LCD 11. , Display. Incidentally, FIG.
In the embodiment of the image processing apparatus shown in, the LCD controller 10 and the CPU 1 are shown as independent configurations, but the LCD controller 10 may be built in the CPU 1. In this case, a predetermined area of the RAM 3 may be used to substitute the VRAM. Furthermore, in this embodiment, an example in which the LCD 11 is used as a display device will be described, but the present invention is not limited to this, and the RG
Anything that can display the moving image data of the B signal,
For example, it may be a CRT (Cathode Ray Tube) or the like.

The storage unit 12 is, for example, an HDD (Hard Disc D
rive) etc., and writes and stores various programs and data based on commands from the CPU 1.
Or read it. The input unit 13 includes, for example, a keyboard and a mouse, and is operated when the user inputs various commands to the image processing apparatus, and outputs a signal corresponding to the operation content to the CPU 1.

Next, the configuration of the DSP 4 will be described with reference to FIG.

The input / output unit 31 is controlled by the control unit 32 and manages input / output of signals transmitted / received via the CPU bus 6. Further, the input / output unit 31 stores the moving image data compressed (encoded) by the MPEG4 system, which is transferred from the RAM 3, in the input buffer 36a of the data buffer 36 and is also stored in the output buffer 36c. The moving image data converted into the RGB signal is read out and output to the LCD controller 10.

The control unit 32 is a so-called microcomputer including a CPU, RAM and ROM, and a DSP
4 controls the entire operation. The decoder 33 is controlled by the control unit 32, decodes moving image data compressed (encoded) by the MPEG4 method and stored in the input buffer 36a, and converts the moving image data into a YUV signal,
It is stored in the intermediate buffer 36b. The RGB conversion unit 34
The YUV signal controlled by the control unit 32 and stored in the intermediate buffer 36b is converted into an RGB signal, and the output buffer 36c
To memorize.

The control register 35 is composed of a plurality of registers for controlling various operations, and includes an interrupt control register 35a and an output buffer overwrite control register 35.
b and the decode control register 35c.

The interrupt register 35a is a register for managing the operation state of the interrupt process executed by the CPU 1. When the interrupt process is one type, the control unit 32 of the DSP 4 controls the input / output unit 31 to cause the CPU 1 to operate. When requesting the interrupt processing, the interrupt control register is set to 1 at the same time. That is, in this state, interrupt processing
The state requested by the CPU 1 will be indicated. Further, when the CPU 1 receives the interrupt request, it sends a signal for controlling (clearing) the interrupt control register to 0 to the DSP 4 as a response, and sets the interrupt control register 35a to 0 in response to the response. By this processing, it can be confirmed that the interrupt processing is recognized by the CPU 1. As a result, it is shown that the next interrupt process can be requested. When there are two types of interrupt processing, the control unit 32 of the DSP 4 controls the input / output unit 31 to control the CPU.
When the interrupt processing is requested to 1, the interrupt control register is configured to operate by 2 bits at the same time, and the state in which the interrupt processing is not requested is set to 00, and when the first interrupt signal is requested, it is set to 01. , May be set to 10 when the second interrupt signal is requested. When two types of interrupt signals are used, a register for identifying the interrupt signal may be provided, or a register may be provided for each type of interrupt signal in terms of hardware.

Output buffer overwrite control register 35b
Is set to 1 when overwriting is prohibited for the output buffer 36c of the data buffer 36, and is set to 0 when overwriting is possible. As a result, depending on the setting state of the output buffer overwrite control register 35b, RG
The operation of the B conversion unit 34 is controlled.

The decode control register 35c is used by the control unit 3
It is controlled by 2 and is set to 1 when decoding is started based on a command from the CPU 1, and is cleared to 0 when decoding is completed. As a result, the operating state of the decoder 33 is indicated.

The data buffer 36 is the input buffer 36.
The moving image data encoded by the MPEG4 system, which is input from the RAM 3 is stored in a. The intermediate buffer 36b is
The decoder 33 stores the YUV signal obtained by decoding the moving image data stored in the input buffer 36a. Further, the output buffer 36c stores an RGB signal that is color space converted from the YUV signal stored in the intermediate buffer 36b by the RGB conversion unit 34. In this way, the data buffer 36 continuously moves and stores a predetermined area on the data buffer 36 depending on the conversion state of moving image data. Then, finally, the RGB stored in the output buffer 36c
The signal is output to the LCD controller 10 via the input / output unit 31 and displayed on the LCD 11.

The above DSP 4 can be configured as the above hardware, but can also be configured as software by executing each function by a program.

Next, with reference to the flowchart of FIG. 4 and the timing chart of FIG. 5, a process in which the CPU 1 and the DSP 4 read the moving image data recorded in the semiconductor memory 9 and reproduce the moving image will be described. Here, FIG. 5A shows DSP4.
5B is a timing chart of each processing of the CPU 1.

In step S1, the CPU 1 controls the semiconductor memory controller 7 so that the MPEG memory recorded in the semiconductor memory 9 mounted on the semiconductor memory drive 8 is recorded.
Start reading video data compressed in 4 format,
The data is sequentially stored in the RAM 3, the dummy data for one frame is stored in the output buffer 36c of the data buffer 36 of the DSP 4, and a command to set all the registers of the DSP 4 to 0 is sent. In response to this, the control unit 32 of the DSP 4 controls the input / output unit 31 to receive this signal and set all the registers of the control register 35 to 0 based on the instruction of the CPU 1 (processing of the DSP 4 Is not shown).

In step S2, the CPU 1 sends a command to set the output buffer overwrite control register 35b of the DSP 4 to 1, and sets the overwrite disable state. At this time, DS
In P4, the control unit 32 causes the input / output unit 31 via the bus 37.
To receive this command, and set the output buffer overwrite control register 31b to 1, that is, the overwrite inhibit state. In the following description, the operation of the DSP 4 executed based on the command of the CPU 1 is actually D
Although the processing is executed in SP4, the control unit 32 only controls the input / output unit 31 to receive this signal and execute the processing based on the instruction of the CPU 1 as described above.
Since the process is mainly performed by the CPU 1, the following description will be given as the process of the CPU 1 and the description of the process of the DSP 4 will be omitted. Also, the display of the process is omitted on the flowchart.

In step S3, the times t20 to t21 (or time t2 in the timing chart of FIG. 5B).
4 to t25, or times t28 to t29), the CPU 1 causes the input buffer 36a of the DSP 4 to store one frame of the moving image data compressed in the MPEG4 format stored in the RAM3. (Make it copied). Further, in step S4, the CPU 1 causes the DSP
The decoding control register 35c of No. 4 is set to 1, that is, the decoding start state is set.

At this time, in the DSP 4, in step S21, the control unit 32 determines whether or not the decode control register 35c is set to 1, that is, whether or not the decode start state is set, and the decode start state is set. This process is repeated until it is set. For example, step S
By the processing of 4, the decoding control register 3 by the CPU 1
If 5c is set to the decoding start state, the process proceeds to step S22.

In step S22, the time t21 to t23 (or time t in the timing chart of FIG. 5A).
25 to t27, or time t29 to t31), the control unit 32 controls the decoding control register 35
C is controlled to clear the setting to 0, that is, the decoding start state is released, and the decoder 33 is controlled to decode the moving image data compressed by the MPEG4 system stored in the input buffer 36a. Then, a YUV signal is generated and stored (copied) in the intermediate buffer 36b.

At this time, in step S5, the CPU 1 executes time t21 to t2 in the timing chart of FIG. 5B.
2 (or time t25 to t26, or time t
29 to t30), the RG stored in the output buffer 36c as the decoding result of the previous frame.
Output (copy) the B signal to the LCD controller 10 and LC
Display it on D11.

In step S6, the CPU 1 sets the output buffer overwriting control register 35b to 0, that is, the overwritable state.

In step S23, the controller 32
It is determined whether the output buffer overwrite control register 35b is set to 0, that is, whether it is set to the overwritable state, and the process is repeated until the overwritable state is set. As shown in the timing chart of FIG. 5B, when the overwritable state is set at time t22 (or time t26 or time t30) when the display of one frame on the LCD 11 is finished, the process is , And proceeds to step S24.

In step S24, the time t23 to t24 (or the time t in the timing chart of FIG. 5A).
27 to t28, or time t31 to t32), the control unit 32 controls the RGB conversion unit 34 so that the moving image data for one frame including the YUV signal stored in the intermediate buffer 36b is obtained. It is converted into an RGB signal that can be displayed on the LCD 11 and stored in the output buffer 36c.

In step S25, the control section 32
The interrupt control register 35a is controlled to be set to 1, that is, the interrupt processing is requested to the CPU 1, and the processing is
The process returns to step S21 and the subsequent processing is repeated.

In step S7, the CPU 1 refers to the RAM 3 to determine whether or not the frame data of the next moving image data (moving image data of an unprocessed frame) exists, and if it does not exist. , The process ends.
If it is determined in step S7 that the next frame data (moving image data of an unprocessed frame) exists, the CPU 1 refers to the interrupt control register 35a in step S8 to determine whether or not interrupt processing is requested. Is determined and the processing is repeated until it is determined that the interrupt processing is requested. For example, step S
When the interrupt control register 35a is set to 1 by the processing of 25 and it is determined that the interrupt processing is requested, the processing proceeds to step S9.

In step S9, the CPU 1 sets the interrupt control register 35a to 0, indicating that the interrupt request is confirmed, and the process proceeds to step S2.
The subsequent processing is repeated.

In the process of step S5, since the decoding result of the previous frame does not exist only for the first process, the dummy data stored in the process of step S1 is output to the LCD 11. Will be displayed. The dummy data may be, for example, data for displaying the entire surface in a single color (blue or the like), so that the display screen does not feel uncomfortable. Further, only the moving image data of the first frame may not be displayed on the LCD 11 (time t21 to t22 in FIG. 5B).
It may be possible to omit only the processing of).

As described above, since the interrupt processing is requested at the time when the RGB conversion processing is completed (step S25), the decoding processing which is the processing of step S5 (for example, the time shown in the timing chart of FIG. 5A). t2
1 to t23, t25 to t27, or t29 to 31) and a process of displaying moving image data on the LCD 11 which is the process of step S22 (for example, times t21 to t22 and t25 to t2 shown in the timing chart of FIG. 5B).
6 or t29 to 30) are executed in parallel, and the processing time is shortened as a whole. As a result, the moving image can be displayed at high speed, and the frame rate of the moving image data can be improved.

In the above description, when the DSP 4 finishes the RGB conversion processing, the DSP 4 displays the moving image data converted into the RGB signal of the previous frame on the LCD 11 from the output buffer 36c via the LCD controller 10. Decoding process and LCD by outputting interrupt request to CPU1
Although the example in which the display processing of the moving image data to 11 is performed in parallel has been described, the moving image data compressed in the MPEG4 format stored in the RAM3 is stored in the input buffer 36a when the decoding processing is completed. By requesting the interrupt of writing process, RGB conversion process and RAM3 to DS
By executing the process of copying the moving image data to the input buffer 36a of P4 in parallel, the processing time can be further shortened.

Therefore, next, referring to the flow chart of FIG. 6 and the timing chart of FIG. 7, the CPU 1 and the DSP 4 when the interrupt processing request is made into two types are made by the semiconductor memory 9
A process of reading out the moving image data recorded in and reproducing the moving image will be described. In the following, the two types of interrupt signals are the interrupt signals for the process of writing the moving image data compressed in the MPEG4 format stored in the RAM 3 into the input buffer 36a so that they can be identified, Also, the moving image data converted to the RGB signal of the previous frame is output from the output buffer 36c
The interrupt signal for the process of displaying on the LCD 11 via the controller 10 is referred to as an interrupt signal.

In step S41, the CPU 1 controls the semiconductor memory controller 7 to control the MP stored in the semiconductor memory 9 mounted on the semiconductor memory drive 8.
The reading of moving image data compressed in the EG4 format is started and sequentially stored in the RAM 3.

In step S42, the CPU 1 causes the DSP 4
The moving image data compressed in the MPEG4 format is stored in the input buffer 36a. In step S43, CPU1
Sets the decode control register 35c of the DSP 4 to 1.

In step S44, the CPU 1 determines whether or not the interrupt is set, and repeats the processing until the interrupt is set. In step S44, for example, when the interrupt signal is set, it is determined that the interrupt signal is input, and the process is step S45.
Proceed to.

In step S45, the CPU 1 causes the DSP 4
With reference to the interrupt control register 35a, the interrupt signal is acquired, and the value is further cleared to 00. In step S46, the CPU 1 determines whether the interrupt signal is an interrupt signal. In this case, since the interrupt signal is set, the interrupt signal is determined to be the interrupt signal, and the process proceeds to step S47.

In step S47, the CPU 1
As shown at B, as shown at times t40 to 41, DS
MP stored in RAM3 in the input buffer 36a of P4
Stores (copies) one frame of moving image data compressed in EG format. In step S48 (in this case, at time t41), the CPU 1 sets the decode control register 35c to 1, that is, the decode start state.

In step S71, the control unit 32 of the DSP 4 refers to the decode control register 35c to determine whether the decode start state has been set, and repeats until the decode start state is reached. For example, in step S46, the CPU 1 sets the decode control register 35c to 1
If it is determined that the decoding start state is set, the process proceeds to step S72.

In step S72, the control section 32
The decode control register 35c is set to 0, that is, cleared, and the decoder 33 is controlled to be compressed in the MPEG4 format stored in the input buffer 36a as shown at times t41 to t42 in FIG. 7A. One frame of moving image data is decoded, that is, moving image data compressed in the MPEG4 format is decompressed to generate a YUV signal and output (stored) to the intermediate buffer 36b. In step S73, the control unit 32
Controls the interrupt control register 35a to set an interrupt signal.

In step S74, the control section 32
0 by referring to the output buffer overwrite control register 35b
Is determined, that is, whether overwriting is set or not, and the process is repeated until the overwriting is set. For example, in the first process, step S
Output buffer overwrite control register 35b
Is set to 0, that is, the overwritable state is set, and thus the process proceeds to step S75.

In step S75, the control section 32
As shown at times t41 to t43 in FIG. 7A, the RGB conversion unit 34 is controlled to convert the YUV signal stored in the intermediate buffer 36b into an RGB signal, which is stored (copied) in the output buffer 36c and output. The buffer overwrite control register 35b is set to 1 to set the overwrite disable state.

In step S76, the control section 32
(For example, in the present case, at time t44), the interrupt control register 35a is controlled to set the interrupt signal, the process returns to the process of step S71, and the subsequent processes are repeated.

In step S49, the CPU 1 determines whether or not there is unprocessed moving image data, and if it is determined that the unprocessed moving image data exists, the process returns to step S44.

In step S45, the CPU 1 refers to the interrupt control register 35a, obtains the interrupt signal number, and clears the interrupt control register 35a. At this time, in step S46, since the interrupt signal is determined to be the interrupt signal, the process proceeds to step S47.

At step S47, the CPU 1 executes the process shown in FIG.
As shown by B, as shown at times t42 to 43, DS
MP stored in RAM3 in the input buffer 36a of P4
Stores (copies) one frame of moving image data compressed in EG4 format. In step S48,
(At this time, at time t42), the CPU 1 sets the decoding control register 35c to 1, that is, the decoding start state.

In step S71, the control unit 32 of the DSP 4 refers to the decode control register 35c, determines whether or not the decode start state is set, and repeats until the decode start state is reached. In this case, step S48
Since it is determined by the CPU 1 that the decode control register 35c has been set to 1 and the decode start state has been set, the process proceeds to step S72.

At step S72, the control section 32
The decode control register 35c is set to 0, that is, cleared, and the decoder 33 is controlled to be compressed in the MPEG4 format stored in the input buffer 36a as shown at times t44 to t46 in FIG. 7A. One frame of moving image data is decoded, that is, moving image data compressed in the MPEG4 format is decompressed to generate a YUV signal and stored in the intermediate buffer 36b. In step S73, the controller 32 controls the interrupt control register 35a again to set the interrupt signal.

On the other hand, for example, at time t44 in FIG. 7A, when the interrupt signal is set in the interrupt control register 35a by the processing of step S76 (for example, when the interrupt control register 35a is set to 10), step S50. In, it is determined that the interrupt is set, and in step S45, the CPU 1 acquires the interrupt signal as the interrupt signal of the interrupt control register 35a and sets the interrupt control register 35a to 0.
In this case, in step S46, it is determined that the interrupt signal is not, that is, the interrupt signal, and the process proceeds to step S50.

In step S50, the CPU 1 sends the RGB signal of the moving image data for one frame stored in the output buffer 36c to the LCD controller 10 through the LCD controller 10.
11 is displayed. In step S51, CPU1
Sets the output buffer overwrite control register 35b to 0, that is, to the overwritable state, the process proceeds to step S49, and the subsequent processes are repeated.

By the above processing, as shown in FIG. 7, RGB conversion processing of the decoded one frame of moving image data (time t42 to t44, t46 to t in FIG. 7A) is performed.
48, or the processing from t50 to t52) and the processing to store the moving image data compressed in the MPEG4 format for one frame in the input buffer 36a (time t42 to t in FIG. 7B).
43, t46 to t47, or t50 to 51) are processed in parallel and the moving image data compressed in the MPEG4 format is decoded (time t44 to t46 or t48 to t50 in FIG. 7A). ) And the processing of displaying one frame of moving image data that has been subjected to RGB conversion on the LCD 11 (time t44 to t45 or t48 to t49 in FIG. 7B) are processed in parallel.
The processing time as a whole is further shortened as compared with the processing described with reference to the flowchart of FIG. 4 and the timing chart of FIG. As a result, the moving image can be displayed at a higher speed, and the frame rate of the moving image data can be further improved.

In the above example, the case where the compression format of the moving image data is MPEG4 has been described, but moving image data compressed in a format for decoding processing in frame units may be used. MPEG1, 2 and Moti
It may be on JPEG. Although the semiconductor memory 9 has been described as an example of the recording medium on which the moving image data is recorded, any other recording medium may be used as long as it can record the moving image data, for example, a magnetic disk, an optical disc, Alternatively, it may be a magneto-optical disk or the like.

The series of processes described above can be executed not only by hardware but also by software. When a series of processes is executed by software, various functions can be executed by installing a computer in which a program configuring the software is incorporated in dedicated hardware or various programs. It is installed from a recording medium into a possible general-purpose personal computer or the like.

As shown in FIG. 2, this recording medium is provided to the user in a state where it is installed in the image processing apparatus in advance.
Not only the storage unit 12 in which the program is recorded,
A magnetic disk 111 (including a flexible disk) on which a program is recorded and an optical disk 112 (CD-ROM (Compact Disk-Read Only Memor) which are distributed to provide a program to a user separately from a computer.
y), DVD (including Digital Versatile Disk), magneto-optical disk 113 (including MD (Mini-Disc) (registered trademark)), or semiconductor memory 114 (including Memory Stick) It

In the present specification, the steps for writing the program recorded on the recording medium include, of course, the processing performed in a time series in the order described, but need not necessarily be performed in a time series. , Which include processes executed in parallel or individually.

[0075]

According to the first image processing apparatus and method and the program of the present invention, the image data encoded by the predetermined method is stored, and the stored image data encoded by the predetermined method is stored. , The decoded image data is stored, the stored, decoded image data is color space converted, the color space converted image data is stored, and the stored color space converted image data is stored. The stored color-space-converted image data is transferred to the display device and displayed at the timing of decoding the image data that has been transferred to the display device, displayed, and stored, and encoded by a predetermined method. I tried to control it.

A second image processing apparatus and method of the present invention,
According to the program, the image data encoded by the predetermined method is stored in the first area of the data buffer, the stored image data encoded by the predetermined method is decoded, and the decoded image data Is stored in the second area of the data buffer, the stored decoded image data is color space converted, and the color space converted image data is stored in the third area of the data buffer and stored. The image data that has been subjected to color space conversion is transferred to a display device, is displayed, and the stored image data that has been subjected to color space conversion is stored at the timing of decoding the stored image data that has been encoded by a predetermined method. It was transferred to a display device and controlled to be displayed.

In either case, as a result, the entire processing time is shortened, the moving image can be displayed at a higher speed, and the frame rate of the moving image data can be further improved. It will be possible.

[Brief description of drawings]

FIG. 1 is a timing chart showing the operation timings of a conventional CPU and DSP.

FIG. 2 is a block diagram illustrating a configuration of an embodiment of an image processing apparatus to which the present invention has been applied.

FIG. 3 is a block diagram showing the configuration of the DSP of FIG.

FIG. 4 is a flowchart illustrating an image reproduction process.

5 is a timing chart showing the operation timings of the CPU and DSP of FIG. 1 during image reproduction processing.

FIG. 6 is a flowchart illustrating an image reproduction process when the number of interrupt processes is increased.

7 is a timing chart showing operation timings of the CPU and DSP of FIG. 1 during image reproduction processing when the number of interrupt processings is increased.

[Explanation of symbols]

1 CPU, 2 ROM, 3 RAM, 4 DSP, 7 semiconductor memory controller, 8 semiconductor memory drive, 9 semiconductor memory, 10 LCD controller, 11 LCD, 31 input / output section, 32 control section, 33 RGB conversion section, 34 decoder, 35 control register, 35a interrupt control register, 35b output buffer overwrite control register, 35c
Decode control register, 36 data control buffer,
36a input buffer, 36b intermediate buffer, 36c
Output buffer

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/92 G09G 5/00 555A 7/24 H04N 5/92 H (72) Inventor Hirotaka Kondo Shinagawa, Tokyo 6-7-35 Kita-Shinagawa, Sony F-terms in Sony Corporation (reference) 5C006 AF01 AF26 BB11 BC16 FA11 5C052 AA17 AB04 CC11 DD08 EE03 GA06 GA07 GB06 GB07 GC05 GE04 GF02 GF03 5C053 FA27 GB37 JA07 KA03 KA24 LA06 5C026 KK11 SS0059 KK11 UA05 UA32 UA35 UA36 UA38 5C082 AA02 BA41 BB15 BB25 BB29 BB44 CB01 DA26 DA51 DA54 DA64 DA87 MM02

Claims (8)

[Claims] 1. An image processing apparatus for displaying image data encoded by a predetermined method on a display device, comprising: first storage means for storing image data encoded by the predetermined method; Decoding means for decoding the image data encoded by the predetermined method stored by the storage means, second storage means for storing the image data decoded by the decoding means, and the second storage means. A conversion unit that converts the decoded image data stored by the storage unit corresponding to the display device, and a third unit that stores the image data converted by the conversion unit corresponding to the display device Storage means and the image data stored in the third storage means and converted in correspondence with the display device, Transfer means for transferring and displaying the image data on the display device, and the decoding means at a timing at which the image data encoded by the predetermined method stored in the first storage means is decoded by the third storage means. The image data stored by the above, which is converted corresponding to the display device, is transferred to the display device, and timing control means for controlling the transfer means to display the image is provided. Processing equipment. 2. The timing control means causes the third storage means to operate at a timing at which the decoding means decodes image data encoded by a predetermined method and stored in the first storage means. Stored, converted corresponding to the display device,
In addition to transferring the image data of one frame before to the display device and controlling the transfer means to display the image data, the third storage means is converted by the conversion means in correspondence with the display device. The image processing apparatus according to claim 1, wherein the first storage unit is controlled so as to store the image data after one frame at a timing at which the image data is stored. 3. An image processing method of an image processing apparatus for displaying image data encoded by a predetermined method on a display device, comprising a first storage step of storing the image data encoded by the predetermined method. A decoding step of decoding the image data encoded by the predetermined method stored in the processing of the first storing step, and a second storing step of storing the image data decoded in the processing of the decoding step. A storage step; a conversion step of converting the decoded image data stored in the processing of the second storage step corresponding to the display device; and a conversion step of converting the image data corresponding to the display device. Stored in the processing of the third storage step, and a third storage step of storing the image data converted by Further, the transfer step of transferring the image data converted corresponding to the display device to the display device for display, and the process of the decoding step are stored in the process of the first storage step. At the timing of decoding the image data encoded by a predetermined method, the image data converted in correspondence with the display device, which is stored in the process of the third storage step, is stored in the display device. And a timing control step of controlling the processing of the transfer step so as to transfer and display the image. 4. A first storage for controlling storage of image data encoded by the predetermined method in a computer controlling an image processing device for displaying image data encoded by the predetermined method on a display device. A control step, a decoding control step for controlling the decoding of the image data encoded by the predetermined method, the storage of which is controlled by the processing of the first storage control step; and the decoding of the decoding control step. A second storage control step for controlling storage of the controlled image data; and conversion of the decoded image data, the storage of which is controlled by the processing of the second storage control step, corresponding to the display device. A conversion control step for controlling the conversion, and before the conversion is controlled corresponding to the display device in the processing of the conversion control step. Third of controlling storage of the image data
Controlling the storage of the image data, the storage of which has been controlled by the processing of the third storage control step, which has been converted corresponding to the display device, to the display device, and to control the display. The transfer control step and the processing of the decoding control step control the decoding of the image data encoded by a predetermined method whose storage is controlled in the processing of the first storage control step The transfer of the image data converted in correspondence with the display device, the storage of which is controlled in the process of the storage control step, to the display device and the process of the transfer control step so as to control the display. A program for executing a timing control step for controlling. 5. An image processing apparatus having a data buffer for displaying image data encoded by a predetermined method on a display device, wherein the image data encoded by the predetermined method is
First storage means for storing in a first area of the data buffer; decoding means for decoding the image data encoded by the predetermined method stored by the first storage means; and the decoding means. Second storage means for storing the image data decoded by the second area of the data buffer, and the decoded image data stored by the second storage means in the display device. Converting means for converting correspondingly; third storing means for storing the image data converted corresponding to the display device by the converting means in a third area of the data buffer; Transfer means for transferring, to the display device, the image data stored in the storage means and converted in correspondence with the display device; The conversion means converts the image data encoded by the predetermined method stored in the first storage means at a timing corresponding to the display device stored in the third storage means. And a timing control unit that controls the transfer unit to transfer the displayed image data to the display unit and display the image data. 6. The timing control means causes the third storage means to operate at a timing at which the decoding means decodes image data encoded by a predetermined method and stored in the first storage means. Stored, converted corresponding to the display device,
In addition to transferring the image data of one frame before to the display device and controlling the transfer means to display the image data, the third storage means is converted by the conversion means in correspondence with the display device. The image processing apparatus according to claim 5, wherein the first storage unit is controlled so as to store the image data after one frame at a timing at which the image data is stored. 7. An image processing method for an image processing apparatus having a data buffer and displaying image data encoded by a predetermined method, wherein the image data encoded by the predetermined method is:
A first storage step of storing in a first area of the data buffer; a decoding step of decoding the image data encoded by the predetermined method stored in the processing of the first storage step; A second storage step of storing the image data decoded in the decoding step in a second area of the data buffer; and the decoded image stored in the second storage step. A conversion step of converting data corresponding to the display device; and a third step of storing the image data converted corresponding to the display device in the processing of the conversion step in a third area of the data buffer. A storage step, and the image data stored in the processing of the third storage step and converted in correspondence with the display device, The transfer step of transferring and displaying the image on a display device, and the processing of the decoding step, at the timing of decoding the image data encoded by the predetermined method stored in the processing of the first storage step A timing control step of controlling the processing of the transfer step so as to transfer the image data converted in correspondence with the display device stored in the processing of the storage step of No. An image processing method comprising: 8. A computer having a data buffer and controlling an image processing apparatus for displaying image data encoded by a predetermined method, comprising:
A first storage control step for controlling storage in the first area of the data buffer; and image data encoded by the predetermined method, the storage of which is controlled by the processing of the first storage control step. A decoding control step for controlling decoding; a second storage control step for controlling storage of the image data, the decoding of which is controlled by the processing of the decoding control step, in a second area of the data buffer; A conversion control step of controlling the conversion of the decoded image data, the storage of which is controlled by the processing of the storage control step 2, corresponding to the display device; and the conversion control step, which corresponds to the display device. A third storage control step of controlling storage of the conversion-controlled image data in a third area of the data buffer; Transfer of the image data converted in correspondence with the display device, the storage of which is controlled by the process of the third storage control step, to the display device, a transfer control step of controlling display, and the decoding The display stored in the process of the third storage step at a timing when the process of the control step decodes the image data encoded by the predetermined method stored in the process of the first storage step. Of the image data whose conversion corresponding to the device is controlled,
A program for executing a transfer to the display device and a timing control step for controlling processing of the transfer control step for controlling display.