JP2006229484A - Image processing method and image processing unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method capable of quickly starting display processing corresponding to a designated display mode after receiving the designation. <P>SOLUTION: In an STC generator 25, correction clock data STC are generated based on reference clock data STC_S and correction data STC_offset_c corresponding to a display mode. An image output part 29 performs the image output of frame data corresponding to PTS conforming to the STC. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method and apparatus for sequentially displaying a plurality of picture data.

A program stream defined by MPEG (Moving Picture Experts Group) or the like includes a plurality of frame data and display time data PTS indicating the display time of the frame data.
After decoding the frame data, the image decoding apparatus associates this with the display time data PTS and stores it in the buffer memory.
For example, the image decoding device compares the display time indicated by the reference clock data generated by the built-in reference clock generator with the display time indicated by the display time data PTS of the frame data stored in the buffer memory. If they match, frame data display processing corresponding to the display time data PTS is performed.
In the image decoding device, for example, in a special display mode such as slow playback, the display time data PTS is corrected according to the display mode, and the corrected display time data PTS and the frame data are associated with each other in the buffer memory. I remember it.
JP 2004-7832 A

However, in the above-described image decoding apparatus, when the special display mode is instructed, it is difficult to correct the display time data PTS for the frame data already stored in the buffer memory. The display time data PTS is corrected in accordance with the instructed display mode with respect to the display time data PTS to be written into the buffer memory after the instruction is given.
Therefore, there is a long time lag from when the display mode change instruction is issued until the display mode is actually changed.

  In order to solve the above-described problems of the prior art, the present invention provides an image processing method and apparatus capable of starting display processing according to a designated display mode in a short time after receiving the designation. Objective.

  In order to solve the above-described problems of the prior art and achieve the above-described object, an image processing method according to a first aspect of the present invention is associated with each of a plurality of picture data, and the display timing of the corresponding picture data An image processing method for sequentially performing the display processing of the plurality of picture data based on a plurality of display timing data that defines the reference timing data according to an instructed display mode. The display timing data and the correction reference timing data generated in the first step are compared, and the display timing data corresponding to the correction reference timing data is determined by the comparison. A second step of performing the display process of the picture data.

  An image processing apparatus according to a second aspect of the present invention is based on a plurality of display timing data associated with each of a plurality of picture data and defining a display timing of the corresponding picture data, respectively. An image processing apparatus that sequentially performs display processing, wherein generating means that corrects reference timing data according to an instructed display mode to generate corrected reference timing data, and the display timing data and the generating means Display processing means for comparing with the correction reference timing data and performing the display processing of the picture data corresponding to the display timing data matching the correction reference timing data by the comparison.

The operation of the image processing apparatus according to the second aspect of the invention is as follows.
The generation unit corrects the reference timing data according to the instructed display mode to generate corrected reference timing data.
Then, the display processing means compares the display timing data with the correction reference timing data generated by the generation means, and the display of the picture data corresponding to the display timing data that matches the correction reference timing data by the comparison Process.

  An image processing apparatus according to a third aspect of the invention relates to the plurality of picture data based on a plurality of display timing data associated with each of a plurality of picture data and defining a display timing of the corresponding picture data. An image processing apparatus that sequentially performs display processing, wherein reference timing generation means for generating reference timing data, correction data generation means for generating correction data according to an instructed display mode, and the reference timing generation means Generating means for generating correction reference timing data based on the reference timing data and the correction data generated by the correction data generating means; the display timing data; and the correction reference timing data generated by the generating means. And the comparison matches the correction reference timing data. And a display processing means for performing the display processing of the picture data corresponding to the display timing data.

The operation of the image processing apparatus according to the third aspect of the invention is as follows.
Reference timing generation means generates reference timing data.
Further, the correction data generation means generates correction data according to the instructed display mode.
Then, the generation unit generates correction reference timing data based on the reference timing data generated by the reference timing generation unit and the correction data generated by the correction data generation unit.
Then, the display processing means compares the display timing data with the correction reference timing data generated by the generation means, and the picture data corresponding to the display timing data that matches the correction reference timing data by the comparison. The display process is performed.

  An image processing apparatus according to a fourth aspect of the invention is an image processing apparatus that sequentially performs display processing of a plurality of picture data, and generates the plurality of picture data and the display timing data of the plurality of picture data. Display timing data generating means for generating the display timing data for each of the plurality of picture data included in the module data based on the module data including the attribute data used for the display, and in accordance with the instructed display mode Comparing the generating unit for correcting the reference timing data to generate the corrected reference timing data, the display timing data generated by the display timing data generating unit and the correction reference timing data generated by the generating unit, and comparing By means of the display data that matches the correction reference timing data. And a display processing means for performing the display processing of the picture data corresponding to the timing data.

The operation of the image processing apparatus according to the fourth aspect of the invention is as follows.
The display timing data generating means includes the plurality of picture data and module data including attribute data used to generate the display timing data of the plurality of picture data. The display timing data for each of the picture data is generated.
Further, the generation unit corrects the reference timing data according to the instructed display mode and generates corrected reference timing data.
Then, the display processing unit compares the display timing data generated by the display timing data generating unit with the correction reference timing data generated by the generation unit, and the display that matches the correction reference timing data by the comparison The display process of the picture data corresponding to the timing data is performed.

  According to the present invention, it is possible to provide an image processing method and apparatus capable of starting display processing according to a designated display mode in a short time after receiving the designation.

Hereinafter, an image processing method and apparatus according to an embodiment of the present invention will be described.
First, the correspondence between the components of the present embodiment and the components of the present invention will be described.
The frame data FR of this embodiment is an example of picture data of the present invention.
Further, the reference clock data STC_S of the present embodiment is an example of the reference timing data of the present invention, the correction clock data STC of the present embodiment is an example of the correction reference timing data of the present invention, and the correction data STC_offset_c of the present embodiment. Is an example of the correction data of the present invention.
Further, the display time data PTS of the present embodiment is an example of display timing data of the present invention.

The process shown in FIGS. 11 and 12 is an example of the first step of the present invention, and steps ST6 and ST7 shown in FIG. 10 are an example of the second step of the present invention.
Step ST4 shown in FIG. 10 is an example of the third step of the present invention.

The STC correction unit 23 and the STC generation unit 25 shown in FIG. 1 correspond to the generation means of the second and fourth aspects of the invention, and the image output unit 29 corresponds to the display processing means of the second, third, and fourth aspects of the invention. is doing.
The reference clock generation unit 21 corresponds to the reference timing generation unit of the third aspect of the invention, the STC correction unit 23 corresponds to the correction data generation unit of the invention of the third aspect, and the STC generation unit 25 corresponds to the third point. This corresponds to the generation means of the invention of this aspect.
Further, the PTS adding unit 19 corresponds to the display timing data generating means of the fourth aspect of the invention.

FIG. 1 is an overall configuration diagram of an image processing apparatus 1 according to an embodiment of the present invention.
As shown in FIG. 1, the image processing apparatus 1 includes, for example, an operation unit 11, an interface 13, a stream analysis unit 15, an image decoding unit 17, a PTS addition unit 19, a reference clock generation unit 21, an STC correction unit 23, and an STC generation. A unit 25, an image accumulation amount monitoring unit 27, and an image output unit 29.
[Interface 13]
The interface 13 inputs an encoded stream STM read from a recording medium such as a DVD, for example.
The encoded stream STM is, for example, a program stream, and is composed of a plurality of pack data PACKs as shown in FIG. 2, and each pack data PACK includes pack header data P_H, system header data S_H, and a plurality of pack data PACKs. It includes pes packet data PES_P.
The pes packet data PES_P includes pes packet header data PES_P_H and an elementary stream ES.
The elementary stream ES includes one or a plurality of encoded frame data FR.
In the present embodiment, MPEG (Moving Picture Experts Group) or the like is used as the encoding. In addition, the encoding is, for example, H.264. H.264 / AVC (Advanced Video Coding) may be used.
The pes packet header data PES_P_H includes display time data (display timing data) PTS for a plurality of frame data FR in the same elementary stream ES.
Further, the pes packet header data PES_P_H further includes attribute data PRP of a plurality of frame data FR in the same elementary stream ES.
The attribute data PRP indicates, for example, information (temporal_reference, picture_coding_type) of a plurality of frame data FR in the same elementary stream ES.

[Stream analysis unit 15]
The stream analysis unit 15 separates the frame data FR from the encoded stream STM input from the interface 13 and outputs it to the image decoding unit 17.
In addition, the stream analysis unit 15 separates information necessary for decoding such as attribute data PRP included in header data in the encoded stream STM from the encoded stream STM and outputs the separated information to the image decoding unit 17.
Further, the stream analysis unit 15 outputs the display time data PTS and the attribute data PRP in the encoded stream STM to the PTS adding unit 19.

[Image decoding unit 17]
The image decoding unit 17 decodes the frame data FR input from the stream analysis unit 15 and outputs it to the PTS adding unit 19.

[PTS addition unit 19]
The PTS adding unit 19 displays the display time of the frame data FR to which the display time data PTS is not added from the frame data FR input from the stream analyzing unit 15 based on the attribute data PRP input from the stream analyzing unit 15. Data PTS is generated.
The PTS adding unit 19 outputs the decoded frame data FR input from the image decoding unit 17 and the display time data PTS corresponding to the decoded frame data FR to the image output unit 29 and writes them in the buffer memory BUF in association with each other.

[Reference clock generator 21]
The reference clock generation unit 21 generates reference clock data (signal) STC_S of the image processing apparatus 1.
Specifically, as shown in FIG. 3 and the like, the reference clock generator 21 (within a predetermined time interval of the present invention) within a period (one picture time) in which a single frame data (picture data) is displayed. ) The reference clock data STC_S is generated so that the value increases linearly by a predetermined number (for example, 3003).
The one picture time has a frequency of 90 kHz and 3003 for NTSC and a frequency of 90 kHz and 3600 for PAL. In the present embodiment, “3003” corresponding to NTSC is exemplified as the predetermined number.

[STC correction unit 23]
Based on the operation signal from the operation unit 11, the STC correction unit 23 increases the value stepwise by the predetermined number at a time interval corresponding to the display mode mode instructed by the operation signal. For example, the correction data STC_offset_c shown in FIG. 3 and the like are generated and output to the STC generation unit 25.
Here, the initial value STC_offset_c_ini of the correction data STC_offset_c is expressed by the following equation (1).
In the following formula (1), STC_ini is an initial value of the reference clock data STC_S.
Nframe indicates a value for adjusting the timing for starting the display processing of the frame data FR.
PTS_1st indicates display time data PTS of the frame data FR that is first displayed in the encoded stream STM.

[Equation 1]
STC_offset_c_ini
= STC_ini + Nframe-PTS_1st
... (1)

  Further, the STC correction unit 23 generates correction data STC_offset_c according to the following equation (2).

[Equation 2]
STC_offset_c = STC_offset_c + offset (mode)
... (2)

  In the above equation (2), offset (mode) is a function that increases the value stepwise by the predetermined number in a time pattern corresponding to the display mode mode.

[STC generator 25]
The STC generation unit 25 subtracts the correction data STC_offset_c input from the STC correction unit 23 from the value indicated by the reference clock data STC_S input from the reference clock generation unit 21, as shown in the following formula (3). Is output to the image output unit 29.

[Equation 3]
STC = STC_S-STC_offset_c
... (3)

[Image accumulation amount monitoring unit 27]
The image accumulation amount monitoring unit 27 detects a state in which the frame data FR to be output is not stored in the buffer memory BUF of the image output unit 29 (image depletion state).
For example, the image accumulation amount monitoring unit 27 receives final output time data FOT indicating the final output time of the frame data FR in the encoded stream STM from the stream analysis unit 15.
The final output time data FOT is a parameter called E_PTM in a DVD (Digital Versatile Disk).

When the image accumulation amount monitoring unit 27 detects that the frame data FR to be subsequently displayed is not stored in the buffer memory BUF of the image output unit 29, the image accumulation amount monitoring unit 27 receives the corrected clock data STC input from the STC generation unit 25 and the stream analysis. The final output time data FOT input from the unit 15 is compared.
Then, the image accumulation amount monitoring unit 27 determines that the image is exhausted when STC> FOT by the above comparison.
The image exhaustion state occurs, for example, when the input of the encoded stream STM to the interface 13 is stopped or an unreproducible error occurs in the encoded stream STM.
Thereafter, the image depletion state can be restored to the normal state by resuming the input of the normal encoded stream STM, and the image accumulation amount monitoring unit 27 also detects the return state.
When detecting the image depletion state, the image accumulation amount monitoring unit 27 notifies the STC correction unit 23 to that effect.

[Image output unit 29]
The image output unit 29 includes a buffer memory BUF that stores the decoded frame data FR input from the PTS adding unit 19 and the display time data PTS in association with each other.
The image output unit 29 stores the correction clock data STC input from the STC generation unit 25 and the buffer memory BUF for each processing time (one picture time) of one picture data for displaying a single frame data (picture data). The stored display time data PTS of the candidate frame data FR to be displayed next is compared.
When the corrected clock data STC and the display time data PTS coincide with each other as a result of the comparison, the image output unit 29 outputs the frame data FR to a display device (not shown).
On the other hand, if PTS> STC, that is, if the current display time has not reached the display time of the frame data FR to be processed next, the image output unit 29 does not output the candidate frame data FR. That is, the image display of the frame data FR that is currently outputting an image is maintained.

The operation during special playback will be described below.
[Pause operation]
When stop (PAUSE) is instructed as the display mode mode according to the operation of the operation unit 11 by the user, the STC correction unit 23 changes the offset (mode) of the above equation (2) for each one picture time. Increment stepwise by the predetermined number.
As a result, as shown in FIG. 3, the correction data STC_offset_c is incremented by the predetermined number (3003) for each picture time according to the above equation (2), and at the timing when the comparison processing is performed in the image output unit 29. The corrected clock data STC becomes the same as the value at the time of the previous comparison processing, and the relationship of “STC <PTS” continues to hold with respect to the display time data PTS of the frame data FR to be displayed next. The frame data FR is not output as an image.
That is, the frame data FR output from the image output unit 29 is stopped.
Thereafter, when stop cancellation (PAUSE cancellation) is instructed according to the operation of the operation unit 11 by the user, the STC correction unit 23 stops incrementing the offset (mode) as shown in FIG. As a result, the correction clock data STC generated by the STC generation unit 25 increases (counts up) linearly as shown in FIG. Then, in the image output unit 29, the display time data PTS of the frame data FR to be displayed next coincides with the correction clock data STC, and at that timing, the next frame data FR is output as an image, and the display image is displayed. Updated.

[Slow playback operation]
Hereinafter, 1 / N-times slow playback will be described.
Here, N is an integer.
When 1 / N-times slow playback is instructed as the display mode mode in accordance with the operation of the operation unit 11 by the user, the image output unit 29 sets the count value of the counter-COUNTER to “1”.
The image output unit 29 increments the count value of the counter-COUNTer by 1 each time the comparison process is performed.
The counter COUNTER resets the count value to 1 when the count value reaches N in the case of 1 / N double speed slow playback.
The STC correction unit 23 increments the correction data STC_offset_c step by step for each picture time on condition that the count value of the counter COUNTER is not “1”.
The STC correction unit 23 does not increment the correction data STC_offset_c when the count value of the counter-COUNTER is “1”. That is, the correction clock data STC output from the STC generation unit 25 is linearly incremented. Thereby, the coincidence is detected by the comparison process by the image output unit 29, and the next frame data FR is output as an image.

As shown in FIG. 4, the STC correction unit 23 increments the count value of the counter-COUNTer from “1” to “2” when the 1/5 speed slow playback is instructed in the 1 × speed playback state. In a one-picture time interval, a predetermined number is incremented step by step until “5” is reached.
As a result, as shown in FIG. 4, the corrected clock data STC output from the STC generation unit 25 holds the same value at the timing of the comparison processing in the image output unit 29 for a period of four pictures, and a new frame Data FR is not output as an image.
The STC correction unit 23 resets the count value of the counter-COUNTer to “1” next to “5”, whereby the correction clock data STC output from the STC generation unit 25 is a predetermined number at 5 picture time intervals. Is incremented only by
That is, with one picture time as one cycle, the value of the correction clock data STC is held in the comparison process of four consecutive cycles, and the value of the correction clock data is updated in one cycle.
As a result, by the comparison process of the image output unit 29, the display time data PTS of the next candidate frame data FR and the correction clock data STC coincide with each other at a 5-picture time interval, and the frame data FR is output as an image. That is, 1 / 5-speed slow playback is performed.

FIG. 5 is a diagram for explaining a case where, in the 1 / 5-speed slow playback shown in FIG. 4, an instruction for 1 / 3-speed slow playback is input immediately after the count value of the counter-COUNTER becomes “5”. FIG.
In this case, as shown in FIG. 5, after an instruction for 1 / 3-speed slow playback is input, the STC correction unit 23 counts up until the counter COUNTER becomes “3” at one picture time interval. And reset to “1” after “3”.
As a result, the display time data PTS of the next candidate frame data FR and the correction clock data STC are immediately after the 1 / 3-times slow playback instruction is input by the comparison processing of the image output unit 29 at intervals of 3 pictures. Match, and the frame data FR is output as an image. In other words, 1 / 3-speed slow playback is performed immediately after an instruction for 1 / 3-speed slow playback is input.

FIG. 6 is a diagram for explaining a case where an instruction for 1 / 3-speed slow playback is input before the count value of the counter-COUNTER becomes “5” in the 1 / 5-speed slow playback shown in FIG. It is.
In this case, as shown in FIG. 6, after the instruction for 1 / 3-speed slow playback is input, the counter-COUNTer resets the count value to “1”, and then counts until it reaches “3”. Up and reset to “1” after “3”.
As a result, immediately after the instruction for 1 / 3-speed slow playback is input, the display time data PTS of the next candidate frame data FR and the correction clock data STC are compared at a 3-picture time interval by the comparison process of the image output unit 29. Match, and the frame data FR is output as an image. In other words, 1 / 3-speed slow playback is performed immediately after an instruction for 1 / 3-speed slow playback is input.

FIG. 7 is a diagram for explaining a case where an instruction for 1 / 5-speed slow playback is input during 1 / 3-speed slow playback.
In this case, the count value at the timing when the instruction for 1 / 5-speed slow playback is input is held, and the count value is incremented until it reaches “5” every picture time.
Thus, immediately after the instruction for 1 / 5-speed slow playback is input, the display time data PTS of the next candidate frame data FR and the correction clock data STC are compared at the 5-picture time interval by the comparison process of the image output unit 29. Match, and the frame data FR is output as an image. That is, immediately after the 1 / 5-speed slow playback instruction is input, 1 / 5-speed slow playback is performed.

[Frame advance]
FIG. 8 is a diagram for explaining an operation when a frame advance instruction is input after a stop (PAUSE) instruction is input.
In this case, the STC correction unit 23 increments the correction data STC_offset_c step by step by the predetermined number (3003) every picture time, as described with reference to FIG. , “STC <PTS” continues to hold, and the next frame data FR is not output as an image.
When the first frame advance instruction is input by the operation signal S11, the STC correction unit 23 stops incrementing the correction data STC_offset_c at the timing, whereby the next display candidate frame data FR is obtained from the image output unit 29. The image is output.
Thereafter, the STC correction unit 23 increments the correction data STC_offset_c by a predetermined number every picture time, and stops the image output of the frame data FR from the image output unit 29.
In the first frame advance shown in FIG. 8, there is a case where the difference between the display time data PTS is the predetermined number (3003) between the frame data FR of the next display candidate and the frame data FR being displayed. Show.
On the other hand, in the second frame advance shown in FIG. 8, the difference in the display time data PTS between the frame data FR of the next display candidate and the frame data FR being displayed is other than the predetermined number (3003). Shows the case.
In the present embodiment, the STC correction unit 23 generates correction data STC_offset_c according to the following formulas (4) and (5).
In the following formula (5), PTS. “next” indicates the display time data PTS of the frame data FR of the next display candidate.

[Equation 4]
STC_offset_c = STC_offset_c + offset (frame advance)
(4)

[Equation 5]
offset (frame advance) = STC_S-3003 (predetermined number) -PTS. next
... (5)

  In this way, by defining the offset (frame advance) based on the display time data PTS of the frame data FR of the next display candidate, the next display candidate can be appropriately displayed even for the second frame advance shown in FIG. The frame data FR can be output as an image.

[In case of underflow]
FIG. 9 is a diagram for explaining an operation example when the buffer memory BUF of the image output unit 29 shown in FIG. 1 underflows.
When detecting the underflow of the buffer memory BUF, the image accumulation amount monitoring unit 27 notifies the STC correction unit 23 to that effect.
When the notification is input, the STC correction unit 23 sets the predetermined number to the offset (mode) of the above equation (2) for each picture time until a return notification to the normal state is input from the image accumulation amount monitoring unit 27. As STC_offset_c.
As a result, the comparison processing (comparison processing between STC and PTS) of the image output unit 29 using the corrected clock data STC output from the STC generation unit 25 is continuously determined to be inconsistent, and the image output of the frame data FR is Not updated.

In the image output unit 29, image display is performed in a normal synchronization state in a state where the correction clock data STC and the display time data PTS coincide. After that, when restoration of the image exhaustion state is detected in the image accumulation amount monitoring unit 27 by restarting input of the encoded stream STM, the offset (mode) is set to “0” in the STC generation unit 25 by the restoration notification. Thus, the increment of the correction clock data STC is resumed.
Normally, when the input of the encoded stream STM is temporarily stopped and then the input is resumed, a portion subsequent to the previously input portion of the encoded stream STM is often input.
However, when the input of the encoded stream STM is resumed, underflow occurs when a portion discontinuous with the previously input portion is input.
In this case, the STC correction unit 23 uses the display time data PTS of the first frame data FR in the encoded stream STM immediately after the return as offset_restart to generate correction data STC_offset_c according to the following equation (6).
[Equation 6]
STC_offset_c = STC_S-offset_restart
(6)

[Example of overall operation of image processing apparatus 1]
Hereinafter, an example of the overall operation of the image processing apparatus 1 shown in FIG. 1 will be described.
FIG. 10 is a flowchart for explaining an overall operation example of the image processing apparatus 1 shown in FIG.
Hereinafter, each step shown in FIG. 10 will be described.
Each step shown in FIG. 10 does not need to be executed in a sequence. For example, the plurality of steps shown in FIG. 10 may perform processing on different frame data in parallel.
Step ST1:
The stream analysis unit 15 separates the frame data FR from the encoded stream STM input from the interface 13 and outputs it to the image decoding unit 17.
In addition, the stream analysis unit 15 separates information necessary for decoding such as attribute data PRP included in header data in the encoded stream STM from the encoded stream STM and outputs the separated information to the image decoding unit 17.
Further, the stream analysis unit 15 outputs the display time data PTS and the attribute data PRP in the encoded stream STM to the PTS adding unit 19.

Step ST2:
The image decoding unit 17 decodes the frame data FR input from the stream analysis unit 15 and outputs it to the PTS adding unit 19.
Step ST3:
The PTS adding unit 19 determines whether or not the display time data PTS is added to the frame data FR input from the stream analyzing unit 15. If it is determined that the display time data PTS is not added, the PTS adding unit 19 proceeds to step ST4 and determines that it is added. Then, the process proceeds to step ST5.

Step ST4:
Of the frame data FR input from the stream analysis unit 15, the PTS adding unit 19 displays the display time data of the frame data FR to which the display time data PTS is not added based on the attribute data PRP input from the stream analysis unit 15. A PTS is generated and added.
Step ST5:
The PTS adding unit 19 writes the frame data FR with the display time data PTS added to the buffer memory BUF of the image output unit 29.

Step ST6:
The image output unit 29 receives the correction clock data STC input from the STC generation unit 25 and the display time data PTS of the candidate frame data FR to be displayed next stored in the buffer memory BUF for each picture time. If they are compared and determined to match, the process proceeds to step ST7. If not, the comparison is performed again after one picture time.
Step ST7:
The image output unit 29 outputs the next candidate frame data FR to a display device (not shown).

[Example of generation operation of correction clock data STC]
11 and 12 are diagrams for explaining an operation example of generating the correction clock data STC.
Hereinafter, each step of FIG. 11 and FIG. 12 will be described.
Step ST11:
The STC correction unit 23 determines whether or not the stop operation is being performed according to the operation signal S11 from the operation unit 11. If the STC correction unit 23 determines that the stop operation is being performed, the STC correction unit 23 proceeds to step ST12. It progresses to step ST21 shown.

Step ST12:
The STC correction unit 23 determines whether or not frame advance is being performed according to the operation signal from the operation unit 11. If it is determined that frame advance is being performed, the process proceeds to step ST13, and if not, the process proceeds to step ST15. .
Step ST13:
The STC correction unit 23 determines whether or not the storage of the frame data FR in the buffer memory BUF is in a depleted state based on the notification from the image accumulation amount monitoring unit 27. If not, the process proceeds to step ST14.

Step ST14:
The STC correction unit 23 generates correction data STC_offset_c according to the above-described equations (4) and (5), and outputs this to the STC generation unit 25.
Step ST15:
The STC correction unit 23 generates correction data STC_offset_c by the method described in the stop operation described above, and outputs this to the STC generation unit 25.

Step ST16:
The STC correction unit 23 determines whether or not the storage of the frame data FR in the buffer memory BUF is in a depleted state based on the notification from the image accumulation amount monitoring unit 27. If the STC correction unit 23 determines that the storage is in a depleted state, the process proceeds to step ST17. If not, the process proceeds to step ST18.
Step ST17:
The STC correction unit 23 generates correction data STC_offset_c by the method described in the above underflow operation, and outputs this to the STC generation unit 25.
Step ST18
The STC generation unit 25 generates the correction clock data STC by the above equation (3) based on the generated correction data STC_offset_c and the reference clock data STC_S, and outputs this to the image output unit 29.

Step ST21:
The STC correction unit 23 specifies N of the 1 / N double speed slow reproduction instruction according to the operation signal from the operation unit 11.
Step ST22:
The STC correction unit 23 determines whether or not N specified in step ST21 is 1. If it is not 1, the STC correction unit 23 proceeds to step ST22. If it is determined to be 1, the process proceeds to step ST26.

Step ST23:
The STC correction unit 23 determines whether or not the count value of the counter-COUNTER is less than N. If it is determined that it is less than N, the process proceeds to step ST24. If it is determined that it is not less than N, the process proceeds to step ST26.
Step ST24:
The STC correction unit 23 generates correction data STC_offset_c by the method described in the above-described slow reproduction, and outputs this to the STC generation unit 25.
Step ST25:
The STC correction unit 23 increments the count value of the counter-COUNTer by 1.
Step ST26:
The STC correction unit 23 resets the count value of the counter COUNTER to “1”.

As described above, according to the image processing apparatus 1, not the display time data PTS but the correction data generated by the STC correction unit 23 according to the display mode (special playback mode) specified using the operation unit 11. Based on STC_offset_c, the correction clock data STC generated by the STC generation unit 25 is corrected.
Then, the image output unit 29 compares the corrected clock data STC thus corrected with the display time data PTS of the frame data FR stored in the image output unit 29, and determines whether or not to output the frame data FR as an image. Determine whether.
As a result, according to the image processing apparatus 1, it is possible to shorten the time from when the display mode is designated to when the image output corresponding to the designated display mode is started compared to the conventional case.
Further, according to the image processing apparatus 1, an error such that normal input of the encoded stream data STM is stopped by the cooperative operation of the image accumulation amount monitoring unit 27 and the STC correction unit 23 described above. Even in such an environment, it is possible to continue playback without breaking synchronization. The system detects the depletion state, and can perform reproduction while maintaining synchronization while notifying the outside of the situation, so that it is possible to give flexibility to control from the outside.
Further, according to the image processing apparatus 1, it is possible to determine whether the current output is the same as the previous timing of outputting the frame data FR or the timing of updating by the action of the STC correction unit 23 and the STC generation unit 25. The output unit 29 can determine. As a result, for example, if the same frame data FR as before is output, the Interlace image is output in the order of Top / Btm / Top / Btm, and the time is reversed at the second output time. Will occur. In order to prevent this, in the second and subsequent outputs, instead of outputting the Top / Btm image, a method is generally known in which an interpolation image is created from the Btm image and output. In the image output unit 29, the correction clock data STC from the STC generation unit 25 can be used for the determination by the image output unit 29 whether to output this interpolated image or a normal image.

The present invention is not limited to the embodiment described above.
In the above-described embodiment, the case where both the reference clock data STC_S and the correction data STC_offset_c are incremented is illustrated, but the present invention may decrement both.

  In the above-described embodiment, the frame data is exemplified as the picture data of the present invention, but other image data such as field data may be used.

  The present invention is applicable to an image processing system that sequentially displays a plurality of frame data.

FIG. 1 is an overall configuration diagram of a communication system according to a first embodiment of this invention. FIG. 2 is a diagram for explaining the format of the stream STM shown in FIG. FIG. 3 is a diagram for explaining the PAUSE operation of the image processing apparatus shown in FIG. FIG. 4 is a diagram for explaining the slow reproduction operation of the image processing apparatus shown in FIG. FIG. 5 is a diagram for explaining a case where, in the 1 / 5-speed slow playback shown in FIG. 4, an instruction for 1 / 3-speed slow playback is input immediately after the count value of the counter-COUNTER becomes “5”. FIG. FIG. 6 is a diagram for explaining a case where an instruction for 1 / 3-speed slow playback is input before the count value of the counter-COUNTER becomes “5” in the 1 / 5-speed slow playback shown in FIG. It is. FIG. 7 is a diagram for explaining a case where an instruction for 1 / 5-speed slow playback is input during 1 / 3-speed slow playback. FIG. 8 is a diagram for explaining an operation when a frame advance instruction is input after a stop (PAUSE) instruction is input. FIG. 9 is a diagram for explaining an operation example when the buffer memory BUF of the image output unit 29 shown in FIG. 1 underflows. FIG. 10 is a flowchart for explaining an example of the overall operation of the image processing apparatus 1 shown in FIG. FIG. 11 is a diagram for explaining an example of the generation operation of the correction clock data STC. FIG. 12 is a continuation diagram of FIG. 11 for explaining an example of the operation of generating the correction clock data STC.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 11 ... Operation part, 13 ... Interface, 15 ... Stream analysis part, 17 ... Image decoding part, 19 ... PTS addition part, 21 ... Reference clock generation part, 23 ... STC correction part, 25 ... STC generation 27: Image accumulation amount monitoring unit, 29: Image output unit, BUF: Buffer memory, COUNTER ... Counter, FR ... Frame data, STC ... Correction clock data, STC_S ... Reference clock data, STC_offset_c ... Correction data, PTS ... Display Time data

Claims (9)

An image processing method for sequentially performing the display processing of the plurality of picture data based on a plurality of display timing data associated with each of a plurality of picture data and defining the display timing of the corresponding picture data,
A first step of correcting the reference timing data according to the instructed display mode to generate corrected reference timing data;
The display timing data is compared with the correction reference timing data generated in the first step, and the display processing of the picture data corresponding to the display timing data that matches the correction reference timing data by the comparison is performed. An image processing method comprising: a second step. Based on module data including a plurality of the picture data and attribute data used to generate the display timing data of the plurality of picture data, each of the plurality of picture data included in the module data A third step of generating display timing data;
The image processing method according to claim 1, wherein the second step compares the display timing data generated in the third step with the correction reference timing data. The second step performs the comparison at a predetermined time interval,
In the first step, the correction reference timing is set so that the display timing data of the picture data to be displayed next and the correction reference timing data coincide with each other at a time interval corresponding to the instructed display mode. The image processing method according to claim 1, wherein data is generated. A fourth step of sequentially incrementing or decrementing the value indicated by the reference timing data so as to increase by a predetermined number at the predetermined time interval;
The first step generates correction data for incrementing or decrementing the value by the predetermined number in a time pattern according to the instruction, and the reference timing data and the correction data generated in the fourth step are generated. The image processing method according to claim 3, wherein the correction reference timing data is generated based on the image data. In the first step, when the instruction indicates 1 / N (N is an integer) double-speed slow playback, the correction is performed in continuous (N−1) periods when the predetermined time interval is one period. The image processing method according to claim 4, wherein the value of the reference timing data is held, and the value of the corrected reference timing data is incremented or decremented by the predetermined number in the subsequent one cycle. A fifth step of generating a count value that counts up to the N at the predetermined time interval and then returns to the initial value “1”;
The first step holds the value of the correction reference timing data when the count value generated in the fifth step is less than N, and when the count value becomes N, the correction reference timing data Increment the value by the predetermined number,
When an instruction to change the double speed of the 1 / N double speed slow playback is received, the count value is held when the count value generated in the fifth step is less than the N after the change, and the count value The image processing method according to claim 5, wherein the count value is reset to “1” when the value is equal to or greater than N after the change. An image processing apparatus that sequentially performs the display processing of the plurality of picture data based on a plurality of display timing data that is associated with each of a plurality of picture data and defines the display timing of the corresponding picture data,
Generating means for correcting the reference timing data according to the instructed display mode to generate corrected reference timing data;
Display processing for comparing the display timing data with the correction reference timing data generated by the generating means and performing the display processing of the picture data corresponding to the display timing data that matches the correction reference timing data by the comparison And an image processing apparatus. An image processing apparatus that sequentially performs the display processing of the plurality of picture data based on a plurality of display timing data that is associated with each of a plurality of picture data and defines the display timing of the corresponding picture data,
Reference timing generation means for generating reference timing data;
Correction data generating means for generating correction data according to the instructed display mode;
Generation means for generating correction reference timing data based on the reference timing data generated by the reference timing generation means and the correction data generated by the correction data generation means;
Display processing for comparing the display timing data with the correction reference timing data generated by the generating means and performing the display processing of the picture data corresponding to the display timing data that matches the correction reference timing data by the comparison And an image processing apparatus. An image processing apparatus that sequentially performs display processing of a plurality of picture data,
Based on module data including a plurality of the picture data and attribute data used to generate the display timing data of the plurality of picture data, each of the plurality of picture data included in the module data Display timing data generating means for generating display timing data;
Generating means for correcting the reference timing data according to the instructed display mode to generate corrected reference timing data;
The display timing data generated by the display timing data generation means is compared with the correction reference timing data generated by the generation means, and the picture corresponding to the display timing data that matches the correction reference timing data by the comparison An image processing apparatus comprising: display processing means for performing the display processing of data.

Images