WO2005112446A1

WO2005112446A1 - Image processing device

Info

Publication number: WO2005112446A1
Application number: PCT/JP2005/007750
Authority: WO
Inventors: Junya Kaku
Original assignee: Sanyo Electric Co., Ltd.
Priority date: 2004-05-17
Filing date: 2005-04-18
Publication date: 2005-11-24
Also published as: US20080008453A1; CN100563321C; JP2005328432A; CN1934859A; JP4190458B2

Abstract

Each of GOPs includes image data of frames in which the first frame in the forward direction is subjected intra-encoding and the second frame subsequent to the first frame in the forward direction is subjected to inter-encoding. A CPU (36) successively specifies the GOP blocks in the reverse direction. An MPEG4 codec (38) decodes the image data in the frames contained in the specified GOP in the forward direction. A JPEG code (40) subjects each of the image data in the frames decoded to intra-encoding in the forward direction. A CPU (36) judges whether the number of frames subjected intra-encoding is equal to more than a threshold value. When the judgment result of the CPU (36) is Yes, the JPEG codec (40) decodes the image data in the frames subjected to the intra-encoding, in the reverse direction.

Description

Specification ·

Image processing equipment

The present invention relates to an image processing apparatus, and more particularly, to an image processing apparatus that processes image data of a plurality of screens including, for example, an intra-coded screen and an inter-coded screen. Conventional technology ''

An example of this type of conventional device is disclosed in Japanese Patent Application Publication No. 2003-52020 published on Feb. 21, 2003. According to this conventional technique, when reverse playback is instructed, B-pictures and P-pictures forming an MPEG video stream are decoded in the forward direction (first time axis direction), and the decoded image data is decoded in the reverse direction. Is re-encoded along the direction (second time axis direction) into B-pictures, and a re-encoded data sequence consisting of the I-pictures forming the MPEG video stream and the re-encoded B-pictures is generated. . The MPEG video decoder decodes the re-encoded data sequence thus generated in the reverse direction, and outputs the decoded image data to the display circuit. Thereby, smooth reverse reproduction is realized. However, in the conventional technology, there is a problem that the circuit scale is increased because two MPEG decoders need to be prepared. Summary of the Invention

Therefore, a main object of the present invention is to provide a novel image processing device.

Another object of the present invention is to reduce the circuit scale, and to provide image data of a plurality of screens which have been subjected to intra-encoding and in-encoding along the first time axis direction as the first time axis direction. The object of the present invention is to provide an image processing apparatus capable of reproducing in the second time axis direction in the reverse direction.

According to claim 1, the image processing device comprises: a first time axis; A plurality of blocks each containing image data of multiple screens in which the first screen is subjected to intra coding and the screen following the first screen in the first time axis direction is inter-coded. Designation means for sequentially designating along the second time axis direction; first decoding means for sequentially decoding image data of a plurality of screens included in the block designated by the designation means along the first time axis direction; (1) an encoding means for performing the intra-encoding along the first time axis direction on each of the image data of a plurality of screens decoded by the decoding means; and the plurality of screens coded by the coding means. Second decoding means for sequentially decoding the image data in the second time axis direction.

Each of the plurality of blocks is composed of a plurality of screens in which the first screen in the first time axis direction has been subjected to the encoder coding and the screen subsequent to the first screen in the first time axis direction has been subjected to the inter-coding. Includes image data. The specifying means sequentially specifies the plurality of blocks along a second time axis direction opposite to the first time axis direction. The first decoding means decodes the image data of a plurality of screens included in the specified block in order along the first time axis direction, and the encoding means decodes each of the decoded image data of the plurality of screens. Intra coding is performed along the first time axis direction. The encoded image data of a plurality of screens is sequentially decoded by the second decoding means along the second time axis direction.

In other words, the image data decoded in the first time axis direction by the first encoding means is subjected to the encoder encoding in the first time axis direction by the encoding means. The image data that has been subjected to the intra coding is thereafter decoded in the second time axis direction by the second decoding means. By employing intra coding, it is possible to reproduce an image in the second time axis direction while suppressing the circuit scale.

According to claim 2 dependent on claim 1, the image processing device further comprises a determining means for determining whether or not the number of screens subjected to intra coding is equal to or greater than a threshold value, and the second decoding means includes a determining means. When the determination result is affirmative, the decoding process is performed.

The intra-coding process of the image data is executed along the first time axis direction, while the decoding process of the intra-coded image data is executed along the second time axis direction. Therefore, in claim 2, it is determined whether or not the number of screens subjected to intra-encoding exceeds a threshold value, and the decoding process is executed when the determination result is positive. In this way, processing failure is avoided. According to claim 3 depending on claim 1 or 2, the processing of the encoding means and the processing of the second decoding means follow the JPEG method.

According to claim 4 which depends on any of claims 1 to 3, the first time axis direction is forward and the second time axis direction is reverse.

According to claim 5, the image processing program executed by the processor of the image processing device comprises: the first screen in the first time axis direction is intra-coded and the first screen in the first time axis direction; A designation step of sequentially designating a plurality of blocks each including image data of a plurality of screens in which a screen subsequent to the screen is subjected to the inter-encoding along a second time axis direction opposite to the first time axis direction; A first decoding step of sequentially decoding the image data of a plurality of screens included in the block specified by the specifying step along the first time axis direction; a multi-screen image data decoded by the first decoding step; A coding step of performing the intra coding along the first time axis direction; and decoding the image data of a plurality of screens coded by the coding step in order along the second time axis direction Second decryption step.

Similarly to claim 1, the image data decoded in the first time axis direction by the first decoding step is subjected to the intra coding in the first time axis direction by the encoding step. The image data that has been intra-coded is then decoded in the second time axis direction in a second decoding step. By employing intra-coding, it is possible to reproduce image data in the second time axis direction while suppressing the circuit scale.

According to claim 6, the image processing device comprises: the first screen in the first time axis direction is subjected to the intra coding, and the screen following the first screen in the first time axis direction is subjected to the inter coding. A processor that sequentially specifies a plurality of blocks each including the image data of a plurality of screens along a second time axis direction opposite to the first time axis direction; a plurality of blocks included in a block specified by the processor. A decoder for sequentially decoding the image data of the screen along the first time axis direction; and a decoder for decoding the image data of the plurality of screens decoded by the decoder along the first time axis direction. Cotec that decodes image data of multiple screens that have been subjected to intra coding in order along the second time axis direction.

Each of the plurality of blocks has the first screen in the first time axis direction subjected to intra coding. The screen following the first screen in the first time axis direction includes image data of a plurality of screens that have been subjected to inter-coding. The processor sequentially designates the plurality of blocks along a second time axis direction opposite to the first time axis direction. The decoder sequentially decodes image data of a plurality of screens included in the specified block along the first time axis direction. The codec performs intra-encoding on each of the decoded image data of the plurality of screens along the first time axis direction, and converts the encoded image data of the plurality of screens along the second time axis direction. Decode in order.

As in claim 1 or 5, the image data can be reproduced in the second time axis direction while suppressing the circuit scale.

According to claim 6 which is dependent on claim 6, the processor instructs the codec to perform the encoding process and the decoding process alternately, and the codec executes the process according to the command from the processor.

The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings. Brief Description of Drawings

FIG. 1 is a block diagram showing a configuration of one embodiment of the present invention;

FIG. 2 is an illustrative view showing one example of a mapping state of the SDRAM applied to the embodiment of FIG. 1;

FIG. 3 is an illustrative view showing a part of the operation of the embodiment in FIG. 1;

FIG. 4 is an illustrative view showing another portion of the operation of the FIG. 1 embodiment;

FIG. 5A is an illustrative view showing another portion of the operation of the embodiment in FIG. 1;

FIG. 5 (B) is an illustrative view showing another part of the operation of the FIG. 1 embodiment; FIG. 5 (C) is an illustrative view showing still another part of the operation of the FIG. 1 embodiment; 5 (D) is an illustrative view showing another portion of the operation of the embodiment in FIG. 1;

FIG. 6 is a flowchart showing a part of the operation of the CPU applied to the embodiment of FIG. 1; FIG. 7 is a flowchart showing another part of the operation of the CPU applied to the embodiment of FIG.

Ό;

FIG. 8 is a front view showing another part of the operation of the CPU applied to the embodiment of FIG. And;

FIG. 9 is a front view showing still another part of the operation of the CPU applied to the embodiment of FIG. 1;

FIG. 10 is a flowchart showing another part of the operation of the CPU applied to the embodiment of FIG. 1;

FIG. 11 is a flowchart showing another part of the operation of the CPU applied to the embodiment of FIG. 1; and

FIG. 12 is a flowchart showing yet another portion of the operation of the CPU applied to the embodiment in FIG. BEST MODE FOR CARRYING OUT THE INVENTION

Referring to FIG. 1, a digital video camera 10 of this embodiment includes an optical lens 12. The optical image of the object scene is applied to the image plane of the image sensor 14 through the optical lens 12. On the imaging surface, a charge corresponding to an optical image of the object scene, that is, a raw image signal is generated by photoelectric conversion.

When the camera mode is selected by the mode key 46a provided on the key input device 46, through image processing, that is, processing for displaying a real-time moving image of the object scene on the LCD monitor 30 is executed. First, the CPU 36 instructs the driver 16 to repeat pre-exposure and thinning-out reading. The driver 16 repeatedly executes the pre-exposure of the image sensor 14 and the thinning-out reading of the raw image signal generated thereby. The pre-exposure and the thinning-out reading are executed in response to the vertical synchronization signal Vsync 1 output from the frequency divider 34 based on the clock CLK1. The vertical sync signal V sync 1 is generated once every 130 seconds, and the low-resolution raw image signal corresponding to the optical image of the object scene is output from the image sensor 14 at a frame rate of 30 fps. .

The output raw image signal of each frame is subjected to a series of processes of noise removal, level adjustment, and AZD conversion by the circuit 18 and is a digital signal. The raw image data is obtained The signal processing circuit 20 adds white balance adjustment, color separation, and white balance to the raw image data output from the CDS AGCZAD circuit 18. Performs processing such as YUV conversion to generate image data in YUV format. The generated image data of each frame is written into the SDRAM 26 by the memory control circuit 24.

The video encoder 28 requests the memory control circuit 24 to read image data in response to the vertical synchronizing signal Vsync2 output from the frequency divider 32 based on the clock CLK2. The vertical synchronizing signal Vsync2 is also generated once every 30 seconds, and the image data is transferred from the SDRAM 26 to the video encoder 28 at a frame rate of 30 fps.

The video encoder 28 converts the transferred image data into a composite video signal according to the NTSC format, and supplies the converted composite video signal to the LCD monitor 30. As a result, a through image of the object scene is displayed on the monitor screen. Although description is omitted as appropriate below, access to the SDRAM 26 is always performed through the memory control circuit 24.

When the still image shooting key 46e is operated, the CPU 36 instructs the driver 16 to perform one main exposure and one readout of all pixels. The driver 16 executes the main exposure of the image sensor 14 and the reading of all the pixels of the raw image signal generated by the main exposure once. As a result, a high-resolution raw image signal corresponding to the optical image of the object scene is output from the image sensor 14. The output raw image signal is converted into a YUV format still image data by the same processing as described above, and the converted still image data is written to the SDRAM 26.

The CPU 36 also issues a compression instruction to the JPEG codec 40. The JPEG codec 40 reads still image data from the SDRAM 26, performs JPEG compression on the read still image data, and writes the compressed still image data, that is, JPEG data, to the SDRAM 26. Thereafter, the CPU 36 reads the JPEG data from the SDRA M26, and records the JPEG file including the read JPEG data on the recording medium 44 through the IZF 42.

The recording medium 44 is a detachable semiconductor memory, and can be accessed by the IZF 42 when the recording medium 44 is mounted in a slot (not shown).

When the through image is displayed on the LCD monitor 30, When operated, the CPU 36 activates the MPEG4 codec 34. The MPEG-4 codec 34 reads the image data of each frame from the SDRAM 26 every time the vertical synchronization signal Vsync 1 is generated, and performs a compression process on the read image data in accordance with the MPEG 4 format simple profile. Apply. In the image data, intra coding is applied approximately once every 15 frames, and inter coding is applied to the remaining frames. The generated compressed moving image data, that is, MPEG data, is written to the SDRAM 26.

Intra-coded frames are defined as "I-frames", and inter-coded frames are defined as "P-frames". Also, a lump composed of an I frame and a plurality of P frames following it is defined as a "GOP (Group Of Pictures)". Then, the MPEG data has the data structure shown in FIG. Each GOP is assigned an identification number starting with "0".

The CPU 40 periodically reads out the MPEG data stored in the SDRAM 26 and records the read MPEG data on the recording medium 44 through the I / F 42. When the moving image shooting key 46 d is operated again, the CPU 36 disables the MPEG4 codec 38 and records the MPEG data remaining in the SDRAM 26 on the recording medium 44. In the recording medium 44, an MPEG file including a plurality of frames of MPEG data is created.

When the playback mode is selected by the mode key 46a and the desired JPEG file is selected by the cursor key 46c and the set key 46b, the CPU 36 stores the JPEG data stored in the JPEG file in the recording medium 44. To the SD RAM 26, and gives a decompression instruction to the JPEG codec 40. The JPEG decoder 40 reads the JPEG data from the SDRAM 26, decompresses the read JPEG data, and writes the decompressed image data to the SDRAM 26. The video encoder 28 reads the image data from the SDRAM 26 every time the vertical synchronization signal Vsync 2 is generated, converts the read image data into an NTSC C format composite video signal, and converts the converted composite video signal. Signal to the LCD monitor 30. As a result, a still image is displayed on the LCD monitor 30. When a desired MPEG file is selected by the cursor key 46c and the set key 46b in a state where the reproduction mode is selected, the reproduction processing of this MPEG file is executed. First, the CPU 36 transfers the MPEG data of the first frame stored in the MPEG file from the recording medium 44 to the SDRAM 26, and gives a decompression instruction to the MPEG4 codec 38. The MPEG codec 38 reads the MPEG data of the first frame from the SDRAM 26, decompresses the read MPEG data, and writes the decompressed image data to the SDRAM 26. The video encoder 28 performs the same processing as described above. As a result, the still image of the first frame is displayed on the LCD monitor 30.

Here, when the set key 46b is operated again, the moving image is reproduced. The CPU 36 transfers the MPEG data stored in the desired MPEG file to the SDRAM 26 by 1 GOP at a cycle corresponding to 1 GOP, and responds to the vertical synchronization signal Vsync 1 to the MPEG4 codec 38 To the decompression instruction. The MPEG4 codec 38 performs the same processing as described above in response to the vertical synchronization signal Vsync1. The video encoder 28 reads image data from the SDRAM 26 every time the vertical sync signal Vsync 2 is generated, converts the read image data into an NTSC C-format composite video signal, and converts the image data. The supplied composite video signal is supplied to the LCD monitor 30. As a result, the moving image following the first frame is displayed on the LCD monitor 30.

When the right direction is designated by the cursor key 36c while the moving image is being reproduced in the forward direction, the moving image is reproduced in the reverse direction. At this time, the CPU 36 executes the MPEG decompression task shown in FIGS. 6 and 7, the JPEG compression task shown in FIGS. 8 to 10, the JPEG decompression task shown in FIG. 11, and the display task shown in FIG. 12 in parallel. .

The CPU 36 executes the tasks shown in FIGS. 6 to 12 under the control of a multitask OS such as ITRON. A control program corresponding to such a task is stored in the flash memory 22.

In the playback mode, the SDRAM 26 is mapped as shown in FIG. Bank 26a Bank 0), Bank 26b (Bank 1) and Bank 26c (Bank 2) Each is an area for storing one frame of decompressed image data. The JPEG data area 26c is an area for storing JPEG data of each frame. The J PEG index area 26e is an area for storing the start address value of the J PEG data of each frame stored in the J PEG data overnight area 26d, and includes 45 columns JPEG—index [0] to JPEG—. It is formed by index [44]. The MPEG 4 data area 26f is an area for storing the MPEG data read from the recording medium 44.

The transfer of MPEG data from the recording medium 44 to the MPEG4 data area 26f is periodically executed by a task (not shown).

Referring to FIG. 6, in step S1, various variables are initialized. Specifically, set the GO P number gop—mmi to “#”, set the number of frames vop_num to “*”, set the column number K to “vop—mini—1”, and set the address value jenc— Set adr to "JPG —START". In addition, the flags mdec—end, jenc_flg, jdec—flg and disp fig are set to 0, the frame numbers mdec_num, jenc num, and jdec—nrnn are set to “0”, and the bank number is set. Set mbank, jbank and dbank to "0".

Here, the GOP number gop_num is the identification number of the G〇P of interest, and “#” indicates the identification number of the GOP to which the frame that was being played back when the reverse moving image playback was instructed belongs. The number of frames vop—num is the number of frames to be played from the GOP of interest, and “*” is “1” for the number of the frame that was playing when the reverse video playback was instructed. Indicates the added value.

When reverse playback starts from the frame to which frame number “2” is assigned among 15 frames belonging to GOP (n + 1) shown in FIG. 3, “#” indicates “n + 1” and “*” indicates Indicates "3".

Column number K is the identification number of the column formed in JPEG index area 26e shown in FIG. The address value jenc-adr is the address value at which writing of the JPEG data is started, and "JPG-START" is the start address value of the JPEG data area 26d.

The flag mdec end indicates that the playback frame reaches the first frame of the MPEG file. This is a flag for identifying whether or not the operation has been performed. "0" means not reached, "1" means reached. The flag jenc1 fig is a flag for identifying whether or not to permit the JPEG codec 40 compression processing. "0" means forbidden, "1" means allowed. The flag jdec-fig is a flag for identifying whether or not to permit the decompression processing of the JPEG codec 40. "0" means forbidden and "1" means allowed. The flag disp-flg is a flag for identifying whether or not the encoding process of the video encoder 28 is permitted. "0" means prohibition, "1" means permission.

The frame number mdec-num is an identification number of a frame to be expanded by the MPEG4 codec 38. The frame number jenc-num is an identification number of a frame to be compressed by the JPEG codec 40. The frame number jdec-num is the identification number of the frame to be expanded by the JPEG codec 40.

The bank number mbank is an identification number of a bank in which one frame of image data expanded by the MPEG4 codec 38 is to be stored. The bank number jbank is used to identify a bank in which one frame of image data to be compressed by the JPEG codec 40 or a bank in which one frame of image data decompressed by the JPEG codec 40 is to be stored. Number. The bank number dbank is the identification number of the bank in which one frame of image data to be displayed on the LCD module 30 is stored.

In step S3, it is determined whether or not the vertical synchronization signal Vsync 1 has occurred. If YES, it is determined in step S5 whether or not the flag mdec-end force is "1". If mdec—end = 0, the process proceeds to step S9 and subsequent steps. If mdec—end = l, the flag jdec—flg is set to “1” in step S7, and the process returns to step S3.

The flag mdec-end is set to "1" in step S37 described later when the playback frame reaches the first frame of the MPEG file. Step S7 is a step for setting the flag; jdec-flg to "1" in place of step S69 described later after JPEG compression of the first frame of the MPEG file is completed. . In step S9, the frame number mdec_num is set to the frame number jenc_num. In step S11, the number of frames vop—num is set to the number of frames] 'vop_nmn, in step S13, the GOP number gop—num is set to the GOP number jgop—num, and in step S15, the bank number mbank is the bank number; Set to jbank.

Frame number mdec num, frame number vo _ num, GP ¾> go— _ num and bank number mbank are referenced in the MPEG decompression task, and frame number jenc— num, frame number jvop _ num, and GOP number j go _ num. The jbank number is also referred to in the JPEG compression task. By the processing of steps S9 to S15, synchronization is ensured for the parameters related to each other. Note that, as is clear from the processing in step S1, the first processing in step S9 has no meaning.

In step S17, a one-frame decompression instruction is issued to the MPEG4 codec 38. The issued decompression instruction includes a GOP number gop-num, a frame number mdec-num, and a puncture number mbank. The MPEG4 codec 38 reads and reads one frame of MPEG data specified by the GOP number gop—nrnn and the frame number mdec—num from the MPEG4 data area 26 # shown in FIG. The expanded MPEG data is expanded, and the expanded image data is written to the bank corresponding to the bank number mbank among the banks 0 to 2 shown in FIG. When the decompression process is completed, YES is determined in step S19, and the flag jenc-fig is set to "1" in step S21. As a result, the processing after step S43 shown in FIG. 8 is started. In step S23, the bank number mbank is updated according to Equation 1. According to Equation 1, the remainder obtained by dividing "mbank + l" by "3" is set as the bank number mbank.

[Formula 1]

mb ank = (mb ank + l)% 3

In step S25, the frame number mdec-num is incremented, and in step S27, it is determined whether or not the incremented frame number mdec-num matches the frame number vop-num. If NO here, it is considered that MPEG expansion of the frame to be reproduced from the GOP of interest has not been completed yet, and the direct step S Return to 3.

On the other hand, if YES is determined in step S27, the GOP number gop—nuni is decremented in step S29 to change the GOP of interest. In step S31, it is determined whether or not the decremented G P number gop_num is less than "0".

If the GOP number gop—nmn is equal to or greater than “0”, the flow advances to step S33 to set the number of frames vop-num to the number of GOP frames corresponding to the G0P number gop-nvrai. After that, in step S35, the frame number mdec-num is set to "0", and the process returns to step S3. On the other hand, if the GOP number gop-num is less than "0", the flag mdec-end is set to "1" in step S37 to end the MPEG extension processing. Upon completion of the process in the step S37, the process returns to the step S3.

By executing the MPEG decompression task, as shown in FIG. 3, a plurality of GOPs forming the MPEG data are sequentially specified in the reverse direction, and a plurality of frames forming the specified GOP are sequentially processed. Direction is specified. The MPEG4 codec 38 performs a decompression process on the specified frame. The expanded image data is written to each of banks 0 to 2 in the order of bank 0 → bank 1 → bank 2 → bank 0 →. Referring to FIG. 8, in step S41, it is determined whether or not flag jenc-flg indicates "1". If YES, the process proceeds to step S43. As described above, the flag jenc-fig is updated to "1" in step S21 when the first MPEG expansion is completed. Therefore, the processing after step S43 is started when one frame of image data is secured in the bank corresponding to the bank number mbank.

In step S43, a compression instruction is issued to the JPEG codec 40. The issued compression instruction contains the bank number; jbank (= mbank) and the address value; jenc — adr. The J PEG codec 40 reads one frame of image data from the bank corresponding to bank number] 'bank among banks 0 to 2 shown in FIG. 4, performs J PEG compression on the read image data, Then, the generated JPEG data is written after the address corresponding to the address value jenc-adr of the JPEG data area 26 d shown in FIG.

When J PEG compression is completed, YES is determined in step S45, and step S47 Writes the address value jenc-adr to the column JPEG-index [K] in the JPEG index area 26 e shown in Fig. 2. In step S49, the address value jenc-adr is updated according to equation 2, and in step S51 it is determined whether the condition of equation 3 is satisfied.

[Formula 2]

jenc— adr = jenc one adr + flt

[Equation 3]

Jenc—adr + (compressed size + a) ≤JPEG_END

Note that the “compressed size” shown in Expressions 2 and 3 is the size of the JPEG data created by the process in the immediately preceding step S43. “JPEG-END” is the end address value of the JPEG data area 26 d shown in FIG. 2, and “HI” is the margin.

When the condition of Expression 3 is satisfied, the process directly proceeds to step S55. On the other hand, if the condition of Expression 3 is not satisfied, the address value; jenc—adr is set to “JPEG—START” in step S53, and the process proceeds to step S55. In step S55, the column number K is decremented, and in subsequent step S57, it is determined whether or not the column number K is less than "0". If the column number K is "0" or more, go directly to step S61. On the other hand, if the column number is less than "0", the column number K is set to "MAX-J-IDX-1" in step S59, and then the process proceeds to step S61. Here, “MAX—J—IDX” is the total number of columns formed in the JPEG index area 26 d. As can be seen from FIG. 2, "MAX-J-IDX" indicates "45" in this embodiment.

In step S61, it is determined whether or not the frame number jenc-num matches "jvop-num-1". That is, it is determined whether or not the frame subjected to JPEG compression is the last frame to be reproduced from the focused GOP. If NO here, it is determined in step S63 whether or not the GOP number jgop_nmn is equal to or more than "# -2", and the address value written in the column JPEG__indextjdec—mmi] exceeds "0" It is determined in step S65 whether or not this is the case.

In step S63, a sufficient number of frames of JPEG data are This is a step for determining whether or not the data is stored in the file 26d. Step S65 is a step for determining whether or not a valid address value is set in the column JPEG-indexlj'dec-num].

If one of steps S63 and S65 is determined to be N〇, the process directly proceeds to step S69. If YES is determined in both steps S65 and S67, the flag jdec-fig is set to "1" in step S67, and the process proceeds to step S69. By the processing of step S67, the processing of step S93 and thereafter shown in FIG. 11 is started. In step S69, a flag is set to prohibit the JPEG compression processing;

If YES is determined in the step S61, the column number K is updated to “K + jvop_num” in a step S71, and in a step S73, the updated column number K is equal to or more than “MAX—J-IDX”. Determine whether or not. If “NO” here, the process directly proceeds to the step S77. If “YES”, the variable K is updated to “K—MAX—J—IDX” in a step S75, and then the process proceeds to a step S77. As a result, the columns formed in the JPEG index area 26e are designated cyclically. -In step S77, the GOP number jgop-nrnn is decremented. In step S81, it is determined whether or not the updated GOP number jgop-nrnn is "0". If YES is determined here, the flag jenc-fig is set to "0" in a step S79, and the process returns to the step S41.

On the other hand, if NO is determined in the step S77, the processes in and after the step S81 are executed. In step S81, the number of frames of the GOP corresponding to the GOP number jgop-nrnn is added to the column number K, and the column number K is updated with the added value obtained thereby. In step S83, it is determined whether or not the updated column number K is equal to or greater than "MAX-J-IDX". If NO, the process directly proceeds to step S63. If YES, the column number K is updated to “K—MAX__J—IDX” in step S85, and then the process proceeds to step S63.

With this JPEG compression, image data of each frame decompressed in the forward direction from each GOP specified in the reverse direction is subjected to JPEG compression in the forward direction. The JPEG data of each frame obtained in this way is stored in the JPEG data area 26 d Is written to. The start address value of the JPEG data of each frame is written to the JPEG index area 26e in the manner shown in FIGS. 5 (A) to 5 (D).

Focusing on the JPEG index area 26e, first, the three start address values for the three frames of GOP (n + 1) are stored in columns JPEG—index [2] to JPEG—index [0] as shown in FIG. ] Respectively. Next, the 15 start address values for the 15 frames of GOP (n) are written to columns JPEG-index [17] to JPEG-index [3], respectively, as shown in Fig. 5 (B).

When JPEG compression for GOP (n-1) is started, YES is determined in step S63 (and S65), and the flag; jdec-fig is set to "1" in step S67. That is, decompression processing of JPEG data is permitted. The start address value stored in the JPEG index area 26e is read by the JPEG decompression task described later in the manner shown in FIGS. 5C and 5D.

Of the 15 start address values for the 15 frames of GOP (n-2), the first 3 start address values are written to JPEG-index [2] to JPEG-index [0], respectively, and the remaining 12 The first address values are written in JPEG-index [44] to JPEG-index [32], respectively (see Fig. 5 (D)).

Referring to FIG. 11, in step S91, it is determined whether or not flag jdec-fig indicates "1". If YES, a one-frame decompression instruction is given to JPEG codec 40 in step S93. The decompression instruction includes the address value written in the column; jpeg—indextjdec—num] and the bank number; jbank. The JPEG codec 40 reads one frame of JPEG data from the JPEG data area 26e according to the address value included in the decompression instruction, decompresses the read JPEG data, and decompresses the decompressed image data. Bank number; Write to the bank corresponding to jbank.

When the JPEG decompression process is completed, YES is determined in step S95, the flag disp-flg is set to "1" in step S97, and the column number jdec-mim is incremented in step S99. In step S101, it is determined whether or not the updated column number; jdec-nmn is greater than or equal to "MAX-J-IDX". If NO here, go directly to step S105, but if YES, step S1 In column 03, the column number jdec—num is set to “0”, and the process proceeds to step S105. In step S105, the flag; jdec-fig is set to "0", and the process returns to step S91.

By such a JPEG decompression task, the start address value of the JPEG data of each frame is read from the JPEG index area 26d in the manner shown in FIGS. 5 (A) to 5 (D).

Referring to FIG. 12, in step S111, it is determined whether or not flag disp_fig is "1". If “YES” here, the generation of the vertical synchronization signal Vsync 2 is waited for in step S113, and then the puncture number dbank is updated in accordance with equation 4 in step S115. According to Equation 4, the remainder obtained by dividing "; jbank + 2" by "3" is set as the bank number dbank.

[Formula 4]

dbank = jbank + 2)% 3

In step S117, the updated bank number dbank is set in the video encoder 28, and the process returns to step S111.

The banks 0 to 2 shown in FIG. 2 are cyclically designated by the powerful display task, and a moving image moving in the opposite direction is displayed on the LCD monitor 30.

As can be understood from the above description, each of the plurality of GOPs is an image of a plurality of frames in which the first frame in the forward direction is subjected to the intra coding and the frame following the first frame in the forward direction is subjected to the inter coding. Includes data. The CPU 36 sequentially specifies the plurality of GOP blocks in the reverse direction (S29). The MP EG 4 codec 38 forwardly decodes the image data of a plurality of frames included in the specified GOP, and the JP EG codec 40 forwardly decodes each of the decoded image data of the plurality of frames. Intra coding is performed. The CPU 36 determines whether or not the number of frames subjected to the intra code is equal to or larger than a threshold (S63). When the determination result of the CPU 36 is affirmative, the J PEG codec 40 decodes the image data of a plurality of frames that have been intra-coded in the reverse direction.

In other words, the image data decoded in the forward direction by the MPEG4 codec 38 is intra-coded in the forward direction by the JPEG codec 40. Intra The encoded image data is then decoded in the reverse direction by the JPEG codec 40. By adopting intra coding, it is possible to reproduce the image in the opposite direction while suppressing the circuit scale.

In this embodiment, the JPEG system is adopted for the intra coding of the image data of each frame. However, the JPEG2000 system may be adopted instead of the JPEG system.

While this invention has been described and illustrated in detail, it is obvious that it is used by way of example and example only and should not be construed as limiting, the spirit and scope of the invention being set forth in the appended claims. Limited only by the language of the claim.

Claims

The scope of the claims

1. An image processing device comprising:

A plurality of blocks each including image data of a plurality of screens in which the first screen in the first time axis direction is subjected to the intra coding and the screen following the first screen in the first time axis direction is subjected to the inter coding. Designation means for sequentially designating along a second time axis direction opposite to the first time axis direction;

First decoding means for sequentially decoding image data of a plurality of screens included in the block designated by the designation means along the first time axis direction;

Encoding means for performing intra-encoding on each of the image data of a plurality of screens decoded by the first decoding means along the first time axis direction; and

Second decoding means for sequentially decoding the image data of a plurality of screens encoded by the encoding means along the second time axis direction.

2. An image processing apparatus according to claim 1, further comprising: a determination unit configured to determine whether the number of screens subjected to the intra encoding is equal to or greater than a threshold value, wherein the second decoding unit includes the determination unit. When the result of the determination is positive, the decoding process is performed. ,

3. An image processing device according to claim 1 or 2, wherein the processing of the encoding means and the processing of the second decoding means follow the JPEG system.

4. An image processing apparatus according to any one of claims 1 to 3, wherein the first time axis direction is a forward direction, and the second time axis direction is a reverse direction.

5. An image processing program executed by a processor of the image processing apparatus, comprising:

A plurality of screens each including image data of a plurality of screens in which the first screen in the first time axis direction is subjected to the intra coding and the screen following the first screen in the first time axis direction is subjected to the inter coding. A designating step of sequentially designating the blocks along a second time axis direction opposite to the first time axis direction;

A first decoding step of sequentially decoding image data of a plurality of screens included in the block specified by the specified step along the first time axis direction;

An encoding step of performing intra-encoding along the first time axis direction on each of the image data of a plurality of screens decoded by the first decoding step; and A second decoding step of sequentially decoding the image data of a plurality of screens encoded by the encoding step along the second time axis direction.

6. An image processing device comprising:

A plurality of blocks each including image data of a plurality of screens in which a first screen in the first time axis direction is subjected to intra coding and a screen subsequent to the first screen in the first time axis direction is inter-coded. A processor that sequentially specifies along a second time axis direction opposite to the first time axis direction;

A decoder for sequentially decoding image data of a plurality of screens included in a block designated by the processor along the first time axis direction; and

Each of the image data of the plurality of screens decoded by the decoder is subjected to the intra coding along the first time axis direction, and the image data of the plurality of screens subjected to the inner coding is processed in the second time axis direction. A codec that decodes in order.

7. An image processing device according to claim 6, wherein the processor alternately instructs the codec to perform an encoding process and a decoding process, and the codec executes a process according to the instruction from the processor.