JP2014165651A - Image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem of an increase in circuit scale which is incurred because an operation mode is selected from determination results regarding decoding failure, motion vector prediction, etc., before switching an operation from one to another.SOLUTION: The present invention comprises: an image generation unit 1 for generating a frame image; a mode switchover determination unit 2 for determining the operation mode of a frame image; a memory control unit 3 for writing or reading the frame image to and from memory; an image processing unit 6 for performing image processing and outputting from the frame image read out by the memory control unit 3 in accordance with the operation mode; and a partial reconfiguration control unit 7 for controlling, on the basis of the mode switchover determination unit 2, some circuit of the memory control unit 3 so as to be partially reconfigurable.

Description

  The present invention relates to an image processing apparatus that processes frame image data.

  2. Description of the Related Art Conventionally, there is an image processing apparatus that handles frame images in which one screen is a frame unit as moving image data. For example, encoded stream data is input, and the input stream data is decoded to generate and output a frame image in units of frames. Alternatively, the interlace image data is input, and the progressive frame image data is generated and output from the field unit image data. The frame image output from such an image processing device is input to the display device and displayed at a predetermined frame rate.

  In such an image processing apparatus, if there is a data error in the encoded stream data, the frame image affected by the data error will not be correctly decoded, and therefore there is a data error in the encoded stream data. In this case, there is a problem in that a frame image having a decoding error is displayed from the display device, and the displayed moving image is uncomfortable. Further, the display device outputs at a predetermined frame rate, and the image processing device needs to output frame images at predetermined time intervals. At this time, if the frame processing is not performed in time and a frame is missing, one frame is missing from the display device, and the displayed moving image is uncomfortable. was there.

  Therefore, when the encoded stream data is input and the input stream data is decoded to generate and output a frame image in units of frames, a memory for storing the decoded frame image in units of frames is provided, and a decoding error is provided. When a frame image in which a frame exists is generated, a new frame image is created by using the data of the corresponding position of the immediately preceding and / or immediately following frame image for all or part of the frame image that has failed to be decoded. There was a way. (For example, refer to Patent Document 1).

  In addition, a memory for storing the generated frame images in units of frames is provided, and when one frame is missing because the generation of the frame images is not in time for a predetermined period, before and after the missing frame images that are hindered to be displayed correctly There is generally a method of generating and supplementing an interpolation frame from a frame image. As a method of generating an interpolation frame with little subjective deterioration by simple processing, a motion vector is predicted from the generated frame image and predicted. There is a method of generating an interpolation frame when the variation of the motion vector is small, and not performing the interpolation process using the motion vector when the variation is large. (For example, refer to Patent Document 2).

JP 2001-148859 A (page 4-7, FIG. 1) JP 2011-77948 A (page 4-8, FIG. 2)

  However, in the methods such as Patent Document 1 and Patent Document 2, the operation mode is selected from the determination results such as decoding failure and motion vector prediction, and the operation is switched. Therefore, it is necessary to mount a circuit corresponding to each operation mode in advance, and when there are a plurality of operation modes, there is a problem that the circuit scale increases.

  The present invention has been made to solve the above-described problems. Even when there are a plurality of operation modes as described above, each processing process is performed without increasing the circuit scale. It is an object of the present invention to obtain an image processing apparatus that outputs a frame image that does not hinder correct display.

  In the image processing apparatus according to the present invention, an image generation unit that generates and outputs a frame image, a mode switching determination unit that determines an operation mode of the frame image, a memory control unit that writes or reads the frame image into a memory, and The memory control unit performs image processing on the frame image read from the memory based on the operation mode, and outputs a part of the circuit of the memory control unit so that partial reconfiguration is possible. A partial reconfiguration control unit, the memory control unit using a programmable logic device capable of partial reconfiguration, wherein the partial reconfiguration control unit is based on the operation mode. Of the memory control circuit It is characterized in that configuration a portion Chi.

  The present invention includes a partial reconfiguration control unit that controls a part of the circuits of the memory control unit and the image processing unit so as to enable partial reconfiguration, and the result of the mode switching determination unit that determines the operation mode of the frame image. Image processing that outputs frame images that do not hinder processing and display correctly without increasing the circuit scale, even if there are multiple operation modes by performing partial reconfiguration based on A device can be obtained.

1 is a configuration diagram of an image processing apparatus showing Embodiment 1 of the present invention. It is a figure which shows the mode switching determination part of this invention. It is a figure which shows the DRAM control part at the time of the normal mode of this invention. It is a figure which shows the SRAM control part at the time of the normal mode of this invention. It is a figure which shows the output of the image process part at the time of normal mode of this invention. It is a figure which shows the DRAM control part at the time of the interpolation mode of this invention. It is a figure which shows the SRAM control part at the time of the interpolation mode of this invention. It is a figure which shows the timing chart at the time of the 6th flame | frame defect of this invention. It is a figure which shows the method of motion vector prediction. It is a figure which shows the output of the image process part at the time of the 6th flame | frame defect of this invention. It is a figure which shows the interpolation process with respect to the input in the image process part of this invention.

Embodiment 1 FIG.
FIG. 1 shows an image processing apparatus according to Embodiment 1 for carrying out the present invention. In FIG. 1, the frame image generated by the image generation unit 1 is written to or read from the memory 4 by the memory control unit 3, the frame image read by the memory control unit 3 is supplied to the image processing unit 6, and the image output is output. To do. Further, it is determined whether or not the output from the image generation unit 1 has been normally output by the mode switching determination unit 2, and whether or not the output from the image generation unit 1 has been normally output as a determination result and the switching thereof. A signal indicating the timing (mode switching instruction signal) is output to the memory control unit 3, the image processing unit 6, and the partial reconfiguration control unit 7, and the partial reconfiguration control unit 7 provides partial reconfiguration to the configuration circuit. A partial reconfiguration data is output from the configuration circuit, and a part of the memory control unit 3 is partially reconfigured. In addition, the motion vector prediction unit 5 predicts a motion vector from a plurality of frame image outputs from the memory control unit 3 and outputs a prediction result to the memory control unit 3. Hereinafter, the state in which the output from the image generation unit 1 is normally output as the determination result of the mode switching determination unit 2 is referred to as a normal mode, and the state in which the output is not normally output is referred to as an interpolation mode.

  Next, details of each component will be described. The image generation unit 1 generates a frame image and outputs it to the memory control unit 3. At this time, a process start pulse indicating generation start is output for each frame, and a process completion pulse indicating completion of frame image generation is output. These processing start pulse and processing completion pulse are supplied to the mode switching determination unit 2. The image generation unit 1 receives, for example, encoded stream data, decodes the input stream data, generates a frame unit frame image, and outputs the frame image.

  As another aspect, progressive frame image data is generated from field-based image data using interlaced image data as input. As another aspect, a frame image is generated by performing a frame rate conversion process for converting a frame image input at a certain frame rate into a predetermined frame rate. As another aspect, a frame image having a predetermined image size is generated from a frame image input with a certain image size.

  The mode switching determination unit 2 determines whether or not the output from the image generation unit 1 has been normally output. FIG. 2 is a diagram illustrating an example of the mode switching determination unit 2. It is determined whether or not the time interval from the process start pulse input from the image generation unit 1 to the process completion pulse being input is within a predetermined period. The counter 21 returns to the initial value when the processing start pulse is input, performs counting with a predetermined clock, and stops the counting when the processing completion pulse is input.

  The comparison circuit 22 compares the count value of the counter 21 with the set value. When the comparison circuit 22 returns to the initial value within a predetermined range, the comparison circuit 22 determines that the output from the image generation unit 1 has been normally output, and the predetermined range. If the value does not return to the initial value, it is determined that the output from the image generation unit 1 has not been normally output. Alternatively, when the count value stops within a predetermined range, it is determined that the output from the image generation unit 1 has been normally output, and when the count value does not stop within the predetermined range, the output from the image generation unit 1 Judge that the output was not output normally.

  Here, the set value is a value corresponding to an assumed image generation processing time, and is set based on a time for displaying at a desired frame rate and a predetermined clock used for the counter.

  Returning to FIG. 1, the memory control unit 3 will be described. The memory control unit 3 writes the frame image from the image generation unit 1 into the memory 4, reads it at a predetermined timing, and outputs the frame image. The memory 4 has a capacity capable of storing a plurality of frame images and is configured to be able to output a plurality of frame images simultaneously. In FIG. 1, the first memory is composed of a memory DRAM 41 composed of DRAM (Dynamic Random Access Memory), and the second memory is composed of a memory SRAM 42 composed of SRAM (Static Random Access Memory). Show.

  At this time, the memory control unit 3 accesses the DRAM 41 that is the first memory to perform writing or reading, and the SRAM control unit that accesses the SRAM 42 that is the second memory to perform writing or reading. 33 and an address generation unit 31 that generates addresses of the memory DRAM 41 and the SRAM 42 accessed by the DRAM control unit 32 and the SRAM control unit 33, respectively.

  The memory control unit 3 outputs the frame image data to the image processing unit 6 that processes the frame image data from the input memory control unit 3 and outputs it to the display device based on the determination result of the mode switching determination unit 2. . When the determination result of the mode switching determination unit 2 indicates the normal mode, the SRAM control unit 33 outputs the frame image data read from the SRAM 42. On the other hand, when the determination result of the mode switching determination unit 2 indicates the interpolation mode, the frame control data read from the DRAM 41 by the DRAM control unit 32 and the frame image data read from the SRAM 42 by the SRAM control unit 33 are output. Details of the address generation unit 31 and details of how to read a frame image to be read by the memory control unit 3 for output to the image processing unit 6 will be described later.

  Note that at least the memory control unit 3 is composed of a programmable logic device capable of partial reconfiguration, and the address generation unit 31 which is a part of the circuit of the memory control unit 3 is subject to partial reconfiguration. It becomes the circuit which becomes.

  A programmable logic device capable of partial reconfiguration is, for example, a programmable logic device such as a field programmable gate array (FPGA), and in recent years, only a part of the circuit developed on the FPGA is rewritten during the operation of the circuit. A partial reconfiguration technique that can be used has been put into practical use. Conventionally, even if a part of the circuit is changed, it is necessary to reconfigure all the circuits mounted on the FPGA after stopping the operation of the FPGA once. This is a technique that allows only a part of a circuit to be configured in another circuit while the circuit is left. In other words, if the partial reconfiguration technique is used, it is possible to realize another function without increasing hardware resources.

  However, the area on the FPGA of the region to be partially reconfigured is determined by the circuit scale of the circuit having the maximum circuit scale among a plurality of circuits that perform partial reconfiguration in the region. In addition, since the time required for configuration is proportional to the area of the area to be configured, considering that partial reconfiguration must be executed within the processing idle time during operation, a large area is unnecessarily large. When it is a configuration target, there is a problem that the time required for the configuration becomes long and the processing time is reduced.

  Therefore, in the present invention, the entire memory control unit 3 is not subject to partial reconfiguration, but only the address generation unit 31 which is a partial circuit of the memory control unit 3 is subject to partial reconfiguration. The circuit scale and configuration time required for partial reconfiguration can be suppressed.

  The address generation unit 31 is a target for partial reconfiguration, and the read address generation unit 311 and the read address generation unit 312 or the prediction block address generation unit 313 and the prediction block address generation unit 314 are configured to be configurable. Yes.

  FIG. 3 shows the DRAM control unit 32 when the read address generation unit 311 is configured in the address generation unit 31. FIG. 4 shows the SRAM control unit 33 when the read address generation unit 312 is configured in the address generation unit 31.

  In FIG. 3, the synchronization control unit 321 receives a synchronization signal based on a predetermined frame rate and, based on the determination result of the mode switching determination unit 2, sends a control signal indicating the timing for making a write request to the DRAM 41 to the write address. It supplies to the production | generation part 324. In addition, a control signal indicating the timing of making a read request to the DRAM 41 is supplied to the read address generation unit 311. Difference in output timing between a control signal indicating the timing for making a write request to the DRAM 41 and a control signal indicating the timing for making a read request to the DRAM 41 when the determination result of the mode switching determination unit 2 is the normal mode and the interpolation mode Will be described later.

  The image data input unit 322 stores a period until the frame image input from the image generation unit 1 is written to the DRAM 41 based on a control signal from the synchronization control unit 321.

  The base address generation unit 323 generates a base address based on a control signal indicating the timing at which a write request or a read request to the DRAM 41 is sent from the synchronization control unit 321 to the write address generation unit 324 and the read address generation unit 311. Output. When the determination result of the mode switching determination unit 2 determines that the output from the image generation unit 1 is normally output, the base address is the base address corresponding to the head address of the frame to be written to the write address generation unit 324. The base address corresponding to the head address of the frame to be read is output to the read address generation unit 311. This is because the address range of the DRAM 41 related to one frame is grasped in advance, and an address value is generated by adding a value corresponding to the address range of the DRAM 41 related to one frame to the head address value of the previous frame. realizable.

  The address value to be added is not the address range itself of the DRAM 41 for one frame, but the head address is set to a head address convenient for accessing the DRAM 41 by using a fixed value larger than the address range of the DRAM 41 for one frame. It is good.

  The write address generation unit 324 generates a write address to the DRAM 41 by incrementing one address at a time from the base address corresponding to the head address of the frame to be written from the base address generation unit 323, and outputs the write address to the DRAM interface unit 325. .

  The read address generation unit 311 generates a read address to the DRAM 41 by incrementing one address at a time from the base address corresponding to the head address of the frame to be read from the base address generation unit 323, and outputs the read address to the DRAM interface unit 325. .

  The DRAM interface unit 325 writes the frame image data stored in the image data input unit 322 to the DRAM 41 based on the input write address from the write address generation unit 324. Further, the frame image data is read from the DRAM 41 based on the input read address from the read address generation unit 311 and output to the image data output unit 326.

  The image data output unit 326 outputs the frame image data read by the DRAM interface unit 325 to the SRAM control unit 33.

  In FIG. 4, the synchronization control unit 331 receives a synchronization signal based on a predetermined frame rate and, based on the determination result of the mode switching determination unit 2, sends a control signal indicating the timing for making a write request to the SRAM 42 to the write address. It supplies to the production | generation part 334. In addition, a control signal indicating the timing for making a read request to the SRAM 42 is supplied to the read address generation unit 312. Difference in output timing between a control signal indicating a timing for requesting writing to the SRAM 42 and a control signal indicating a timing for performing a read request to the SRAM 42 when the determination result of the mode switching determination unit 2 is the normal mode and the interpolation mode. Will be described later.

  The image data input unit 332 stores a period until the frame image input from the DRAM control unit 32 is written to the SRAM 42 based on the control signal from the synchronization control unit 331.

  The base address generation unit 333 generates a base address based on a control signal indicating the timing of performing a write request or read request to the SRAM 42 from the synchronization control unit 331, and sends the base address to the write address generation unit 334 and the read address generation unit 312. Output. When the determination result of the mode switching determination unit 2 determines that the output from the image generation unit 1 is normally output, the base address is the base address corresponding to the head address of the frame to be written to the write address generation unit 334. The base address corresponding to the head address of the frame to be read is output to the read address generation unit 312. This is because the address range of the SRAM 42 related to one frame is grasped in advance, and an address value is generated by adding a value corresponding to the address range of the SRAM 42 related to one frame to the head address value of the previous frame. realizable.

  The address value to be added is not the address range of the SRAM 42 for one frame, but the head address is adjusted to a head address convenient for accessing the SRAM 42 by using a fixed value larger than the address range of the SRAM 42 for one frame. It is good.

  In addition, by always using the same start address for writing and reading frames, it is possible to use an SRAM having a capacity capable of storing frame image data for one frame, so that the cost can be reduced. However, it goes without saying that it is necessary to perform address control so that writing to the SRAM 42 does not overtake reading.

  The write address generation unit 334 generates a write address to the SRAM 42 by incrementing one address at a time from the base address corresponding to the head address of the frame to be written from the base address generation unit 333, and outputs the write address to the SRAM interface unit 335. .

  The read address generation unit 312 generates a read address to the SRAM 42 by incrementing the base address corresponding to the start address of the frame to be read from the base address generation unit 333 one by one, and outputs the read address to the SRAM interface unit 335. .

  The SRAM interface unit 335 writes the frame image data stored in the image data input unit 332 to the SRAM 42 based on the input write address from the write address generation unit 334. Further, the frame image data is read from the SRAM 42 based on the input read address from the read address generation unit 312 and output to the image data output unit 336.

  The image data output unit 336 outputs the frame image data read by the SRAM interface unit 335 to the image processing unit 6.

  FIG. 5 is a diagram illustrating the output of the image processing unit 6 in the normal mode. In this case, the address generation unit 31 is configured with a read address generation unit 311 and a read address generation unit 312, the DRAM control unit 32 has the configuration shown in FIG. 3, and the SRAM control unit 33 has the configuration shown in FIG. 4. It is a configuration.

  The upper part of FIG. 5 shows the output of the DRAM controller 32, the middle part shows the output of the SRAM controller 33, and the lower part shows the output of the image processor 6 in time series. P1 to P12 indicate frame images, and P1 is assumed to be frame number 1 and P12 is assumed to be frame number 12.

  When it is determined that the determination result of the mode switching determination unit 2 is normally output for the outputs P1 to P12 from the image generation unit 1 (in normal mode), the frame image output from the image generation unit 1 is normal. Since it has been generated, the image processing unit 6 merely outputs the image at a predetermined frame rate without any particular processing. In addition, the SRAM control unit 33 can store the next frame image by performing address control so as to write the output of the next frame image from the DRAM control unit 32 to the SRAM 42 while reading.

  Next, the memory control unit 3 when the determination result of the mode switching determination unit 2 determines that the output from the image generation unit 1 is not normally output (in the interpolation mode) will be described.

  When the determination result of the mode switching determination unit 2 determines that the output from the image generation unit 1 is not normally output, the prediction block address generation unit 313 and the prediction block address generation unit 314 are configured in the address generation unit 31. Is done.

  FIG. 6 shows the DRAM control unit 32 when the predicted block address generation unit 313 is configured in the address generation unit 31. FIG. 7 shows the SRAM control unit 33 when the predicted address generation unit 314 is configured in the address generation unit 31. Here, the synchronization control unit 321, the image data input unit 322, the base address generation unit 323, the base address generation unit 333, the write address generation unit 324, the DRAM interface unit 325, the image data output unit 326, shown in FIGS. Since the synchronization control unit 331, the image data input unit 332, the write address generation unit 334, the SRAM interface unit 335, and the image data output unit 336 have the functions and operations described above, description thereof will be omitted.

  The prediction block address generation unit 313 generates a read address to the DRAM 41 for supplying frame image data for the motion vector prediction unit 5 to predict a motion vector in the first period (motion vector prediction period). Thereafter, based on the motion vector prediction result from the motion vector prediction unit 5 in the second period, the image processing unit 6 generates a read address to the DRAM 41 for supplying frame image data for generating an interpolation frame. To do.

  Similarly, the prediction block address generation unit 314 generates a read address to the SRAM 42 for supplying frame image data for the motion vector prediction unit 5 to predict a motion vector in the first period (motion vector prediction period). To do. Thereafter, based on the motion vector prediction result from the motion vector prediction unit 5 in the second period, the image processing unit 6 generates a read address to the SRAM 42 for supplying frame image data for generating an interpolation frame. To do.

  FIG. 8 is a diagram illustrating a timing chart when the determination result of the mode switching determination unit 2 determines that the sixth frame (P6) from the image generation unit 1 is not normally output. FIG. 8 shows that the output from the image generation unit 1 is not normally output from the mode switching determination unit 2 at the time T1, and the output from the mode switching determination unit 2 to the image generation unit 1 is normal at the time T2. A mode switching determination instruction signal indicating that it is output is output.

  The partial reconfiguration control unit 7 receives the mode switching determination instruction signal from the mode switching determination unit 2 and controls the partial reconfiguration to the address generation unit 31 to the external configuration circuit. The external configuration circuit includes configuration data for the read address generation unit 311 and the read address generation unit 312, and configuration data for the prediction block address generation unit 313 and the prediction block address generation unit 314 for the address generation unit 31. And store.

  When the read address generation unit 311 and the read address generation unit 312 are currently configured in the address generation unit 31 and the mode switching determination instruction signal indicates the interpolation mode (H level in FIG. 8), the external configuration Instruct the address generation unit 31 to configure the prediction block address generation unit 313 and the prediction block address generation unit 314. Nothing is done to indicate the normal mode.

  On the other hand, when the prediction block address generation unit 313 and the prediction block address generation unit 314 are currently configured in the address generation unit 31 and the mode switching determination instruction signal indicates the normal mode (L level in FIG. 8). Then, an external configuration circuit is instructed to configure the read address generator 311 and the read address generator 312 to the address generator 31. Nothing is done to indicate the interpolation mode.

  In FIG. 8, the prediction block address generation unit 313 and the prediction block address generation unit 314 are partially reconfigured from the time T1 to the address generation unit 31, and the read address generation unit 311 and the read address generation unit are transferred to the address generation unit 31 from the time T2. 312 is partially reconfigured. That is, the prediction block address generation unit 313 and the prediction block address generation unit 314 are configured in the address generation unit 31 after the time required for the partial reconfiguration from the time T1.

  At this time, since the memory control unit 3 supplies frame image data for the motion vector prediction unit 5 to predict the motion vector in the first period (motion vector prediction period), the DRAM 41 receives the seventh frame (P7 ) Frame image data, and the frame image data of the fifth frame (P5) is supplied from the SRAM.

  Thereafter, based on the motion vector prediction result predicted by the motion vector prediction unit 5 in the first period, the image processing unit 6 supplies frame image data for generating an interpolation frame in the second period. The frame image data of the seventh frame (P7) is supplied from the DRAM 41, and the frame image data of the fifth frame (P5) is supplied from the SRAM.

  Next, the operation of the motion vector prediction unit 5 when using an algorithm called a block matching method, which is a general method of motion vector prediction used in MPEG (Moving Picture Experts Group) and the like, will be described.

  FIG. 9 is a diagram showing a motion vector prediction method. In other words, the motion vector of the sixth frame is predicted using the block matching method from the fifth frame and the seventh frame that are before and after the sixth frame. In FIG. 9, one frame is divided into, for example, blocks of 16 × 16 pixels (hereinafter simply referred to as blocks). Here, a motion vector for the block B65 among the divided blocks (B61 to B69) is obtained. Here, the sixth frame that is the target of motion vector prediction is P6 '.

  A range indicated by a broken line with respect to the previous frame P5 in the SRAM 42 and the next frame P7 in the DRAM 41 is set as a search range for B65. First, as shown in FIG. 9A, the target block on the P5 side is the upper left block of the broken line area of the previous frame P5, and the next frame P7 that is in a point relationship with the target block on the P5 side and in point symmetry with B65. The lower right block of the broken line area is set as the target block on the P7 side, and the correlation of the image data is obtained for each target block.

  Next, as shown in FIG. 9B, the target block on the P5 side is a block moved to the right by one pixel, and the broken line of the next frame P7 that is in point symmetry with the target block on the P5 side at B65 The block of the area is set as the target block on the P7 side, and the correlation of the image data is taken for each target block.

  In this way, the image data is correlated for all combinations of the target blocks in the range indicated by the broken line while shifting the target blocks by one pixel. FIG. 9C shows the positional relationship with the center of the range represented by the broken line as the target block, and FIG. 9D shows the final positional relationship of the combination candidates. A movement amount between two blocks having a positional relationship having the maximum correlation value among all correlation values obtained for the broken line area in this way is set as a motion vector for the block B65.

  By performing this for all the blocks (B61 to B69) on P6 ', the motion vector of P6' can be predicted by obtaining the motion vectors for all the blocks.

  As described above, in the first period, the prediction block address generation unit 313 inputs the pixel data for each block for obtaining the correlation of each block by the motion vector prediction unit 5 to the motion vector prediction unit 5. Address is generated and output to the DRAM interface unit 325. Similarly, in the first period, the prediction block address generation unit 314 has a positional relationship that is point-symmetric with respect to the prediction block address generation unit 313 and the target block so that the motion vector prediction unit 5 obtains the correlation of each block. ) An address to the SRAM 42 for inputting pixel data for each block to the motion vector prediction unit 5 is generated and output to the SRAM interface unit 335.

  Next, an address generation policy in the second period will be described. For example, if the horizontal axis direction of the frame is the x axis, the vertical direction is the y axis, the value of the obtained motion vector is (Δx, Δy), and the coordinates of the block on the interpolation frame P6 ′ are (x, y), the previous frame The coordinates of the block on P5 (x−Δx / 2, y−Δy / 2) and the coordinates of the block on the next frame P7 (x + Δx / 2, y + Δy / 2) are the coordinates of the two blocks necessary for generating the interpolation frame. It becomes. Necessary for generating an interpolated frame of the previous frame and the next frame to the image generation unit 6 by converting each read address into an address shifted from the base address based on the motion vector thus predicted. Pixel information can be input.

    FIG. 10 is a diagram illustrating the output of the image processing unit 6 in the interpolation mode in the sixth frame (P6). The upper part of FIG. 10 shows the output of the DRAM controller 32, the middle part shows the output of the SRAM controller 33, and the lower part shows the output of the image processor 6 in time series. As described above, when interpolating the sixth frame, frame image data before and after the interpolation target frame is output from the DRAM control unit 32 and the SRAM control unit 33, respectively, and the image processing unit 6 is obtained in the interpolation mode. An interpolation frame can be generated by simply averaging each frame image data.

  FIG. 11 is a diagram showing an interpolation process for an input in the image processing unit 6 of the present invention. FIG. 11A shows a moving image in which one vehicle runs in three frames from the lower left to the lower right of the frame in the normal mode image from the fifth frame (P5) to the seventh frame (P7). .

  FIG. 11B shows that the partial reconfiguration for the interpolation mode was not performed when the mode switching determination unit 2 determined that the output from the image generation unit 1 was not normally output for the sixth frame. Show the case. In this case, even if the fifth frame (P5) and the seventh frame (P7) are read as they are, two cars appear in the image processing unit 6 that performs simple averaging.

  In FIG. 11C, the partial reconfiguration for the interpolation mode is performed when the mode switching determination unit 2 determines that the output from the image generation unit 1 is not normally output for the sixth frame. Show the case. In this case, as described above, the frame images of P5 and P7 are read so that correction can be performed by simple averaging, so that the interpolated frame P6 ′ obtains a frame image close to P6 in FIG. be able to.

  In the second period, reading is performed at a rate based on a predetermined frame rate specified for output at a specified predetermined frame rate. However, the first period is a specified predetermined frame rate. It is not limited to the frame rate, and the reading period may be shortened by reading at high speed and outputting it to the motion vector predicting unit 5.

  For example, when the SRAM control unit 33 writes the frame image data of the seventh frame from the DRAM control unit 32 when interpolating the sixth frame, the SRAM control unit 33 obtains the motion vector, so the DRAM control unit Since it knows how much the frame image data of the seventh frame from 32 is shifted from the reference, it is corrected and written to the SRAM 42. For the shortage of the frame image data of the seventh frame, that is, the portion that has not been read in the second period due to the seventh frame being shifted and read in the second period, the DRAM control unit 32 performs the second period. It is unnecessary to read all the frame image data from the DRAM 41 again after the second period by reading the data later and writing it to the SRAM 42 by the SRAM control unit 33.

  As described above, according to the image processing apparatus according to the first embodiment, by performing partial reconfiguration on a part of the DRAM control unit 32 and the SRAM control unit 33, the read address generation unit 311 and the read operation are performed. Since the address generation unit 312, the prediction block address generation unit 313, and the prediction block address generation unit 314 are not provided at the same time, the circuit scale can be reduced. In particular, when using a device with limited circuit scale capacity, it is necessary to reduce the circuit scale as much as possible when the usage rate for the circuit scale capacity of the device is high. The ability to automatically control some of the circuits that operate exclusively using partial reconfiguration has the great effect of reducing manufacturing costs.

  In addition, among the functional blocks that need to be switched in order to obtain the effect of improving the quality of moving images, only the address generator 31 that is the minimum functional block is targeted for partial reconfiguration. The time required for the operation can be reduced.

  Furthermore, in the description using the diagrams shown in FIGS. 8 to 10, the frame in the interpolation mode is one of the sixth frames. However, for example, when the seventh frame is also in the interpolation mode, the correlation is taken. The fifth frame (P5) and the motion vector are obtained by setting the next frame to be processed as the eighth frame (P8), and the DRAM control unit in the second period based on the respective motion vectors of the sixth frame and the seventh frame Needless to say, by interpolating by outputting P8 from 32 and P5 from the SRAM control unit 33, it is possible to cope with continuous interpolation.

DESCRIPTION OF SYMBOLS 1 Image generation part 2 Mode switching determination part 3 Memory control part 4 Memory 5 Motion vector prediction part 6 Image processing part 7 Partial reconfiguration control part

Claims (6)

An image generation unit that generates and outputs a frame image;
A mode switching determination unit for determining an operation mode of the frame image;
A memory control unit for writing or reading the frame image into a memory;
An image processing unit that performs image processing on the frame image read from the memory by the memory control unit based on the operation mode;
A partial reconfiguration control unit that controls a part of the circuit of the memory control unit so as to enable partial reconfiguration;
The memory control unit is configured using a programmable logic device capable of partial reconfiguration,
The partial reconfiguration control unit configures a part of the circuit of the memory control unit based on the operation mode. When the frame image generated by the image processing unit is output in a predetermined frame period or when a correctly decoded frame image is output, the mode switching determination unit is When the operation mode is determined to have been normally output and the operation mode is set to the normal mode, and a new frame image is not output during the predetermined frame period or a correctly decoded frame image is not output. , It is determined that the output from the image generation unit was not normally output, the operation mode is a normal mode, the operation mode is an interpolation mode,
The image processing unit selects and outputs the frame image read from the memory control unit when the mode switching determination unit determines that the normal mode is selected. The image processing apparatus according to claim 1, wherein an interpolation frame is generated from the plurality of frame images read out from the unit and output. The memory control unit includes a read address generation unit that generates a read address of a read target frame and a prediction block address generation unit that generates read addresses of a plurality of frames before and after the read target frame. It can be switched by the part,
The partial reconfiguration control unit controls the partial reconfiguration so that the read address generation unit is configured in the memory control unit when the mode switching determination unit determines that the normal mode is selected, and the interpolation mode 3. The image processing apparatus according to claim 2, wherein when it is determined, partial reconfiguration is controlled so that the prediction block address generation unit is configured in the memory control unit. The memory is composed of a first memory and a second memory,
The memory control unit writes the frame image read from the first memory to the second memory. When the mode switching determination unit determines that the normal mode is selected, the memory control unit generates a second image generated by the read address generation unit. The frame image read from the second memory based on the read address for the second memory is output as a read target frame, and when the mode switching determination unit determines the interpolation mode, the prediction block address generation unit generates A plurality of frame images read from the first memory and frame images read from the second memory based on a read address for one memory and a read address for a second memory, before and after the read target frame It is output as a frame of The image processing apparatus according to Motomeko 3. A motion vector prediction unit that performs a motion vector prediction from the frame image between successive frames generated by the image generation unit and predicts a pixel block used for interpolation;
The predicted block address generation means generates a read address to the first memory and a read address to the second memory for output to the image processing unit based on the pixel block predicted by the motion vector prediction unit. The image processing apparatus according to claim 4, wherein the image processing apparatus is generated. The first memory is a DRAM (Dynamic Random Access Memory),
The image processing apparatus according to claim 4, wherein the second memory is an SRAM (Static Random Access Memory).

Images