JP2009124547A - Image processor, and image recording and reproducing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent interruption of super-resolution processing due to a failure of positional difference estimation, in special reproduction such as slow reproduction or still reproduction. <P>SOLUTION: A super-resolution processing part 21 increases resolution by combining an input display frame F1 with a reference frame F2. A positional difference estimation part 25 estimates the positional difference between corresponding images of the display frame and the reference frame. A super-resolution processing control part 26 outputs an image from the super-resolution processing part 21 in succeeding in the positional difference estimation, and repeats the process of the super-resolution processing part 21 by changing the reference frame to another frame in failing in the positional difference estimation. The repetition process is aimed particularly at a slow reproduction image and a still reproduction image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image high-resolution processing technique, and more particularly to a high-resolution processing technique during special reproduction in an image recording / reproducing apparatus.

  Recent image display devices display video in various screen sizes suitable for applications from large screen televisions to portable terminals. At this time, an image signal input from broadcasting, communication, a storage medium, or the like is not displayed as it is, but it may be necessary to convert the resolution of the image signal in accordance with the screen size to be displayed. At that time, a technique for improving the resolution as well as simply interpolating the number of pixels by an interpolation low-pass filter using a generally known sinc function or a spline function has been proposed.

  For example, Patent Document 1 discloses a processing technique for increasing the number of pixels while increasing the resolution by combining a plurality of input image frames (hereinafter simply referred to as frames) into one frame. . In this technique, a plurality of sets of data having different sampling positions are used to cancel aliasing components generated by sampling and restore high-frequency components of the image signal. Hereinafter, this processing is referred to as “super-resolution processing”. In super-resolution processing, a single frame is synthesized using data of a plurality of adjacent moving image frames, and can be applied to increase the resolution of a reproduced image of a video recording / reproducing apparatus.

  On the other hand, the playback mode of the video recording / playback apparatus has not only normal playback at standard speed but also special playback functions with different speeds such as fast forward, slow, and still. Slow and still playback are used in special playback for the user to slowly watch the video scene. Therefore, in these special playback modes, the image quality of the content tends to be conspicuous compared to the normal playback video. As a technique for improving the image quality during special reproduction, for example, Patent Document 2 discloses that the image quality of a displayed video is improved by controlling the display time of each field during slow reproduction.

JP-A-8-336046 Japanese Patent Laid-Open No. 11-27633

  For example, in the super-resolution processing described in Patent Document 1, generally, an image frame is divided into a plurality of blocks, and the positional difference between the display frame and the reference frame is estimated in units of blocks to increase the resolution. Here, depending on the video scene and the pattern of a plurality of inputted frames, there may be a block in which the position difference estimation result cannot be obtained correctly.

  As an example, assume a video scene in which a new object appears in a display frame. When the reference frame for composition is acquired from the past only with respect to the display frame, the object does not exist in the reference frame. Therefore, the evaluation value based on the position difference estimation indicates an abnormal value exceeding the threshold value, and the position difference estimation has failed in the block including the object.

  Conventionally, in a block in which position difference estimation has failed, the original super-resolution processing is interrupted and switched to pixel interpolation processing using an interpolation low-pass filter or the like. For this reason, in a block in which position difference estimation has failed, high resolution (high image quality) cannot be achieved even if the number of pixels increases. In addition, if a plurality of reference frames are used for one display frame, the position difference estimation failure can be reduced. However, increasing the number of reference frames can increase the circuit scale and processing for super-resolution processing. It increases time and is practically difficult.

  As described above, slow playback and still playback in an image recording / playback apparatus are often used in the case of a video scene that a user wants to watch. In particular, if the position difference estimation fails at the time of special reproduction, the image quality becomes worse as compared with normal reproduction, which is not preferable. Note that the technique described in Patent Document 2 is a technique different from the super-resolution processing and is not related to the success or failure of position difference estimation.

  An object of the present invention is to provide a high-resolution image by avoiding interruption of super-resolution processing due to failure of position difference estimation in special reproduction such as slow reproduction or still reproduction.

  The image processing apparatus of the present invention includes a super-resolution processing unit that increases the resolution by synthesizing an image of another reference frame with an input display frame image, and an image corresponding to the display frame and the reference frame. And a super-resolution processing control unit that controls the operation of the super-resolution processing unit based on the estimation result of the position-difference estimation unit, the super-resolution processing control unit If the position difference estimation unit succeeds in position difference estimation, the image from the super-resolution processing unit is output. If the position difference estimation unit fails in position difference estimation, the reference frame is transferred to another frame. And the super resolution processing unit performs the process.

  The image recording / playback apparatus of the present invention includes a content storage unit that stores image content, a recording / playback unit that records the input image content in the content storage unit, and plays back the content from the content storage unit. A resolution enhancement processing unit that converts the resolution and outputs, a recording / playback unit, and a control unit that controls the resolution enhancement processing unit. The resolution enhancement processing unit combines a reference frame image with a display frame image of the reproduced image content to increase the resolution, the display frame, the reference frame, A position difference estimator that estimates the position difference of the corresponding image of the image, and a super-resolution processing controller that controls the operation of the super-resolution processor based on the estimation result of the position difference estimator. The resolution processing control unit outputs an image from the super-resolution processing unit when the position difference estimation unit succeeds in the position difference estimation, and when the position difference estimation unit fails in the position difference estimation, The playback unit causes the content storage unit to read another frame instead of the reference frame, and changes the reference frame to perform the processing of the super-resolution processing unit.

  Here, when the recording / playback unit performs slow playback of the image content, the super-resolution processing control unit changes the reference frame within the range of the number of repetitions set in accordance with the playback speed, and performs the super-resolution processing. Repeat the process. When the recording / playback unit performs still playback of the image content, the super-resolution processing control unit changes the reference frame over a still playback period or a user-specified time and repeats the processing of the super-resolution processing unit. .

  According to the present invention, it is possible to avoid interruption of super-resolution processing due to failure of position difference estimation and provide a high-resolution image. This is particularly effective for special playback such as slow playback and still playback.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a first embodiment of an image recording / reproducing apparatus according to the present invention. The image recording / playback apparatus 1 includes an input unit 11, a recording / playback unit 12, a high resolution processing unit 13, a buffer 14, a control unit 15, an operation unit 16, and a content storage unit 17. Connected to.

  For example, video (moving image) content such as a television broadcast signal is input to the input unit 11. The recording / playback unit 12 processes the input video content and records it in the content storage unit 17. The content storage unit 17 is composed of a large-capacity storage medium such as an HDD or a DVD, and the video content is stored and stored in a predetermined compression encoding format or in an image frame sequence. The recording / playback unit 12 plays back video content from the content storage unit 17. The playback mode has special playback functions as well as normal playback. At the time of reproduction, a decoding process or the like is performed according to the storage format of the content, and output to the high resolution processing unit 13. For example, when digital broadcast content is input to the input unit 11, the recording / playback unit 12 decodes the stream content encoded by MPEG-2 to generate an image frame sequence, and outputs the frame sequence to the high resolution processing unit 13. .

  The resolution enhancement processing unit 13 performs resolution enhancement processing using the frame sequence (including the display frame F1 and the reference frame F2 as will be described later) input from the recording / playback unit 12, and the processing result is sent to the buffer 14. Write. The high-resolution video data written in the buffer 14 is output to the display device 18 and displayed at a predetermined timing. As the display device 18, for example, a liquid crystal display panel, a plasma display panel, or the like is used.

  The control unit 15 is composed of, for example, a CPU and a software program, and controls the recording / playback unit 12 to perform video content recording, playback, and special playback operations. Further, the high resolution processing unit 13 is controlled to perform the high resolution processing, and the recording / playback unit 12 is supplied with a frame necessary for the high resolution processing. The operation unit 16 is a user interface unit that receives an operation from the user, and includes a remote controller and a group of operation buttons. An instruction signal by a user operation is sent to the control unit 15.

FIG. 2 is a diagram illustrating a configuration example of the high resolution processing unit 13 in FIG.
The high resolution processing unit 13 receives the display frame F1 and the reference frame F2 from the recording / playback unit 12, and the control information from the control unit 15. Here, it is assumed that one frame immediately before (the past) the display frame F1 is selected as the reference frame F2. Here, it is of course possible to use a plurality of frames as the reference frame F2, but the processing circuit can be most simplified by using one reference frame. The display frame F1 is input to the super-resolution processing unit 21 and the interpolation processing unit 22. The reference frame F2 is input to the super-resolution processing unit 21.

  The position difference estimation unit 25 estimates the position difference between the display frame F1 and the reference frame F2, and sends the position difference information to the super-resolution processing control unit 26. Here, the position difference estimation unit 25 divides one frame into block areas each having 16 × 16 pixels as a unit, and performs the position difference estimation in units of blocks.

  The super-resolution processing unit 21 performs super-resolution processing on the input image of the display frame F1 using the reference frame F2. The interpolation processing unit 22 performs pixel interpolation processing using an interpolation low-pass filter or the like on the input image of the display frame F1. Here, the super-resolution processing unit 21 and the interpolation processing unit 22 divide the frame into block regions, and perform the above processing in units of blocks.

  The selection unit 23 receives the super-resolution image P1 processed by the super-resolution processing unit 21, the interpolation image P2 processed by the interpolation processing unit 22, and the selection signal Sel from the super-resolution processing control unit 26. The The selection unit 23 selects either the super-resolution image P1 or the interpolation image P2 according to the selection signal, and outputs the selected image to the buffer write control unit 24. The buffer write control unit 24 writes the input processed image in an appropriate position in the buffer 14 in correspondence with the position in the frame of the processing block.

  The super-resolution processing control unit 26 receives the position difference information sent from the position difference estimation unit 25 and determines whether the position difference estimation is successful. When the position difference estimation is successful, the selection unit 23 is caused to select the super-resolution image P1 from the super-resolution processing unit 21. When the position difference estimation fails, the reference frame F2 is changed to another frame, and super-resolution processing is performed using a new reference frame F2 ′. Therefore, the control unit 15 is instructed to read the changed reference frame. In this case, for example, a frame immediately after (the future) the display frame F1 is selected as the new reference frame F2 ′. Also in this case, the position difference estimation unit 25 performs position difference estimation again. If the position difference estimation is successful, the selection unit 23 is made to select the super-resolution image P1. If the position difference estimation fails again, the reference frame F2 ′ is further changed to another frame F2 ″, and repeated super-resolution processing is performed until the position difference estimation is successful.

  The number of repetitions setting unit 28 of the super-resolution processing control unit 26 sets the upper limit number of repetitions of super-resolution processing for the same display frame. The repetition count setting unit 28 is constituted by a control register, for example. If the position difference estimation is not successful even if the number of repetitions reaches this upper limit number, the selection unit 23 is made to select the processed image P2 from the interpolation processing unit 22.

  Further, the super-resolution processing success / failure storage unit 27 stores a flag indicating whether or not the position difference estimation has succeeded for each unit block of the high resolution processing. If the super-resolution processing is successful, for example, the flag is set to “1”, and if the super-resolution processing fails, the flag is set to “0”. When one input image frame has a 1920 × 1080 pixel size and a processing unit block has a 16 × 16 pixel size, the capacity required for the super-resolution processing success / failure storage unit 27 is 8100 bits.

Here, the operation of the position difference estimation unit 25 will be described.
The display frame F1 is divided into rectangular blocks of 16 × 16 pixels. In the reference frame F2, a predetermined image range centered on a rectangular area at the same position as the processing target block B1 of the display frame F1 is considered. This image range is called a search range, and the search range is, for example, an area of horizontal 128 pixels × vertical 64 pixels.

  In the position difference estimation, a 16 × 16 pixel rectangular block is moved while being shifted pixel by pixel within the search range on the reference frame F2, and the difference between the pixels corresponding to the processing target block B1 on the display frame F1 at each moving position. Calculate the sum of absolute values. This sum of absolute differences is calculated, for example, for the luminance component of the video. From the obtained sum of absolute differences at each movement position, the one with the smallest value is selected and compared with a preset threshold value. When the difference absolute value sum is equal to or smaller than the threshold value, it is determined that the position difference estimation of the processing target block B1 has succeeded, and the distance from the rectangular block B2 at the moving position that minimizes the difference absolute value sum is used as the position difference data. When the difference absolute value sum exceeds the threshold value, it is determined that a block similar to the processing target block does not exist within the search range of the reference frame F2, and it is determined that the position difference estimation has failed.

  Although the position difference estimation in units of integer pixels is described here, it is also possible to generate ½ pixels by linear interpolation within the search range and perform position difference estimation with ½ pixel accuracy by the same calculation method as described above. .

  FIG. 3 is a flowchart showing the flow of high resolution processing in the present embodiment. In this example, in slow playback or still playback, one frame is repeatedly performed until high resolution processing (super-resolution processing) succeeds.

  In step 101, when slow playback or still playback is instructed by a user operation, the processing target frame F1 to be displayed is read from the recording / playback unit 12.

  In step 102, the super-resolution processing control unit 26 instructs the control unit 15 to read the reference frame F2 for synthesis from the recording / playback unit 12. For the reference frame F2, the frame immediately before the processing target frame F1 is designated in the first reading. In the second reading for repetitive processing because the super-resolution processing has failed, the frame immediately after the processing target frame F1 is designated. Further, in the third reading for repeated processing, the frame immediately before the processing target frame F1 is designated. Thereafter, the previous and subsequent frames are read alternately, such as the second frame after F1.

  In step 103, the high resolution processing unit 13 performs high resolution processing in units of blocks. Here, the super-resolution processing unit 21 and the interpolation processing unit 21 are operated in parallel, and each processed image is output to the selection unit 23. At that time, the position difference estimation unit 25 estimates the position difference between the processing target frame F1 and the reference frame F2. The super-resolution processing control unit 26 determines whether or not the position difference estimation is successful, causes the selection unit 23 to select an image according to the result, and sets the flag of the block in the super-resolution processing success / failure storage unit 27 to “1” ( Success) or “0” (failure). If the position difference estimation fails, the super-resolution process is repeated. Therefore, the controller 15 is instructed to read a new reference frame. Step 103 will be described in detail with reference to FIG.

  In step 104, it is determined whether or not the position difference estimation has succeeded for all the blocks of the processing target frame F1. If it is determined that the process has succeeded, the process proceeds to step 106, and the resolution enhancement processing for the display frame is completed. If it is determined in step 104 that the position difference estimation has not succeeded for all the blocks, the process proceeds to step 105, where iterative process determination is performed.

  In step 105, it is determined whether or not the resolution enhancement process has been performed a preset number of times. If the number of repetitions has been reached, the process proceeds to step 106 and the resolution enhancement processing for the display frame is terminated. If the number of repetitions has not been reached, the process returns to step 102, and the above-described high resolution processing is repeated using a new reference frame.

  As a result of the above processing, the frame to be displayed is subjected to high resolution processing for each block and written to the buffer 14. Then, it is displayed on the display device 18 in accordance with the slow playback or still playback conditions requested by the user.

  Next, FIG. 4 is a flowchart showing details of the high resolution processing performed in units of blocks in step 103 of FIG. In this block unit processing, each block may be processed in parallel or sequentially.

  In step 201, high resolution processing is started in units of processing blocks. Each block may be processed in parallel or sequentially.

  In step 202, the value of the flag of the processing block stored in the super-resolution processing success / failure storage unit 27 is referred to. When the flag = “1”, the processing block has already succeeded in position difference estimation and super-resolution processing has been performed, and the process proceeds to step 210 and the processing of the block is terminated. When the flag = “0”, the previous position difference estimation has failed, and the process proceeds to step 203 where the super-resolution processing is performed again using a new reference frame. Note that since the position difference estimation flag is not given to the block for which resolution enhancement processing is performed for the first time, in this case, it is regarded as “0” and the process proceeds to step 203.

  In step 203, the position difference estimation unit 25 performs position difference estimation between the new reference frame and the processing target frame.

  In step 204, it is determined whether the position difference estimation is successful. As described above, whether or not the position difference estimation is successful is determined based on whether or not the sum of absolute differences between corresponding pixels is within a threshold value.

  When the position difference estimation is successful (Yes in Step 204), the process proceeds to Step 205 and the super-resolution processing unit 21 performs the process. In step 206, the selection unit 23 selects the super-resolution image P1, outputs it to the buffer write control unit 24, and writes it to the buffer 14. As a result, the image of the block is updated to an image with a higher resolution. In step 207, the flag of the processing block is rewritten to “1”, and the process proceeds to step 210 to end the processing of the block.

  If the position difference estimation has failed (No in step 204), the process proceeds to step 208 and the interpolation processing unit 22 performs the process. In step 209, the selection unit 23 selects the interpolated image P2, outputs it to the buffer write control unit 24, and writes it to the buffer 14. In this case, the flag of the processing block remains “0”, and the process proceeds to step 210 to end the processing of the block. In the second and subsequent high resolution processing, the interpolation image by the interpolation processing unit 22 is already recorded in the buffer 14. Since the interpolation image does not need to be updated, it is not necessary to operate the interpolation processing unit 22.

  In this way, the efficiency of the high-resolution processing can be improved by adaptively performing the super-resolution processing and the interpolation processing for each block according to the flag value of the super-resolution processing success / failure storage unit 27. .

  FIG. 5 is a diagram showing an example of the number of repetitions used for the repetition process determination (step 105). This number of times is set in the repetition number setting unit 28.

  In the normal reproduction, the high resolution processing is performed once, whereas in the 1 / N-times slow reproduction, the high resolution processing is performed N times (in the case of the high image quality mode). For example, the resolution enhancement process can be performed twice during ½ × speed slow playback, and four times during ¼ × speed slow playback. In still reproduction, the resolution enhancement process is repeatedly performed while the pause button is pressed.

  Further, in the case of the power saving mode, in the 1 / N double speed slow reproduction, the high resolution processing is reduced to N / 2 times. In still reproduction, super-resolution processing is repeated for a time specified by the user.

  These setting conditions are merely examples, and may be set as appropriate according to the usage status of the apparatus. It is preferable to prepare some setting conditions for each playback mode in advance so that the user can select them. Further, by adding a setting condition for setting the number of repetitions to 0, it is possible to prevent the super-resolution repetition process during special reproduction from being performed.

  As described above, in this embodiment, repeated processing is performed in the case of special playback with a time margin for image processing of a display frame, such as slow playback and still playback. Thereby, it is possible to avoid interruption of super-resolution processing due to failure of position difference estimation and provide a high-resolution image.

  FIG. 6 is a block diagram showing a second embodiment of the image recording / reproducing apparatus according to the present invention. The image recording / playback apparatus 1 of this embodiment has a configuration in which a smoothing processing unit 19 is additionally connected to the buffer 14 in the first embodiment (FIG. 1).

  The smoothing processing unit 19 performs an image smoothing process on the image frame obtained by the high resolution processing unit 13. The purpose is that repetition of super-resolution processing may cause adjacent blocks in a frame to be generated from different reference frames, resulting in image discontinuities at the block boundaries. The smoothing processing unit 19 performs a filter calculation process for removing the discontinuous portion. When determining which region in the frame is to be smoothed, for example, reference is made to the success / failure flag stored in the super-resolution processing success / failure storage unit 27, and the boundary portion of blocks having different flag values is targeted. do it.

  In addition, in order to support the calculation of the smoothing processing unit 19, the buffer 14 has a configuration in which the number of buffer surfaces is increased, so that a desired high-resolution processed video can be supplied to the display device 18.

  FIG. 7 is a block diagram showing a third embodiment of the image recording / reproducing apparatus according to the present invention. In this embodiment, the configuration of the high resolution processing unit 13 in the first embodiment (FIG. 2) is changed. That is, the selection unit 29 is arranged in front of the super-resolution processing unit 21 and the interpolation processing unit 22 to select the supply destination of the display frame F1.

  The position difference estimation unit 25 compares the display frame F1 with the reference frame F2, and performs position difference estimation in the super-resolution processing. The super-resolution processing control unit 26 determines whether or not the position-difference estimation is successful based on the position-difference information. Send Sel. That is, the super-resolution processing unit 21 is selected when the position difference estimation is successful, and the interpolation processing unit 22 is selected when it fails.

  The selection unit 29 activates the selection destination processing unit and outputs the display frame F1 to the selection destination. The selected processing unit performs super-resolution processing or interpolation processing, and either the super-resolution image P1 or the interpolation image P2 is output and written to the buffer 14 by the buffer write control unit 24. Although the output selection unit for the super-resolution image P1 or the interpolation image P2 is omitted here, this may be added.

  In this embodiment, since only one of the modules of the super-resolution processing unit 21 or the interpolation processing unit 22 needs to be operated, there is an effect of reducing the power consumption of the apparatus.

  An embodiment of an image processing apparatus according to the present invention will be described. Embodiments 1 to 3 described above are image recording / playback apparatuses that increase the resolution when playing back recorded video content. On the other hand, the present embodiment is a case where the present invention is applied to an image processing apparatus of a television receiver. In this case, the broadcast video content is displayed in real time, but the high resolution processing is repeatedly executed to display the high resolution image. This will be described below.

  In digital broadcasting, video compressed using image coding technology is broadcast. For example, terrestrial digital broadcasting and BS digital broadcasting are MPEG-2 standards, and one-segment broadcasting is H.264. Video is compressed using the H.264 / MPEG-4 AVC standard. In the television, received data is decoded by a video decoder and displayed on a display.

  In the image coding as described above, a technique called inter-frame prediction using difference information between image frames is used. In this inter-frame prediction, when a macroblock (16 × 16 pixel region) that is a unit of encoding processing is used, when the difference amount between image frames = 0, the macroblock is transmitted with information “skip”. Macroblock_address_increment in MPEG-2; A syntax element such as mb_skip_run in H.264 corresponds to this. In this embodiment, this “skip” information is used.

  When all the macroblocks in the frame are “skip” for the input frame, this indicates that the frame is the same image as the previous frame. Hereinafter, such a frame is referred to as a “skip frame”.

  When skip frames are consecutive, the number of consecutive frames is detected. When one or more skip frames are consecutive, repeated super-resolution processing is executed while changing the reference frame. A case where N skip frames are consecutive corresponds to the 1 / (N + 1) -speed slow playback in the above-described embodiment, and, for example, a maximum of (N + 1) times of super-resolution processing is performed. In this case, since the same image is input to consecutive N frames, a new reference frame used for the repetitive processing is selected from those separated by (N + 1) frames in the front-rear direction.

  As described above, the present invention can be similarly applied not only to special playback (slow playback and still playback) of an image recording / playback apparatus but also to an image processing apparatus in which the same image frame is continuously input.

The block diagram which shows the 1st Example of the image recording / reproducing apparatus which concerns on this invention. The figure which shows the structural example of the high resolution process part. The flowchart which shows the flow of high resolution processing. The flowchart which shows the detail of the high resolution process performed in the block unit in FIG. The figure which shows an example of the repetition frequency used for repetition process determination. The block diagram which shows the 2nd Example of the image recording / reproducing apparatus by this invention. The block diagram which shows the 3rd Example of the image recording / reproducing apparatus by this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image recording / reproducing apparatus 11 ... Input part 12 ... Recording / reproducing part 13 ... High resolution process part 14 ... Buffer 15 ... Control part 16 ... Operation part 17 ... Content storage part 18 ... Display apparatus 19 ... Smoothing process part 21 ... Super Resolution processing unit 22 ... Interpolation processing unit 23, 29 ... Selection unit 24 ... Buffer write control unit 25 ... Position difference estimation unit 26 ... Super-resolution processing control unit 27 ... Super-resolution processing success / failure storage unit 28 ... Repetition count setting unit .

Claims (9)

An image processing device for converting the resolution of an input image,
A super-resolution processing unit that increases the resolution by synthesizing the image of another reference frame with the image of the input display frame;
A position difference estimation unit that estimates a position difference between corresponding images of the display frame and the reference frame;
A super-resolution processing control unit that controls the operation of the super-resolution processing unit based on the estimation result of the position difference estimation unit,
The super-resolution processing control unit outputs an image from the super-resolution processing unit when the position difference estimation unit succeeds in position difference estimation,
An image processing apparatus, wherein when the position difference estimation unit fails in position difference estimation, the reference frame is changed to another frame and the process of the super-resolution processing unit is performed. The image processing apparatus according to claim 1,
An interpolation processing unit that increases the number of pixels by interpolating pixels from an image in the same frame with respect to the image of the display frame;
The super-resolution processing control unit, when the position difference estimation unit fails in position difference estimation, and the position difference estimation fails even if the reference frame is changed to another frame, the image from the interpolation processing unit is displayed. An image processing apparatus for outputting. The image processing apparatus according to claim 1, wherein:
In the super-resolution processing control unit, a success / failure storage unit that stores in the image processing unit whether the position difference estimation unit succeeded or failed in position difference estimation;
It has a repetition number setting part for setting the upper limit number of repetitions or the repetition time of super-resolution processing for the same display frame,
An image processing apparatus that repeats the process of the super-resolution processing unit with reference to the information on the success or failure of the position difference estimation and the repetition upper limit number or repetition time. The image processing apparatus according to claim 1, wherein:
The position difference estimation unit obtains a difference absolute value between corresponding pixels of the display frame and the reference frame for each pixel processing unit,
The super-resolution processing control unit compares the sum of the absolute difference values with a preset threshold value, and determines that the position difference estimation is successful when the sum of the absolute difference values is equal to or less than the threshold value. An image processing apparatus, wherein when the sum of absolute values exceeds a threshold, it is determined that the position difference estimation has failed. The image processing apparatus according to claim 1, wherein:
For the reference frame used by the super-resolution processing unit, the super-resolution processing control unit specifies the frame immediately before the display frame for the first time. An image processing apparatus characterized by alternately specifying the preceding and following frames in the order of the two previous frames. The image processing apparatus according to claim 1, wherein:
An image processing apparatus, comprising: a smoothing processing unit that performs a smoothing process on a discontinuous portion caused by a change of a reference frame on an image from the super-resolution processing unit. An image recording / playback apparatus for recording and playing back an input image,
A content storage unit for storing image content;
A recording / playback unit that records image content to be input to the content storage unit and plays back the content from the content storage unit;
A high-resolution processing unit that converts and outputs the resolution of the reproduced image content;
A control unit for controlling the recording / playback unit and the high resolution processing unit;
The high resolution processing unit
A super-resolution processing unit that increases the resolution by synthesizing an image of another reference frame with an image of a display frame of the reproduced image content;
A position difference estimation unit that estimates a position difference between corresponding images of the display frame and the reference frame;
A super-resolution processing control unit that controls the operation of the super-resolution processing unit based on the estimation result of the position difference estimation unit,
The super-resolution processing control unit outputs an image from the super-resolution processing unit when the position difference estimation unit succeeds in position difference estimation,
If the position difference estimation unit fails to estimate the position difference, the recording / playback unit causes the content storage unit to read another frame instead of the reference frame, changes the reference frame, and performs the super-resolution processing. An image recording / reproducing apparatus characterized in that the processing of a part is performed. The image recording / reproducing apparatus according to claim 7,
When the recording / playback unit performs slow playback of the image content, the super-resolution processing control unit changes the reference frame within the range of the number of repetitions set according to the playback speed, and performs processing of the super-resolution processing unit An image recording / reproducing apparatus characterized by repeating the above. The image recording / reproducing apparatus according to claim 7,
When the recording / playback unit performs still playback of image content, the super-resolution processing control unit changes the reference frame over a still playback period or a user-specified time and repeats the processing of the super-resolution processing unit. An image recording / reproducing apparatus as a feature.

Images