WO2015053352A1 - Display device and super-resolution processing method - Google Patents

Display device and super-resolution processing method Download PDF

Info

Publication number
WO2015053352A1
WO2015053352A1 PCT/JP2014/077033 JP2014077033W WO2015053352A1 WO 2015053352 A1 WO2015053352 A1 WO 2015053352A1 JP 2014077033 W JP2014077033 W JP 2014077033W WO 2015053352 A1 WO2015053352 A1 WO 2015053352A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
image signal
original
interpolation
rate conversion
Prior art date
Application number
PCT/JP2014/077033
Other languages
French (fr)
Japanese (ja)
Inventor
昌彦 瀧口
晃 小池
Original Assignee
シャープ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by シャープ株式会社 filed Critical シャープ株式会社
Publication of WO2015053352A1 publication Critical patent/WO2015053352A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/02—Improving the quality of display appearance
    • G09G2320/0257—Reduction of after-image effects
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/10—Special adaptations of display systems for operation with variable images
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G09G2340/0435—Change or adaptation of the frame rate of the video stream

Definitions

  • the present invention relates to a display device and a super-resolution processing method, and more particularly to a display device that performs super-resolution processing using a plurality of frames and a super-resolution processing method using the device.
  • a liquid crystal panel capable of displaying high-resolution images such as 4K2K (3840 pixels ⁇ 2160 pixels) and 8K4K (7680 pixels ⁇ 4320 pixels), which have higher resolution than 2K1K full high-definition images (1920 pixels ⁇ 1080 pixels).
  • Development is progressing.
  • Development of a high-resolution television using such a high-resolution liquid crystal panel is also progressing, but the actual situation is that the environment for providing content utilizing this high-resolution has not been sufficiently prepared. For this reason, by mounting an upconverter in a television, the resolution of a normal television image is converted to achieve higher definition.
  • a method of performing resolution conversion by inter-frame interpolation processing using a plurality of frames is also called super-resolution processing.
  • the intra-frame interpolation process such as the linear interpolation method or the three-dimensional convolutional interpolation method, there are significant merits in image quality such as a noise reduction effect.
  • the above super-resolution processing is roughly divided into motion estimation processing for extracting feature points from a plurality of images and matching the positions by matching the feature points extracted in the individual images, and integrating the individual images.
  • Three stages of processing are required: a reconstruction process that generates a high-resolution image, and a sharpening process that improves image blurring and removes noise.
  • motion vectors are calculated between frames. For motion images that contain multiple images that do not move continuously, such as motion images that contain still images or motion pictures that have been pulled down from a movie, the motion vector is set appropriately. Extraction cannot be performed, and the effect of increasing the resolution by super-resolution processing cannot be obtained.
  • Patent Document 1 describes a technique that can increase the resolution of a moving image signal including a plurality of images that do not move continuously. In this method, it is determined whether or not an adjacent image of the moving image has a motion, and a super-resolution process is performed on the image having a motion.
  • the FRC function when the FRC function is installed in the high-resolution television, if the FRC is applied after the 4K image conversion by the super-resolution processing, the FRC processing is naturally performed on the 4K image. That is, the circuit configuration for performing motion prediction, motion compensation, etc., including the memory size and memory bandwidth required for FRC, is simply four times as large as that for 2K image processing. For this reason, it is preferable to apply FRC to a 2K image and then upconvert to a 4K image by super-resolution processing because an increase in circuit scale can be suppressed.
  • the interpolation frames 1.5, 2.5, 3.5,... are composed of two original frames, information on the original original frame may be missing. Specifically, the case where the noise is smoothed and the correct amount of noise is not included, the case where the details of the original frame are smoothed and the original fineness of the original frame is impaired, etc. can be considered.
  • the interpolation frame 1.5 and the interpolation frame 2.5 are referred to as reference frames.
  • the original information of the original frame is not included in the frame, and the image quality of the 4K frame 2a after the super-resolution processing is deteriorated.
  • the present invention has been made in view of the above-described circumstances, and in a display device having a frame rate conversion function, super-resolution processing using a plurality of frames without reducing the image quality while suppressing the circuit scale.
  • the purpose is to be able to.
  • the first technical means of the present invention interpolates a frame of an interpolated image signal subjected to motion compensation processing between frames of the input image signal, so that the frame of the input image signal
  • a display device comprising a frame rate conversion unit for converting a number, wherein the motion compensation processing is performed from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion by the frame rate conversion unit.
  • An original frame selection unit that selects an original frame of the input image signal that is not, and a resolution of the predetermined frame in each image signal after the frame rate conversion is set to a pixel value based on the original frame selected by the original frame selection unit. It was equipped with a super-resolution processing unit that converted by interpolation processing Than is.
  • the frame rate conversion unit applies the same original as the input image signal according to a predetermined frame rate for a predetermined number of original frames from the head of the input image signal.
  • the frame is repeatedly output to the original frame selection unit, and the original frame selection unit interpolates the original frame of the input image signal and the interpolated image signal from the arrangement pattern of the original frame repeatedly output from the frame rate conversion unit.
  • An interpolation pattern with a frame is specified, and an original frame of the input image signal and an interpolation frame of the interpolation image signal are discriminated based on the specified interpolation pattern.
  • the frame rate conversion unit receives interpolation pattern information indicating an interpolation pattern between an original frame of the input image signal and an interpolation frame of the interpolation image signal.
  • the original frame selection unit is configured to determine an original frame of the input image signal and an interpolation frame of the interpolation image signal based on the interpolation pattern information. is there.
  • the frame rate conversion unit includes an original frame of the input image signal, an interpolation frame of the interpolation image signal, and a frame of each image signal after the frame rate conversion. Identification information for identifying the image signal is output, and the frame of each image signal to which the identification information is added is output to the original frame selection unit, and the original frame selection unit is added to the frame of each image signal. Based on the identification information, the original frame of the input image signal and the interpolation frame of the interpolated image signal are discriminated.
  • the fifth technical means is characterized in that, in any one of the first to fourth technical means, the frame rate conversion unit and the original frame selection unit share a frame memory.
  • the sixth technical means includes a frame rate conversion step of converting the number of frames of the input image signal by interpolating a frame of the interpolated image signal subjected to motion compensation processing between the frames of the input image signal.
  • a super-resolution processing method wherein an original of an input image signal that has not been subjected to the motion compensation processing from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion in the frame rate conversion step
  • An original frame selection step for selecting a frame; and a resolution for converting the resolution of the predetermined frame in each image signal after the frame rate conversion by interpolation processing of pixel values based on the original frame selected in the original frame selection step.
  • a resolution processing step It is.
  • an original frame that has not been subjected to motion compensation processing can be selected as a reference frame, thereby reducing the circuit scale.
  • FIG. 1 is a diagram showing an example of an appearance when a display device according to the present invention is applied to a liquid crystal television.
  • 1 shows a liquid crystal television which is an example of a display device.
  • the liquid crystal television 1 is remotely operated by a remote controller R, can receive a digital broadcast signal having a resolution such as 2K1K (full high-definition) or 4K2K, and can display the program video.
  • FIG. 2 is a block diagram illustrating a configuration example of a display device according to an embodiment of the present invention.
  • a display device exemplified as the liquid crystal television 1 (hereinafter referred to as the display device 1) includes a tuner 2 that receives a digital broadcast signal and a frame of an input image signal included in the digital broadcast signal received by the tuner 2.
  • a frame rate conversion unit 3 that converts the number of frames of the input image signal by interpolating frames of the interpolated image signal that has undergone motion compensation processing; a first frame memory 4 that is connected to the frame rate conversion unit 3; An original frame selection unit 5 that selects an original frame of an input image signal that has not undergone motion compensation processing from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion by the frame rate conversion unit 3; A second frame memory 6 connected to the original frame selector 5 and a frame rate; A super-resolution processing unit 7 that converts the resolution of a predetermined frame in each image signal after conversion by interpolation processing of pixel values based on the original frame selected by the original frame selection unit 5 and a high-resolution image such as 4K2K are displayed. And a display unit 8 including a possible high-resolution liquid crystal panel.
  • the first frame memory 4 stores frames necessary for motion compensation processing in the frame rate conversion unit 3.
  • the original frame selection unit 5 stores the original frame to be referred to in the super-resolution processing in the second frame memory 6, and reads out the original frame necessary for performing the super-resolution processing from the second frame memory 6. The data is sent to the super-resolution processing unit 7.
  • the super-resolution processing is roughly divided into a motion estimation process for extracting feature points from a plurality of images and matching the positions by matching the feature points extracted in the individual images, and the individual images.
  • a three-stage process including a reconstruction process for generating a high-resolution image by integrating the two and a sharpening process for improving image blurring and removing noise.
  • FIG. 3 is a diagram for explaining a super-resolution processing method by the display device according to the first embodiment of the present invention.
  • the original frame 1 is converted at the frame rate conversion of 120 Hz.
  • An interpolation frame 1.5 is generated from the original frame 2
  • an interpolation frame 2.5 is generated from the original frame 2 and the original frame 3
  • an interpolation frame 3.5 is generated from the original frame 3 and the original frame 4.
  • the original frames 1, 2, 3, 4,... are frames of input image signals that have not been subjected to motion compensation processing, and the interpolation frames 1.5, 2.5, 3.5,. It is a frame of an interpolated image signal that has been processed.
  • An interpolated frame 1.5 is interpolated between the original frames 1 and 2
  • an interpolated frame 2.5 is interpolated between the original frames 2 and 3.
  • an interpolation frame is interpolated between the original frames, and the frame rate is converted to 120 Hz.
  • the original frame selection unit 5 selects an input image signal that has not undergone motion compensation processing from at least a front or rear frame of a predetermined frame in each image signal (original frame and interpolation frame) after frame rate conversion. Select the original frame. For example, as in this example, when a frame rate conversion of a general 2K image of 60 Hz is performed to 120 Hz, the frame of each image signal after the frame rate conversion is an original frame 1, an interpolation frame 1.5, an original frame 2, interpolation frames 2.5,... Are arranged in this order.
  • the original frame selection unit 5 selects every other frame after selecting the first frame (original frame 1), it can extract only the original frame while avoiding the interpolation frame.
  • the odd numbered frame may be selected because the odd numbered frame is the original frame and the even numbered frame is the interpolated frame.
  • a frame in which a circle is added to the frame number after frame rate conversion indicates that the frame is an original frame selected by the original frame selection unit 5. Specifically, when the original frame 2 is the predetermined frame, either the original frame 1, the original frame 3, or both the original frames 1 and 3 are selected.
  • the super-resolution processing unit 7 converts the resolution of a predetermined frame in each image signal after frame rate conversion by interpolation processing of pixel values based on the original frame selected by the frame rate selection unit 5. Specifically, when super-resolution processing is performed on the original frame 1, the original frame 2 is referred to, and pixel values are interpolated based on the original frame 2. Then, the original frame 1 after the super-resolution processing is output as a 4K frame 1a. Similarly, when super-resolution processing is performed on the interpolation frame 1.5, the original frame 1 and the original frame 2 are referred to, and pixel values are subjected to interpolation processing based on the original frames 1 and 2.
  • the interpolated frame 1.5 after the super-resolution processing is output as a 4K frame 1.5a. Furthermore, when super-resolution processing is performed on the original frame 2, the original frame 1 and the original frame 3 are referred to, and pixel values are interpolated based on the original frames 1 and 3.
  • the original frame 2 after the super-resolution processing is output as a 4K frame 2a.
  • super-resolution processing is performed on each image signal (original frame and interpolation frame) after frame rate conversion, and the resolution is converted to 4K.
  • the 4K frames 1a, 1.5a, 2a, 2.5a,... After super-resolution processing are output as 4K (3840 pixels ⁇ 2160 pixels), 120 Hz image signals. Since these 4K frames are generated by using one or more original frames as a reference frame during the super-resolution processing, it is possible to prevent deterioration in image quality while suppressing an increase in circuit scale.
  • the original frame and the interpolated frame are not necessarily arranged alternately after the frame rate conversion. That is, when a 2: 3 pull-down conversion signal, a 2: 2 pull-down conversion signal, or the like is input, the original frame selection unit 5 applies one for each image signal frame after frame rate conversion as described above. If every other frame is extracted, there is a possibility that an interpolation frame is selected by mistake.
  • the frame rate conversion unit 3 is controlled to repeatedly output the same frame as the input image for only a few frames from the beginning, whereby the original frame selection unit 5 determines the position of the original frame. Make it detectable. That is, the frame rate conversion unit 3 repeatedly outputs the same original frame as the input image signal to the original frame selection unit 5 according to a predetermined frame rate for a predetermined number of original frames from the beginning of the input image signal.
  • the original frame selection unit 5 specifies an interpolation pattern between the original frame of the input image signal and the interpolation frame of the interpolation image signal from the original frame arrangement pattern repeatedly output from the frame rate conversion unit 3, and specifies Based on the interpolated pattern, the original frame of the input image signal and the interpolated frame of the interpolated image signal are discriminated.
  • FIG. 4 is a diagram for explaining a super-resolution processing method by the display device according to the second embodiment of the present invention.
  • original frames 1, 2, 3, 4,... Constituting a 2K, 60 Hz full high-definition image are input as input image signals.
  • the frame rate conversion unit 3 performs frame rate conversion on the frame of the input image signal.
  • the same original frame as that of the input image signal is determined according to a predetermined frame rate.
  • the original frame selector 5 since the input image signal is a full high-definition signal and the frame rate is “double”, for example, the two original frames 1 and 2 are repeatedly output from the head twice.
  • the frame rate conversion unit 3 repeatedly outputs the two original frames 1 and 2 from the top in accordance with the 120 Hz frame rate, and then performs normal frame rate conversion for the original frame 3 and subsequent frames. That is, an interpolation frame 3.5 is generated from the original frame 3 and the original frame 4, and this interpolation frame 3.5 is inserted between the original frame 3 and the original frame 4. Thereafter, the same processing is executed for the subsequent original frame.
  • the original frame selection unit 5 determines the arrangement pattern of the original frames repeatedly output from the frame rate conversion unit 3.
  • the original frame selection unit 5 of this example cannot determine whether the frame output from the frame rate conversion unit 3 is an original frame or an interpolation frame, when the same frame is repeatedly input, Assume that the frame is determined to be an original frame.
  • the original frame selection unit 5 determines that the two original frames are arranged patterns that are repeated twice, the original frame selection unit 5 specifies an interpolation pattern when the input image signal is a 2K, 60 Hz full high-definition signal. In the case of this full high-definition signal, the interpolation pattern between the original frame and the interpolation frame is uniquely specified.
  • the original frame selection unit 5 identifies an interpolation pattern in which the odd number is the original frame from the arrangement patterns of the first to fourth original frames 1 and 2, and thereby the fifth frame, that is, the original frame Select frame 3. Thereafter, similarly, the seventh original frame 4 is selected.
  • a frame with a circle added to the frame number after frame rate conversion indicates that it is an original frame selected by the original frame selection unit 5.
  • the original frame 1 uses the original frame 2 as a reference frame
  • the original frame 2 uses the original frame 1 as a reference frame, respectively.
  • Super-resolution processing is performed to make the resolution 4K (4K frames 1a and 2a).
  • the original frames 1 and 2 may not be subjected to super-resolution processing, and the resolution may be changed to 4K by intra-frame copy processing such as nearest neighbor method or intra-frame interpolation processing such as linear interpolation or three-dimensional convolution interpolation. .
  • intra-frame copy processing such as nearest neighbor method or intra-frame interpolation processing such as linear interpolation or three-dimensional convolution interpolation.
  • super-resolution processing is performed using the original frames 3, 4,... Selected by the original frame selection unit 5, and 4K frames 3a, 3.5a, 4a, 4.5a,. Is generated.
  • the basic concept is the same when the input image signal is a 60 Hz 2: 3 pull-down conversion signal, a 2: 2 pull-down conversion signal, or a 24 Hz movie film signal. These examples will be described with reference to FIG. 5 below.
  • FIG. 5 is a diagram for explaining an example of a frame repeat pattern and an interpolation pattern for each type of input image signal.
  • the frame repeat pattern is a type in which the same image is repeatedly sent, and at least one scene is transmitted with the same frame repeat pattern.
  • FIG. 5A shows a case where the input image signal is a 60 Hz video (full high-definition) signal
  • FIG. 5B shows a case where the input image signal is a 60 Hz 2: 3 pull-down conversion signal.
  • FIG. 5D shows a case where the input image signal is a movie film signal of 24 Hz.
  • a white frame indicates an original frame
  • a hatched frame indicates an interpolation frame.
  • the input image signal and the interpolation pattern shown in this example are examples, and the present invention is not limited to these.
  • FIG. 5 (A) is an example in the case where the input image signal is a full high-definition signal of 60 Hz, as described in FIG. 4 above. That is, the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 2, 3, 4,... From the head of the input image signal to the original frame selection unit 5 twice (frame repeat). That is, the process is repeated twice for one original frame 1 and further twice for one original frame 2. The same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify that this is a full high-definition signal, it is necessary to repeat at least two original frames 1 and 2 twice from the beginning of the input image signal.
  • the interpolation pattern is specified as one pattern in which odd-numbered original frames and even-numbered interpolation frames are alternately arranged. That is, the original frame selection unit 5 specifies an interpolation pattern when the input image signal is a full high-definition signal of 60 Hz from the frame repeat pattern (that is, the arrangement pattern) from the frame rate conversion unit 3, and this interpolation Based on the pattern, it is possible to determine that the odd-numbered frame is the original frame and the even-numbered frame is the interpolation frame. As a result, only the original frame can be selected while avoiding the interpolation frame.
  • the example of FIG. 5B is an example in which the input image signal is a 2: 3 pull-down conversion signal of 60 Hz.
  • the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 1, 2, 2, 2, 3,... From the head of the input image signal to the original frame selection unit 5 twice each time ( Frame repeat).
  • Frame repeat since one original frame 1 is repeated twice, the original frame 1 is repeated four times in total, and further, since one original frame 2 is repeated twice, the original frame 2 is repeated six times in total.
  • the same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify the 2: 3 pull-down conversion signal, it is necessary to repeat at least two original frames 1 and 2 from the head of the input image signal 4 times and 6 times, respectively.
  • the original frame selection unit 5 identifies an interpolation pattern when the input image signal is a 2: 3 pull-down conversion signal of 60 Hz from the frame repeat pattern from the frame rate conversion unit 3, and based on this interpolation pattern
  • the original frame and the interpolation frame can be discriminated. For example, if the interpolation pattern is “interpolation 3”, the original frame is arranged every four frames. That is, since the first, sixth, eleventh, sixteenth, sixteenth, twenty-sixth,.
  • the example of FIG. 5C is an example in which the input image signal is a 2: 2 pull-down conversion signal of 60 Hz.
  • the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 1, 2, 2, 3,... From the head of the input image signal to the original frame selection unit 5 twice (frame repeat). ). That is, since one original frame 1 is repeated twice, the original frame 1 is repeated four times in total, and further, since one original frame 2 is repeated twice, the original frame 2 is repeated four times in total. The same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify the 2: 2 pull-down conversion signal, it is necessary to repeat at least two original frames 1 and 2 four times from the beginning of the input image signal.
  • the original frame selection unit 5 identifies an interpolation pattern when the input image signal is a 2: 2 pull-down conversion signal of 60 Hz from the frame repeat pattern from the frame rate conversion unit 3, and based on this interpolation pattern
  • the original frame and the interpolation frame can be discriminated. For example, if the interpolation pattern is “interpolation 2”, the original frame is arranged every three frames. That is, since the first, fifth, ninth, thirteenth, seventeenth, twenty-first,.
  • the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 2, 3,... From the beginning of the input image signal to the original frame selection unit 5 five times (frame repeat). That is, it repeats 5 times for one original frame 1 and further 5 times for one original frame 2. The same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify the movie film signal, it is necessary to repeat at least two original frames 1 and 2 five times from the beginning of the input image signal.
  • the original frame selection unit 5 identifies an interpolation pattern when the input image signal is a 24 Hz movie film signal from the frame repeat pattern from the frame rate conversion unit 3, and based on this interpolation pattern, A frame and an interpolation frame can be discriminated. For example, if the interpolation pattern is “interpolation 1”, the original frame is arranged every four frames. That is, since the first, sixth, eleventh, sixteenth, sixteenth, twenty-sixth,.
  • the frame rate conversion unit 3 interrupts the interpolation for the longest time in the case of the 2: 3 pull-down conversion signal in FIG. 5B, but it is only 11 frames at the maximum. Therefore, the effect on image quality is considered to be extremely small.
  • the selection of the original frame is simplified, and the frame rate conversion unit 3, the original frame selection unit 5 and the super-resolution processing unit 7 are mounted on separate chips, making it difficult to share information. It is particularly effective in such cases.
  • FIG. 6 is a block diagram showing a configuration example of a display device according to the third embodiment of the present invention.
  • the position of the original frame is detected by repeatedly outputting the same frame as the input image for only a few frames from the beginning of the input image signal.
  • the position of the original frame is detected.
  • the interpolated pattern information 9 is used for detection. That is, the frame rate conversion unit 3 outputs the interpolation pattern information 9 indicating the interpolation pattern between the original frame of the input image signal and the interpolation frame of the interpolation image signal to the original frame selection unit 5.
  • the original frame selection unit 5 determines an original frame of the input image signal and an interpolation frame of the interpolation image signal based on the interpolation pattern information 9 from the frame rate conversion unit 3.
  • the interpolation pattern information 9 can be exemplified by the interpolation pattern shown in FIG.
  • the interpolation pattern since the arrangement relationship between the original frame and the interpolation frame is known for each type of input image signal, only the original frame can be selected while avoiding the interpolation frame.
  • the frame rate conversion unit 3 outputs interpolation pattern information (see FIG. 5A) of the full high-definition signal to the original frame selection unit 5. From this interpolation pattern information, the original frame selection unit 5 specifies that the odd number is the original frame and the even number is the interpolation frame.
  • the position of the original frame is detected from the respective interpolation pattern information. Can do. According to the present embodiment, since it is only necessary to send the interpolation pattern information for selecting the original frame, the original frame can be selected more easily.
  • the frame rate conversion unit 3 adds identification information for identifying the original frame of the input image signal and the interpolation frame of the interpolated image signal to each frame of the image signal after the frame rate conversion.
  • the frame of each image signal to which the identification information is added may be output to the original frame selection unit 5.
  • the original frame selection unit 5 determines the original frame of the input image signal and the interpolation frame of the interpolated image signal based on the identification information given to the frame of each image signal.
  • the identification information of the present embodiment for example, “0” is assigned if the frame of each image signal after frame rate conversion is an original frame, and “1” is assigned if the frame is an interpolation frame. .
  • the original frame selection unit 5 determines that the frame is an original frame if the identification information given to the frame of each image signal is “0”, and if the identification information is “1”, the original frame is selected. It is determined as an interpolation frame. This makes it possible to select only the original frame while avoiding the interpolation frame. According to this embodiment, the original frame can be selected easily and reliably.
  • FIG. 7A shows an example of image data to which identification information is not given
  • FIG. 7B shows an example of image data to which identification information is given.
  • a method of embedding this identification information as data of several bytes including a header at the head of the first line of image data (one frame) can be considered. If it is the beginning of the first line of one frame, it is not noticeable because it is an end portion on the screen, and it is extremely rare that the same data is continuously arranged in a plurality of frames. Can be recognized as.
  • FRM0 and FRM1 can be used with 4-byte data.
  • the 4 bytes at the left end of the first line are identification information.
  • FIG. 8 is a block diagram showing a configuration example of a display device according to the fifth embodiment of the present invention.
  • the frame rate conversion unit 3 and the original frame selection unit 5 share the frame memory 10.
  • the original frame selection unit 5 may select the original frame necessary for the super-resolution processing from the frame memory 10.
  • the original frame selection method is the same as that when a separate frame memory is used as described above. According to the present embodiment, the selection of the original frame is simplified and only one frame memory is required, so that the frame memory can be saved.
  • the display device converts the number of frames of the input image signal by interpolating the frames of the interpolated image signal subjected to the motion compensation process between the frames of the input image signal.
  • An input image signal that is not subjected to the motion compensation processing from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion by the frame rate conversion unit.
  • An original frame selection unit for selecting the original frame, and the resolution of the predetermined frame in each image signal after the frame rate conversion is converted by interpolation processing of pixel values based on the original frame selected by the original frame selection unit A super-resolution processing unit.
  • an original frame that has not been subjected to motion compensation processing can be selected as a reference frame.
  • the frame rate conversion unit repeatedly outputs the same original frame as the input image signal to the original frame selection unit according to a predetermined frame rate for a predetermined number of original frames from the beginning of the input image signal
  • the original frame selection unit specifies an interpolation pattern between the original frame of the input image signal and the interpolation frame of the interpolation image signal from the arrangement pattern of the original frame repeatedly output from the frame rate conversion unit, It is desirable to discriminate between the original frame of the input image signal and the interpolation frame of the interpolated image signal based on the identified interpolation pattern. According to this, when the original frame can be easily selected and the frame rate conversion unit, the original frame selection unit and the super-resolution processing unit are mounted on separate chips, it is difficult to share information. Is particularly effective.
  • the frame rate conversion unit outputs interpolation pattern information indicating an interpolation pattern between an original frame of the input image signal and an interpolation frame of the interpolation image signal to the original frame selection unit, and the original frame selection unit Preferably, based on the interpolation pattern information, an original frame of the input image signal and an interpolation frame of the interpolated image signal are discriminated. According to this, since it is only necessary to send the interpolation pattern information for selecting the original frame, the original frame can be selected more easily.
  • the frame rate conversion unit gives identification information for identifying the original frame of the input image signal and the interpolation frame of the interpolated image signal to the frame of each image signal after the frame rate conversion
  • the frame of each image signal to which identification information is assigned is output to the original frame selection unit, and the original frame selection unit is configured to output the input image signal based on the identification information given to the frame of each image signal. It is desirable to discriminate between the original frame and the interpolated frame of the interpolated image signal. According to this, it is possible to easily and reliably select the original frame.
  • the frame rate converter and the original frame selector share a frame memory. According to this, the selection of the original frame can be simplified and the frame memory can be saved.
  • a super-resolution processing method comprising a frame rate conversion step of converting the number of frames of an input image signal by interpolating a frame of an interpolated image signal subjected to motion compensation processing between frames of the input image signal.
  • the original frame selection for selecting the original frame of the input image signal not subjected to the motion compensation process from at least the front or rear frame of the predetermined frame in each image signal after the frame rate conversion in the frame rate conversion step
  • a super-resolution processing step of converting the resolution of the predetermined frame in each image signal after the frame rate conversion by interpolation processing of pixel values based on the original frame selected in the original frame selection step.
  • an original frame that has not been subjected to motion compensation processing can be selected as a reference frame. It is possible to obtain a high-resolution image by performing super-resolution processing without reducing the image quality while suppressing the circuit scale.

Abstract

　A display device provided with a frame rate conversion function, wherein super-resolution processing using a plurality of frames is performed while suppressing an increase in circuit scale but not degrading picture quality. A display device (1) is provided with: a frame rate conversion unit (3) for converting the number of frames in an input image signal by interpolating, between frames of the input image signal, a frame of an interpolation image signal to which a motion compensation process was applied; an original frame selection unit (5) for selecting, at least from among frames ahead or behind a prescribed frame in each image signal after frame rate conversion by the frame rate conversion unit (3), the original frame of the input image signal to which the motion compensation process was not applied; and a super-resolution processing unit (7) for converting the resolution of the prescribed frame in each image signal after frame rate conversion by interpolation processing of a pixel value based on the original frame selected by the original frame selection unit (5).

Description

表示装置及び超解像処理方法Display device and super-resolution processing method
　本発明は、表示装置及び超解像処理方法に関し、より詳細には、複数フレームを用いて超解像処理を行う表示装置及び該装置による超解像処理方法に関する。 The present invention relates to a display device and a super-resolution processing method, and more particularly to a display device that performs super-resolution processing using a plurality of frames and a super-resolution processing method using the device.
　近年、２Ｋ１Ｋのフルハイビジョン画像（１９２０画素×１０８０画素）よりも解像度の高い、４Ｋ２Ｋ（３８４０画素×２１６０画素）や８Ｋ４Ｋ（７６８０画素×４３２０画素）などの高解像度画像の表示を可能とする液晶パネルの開発が進んでいる。そして、このような高解像度液晶パネルを用いた高解像度テレビの開発も進んでいるが、この高解像度を活かしたコンテンツを提供する環境はまだ十分に整備されていないのが実情である。このため、テレビ内にアップコンバータを搭載することにより、通常のテレビ画像の解像度を変換してより高精細化を図ることが行なわれている。 In recent years, a liquid crystal panel capable of displaying high-resolution images such as 4K2K (3840 pixels × 2160 pixels) and 8K4K (7680 pixels × 4320 pixels), which have higher resolution than 2K1K full high-definition images (1920 pixels × 1080 pixels). Development is progressing. Development of a high-resolution television using such a high-resolution liquid crystal panel is also progressing, but the actual situation is that the environment for providing content utilizing this high-resolution has not been sufficiently prepared. For this reason, by mounting an upconverter in a television, the resolution of a normal television image is converted to achieve higher definition.
　通常の２Ｋ画像を４Ｋ画像または８Ｋ画像に変換するアップコンバート技術の中でも、複数フレームを用いたフレーム間補間処理により解像度変換を行う方法は、超解像処理とも呼ばれ、従来の最近隣法などのフレーム内複写処理、線形補間法や３次元畳み込み補間法などのフレーム内補間処理により解像度変換を行う方法と比べ、ノイズリダクション効果など、画質的に大きなメリットがある。 Among up-conversion techniques that convert normal 2K images into 4K images or 8K images, a method of performing resolution conversion by inter-frame interpolation processing using a plurality of frames is also called super-resolution processing. Compared with the intra-frame copying process, the intra-frame interpolation process such as the linear interpolation method or the three-dimensional convolutional interpolation method, there are significant merits in image quality such as a noise reduction effect.
　上記の超解像処理は、大きく分けて、複数の画像から特徴点を抽出し個々の画像で抽出された特徴点同士をマッチングさせて位置を合わせる運動推定処理と、個々の画像を統合して高解像度画像を生成する再構成処理と、画像のぼやけを改善してノイズを削除する鮮明化処理という３段階の処理が必要とされる。特に運動推定処理では、フレーム間で動きベクトルを求めるため、静止状態を含む動画や、映画からプルダウン変換した動画など、連続して動きがない複数の画像を含む動画の場合、動きベクトルを適切に抽出できず、超解像処理による高解像度化の効果を得ることができない。 The above super-resolution processing is roughly divided into motion estimation processing for extracting feature points from a plurality of images and matching the positions by matching the feature points extracted in the individual images, and integrating the individual images. Three stages of processing are required: a reconstruction process that generates a high-resolution image, and a sharpening process that improves image blurring and removes noise. In particular, in motion estimation processing, motion vectors are calculated between frames. For motion images that contain multiple images that do not move continuously, such as motion images that contain still images or motion pictures that have been pulled down from a movie, the motion vector is set appropriately. Extraction cannot be performed, and the effect of increasing the resolution by super-resolution processing cannot be obtained.
　これに対して、例えば、特許文献１には、連続して動きがない複数の画像を含む動画の画像信号に対して高解像度化できる技術が記載されている。これは、動画の隣接した画像に動きがあるか否かを判断し、動きのある画像について超解像処理を行うものである。 On the other hand, for example, Patent Document 1 describes a technique that can increase the resolution of a moving image signal including a plurality of images that do not move continuously. In this method, it is determined whether or not an adjacent image of the moving image has a motion, and a super-resolution process is performed on the image having a motion.
　一方、放送信号は６０Ｈｚでテレビに入力されるが、液晶テレビのようなホールド型表示装置に動きのある画像を表示させた場合、観る者には動き部分の輪郭がぼけて知覚されてしまうという、所謂、動きぼけの問題がある。このため、動きぼけを改善するために、入力画像信号のフレーム間に動き補償処理を施した補間画像信号（補間フレーム）を内挿し、フレームレートを例えば１２０Ｈｚに変換することが行われている。これはフレームレート変換（ＦＲＣ：Frame Rate Conversion）機能といい、上記の高解像度テレビにも搭載されている。 On the other hand, the broadcast signal is input to the television at 60 Hz. However, when a moving image is displayed on a hold-type display device such as a liquid crystal television, the viewer sees the outline of the moving portion as blurred. There is a so-called motion blur problem. For this reason, in order to improve motion blur, an interpolation image signal (interpolation frame) subjected to motion compensation processing is interpolated between frames of the input image signal, and the frame rate is converted to, for example, 120 Hz. This is called a frame rate conversion (FRC: Frame 、 Rate 機能 Conversion) function, and is also installed in the above high-definition television.
特開２００９－１８８４７０号公報JP 2009-188470 A
　ここで、高解像度テレビにＦＲＣ機能を搭載する場合、超解像処理により４Ｋ画像変換後にＦＲＣをかけると、当然ながら４Ｋ画像に対してＦＲＣ処理を行うことになる。すなわち、ＦＲＣに必要なメモリサイズ、メモリバンド幅をはじめ、動き予測や動き補償等を行うための回路構成が、２Ｋ画像処理時の単純に４倍になってしまう。このため、２Ｋ画像にＦＲＣをかけた後に、超解像処理により４Ｋ画像にアップコンバートするほうが、回路規模の増大を抑制することができるため望ましい。 Here, when the FRC function is installed in the high-resolution television, if the FRC is applied after the 4K image conversion by the super-resolution processing, the FRC processing is naturally performed on the 4K image. That is, the circuit configuration for performing motion prediction, motion compensation, etc., including the memory size and memory bandwidth required for FRC, is simply four times as large as that for 2K image processing. For this reason, it is preferable to apply FRC to a 2K image and then upconvert to a 4K image by super-resolution processing because an increase in circuit scale can be suppressed.
　しかしながら、従来構成のままだと超解像処理による高画質が期待できないという問題がある。これについて図９に基づいて説明する。２Ｋ，６０Ｈｚの入力画像信号を構成するオリジナルフレーム１，２，３，４，…が入力された場合、１２０Ｈｚのフレームレート変換の際に、オリジナルフレーム１とオリジナルフレーム２から補間フレーム１．５が生成され、同様にして、オリジナルフレーム２とオリジナルフレーム３から補間フレーム２．５が生成され、オリジナルフレーム３とオリジナルフレーム４から補間フレーム３．５が生成される。そして、これらオリジナルフレームの間に、補間フレームが内挿され、フレームレートが２倍の１２０Ｈｚに変換される。 However, there is a problem that high image quality by super-resolution processing cannot be expected with the conventional configuration. This will be described with reference to FIG. When original frames 1, 2, 3, 4,... Constituting an input image signal of 2K, 60 Hz are input, an interpolated frame 1.5 from the original frame 1 and the original frame 2 is converted at the frame rate conversion of 120 Hz. Similarly, an interpolation frame 2.5 is generated from the original frame 2 and the original frame 3, and an interpolation frame 3.5 is generated from the original frame 3 and the original frame 4. An interpolated frame is interpolated between these original frames, and the frame rate is doubled and converted to 120 Hz.
　上記において、補間フレーム１．５，２．５，３．５，…は、２つのオリジナルフレームが合成されたものであるため、本来のオリジナルフレームの情報が欠落している場合がある。具体的には、ノイズが平滑化され正しいノイズ量が含まれていない場合や、オリジナルフレームの細部が平滑化されオリジナルフレーム本来の精細感が損なわれている場合などが考えられる。 In the above, since the interpolation frames 1.5, 2.5, 3.5,... Are composed of two original frames, information on the original original frame may be missing. Specifically, the case where the noise is smoothed and the correct amount of noise is not included, the case where the details of the original frame are smoothed and the original fineness of the original frame is impaired, etc. can be considered.
　従って、例えば、２Ｋのオリジナルフレーム２を４Ｋ化する超解像処理を行う場合、参照フレームとして、補間フレーム１．５と補間フレーム２．５を参照することになるが、上記より、これらの補間フレームにはオリジナルフレーム本来の情報が含まれておらず、超解像処理後の４Ｋフレーム２ａの画質が劣化したものとなる。これは、超解像処理後の他の４Ｋフレーム１ａ，３ａ，４ａ，…についても同様である。 Therefore, for example, when performing super-resolution processing for converting the 2K original frame 2 into 4K, the interpolation frame 1.5 and the interpolation frame 2.5 are referred to as reference frames. The original information of the original frame is not included in the frame, and the image quality of the 4K frame 2a after the super-resolution processing is deteriorated. The same applies to the other 4K frames 1a, 3a, 4a,... After the super-resolution processing.
　本発明は、上述のような実情に鑑みてなされたもので、フレームレート変換機能を備えた表示装置において、回路規模を抑制しつつ、画質劣化することなく、複数フレームを用いた超解像処理を行えるようにすることを目的とする。 The present invention has been made in view of the above-described circumstances, and in a display device having a frame rate conversion function, super-resolution processing using a plurality of frames without reducing the image quality while suppressing the circuit scale. The purpose is to be able to.
　上記課題を解決するために、本発明の第１の技術手段は、入力画像信号のフレーム間に、動き補償処理を施した補間画像信号のフレームを内挿することにより、前記入力画像信号のフレーム数を変換するフレームレート変換部を備えた表示装置であって、前記フレームレート変換部によるフレームレート変換後の各画像信号における所定フレームの少なくとも前方または後方のフレームの中から前記動き補償処理を施していない入力画像信号のオリジナルフレームを選択するオリジナルフレーム選択部と、前記フレームレート変換後の各画像信号における前記所定フレームの解像度を、前記オリジナルフレーム選択部により選択されたオリジナルフレームに基づく画素値の補間処理により変換する超解像処理部とを備えたことを特徴としたものである。 In order to solve the above-described problem, the first technical means of the present invention interpolates a frame of an interpolated image signal subjected to motion compensation processing between frames of the input image signal, so that the frame of the input image signal A display device comprising a frame rate conversion unit for converting a number, wherein the motion compensation processing is performed from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion by the frame rate conversion unit. An original frame selection unit that selects an original frame of the input image signal that is not, and a resolution of the predetermined frame in each image signal after the frame rate conversion is set to a pixel value based on the original frame selected by the original frame selection unit. It was equipped with a super-resolution processing unit that converted by interpolation processing Than is.
　第２の技術手段は、第１の技術手段において、前記フレームレート変換部は、前記入力画像信号の先頭から所定数のオリジナルフレームについて、予め定められたフレームレートに従って、前記入力画像信号と同じオリジナルフレームを繰り返し前記オリジナルフレーム選択部に出力し、前記オリジナルフレーム選択部は、前記フレームレート変換部から繰り返し出力されたオリジナルフレームの配置パターンから、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとの内挿パターンを特定し、該特定した内挿パターンに基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することを特徴としたものである。 According to a second technical means, in the first technical means, the frame rate conversion unit applies the same original as the input image signal according to a predetermined frame rate for a predetermined number of original frames from the head of the input image signal. The frame is repeatedly output to the original frame selection unit, and the original frame selection unit interpolates the original frame of the input image signal and the interpolated image signal from the arrangement pattern of the original frame repeatedly output from the frame rate conversion unit. An interpolation pattern with a frame is specified, and an original frame of the input image signal and an interpolation frame of the interpolation image signal are discriminated based on the specified interpolation pattern.
　第３の技術手段は、第１の技術手段において、前記フレームレート変換部は、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとの内挿パターンを示す内挿パターン情報を前記オリジナルフレーム選択部に出力し、前記オリジナルフレーム選択部は、前記内挿パターン情報に基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することを特徴としたものである。 According to a third technical means, in the first technical means, the frame rate conversion unit receives interpolation pattern information indicating an interpolation pattern between an original frame of the input image signal and an interpolation frame of the interpolation image signal. The original frame selection unit is configured to determine an original frame of the input image signal and an interpolation frame of the interpolation image signal based on the interpolation pattern information. is there.
　第４の技術手段は、第１の技術手段において、前記フレームレート変換部は、前記フレームレート変換後の各画像信号のフレームに、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとを識別するための識別情報を付与し、該識別情報が付与された前記各画像信号のフレームを前記オリジナルフレーム選択部に出力し、前記オリジナルフレーム選択部は、前記各画像信号のフレームに付与された識別情報に基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することを特徴としたものである。 According to a fourth technical means, in the first technical means, the frame rate conversion unit includes an original frame of the input image signal, an interpolation frame of the interpolation image signal, and a frame of each image signal after the frame rate conversion. Identification information for identifying the image signal is output, and the frame of each image signal to which the identification information is added is output to the original frame selection unit, and the original frame selection unit is added to the frame of each image signal. Based on the identification information, the original frame of the input image signal and the interpolation frame of the interpolated image signal are discriminated.
　第５の技術手段は、第１～第４のいずれか１の技術手段において、前記フレームレート変換部と前記オリジナルフレーム選択部とでフレームメモリを共有することを特徴としたものである。 The fifth technical means is characterized in that, in any one of the first to fourth technical means, the frame rate conversion unit and the original frame selection unit share a frame memory.
　第６の技術手段は、入力画像信号のフレーム間に、動き補償処理を施した補間画像信号のフレームを内挿することにより、前記入力画像信号のフレーム数を変換するフレームレート変換ステップを備えた超解像処理方法であって、前記フレームレート変換ステップにてフレームレート変換後の各画像信号における所定フレームの少なくとも前方または後方のフレームの中から前記動き補償処理を施していない入力画像信号のオリジナルフレームを選択するオリジナルフレーム選択ステップと、前記フレームレート変換後の各画像信号における前記所定フレームの解像度を、前記オリジナルフレーム選択ステップにて選択されたオリジナルフレームに基づく画素値の補間処理により変換する超解像処理ステップとを備えたことを特徴としたものである。 The sixth technical means includes a frame rate conversion step of converting the number of frames of the input image signal by interpolating a frame of the interpolated image signal subjected to motion compensation processing between the frames of the input image signal. A super-resolution processing method, wherein an original of an input image signal that has not been subjected to the motion compensation processing from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion in the frame rate conversion step An original frame selection step for selecting a frame; and a resolution for converting the resolution of the predetermined frame in each image signal after the frame rate conversion by interpolation processing of pixel values based on the original frame selected in the original frame selection step. And a resolution processing step. It is.
　本発明によれば、フレームレート変換後に、複数フレームを用いた超解像処理を行う際に、参照フレームとして、動き補償処理を施していないオリジナルフレームを選択することができるため、回路規模を抑制しつつ、画質劣化することなく、超解像処理を行って高解像度の画像を得ることができる。 According to the present invention, when performing super-resolution processing using a plurality of frames after frame rate conversion, an original frame that has not been subjected to motion compensation processing can be selected as a reference frame, thereby reducing the circuit scale. However, it is possible to obtain a high-resolution image by performing super-resolution processing without image quality degradation.
本発明による表示装置を液晶テレビに適用した場合の外観の一例を示す図である。It is a figure which shows an example of the external appearance at the time of applying the display apparatus by this invention to a liquid crystal television. 本発明の一実施形態に係る表示装置の構成例を示すブロック図である。It is a block diagram which shows the structural example of the display apparatus which concerns on one Embodiment of this invention. 本発明の第１の実施形態に係る表示装置による超解像処理方法を説明するための図である。It is a figure for demonstrating the super-resolution processing method by the display apparatus which concerns on the 1st Embodiment of this invention. 本発明の第２の実施形態に係る表示装置による超解像処理方法を説明するための図である。It is a figure for demonstrating the super-resolution processing method by the display apparatus which concerns on the 2nd Embodiment of this invention. 入力画像信号の種類毎のフレームリピートパターン及び内挿パターンの一例を説明するための図である。It is a figure for demonstrating an example of the frame repeat pattern and interpolation pattern for every kind of input image signal. 本発明の第３の実施形態に係る表示装置の構成例を示すブロック図である。It is a block diagram which shows the structural example of the display apparatus which concerns on the 3rd Embodiment of this invention. 本発明の第４の実施形態に係る識別情報の一例を説明するための図である。It is a figure for demonstrating an example of the identification information which concerns on the 4th Embodiment of this invention. 本発明の第５の実施形態に係る表示装置の構成例を示すブロック図である。It is a block diagram which shows the structural example of the display apparatus which concerns on the 5th Embodiment of this invention. 従来技術を説明する図である。It is a figure explaining a prior art.
　以下、添付図面を参照しながら、本発明の表示装置及び該装置による超解像処理方法に係る好適な実施の形態について説明する。 Hereinafter, preferred embodiments according to a display device of the present invention and a super-resolution processing method using the device will be described with reference to the accompanying drawings.
　図１は、本発明による表示装置を液晶テレビに適用した場合の外観の一例を示す図で、図中、１は表示装置の一例である液晶テレビを示す。この液晶テレビ１は、リモコンＲにより遠隔操作され、２Ｋ１Ｋ（フルハイビジョン）や４Ｋ２Ｋなどの解像度を持つデジタル放送信号を受信し、その番組映像を表示させることができる。 FIG. 1 is a diagram showing an example of an appearance when a display device according to the present invention is applied to a liquid crystal television. In the figure, 1 shows a liquid crystal television which is an example of a display device. The liquid crystal television 1 is remotely operated by a remote controller R, can receive a digital broadcast signal having a resolution such as 2K1K (full high-definition) or 4K2K, and can display the program video.
（第１の実施形態）
　図２は、本発明の一実施形態に係る表示装置の構成例を示すブロック図である。上記の液晶テレビ１として例示される表示装置（以下、表示装置１という）は、デジタル放送信号を受信するチューナ２と、チューナ２で受信したデジタル放送信号に含まれる入力画像信号のフレーム間に、動き補償処理を施した補間画像信号のフレームを内挿することにより、入力画像信号のフレーム数を変換するフレームレート変換部３と、フレームレート変換部３と接続される第１フレームメモリ４と、フレームレート変換部３によるフレームレート変換後の各画像信号における所定フレームの少なくとも前方または後方のフレームの中から動き補償処理を施していない入力画像信号のオリジナルフレームを選択するオリジナルフレーム選択部５と、オリジナルフレーム選択部５と接続される第２フレームメモリ６と、フレームレート変換後の各画像信号における所定フレームの解像度を、オリジナルフレーム選択部５により選択されたオリジナルフレームに基づく画素値の補間処理により変換する超解像処理部７と、４Ｋ２Ｋなどの高解像度画像を表示可能な高解像度液晶パネルなどで構成される表示部８とを備える。 (First embodiment)
FIG. 2 is a block diagram illustrating a configuration example of a display device according to an embodiment of the present invention. A display device exemplified as the liquid crystal television 1 (hereinafter referred to as the display device 1) includes a tuner 2 that receives a digital broadcast signal and a frame of an input image signal included in the digital broadcast signal received by the tuner 2. A frame rate conversion unit 3 that converts the number of frames of the input image signal by interpolating frames of the interpolated image signal that has undergone motion compensation processing; a first frame memory 4 that is connected to the frame rate conversion unit 3; An original frame selection unit 5 that selects an original frame of an input image signal that has not undergone motion compensation processing from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion by the frame rate conversion unit 3; A second frame memory 6 connected to the original frame selector 5 and a frame rate; A super-resolution processing unit 7 that converts the resolution of a predetermined frame in each image signal after conversion by interpolation processing of pixel values based on the original frame selected by the original frame selection unit 5 and a high-resolution image such as 4K2K are displayed. And a display unit 8 including a possible high-resolution liquid crystal panel.
　第１フレームメモリ４は、フレームレート変換部３での動き補償処理に必要なフレームを格納する。また、オリジナルフレーム選択部５は、超解像処理で参照予定のオリジナルフレームを第２フレームメモリ６に格納し、超解像処理を行う際に必要なオリジナルフレームを第２フレームメモリ６から読み出して超解像処理部７に送る。 The first frame memory 4 stores frames necessary for motion compensation processing in the frame rate conversion unit 3. The original frame selection unit 5 stores the original frame to be referred to in the super-resolution processing in the second frame memory 6, and reads out the original frame necessary for performing the super-resolution processing from the second frame memory 6. The data is sent to the super-resolution processing unit 7.
　本例の場合、デジタル放送信号として、２Ｋ（１９２０画素×１０８０画素），６０Ｈｚのフルハイビジョン画像が入力され、１２０Ｈｚのフレームレート変換後に、複数フレームを用いた超解像処理により４Ｋ（３８４０画素×２１６０画素）の高解像度画像に変換される。そして、超解像処理後の高解像度画像は表示部８に表示される。超解像処理とは、前述したように、大きく分けて、複数の画像から特徴点を抽出し個々の画像で抽出された特徴点同士をマッチングさせて位置を合わせる運動推定処理と、個々の画像を統合して高解像度画像を生成する再構成処理と、画像のぼやけを改善してノイズを削除する鮮明化処理という３段階の処理が必要とされる。上記の運動推定処理では、フレーム間で動きベクトルを算出するため、あるフレームを超解像処理する場合、当該フレームの前方フレーム、または、後方フレーム、または、前方及び後方の両フレームが参照される。どのフレームを参照するかは超解像処理のアルゴリズムに基づき適宜設定することができる。 In the case of this example, a 2K (1920 pixels × 1080 pixels), 60 Hz full high-definition image is input as a digital broadcast signal, and after a frame rate conversion of 120 Hz, 4K (3840 pixels × 2160 pixels). Then, the high-resolution image after the super-resolution processing is displayed on the display unit 8. As described above, the super-resolution processing is roughly divided into a motion estimation process for extracting feature points from a plurality of images and matching the positions by matching the feature points extracted in the individual images, and the individual images. Are required to perform a three-stage process including a reconstruction process for generating a high-resolution image by integrating the two and a sharpening process for improving image blurring and removing noise. In the motion estimation process described above, since a motion vector is calculated between frames, when super-resolution processing is performed on a certain frame, the front frame, the rear frame, or both the front and rear frames of the frame are referred to. . Which frame is to be referred to can be appropriately set based on the super-resolution algorithm.
　図３は、本発明の第１の実施形態に係る表示装置による超解像処理方法を説明するための図である。前述の図９の場合と同様に、２Ｋ，６０Ｈｚのフルハイビジョン画像を構成するオリジナルフレーム１，２，３，４，…が入力された場合、１２０Ｈｚのフレームレート変換の際に、オリジナルフレーム１とオリジナルフレーム２から補間フレーム１．５が生成され、同様にして、オリジナルフレーム２とオリジナルフレーム３から補間フレーム２．５が生成され、オリジナルフレーム３とオリジナルフレーム４から補間フレーム３．５が生成される。 FIG. 3 is a diagram for explaining a super-resolution processing method by the display device according to the first embodiment of the present invention. As in the case of FIG. 9 described above, when original frames 1, 2, 3, 4,... Constituting a 2K, 60 Hz full high-definition image are input, the original frame 1 is converted at the frame rate conversion of 120 Hz. An interpolation frame 1.5 is generated from the original frame 2, and similarly, an interpolation frame 2.5 is generated from the original frame 2 and the original frame 3, and an interpolation frame 3.5 is generated from the original frame 3 and the original frame 4. The
　つまり、オリジナルフレーム１，２，３，４，…は、動き補償処理の施されていない入力画像信号のフレームであり、補間フレーム１．５，２．５，３．５，…は、動き補償処理の施された補間画像信号のフレームである。そして、オリジナルフレーム１，２の間には、補間フレーム１．５が内挿され、オリジナルフレーム２，３の間に、補間フレーム２．５が内挿される。以下同様にオリジナルフレーム間に補間フレームが内挿され、フレームレートが１２０Ｈｚに変換される。 That is, the original frames 1, 2, 3, 4,... Are frames of input image signals that have not been subjected to motion compensation processing, and the interpolation frames 1.5, 2.5, 3.5,. It is a frame of an interpolated image signal that has been processed. An interpolated frame 1.5 is interpolated between the original frames 1 and 2, and an interpolated frame 2.5 is interpolated between the original frames 2 and 3. Similarly, an interpolation frame is interpolated between the original frames, and the frame rate is converted to 120 Hz.
　次に、オリジナルフレーム選択部５は、フレームレート変換後の各画像信号（オリジナルフレームと補間フレーム）における所定フレームの少なくとも前方または後方のフレームの中から、動き補償処理を施していない入力画像信号のオリジナルフレームを選択する。例えば、本例のように、一般的な６０Ｈｚの２Ｋ画像をフレームレート変換して１２０Ｈｚにする場合、フレームレート変換後の各画像信号のフレームは、オリジナルフレーム１、補間フレーム１．５、オリジナルフレーム２、補間フレーム２．５、…の順に配置されることになる。 Next, the original frame selection unit 5 selects an input image signal that has not undergone motion compensation processing from at least a front or rear frame of a predetermined frame in each image signal (original frame and interpolation frame) after frame rate conversion. Select the original frame. For example, as in this example, when a frame rate conversion of a general 2K image of 60 Hz is performed to 120 Hz, the frame of each image signal after the frame rate conversion is an original frame 1, an interpolation frame 1.5, an original frame 2, interpolation frames 2.5,... Are arranged in this order.
　従って、オリジナルフレーム選択部５は、先頭フレーム（オリジナルフレーム１）を選択後に、１フレームおきにフレームを選択していけば、補間フレームを避けて、オリジナルフレームのみを抽出することができる。つまり、フレームの配置パターンが、奇数番目のフレームがオリジナルフレーム、偶数番目のフレームが補間フレームとなることから、奇数番目のフレームを選択していけばよい。図３において、フレームレート変換後のフレーム番号に丸印が付与されているフレームは、オリジナルフレーム選択部５により選択されたオリジナルフレームであることを示す。具体的には、オリジナルフレーム２を上記の所定フレームとした場合、オリジナルフレーム１、または、オリジナルフレーム３、または、オリジナルフレーム１，３の両方のいずれかが選択される。 Therefore, if the original frame selection unit 5 selects every other frame after selecting the first frame (original frame 1), it can extract only the original frame while avoiding the interpolation frame. In other words, the odd numbered frame may be selected because the odd numbered frame is the original frame and the even numbered frame is the interpolated frame. In FIG. 3, a frame in which a circle is added to the frame number after frame rate conversion indicates that the frame is an original frame selected by the original frame selection unit 5. Specifically, when the original frame 2 is the predetermined frame, either the original frame 1, the original frame 3, or both the original frames 1 and 3 are selected.
　次に、超解像処理部７は、フレームレート変換後の各画像信号における所定フレームの解像度を、フレームレート選択部５により選択されたオリジナルフレームに基づく画素値の補間処理により変換する。具体的には、オリジナルフレーム１を超解像処理する場合、オリジナルフレーム２を参照し、オリジナルフレーム２に基づき画素値が補間処理される。そして、超解像処理後のオリジナルフレーム１は４Ｋフレーム１ａとして出力される。同様に、補間フレーム１．５を超解像処理する場合、オリジナルフレーム１とオリジナルフレーム２を参照し、これらオリジナルフレーム１，２に基づき画素値が補間処理される。そして、超解像処理後の補間フレーム１．５は４Ｋフレーム１．５ａとして出力される。さらに、オリジナルフレーム２を超解像処理する場合、オリジナルフレーム１とオリジナルフレーム３を参照し、これらオリジナルフレーム１，３に基づき画素値が補間処理される。そして、超解像処理後のオリジナルフレーム２は４Ｋフレーム２ａとして出力される。以下同様に、フレームレート変換後の各画像信号（オリジナルフレーム及び補間フレーム）に超解像処理が施され、解像度が４Ｋ化される。 Next, the super-resolution processing unit 7 converts the resolution of a predetermined frame in each image signal after frame rate conversion by interpolation processing of pixel values based on the original frame selected by the frame rate selection unit 5. Specifically, when super-resolution processing is performed on the original frame 1, the original frame 2 is referred to, and pixel values are interpolated based on the original frame 2. Then, the original frame 1 after the super-resolution processing is output as a 4K frame 1a. Similarly, when super-resolution processing is performed on the interpolation frame 1.5, the original frame 1 and the original frame 2 are referred to, and pixel values are subjected to interpolation processing based on the original frames 1 and 2. The interpolated frame 1.5 after the super-resolution processing is output as a 4K frame 1.5a. Furthermore, when super-resolution processing is performed on the original frame 2, the original frame 1 and the original frame 3 are referred to, and pixel values are interpolated based on the original frames 1 and 3. The original frame 2 after the super-resolution processing is output as a 4K frame 2a. Similarly, super-resolution processing is performed on each image signal (original frame and interpolation frame) after frame rate conversion, and the resolution is converted to 4K.
　このようにして、超解像処理後の４Ｋフレーム１ａ，１．５ａ，２ａ，２．５ａ，…は、４Ｋ（３８４０画素×２１６０画素），１２０Ｈｚの画像信号として出力される。これらの４Ｋフレームは、超解像処理の際に１つ以上のオリジナルフレームを参照フレームとして生成されているため、回路規模の増大を抑えながら、画質の劣化を防止することができる。 In this way, the 4K frames 1a, 1.5a, 2a, 2.5a,... After super-resolution processing are output as 4K (3840 pixels × 2160 pixels), 120 Hz image signals. Since these 4K frames are generated by using one or more original frames as a reference frame during the super-resolution processing, it is possible to prevent deterioration in image quality while suppressing an increase in circuit scale.
（第２の実施形態）
　ここで、日本のテレビ放送の場合、１秒間に６０フレームを含むテレビ信号となるが、映画の場合には１秒間に２４コマが記録される。従って、映画としてフィルム上に記録された画像をテレビ信号に変換して放送または記録するためには、２４Ｈｚから６０Ｈｚへの変換が必要とされる。このような変換方法として、２：３プルダウンと呼ばれる方法が知られている。これは、フィルム上の各コマから、それぞれ２フレーム、３フレーム、２フレーム、３フレーム、…を生成することで、１秒間に６０フレームのテレビ信号を作り出すものである。また、コンテンツによっては１秒間に３０コマを記録する場合もある。この場合、３０Ｈｚから６０Ｈｚへの変換を行うが、このような変換方法は２：２プルダウンと呼ばれる。これは、フィルム上の各コマから、それぞれ２フレームずつ生成することで、１秒間に６０フレームのテレビ信号を作り出している。 (Second Embodiment)
Here, in the case of Japanese television broadcasting, the television signal includes 60 frames per second, but in the case of a movie, 24 frames are recorded per second. Therefore, in order to convert an image recorded on a film as a movie into a television signal for broadcasting or recording, conversion from 24 Hz to 60 Hz is required. As such a conversion method, a method called 2: 3 pull-down is known. This generates 2 frames, 3 frames, 2 frames, 3 frames,... From each frame on the film, thereby creating a television signal of 60 frames per second. Depending on the content, 30 frames may be recorded per second. In this case, conversion from 30 Hz to 60 Hz is performed. Such a conversion method is called 2: 2 pull-down. This creates 60 frames of TV signals per second by generating 2 frames from each frame on the film.
　上記において、テレビ信号として、通常の２Ｋ画像信号だけでなく、２：３プルダウン変換信号、２：２プルダウン変換信号等が入力される場合について想定する。この場合、フレームレート変換後にオリジナルフレームと補間フレームとが交互に配置されているとは限らない。つまり、２：３プルダウン変換信号、２：２プルダウン変換信号等が入力された場合、オリジナルフレーム選択部５では、上述のように、フレームレート変換後の各画像信号のフレームに対して、１つおきにフレームを抽出していくと、誤って補間フレームを選択してしまう虞がある。 In the above, it is assumed that not only a normal 2K image signal but also a 2: 3 pull-down conversion signal, a 2: 2 pull-down conversion signal, etc. are input as a television signal. In this case, the original frame and the interpolated frame are not necessarily arranged alternately after the frame rate conversion. That is, when a 2: 3 pull-down conversion signal, a 2: 2 pull-down conversion signal, or the like is input, the original frame selection unit 5 applies one for each image signal frame after frame rate conversion as described above. If every other frame is extracted, there is a possibility that an interpolation frame is selected by mistake.
　これを回避するために、本実施形態では、フレームレート変換部３を制御し、先頭から数フレームのみ入力画像と同じフレームを繰り返し出力し、これにより、オリジナルフレーム選択部５がオリジナルフレームの位置を検出できるようにする。すなわち、フレームレート変換部３は、入力画像信号の先頭から所定数のオリジナルフレームについて、予め定められたフレームレートに従って、入力画像信号と同じオリジナルフレームを繰り返しオリジナルフレーム選択部５に出力する。そして、オリジナルフレーム選択部５は、フレームレート変換部３から繰り返し出力されたオリジナルフレームの配置パターンから、入力画像信号のオリジナルフレームと、補間画像信号の補間フレームとの内挿パターンを特定し、特定した内挿パターンに基づいて、入力画像信号のオリジナルフレームと、補間画像信号の補間フレームとを判別する。 In order to avoid this, in the present embodiment, the frame rate conversion unit 3 is controlled to repeatedly output the same frame as the input image for only a few frames from the beginning, whereby the original frame selection unit 5 determines the position of the original frame. Make it detectable. That is, the frame rate conversion unit 3 repeatedly outputs the same original frame as the input image signal to the original frame selection unit 5 according to a predetermined frame rate for a predetermined number of original frames from the beginning of the input image signal. Then, the original frame selection unit 5 specifies an interpolation pattern between the original frame of the input image signal and the interpolation frame of the interpolation image signal from the original frame arrangement pattern repeatedly output from the frame rate conversion unit 3, and specifies Based on the interpolated pattern, the original frame of the input image signal and the interpolated frame of the interpolated image signal are discriminated.
　図４は、本発明の第２の実施形態に係る表示装置による超解像処理方法を説明するための図である。本例では、入力画像信号として、２Ｋ，６０Ｈｚのフルハイビジョン画像を構成するオリジナルフレーム１，２，３，４，…が入力される場合について想定する。フレームレート変換部３は、入力画像信号のフレームに対してフレームレート変換を行うが、入力画像信号の先頭から所定数のオリジナルフレームについて、予め定められたフレームレートに従って、入力画像信号と同じオリジナルフレームを繰り返しオリジナルフレーム選択部５に出力する。なお、ここでは、入力画像信号がフルハイビジョン信号であり、フレームレートが“２倍”であるため、例えば、先頭から２つのオリジナルフレーム１，２が２回ずつ繰り返し出力される。 FIG. 4 is a diagram for explaining a super-resolution processing method by the display device according to the second embodiment of the present invention. In this example, it is assumed that original frames 1, 2, 3, 4,... Constituting a 2K, 60 Hz full high-definition image are input as input image signals. The frame rate conversion unit 3 performs frame rate conversion on the frame of the input image signal. However, for a predetermined number of original frames from the head of the input image signal, the same original frame as that of the input image signal is determined according to a predetermined frame rate. Are repeatedly output to the original frame selector 5. Here, since the input image signal is a full high-definition signal and the frame rate is “double”, for example, the two original frames 1 and 2 are repeatedly output from the head twice.
　なお、フレームレート変換部３は、１２０Ｈｚのフレームレートに従って、先頭から２つのオリジナルフレーム１，２を２回ずつ繰り返し出力した後、オリジナルフレーム３以降について通常のフレームレート変換を行う。つまり、オリジナルフレーム３とオリジナルフレーム４から補間フレーム３．５を生成し、この補間フレーム３．５を、オリジナルフレーム３とオリジナルフレーム４との間に内挿する。以下後続のオリジナルフレームについて同様の処理が実行される。 The frame rate conversion unit 3 repeatedly outputs the two original frames 1 and 2 from the top in accordance with the 120 Hz frame rate, and then performs normal frame rate conversion for the original frame 3 and subsequent frames. That is, an interpolation frame 3.5 is generated from the original frame 3 and the original frame 4, and this interpolation frame 3.5 is inserted between the original frame 3 and the original frame 4. Thereafter, the same processing is executed for the subsequent original frame.
　オリジナルフレーム選択部５は、フレームレート変換部３から繰り返し出力されたオリジナルフレームの配置パターンを判定する。ここで、本例のオリジナルフレーム選択部５では、フレームレート変換部３から出力されたフレームがオリジナルフレームなのか、補間フレームなのか判断できないため、同じフレームが２回繰り返し入力された場合に、当該フレームをオリジナルフレームと判定するものとする。そして、オリジナルフレーム選択部５は、２つのオリジナルフレームが２回ずつ繰り返される配置パターンであると判定した場合、入力画像信号が２Ｋ，６０Ｈｚのフルハイビジョン信号である場合の内挿パターンを特定する。このフルハイビジョン信号の場合、オリジナルフレームと補間フレームとの内挿パターンが一意に特定される。 The original frame selection unit 5 determines the arrangement pattern of the original frames repeatedly output from the frame rate conversion unit 3. Here, since the original frame selection unit 5 of this example cannot determine whether the frame output from the frame rate conversion unit 3 is an original frame or an interpolation frame, when the same frame is repeatedly input, Assume that the frame is determined to be an original frame. When the original frame selection unit 5 determines that the two original frames are arranged patterns that are repeated twice, the original frame selection unit 5 specifies an interpolation pattern when the input image signal is a 2K, 60 Hz full high-definition signal. In the case of this full high-definition signal, the interpolation pattern between the original frame and the interpolation frame is uniquely specified.
　具体的には、奇数番目がオリジナルフレーム、偶数番目が補間フレームとなる内挿パターンであることから、奇数番目のフレームを選択すれば、補間フレームを避けて、オリジナルフレームのみを抽出することができる。つまり、オリジナルフレーム選択部５は、１～４番目までのオリジナルフレーム１，２の配置パターンから、奇数番目がオリジナルフレームとなる内挿パターンを特定し、これにより、５番目のフレーム、すなわち、オリジナルフレーム３を選択する。以下、同様に、７番目のオリジナルフレーム４を選択する。図４において、図３と同様に、フレームレート変換後のフレーム番号に丸印が付与されているフレームは、オリジナルフレーム選択部５により選択されたオリジナルフレームであることを示す。 Specifically, since an odd number is an original frame and an even number is an interpolation pattern, if an odd number frame is selected, only the original frame can be extracted while avoiding the interpolation frame. . That is, the original frame selection unit 5 identifies an interpolation pattern in which the odd number is the original frame from the arrangement patterns of the first to fourth original frames 1 and 2, and thereby the fifth frame, that is, the original frame Select frame 3. Thereafter, similarly, the seventh original frame 4 is selected. In FIG. 4, as in FIG. 3, a frame with a circle added to the frame number after frame rate conversion indicates that it is an original frame selected by the original frame selection unit 5.
　次に、超解像処理部７では、２回ずつ繰り返し出力されたオリジナルフレーム１，２について、オリジナルフレーム１はオリジナルフレーム２を参照フレームとして、オリジナルフレーム２はオリジナルフレーム１を参照フレームとして、それぞれ超解像処理を行い、解像度を４Ｋ化する（４Ｋフレーム１ａ，２ａ）。あるいは、オリジナルフレーム１，２については超解像処理を行わず、最近隣法などのフレーム内複写処理や、線形補間や３次元畳み込み補間などのフレーム内補間処理により、解像度を４Ｋ化してもよい。そして、オリジナルフレーム３以降については、オリジナルフレーム選択部５で選択されたオリジナルフレーム３，４，…を用いて超解像処理を行い、４Ｋフレーム３ａ，３．５ａ，４ａ，４．５ａ，…を生成する。 Next, in the super-resolution processing unit 7, for the original frames 1 and 2 repeatedly output twice, the original frame 1 uses the original frame 2 as a reference frame, the original frame 2 uses the original frame 1 as a reference frame, respectively. Super-resolution processing is performed to make the resolution 4K (4K frames 1a and 2a). Alternatively, the original frames 1 and 2 may not be subjected to super-resolution processing, and the resolution may be changed to 4K by intra-frame copy processing such as nearest neighbor method or intra-frame interpolation processing such as linear interpolation or three-dimensional convolution interpolation. . Then, with respect to the original frame 3 and later, super-resolution processing is performed using the original frames 3, 4,... Selected by the original frame selection unit 5, and 4K frames 3a, 3.5a, 4a, 4.5a,. Is generated.
　なお、入力画像信号が、６０Ｈｚの２：３プルダウン変換信号、２：２プルダウン変換信号、あるいは、２４Ｈｚの映画フィルム信号等である場合も基本的な考え方は同様である。これらの例について以下の図５に基づいて説明する。 The basic concept is the same when the input image signal is a 60 Hz 2: 3 pull-down conversion signal, a 2: 2 pull-down conversion signal, or a 24 Hz movie film signal. These examples will be described with reference to FIG. 5 below.
　図５は、入力画像信号の種類毎のフレームリピートパターン及び内挿パターンの一例を説明するための図である。フレームリピートパターンとは、同一の画像が繰り返し送られる型のことであり、少なくとも１つのシーンについては同一のフレームリピートパターンによって映像が送られる。図５（Ａ）は入力画像信号が６０Ｈｚのビデオ（フルハイビジョン）信号の場合を示し、図５（Ｂ）は入力画像信号が６０Ｈｚの２：３プルダウン変換信号の場合を示し、図５（Ｃ）は入力画像信号が６０Ｈｚの２：２プルダウン変換信号の場合を示し、図５（Ｄ）は入力画像信号が２４Ｈｚの映画フィルム信号の場合を示す。図中、白抜きのフレームはオリジナルフレーム、ハッチングのフレームは補間フレームを示す。なお、本例に示す入力画像信号、内挿パターンは一例であり、これらに限定されるものではない。 FIG. 5 is a diagram for explaining an example of a frame repeat pattern and an interpolation pattern for each type of input image signal. The frame repeat pattern is a type in which the same image is repeatedly sent, and at least one scene is transmitted with the same frame repeat pattern. FIG. 5A shows a case where the input image signal is a 60 Hz video (full high-definition) signal, and FIG. 5B shows a case where the input image signal is a 60 Hz 2: 3 pull-down conversion signal. ) Shows a case where the input image signal is a 2: 2 pull-down conversion signal of 60 Hz, and FIG. 5D shows a case where the input image signal is a movie film signal of 24 Hz. In the figure, a white frame indicates an original frame, and a hatched frame indicates an interpolation frame. The input image signal and the interpolation pattern shown in this example are examples, and the present invention is not limited to these.
　図５（Ａ）の例は、入力画像信号が６０Ｈｚのフルハイビジョン信号の場合の例であり、上述の図４で説明した通りである。すなわち、フレームレート変換部３からは、入力画像信号の先頭から所定数のオリジナルフレーム１，２，３，４，…が２回ずつ繰り返しオリジナルフレーム選択部５に出力される（フレームリピート）。つまり、１つのオリジナルフレーム１について２回繰り返し、さらに、１つのオリジナルフレーム２について２回繰り返す。所定数の決め方により後続のオリジナルフレームについても同様の処理を行う。従って、このフルハイビジョン信号であることを特定するためには、入力画像信号の先頭から少なくとも２つのオリジナルフレーム１，２を２回ずつ繰り返す必要がある。 The example of FIG. 5 (A) is an example in the case where the input image signal is a full high-definition signal of 60 Hz, as described in FIG. 4 above. That is, the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 2, 3, 4,... From the head of the input image signal to the original frame selection unit 5 twice (frame repeat). That is, the process is repeated twice for one original frame 1 and further twice for one original frame 2. The same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify that this is a full high-definition signal, it is necessary to repeat at least two original frames 1 and 2 twice from the beginning of the input image signal.
　そして、この場合の内挿パターンは、奇数番目のオリジナルフレームと偶数番目の補間フレームとが交互に配置される１つのパターンに特定される。つまり、オリジナルフレーム選択部５は、フレームレート変換部３からのフレームリピートパターン（すなわち、配置パターン）から、入力画像信号が６０Ｈｚのフルハイビジョン信号である場合の内挿パターンを特定し、この内挿パターンに基づいて、奇数番目のフレームがオリジナルフレーム、偶数番目のフレームが補間フレームと判別することができる。これにより、補間フレームを避けて、オリジナルフレームのみを選択することができる。 In this case, the interpolation pattern is specified as one pattern in which odd-numbered original frames and even-numbered interpolation frames are alternately arranged. That is, the original frame selection unit 5 specifies an interpolation pattern when the input image signal is a full high-definition signal of 60 Hz from the frame repeat pattern (that is, the arrangement pattern) from the frame rate conversion unit 3, and this interpolation Based on the pattern, it is possible to determine that the odd-numbered frame is the original frame and the even-numbered frame is the interpolation frame. As a result, only the original frame can be selected while avoiding the interpolation frame.
　図５（Ｂ）の例は、入力画像信号が６０Ｈｚの２：３プルダウン変換信号の場合の例である。この場合、フレームレート変換部３からは、入力画像信号の先頭から所定数のオリジナルフレーム１，１，２，２，２，３，…が２回ずつ繰り返しオリジナルフレーム選択部５に出力される（フレームリピート）。つまり、１つのオリジナルフレーム１について２回繰り返すため、オリジナルフレーム１を計４回繰り返し、さらに、１つのオリジナルフレーム２について２回繰り返すため、オリジナルフレーム２を計６回繰り返す。所定数の決め方により後続のオリジナルフレームについても同様の処理を行う。従って、この２：３プルダウン変換信号であることを特定するためには、入力画像信号の先頭から少なくとも２つのオリジナルフレーム１，２をそれぞれ４回、６回繰り返す必要がある。 The example of FIG. 5B is an example in which the input image signal is a 2: 3 pull-down conversion signal of 60 Hz. In this case, the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 1, 2, 2, 2, 3,... From the head of the input image signal to the original frame selection unit 5 twice each time ( Frame repeat). In other words, since one original frame 1 is repeated twice, the original frame 1 is repeated four times in total, and further, since one original frame 2 is repeated twice, the original frame 2 is repeated six times in total. The same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify the 2: 3 pull-down conversion signal, it is necessary to repeat at least two original frames 1 and 2 from the head of the input image signal 4 times and 6 times, respectively.
　そして、この場合の内挿パターンには、内挿１～内挿３などのように複数のパターンが存在するため、どの内挿パターンにするかを予め決めておけばよい。つまり、オリジナルフレーム選択部５は、フレームレート変換部３からのフレームリピートパターンから、入力画像信号が６０Ｈｚの２：３プルダウン変換信号である場合の内挿パターンを特定し、この内挿パターンに基づいて、オリジナルフレームと、補間フレームとを判別することができる。例えば、内挿パターンが「内挿３」であれば、オリジナルフレームは４フレームおきに配置されることになる。すなわち、内挿後のフレームは先頭から１，６，１１，１６，２１，２６，…番目がオリジナルフレームとなるため、この順序に従ってオリジナルフレームのみを選択することができる。 In this case, since there are a plurality of patterns such as interpolation 1 to 3 in the interpolation pattern, it is only necessary to determine in advance which interpolation pattern is to be used. That is, the original frame selection unit 5 identifies an interpolation pattern when the input image signal is a 2: 3 pull-down conversion signal of 60 Hz from the frame repeat pattern from the frame rate conversion unit 3, and based on this interpolation pattern Thus, the original frame and the interpolation frame can be discriminated. For example, if the interpolation pattern is “interpolation 3”, the original frame is arranged every four frames. That is, since the first, sixth, eleventh, sixteenth, sixteenth, twenty-sixth,.
　図５（Ｃ）の例は、入力画像信号が６０Ｈｚの２：２プルダウン変換信号の場合の例である。この場合、フレームレート変換部３からは、入力画像信号の先頭から所定数のオリジナルフレーム１，１，２，２，３，…が２回ずつ繰り返しオリジナルフレーム選択部５に出力される（フレームリピート）。つまり、１つのオリジナルフレーム１について２回繰り返すため、オリジナルフレーム１を計４回繰り返し、さらに、１つのオリジナルフレーム２について２回繰り返すため、オリジナルフレーム２を計４回繰り返す。所定数の決め方により後続のオリジナルフレームについても同様の処理を行う。従って、この２：２プルダウン変換信号であることを特定するためには、入力画像信号の先頭から少なくとも２つのオリジナルフレーム１，２をそれぞれ４回ずつ繰り返す必要がある。 The example of FIG. 5C is an example in which the input image signal is a 2: 2 pull-down conversion signal of 60 Hz. In this case, the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 1, 2, 2, 3,... From the head of the input image signal to the original frame selection unit 5 twice (frame repeat). ). That is, since one original frame 1 is repeated twice, the original frame 1 is repeated four times in total, and further, since one original frame 2 is repeated twice, the original frame 2 is repeated four times in total. The same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify the 2: 2 pull-down conversion signal, it is necessary to repeat at least two original frames 1 and 2 four times from the beginning of the input image signal.
　そして、この場合の内挿パターンには、内挿１～内挿２などのように複数のパターンが存在するため、どの内挿パターンにするかを予め決めておけばよい。つまり、オリジナルフレーム選択部５は、フレームレート変換部３からのフレームリピートパターンから、入力画像信号が６０Ｈｚの２：２プルダウン変換信号である場合の内挿パターンを特定し、この内挿パターンに基づいて、オリジナルフレームと、補間フレームとを判別することができる。例えば、内挿パターンが「内挿２」であれば、オリジナルフレームは３フレームおきに配置されることになる。すなわち、内挿後のフレームは先頭から１，５，９，１３，１７，２１，…番目がオリジナルフレームとなるため、この順序に従ってオリジナルフレームのみを選択することができる。 In this case, since there are a plurality of patterns such as interpolation 1 to interpolation 2 in the interpolation pattern, it is only necessary to determine in advance which interpolation pattern is used. That is, the original frame selection unit 5 identifies an interpolation pattern when the input image signal is a 2: 2 pull-down conversion signal of 60 Hz from the frame repeat pattern from the frame rate conversion unit 3, and based on this interpolation pattern Thus, the original frame and the interpolation frame can be discriminated. For example, if the interpolation pattern is “interpolation 2”, the original frame is arranged every three frames. That is, since the first, fifth, ninth, thirteenth, seventeenth, twenty-first,.
　図５（Ｄ）の例は、入力画像信号が２４Ｈｚの映画フィルム信号の場合の例である。この場合、フレームレート変換部３からは、入力画像信号の先頭から所定数のオリジナルフレーム１，２，３，…が５回ずつ繰り返しオリジナルフレーム選択部５に出力される（フレームリピート）。つまり、１つのオリジナルフレーム１について５回繰り返し、さらに、１つのオリジナルフレーム２について５回繰り返す。所定数の決め方により後続のオリジナルフレームについても同様の処理を行う。従って、この映画フィルム信号であることを特定するためには、入力画像信号の先頭から少なくとも２つのオリジナルフレーム１，２を５回ずつ繰り返す必要がある。 5D is an example when the input image signal is a movie film signal of 24 Hz. In this case, the frame rate conversion unit 3 repeatedly outputs a predetermined number of original frames 1, 2, 3,... From the beginning of the input image signal to the original frame selection unit 5 five times (frame repeat). That is, it repeats 5 times for one original frame 1 and further 5 times for one original frame 2. The same processing is performed for the subsequent original frames according to a predetermined number of determination methods. Therefore, in order to specify the movie film signal, it is necessary to repeat at least two original frames 1 and 2 five times from the beginning of the input image signal.
　そして、この場合の内挿パターンは、内挿１～内挿２などのように複数のパターンが存在するため、どの内挿パターンにするかを予め決めておけばよい。つまり、オリジナルフレーム選択部５は、フレームレート変換部３からのフレームリピートパターンから、入力画像信号が２４Ｈｚの映画フィルム信号である場合の内挿パターンを特定し、この内挿パターンに基づいて、オリジナルフレームと、補間フレームとを判別することができる。例えば、内挿パターンが「内挿１」であれば、オリジナルフレームは４フレームおきに配置されることになる。すなわち、内挿後のフレームは先頭から１，６，１１，１６，２１，２６，…番目がオリジナルフレームとなるため、この順序に従ってオリジナルフレームのみを選択することができる。 In this case, since there are a plurality of interpolation patterns such as interpolation 1 to interpolation 2 and the like, it is sufficient to determine in advance which interpolation pattern to use. That is, the original frame selection unit 5 identifies an interpolation pattern when the input image signal is a 24 Hz movie film signal from the frame repeat pattern from the frame rate conversion unit 3, and based on this interpolation pattern, A frame and an interpolation frame can be discriminated. For example, if the interpolation pattern is “interpolation 1”, the original frame is arranged every four frames. That is, since the first, sixth, eleventh, sixteenth, sixteenth, twenty-sixth,.
　上記において、フレームリピートの際に、フレームレート変換部３では、図５（Ｂ）の２：３プルダウン変換信号のときが最も長く内挿を中断することになるが、最大でも１１フレーム分に過ぎないため、画質に与える影響は極めて少ないものと考えられる。
　本実施形態によれば、オリジナルフレームの選択を簡単にすると共に、フレームレート変換部３と、オリジナルフレーム選択部５及び超解像処理部７とが別々のチップに実装され、情報の共有が困難な場合に特に有効である。 In the above, in the frame repeat, the frame rate conversion unit 3 interrupts the interpolation for the longest time in the case of the 2: 3 pull-down conversion signal in FIG. 5B, but it is only 11 frames at the maximum. Therefore, the effect on image quality is considered to be extremely small.
According to the present embodiment, the selection of the original frame is simplified, and the frame rate conversion unit 3, the original frame selection unit 5 and the super-resolution processing unit 7 are mounted on separate chips, making it difficult to share information. It is particularly effective in such cases.
（第３の実施形態）
　図６は、本発明の第３の実施形態に係る表示装置の構成例を示すブロック図である。上述の第２の実施形態では、入力画像信号の先頭から数フレームのみ入力画像と同じフレームを繰り返し出力することで、オリジナルフレームの位置を検出していたが、本実施形態では、オリジナルフレームの位置の検出に内挿パターン情報９を用いる点が異なる。すなわち、フレームレート変換部３は、入力画像信号のオリジナルフレームと補間画像信号の補間フレームとの内挿パターンを示す内挿パターン情報９をオリジナルフレーム選択部５に出力する。そして、オリジナルフレーム選択部５は、フレームレート変換部３からの内挿パターン情報９に基づいて、入力画像信号のオリジナルフレームと、補間画像信号の補間フレームとを判別する。 (Third embodiment)
FIG. 6 is a block diagram showing a configuration example of a display device according to the third embodiment of the present invention. In the second embodiment described above, the position of the original frame is detected by repeatedly outputting the same frame as the input image for only a few frames from the beginning of the input image signal. However, in this embodiment, the position of the original frame is detected. The difference is that the interpolated pattern information 9 is used for detection. That is, the frame rate conversion unit 3 outputs the interpolation pattern information 9 indicating the interpolation pattern between the original frame of the input image signal and the interpolation frame of the interpolation image signal to the original frame selection unit 5. Then, the original frame selection unit 5 determines an original frame of the input image signal and an interpolation frame of the interpolation image signal based on the interpolation pattern information 9 from the frame rate conversion unit 3.
　上記の内挿パターン情報９として、具体的には、図５に示した内挿パターンを例示できる。この内挿パターンによれば、入力画像信号の種類毎に、オリジナルフレームと補間フレームの配置関係が分かるため、補間フレームを避けて、オリジナルフレームのみを選択することが可能となる。フレームレート変換部３は、入力画像信号が例えば６０Ｈｚのフルハイビジョン信号であった場合、フルハイビジョン信号の内挿パターン情報（図５（Ａ）を参照）をオリジナルフレーム選択部５に出力する。オリジナルフレーム選択部５は、この内挿パターン情報から、奇数番目がオリジナルフレーム、偶数番目が補間フレームであると特定する。同様に、入力画像信号が６０Ｈｚの２：３プルダウン変換信号、２：２プルダウン変換信号、２４Ｈｚの映画フィルム信号などである場合にも、それぞれの内挿パターン情報からオリジナルフレームの位置を検出することができる。
　本実施形態によれば、オリジナルフレームの選択のために内挿パターン情報を送るだけでよいため、オリジナルフレームの選択をより簡単に行うことができる。 Specifically, the interpolation pattern information 9 can be exemplified by the interpolation pattern shown in FIG. According to this interpolation pattern, since the arrangement relationship between the original frame and the interpolation frame is known for each type of input image signal, only the original frame can be selected while avoiding the interpolation frame. When the input image signal is a full high-definition signal of 60 Hz, for example, the frame rate conversion unit 3 outputs interpolation pattern information (see FIG. 5A) of the full high-definition signal to the original frame selection unit 5. From this interpolation pattern information, the original frame selection unit 5 specifies that the odd number is the original frame and the even number is the interpolation frame. Similarly, when the input image signal is a 60 Hz 2: 3 pulldown conversion signal, 2: 2 pulldown conversion signal, 24 Hz movie film signal, or the like, the position of the original frame is detected from the respective interpolation pattern information. Can do.
According to the present embodiment, since it is only necessary to send the interpolation pattern information for selecting the original frame, the original frame can be selected more easily.
（第４の実施形態）
　また、第４の実施形態として、フレームレート変換部３は、フレームレート変換後の各画像信号のフレームに、入力画像信号のオリジナルフレームと補間画像信号の補間フレームとを識別するための識別情報を付与し、識別情報が付与された各画像信号のフレームをオリジナルフレーム選択部５に出力するようにしてもよい。そして、オリジナルフレーム選択部５は、各画像信号のフレームに付与された識別情報に基づいて、入力画像信号のオリジナルフレームと、補間画像信号の補間フレームとを判別する。 (Fourth embodiment)
As a fourth embodiment, the frame rate conversion unit 3 adds identification information for identifying the original frame of the input image signal and the interpolation frame of the interpolated image signal to each frame of the image signal after the frame rate conversion. The frame of each image signal to which the identification information is added may be output to the original frame selection unit 5. Then, the original frame selection unit 5 determines the original frame of the input image signal and the interpolation frame of the interpolated image signal based on the identification information given to the frame of each image signal.
　本実施形態の識別情報としては、例えば、フレームレート変換後の各画像信号のフレームが、オリジナルフレームであれば「０」を付与し、補間フレームであれば「１」を付与することが考えられる。オリジナルフレーム選択部５は、各画像信号のフレームに付与された識別情報が「０」であれば、当該フレームをオリジナルフレームと判定し、また、識別情報が「１」であれば、当該フレームを補間フレームと判定する。これにより、補間フレームを避けて、オリジナルフレームのみを選択することが可能となる。
　本実施形態によれば、オリジナルフレームの選択を簡単且つ確実に行うことができる。 As the identification information of the present embodiment, for example, “0” is assigned if the frame of each image signal after frame rate conversion is an original frame, and “1” is assigned if the frame is an interpolation frame. . The original frame selection unit 5 determines that the frame is an original frame if the identification information given to the frame of each image signal is “0”, and if the identification information is “1”, the original frame is selected. It is determined as an interpolation frame. This makes it possible to select only the original frame while avoiding the interpolation frame.
According to this embodiment, the original frame can be selected easily and reliably.
　ここで、フレーム内に識別情報を付与する具体的な方法について図７に基づき説明する。図７（Ａ）は識別情報が付与されていない画像データの例を示し、図７（Ｂ）は識別情報が付与された画像データの例を示す。この識別情報を、例えば、画像データ（１フレーム）の第１ライン先頭にヘッダ込みで数バイトのデータとして埋め込む方法が考えられる。１フレームの第１ライン先頭であれば、画面上の端部分なので目立つことがなく、また、複数フレームで継続的に同様のデータが並ぶことは極めてまれであることから、埋め込んだデータを識別情報として認識することができる。このようなデータの例としては、４バイトデータでＦＲＭ０、ＦＲＭ１を用いることができる。例えば、ＦＲＭ０であれば補間フレーム、ＦＲＭ１であればオリジナルフレームと判定することができる。図７（Ｂ）の例では１ライン目左端の４バイトが識別情報となっている。 Here, a specific method for providing identification information in a frame will be described with reference to FIG. FIG. 7A shows an example of image data to which identification information is not given, and FIG. 7B shows an example of image data to which identification information is given. For example, a method of embedding this identification information as data of several bytes including a header at the head of the first line of image data (one frame) can be considered. If it is the beginning of the first line of one frame, it is not noticeable because it is an end portion on the screen, and it is extremely rare that the same data is continuously arranged in a plurality of frames. Can be recognized as. As an example of such data, FRM0 and FRM1 can be used with 4-byte data. For example, if it is FRM0, it can be determined as an interpolation frame, and if it is FRM1, it can be determined as an original frame. In the example of FIG. 7B, the 4 bytes at the left end of the first line are identification information.
（第５の実施形態）
　図８は、本発明の第５の実施形態に係る表示装置の構成例を示すブロック図である。本実施形態では、フレームレート変換部３とオリジナルフレーム選択部５とでフレームメモリ１０を共有する。この場合、フレームメモリ１０には基本的にオリジナルフレームのみが格納されるため、オリジナルフレーム選択部５は、このフレームメモリ１０から超解像処理に必要なオリジナルフレームを選択すればよい。なお、オリジナルフレームの選択方法は、前述のように別個のフレームメモリを用いた場合と同様である。
　本実施形態によれば、オリジナルフレームの選択を簡単にすると共に、１つのフレームメモリで済むため、フレームメモリの節約になる。 (Fifth embodiment)
FIG. 8 is a block diagram showing a configuration example of a display device according to the fifth embodiment of the present invention. In the present embodiment, the frame rate conversion unit 3 and the original frame selection unit 5 share the frame memory 10. In this case, since only the original frame is basically stored in the frame memory 10, the original frame selection unit 5 may select the original frame necessary for the super-resolution processing from the frame memory 10. The original frame selection method is the same as that when a separate frame memory is used as described above.
According to the present embodiment, the selection of the original frame is simplified and only one frame memory is required, so that the frame memory can be saved.
　以上説明したように、本発明による表示装置は、入力画像信号のフレーム間に、動き補償処理を施した補間画像信号のフレームを内挿することにより、前記入力画像信号のフレーム数を変換するフレームレート変換部を備えた表示装置であって、前記フレームレート変換部によるフレームレート変換後の各画像信号における所定フレームの少なくとも前方または後方のフレームの中から前記動き補償処理を施していない入力画像信号のオリジナルフレームを選択するオリジナルフレーム選択部と、前記フレームレート変換後の各画像信号における前記所定フレームの解像度を、前記オリジナルフレーム選択部により選択されたオリジナルフレームに基づく画素値の補間処理により変換する超解像処理部とを備える。これによれば、フレームレート変換後に、複数フレームを用いた超解像処理を行う際に、参照フレームとして、動き補償処理を施していないオリジナルフレームを選択することができるため、回路規模を抑制しつつ、画質劣化することなく、超解像処理を行って高解像度の画像を得ることができる。 As described above, the display device according to the present invention converts the number of frames of the input image signal by interpolating the frames of the interpolated image signal subjected to the motion compensation process between the frames of the input image signal. An input image signal that is not subjected to the motion compensation processing from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion by the frame rate conversion unit. An original frame selection unit for selecting the original frame, and the resolution of the predetermined frame in each image signal after the frame rate conversion is converted by interpolation processing of pixel values based on the original frame selected by the original frame selection unit A super-resolution processing unit. According to this, when performing super-resolution processing using a plurality of frames after frame rate conversion, an original frame that has not been subjected to motion compensation processing can be selected as a reference frame. However, it is possible to obtain a high-resolution image by performing super-resolution processing without image quality deterioration.
　また、前記フレームレート変換部は、前記入力画像信号の先頭から所定数のオリジナルフレームについて、予め定められたフレームレートに従って、前記入力画像信号と同じオリジナルフレームを繰り返し前記オリジナルフレーム選択部に出力し、前記オリジナルフレーム選択部は、前記フレームレート変換部から繰り返し出力されたオリジナルフレームの配置パターンから、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとの内挿パターンを特定し、該特定した内挿パターンに基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することが望ましい。これによれば、オリジナルフレームの選択を簡単に行うことができると共に、フレームレート変換部と、オリジナルフレーム選択部及び超解像処理部とが別々のチップに実装され、情報の共有が困難な場合に特に有効である。 The frame rate conversion unit repeatedly outputs the same original frame as the input image signal to the original frame selection unit according to a predetermined frame rate for a predetermined number of original frames from the beginning of the input image signal, The original frame selection unit specifies an interpolation pattern between the original frame of the input image signal and the interpolation frame of the interpolation image signal from the arrangement pattern of the original frame repeatedly output from the frame rate conversion unit, It is desirable to discriminate between the original frame of the input image signal and the interpolation frame of the interpolated image signal based on the identified interpolation pattern. According to this, when the original frame can be easily selected and the frame rate conversion unit, the original frame selection unit and the super-resolution processing unit are mounted on separate chips, it is difficult to share information. Is particularly effective.
　また、前記フレームレート変換部は、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとの内挿パターンを示す内挿パターン情報を前記オリジナルフレーム選択部に出力し、前記オリジナルフレーム選択部は、前記内挿パターン情報に基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することが望ましい。これによれば、オリジナルフレームの選択のために内挿パターン情報を送るだけでよいため、オリジナルフレームの選択をより簡単に行うことができる。 Further, the frame rate conversion unit outputs interpolation pattern information indicating an interpolation pattern between an original frame of the input image signal and an interpolation frame of the interpolation image signal to the original frame selection unit, and the original frame selection unit Preferably, based on the interpolation pattern information, an original frame of the input image signal and an interpolation frame of the interpolated image signal are discriminated. According to this, since it is only necessary to send the interpolation pattern information for selecting the original frame, the original frame can be selected more easily.
　また、前記フレームレート変換部は、前記フレームレート変換後の各画像信号のフレームに、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとを識別するための識別情報を付与し、該識別情報が付与された前記各画像信号のフレームを前記オリジナルフレーム選択部に出力し、前記オリジナルフレーム選択部は、前記各画像信号のフレームに付与された識別情報に基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することが望ましい。これによれば、オリジナルフレームの選択を簡単且つ確実に行うことができる。 Further, the frame rate conversion unit gives identification information for identifying the original frame of the input image signal and the interpolation frame of the interpolated image signal to the frame of each image signal after the frame rate conversion, The frame of each image signal to which identification information is assigned is output to the original frame selection unit, and the original frame selection unit is configured to output the input image signal based on the identification information given to the frame of each image signal. It is desirable to discriminate between the original frame and the interpolated frame of the interpolated image signal. According to this, it is possible to easily and reliably select the original frame.
　また、前記フレームレート変換部と前記オリジナルフレーム選択部とでフレームメモリを共有することが望ましい。これによれば、オリジナルフレームの選択を簡単にすると共に、フレームメモリを節約することができる。 In addition, it is desirable that the frame rate converter and the original frame selector share a frame memory. According to this, the selection of the original frame can be simplified and the frame memory can be saved.
　入力画像信号のフレーム間に、動き補償処理を施した補間画像信号のフレームを内挿することにより、前記入力画像信号のフレーム数を変換するフレームレート変換ステップを備えた超解像処理方法であって、前記フレームレート変換ステップにてフレームレート変換後の各画像信号における所定フレームの少なくとも前方または後方のフレームの中から前記動き補償処理を施していない入力画像信号のオリジナルフレームを選択するオリジナルフレーム選択ステップと、前記フレームレート変換後の各画像信号における前記所定フレームの解像度を、前記オリジナルフレーム選択ステップにて選択されたオリジナルフレームに基づく画素値の補間処理により変換する超解像処理ステップとを備える。これによれば、上記と同様に、フレームレート変換後に、複数フレームを用いた超解像処理を行う際に、参照フレームとして、動き補償処理を施していないオリジナルフレームを選択することができるため、回路規模を抑制しつつ、画質劣化することなく、超解像処理を行って高解像度の画像を得ることができる。 A super-resolution processing method comprising a frame rate conversion step of converting the number of frames of an input image signal by interpolating a frame of an interpolated image signal subjected to motion compensation processing between frames of the input image signal. The original frame selection for selecting the original frame of the input image signal not subjected to the motion compensation process from at least the front or rear frame of the predetermined frame in each image signal after the frame rate conversion in the frame rate conversion step And a super-resolution processing step of converting the resolution of the predetermined frame in each image signal after the frame rate conversion by interpolation processing of pixel values based on the original frame selected in the original frame selection step. . According to this, similarly to the above, when performing super-resolution processing using a plurality of frames after frame rate conversion, an original frame that has not been subjected to motion compensation processing can be selected as a reference frame. It is possible to obtain a high-resolution image by performing super-resolution processing without reducing the image quality while suppressing the circuit scale.
１…表示装置、２…チューナ、３…フレームレート変換部、４…第１フレームメモリ、５…オリジナルフレーム選択部、６…第２フレームメモリ、７…超解像処理部、８…表示部、９…内挿パターン情報、１０…フレームメモリ。 DESCRIPTION OF SYMBOLS 1 ... Display apparatus, 2 ... Tuner, 3 ... Frame rate conversion part, 4 ... 1st frame memory, 5 ... Original frame selection part, 6 ... 2nd frame memory, 7 ... Super-resolution processing part, 8 ... Display part, 9 ... Interpolation pattern information, 10 ... Frame memory.

Claims (6)

  1. 　入力画像信号のフレーム間に、動き補償処理を施した補間画像信号のフレームを内挿することにより、前記入力画像信号のフレーム数を変換するフレームレート変換部を備えた表示装置であって、
    　前記フレームレート変換部によるフレームレート変換後の各画像信号における所定フレームの少なくとも前方または後方のフレームの中から前記動き補償処理を施していない入力画像信号のオリジナルフレームを選択するオリジナルフレーム選択部と、前記フレームレート変換後の各画像信号における前記所定フレームの解像度を、前記オリジナルフレーム選択部により選択されたオリジナルフレームに基づく画素値の補間処理により変換する超解像処理部とを備えたことを特徴とする表示装置。     A display device comprising a frame rate conversion unit for converting the number of frames of the input image signal by interpolating a frame of the interpolated image signal subjected to motion compensation processing between frames of the input image signal,
    An original frame selection unit that selects an original frame of an input image signal that has not been subjected to the motion compensation process from at least a front frame or a rear frame of a predetermined frame in each image signal after frame rate conversion by the frame rate conversion unit; A super-resolution processing unit that converts the resolution of the predetermined frame in each image signal after the frame rate conversion by interpolation processing of pixel values based on the original frame selected by the original frame selection unit. Display device.
  2. 　請求項１に記載の表示装置において、前記フレームレート変換部は、前記入力画像信号の先頭から所定数のオリジナルフレームについて、予め定められたフレームレートに従って、前記入力画像信号と同じオリジナルフレームを繰り返し前記オリジナルフレーム選択部に出力し、
    　前記オリジナルフレーム選択部は、前記フレームレート変換部から繰り返し出力されたオリジナルフレームの配置パターンから、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとの内挿パターンを特定し、該特定した内挿パターンに基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することを特徴とする表示装置。     2. The display device according to claim 1, wherein the frame rate conversion unit repeatedly performs the same original frame as the input image signal according to a predetermined frame rate for a predetermined number of original frames from the top of the input image signal. Output to the original frame selector,
    The original frame selection unit specifies an interpolation pattern between the original frame of the input image signal and the interpolation frame of the interpolated image signal from the original frame arrangement pattern repeatedly output from the frame rate conversion unit, and A display device, wherein an original frame of the input image signal and an interpolation frame of the interpolated image signal are discriminated based on the interpolated pattern.
  3. 　請求項１に記載の表示装置において、前記フレームレート変換部は、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとの内挿パターンを示す内挿パターン情報を前記オリジナルフレーム選択部に出力し、
    　前記オリジナルフレーム選択部は、前記内挿パターン情報に基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することを特徴とする表示装置。     2. The display device according to claim 1, wherein the frame rate conversion unit supplies interpolation pattern information indicating an interpolation pattern between an original frame of the input image signal and an interpolation frame of the interpolation image signal to the original frame selection unit. Output,
    The display device according to claim 1, wherein the original frame selection unit determines an original frame of the input image signal and an interpolation frame of the interpolation image signal based on the interpolation pattern information.
  4. 　請求項１に記載の表示装置において、前記フレームレート変換部は、前記フレームレート変換後の各画像信号のフレームに、前記入力画像信号のオリジナルフレームと前記補間画像信号の補間フレームとを識別するための識別情報を付与し、該識別情報が付与された前記各画像信号のフレームを前記オリジナルフレーム選択部に出力し、
    　前記オリジナルフレーム選択部は、前記各画像信号のフレームに付与された識別情報に基づいて、前記入力画像信号のオリジナルフレームと、前記補間画像信号の補間フレームとを判別することを特徴とする表示装置。     2. The display device according to claim 1, wherein the frame rate conversion unit identifies an original frame of the input image signal and an interpolation frame of the interpolation image signal as a frame of each image signal after the frame rate conversion. The identification information, and outputs the frame of each image signal with the identification information to the original frame selection unit,
    The original frame selection unit determines an original frame of the input image signal and an interpolation frame of the interpolated image signal based on identification information given to the frame of each image signal. .
  5. 　請求項１～４のいずれか１項に記載の表示装置において、前記フレームレート変換部と前記オリジナルフレーム選択部とでフレームメモリを共有することを特徴とする表示装置。 5. The display device according to claim 1, wherein a frame memory is shared between the frame rate conversion unit and the original frame selection unit.
  6. 　入力画像信号のフレーム間に、動き補償処理を施した補間画像信号のフレームを内挿することにより、前記入力画像信号のフレーム数を変換するフレームレート変換ステップを備えた超解像処理方法であって、
    　前記フレームレート変換ステップにてフレームレート変換後の各画像信号における所定フレームの少なくとも前方または後方のフレームの中から前記動き補償処理を施していない入力画像信号のオリジナルフレームを選択するオリジナルフレーム選択ステップと、前記フレームレート変換後の各画像信号における前記所定フレームの解像度を、前記オリジナルフレーム選択ステップにて選択されたオリジナルフレームに基づく画素値の補間処理により変換する超解像処理ステップとを備えたことを特徴とする超解像処理方法。     A super-resolution processing method comprising a frame rate conversion step of converting the number of frames of an input image signal by interpolating a frame of an interpolated image signal subjected to motion compensation processing between frames of the input image signal. And
    An original frame selection step of selecting an original frame of the input image signal that has not been subjected to the motion compensation processing from at least a front or rear frame of a predetermined frame in each image signal after the frame rate conversion in the frame rate conversion step; And a super-resolution processing step of converting the resolution of the predetermined frame in each image signal after the frame rate conversion by interpolation processing of pixel values based on the original frame selected in the original frame selection step. A super-resolution processing method characterized by the above.
PCT/JP2014/077033 2013-10-10 2014-10-09 Display device and super-resolution processing method WO2015053352A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013-212908 2013-10-10
JP2013212908 2013-10-10

Publications (1)

Publication Number Publication Date
WO2015053352A1 true WO2015053352A1 (en) 2015-04-16

Family

ID=52813167

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/077033 WO2015053352A1 (en) 2013-10-10 2014-10-09 Display device and super-resolution processing method

Country Status (1)

Country Link
WO (1) WO2015053352A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018142945A1 (en) * 2017-01-31 2018-08-09 ソニー株式会社 Information processing device and method
CN111147893A (en) * 2018-11-02 2020-05-12 华为技术有限公司 Video self-adaption method, related equipment and storage medium
CN111182268A (en) * 2016-01-29 2020-05-19 深圳市大疆创新科技有限公司 Video data transmission method, system, equipment and shooting device
US11024016B2 (en) 2019-04-05 2021-06-01 Samsung Electronics Co., Ltd. Image processing apparatus and image processing method thereof

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006041603A (en) * 2004-07-22 2006-02-09 Seiko Epson Corp Image processing apparatus, image processing method, and image processing program
JP2007151080A (en) * 2005-10-27 2007-06-14 Canon Inc Image processor and image processing method
JP2008160565A (en) * 2006-12-25 2008-07-10 Toshiba Corp Image display apparatus, image signal processing apparatus, and image signal processing method
JP2009005119A (en) * 2007-06-22 2009-01-08 Sanyo Electric Co Ltd Image processing method, image processing apparatus, and electronic appliance with image processing apparatus
JP2009296193A (en) * 2008-06-04 2009-12-17 Hitachi Ltd Image signal processing method and apparatus

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006041603A (en) * 2004-07-22 2006-02-09 Seiko Epson Corp Image processing apparatus, image processing method, and image processing program
JP2007151080A (en) * 2005-10-27 2007-06-14 Canon Inc Image processor and image processing method
JP2008160565A (en) * 2006-12-25 2008-07-10 Toshiba Corp Image display apparatus, image signal processing apparatus, and image signal processing method
JP2009005119A (en) * 2007-06-22 2009-01-08 Sanyo Electric Co Ltd Image processing method, image processing apparatus, and electronic appliance with image processing apparatus
JP2009296193A (en) * 2008-06-04 2009-12-17 Hitachi Ltd Image signal processing method and apparatus

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111182268A (en) * 2016-01-29 2020-05-19 深圳市大疆创新科技有限公司 Video data transmission method, system, equipment and shooting device
CN111182268B (en) * 2016-01-29 2021-08-17 深圳市大疆创新科技有限公司 Video data transmission method, system, equipment and shooting device
WO2018142945A1 (en) * 2017-01-31 2018-08-09 ソニー株式会社 Information processing device and method
CN111147893A (en) * 2018-11-02 2020-05-12 华为技术有限公司 Video self-adaption method, related equipment and storage medium
CN111147893B (en) * 2018-11-02 2021-10-22 华为技术有限公司 Video self-adaption method, related equipment and storage medium
US11509860B2 (en) 2018-11-02 2022-11-22 Huawei Technologies Co., Ltd. Video adaptation method, related device, and storage medium
US11024016B2 (en) 2019-04-05 2021-06-01 Samsung Electronics Co., Ltd. Image processing apparatus and image processing method thereof

Similar Documents

Publication Publication Date Title
US8300087B2 (en) Method and system for response time compensation for 3D video processing
US8385422B2 (en) Image processing apparatus and image processing method
JP5639089B2 (en) Method and apparatus for displaying picture sequences
WO2011039920A1 (en) Three-dimensional image processing device and control method therefor
JP2010041337A (en) Image processing unit and image processing method
JP2003501902A (en) Video signal encoding
WO2015053352A1 (en) Display device and super-resolution processing method
JP2005192199A (en) Real time data stream processor
JP2014077993A (en) Display device
EP2309766A2 (en) Method and system for rendering 3D graphics based on 3D display capabilities
US20110069225A1 (en) Method and system for transmitting and processing high definition digital video signals
JP2008311981A (en) Image signal processing apparatus, image signal processing method, program, camera instrument, image display apparatus, and image signal output apparatus
JP4966400B2 (en) Image output apparatus and image output method
JP2007180765A (en) Video processor, video display device and video processing method
JP2005045787A (en) Video signal processing apparatus to generate both progressive and interlace video signals
JP4548518B2 (en) Video signal display system, video signal reproduction apparatus, and video signal display method
US20100079666A1 (en) Video processing apparatus and video processing method
JP4747214B2 (en) Video signal processing apparatus and video signal processing method
US8270773B2 (en) Image processing apparatus and image processing method
KR100943902B1 (en) Ordinary image processing apparatus for digital TV monitor
JP4869302B2 (en) Image processing apparatus and image processing method
JP5328549B2 (en) Image processing apparatus and control method thereof
JP4991884B2 (en) Image processing apparatus and image processing method
JP2006332904A (en) Contour emphasizing circuit
US20090175600A1 (en) Method of scaling subpicture data and related apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14852651

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14852651

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP