JP2012015669A - Display device and image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a failure, with respect to a video which already has the failure when the video is input to a device.SOLUTION: A display device comprises: a determination unit for determining the failure of the video by detecting discontinuity of an edge included in an input video; a video processing unit which subjects the input video to processing for higher image quality or processing for higher frame rate; and a display unit for displaying a video processed by the video processing unit. The video processing unit, in accordance with the determination result of the determination unit, changes the strength and level of the processing for the higher image quality or changes the generation processing of an interpolation frame in the processing for the higher frame rate.

Description

The present invention relates to an image (video) image quality enhancement process.

  In recent years, the number of video frames has been increased and the resolution in the time direction has been increased to improve the unnaturalness such as blurring and rattling in moving image display. In many cases, a rate conversion processing function is provided. However, the frame rate conversion processing function requires a very advanced technique of predicting and creating a frame that does not exist originally, and a part of the video after the frame rate conversion processing becomes a video (image) different from the original (hereinafter referred to as “video”). This phenomenon may be expressed as “video failure” or “image failure”).

  For example, Patent Literature 1 discloses a technique for improving a failure caused by a frame rate conversion process in a television receiver or a display device.

JP2009-278578

  However, the technique disclosed in Patent Document 1 is to reduce the failure of the video due to the frame rate conversion process in the device that performs the frame rate conversion process. For the video that has already failed at the time of input to the device. The failure cannot be improved.

  The present invention has been made in view of the above problems, and an object of the present invention is to reduce the failure of a video that has already failed when it is input to the apparatus.

  In order to achieve the above purpose, a determination unit that detects the discontinuity of the edge included in the input video and determines the failure of the video, and a video processing that performs the high image quality processing or the high frame rate processing on the input video And a display unit for displaying the video processed by the video processing unit, the video processing unit, depending on a determination result of the determination unit, strength of the image quality improvement processing, level change, or high frame What is necessary is just to make it the structure which changes the production | generation process of the interpolation frame in a rate conversion process.

  According to the present invention, it is possible to reduce the failure of a video that has already failed when it is input to the apparatus.

It is a block diagram which shows an example of the image processing apparatus which concerns on one Example of this invention. 2 shows an example of the configuration of a failure detection unit according to an embodiment of the present invention. It is a figure for demonstrating an example of the edge detection which concerns on one Example of this invention. It is a figure for demonstrating an example of the polarity of the edge detection which concerns on one Example of this invention. It is a figure for demonstrating an example of edge continuity determination which concerns on one Example of this invention. It is a block diagram which shows an example of the image processing apparatus which concerns on one Example of this invention. 2 shows an example of a configuration of a level determination unit according to an embodiment of the present invention. It is a figure for demonstrating an example of the dispersion | distribution determination of the level determination part which concerns on one Example of this invention. It is a block diagram which shows an example of the image processing apparatus which concerns on one Example of this invention. It is a figure for demonstrating an example of the failure pattern detection which concerns on one Example of this invention. It is a block diagram which shows an example of the image processing apparatus which concerns on one Example of this invention. It is a figure for demonstrating the format of FramePacking data in HDMI ver.1.4a. 2 shows an exemplary configuration of a synchronization generation unit according to an embodiment of the present invention. It is a figure for demonstrating an example of operation | movement of the synchronous production | generation part which concerns on one Example of this invention. 2 shows an example of the configuration of a failure detection unit according to an embodiment of the present invention. It is a figure for demonstrating an example of the system which concerns on one Example of this invention. It is a figure for demonstrating an example of the video failure by a frame rate conversion process. It is a figure for demonstrating an example of the system which concerns on one Example of this invention. 2 shows an example of a configuration of an adaptive enhancer processing unit according to an embodiment of the present invention. 2 shows an example of a configuration of a nonlinear processing unit of an adaptive enhancer processing unit according to an embodiment of the present invention. 7 shows an example of input / output characteristics of a nonlinear processing unit of an adaptive enhancer processing unit according to an embodiment of the present invention. 1 is a block diagram illustrating an example of an image processing apparatus according to an embodiment of the present invention.

  A system example including the image processing apparatus according to the first embodiment of the present invention will be described with reference to FIG. In FIG. 16, in the broadcasting station side system 16001, the video imaged by the photographing camera 16003 is converted into a frame rate by the frame rate conversion processing unit 16004, and video data compression is performed by the encoding processing unit 16005. The compressed video data is transmitted as transmission data 16006 via a broadcast wave or a communication network. The transmitted transmission data 16006 is received by the image processing apparatus 16002 according to the first embodiment of the present invention, and image processing is performed.

  Note that the image processing device 16002 according to the first embodiment of the present invention may be a display device that displays an image-processed video on a display unit, or an image output device that outputs an image-processed video to another device. Further, the image processing device 16002 may be configured as a television broadcast receiving device including a tuner and a demodulation unit. In this case, video data included in the television broadcast signal and transmitted can also be processed.

  As an example of the frame rate conversion processing in the frame rate conversion processing unit 16004 of the broadcasting station side system 16001, for example, in order to broadcast images shot in the PAL area such as Europe and China in the NTSC area of Japan, 50Hz to 60Hz And frame rate conversion processing.

  At this time, when a video (image) failure occurs due to the frame rate conversion processing in the frame rate conversion processing unit 16004, the video failure remains in the video data included in the transmission data 16006.

  Here, an example of the occurrence of the video failure will be described with reference to FIG. For example, as shown in FIG. 17A, it is difficult to accurately detect a motion vector for an image including a repeated pattern pattern. When frame rate conversion processing using a motion vector is performed on such a video, a part of the video after the frame rate conversion processing may be different from the original video, as shown in FIG. Video breakdown will occur.

  As another example of the system including the image processing apparatus according to the first embodiment of the present invention, the example of FIG. In FIG. 18, a playback device 18001 is a device that plays back video data. The playback device 18001 plays back video data using a decoder unit 18003, performs frame rate conversion processing using a frame rate conversion processing unit 18004, and outputs the video data. The output video data is input as transmission data 18005 to the image processing apparatus 18002 via a wired or wireless transmission path.

  Note that the video data to be played back by the playback device 18001 may be video data recorded on an optical disk such as a DVD or BD, or a magnetic disk such as an HDD. In this case, the playback device 18001 uses a video playback unit from a recording medium. It is necessary to have. Also, video data included in the digital broadcast signal may be played back. In this case, the playback device 18001 needs to include a tuner, a demodulator, or the like in order to generate compressed video data from the digital broadcast signal.

  Also in the case of FIG. 18, when a video failure occurs due to the frame rate conversion processing in the frame rate conversion processing unit 18004, the video failure remains in the video data in the transmission data input to the image processing device 18002. .

  Next, the configuration of the image processing apparatus 16002 in the system of FIG. 16 or the image processing apparatus of the present invention which becomes the image processing apparatus 18002 in the system of FIG. 18 will be described.

  FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus according to the first embodiment of the present invention.

  In FIG. 1, 1001 is a decoder unit, 1002 is a failure detection unit, 1003 is an image quality enhancement processing unit, 1004 is a frame rate conversion processing (hereinafter referred to as FRC: Frame Rate Conversion) unit, 1005 is a timing controller unit, and 1006 is A display unit, 1007 is a memory I / F unit, and 1008 is a frame memory.

The decoder unit 1001 decodes compressed data such as MPEG and H.264. If the image processing apparatus according to the first embodiment is the image processing apparatus 18002 of the system shown in FIG. 18, the decoder unit 1001 is not necessary. This is because decoded data is input. This also applies to image processing apparatuses of the following embodiments.
The failure detection unit 1002 detects the failure of the video included in the input video. The failure detection unit 1002 outputs a failure determination signal 2007, which will be described later, to the image quality improvement processing unit 1003 and the FRC unit 1004. The high image quality processing unit 1003 performs high quality image processing such as resolution conversion, IP (Interlace / Progressive) conversion, noise reduction, and the like. The FRC unit 1004 uses the memory I / F unit 1007 and the frame memory 1008 to perform frame rate conversion processing. The timing controller unit 1005 performs timing adjustment for displaying the video output after the frame rate conversion processing obtained from the FRC unit 1004 on the display unit 1006.

  FIG. 2 shows an example of the configuration of the failure detection unit 1002 in FIG.

  In FIG. 2, 2001 is a luminance level detection unit, 2002 is a horizontal edge detection unit, 2003 is a vertical edge detection unit, 2004 is an edge continuity determination unit, 2005 is a failure determination unit, 2006 is a video delay unit, and 2007 is a failure. This is a determination signal.

  The luminance level detection unit 2001 detects the luminance level of the input video. Here, for example, as shown in FIG. 3, the dynamic range of the input signal is divided by predetermined thresholds TH1 to TH3 to classify which region L1 to L4 each pixel belongs to.

  The horizontal edge detection unit 2002 and the vertical edge detection unit 2003 detect the edge amount (intensity) and polarity (rising / falling) in the horizontal direction and the vertical direction, respectively. Here, the edge amount (intensity) is a difference between pixel values of adjacent pixels. For example, as shown in FIG. 4A, the edge amount (intensity) is divided by predetermined threshold values TH4 to TH6 to classify which region the edge amount (intensity) of each pixel belongs to from E0 to E4. . Here, E0 is a point having no edge (no luminance difference). The above classification is performed independently in the horizontal and vertical directions. Also, as shown in FIG. 4, when there is an edge, it is determined whether the polarity is rising (FIG. 4 (b)) or falling (FIG. 4 (c)). Further, as the detection order here, there is no problem even in the horizontal direction after the vertical direction.

  The edge continuity determination unit 2004 determines edge continuity using the luminance level information and edge information. Specifically, the luminance level information and edge information of the surrounding eight pixels are used as shown in FIG. In FIG. 5, A0 is a target pixel with an edge.

An example of the condition for determining edge continuity will be described. For example, when the following conditions are satisfied, it is determined that there is no edge continuity.
Condition (1) In the case where there are a number of pixels having a luminance level of the same classification as the target pixel and having an edge amount different from the predetermined threshold (X1) by the predetermined threshold (N1) Condition (2) The same classification as the target pixel When the number of pixels having different edge directions (horizontal / vertical) at the luminance level is greater than or equal to the predetermined threshold (N2) Condition (3) The luminance level is the same as that of the target pixel and the edge has a different polarity When the number of pixels is greater than or equal to the predetermined threshold (N3), each of the conditions (1) to (3) may be used alone as an edge continuity determination condition, and the conditions (1) to (3) Some conditions or all conditions may be combined, and when all of these conditions are satisfied, it may be determined that there is no edge continuity.

  Here, the definition of a pixel having an edge needs to be determined in advance. For example, “brightness level L2 or higher and edge amount (intensity) E3 or higher is assumed to have an edge”.

  The meaning of the condition is that an edge indicates a characteristic part of an image, and an image in which only one pixel is present in isolation is usually difficult to occur. Since such pixels usually indicate the contour or pattern of an object in many cases, there is a high possibility that one or more pixels having the same characteristics exist around the pixel. In addition, it is difficult for an image having such a characteristic pixel to surround many kinds of pixels.

  Here, it is known from experience that there are many phenomena in which the image is broken due to the frame rate conversion process in which the edge portion, the outline of the object, etc. are broken (edge continuity is broken). By determining edge continuity, it is possible to accurately detect a video failure due to frame rate conversion processing. Furthermore, by adding the condition of edges having the same luminance level, it is possible to improve the detection accuracy of a portion that is discontinuity of edge components belonging to the same object and contour.

  In the failure determination 2005, the portion (pixel) determined to have no edge continuity by the edge continuity determination is counted in one frame, and the case where the count value is equal to or greater than a predetermined threshold (M) is “failed” If it is smaller than the threshold value (M), it is determined that there is no failure and is output as a failure determination signal 2007.

  The delay unit 2006 delays and outputs the input data by the delay amount corresponding to the failure detection system circuit.

  The failure determination signal 2007 is sent to the image quality enhancement processing unit 1003 and the FRC processing unit 1004, respectively.

  In the image quality improvement processing unit 1003, for a frame in which the failure determination signal 2007 indicates “failure”, for example, edge enhancement enhancer and sharpness processing is made weaker than processing for a frame that indicates “failure” . Alternatively, for a frame in which the failure determination signal 2007 indicates “failure”, the processing is turned off.

  FIG. 19 shows an example of a configuration when the image quality enhancement processing unit 1003 performs enhancer processing. In FIG. 19, 19001 is a vertical high-pass filter, 19002 is a horizontal high-pass filter, 19003 is a delay unit, 19004 and 19006 are adders, 19005 is a nonlinear processing unit, and 19007 is a selector unit.

  The input signal 19009 is divided into two systems: an input to the delay unit 19003 and the horizontal / vertical high-pass filter processing units 19001 and 19002. The horizontal / vertical high-pass filters 19001 and 19002 apply filter processing to the input signal 19009 to extract the frequency component of the signal to be enhanced. In FIG. 19, the vertical enhancement component and the horizontal enhancement component are individually extracted and added by an adder 19004 to generate an enhancement component. The enhancer component is subjected to non-linear processing such as noise removal, gain adjustment, and limiter by a non-linear processing unit 19005 to generate a final enhance signal.

  An adder 19006 adds an enhancement signal to the timing-adjusted input signal from the delay unit 19003, thereby generating a signal in which a predetermined high frequency component is enhanced. The selector 19007 selects which signal to use before or after the enhancement process by the failure determination signal 2007. Thereby, ON / OFF of the enhancement process can be controlled.

  The failure determination signal 2007 is also input to the non-linear processing unit 19005 so that the enhancement amount can be controlled.

  FIG. 20 shows an example of the configuration of the nonlinear processing unit 19005 in FIG. In FIG. 20, 20001 is a coring processing unit, 20002 is a gain adjustment unit, and 20003 is an amplitude limiter unit. The coring processing unit 20001 sets the level of the output signal with respect to the input signal equal to or lower than the predetermined threshold value CORE to 0 and prevents the noise component from being extracted as an enhancement component. The gain adjustment unit 20002 increases the sensitivity of the enhancement component according to the predetermined gain amount GAIN. The amplitude limiter unit 20003 has a predetermined level in order to prevent excessive enhancement processing such as overexposure or blackout caused by excessive enhancement by the gain adjustment for a large amplitude component near the dynamic range. Control is performed so that the amplitude level of the large amplitude enhancement signal within the range of LIM1 and LIM2 is constant or lowered. As described above, the nonlinear processing unit 19005 exhibits input / output characteristics as shown in FIG. 21, for example. By adaptively changing the coring amount, gain amount, and limiter level according to the failure determination signal 2007, it is possible to perform suitable image quality improvement processing for a video in which the failure has occurred.

  Further, as an example of another process in the quality enhancement process 1003, IP conversion, resolution conversion, noise reduction, and the like may be performed. In this case, the configuration of the high-quality processing 1003 is a configuration in which adaptive processing according to the failure status of the input video is performed by the failure determination signal and the strength or level of the processing is changed in the same manner as the enhancer processing described above. That's fine. As a result, a more suitable image can be created.

  In the FRC processing unit 1004, since the reliability of the motion vector is low with respect to the frame in which the failure determination signal 2007 indicates “failure”, in the interpolated frame generation using the frame, for example, the same frame repetition or linearity of the preceding and following frames is used. An interpolation frame generation process that does not use a motion vector such as interpolation may be selected. Further, even when an interpolation frame generation process that does not use a motion vector is used, an interpolation frame generation process that is less affected by a motion vector may be performed by shortening the motion vector. As described above, with respect to a frame in which the failure determination signal 2007 indicates “failure occurs”, the interpolation frame generation processing using the motion vector is limited, and the interpolation frame generation processing in which the failure is hardly visible is selected. It should be noted that regarding the frame in which the failure determination signal 2007 indicates “no failure”, it is determined that there is no problem in reducing the reliability of the motion vector due to the failure of the input video itself, and the FRC processing unit 1004 preferably includes other factors. What is necessary is just to select the interpolation frame production | generation process to judge. That is, it is not necessary to limit the interpolation frame generation process using the motion vector. According to the image processing apparatus of the first embodiment of the present invention described above, when the video has already failed in the input video signal, the presence or absence of the failure can be determined. In signal processing, it is possible to perform more suitable image quality improvement processing adapted to it. In other words, it is possible to more suitably perform high-quality video.

  In the present embodiment, the failure of the input video is described as an example by the frame rate conversion process. However, the present invention is not limited to this. Various encoding, decoding processes, IP conversion, resolution conversion, super-resolution The disclosed technique according to the present embodiment can be applied to any video failure caused by various image quality enhancement processing such as image processing and noise reduction processing.

  Next, a second embodiment of the present invention will be described. The second embodiment of the present invention is different from the image processing apparatus according to the first embodiment in the configuration and operation of the failure detection unit, and the other configurations and operations are the same. Since the system shown in FIG. 16 and FIG. 18 is the same as that of the first embodiment, the description thereof is omitted.

  FIG. 6 is a block diagram illustrating an example of the configuration of the failure detection unit 6002 according to the second embodiment of the present invention.

  In FIG. 6, the same constituent elements as those shown in FIG. 2 are denoted by the same reference numerals, and the description thereof is omitted.

  The failure detection unit 6002 according to the second embodiment can determine the level of edge continuity and can perform subsequent image quality improvement processing according to the level.

  Hereinafter, the operation of portions different from those of the first embodiment described above will be described.

  In the failure detection unit 6002 shown in FIG. 6, reference numeral 6001 denotes a level determination unit that detects the level of edge continuity in a frame. The failure detection unit 6002 of Example 2 shown in FIG. 6 is different from the failure detection unit 1002 of Example 1 shown in FIG. 2 in that the level determination unit 6001 is added.

  FIG. 7 is a block diagram illustrating an example of the configuration of the level determination unit 6001 in FIG. In FIG. 7, reference numeral 7001 denotes an edge discontinuous number total number determining unit, 7002 an edge discontinuous dispersion determining unit, and 7003 a level determining unit.

  The total number of edge discontinuity determination unit 7001 counts the number of edge discontinuities in one frame, and divides the count value according to a threshold value. Here, the process of counting the number of edge discontinuities in one frame may be the same as the process of counting the number of edge discontinuities in the edge continuity determination unit 2004 described in the first embodiment. Therefore, the count processing may be performed by the edge discontinuity total number determination unit 7001, but the count result of the edge continuity determination unit 2004 may be acquired and used.

The classification of the calculated count value will be described. For example, two threshold values TH1 and TH2 (TH1 <TH2) are prepared and classified as shown in Table 1. That is, classification is performed using the count value as an index value for the number of edge discontinuities in the frame.

Next, the dispersion determination unit 7002 acquires position information (coordinate information in the frame) of the pixels determined to be discontinuous in the preceding edge continuity determination 2004, and detects the dispersion information. As the position information, for example, as shown in FIG. 8, one frame is divided into six regions, and the number of edge discontinuous pixels in each region is counted. A predetermined threshold TH3 is prepared for the count number, and a predetermined threshold TH4 is prepared for the number of areas, and classification is performed as shown in Table 2. In other words, classification is performed by using the number of areas whose count number is equal to or greater than the predetermined threshold TH3 as an index value for spatial dispersion of edge discontinuity in the input video.

  The level determination unit 7003 performs a determination process for determining an edge discontinuity level as shown in Table 3, for example, based on the results from the total number of discontinuity determination unit 7001 and the concentration degree determination unit 7002.

That is, as the number of edge discontinuities in a frame increases and the variance increases, the edge discontinuity level is determined to be higher. Conversely, as the number of edge discontinuities decreases and the variance decreases, the edge discontinuity level decreases. Is determined.

  As a specific determination process, the level determination unit 7003 stores a determination table as shown in Table 3 in advance, and performs edge discontinuity according to the results of the discontinuous number total number determination unit 7001 and the concentration degree determination unit 7002. You just have to decide the level. The level determination unit 7003 includes the determined edge discontinuity level in the failure determination signal 2007, and outputs it to a subsequent processing unit such as the image quality improvement processing unit 1003 or the FRC unit 1004.

  The image quality enhancement processing unit 1003, the FRC unit 1004, and the like adaptively switch processing according to the edge discontinuity level determined by the determination. Specifically, as the discontinuity level increases, the image quality improvement processing unit 1003 weakens the enhancer and sharpness processing, and the FRC unit 1004 uses an interpolation frame that does not use a motion vector in the interpolation frame (for example, linear An adaptive process such as increasing the insertion rate of the interpolation frame) is performed. In the example of Table 3, when the discontinuity level is “low”, the enhancement and sharpness processing of the image quality enhancement processing unit 1003 are not limited, and the discontinuity level increases to “medium” and “high”. You may increase the limit as you go. When the discontinuity level is “high”, the enhancer and sharpness processing may be turned off.

  In addition, when the discontinuity level is “low”, the FRC unit 1004 does not restrict the insertion of the interpolation frame generated using the motion vector in the interpolation frame, and the discontinuity level increases to “medium” and “high”. Accordingly, the insertion rate of an interpolation frame that does not use a motion vector (for example, a linear interpolation frame) may be increased instead of the interpolation frame that is generated using the motion vector. In addition, when the discontinuity level is “high”, insertion of an interpolation frame generated using a motion vector may be prohibited.

  According to the image processing apparatus of the second embodiment of the present invention described above, the failure rate in the video is determined in consideration of not only the total number of edge discontinuities but also the dispersion of edge discontinuities. By changing the subsequent signal processing method adaptively according to the ratio, it is possible to improve the image quality of the video more suitably.

  Next, Embodiment 3 of the present invention will be described. FIG. 9 is a block diagram illustrating an example of the configuration of the image processing apparatus according to the third embodiment of the present invention.

  The image processing apparatus according to the third embodiment deletes a broken frame from the input video by detecting the presence or absence of the broken frame of the input video or the periodicity of the appearance of the broken frame, and deletes the FRC process again in the image processing apparatus. It is possible to reduce failure by replacing the frame with a new frame.

  9, the same components as those shown in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. Hereinafter, the operation of parts different from those of the first embodiment will be described.

  In FIG. 9, 9001 is a failure determination signal, 9002 is an FRC mode signal, and 9003 is a pattern detection unit.

  The configuration of the failure detection unit 1002 is the same as that of the first embodiment. The failure detection unit 1002 outputs a failure determination signal 2007 for each frame as in the first embodiment. The failure determination signal 2007 is input to the pattern detection unit 9003. The pattern detection unit 9003 obtains a plurality of frames of input data and a failure determination result by the failure detection unit 1002, and detects a pattern of a frame in which the failure has occurred.

  An example of the detection will be described. As shown in FIG. 10, for example, pattern detection can be performed based on failure determination results for six consecutive frames. Here, for example, it is assumed that the input signal to the failure detection unit 1002 is 60 Hz. In FIG. 10, “present” and “none” represent a frame in which a failure has occurred, and “no” represents a frame in which no failure has occurred. The pattern in FIG. 10A is a pattern when all the interpolation frames generated on the transmission side are broken in the FRC process of 24 Hz → 60 Hz (2.5 times) on the transmission side. Similarly, (b) and (c) are the patterns when all the interpolated frames generated on the transmission side have failed in FRC processing at double speed (30 Hz → 60 Hz, etc.) and 50 Hz → 60 Hz (1.2 times), respectively. It is. In some cases, regularity may not be seen in the appearance of a broken frame as shown in (d).

  The memory I / F unit 1007 reads only a frame without failure according to the detected pattern, and outputs it to the signal processing unit at the subsequent stage. That is, in FIG. 9A, only the first, fourth, seventh,... Frame data are read. That is, the memory I / F unit 1007 has a function as a frame decimation processing unit that can perform a decimation process of a frame with a failure.

  In addition, the memory I / F unit 1007 outputs an FRC mode signal 9002 indicating that only the frame without failure is passed to the FRC unit 1004 in consideration of the delay number of the image quality improvement processing unit 1003. The failure determination signal 9001 is the same as the failure determination signal 2007 in the first embodiment. In this case, a failure determination signal 9001 indicating that there is no failure in the frame output from the memory I / F unit 1007 is output to the image quality improvement processing unit 1003 and the FRC unit 1004.

  For example, in FIG. 9, when the display frame rate on the display unit 1006 is 120 Hz, and in the case of FIGS. 10 (a), (b), and (c), FRC processing of 5 × speed, 4 × speed, and 2.4 × speed is required. FRC mode signal 9002 is output. In accordance with the FRC mode signal 9002, the FRC processing unit 1004 performs FRC processing for generating an interpolation frame using the frame without failure acquired from the memory I / F unit 1007. As a result of the FRC process, the frame rate is increased from that of the input video, and as a result, the broken “present” frame in the input video is replaced with an interpolation frame newly generated by the FRC process in the image processing apparatus according to the third embodiment. It will be.

  Also, as shown in FIG. 10 (d), when there is no regularity in the appearance of the broken frame, the memory I / F unit 1007 reads all the frames including the broken frame, and the image quality improving processing unit 1003 Outputs a failure determination signal 9001 and outputs an FRC mode signal 9002 to the FRC processor that double-speed FRC processing is required. In the image quality improvement processing unit 1003, the enhancement / sharpness processing is weakened or turned off for a frame whose failure determination signal 2009 indicates failure. In the image quality improvement processing unit 1003, there is no need to perform such image quality improvement processing for a frame whose failure determination signal 2009 indicates no failure. Similarly, in the FRC processing unit 1004, for a frame whose failure determination signal 9001 indicates that there is a failure, for example, interpolation frame generation processing that makes it difficult to see the failure by repeating the same frame, linear interpolation of the preceding and following frames, shortening of the motion vector, etc. Select and limit interpolation processing using motion vectors. Even in the FRC processing unit 1004, such a restriction is not necessary for a frame whose failure determination signal 2009 indicates no failure. That is, when there is no regularity in the appearance of the broken frame as shown in FIG. 10D, the same processing as that of the image processing apparatus according to the first embodiment is performed to replace the broken frame with a new interpolation frame. It is not necessary to do.

  According to the image processing apparatus of the third embodiment of the present invention described above, by detecting a regular pattern of a failed frame, the failed frame is deleted, and the subsequent FRC process is performed again. Thus, the broken frame can be replaced with a new interpolation frame. Further, when a regular pattern is not detected, it is possible to correct a broken frame as in the first embodiment.

  In the present embodiment, the configuration of the failure detection unit is the same as that of the first embodiment, but the configuration is not limited to this, and the configuration of the failure detection unit in the second embodiment may be the same.

  Next, a fourth embodiment of the present invention will be described. The configuration of the image processing apparatus according to the fourth embodiment of the present invention may be configured as shown in FIG. 22, for example. The image processing apparatus shown in FIG. 22 has a configuration obtained by removing the pattern detection unit 9003 from the image processing apparatus according to the third embodiment shown in FIG.

  22, the same components as those shown in FIG. 9 are denoted by the same reference numerals, and the description thereof is omitted.

In the pattern detection unit 9003 of the image processing apparatus according to the third embodiment of the present invention, for example, as shown in FIG. 10, the appearance pattern of a failed frame is detected from the failure determination result of a plurality of frames, and a regular appearance pattern is detected. The process was switched between when it was detected and when it was not.
In contrast, in the image processing apparatus according to the fourth embodiment of the present invention, when the failure determination signal 2007 is input to the memory I / F unit 1007 for each frame detected by the failure detection unit 1002, the memory I / F unit At 1007, only a frame without failure is read and output to the signal processing unit at the subsequent stage. In this case, the memory I / F unit 1007 outputs only a frame without failure in any of the cases of FIGS. 10A to 10D regardless of the presence or absence of the regularity of the appearance of the broken frame. In this case, a failure determination signal 9001 indicating that there is no failure in the frame output from the memory I / F unit 1007 is output to the image quality improvement processing unit 1003 and the FRC unit 1004. Further, the FRC unit 1004 outputs an FRC mode signal 9002 indicating that all the failed frames have been deleted and need to be replaced by interpolation processing of FRC processing.

  The image quality improvement processing unit 1003 performs the image quality improvement processing without limitation because all the input video frames are determined to have no failure.

  The FRC unit 1004 first uses the frame determined not to fail, generates an interpolated frame that replaces the frame determined not to be output from the memory I / F unit 1007, and causes a frame in the input video to fail. Is replaced with a new interpolation frame. When a frame rate higher than the frame rate of the input video is necessary, an interpolation frame is further generated and converted into a video with a high frame rate. Note that in this case, the reliability of the motion vector is not low because the frames input to the FRC unit 1004 are all subjected to image quality enhancement processing on frames determined to have no failure. Therefore, the interpolation processing in the FRC unit 1004 may use not only interpolation processing that does not use motion vectors (for example, linear interpolation) but also interpolation processing that uses motion vectors.

  According to the image processing apparatus of the fourth embodiment of the present invention described above, a broken frame included in the input video is newly determined regardless of whether the appearance pattern of the broken frame included in the input video is regular. Therefore, it is possible to replace the generated interpolated frame with a higher quality image.

  Next, a fifth embodiment of the present invention will be described. FIG. 11 is a block diagram showing an example of the configuration of an image processing apparatus according to Embodiment 5 of the present invention.

  In FIG. 11, the same components as those shown in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.

  When the input signal is a 3D video signal, the image processing apparatus according to the fifth embodiment of the present invention can make the 3D video without a sense of incongruity by combining the image processing amounts for the left and right images. To do.

  Hereinafter, the operation of parts different from the above-described embodiments will be described.

  In FIG. 11, 11001 is a 3D mode signal, 11002 is a synchronization generation unit, 11003 is an internal synchronization signal, 11004 is a left / right video corruption unified determination flag, 11005 is a data compression / decompression unit, and the failure detection unit 11006 is the same as that of the first embodiment. This is a new failure detection unit that replaces the failure detection unit 1002. The 3D correspondence display unit 11007 is a display unit that supports not only 2D video but also 3D video display.

  Here, the 3D mode signal 11001 is a signal indicating a 3D signal format. For example, if the notation in HDMI ver.1.4a is used, FramePacking (described later), Side-by-Side (video for the left and right eyes) is used. Sending in horizontal format: half horizontal resolution), Top-and-Bottom (sending left and right eye video in vertical format: half vertical resolution) It is. In the following description, the decoder unit 1001 in FIG. 11 is described as outputting a 3D video signal in the format described in HDMI ver.1.4a. However, the format conforms to other standards that can handle the 3D video format. But it ’s okay. In the 3D video format, the case of FramePacking will be described as an example.

  Here, the format (1080p @ 24Hz) of FramePacking data in HDMI ver.1.4a is as shown in FIG. In other words, the left and right videos are arranged one above the other, and the video is sent with a dot clock in the vertical synchronization period that is twice that of 2D video.

  FIG. 13 is a diagram for explaining the operation of the synchronization generation unit 11002 in FIG. In FIG. 13, (a) and (b) are a vertical synchronization signal and a data enable waveform obtained from input video data. On the other hand, as shown in (b) and (c), the synchronization generation unit 10002 generates an internal synchronization signal 10003 indicating the separation between the left and right image data. Specifically, a vertical synchronization signal SVp 14001 and data enable: SDe 14002 as shown in FIG. 14 are generated. The horizontal synchronization signal is used as it is.

FIG. 15 shows an example of the configuration of the failure detection unit 11006 in FIG. In FIG. 15, reference numeral 15001 denotes a vertical synchronizing signal of the inputted internal synchronizing signal 10003, 15002 denotes a left / right video failure unified determination unit, and 15003 denotes a level determination output. In the edge continuity determination unit 2004, determination is performed for each vertical synchronization of the internal synchronization signal 10003 generated by the synchronization generation unit 10002. That is, determination is performed for each of the left and right images, and each result is output to the level determination unit 6001. The level determination unit 6001 performs level determination in the same manner as in Table 3 from the edge continuity determination of each of the left and right videos. The result is output to the bankruptcy unified determination unit 15002, and output to the outside 15003. The left and right video failure unified determination unit 15002 performs classification as shown in Table 4 from the level determination result of the left and right videos.

  This determination is performed for each vertical synchronization of input, and one determination result is output as a left and right video failure unified determination flag 11004 for each set of left and right images. Specifically, when it is determined that there is a one-step difference in the failure level between the left and right images, the determination signal is output in a unified high level. Here, by unifying the determination of the left and right levels, it is possible to prevent the difference in video breakdown between the final images of the left and right images from being widened due to the strength of the image quality enhancement process at the later stage. Further, when unifying judgments, it is possible to suppress emphasis on video failure due to a later high image quality processing by unifying to a higher failure level.

  Further, when it is determined that there is a difference of two levels in the failure level, it is determined that there is a failure, and the image quality improvement processing in the subsequent stage is limited or turned off. In other words, if there is a large difference between the failure levels of the left and right images, processing that may increase the limit is limited or eliminated.

  There is a high possibility that the difference in video breakdown in the final images of the left and right images will increase the sense of discomfort when viewing in 3D, but by making the determination shown in Table 4 described above, the increase in discomfort can be suppressed. Is possible.

  When the classification shown in Table 4 is performed, the default unified determination flag 11004 can be realized by a 3-bit signal.

  The image quality improvement processing unit 1003 and the FRC unit 1004 that have received the failure unified determination flag 11004, according to the failure unified determination result shown in Table 4, respectively, have the same or the same level of image quality improvement processing and frame rate for both the left and right images. Perform conversion processing. The 3D image including the left and right images subjected to these processes is displayed in 3D on the 3D correspondence display unit 11007.

  On the other hand, the externally output level determination output 15003 is sent to the data compression / decompression unit 11005. The data compression / decompression unit 11005 is a function that compresses data and stores it in memory for the purpose of reducing the frame memory capacity and SDRAM bandwidth, and decompresses the data when it is used. This is an important process. The lossless method is ideally desirable, but the lossy method is often adopted. In this case, the higher the compression ratio is, the more the video deterioration is caused.

  The data compression / decompression unit 11005 has a configuration in which the compression rate can be varied. In the present embodiment, the compression rate is varied in accordance with the level determination output 15003. If there is a difference between the collapse levels of the left and right images, for example, if the collapse level of the L side image is “low” and the collapse level of the R side image is “high”, increase the compression rate of the L side image, By reducing the compression rate of the R side image accordingly, further divergence of the failure of the L side image and the R image is prevented. However, the compression rate should basically be the same for the L-side image and the R-side image, and it is desirable to apply when the failure level is clearly different as described above.

  According to the image processing apparatus of the fifth embodiment of the present invention described above, when the input signal is a 3D video signal, the processing levels of the left and right video image quality enhancement processing and the frame rate conversion processing are unified. This makes it possible to reduce a sense of incongruity in the final image.

  In addition, in the configuration that performs compression processing when storing video data, if the left and right images of the input 3D signal have different failure levels, the data compression rate of the image with the lower failure level is increased and the failure level is higher. By reducing the data compression ratio, it is possible to obtain a 3D image without a sense of incongruity by balancing the left and right images so that the difference in image failure does not widen.

  In this embodiment, the 3D format has been described using only the FramePacking method. However, the present invention is not limited to this, and the present invention can also be applied to the Side-By-Side method and the Top-and-Bottom method. The FramePacking method, Side-By-Side method, and Top-and-Bottom method differ only in the storage method of the left and right images, and can be handled in the same way as FramePacking method images if necessary scaling processing is performed. Because it can.

  Note that the image processing apparatus according to any of the embodiments described above may be provided with an output unit instead of or in addition to the display unit, and output an image or video to another device.

1001 decoder unit, 1002 failure detection unit, 1003 high image quality processing unit, 1004 frame rate conversion processing unit, 1005 timing control unit, 1006 display unit, 1007 memory I / F unit, 1008 frame memory,
2001 brightness level detection unit, 2002 horizontal edge detection unit, 2003 vertical edge detection unit, 2004 edge continuity determination unit, 2005 failure determination unit, 2006 video delay unit, 2007 failure determination signal,
6001 level judgment part, 6002 failure detection part,
7001 edge discontinuity total number judgment part, 7002 edge discontinuity dispersion judgment part, 7003 level determination part,
9001 failure determination signal, 9002 FRC mode signal,
11001 3D mode signal, 11002 synchronization generation unit, 11003 internal synchronization signal, 11004 failure unified determination flag, 11005 data compression / decompression unit, 11006 failure detection unit, 11007 3D compatible display unit,
14001 vertical sync signal, 14002 data enable,
15001 input vertical sync signal, 15002 failure unified judgment unit, 15003 level judgment output,
16001 broadcast station side signal processing, 16002 TV set internal signal processing, 16003 shooting camera, 16004 frame rate conversion processing unit, encoding processing unit for 16005 data compression, 16006 decoding processing unit, 16007 high image quality processing unit, 16008 frame rate conversion Processing unit, 16009 timing controller unit, 16010 display unit,
18001 TV set internal signal processing, 18002 Recorder side signal processing,
19001 vertical high-pass filter, 19002 horizontal high-pass filter, 19003 delay unit, 19004, 19006 adder, 19005 nonlinear processing unit, 19007 selector unit,
19001 coring processing unit, 19002 gain adjustment unit, 19003 amplitude limiter unit,

Claims (12)

A determination unit that detects the discontinuity of the edge included in the input video and determines the failure of the video;
A video processing unit that performs high image quality processing or high frame rate processing on the input video;
A display unit for displaying the video processed by the video processing unit,
The display apparatus according to claim 1, wherein the video processing unit changes the strength of the image quality enhancement processing, the level change, or the interpolation frame generation processing in the high frame rate processing according to the determination result of the determination unit. The display device according to claim 1,
When the number of edge discontinuities included in the frame of the input video is equal to or greater than a predetermined threshold, the determination unit turns off the image quality improvement processing in the video processing unit, or performs high frame rate processing. In the interpolation frame generation process, an interpolation frame generation process that does not use a motion vector is performed, and an interpolation frame generation process that uses a motion vector is not performed. The display device according to claim 1,
The determination unit uses the total number of occurrences of edge discontinuities included in the frame of the input video as a first index value, and the spatial distribution of occurrence of edge discontinuities in the input video as a second index value. The level of occurrence of edge discontinuity is determined from the combination of the above, and the strength of the image quality improvement processing, the level change, or the interpolation frame generation processing in the high frame rate processing is changed according to the determination result A display device. Whether or not there is regularity in the appearance of a broken frame based on the result of video failure judgment for multiple frames by detecting edge discontinuity for multiple frames included in the input video A determination unit for determining whether or not
A frame thinning processing unit capable of thinning out some frames from the input video according to the determination result of the determination unit;
A video processing unit that performs high image quality processing or high frame rate processing on the video output from the frame thinning processing unit;
A display unit for displaying the video processed by the video processing unit,
When the determination unit determines that the appearance of the broken frame is regular, the frame thinning processing unit outputs a video obtained by thinning the broken frame from the input video, and the video processing unit Perform high frame rate processing according to the determined regularity type,
When the determination unit determines that there is no regularity in the appearance of the broken frame, the frame thinning processing unit outputs the broken frame from the input video without thinning out, and the video processing unit A display device characterized by changing the strength of image quality enhancement processing, level change, or interpolation frame generation processing in high frame rate processing depending on the presence or absence of video failure. A determination unit that detects edge discontinuity for a frame included in the input video and determines a video failure;
A frame thinning processing unit capable of thinning out some frames from the input video according to the determination result of the determination unit;
A video processing unit that performs high frame rate processing on the video output from the frame thinning processing unit;
A display unit for displaying the video processed by the video processing unit,
The display device, wherein the video processing unit replaces the frame thinned out by the frame thinning processing unit with an interpolated frame generated by a high frame rate processing. An input unit for inputting an input 3D image including a set of a left-eye image and a right-eye image;
A determination unit for determining a video failure of the input 3D video;
A video processing unit for performing high image quality processing or high frame rate processing on input 3D video;
A 3D video display unit for 3D display of the 3D video processed by the video processing unit,
The determination unit determines a video breakdown level for each video in a set of a left-eye video and a right-eye video included in the input 3D video, and the left-eye video and the right-eye video Even when the video breakdown levels are different, the video processing unit performs the same level of high image quality processing or high frame rate processing on the left-eye video and the right-eye video. Characteristic display device. A determination unit that detects the discontinuity of the edge included in the input video and determines the failure of the video;
A video processing unit that performs high quality image processing or high frame rate processing on the input video,
The video processing unit changes the strength of image quality enhancement processing, level change, or interpolation frame generation processing in high frame rate processing according to the determination result of the determination unit . The image processing apparatus according to claim 7,
When the number of edge discontinuities included in the frame of the input video is equal to or greater than a predetermined threshold, the determination unit turns off the image quality improvement processing in the video processing unit, or performs high frame rate processing. An image processing apparatus that performs interpolation frame generation processing that does not use motion vectors and does not perform interpolation frame generation processing that uses motion vectors in the interpolation frame generation processing. The image processing apparatus according to claim 7,
The determination unit uses the total number of occurrences of edge discontinuities included in the frame of the input video as a first index value, and the spatial distribution of occurrence of edge discontinuities in the input video as a second index value. The level of occurrence of edge discontinuity is determined from the combination of the above, and the strength of the image quality improvement processing, the level change, or the interpolation frame generation processing in the high frame rate processing is changed according to the determination result An image processing apparatus. Whether or not there is regularity in the appearance of a broken frame based on the result of video failure judgment for multiple frames by detecting edge discontinuity for multiple frames included in the input video A determination unit for determining whether or not
A frame thinning processing unit capable of thinning out some frames from the input video according to the determination result of the determination unit;
A video processing unit that performs high image quality processing or high frame rate processing on the video output from the frame thinning processing unit,
When the determination unit determines that the appearance of the broken frame is regular, the frame thinning processing unit outputs a video obtained by thinning the broken frame from the input video, and the video processing unit Perform high frame rate processing according to the determined regularity type,
When the determination unit determines that there is no regularity in the appearance of the broken frame, the frame thinning processing unit outputs the broken frame from the input video without thinning out, and the video processing unit An image processing apparatus characterized by changing the level of an image quality enhancement process, a level change, or an interpolated frame generation process in a high frame rate process depending on the presence or absence of a video failure. A determination unit that detects edge discontinuity for a frame included in the input video and determines a video failure;
A frame thinning processing unit capable of thinning out some frames from the input video according to the determination result of the determination unit;
A video processing unit that performs high frame rate processing on the video output from the frame thinning processing unit,
The image processing device, wherein the video processing unit replaces the frame thinned out by the frame thinning processing unit with an interpolation frame generated by a high frame rate processing. An input unit for inputting an input 3D image including a set of a left-eye image and a right-eye image;
A determination unit for determining a video failure of the input 3D video;
An image processing unit that performs high image quality processing or high frame rate processing on input 3D video,
The determination unit determines a video breakdown level for each video in a set of a left-eye video and a right-eye video included in the input 3D video, and the left-eye video and the right-eye video Even when the video breakdown levels are different, the video processing unit performs the same level of high image quality processing or high frame rate processing on the left-eye video and the right-eye video. A featured image processing apparatus.

Images