JP2010039135A - Image processor and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely display a graphics image while reducing the storage capacity even when an on-screen display function is used in high resolution display. <P>SOLUTION: An image display 100 includes a scale converting section 123 for scale-converting an image signal of a first resolution into an image signal of a second resolution higher than the first resolution, a flash memory 23 for storing a graphics image of the second resolution, and a high-resolution section 13 for performing super-resolution conversion processing of restoring an image signal of a third resolution higher than the second resolution by increasing a pixel by estimating an original pixel value from the image signal of the second resolution overlaid by the graphics image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method for generating an image signal obtained by increasing the resolution of a plurality of image frames included in input video data.

  With a high-resolution television, a graphics image (on-screen image) such as a setting screen for menus or an electronic program guide is displayed superimposed on the video displayed on the display, and the graphics image can be displayed with an operation button such as a controller. There is an on-screen display function (OSD: On-Screen Display) for setting the menu and the like.

  As a technology for displaying a graphics image in such an on-screen display function, a predetermined conversion ratio in which the number of horizontal pixels of an on-screen image is matched with the number of horizontal pixels of HD (High Definition) resolution (1920 pixels × 1080 lines) And a means for converting and outputting the number of horizontal pixels of the on-screen image at a predetermined conversion ratio in accordance with the number of horizontal pixels of SD (Standard Definition) resolution (720 pixels × 480 lines), A technique is known that can easily convert an on-screen image to HD output or SD output without increasing the processing load (see, for example, Patent Document 1).

  On the other hand, with the spread of high-resolution televisions and displays, the resolution of image frames is increasing. Further, as the resolution of an image signal is increased, the amount of data related to the image processing on the image signal is increased, so that a method capable of performing image processing more efficiently is desired. In particular, super-resolution that realizes high resolution while maintaining the sharpness of image data by restoring the high-resolution image signal by estimating the original pixel value from the low-resolution image signal and increasing the number of pixels Image processing technology called processing has appeared (for example, see Patent Document 2, Patent Document 3, etc.).

With this technology, an SD resolution image such as a DVD or analog video can be increased to a high resolution image such as an HD resolution, and a clear image can be displayed even on a large screen on a high resolution television or display.

  For this reason, in order to increase the resolution of an SD resolution image to HD resolution and display a graphics image such as a menu, the SD resolution is obtained by superimposing the graphics image on the SD resolution image data. It is necessary to convert the image data into HD resolution image data by super-resolution processing.

JP 2006-303631 A JP 2007-336239 A JP 2007-310837 A

  However, when low-resolution image data on which a graphics image is superimposed is converted into high-resolution image data by super-resolution processing and displayed, there is a problem that the graphics image is blurred.

  On the other hand, graphics images of different sizes for each resolution are facilitated in advance, super-resolution processing is performed on the displayed video image, and the resulting high-resolution image has the same resolution as that of the high-resolution image. It is also possible to superimpose images.

  However, in such a method, it is necessary to prepare graphics images with different resolutions in advance, and there is a problem that the storage capacity of a memory or the like for storing the graphics images increases.

  The present invention has been made in view of the above, and an image processing apparatus capable of displaying a graphics image with high definition while reducing the storage capacity even when the on-screen display function is used with high-resolution display. It is another object of the present invention to provide an image processing method.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention includes a conversion unit that performs scale conversion of an image signal having a first resolution into an image signal having a second resolution higher than the first resolution. A storage unit that stores the graphics image of the second resolution, a superimposing unit that superimposes the graphics image on the image signal of the second resolution, and an image signal of the second resolution on which the graphics image is superimposed. A high-resolution conversion unit that performs super-resolution conversion processing for restoring an image signal having a third resolution higher than the second resolution by estimating an original pixel value and increasing the number of pixels. .

  The image processing apparatus according to the present invention further includes a conversion unit that converts the first resolution image signal into a second resolution image signal higher than the first resolution, and a storage that stores the second resolution graphics image. A superimposing unit that superimposes the graphics image on the image signal of the second resolution, and an image signal of the second resolution on which the graphics image is superimposed using the super-resolution technique. And a high-resolution part for increasing the resolution of the high third-resolution image signal.

  According to the image processing method of the present invention, the first resolution image signal is scale-converted into a second resolution image signal higher than the first resolution, and the second resolution image signal is converted into the second resolution image signal. A third resolution higher than the second resolution by superimposing a graphics image having a resolution; and estimating an original pixel value from the image signal having the second resolution on which the graphics image is superimposed to increase the number of pixels. Performing a super-resolution conversion process for restoring the image signal.

  According to the present invention, even when the on-screen display function is used for high-resolution display, it is possible to display a graphics image with high definition while reducing the storage capacity.

  Exemplary embodiments of an image processing apparatus and an image processing method according to the present invention are explained in detail below with reference to the accompanying drawings.

  FIG. 1 is a block diagram schematically showing a system of an image display apparatus 100 according to the present embodiment. As shown in the figure, the image display device 100 includes a video signal input unit 11, a main processing unit 12, a resolution increasing unit 13 corresponding to the image processing device, a moving image improvement processing unit 14, and a display processing unit. 15, a display unit 16, an audio processing unit 17, an audio output unit 18, a flash memory 23, and EEPROMs 21 and 22.

  The video signal input unit 11 is a part to which a video signal to be displayed is input, and receives data transmitted via the digital broadcast receiving unit 111, the IPTV signal processing unit 112, and an IP network such as the Internet. An Internet signal processing unit 113 for receiving and an external input unit 114 for receiving an analog signal input are provided. Here, the “video signal” is a concept including an audio signal (audio data) in addition to an image signal (image data) composed of a still image or a moving image.

  The digital broadcast receiver 111 includes a digital antenna 1111 for receiving digital broadcasts such as BS, CS, and terrestrial waves, a digital tuner 1112 for selecting digital broadcasts, and demodulating the digital broadcasts as digital video signals. And a digital signal demodulating unit 1113 for outputting to the main processing unit 12.

  The IPTV signal processing unit 112 receives an IP broadcast transmitted via a dedicated IP network, and outputs it to the main processing unit 12 as a digital video signal.

  The Internet signal processing unit 113 receives data (still images and moving images) transmitted via an IP network such as the Internet and outputs the data to the main processing unit 12 as a digital video signal.

  The external input unit 114 includes an analog antenna 1141 for receiving an analog broadcast, an analog tuner 1142 for selecting the analog broadcast, and performs signal processing such as A / D conversion on the analog signal to obtain a digital video signal. An external input signal processing unit 1143 for outputting to the main processing unit 12. Note that the external input signal processing unit 1143 has a terminal (not shown) for connecting to an external device such as a game machine, a PC (Personal Computer), or a DVD player, and is input from the external device via the input terminal. Signal processing is also performed on the analog signal to be processed.

  The main processing unit 12 performs an MPEG decoding process on the video signal input from the video signal input unit 11, a separation process of the video signal into image data and audio data, a scale conversion process of the image data, an on-screen display Performs superimposition processing with graphics images such as menu screens and electronic program guides used for functions.

  FIG. 2 is a block diagram illustrating a functional configuration of the main processing unit 12. As shown in FIG. 2, an MPEG decoder 121, an audio / image separation unit 122, a scale conversion unit 123, and a superimposition unit 124 are mainly provided.

  The MPEG decoder 121 decodes MPEG video data input from the video signal input unit 11. The audio / image separation unit 122 separates the decoded video data into audio data and image data (moving image data). Then, the sound / image separation unit 122 sends the sound data to the sound processing unit 17 and sends the image data (moving image data) to the scale conversion unit 123.

  The scale conversion unit 123 converts each frame of the input moving image data into image data having an image size of 1440 × 1080 (horizontal direction 1440 pixels, vertical direction 1080 lines). For example, in the case of moving image data input from a DVD or the like, each frame is converted into an SD resolution (720 × 480) image size of 1440 × 1080.

  When the input moving image data has a resolution of 1440 × 1080 (horizontal direction 1440 pixels, vertical direction 1080 lines), the scale conversion unit 123 does not perform scale conversion.

  Here, the scale conversion performed by the scale conversion unit 123 is different from the super-resolution conversion process described later. The image size is simply changed from an SD size of 720 × 480, an image size of 1280 × 720, or the like to an image size of 1440 × 1080. Only the scale conversion is performed, and processing such as pixel interpolation is not performed.

  The flash memory 23 stores graphics image data such as a menu screen and an electronic program guide that are displayed superimposed on moving image data displayed as an on-screen display function. The image size of the graphics image data is 1440 × 1080, and graphics image data other than this size is not stored.

  The superimposing unit 124 reads out the graphics image data from the flash memory 23 and superimposes the graphics image data on the frame of the moving image data. In addition, the superimposing unit 124 transmits the moving image data including the frames on which the graphics image data is superimposed to the high resolution unit 13 for each frame.

  FIG. 3 is a schematic diagram illustrating an example of graphics image data. As shown in FIG. 3, in the graphics image data, setting menu image data to be displayed as a setting menu is embedded in an area having a size of 1440 × 1080, and the periphery of the image is a video image when superimposed on a frame of the moving image data. It is a transparent area so that is displayed.

  As described above, in this embodiment, graphics image data having an image size of 1440 × 1080 is stored in advance, and frames of low-resolution moving image data such as the input SD resolution or 1280 × 720 are converted into 1440 × 1080. Is converted to the image size of the image, and then superimposed on the graphics image data of the image size of 1440 × 1080 in the flash memory 23, and then the super-resolution conversion process described later is performed on the superimposed frame of 1440 × 1080. ing. For this reason, it is possible to display a graphics image with higher image quality than when super-resolution conversion processing is performed after superimposing low-resolution graphics image data on a low-resolution frame. Further, it is not necessary to store graphics image data for each different image size, and the storage capacity of the flash memory 23 can be reduced.

  Returning to FIG. 1, the high resolution unit 13 inputs the moving image data of the resolution of 1440 × 1080 output from the main processing unit 12 for each frame, performs a super-resolution conversion process to be described later, and performs HD size (1920 × 1080) high-resolution moving image data. Here, the resolution of 1440 × 1080 is referred to as medium resolution for convenience of explanation.

  The moving image data input to the high resolution unit 13 is composed of a plurality of frames having a medium resolution of 1440 × 1080 on which graphics image data having an image size of 1440 × 1080 is superimposed. The moving image data has a frame rate of 60 fps so that 60 frames of medium resolution frame 1 are displayed per second.

  FIG. 4 is a block diagram showing a detailed configuration of the resolution increasing unit 13. As shown in FIG. 4, the resolution enhancement unit 13 mainly includes a preprocessing unit 131, a super resolution conversion processing unit 133, and a postprocessing unit 135.

  The pre-processing unit 131 includes an IP (interlace / progressive) conversion process or a video signal frame (hereinafter referred to as “medium-resolution frame”) that is input from the main processing unit 12 and included in the image signal. Preprocessing such as NR (noise reduction) processing for removing noise is performed, and the processed medium resolution frame is output to the super resolution conversion processing unit 133.

  Here, as the IP conversion processing, for example, it is determined whether the image data is a still image or a moving image by detecting the motion of the image included in the medium resolution frame. Interpolation processing is performed, and when it is determined as a moving image, moving image interpolation processing is performed.

  Noise reduction processing includes image data contour correction, reduction of image blur and glare, correction to prevent excessive equalization (high frequency emphasis), and blurring improvement when the camera moves in the horizontal direction. Can be mentioned.

  The super-resolution conversion processing unit 133 performs image processing (hereinafter referred to as “super-resolution conversion processing”) for increasing the resolution of the medium-resolution frame input from the pre-processing unit 131, and performs high resolution of HD size. Frame of the moving image data (hereinafter referred to as “high resolution frame”) is generated and output to the post-processing unit 135.

  Here, the super-resolution conversion process means that the original pixel value is estimated from the low-resolution or medium-resolution image signal (low-resolution frame or medium-resolution frame) that is the first resolution, and the number of pixels is increased. Means a sharpening process for restoring a high-resolution image signal (high-resolution frame).

  Here, the original pixel value is, for example, an image of the same subject as that obtained when a low-resolution (first resolution) image signal is obtained with a camera having the number of pixels of the high-resolution (second resolution) image signal. It refers to the value of each pixel of the image signal that is sometimes obtained.

  “Estimate and increase pixels” means that the original pixel value is estimated from the surrounding (intra-frame or inter-frame) image by capturing the image characteristics and using the image characteristics that are correlated. Means increasing the number of pixels.

  For the super-resolution conversion processing, it is possible to use a publicly known / public technique disclosed in Japanese Unexamined Patent Application Publication No. 2007-310837, Japanese Unexamined Patent Application Publication No. 2008-98803, Japanese Unexamined Patent Application Publication No. 2000-188680, and the like. As a technique for the super-resolution conversion processing of the present embodiment, for example, a technique for restoring an image having a frequency component higher than the Nyquist frequency determined by the sampling period of the input image is used.

  For example, when using the super-resolution conversion process disclosed in Japanese Patent Application Laid-Open No. 2007-310837, the change pattern of the pixel value in the target image area including the target pixel in the frame for each of a plurality of medium resolution frames. Select a plurality of corresponding points corresponding to a plurality of target image areas closest to the reference frame from the reference frame, set the sample value of the luminance at the corresponding point to the pixel value of the target pixel corresponding to the corresponding point, and Calculating a pixel value of a high-resolution frame corresponding to the reference frame, which is a high-resolution frame having a number of pixels larger than the number of pixels of the reference frame, based on the size of the sample value and the arrangement of a plurality of corresponding points. Thus, the original pixel value is estimated from the medium resolution frame and the number of pixels is increased to restore the high resolution frame.

  In addition, when using the super-resolution conversion process using self-congruent position search in the same frame image disclosed in Japanese Patent Application Laid-Open No. 2008-98803, the error of each pixel in the search area of the medium resolution frame is compared. The minimum first pixel position is calculated, and based on the first pixel position and the first error, the second pixel position around the first pixel, and the second error, The position where the error is minimized is calculated with decimal precision. Then, a decimal precision vector with this position as the end point and the target pixel as the start point is calculated, and an extrapolation vector of the decimal precision vector with the pixel on the screen not included in the search region as the end point is calculated using the decimal precision vector. calculate. Then, based on the decimal precision vector, the extrapolation vector, and the pixel value acquired from the image data, the pixel value of the high-resolution image having a larger number of pixels than the number of pixels included in the image data is calculated. The super-resolution conversion processing unit 133 restores the high-resolution frame by performing such processing to estimate the original pixel value from the medium-resolution frame and increase the number of pixels.

  Also, a super-resolution conversion process using mapping between a plurality of frame images disclosed in Japanese Patent Laid-Open No. 2000-188680 can be used.

  However, the method of super-resolution conversion processing in the super-resolution conversion processing unit 133 is not limited to the above, and by increasing the number of pixels by estimating the original pixel value from a low-resolution or medium-resolution image signal, Any method can be applied as long as it is processing for restoring a resolution image signal.

  Further, the super-resolution conversion processing unit 133 sets a sharpening parameter and performs a super-resolution conversion process using the sharpening parameter.

  The sharpening parameters include, for example, a characteristic parameter of a filter for image quality deterioration correction and an intensity parameter for setting the sharpening intensity.

  In the present embodiment, as described above, the main processing unit 12 performs scale conversion of the input low resolution frame to a resolution of 1440 × 1080 and inputs it to the high resolution unit 13. However, since the frame of the moving image data input after being scale-converted in this way is simply an image that has been input as a low-resolution video signal to the video signal input unit 11 to increase the image size. The amount of information is small as compared with the frame of the moving image data input as a 1440 × 1080 medium resolution video signal from the beginning by the signal input unit 11.

  Therefore, when the super-resolution conversion processing unit 133 performs the super-resolution conversion processing on the frame of the moving image data input from the video signal input unit 11 with the resolution of 1440 × 1080, the frame with the resolution of 1440 × 1080 is used. However, it is difficult to generate a high-definition high-resolution frame.

  For this reason, the super-resolution conversion processing unit 133 according to the present embodiment sets the sharpening parameter to a parameter corresponding to the resolution before the scale conversion for the frame of the moving image data input after the scale conversion. Super-resolution conversion processing is performed.

  Specifically, the super-resolution conversion processing unit 133 changes the characteristic parameter of the filter for image quality degradation correction to a characteristic parameter for correcting a frequency band lower than the frequency band to be corrected in the case of a frame having a resolution of 1440 × 1080. Set and perform super-resolution conversion processing. The band to be a low frequency is determined according to the resolution before the scale conversion, and the lower the resolution, the lower the frequency is set.

  The super-resolution conversion processing unit 133 sets the sharpening intensity parameter to a parameter having an intensity stronger than that in the case of a frame having a resolution of 1440 × 1080, and performs super-resolution conversion processing. The intensity to be set is determined according to the resolution before scale conversion, and is set as a stronger parameter as the resolution is lower.

  Thus, by setting the sharpening parameter as a parameter appropriate for the resolution before the scale conversion and performing the super-resolution conversion process, it becomes possible to display a higher-definition graphics image and video.

  The post-processing unit 135 performs image correction processing such as gamma correction, contrast expansion, gradation correction, and color management on the image of the high-resolution frame generated by the super-resolution conversion processing unit 133, and a subsequent moving image improvement processing unit 14 and sequentially stored in an EEPROM (Electrically Erasable and Programmable Read Only Memory) 21. A configuration in which such image correction processing is not performed may be employed.

  Returning to FIG. 1, the moving image improvement processing unit 14 generates an interpolation frame from image data composed of a plurality of high resolution frames input from the resolution increasing unit 13 and converts the frame to increase the frame rate of the image data. . In the moving image improvement processing unit 14 of the present embodiment, as frame rate conversion processing, interpolation frame generation processing is performed in which motion compensation is performed from two high-resolution frame images to generate an interpolation frame.

  More specifically, the moving image improvement processing unit 14 inputs a high-resolution frame that has been output from the high-resolution conversion unit 13 and has been subjected to the super-resolution conversion process, and on the other hand, is one before the high-resolution frame. The high-resolution frame after the super-resolution conversion processing is read out from the EEPROM 21. Then, a motion vector is calculated from the two high resolution frames and a motion compensation process is performed. Based on the result, an interpolation frame to be interpolated between the two high resolution frames is obtained. As detailed processing of such interpolation frame generation processing, a known method such as Japanese Patent Application Laid-Open No. 2008-35404 may be used. However, the interpolation frame generation processing performed in the moving image improvement processing unit 14 is not limited to such a method, and any method can be applied as long as the method generates an interpolation frame.

  In the present embodiment, frames per second are obtained by interpolating interpolated frames between the frames of moving image data having a frame rate of 60 fps after the super-resolution conversion processing is performed by the high resolution unit 13. It is converted into moving image data having a frame rate of 120 fps, which is 120.

  Here, the EEPROM 22 is a memory for temporarily storing a frame when generating an interpolation frame.

  Note that the frame rate after conversion by the moving image improvement processing unit 14 is not limited to this. In this embodiment, a single interpolation frame is interpolated between the high resolution frames 401, but the present invention is not limited to this, and a plurality of interpolation frames are inserted between the high resolution frames 401. You may comprise as follows.

  Returning to FIG. 1, the display processing unit 15 is a display driver or the like, and controls the display of the moving image data input from the moving image improvement processing unit 14 on the display unit 16. The display unit 16 is a display device such as a liquid crystal display panel, a plasma panel, or an SED (Surface-conduction Electron-emitter Display) panel, and displays a screen corresponding to an image signal under the control of the display processing unit 15. To do.

  The audio processing unit 17 converts the digital audio signal input from the main processing unit 12 into an analog audio signal in a format that can be reproduced by the audio output unit 18, and outputs the analog audio signal to the audio output unit 18. The audio output unit 18 is a speaker or the like, and outputs an analog audio signal input from the audio processing unit 17 as audio.

  Next, a description will be given of high resolution and moving image improvement processing of moving image data configured as described above. FIG. 5 is a flowchart showing the procedure of the entire process related to the display of moving image data according to the present embodiment.

  A video signal or the like of the digital broadcast received by the digital broadcast receiving unit 111 or the like is subjected to predetermined digital demodulation processing or the like by the video signal input unit 11 and is input to the main processing unit 12 as video data. It should be noted that data other than digital broadcasting is also input to the main processing unit 12.

  The main processing unit 12 performs main processing such as format conversion, video signal decoding processing, image data and audio data separation processing, graphics image data superimposition processing on the input video signal (step S11).

  FIG. 6 is a flowchart illustrating a procedure of main processing performed by the main processing unit 12. First, the MPEG decoder 121 inputs video data (step S31), and performs MPEG decoding processing of this video data (step S32).

  Then, the audio / image separation unit 122 separates the decoded video data into audio data and moving image data (image data) (step S33). The sound / image separation unit 122 sends the separated sound data to the sound output unit 17 (step S34). On the other hand, the audio / image separation unit 122 sends the separated moving image data to the scale conversion unit 123.

  The scale conversion unit 123 checks whether or not the image size of the frame constituting the input moving image data is a medium resolution image of 1440 × 1080 (step S35). If the frame image size is smaller than the medium resolution image of 1440 × 1080 such as SD resolution or 1280 × 720 (Step S35: No), each frame constituting the moving image data is converted to 1440 × 1080. The scale is converted to a medium resolution image (step S36).

  On the other hand, in step S35, when the image size of the frame is an image size of 1440 × 1080 (step S35: Yes), the scale conversion process is not performed.

  Next, the superimposing unit 124 reads the graphics image data (1440 × 1080) from the flash memory 23 (step S38), and superimposes the read graphics image data on the frame (step S39).

  Next, the superimposing unit 124 sends the frame on which the graphics image data is superimposed to the high resolution unit 13 (step S40).

  The superimposition process from step S38 to S40 is executed for all the frames constituting the moving image data (steps S37a and S37b). Such superimposition processing is executed only when the user gives an instruction to display a graphics image, such as a menu screen display or an electronic program guide display, by a controller or the like.

  Returning to FIG. 5, when the main process is completed, the resolution increasing unit 13 performs the resolution increasing process for each frame of the moving image data (step S12).

  FIG. 7 is a flowchart showing the procedure of the high resolution processing in step S12. First, the preprocessing unit 131 inputs a medium resolution frame having an image size of 1440 × 1080 constituting the moving image data sequentially output from the main processing unit 12 (step S52). Then, the preprocessing unit 131 performs preprocessing such as IP (interlace / progressive) conversion processing and NR (noise reduction) processing to remove noise included in the image signal on the input medium resolution frame. (Step S53).

  Next, the medium resolution frame subjected to such pre-processing is input to the super resolution conversion processing unit 133, and the super resolution conversion processing unit 133 sharpens according to the image size of the frame before the scale conversion of the medium resolution frame. Parameterization is set (step S54). Specifically, as described above, a parameter corresponding to the resolution before scale conversion is set as a sharpening parameter.

  Then, the super-resolution conversion processing unit 133 performs the above-described super-resolution conversion process on the medium resolution frame using the set sharpening parameter to sharpen (step S55). As a result, the medium resolution frame is converted into a 1920 × 1080 high resolution frame. The high-resolution frame converted from the medium-resolution frame by the super-resolution conversion process is subjected to image correction processing such as gamma correction by the post-processing unit 135 (step S56).

  The post-processing unit 135 stores the high-resolution frame subjected to the image correction processing in the EEPROM 21 (step S57), and further outputs the high-resolution frame to the moving image improvement processing unit 14 (step S58). The processing from step S52 to S58 is performed for all medium resolution frames of the input moving image data (steps S51a and S51b).

  Returning to FIG. 5, when such high resolution processing is performed, the moving image improvement processing unit 14 performs moving image improvement processing, that is, interpolation frame generation and interpolation, on the generated moving image data including high resolution frames. Processing is performed to convert the frame rate of the moving image data to 120 fps (step S13).

  Then, the display processing unit 15 inputs the moving image data with the converted frame rate, and the display processing unit 15 displays the moving image data on the display unit 16 (step S14). As a result, moving image data having high resolution of HD size and smooth movement is displayed on the display unit 16.

  As described above, in the present embodiment, the input SD resolution or the low resolution moving image data frame such as 1280 × 720 is scaled to the image size of 1440 × 480, and then the 1440 × 480 image of the flash memory 23 is converted. Since the super-resolution conversion process described later is performed on the 1440 × 18080 superimposed frame superimposed on the graphics image data of the size, the graphics image can be displayed with high image quality. Further, it is not necessary to store graphics image data for each different image size, and the storage capacity of the flash memory 23 can be reduced.

  In this embodiment, when performing super-resolution conversion processing on a frame having an image size of 1440 × 1080 on which graphics image data is superimposed, the sharpening parameter is set as a parameter appropriate for the resolution before scale conversion. Therefore, it becomes possible to display higher-definition graphics images and videos.

  In the present embodiment, an example in which the image processing apparatus of the present invention is applied to an image display apparatus 100 such as a digital TV having a display unit 16, a display processing unit 15, an audio output unit 18, and an audio processing unit 17 is given. As described above, the image processing apparatus of the present invention can be applied to, for example, a tuner, a set top box, or the like that does not include the display unit 16, the display processing unit 15, the audio output unit 18, and the audio processing unit 17. .

  It should be noted that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is a block diagram schematically showing a system of an image display device 100 according to the present embodiment. It is a block diagram which shows the functional structure of a main process part. It is a schematic diagram which shows an example of graphics image data. It is the block diagram which showed the detailed structure of the high resolution part. It is a flowchart which shows the procedure of the whole process concerning the display of the moving image data of this Embodiment. It is a flowchart which shows the procedure of the main process performed in a main process part. It is a flowchart which shows the procedure of high resolution processing.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Video signal input part 12 Main process part 13 High resolution part 14 Movie improvement process part 15 Display process part 16 Display part 17 Audio | voice process part 18 Audio | voice output part 21,22 EEPROM
23 Flash Memory 100 Image Display Device 112 IPTV Signal Processing Unit 114 External Input Unit 121 MPEG Decoder 122 Audio / Image Separation Unit 123 Scale Conversion Unit 124 Superimposition Unit 131 Preprocessing Unit 133 Super Resolution Conversion Processing Unit 135 Post Processing Unit 1112 Digital Tuner 1141 Analog antenna

Claims (9)

A conversion unit that converts the first resolution image signal into a second resolution image signal higher than the first resolution;
A storage unit for storing the graphics image of the second resolution;
A superimposing unit that superimposes the graphics image on the image signal of the second resolution;
Super-resolution conversion processing for restoring an image signal having a third resolution higher than the second resolution by estimating an original pixel value from the image signal having the second resolution on which the graphics image is superimposed and increasing the number of pixels. A high-resolution part that performs
An image processing apparatus comprising:   The high-resolution unit sets a sharpening parameter used for the super-resolution conversion process according to the first resolution, and the image signal of the second resolution on which the graphics image is superimposed by the set parameter. The image processing apparatus according to claim 1, wherein the super-resolution conversion process is performed upon input.   The high resolution unit converts the characteristic parameter of the image quality degradation correction filter as the sharpening parameter into a characteristic parameter for correcting a frequency band lower than a frequency band to be corrected in the case of the image signal of the second resolution. 3. The image processing according to claim 2, wherein the super-resolution conversion process is performed by inputting the second resolution image signal on which the graphics image is superimposed using the characteristic parameter of the filter. apparatus.   The high resolution unit sets a sharpening intensity parameter as the sharpening parameter to a parameter having a stronger intensity than that in the case of the image signal of the second resolution, and sets the set sharpening intensity parameter. The image processing apparatus according to claim 2 or 3, wherein the super-resolution conversion process is performed by inputting the second resolution image signal on which the previous graphics image is superimposed.   The said conversion part does not perform the said scale conversion with respect to the image signal of the said 1st resolution, when the said 1st resolution and the said 2nd resolution are equal. The image processing apparatus described in one. The second resolution is a resolution of 1440 pixels in the horizontal direction × 1080 lines in the vertical direction,
The image processing apparatus according to claim 1, wherein the third resolution is a resolution of 1920 pixels in the horizontal direction × 1080 lines in the vertical direction. A moving image improvement processing unit that increases a frame rate of the image signal of the third resolution generated by the super-resolution conversion processing;
An image processing apparatus comprising: A conversion unit that converts the image signal of the first resolution into an image signal of the second resolution higher than the first resolution;
A storage unit for storing the graphics image of the second resolution;
A superimposing unit that superimposes the graphics image on the image signal of the second resolution;
A resolution increasing unit that increases the resolution of the second resolution image signal on which the graphics image is superimposed into a third resolution image signal higher than the second resolution using a super-resolution technique;
An image processing apparatus comprising: Scaling the first resolution image signal to a second resolution image signal higher than the first resolution;
Superimposing the second resolution graphics image on the second resolution image signal;
Super-resolution conversion processing for restoring a third resolution image signal higher than the second resolution by estimating the original pixel value from the second resolution image signal on which the graphics image is superimposed and increasing the number of pixels. The steps of
An image processing method comprising:

Images