JP2009168975A - Image processing method, image processor, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress deterioration of image quality by performing frame interpolation and to make it possible to compress image data by a small amount of operations. <P>SOLUTION: This image processing method includes a color space conversion step into a second color space of converting image data in a first color space having predetermined tristimulus values into image data in the second color space having predetermined three stimulus values using a plurality of frames of image data corresponding to an image to be displayed as a processing unit, wherein in the color space conversion step to the second color space, image data with a stimulus value regarding an element influencing human eyes most of the image data with the tristimulus values in the second color space is obtained for each frame of the plurality of frames to be the processing unit, and image data with other stimulus values is obtained in a certain frame of the plurality of frames to be the processing unit. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing method, an image processing apparatus, and an image processing program.

  Various techniques for generating intermediate image data from two frames and displaying an image at an increased frame rate have been proposed to improve the visibility of moving images when displaying moving images on image display devices such as projectors. (For example, refer to Patent Document 1). The technique disclosed in Patent Document 1 (referred to as the first conventional technique) enables a smooth moving image display without flicker.

  On the other hand, various techniques have also been proposed for enabling playback of moving images even in an image display system configured by hardware with low transmission capability by compressing moving image data at a high frame rate (for example, Patent Document 2). reference). The technique (referred to as the prior art) disclosed in Patent Document 2 enables suppression of image degradation accompanying compression of image data and enables stable image compression at a high compression rate.

Japanese Patent Laid-Open No. 2003-69691 JP-A-8-265755

  By generating an intermediate image and performing frame interpolation as in the first prior art, the frame rate can be increased and a smooth moving image can be displayed. However, by simply performing frame interpolation, the image quality is improved. In some cases, there are still issues. For example, when frame interpolation is performed such that a frame of image data corresponding to a specific color component is inserted, a false color is generated, which may cause deterioration in image quality.

  In the second prior art, an image is divided into a plurality of small blocks, and discrete cosine transform or the like is performed for each of the divided blocks. For this reason, there is a problem that the amount of calculation for image compression is large and the calculation is complicated. Further, when processing is performed by dividing an image into blocks, so-called block noise is generated, which may cause deterioration in image quality.

  An object of the present invention is to provide an image processing method, an image processing apparatus, and an image processing program that can suppress image quality deterioration due to frame interpolation and that can also compress image data with a small amount of calculation.

  In the image processing method of the present invention, the second color having a predetermined tristimulus value from the image data of the first color space having a predetermined tristimulus value, with a plurality of frames of image data corresponding to the image to be displayed as a processing unit. A color space conversion step to a second color space for conversion to image data of the space, wherein the color space conversion step to the second color space includes human image data of tristimulus values in the second color space. The stimulus value image data relating to the element that most affects the eye is obtained for each frame of the plurality of frames serving as the processing unit, and the other stimulus value image data includes a plurality of frames serving as the processing unit. It is obtained in one frame.

As described above, according to the image processing method of the present invention, the stimulation value image data relating to the element that most affects the human eye is obtained for each frame of the plurality of frames serving as the processing unit, and images of other stimulation values are obtained. Data is obtained in one frame having a plurality of frames as the processing unit. By performing such processing, it is possible to reduce the amount of image data of two stimulus values out of the three stimulus values, and to compress the data amount of the entire image data corresponding to the image to be displayed. Can do. As a result, image data can be transferred at a low transfer rate, and a moving image can be displayed even in an image display system configured with hardware having a low transmission capability.

  Further, the image processing method of the present invention does not perform processing for dividing an image into a plurality of small blocks when performing image data compression, so there is no image quality degradation due to block noise, and complex cosine transformation such as discrete cosine transform is performed. Since no calculation is performed, the image data can be compressed with a small calculation amount.

In the image processing method of the present invention, image data in a color space other than the first color space is converted into image data in the first color space for each of a plurality of frames as the processing unit. Preferably, a color space conversion step is included, and the converted image data of the first color space is input to the color space conversion step to the second color space.
By including such a first color space conversion step, it is possible to cope with a case where image data corresponding to an image to be displayed has a color space other than the first color space.

The image processing method of the present invention preferably includes an image data output step of outputting the image data of the second color space converted by the color space conversion step to the second color space.
As a result, the image data converted into the second color space by the second color space conversion step can be output to an image display device or the like.

  In the image processing method of the present invention, the first color space is a YCbCr color space represented by luminance (Y) and color difference (Cb, Cr), and the second color space is red (R), green. (G), an RGB color space represented by blue (B), and the color space conversion step to the second color space includes, for the G image data, each of a plurality of frames serving as the processing unit. It is determined for each frame, and it is preferable that the image data of R and B is determined in one frame having a plurality of frames as the processing unit.

  In the image processing method of the present invention, the color space conversion step to the second color space performs a process of converting the image data in the YCbCr color space into image data in the RGB color space. By performing such a color space conversion process, the Y (luminance) component that most affects the human eye can be distributed to R and B as well. Thus, for example, when processing is performed to increase the frame rate of image data corresponding to G with respect to image data corresponding to R and B by performing interpolation of the frame of image data corresponding to G It is possible to suppress false colors that may occur. Thus, according to the image processing method of the present invention, it is possible to suppress false colors that may occur when frame interpolation is performed such as inserting a frame of image data corresponding to a specific color component. Can contribute to the improvement of image quality.

In the image processing method of the present invention, in the color space conversion step to the second color space, the R image data includes a Y value in a specific frame of the plurality of frames serving as the processing unit, and the processing. The G image data is obtained based on the average value of Cr obtained by averaging the Cr values of each frame of the plurality of frames serving as a unit, and the Y value in a specific frame among the plurality of frames serving as the processing unit. And the image data of B is a specific frame of the plurality of frames serving as the processing unit, based on the value of Cb in each frame of the plurality of frames serving as the processing unit and the value of Cr in each frame. The value of Y at
It is preferable to obtain it based on the average value of Cb obtained by averaging the Cb values of the respective frames of the plurality of frames serving as the processing unit.

  This is color space conversion processing when priority is given to motion reproducibility when performing color space conversion processing from the YCbCr color space to the RGB color space. When priority is given to motion reproducibility, Y ( Instead of the average value of luminance, the luminance (Y) obtained in a specific frame (for example, the first frame of a plurality of frames as a processing unit) is used. As described above, by using the luminance (Y) obtained in a specific frame, it is possible to obtain image data giving priority to motion reproducibility.

  In the image processing method of the present invention, in the color space conversion step to the second color space, the R image data includes an average value of Y obtained by averaging the Y values of each frame of the plurality of frames serving as the processing unit. And the average value of Cr obtained by averaging the Cr values of each frame of the plurality of frames serving as the processing unit, and the G image data is obtained by calculating the Y value of each frame of the plurality of frames serving as the processing unit. The average value of Y is obtained based on the Cb value in each frame of the plurality of frames serving as the processing unit and the Cr value in each frame, and the B image data is obtained from the plurality of frames serving as the processing unit. It is preferable to obtain the average value of Y obtained by averaging the Y values of the respective frames and the average value of Cb obtained by averaging the Cb values of the respective frames of the plurality of frames serving as the processing unit.

  This is a color space conversion process when priority is given to color reproducibility when performing a color space conversion process from the YCbCr color space to the RGB color space. By generating image data corresponding to red (R), green (G), and blue (B), the generated image data becomes image data in which generation of false colors is suppressed. As described above, when priority is given to color reproducibility, a color close to the original image can be obtained by using an average value for Y (luminance).

  In the image processing method of the present invention, each frame of the image data corresponding to the image to be displayed is m (m is an integer of 2 or more) times the frame rate of the base image data supplied from the image source. When frame interpolation is performed so as to achieve a frame rate, the number of frames serving as the processing unit is preferably set to m.

  For green (G), intermediate image data is generated, and the frame rate of image data corresponding to green (G) is set to m of the frame rate of image data corresponding to red (R) and blue (B). In this case, the number of frames as a processing unit is set to m. In this way, by setting the number of frames as a processing unit and executing the image processing method of the present invention, the frame rate of the image data corresponding to green (G) corresponds to red (R) and blue (B). It is possible to suppress false colors that may occur when processing for increasing the image data is performed.

In the image processing method of the present invention, the image data corresponding to the image to be displayed is image data given to the projector, and the projector modulates the color light from the light source based on the image data. A first image forming unit that has an element, synthesizes the respective color lights modulated by the three light modulation elements and emits the first image light, and three lights that modulate the color light from the light source based on the image data A second image forming unit that has a modulation element, synthesizes the respective color lights modulated by the three light modulation elements, and emits the second image light; and the first image forming unit and the second image forming unit The first image formation comprising: a polarization combining optical system that combines the first image light and the second image light; and a projection optical system that projects the image light combined by the polarization combining optical system onto a projection surface. Means and number Of the total six light modulation elements provided in the image forming means, four light modulation elements are light modulation elements corresponding to green light, and the remaining two light modulation elements are light modulation elements corresponding to red light and blue light. 4 corresponding to the light source and the green light are used so that each color light modulated by the four light modulation elements corresponding to the green light is sequentially emitted with a predetermined time shift in each frame of the image data. Preferably, the projector is one in which two light modulation elements are controlled.

  This is a case where the image processing method of the present invention is applied to a projector having two image forming means (first image forming means and second image forming means) and one projection optical system. Such a projector has a total of six light modulation elements (such as a liquid crystal panel) in each of the first image forming means and the second image forming means. Of these, four are used as light modulation elements corresponding to blue (G), and image data corresponding to green (G) is four times as many frames as image data corresponding to red (R) and blue (B). It is possible to display at a rate. In such a projector, false interpolation may occur by simply performing frame interpolation by inserting a frame of image data corresponding to green (G). By applying the image processing method of the present invention, Generation of false colors can be suppressed.

  The image processing apparatus of the present invention uses a plurality of frames of image data corresponding to an image to be displayed as a processing unit, and second color having predetermined tristimulus values from image data in a first color space having predetermined tristimulus values. A color space conversion unit to a second color space for converting into image data of the space, and the color space conversion unit to the second color space includes the human eye among the tristimulus value image data of the second color space. The image data of the stimulus value relating to the element that most influences the image is obtained for each frame of the plurality of frames serving as the processing unit, and the image data of the other stimulus values is provided by one of the plurality of frames serving as the processing unit. It is characterized in that it is obtained in a frame.

The image processing apparatus of the present invention can be applied to, for example, a projector having two image forming units and one projection optical system, a direct-view image display apparatus, and the like. When the image processing apparatus of the present invention is applied to a projector having two image forming units and one projection optical system, the effect of suppressing the occurrence of false colors can be obtained as described above.
Further, when the image processing apparatus of the present invention is applied to a direct view type image display apparatus or the like, the image data can be compressed, so that image data requiring a high transfer rate is lower than that of the image display apparatus. It can be sent at a transfer rate. As a result, a moving image can be displayed even in an image display system configured with hardware having a low transmission capability. Also, the image processing apparatus of the present invention preferably has the respective characteristics of the image processing method of the present invention.

  An image processing program of the present invention is an image processing program for causing a computer to execute image processing, and has a first tristimulus value having a plurality of frames of image data corresponding to an image to be displayed as a processing unit. A color space conversion step to a second color space for converting the image data of the color space to image data of a second color space having a predetermined tristimulus value, and the color space conversion step to the second color space comprises: Among the image data of the tristimulus values in the second color space, the image data of the stimulus values relating to the element that most affects the human eye is obtained for each frame of the plurality of frames as the processing unit, and other stimulus values The image data is obtained in one frame having a plurality of frames as the processing unit.

  By causing a computer to execute such an image processing program, the above-described image processing method of the present invention can be realized, and the same effect as the image processing method of the present invention can be obtained. Also, the image processing program of the present invention preferably has the respective characteristics of the image processing method of the present invention.

Hereinafter, embodiments of the present invention will be described. The present invention provides image data in a second color space having a predetermined tristimulus value from image data in a first color space having a predetermined tristimulus value, with a plurality of frames of image data corresponding to the image to be displayed as a processing unit. Perform color space conversion processing to convert to. When performing color space conversion processing for conversion to image data of the second color space, among the image data of the tristimulus values of the second color space, the image data of the stimulus values relating to the elements that most affect the human eye are: Processing is performed for each frame of a plurality of frames serving as a processing unit, and image data of other stimulus values is determined in one frame of the plurality of frames serving as a processing unit.

  In the embodiment described below, the image data of the first color space having predetermined tristimulus values is image data of the YCbCr color space, and the image data of the YCbCr color space is converted into RGB by a predetermined color space conversion process. An example of conversion to color space image data will be described. In the embodiment described below, as input image data, image data in a color space other than the YCbCr color space (referred to as image data in the RGB color space) is given. First, the image data in the RGB color space is converted to YCbCr. Conversion to image data in the color space, conversion of the image data converted to image data in the YCbCr color space into RGB color space, and output of the image data converted into RGB color space to the image display device are performed. Shall.

FIG. 1 is a diagram schematically showing a configuration of an image processing apparatus according to an embodiment of the present invention. As shown in FIG. 1, the image processing apparatus 10 according to the embodiment of the present invention represents image data (assuming that it is image data in the RGB color space) corresponding to an image to be displayed by luminance and color difference. A first color space conversion unit (referred to as a YCbCr color space conversion unit) 11 that converts image data in the first color space (referred to as image data in the YCbCr color space), and image data in the YCbCr color space in the second color space. A second color space conversion unit (RGB color space conversion unit) 12 that converts to image data (image data in the RGB color space), and red (R) and green (converted by the RGB color space conversion unit 12) An image data output unit 13 for outputting image data corresponding to each of G) and blue (B) (image data in the RGB color space) to image display means (not shown).
Note that the color space conversion processing by the YCbCr color space conversion unit 11 and the RGB color space conversion unit 12 is performed using a plurality of frames of image data (four frames in the embodiment of the present invention) as processing units.

  FIG. 2 is a diagram for explaining basic image processing performed by the image processing apparatus 10 shown in FIG. Basic image processing performed by the image processing apparatus 10 will be described with reference to FIGS. 2 and 1. First, as shown in FIG. 2A, image data for four frames corresponding to red (R) (referred to as R1, R2, R3, and R4) and four frames corresponding to green (G). Image data (G1, G2, G3, G4) and four frames of image data (B1, B2, B3, B4) corresponding to blue (B) are sequentially processed from the first frame. Shall be given to

  The numbers “1, 2, 3, 4” attached to the image data for four frames corresponding to red (R), green (G), and blue (B) represent frame numbers, and “1” is It is assumed that it is the first frame (first frame) in the four frames of the processing unit. For example, in the image data R1, R2, R3, R4 for four frames corresponding to red (R), each number is such that R1 is the first frame (first frame) and R2 is the second frame. Indicates a frame number. The same applies to G1, G2, G3, G4 and B1, B2, B3, B4.

Then, in the YCbCr color space conversion unit 11, YCbCr represented by luminance Y, blue color difference Cb, and red color difference Cr as processing unit image data (RGB color space image data) shown in FIG. Convert to color space. Note that the process of converting the processing unit image data (image data in the RGB color space) into the YCbCr color space is referred to as a YCbCr color space conversion step (first color space conversion step) S1. FIG. 2 (b) shows the YCbCr color space conversion step S1.
Image data converted to a Cr color space (referred to as YCbCr color space image data).

In the YCbCr color space conversion data shown in FIG. 2B, the numbers “1, 2, 3, 4” attached to Y, Cb, Cr are the same as the above, and the four frames as the processing unit. Indicates the frame number. That is, Y1, Cb1, and Cr1 are YCbCr color space image data in the first frame (first frame) of the four frames, and Y2, Cb2, and Cr2 are YCbCr color space image data in the second frame, and Y3 and Cb3. , Cr3 are YCbCr color space image data in the third frame, and Y4, Cb4, Cr4 are YCbCr color space image data in the fourth frame.
Ya is the average value of Y1, Y2, Y3, and Y4, Cba is the average value of Cb1, Cb2, Cb3, and Cb4, and Cra is the average value of Cr1, Cr2, Cr3, and Cr4.

  When the YCbCr color space image data converted into the YCbCr color space is obtained in this way, the RGB color space conversion unit 12 now converts the YCbCr color space image data into image data in the RGB color space. Do. The process of converting the YCbCr color space image data into the RGB color space image data is referred to as an RGB color space conversion step (second color space conversion step) S1. In the RGB color space conversion step S2, when converting from the YCbCr color space to the image data in the RGB color space, the image data corresponding to G is generated every four frames, and the images corresponding to R and B are generated. Data generation is performed only for one frame (referred to as the first frame) of the four frames. Thereby, for G, G image data (G1, G2, G3, G4) corresponding to each frame for four frames is obtained, and for R and B, image data R1 and B1 of only the first frame are obtained. (See (d) of FIG. 2).

  FIG. 2C shows a conversion formula from the YCbCr color space to the RGB color space. As shown in FIG. 2C, as a conversion formula from the YCbCr color space to the RGB color space, priority is given to a conversion formula that gives priority to motion reproducibility (referred to as motion priority) and color reproducibility (color priority). Two types of conversion formulas are set.

Here, the conversion formula with motion priority is
R = Yx + Cra × 1.4020 (1)
G = Yx−Cbi × 0.3441−Cri × 0.7139 (2)
B = Yx + Cba × 1.7718 (3)
And

On the other hand, the conversion formula with color priority is
R = Ya + Cra × 1.4020 (4)
G = Ya−Cbi × 0.3441−Cri × 0.7139 (5)
B = Ya + Cba × 1.7718 (6)
And

  In the above formulas (1) to (3), the subscript x of Y indicates a specific frame number among the four frames. Therefore, if the first frame (first frame) of the four frames is used as the specific frame number, it is expressed as Y1.

  In the expressions (2) and (5), the subscript i of Cb and Cr is the frame number at that time. Therefore, Cbi and Cri are the values of Cb and Cr of the frame to be converted into the RGB color space at that time. For example, if the frame number of the conversion target frame is “1” at that time, it is represented as Cb1, Cr1.

Here, when conversion to the RGB color space is performed with priority given to motion, if x of Rx is set to 1 (first frame), R1 and B1 in FIG. 3) From the equation
R1 = Y1 + Cra × 1.4020 (7)
B1 = Y1 + Cba × 1.7718 (8)
It can be obtained as follows.

In addition, G1 to G4 in (d) of FIG.
G1 = Y1-Cb1 * 0.3441-Cr1 * 0.7139 (9)
G2 = Y1-Cb2 * 0.3441-Cr2 * 0.7139 (10)
G3 = Y1-Cb3 * 0.3441-Cr3 * 0.7139 (11)
G4 = Y1-Cb4 * 0.3441-Cr4 * 0.7139 (12)
It can be asked.

  As described above, when priority is given to motion, instead of the average value Ya of Y (luminance), the luminance obtained in a specific frame (for example, the first frame of a plurality of frames to be processed) is used. As described above, by using the luminance (Y) obtained in a specific frame, it is possible to obtain image data giving priority to motion reproducibility.

On the other hand, when priority is given to color, the average brightness Ya is used as shown in the equations (4) to (6). As described above, the luminance Y can be a color close to the original image by using the average luminance Ya. In the case of color priority, R1 and B1 in (d) of FIG. 2 are obtained from the above equations (1) and (3).
R1 = Ya + Cra × 1.4020 (13)
B1 = Ya + Cba × 1.7718 (14)
It can be obtained as follows.

In addition, G1 to G4 in (d) of FIG.
G1 = Ya−Cb1 × 0.3441−Cr1 × 0.7139 (15)
G2 = Ya-Cb2 × 0.3441-Cr2 × 0.7139 (16)
G3 = Ya-Cb3 × 0.3441-Cr3 × 0.7139 (17)
G4 = Ya-Cb4 * 0.3441-Cr4 * 0.7139 (18)
It can be asked.

  As described above, the conversion formula to the RGB color space differs depending on whether motion priority is given or color priority, but whether to give priority to motion or color priority is selected according to the user's preference and the content of the image to be displayed. Can do.

  FIG. 2D shows image data (referred to as RGB color space image data) converted into the RGB color space by the RGB color space conversion step S2. As shown in FIG. 2 (d), the image data corresponding to R and B is obtained only in the first frame (first frame) of the four frames serving as processing units (R1). And B1). Also, the image data corresponding to G is obtained for every frame of the four frames that are processing units (G1, G2, G3, G4). Then, the RGB color space image data as shown in FIG. 2D is output from the image data output unit 13 and subjected to display processing by the image display means.

By performing the image processing as described in FIG. 2, for example, when 4 frames are used as a processing unit, in the RGB color space conversion step S <b> 2, G is subjected to conversion processing for each frame, and R and B With respect to, since it is only necessary to perform conversion processing for one frame (first frame), the amount of calculation can be reduced, and the data amount when transferring the image data converted into the RGB color space to the image display means can be reduced. Can be reduced. Thereby, the transfer rate when transferring the image data to the image display means can be kept low.

  Note that the image data in the RGB color space is once converted into the YCbCr color space, and then converted into the RGB color space again after the conversion into RGB, which is the Y (luminance) component that most affects the human eye. This is because the effect of suppressing the occurrence of false colors can be obtained even when the amount of data is reduced by dispersing the R into B and R. Thereby, even if it is a case of image data with a quick motion, generation | occurrence | production of a false color can be suppressed. Note that the image data in the RGB color space is finally used because the image display means generally performs image display processing in the RGB color space.

  Next, an application example of the present invention using the above-described image processing method (referred to as basic image processing of the present invention) will be described.

[First application example]
In the first application example, the basic image processing of the present invention is applied to a projector having two image forming means (referred to as an image forming unit) and one projection optical system.

  FIG. 3 is a diagram schematically showing an optical configuration of a projector having two image forming units and one projection optical system. As shown in FIG. 3, the projector used in the first application example includes two image forming units 100 and 200 (referred to as a first image forming unit 100 and a second image forming unit 200), a first image forming unit 100, and A polarization combining prism 300 as a polarization combining optical system that combines the respective image lights (first image light and second image light) emitted from the second image forming unit 200, and an image combined by the polarization combining prism 300 A projection optical system 400 that projects light onto a screen SCR as a projection surface.

  The first image forming unit 100 has first to third light source devices 110 to 130 as light source devices, and the second image forming unit 200 has fourth to sixth light source devices 210 to 230 as light source devices. Yes. The first to third light source devices 110 to 130 and the fourth to sixth light source devices 210 to 230 use solid light sources (LEDs) as light sources.

  The first light source device 110 in the first image forming unit 100 includes a plurality of red light sources R1 configured by a plurality of red LEDs that emit red light and a plurality of red LEDs that configure the red light sources R1. The condenser lens 111, the rod integrator 112, and the condenser lens 113 provided on the output side of the rod integrator 112 are included.

  In addition, the second light source device 120 includes a first green light source G1 composed of a plurality of green LEDs that emit green light, and a plurality of collections provided corresponding to the plurality of green LEDs that constitute the first green light source G1. It has an optical lens 121, a rod integrator 122, and a condenser lens 123 provided on the output side of the rod integrator 122.

  The third light source device 130 includes a second green light source G2 composed of a plurality of green LEDs that emit green light, and a plurality of light sources provided corresponding to the plurality of green LEDs constituting the second green light source G2. It has an optical lens 131, a rod integrator 132, and a condensing lens 133 provided on the output side of the rod integrator 132.

The first image forming unit 100 includes light modulation elements 150R, 150G1, and 150G2 corresponding to the first to third light source devices 110 to 130, and light modulation elements 150R and 150G.
It further has a synthesis optical system 160 that synthesizes each color light modulated by 1,150G2. The composite optical system 160 uses a cross dichroic prism, reflects red light modulated by the light modulation element 150R, transmits green light modulated by the light modulation element 150G1, and is modulated by the light modulation element 150G2. It has the function of reflecting green light and combining these color lights.

  The fourth light source device 210 in the second image forming unit 200 is provided corresponding to a blue light source B1 composed of a plurality of blue LEDs emitting blue light and a plurality of blue LEDs constituting the blue light source B1. A plurality of condenser lenses 211, a rod integrator 212, and a condenser lens 213 provided on the output side of the rod integrator 212 are provided.

  The fifth light source device 220 includes a third green light source G3 composed of a plurality of green LEDs that emit green light, and a plurality of light sources provided corresponding to the plurality of green LEDs constituting the third green light source G3. It has an optical lens 221, a rod integrator 222, and a condenser lens 223 provided on the output side of the rod integrator 222.

  The sixth light source device 230 includes a fourth green light source G4 composed of a plurality of green LEDs emitting green light and a plurality of light sources provided corresponding to the plurality of green LEDs constituting the fourth green light source G4. It has an optical lens 231, a rod integrator 232, and a condensing lens 233 provided on the output side of the rod integrator 232.

The second image forming unit 200 combines the light modulation elements 250B, 250G3, and 250G4 corresponding to the fourth to sixth light source devices 210 to 230 and the respective color lights modulated by the light modulation elements 250B, 250G3, and 250G4. It further has a synthesizing optical system 260. This synthetic optical system 260 uses a cross dichroic prism, reflects blue light modulated by the light modulation element 250B, reflects green light modulated by the light modulation element 250G3, and is modulated by the light modulation element 250G4. It has a function of transmitting green light and synthesizing these color lights.
The light modulation elements 150R, 150G1, and 150G2 and the light modulation elements 250B, 250G3, and 250G4 are light modulation elements (liquid crystal panels) using liquid crystals.

  By the way, the wavelength band of green light is about 500 nm to 565 nm. However, in the projector used in the first application example, green light having a wavelength band of about 500 nm to 565 nm is converted into a wavelength band closer to blue light (500 nm). Two types of green light: green light (referred to as first green light) having a wavelength band of ~ 532 nm) and green light (referred to as second green light) in a wavelength band close to yellow light (wavelength band of 533 nm to 565 nm). Use as

  Thus, by using green light as two types of green light, the first green light and the second green light, the composite optical system 160 of the first image forming unit 100 and the composite optical system of the second image forming unit 200 are used. A general cross dichroic prism 260 can be used.

  In the projector used in the first application example, the first green light having a wavelength band of 500 nm to 532 nm is given to the light modulation element 150G1 of the first image forming unit 100 and the light modulation element 250G3 of the second image forming unit 200. The light modulation element 150G2 of the first image forming unit 100 and the light modulation element 250G4 of the second image forming unit 200 are set so that the second green light having a wavelength band of 533 nm to 565 nm is given. And

FIG. 4 is a diagram for explaining the relationship between the operation of the light source and the driving of the light modulation element in the projector shown in FIG. 4A shows the operation of the first green light source G1 corresponding to the light modulation element 150G1, FIG. 4B shows the operation of the second green light source G2 corresponding to the light modulation element 150G2, and FIG. 4C shows the light modulation. Operation of the third green light source G3 corresponding to the element 250G3, FIG. 4D shows the operation of the fourth green light source G4 corresponding to the light modulation element 250G4. FIG. 4E shows the operation of the red light source R1 corresponding to the light modulation element 150R, and FIG. 4F shows the operation of the blue light source B1 corresponding to the light modulation element 250B.

  In FIGS. 4A to 4F, gray areas indicate that the light source is turned on, and white areas indicate that the light source is turned off. 4A to 4D, the green light uses the first green light having a wavelength band of 500 nm to 532 nm and the second green light having a wavelength band of 533 to 565 nm. Are divided into dark gray (first green light) and light gray (second green light).

  As shown in FIGS. 4A to 4D, in the projector used in the first application example, the first green light source G1, the second green light source G2, and the second image forming unit 200 of the first image forming unit 100 are used. The third green light source G3 and the fourth green light source G4 are sequentially turned on every time corresponding to a quarter frame obtained by dividing one frame of image data into four (this is Δt time). It lasts for a time corresponding to 2/4 frame (2Δt time). Accordingly, the first green light source G1, the second green light source G2, the third green light source G3, and the fourth green light source G4 are each a green light source (first green light source G1 and G1) that emits the first green light in the lighting state for 2Δt hours. The third green light source G3) and the green light sources emitting the second green light (the second green light source G2 and the fourth green light source G4) are turned on by overlapping by Δt time.

  For example, regarding the second frame F2, the first green light source G1 and the fourth green color are used in the time corresponding to the first quarter frame obtained by equally dividing the second frame F2 into four (Δt1 time). The light source G4 is turned on at the same time, and the first green light source G1 and the second green light source G2 are turned on at the same time during the time corresponding to the second quarter frame (assumed as Δt2 time). In addition, during the Δt time (Δt3 time) corresponding to the third quarter frame, the second green light source G2 and the third green light source G3 are turned on simultaneously, and the fourth quarter frame is reached. In the corresponding Δt time (referred to as Δt4 time), the third green light source G3 and the fourth green light source G4 are turned on simultaneously.

As described above, the green light source that emits the first green light and the green light source that emits the second green light are simultaneously turned on in order to prevent the human eyes from seeing green as two types of green. It is. That is, by using two types of green light (first green light and second green light), a general cross dichroic prism can be used as a composite optical system. Since it may be visually recognized as a kind of green, in order to prevent this, a green light source that emits first green light and a green light source that emits second green light are turned on simultaneously.
4E and 4F show the operations of the red light source R1 and the blue light source B1, and the red light source R1 and the blue light source B1 perform continuous lighting.

  FIGS. 4G to 4L show the driving time of each light modulation element, FIG. 4G shows the driving time of the light modulation element 150G1, and FIG. 4H shows the light modulation element 150G2. 4 (i) is the driving time of the light modulation element 250G3, FIG. 4 (j) is the driving time of the light modulation element 250G4, FIG. 4 (k) is the driving time of the light modulation element 150R, and FIG. ) Indicates the drive time of the light modulation element 250B, and the gray portion indicates the time during which each light modulation element is driven.

As shown in FIGS. 4G to 4L, the light modulation elements 150G1 and 150G corresponding to green light.
2 and the light modulation elements 250G3 and 250G4 are driven so as to correspond to the lighting of the first to fourth green light sources G1 to G4. At this time, the driving time is a time including the Δt time before and after the lighting time (2Δt time) of the first to fourth green light sources G1 to G4. Therefore, in this case, the driving time of the light modulation elements 150G1 and 150G2 and the light modulation elements 250G3 and 250G4 corresponding to green light is 4Δt time (time corresponding to one frame). Each color light modulated by 150G2 and the light modulation elements 250G3 and 250G4 is emitted for 2Δt time.

  As described above, the time including the time Δt before and after the lighting time (2Δt time) of the first to fourth green light sources G1 to G4 is used as the driving time of each light modulation element. This is because the light source emits light during the stable operation period of the liquid crystal.

  Of the light modulation elements 150G1, 150G2, 250G3, and 250G4 corresponding to green light, the light modulation element 150G1 corresponds to the frame (referred to as the nth frame) of the frames originally included in the image data. For example, three intermediate image data generated from the nth frame and the (n + 1) th frame of the image data are given to the other light modulation elements 150G2, 250G3, and 250G4. It is preferable. For the generation of the intermediate image data, various commonly used intermediate image data generation techniques such as an intermediate image generation technique based on a motion vector can be employed.

The projector used in the first application example is configured as shown in FIG. 3, and the light source and the light modulation element are driven as shown in FIG. 4, so that the frame rate of the image data corresponding to the green light is set to red light. In addition, the image data corresponding to blue light can be set to 4 times.
As described above, according to the projector used in the first application example, the frame rate of the image data corresponding to the green light can be four times that of the image data of the red light and the blue light. It can be data.

  However, when performing frame interpolation such as inserting a frame of image data corresponding to a specific color component (in this case, green) of RGB colors, like the projector used in the first application example, If a frame of image data corresponding to a specific color component is simply inserted as it is, a false color may be generated, which causes deterioration in image quality. In order to suppress the occurrence of such false colors, the projector used in the first application example performs the basic image processing of the present invention as described in FIG. 2 by the image processing apparatus 10 shown in FIG. It is preferable.

  In the first application example, as can be seen from FIG. 4, the image data corresponding to the green light has a frame rate four times that of the image data corresponding to the red light and the image data corresponding to the blue light. Therefore, it is assumed that the processing unit when executing the basic image processing of the present invention is four frames of image data corresponding to green light. Accordingly, it is assumed that the image data corresponding to the red light and the blue light has four frames that hold the same value within the time corresponding to four frames of green light. That is, the processing unit when executing the basic image processing of the present invention is “R1, R1, R1, R1” for image data corresponding to red light, and “G1, for image data corresponding to green light. “G2, G3, G4” and image data corresponding to blue light are “B1, B1, B1, B1”.

  By the way, in the first application example, each of the image data corresponding to RGB and the red light source, the green light source, and the blue light source correspond to each other. Therefore, the image data R1, the image data G1 to G4, Image data B1, red light source R1, green light sources G1 to G4, and blue light source B1 are assigned the same reference numerals to the image data and the corresponding light sources.

  FIG. 5 is a diagram for explaining image processing of the projector used in the first application example. As shown in FIG. 5, the image processing apparatus 10 includes “R1, R1, R1, R1”, “G1, G2, G3, G4”, and “B1, B1, B1, B1” as image data for each processing unit. Given. By inputting such image data to the image processing apparatus 10, the image processing apparatus 10 can perform the basic image processing of the present invention described in FIG.

  That is, as described in FIG. 2, the image processing apparatus 10 performs conversion processing from the RGB color space to the YCbCr color space (YCbCr color space conversion step S1) for the processing unit image data, and thereafter, YCbCr. Conversion processing from the color space to the RGB color space is performed (RGB color space conversion step S2). Thereby, the RGB color space image data as shown in FIG. 5 is output from the image data output unit 13 of the image processing apparatus 10 to the image display means (light modulation element driving units 21R, 21G1 to 21G4, 21B). The

  Here, among the RGB color space image data output from the image data output unit 13, the image data R1 is provided to the light modulation element driving unit 21R, and the image data B1 is provided to the light modulation element driving unit 21B. Further, the image data G1 to G4 are given to the light modulation element driving unit 21G1 and the image data G2 is given to the light modulation element driving unit 21G2 within the time corresponding to the first frame F1 shown in FIG. The image data G3 is supplied to the light modulation element driving unit 21G3, and the image data G4 is supplied to the light modulation element driving unit 21G4.

  As described above, even if the amount of data is reduced by performing the basic image processing (see FIG. 2) of the present invention on the processing unit image data input to the image processing apparatus 10 in FIG. Color generation can be suppressed. That is, the image data in the RGB color space is once converted into the YCbCr color space, and then converted into the RGB color space, so that Y ( (Luminance) component can also be dispersed in R and B, thereby obtaining the effect of suppressing the occurrence of false colors.

[Second application example]
In the second application example, the basic image processing algorithm of the present invention shown in FIG. 2 is applied to image data compression.

  FIG. 6 is a diagram showing a configuration of an image display system as a second application example. The image display system as the second application example includes the image processing apparatus 10 that performs the basic image processing of the present invention shown in FIG. 2 and the image display apparatus 30 as image display means. Note that the image display device 30 is, for example, a direct-view image display device. Note that the image processing apparatus 10 and the image display apparatus 30 may be configured to be directly connected by a cable, or may be configured to be connected via a network.

  In the first application example described above, the frame rate of the image data corresponding to green (G) is four times that of the image data corresponding to red (R) and blue (B). Although the number of frames that have been processed is four, in the second application example, the number of frames in the processing unit of image data can be arbitrarily set. For example, the number of frames can be reduced when the moving image is fast moving, and the number of frames can be increased when the moving image data is slow moving.

Here, for simplification of explanation, it is assumed that the number of frames (assumed to be n) is n = 4, that is, the number of frames as a processing unit is 4. In this case, as shown in FIG. 6, the image processing apparatus 10 includes four frames of “R1, R2, R3, R4”, “G1, G2, G3, G4”, and “B1, B2, B3, B4”. Are sequentially given as processing units. When such image data in the RGB color space is input to the image processing apparatus 10, the image processing apparatus 10 performs the basic image processing of the present invention described in FIG.

  That is, as described with reference to FIG. 2, the image processing apparatus 10 first performs conversion processing from the RGB color space to the YCbCr color space (YCbCr color space conversion step S1), and then from the YCbCr color space to the RGB color space. (RGB color space conversion step S2). Thereby, as shown in FIG. 6, the compressed RGB color space image data is output from the image data output unit 13 of the image processing apparatus 10, and the RGB color space image data is transmitted to the image display device 30. .

  In the image display device 30, for the red (R) and blue (B) of the transmitted image data, the image data R1 and B1 of the first frame (first frame) are held in the memory, and the green (G) Is updated on the memory every time the image data G1, G2,... Is transmitted, and the image data G1, G2,. The image display process is performed based on the above.

  Note that a GPU (Graphics Processing Unit) or the like included in the image display device 30 normally performs a process of updating the image data of each RGB color on the memory every time each RGB color is transmitted. In order to realize the invention, for image data corresponding to green (G), every time image data corresponding to green (G) is transmitted, image data G1, G2, and G2 corresponding to green (G) are transmitted. G3 and G4 are sequentially updated on the memory, and the respective image data R1 and B1 corresponding to red (R) and blue (B) are held until the update of four frames of green (G) is completed. It shall have. Such a function can be achieved by slightly changing the image processing function of the image display device 30.

  As described above, in the second application example, when the image data corresponding to the image to be displayed is image data that requires a high transfer rate, the image processing apparatus uses four frames of the image data as a processing unit. 10, the basic image processing of the present invention is performed, and the image data compressed thereby is transmitted to the image display device 30. As a result, image data having a large amount of data and requiring a high transfer rate can be transmitted to the image display device 30 at a low transfer rate, and even in an image display system configured with hardware having a low transmission capability, Display is possible.

  In the second application example, the number of frames n as a processing unit can be arbitrarily set according to the speed of motion of the moving image as described above. As a setting method, for example, An example is a method in which the sum of motion vector amounts of two frames is obtained and determined based on the obtained sum of motion vector amounts. For example, a threshold value is set for the sum of motion vector amounts, and the value of n is increased from, for example, 1 in order until the calculated motion vector amount exceeds the threshold value, and the motion vector amount exceeds the threshold value. There is also a method of determining n at the time as the number of frames at that time.

  Further, the parameter for determining the speed of motion is not limited to the motion vector amount. For example, in the motion priority processing shown in FIG. 2, the parameter is obtained by summing the mean square of Yx and Yi. It may be a value. In the case of color priority processing, it may be a value obtained by summing the mean square of Ya and Yi. Here, the subscript i of Yi is the frame number to be processed at that time, and Yi is the Y value of YCbCr obtained at the frame number.

6 describes the case where the image processing apparatus 10 and the image display apparatus 30 are provided separately, the image processing apparatus 10 and the image display apparatus 30 may be configured integrally.
Further, the compressed image data output from the image data output unit 13 of the image processing apparatus 10 is not limited to the form transmitted to the image display apparatus 30, but for example, the form recorded on a recording medium. It may be.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in each of the above-described embodiments, the input image is not limited to image data in the RGB color space, and may be image data in the YCbCr color space, for example. In this case, the YCbCr color space conversion process (YCbCr color space conversion step S1) by the YCbCr color space conversion unit 11 in FIGS. 1 and 2 can be omitted.

In addition, in the conversion formula in FIG. 2C, when conversion to the RGB color space is performed as a motion priority, in the above example, as shown in the above formulas (7) to (12), Yx is used and x = 1 and Y1 is set. However, G may be Yi instead of Yx, and the frame number at that time may be used. For example, if the frame number is i = 1, 2, 3, 4, G1 to G4 are
G1 = Y1-Cb1 * 0.3441-Cr1 * 0.7139
G2 = Y2-Cb2 * 0.3441-Cr2 * 0.7139
G3 = Y3-Cb3 * 0.3441-Cr3 * 0.7139
G4 = Y4-Cb4 * 0.3441-Cr4 * 0.7139
It is also possible to ask for.

  Further, as application examples to which the present invention can be applied, a projector (first application example) having two image forming means (referred to as an image forming unit) and one projection optical system, and a direct-view type image display device (first display). Although two application examples) are illustrated, the present invention is not limited to the first application example and the second application example described above.

  In the above-described embodiment, after the RGB color space image data is converted into the YCbCr color space image data by the first color space conversion step, the converted YCbCr color space image data is converted by the second color space conversion step. Although an example (see FIG. 2) of converting to image data in the RGB color space has been described, the image processing method of the present invention is not limited to such color space conversion processing.

  For example, after converting the image data in the RGB color space into image data in the XYZ color space (first color interval conversion step) to convert the image data into the XYZ color space, the converted image data in the XYZ color space Can be performed (second color interval conversion step) for converting the image data into image data in the RGB color space. Further, after converting the image data of the sRGB color space into the image data of the YUV color space (first color interval conversion step) to convert the image data into the image data of the YUV color space, the converted image data of the YUV color space Can also be performed (second color interval conversion step) for converting the image data into image data in the sRGB color space.

1 is a diagram schematically showing a configuration of an image processing apparatus according to an embodiment of the present invention. FIG. 3 is a diagram for explaining basic image processing performed by the image processing apparatus 10 shown in FIG. 1. FIG. 2 is a diagram schematically illustrating an optical configuration of a projector having two image forming units and one projection optical system. The figure explaining the relationship between the operation | movement of the light source in the projector shown in FIG. 3, and the drive of a light modulation element. The figure explaining the image processing of the projector used for a 1st application example. The figure which shows the structure of the image display system as a 2nd application example.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 11 ... YCbCr color space conversion part, 12 ... RGB color space conversion part, 13 ... Image data output part, 30 ... Image display apparatus

Claims (10)

First, a plurality of frames of image data corresponding to an image to be displayed is converted into image data of a first color space having a predetermined tristimulus value and image data of a second color space having a predetermined tristimulus value. A color space conversion step to a two-color space,
In the color space conversion step to the second color space, among the image data of the tristimulus values in the second color space, the image data of the stimulus values related to the element that most affects the human eye is the processing unit. An image processing method characterized in that it is obtained for each of a plurality of frames, and the image data of other stimulus values is obtained in one certain frame of the plurality of frames as the processing unit. The image processing method according to claim 1,
A color space conversion step to a first color space for converting image data of a color space other than the first color space into image data of the first color space for each of a plurality of frames as the processing unit;
An image processing method, wherein the converted image data of the first color space is input to a color space conversion step to the second color space. The image processing method according to claim 1 or 2,
An image processing method comprising: an image data output step of outputting image data of the second color space converted by the color space conversion step to the second color space. In the image processing method in any one of Claims 1-3,
The first color space is a YCbCr color space represented by luminance (Y) and color difference (Cb, Cr), and the second color space is red (R), green (G), and blue (B). RGB color space represented,
In the color space conversion step to the second color space, the G image data is obtained for each of a plurality of frames as the processing unit, and the R and B image data is processed in the processing unit. An image processing method characterized in that it is obtained in one frame of a plurality of frames. The image processing method according to claim 4,
In the color space conversion step to the second color space,
The R image data includes a value of Y in a specific frame among the plurality of frames serving as the processing unit and an average value of Cr obtained by averaging the Cr values of the frames of the plurality of frames serving as the processing unit. Based on,
The G image data includes a Y value in a specific frame among the plurality of frames serving as the processing unit, a Cb value in each frame of the plurality of frames serving as the processing unit, and a Cr value in each frame. Based on,
The B image data includes a Y value in a specific frame of the plurality of frames serving as the processing unit and an average value of Cb obtained by averaging the Cb values of the frames of the plurality of frames serving as the processing unit. An image processing method characterized in that it is obtained on the basis of the above. The image processing method according to claim 4,
In the color space conversion step to the second color space,
The R image data includes an average value of Y obtained by averaging the Y values of each frame of the plurality of frames serving as the processing unit, and an average of Cr obtained by averaging the Cr values of the respective frames of the plurality of frames serving as the processing unit. Based on the value and
The G image data includes an average value of Y obtained by averaging the Y values of each frame of the plurality of frames serving as the processing unit, a value of Cb in each frame of the plurality of frames serving as the processing unit, and a Cr value of each frame. Based on the value and
The B image data includes an average value of Y obtained by averaging the Y values of each frame of the plurality of frames serving as the processing unit, and an average of Cb obtained by averaging the Cb values of the respective frames of the plurality of frames serving as the processing unit. An image processing method characterized in that it is obtained based on a value. In the image processing method according to any one of claims 1 to 6,
Frame interpolation such that each frame of image data corresponding to the image to be displayed has a frame rate that is m (m is an integer of 2 or more) times the frame rate of the base image data supplied from the image source. Is performed, the number of frames as the processing unit is set to m. The image processing method according to claim 7.
The image data corresponding to the image to be displayed is image data given to a projector, and the projector
A first image forming unit that includes three light modulation elements that modulate color light from a light source based on image data, and synthesizes each color light modulated by the three light modulation elements to emit as first image light; A second image forming unit that has three light modulation elements that modulate the color light from the light source based on the image data, and synthesizes the respective color lights modulated by the three light modulation elements and emits them as the second image light. And a polarization combining optical system that combines the first image light and the second image light emitted from the first image forming unit and the second image forming unit, and image light combined by the polarization combining optical system. A total of six light modulation elements provided in the first image forming unit and the second image forming unit, four light modulation elements corresponding to green light, The remaining two light modulation elements are red and blue The light source is used as a light modulation element corresponding to light, and the respective color lights modulated by the four light modulation elements corresponding to the green light are sequentially emitted while being shifted by a predetermined time in each frame of the image data. And an image processing method, wherein the four light modulation elements corresponding to the green light are controlled. First, a plurality of frames of image data corresponding to an image to be displayed is converted into image data of a first color space having a predetermined tristimulus value and image data of a second color space having a predetermined tristimulus value. A color space conversion unit to a two-color space;
The color space conversion unit to the second color space includes a plurality of image data of stimulus values relating to elements that most affect human eyes among the image data of tristimulus values of the second color space. An image processing apparatus characterized in that it is obtained for each frame, and the image data of other stimulus values is obtained in one frame having a plurality of frames as the processing unit. An image processing program for causing a computer to execute image processing,
First, a plurality of frames of image data corresponding to an image to be displayed is converted into image data of a first color space having a predetermined tristimulus value and image data of a second color space having a predetermined tristimulus value. A color space conversion step to a two-color space,
In the color space conversion step to the second color space, among the image data of the tristimulus values in the second color space, the image data of the stimulus values related to the element that most affects the human eye is the processing unit. An image processing program characterized in that it is obtained for each of a plurality of frames, and image data of other stimulus values is obtained in one frame having a plurality of frames as the processing unit.