JP2018072418A - Image processing device and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology capable of improving the image quality of an image displayed by stack projection.SOLUTION: An image processing device of this invention is an image processing device for generating image data for a projection system having a plurality of projection devices including a first projection device for displaying a first image based on first image data in a projection area, and a second projection device for displaying a second image based on second image data in the projection area. The first image data is image data obtained by performing prescribed image processing of original image data, the prescribed image processing includes restriction processing for restricting an outer gradation value of the prescribed range of a gradation value to an inner gradation value of the prescribed range, and the image processing device includes: acquisition means for acquiring restriction information about the performed restriction processing; and generation means for generating image data corresponding to a change in the gradation value by the restriction processing as second image data on the basis of the restriction information acquired by the acquisition means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing method.

  A projection method in which a plurality of projectors project to one projection area of a projection surface is known. Such a projection method is called “stack projection” or the like. In stack projection, a plurality of images (projected images; display images) displayed by a plurality of projectors overlap. As a result, it is possible to increase the brightness of the projection image (a composite image obtained by overlapping a plurality of projection images), increase the gradation of the projection image, and the like. A technique related to stack projection is described in Patent Document 1, for example.

  In a hold drive type display device that continues to display the same frame image over one frame period, it is known that a moving object (moving object) existing in a moving image is blurred and moving image visibility is lowered. Such a phenomenon is called “hold blur”. When a human observes a moving object, the human moves the line of sight smoothly so that the line of sight follows the moving object. However, in the hold drive type display device, the movement of the displayed moving object is not smooth, and the moving object moves while repeating the stillness of one frame period. Then, hold blur occurs due to the difference between the movement of the line of sight and the movement of the displayed moving object. Specifically, hold blur occurs when the line of sight moves (shakes) with respect to a stationary moving object in one frame period. Due to the generation principle of hold blur, the magnitude of hold blur is proportional to the length of the period during which the same frame image is continuously displayed (hold period).

  In an impulse drive type display device that displays a frame image for a moment, it is known that flickering is visually recognized because image display and image non-display are repeated. Flicker places a visual burden on the user. Therefore, flicker causes eye fatigue, discomfort, and the like.

  Therefore, a high frequency emphasis frame and a high frequency suppression frame are generated from each frame of the input image data, and the high frequency emphasis frame and the high frequency suppression frame are alternately output at a frame rate twice the frame rate of the input image data. Techniques to do this have been proposed. The high frequency enhancement frame is a frame in which a spatial high frequency component is emphasized, and the high frequency suppression frame is a frame in which a spatial high frequency component is suppressed. According to this technique, it is possible to improve moving image visibility while suppressing flicker.

  The hold blur is likely to be visually recognized in an image including a high-frequency component (such as an edge of a moving object). According to the above technique, a high-frequency emphasis frame and a high-frequency suppression frame are generated from one frame of input image data, and the high-frequency emphasis frame and the high-frequency suppression frame have a frame rate that is twice the frame rate of the input image data. Are displayed in order. Thereby, the high frequency component of the input image data is intensively displayed in the high frequency emphasis frame. That is, the high-frequency component hold period is reduced to half. As a result, hold blur can be suppressed and moving image visibility can be improved. The low frequency component of the input image data is displayed in one frame period of the input image data (the period of both the high frequency emphasis frame and the high frequency suppression frame). Thereby, flicker can be suppressed.

However, in the above technique, when the gradation value after enhancement of the high frequency component becomes a value outside the specified range, the gradation value is limited to a value inside the specified range (saturation of the gradation value). ). Then, saturation of the gradation value reduces the reproducibility of the image and reduces the effect of improving the moving image visibility. Specifically, due to the saturation of the gradation value, the brightness of the input image data cannot be restored, and the display is performed at a brightness higher or lower than the brightness of the input image data, or more than the gradation of the input image data. Display with low gradation is performed. A method is conceivable in which the amount of change in the gradation value due to the restriction processing that restricts the gradation value to a value inside the specified range is supplemented with a high frequency suppression frame. However, with this method, a state in which each frame of the input image data is displayed twice at a frame rate twice is realized, and the effect of reducing hold blur is lost or a double image is visually recognized.

  Patent Document 2 describes a technique for calculating an average of previous and subsequent frames and generating a high-frequency suppression frame smoothed in the time direction. According to the technique described in Patent Document 2, the change in the gradation value due to the restriction process is smoothed, and the effect of improving the moving image visibility is easily obtained. However, with the technique described in Patent Document 2, hold blur and overlapping images that cause a reduction in moving image visibility are less noticeable, but the size of an image area in which hold blur and overlapping images are generated increases. As a result, depending on the magnitude of the motion of the moving object, the shape of the moving object, the luminance of the moving object (the luminance of the input image data), etc., there is a possibility that the effect of improving the moving image visibility cannot be sufficiently obtained.

JP 2011-002666 A JP 2013-141198 A

  An object of the present invention is to provide a technique capable of improving the image quality of an image displayed by stack projection.

The first aspect of the present invention is:
A first projection device that displays a first image based on the first image data in the projection area by performing projection on the projection area of the projection surface;
A second projection device that displays a second image based on second image data in the projection area by performing projection on the projection area;
An image processing device for generating image data for a projection system having a plurality of projection devices including:
The first image data is image data obtained by performing predetermined image processing on the original image data,
The predetermined image processing includes a limiting process of limiting a gradation value outside a predetermined range of gradation values to a gradation value inside the predetermined range;
The image processing apparatus includes:
Obtaining means for obtaining restriction information on the restriction processing performed;
Generation means for generating, as the second image data, image data corresponding to a change in gradation value by the restriction process based on the restriction information obtained by the obtaining means;
An image processing apparatus comprising:

The second aspect of the present invention is:
A first projection device that displays a first image based on the first image data in the projection area by performing projection on the projection area of the projection surface;
A second projection device that displays a second image based on second image data in the projection area by performing projection on the projection area;
An image processing method for generating image data for a projection system having a plurality of projection devices including:
The first image data is image data obtained by performing predetermined image processing on the original image data,
The predetermined image processing includes a limiting process of limiting a gradation value outside a predetermined range of gradation values to a gradation value inside the predetermined range;
The image processing method includes:
Obtaining restriction information regarding the restriction processing that has been performed;
Generating, as the second image data, image data corresponding to a change in gradation value by the restriction process based on the obtained restriction information;
An image processing method characterized by comprising:

  A third aspect of the present invention is a program for causing a computer to execute each step of the image processing method according to the second aspect of the present invention.

  According to the present invention, the image quality of an image displayed by stack projection can be improved.

FIG. 1 is a block diagram illustrating a configuration example of a projection system according to a first embodiment. FIG. 3 is a block diagram illustrating a configuration example of a high frequency emphasis / high frequency suppression unit according to the first embodiment. FIG. 10 is a diagram illustrating a processing example of the projection system according to the first embodiment. FIG. 6 is a block diagram illustrating a configuration example of a projection system according to the second embodiment. FIG. 6 is a block diagram illustrating a configuration example of a projection system according to the second embodiment. FIG. 6 is a block diagram illustrating a configuration example of a projection system according to a third embodiment. FIG. 10 is a diagram illustrating a processing example of the projection system according to the third embodiment. FIG. 9 is a block diagram showing an example of the configuration of a projection system according to Embodiment 4. FIG. 10 is a diagram illustrating a processing example of the projection system according to the fourth embodiment. FIG. 9 is a block diagram showing an example of the configuration of a projection system according to a fifth embodiment. FIG. 10 is a diagram illustrating a processing example of the projection system according to the fifth embodiment. Figure showing a conventional example

<Example 1>
Embodiment 1 of the present invention will be described below. The projection system according to the present embodiment includes a plurality of projection apparatuses (projectors). Each projection device displays an image in the projection area by performing projection on the projection area of the projection plane. The projection surface is the surface of the screen board, the surface of the screen sheet, the wall surface, or the like. The projection area is the same among the plurality of projection apparatuses. Further, the projection (display) of each projection apparatus is synchronized among the plurality of projection apparatuses. That is, stack projection is performed in the projection system according to the present embodiment.

Although the number of projection apparatuses is not particularly limited, in the present embodiment, two projection apparatuses (a first projection apparatus and a second projection apparatus) are used in order to simplify the description. The first projection device displays a first image based on the first image data in the projection area by performing projection on the projection area. The second projection device displays a second image based on the second image data in the projection area by performing projection on the projection area. In this embodiment, an example in which a liquid crystal projector having a light emitting unit (light source) and a liquid crystal panel is used as a projection device will be described. The projection device is not limited to a liquid crystal projector. For example, instead of the liquid crystal element of the liquid crystal panel, MEMS (Micro
A MEMS shutter projector using an Electro Mechanical System (shutter) can also be used as a projection device.

  In addition, the luminance characteristics of the projection image (the image displayed by the projection apparatus; the display image) are not particularly limited. However, in order to simplify the description, in this embodiment, the luminance characteristics of the first image are the same as those of the second image. It shall be equal to the luminance characteristic. The luminance characteristic of the projected image is a characteristic of the correspondence relationship between the gradation value of the image data and the luminance of the projected image. Therefore, it can be said that “the luminance characteristics of the projected image are equal between the projection devices” is “the correspondence between the gradation value of the image data and the luminance of the projection image is equal between the projection devices”. Specifically, the peak luminance of the first image (the upper limit of the luminance of the first image) is equal to the peak luminance of the second image, and the gradation characteristic (gamma) of the first image is the gradation characteristic of the second image. Is equal to Further, it is assumed that the frame rate of the first image is equal to the frame rate of the second image.

  In the present embodiment, an example in which the first projection device generates first image data and second image data based on input image data and outputs the second image data to the second projection device will be described. Note that generation of image data for the projection system may be performed by an apparatus different from the first projection apparatus. For example, the first projection device generates first image data based on the image data input to the first projection device, and the second projection device performs the second based on the image data input to the second projection device. Image data may be generated. The second projection device may generate the first image data and the second image data based on the input image data. The image processing device that generates at least one of the first image data and the second image data may be a separate device from the projection device.

  FIG. 1 is a block diagram illustrating a configuration example of a projection system according to the present embodiment. As shown in FIG. 1, the projection system according to the present embodiment includes a first projection device and a second projection device. The first image data described above is moving image data obtained by performing predetermined image processing on the first original image data that is moving image data. Although the predetermined image processing is not particularly limited, in the present embodiment, predetermined image processing is performed by the frame rate conversion unit 101, the high frequency enhancement / high frequency suppression unit 102, the saturation determination unit 103, and the inter-device distribution unit 105. Done.

  The frame rate conversion unit 101 performs frame rate conversion processing on input image data (first original image data). By frame rate conversion processing, N frames (N is an integer of 2 or more) are generated from each frame of the input image data. Then, the converted moving image data, which is moving image data including a plurality of generated frames and having a frame rate N times the frame rate of the input image data, is generated by the frame rate conversion process.

  The method of frame rate conversion processing is not particularly limited, but in this embodiment, moving image data in which each frame of the input image data is repeated twice at a frame rate twice the frame rate of the input image data is generated as converted moving image data. Is done. Specifically, the frame rate conversion unit 101 uses the frame data of the input image data (one frame of image data) as the frame data of the converted moving image data during one frame period of the input image data. Output twice to the suppression unit 102. The frame rate conversion unit 101 performs this process for each frame of the input image data.

  The high-frequency emphasis / high-frequency suppression unit 102 performs an emphasis process for emphasizing a spatial high-frequency component on the input frame data (image data corresponding to one frame of the converted moving image data). Thereby, high frequency emphasis frame data is generated. Further, the high frequency emphasis / high frequency suppression unit 102 performs a suppression process for suppressing the spatial high frequency component on the input frame data. Thereby, high frequency suppression frame data is generated. Then, the high frequency emphasis / high frequency suppression unit 102 outputs the high frequency emphasis frame data to the saturation determination unit 103 and the asymmetry processing unit 104, and outputs the high frequency suppression frame data to the changeover switch 108.

  FIG. 2 is a block diagram showing a specific example of the configuration of the high frequency emphasis / high frequency suppression unit 102. The LPF unit 113 is a two-dimensional spatial low-pass filter. The LPF unit 113 extracts spatial low-frequency components from the frame data input to the high-frequency emphasis / high-frequency suppression unit 102, and outputs the extracted low-frequency component data as high-frequency suppression frame data. The method of the low pass filter is not particularly limited. For example, a general two-dimensional FIR filter or the like can be used as the LPF unit 113. The subtractor 114 subtracts the high frequency suppression frame data from the frame data input to the high frequency emphasis / high frequency suppression unit 102. Thereby, data (high frequency component data) representing the spatial high frequency component of the frame data input to the high frequency emphasis / high frequency suppression unit 102 is obtained. The adder 115 adds the high frequency component data to the frame data input to the high frequency emphasis / high frequency suppression unit 102. Then, the adder 115 outputs the addition result as high frequency emphasized frame data.

  In FIG. 2, “H” is high frequency component data, and “L” is low frequency component data. As shown in FIG. 2, when frame data “L + H” including a low frequency component and a high frequency component is input to the high frequency emphasis / high frequency suppression unit 102, “L + 2H” is output as high frequency emphasis frame data. Then, “L” is output as the high frequency suppression frame data. As described above, in this embodiment, the frame data of the input image data is output twice to the high frequency emphasis / high frequency suppression unit 102 as the frame data of the converted moving image data in one frame period of the input image data. In the first projection device, an image based on the high frequency emphasis frame data is displayed as the first frame display, and an image based on the high frequency suppression frame data is displayed as the second frame display. The first frame is one of an odd-numbered frame and an even-numbered frame among the plurality of frames of the converted video data, and the second frame is an odd-numbered frame and an even-numbered frame among the plurality of frames of the converted video data. The other of the second frame. That is, in the first projection device, an image based on the high frequency emphasis frame data and an image based on the high frequency suppression frame data are alternately displayed. Therefore, for one frame of the input image data, the perceived brightness of the first image (the brightness perceived by the user) is an average “L + H” of the brightness of the high-frequency emphasized frame data and the brightness of the high-frequency suppressed frame data. That is, the luminance of the input image data can be reproduced as the perceived luminance of the first image.

  In the example of FIG. 2, the high frequency emphasis frame data and the high frequency suppression frame data are generated from the frame data of the converted video data for each frame of the converted video data. However, the present invention is not limited to this. As described above, in the first projection device, an image based on the high-frequency emphasized frame data is displayed as the first frame, and an image based on the high-frequency suppression frame data is displayed as the second frame. Done. Therefore, the enhancement process for the first frame and the suppression process for the second frame may be performed, and the suppression process for the first frame and the enhancement process for the second frame may be omitted. That is, the generation / output of high frequency suppression frame data may be omitted during the first frame period, and the generation / output of high frequency emphasized frame data may be omitted during the second frame period.

  Based on the high frequency emphasized frame data input to the saturation determining unit 103, the saturation determining unit 103 determines whether there is a saturated pixel in the high frequency emphasized frame data. A saturated pixel is a pixel having a gradation value outside a predetermined range of gradation values. The predetermined range is, for example, a range of gradation values that can be displayed by the first projection device. That is, the predetermined range is a range from a gradation value corresponding to the lower limit of the luminance of the first image to a gradation value corresponding to the upper limit of the luminance of the first image. The predetermined range is not particularly limited. The predetermined range may be a fixed range determined in advance by the manufacturer, or may be a range that is appropriately changed according to an instruction from the user, a use environment of the projection system, and the like.

Then, saturation determination section 103 outputs the result of the determination to asymmetry processing section 104.
In the present embodiment, the saturation determination unit 103 outputs the determination result “saturated pixel exists” when the saturated pixel exists in the high frequency emphasized frame data, and the determination result when the saturated pixel does not exist in the high frequency emphasized frame data. “No saturated pixel” is output.

  The asymmetry processing unit 104 generates second original image data that is the source of the second image data according to the determination result from the saturation determination unit 103, and outputs the second original image data to the inter-apparatus distribution unit 105. . Further, the asymmetry processing unit 104 distributes the high frequency emphasis frame data output from the high frequency emphasis / high frequency suppression unit 102 as the third original image data that is the source of the frame data of the first image data. To the unit 105. In this embodiment, the inter-apparatus distribution unit 105 performs a restriction process that restricts (clips) the gradation value outside the predetermined range to the gradation value inside the predetermined range. Specifically, the inter-device distribution unit 105 performs a restriction process on the high frequency emphasized frame data. “High-frequency emphasis frame data output from the high-frequency emphasis / high-frequency suppression unit 102” can also be said to be “frame data of pre-restriction image data (first image data before being subjected to restriction processing)”.

  In this embodiment, when the determination result “with saturated pixel” is output from the saturation determination unit 103, the asymmetry processing unit 104 uses image data different from the third original image data as the second original image data. Output. “Image data different from the third original image data” can also be said to be “image data asymmetric with the third original image data”. On the other hand, when the determination result “no saturated pixel” is output from the saturation determination unit 103, the asymmetry processing unit 104 outputs the same image data as the third original image data as the second original image data. “The same image data as the third original image data” can be said to be “image data symmetrical to the third original image data”.

  Specifically, when the determination result “with saturated pixels” is output from the saturation determination unit 103, the asymmetry processing unit 104 is an image in which the gradation value of each pixel is a predetermined value (predetermined offset value). The data is output as second original image data. On the other hand, when the determination result “no saturated pixel” is output from the saturation determination unit 103, the asymmetry processing unit 104 outputs the high frequency emphasis frame data output from the high frequency emphasis / high frequency suppression unit 102 to Also output as binary image data. The offset value may be a fixed value determined in advance by the manufacturer, or may be a value that is appropriately changed according to an instruction from the user, a usage environment of the projection system, and the like.

  As described above, in the present embodiment, image data different from pre-restriction image data (frame data of pre-restriction image data) is used as the second original image data in the case where saturated pixels exist. Then, image data equal to the pre-restriction image data (frame data of the pre-restriction image data) is used as the second original image data when there is no saturated pixel. “Second original image data when saturated pixels are present” includes “second original image data when restriction processing is performed” and “second original image data when gradation value changes due to restriction processing”. , Etc. The “second original image data when there is no saturated pixel” includes the “second original image data when the restriction process is not performed” and the “second original image data when the gradation value is not changed by the restriction process”. It can also be said to be “original image data”. Note that image data different from the pre-restriction image data may be used as the second original image data regardless of whether there is a saturated pixel.

The inter-apparatus distribution unit 105 generates restriction frame data by performing restriction processing on the third original image data (high frequency emphasis frame data) output from the saturation determination unit 103. Then, the inter-apparatus distribution unit 105 outputs the restricted frame data. If there is no saturated pixel in the third original image data, the inter-apparatus distribution unit 105 outputs the same image data as the third original image data as restricted frame data. When saturated pixels exist in the third original image data, the inter-apparatus distribution unit 105 restricts the gradation value of the saturated pixels in the third original image data to a value inside a predetermined range, thereby limiting frame data. Is generated. Then, the inter-apparatus distribution unit 105 outputs the restricted frame data. Further, the inter-apparatus distribution unit 105 acquires (generates) restriction information regarding the performed restriction process by determining the amount of change in the gradation value due to the performed restriction process.

In addition, the inter-apparatus distribution unit 105 generates image data corresponding to a change in the gradation value by the restriction process as compensation frame data based on the restriction information. Specifically, the inter-apparatus distribution unit 105 generates, as compensation frame data, image data that compensates for the change in gradation value due to the restriction process based on the restriction information. Then, the inter-apparatus distribution unit 105 outputs compensation frame data. In the present embodiment, the inter-apparatus distribution unit 105 generates compensation frame data using the second original image data output from the saturation determination unit 103 and the amount of change in the gradation value due to the restriction process. Specifically, the inter-apparatus distribution unit 105 generates the compensation frame data by adding the change amount of the gradation value by the restriction process to the gradation value of the second original image data. More specifically, image data satisfying the following conditions 1 and 2 is generated as compensation frame data. The restricted pixel is a pixel whose gradation value is restricted to a value inside a predetermined range by the restriction process, and is a saturated pixel. When there is no saturated pixel in the third original image data, the inter-apparatus distribution unit 105 outputs the same image data as the second original image data and the third original image data as compensation frame data. When saturated pixels exist in the third original image data, the inter-apparatus distribution unit 105 outputs image data different from the second original image data and the third original image data as compensation frame data.

Condition 1: The gradation value of the pixel corresponding to the restricted pixel is a gradation value obtained by adding the change amount of the gradation value by the restriction process to the gradation value of the second original image data.
Condition 2: The gradation value of a pixel corresponding to a pixel other than the restricted pixel is the gradation value of the second original image data.

  Note that the compensation frame data generation method is not particularly limited as long as the change in the gradation value due to the restriction process is compensated. For example, the change amount of the gradation value of the restricted pixel may be supplemented by the pixel corresponding to the restricted pixel and the surrounding pixels.

  In accordance with the processing result of the asymmetry processing unit 104, the luminance restoration unit 106 reduces the emission luminance so that the luminance variation of the projection image (the synthesized image obtained by overlapping the first image and the second image) is suppressed. decide. Then, the luminance restoration unit 106 outputs luminance control data for controlling the light emission luminance of the light emitting unit to the determined light emission luminance. The luminance control data output from the luminance restoring unit 106 is used for controlling the emission luminance of the light emitting units 111A and 111B. Specifically, the luminance control data output from the luminance restoration unit 106 is used in the period of the first frame.

  In this embodiment, limited frame data is obtained using the third original image data, and compensation frame data is obtained using the second original image data. Then, during the period in which the image based on the limited frame data is displayed by the first projection device, the image based on the compensation frame data is displayed by the second projection device.

  In the present embodiment, when the third original image data does not include saturated pixels, the same image data as the third original image data is adopted as the second original image data. It can be said that “when the third original image data does not include saturated pixels” “when the limited frame data does not include limited pixels”. Then, the luminance restoration unit 106 outputs luminance control data corresponding to a predetermined reference luminance when the same image data as the third original image data is adopted as the second original image data. The third original image data is high frequency emphasized frame data obtained from the input image data. For this reason, when the same image data as the third original image data is adopted as the second original image data, display with luminance based on the input image data is performed in each of the first projection device and the second projection device.

On the other hand, when the third original image data includes saturated pixels, image data in which each gradation value is an offset value is adopted as the second original image data. It can be said that “when the third original image data includes saturated pixels” “when the limited frame data includes limited pixels”. In such a case, the luminance based on the input image data is not realized by the second projection device. Therefore, if the light emission brightness of the light emitting units 111A and 111B is always controlled to the reference brightness, the brightness of the composite image in the display period of the limited frame data including the limited pixels and the composite image in the display period of the limited frame data not including the limited pixels are displayed. A difference from the brightness occurs. For example, in the display period of restricted frame data that does not include restricted pixels, the composite image is displayed with a lower brightness than the brightness of the composite image in the display period of restricted frame data that does not include restricted pixels. In the present embodiment, in the display period of restricted frame data that does not include restricted pixels, the composite image is displayed with a brightness that is half the brightness of the composite image in the display period of restricted frame data that does not include restricted pixels. Therefore, in the present embodiment, the luminance restoration unit 106 uses the luminance control data corresponding to the light emission luminance that is twice the reference luminance when image data whose gradation values are offset values is adopted as the second original image data. Is output. Thereby, the brightness | luminance fluctuation | variation of a synthesized image can be suppressed.

  The reference brightness determination unit 107 outputs brightness control data corresponding to the reference brightness. The luminance control data output from the reference luminance determining unit 107 is also used for controlling the emission luminance of the light emitting units 111A and 111B. Specifically, the brightness control data output from the reference brightness determining unit 107 is used in the period of the second frame.

  As described above, in this embodiment, the luminance control data corresponding to the processing result of the asymmetry processing unit 104 is used in the first frame period, and the luminance control data corresponding to the reference luminance is used in the second frame period. Is used. By changing the luminance control data in the first frame period according to the processing result of the asymmetry processing unit 104, the luminance balance (the luminance of the synthesized image in the first frame period and the luminance of the synthesized image in the second frame period) Fluctuations in the balance) can also be suppressed. Furthermore, fluctuations in frequency balance (balance between the spatial frequency of the composite image and the spatial frequency of the input image data) can be suppressed.

  The reference luminance may be a fixed value determined in advance by the manufacturer, or may be a value that is appropriately changed according to an instruction from the user, a usage environment of the projection system, and the like. In this embodiment, since the light emitting unit emits light with the light emission luminance corresponding to the output luminance control data, “the process of outputting the luminance control data” can be said to be “the process of controlling the light emission luminance of the light emitting unit”. .

  Note that a method for controlling the light emission luminance of the light emitting unit, luminance control data, and the like are not particularly limited. For example, when the PWM control for controlling the light emission luminance of the light emitting unit by controlling the pulse width of the pulse signal supplied to the light emitting unit, data indicating the pulse width can be used as the luminance control data. When the PAM control for controlling the light emission luminance of the light emitting unit by controlling the pulse amplitude of the pulse signal, data indicating the pulse amplitude can be used as the luminance control data. When PHM control for controlling the light emission luminance of the light emitting unit by controlling both the pulse width and the pulse amplitude, data indicating both the pulse width and the pulse amplitude can be used as the luminance control data.

In the present embodiment, the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B are controlled using common luminance control data, but the present invention is not limited to this. For example, the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B may be individually controlled. Only one of the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B may be changed without changing one of the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B. Specifically, at least one of the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B may be controlled so that the following condition 3 is satisfied in the display period of the frame including the limited pixels. Then, at least one of the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B may be controlled so that the following condition 4 is satisfied in a frame display period that does not include the limited pixels. In the following conditions 3 and 4, the predetermined luminance is, for example, twice the reference luminance.

Condition 3: The total of the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B is twice the predetermined luminance.
Condition 4: The sum of the light emission luminance of the light emitting unit 111A and the light emission luminance of the light emitting unit 111B is a predetermined luminance.

The changeover switch 108 is a switch for switching an image data transmission path, a luminance control data transmission path, and the like. In the present embodiment, the changeover switch 108 switches the transmission path so that the following processing is realized.

A process of leading the limited frame data output from the inter-apparatus distribution unit 105 to the synchronization forming unit 109A as the first image data (the frame data of the first image data) in the first frame period. In the processing, the compensation frame data output from the inter-apparatus distribution unit 105 is guided to the synchronization forming unit 109B as the second image data. The second frame is output from the high frequency emphasis / high frequency suppression unit 102 in the second frame period. Processing for guiding the high-frequency suppression frame data to the synchronization forming unit 109A as the first image data (frame data of the first image data). Output from the high-frequency emphasis / high-frequency suppression unit 102 during the second frame period. Processing for guiding the high-frequency suppression frame data as the second image data to the synchronization forming unit 109B. In the period of the first frame, luminance recovery is performed. Processing for deriving the luminance control data output from the base unit 106 to the synchronization forming units 109A and 109B. In the period of the second frame, the luminance control data output from the reference luminance determining unit 107 to the synchronization forming units 109A and 109B. Processing

  The synchronization forming unit 109A outputs the image data input to the synchronization forming unit 109A to the liquid crystal panel 110A, and outputs the brightness control data input to the synchronization forming unit 109A to the light emitting unit 111A. The synchronization forming unit 109B outputs the image data input to the synchronization forming unit 109B to the liquid crystal panel 110B, and outputs the luminance control data input to the synchronization forming unit 109B to the light emitting unit 111B.

  The synchronization forming units 109A and 109B output the image data and the brightness control data so that the display of the first image and the display of the second image are performed at the same timing for each frame of the converted moving image data (synchronization processing). ). In the present embodiment, the synchronization forming units 109A and 109B perform image control so that the control of the modulation rate (transmittance, reflectance, etc.) of the liquid crystal panel 110A and the control of the modulation rate of the liquid crystal panel 110B are performed at the same timing. Output data. The synchronization forming units 109A and 109B output luminance control data so that the modulation rate control of the liquid crystal panels 110A and 110B and the light emission of the light emitting units 111A and 111B are performed at the same timing. Specifically, the synchronization forming units 109A and 109B have a storage unit that stores data (image data, luminance control data, etc.). Then, the synchronization forming units 109A and 109B read the data from the storage unit and output the read data at a timing that considers the time required to transmit data from the first projection device to the second projection device.

The liquid crystal panel 110A modulates (transmits or reflects) light from the light emitting unit 111A based on image data input to the liquid crystal panel 110A. Specifically, the liquid crystal panel 110A includes a plurality of liquid crystal elements, and controls the modulation rate of each liquid crystal element based on image data input to the liquid crystal panel 110A. The liquid crystal panel 110A modulates the light from the light emitting unit 111A with a modulation rate based on the image data input to the liquid crystal panel 110A. The light after being modulated by the liquid crystal panel 110A is applied to the projection lens 112A. Similarly, the liquid crystal panel 110B modulates the light from the light emitting unit 111B based on the image data input to the liquid crystal panel 110B. The light after being modulated by the liquid crystal panel 110B is applied to the projection lens 112B.

  The light emitting unit 111A irradiates the liquid crystal panel 110A with light having a luminance corresponding to the luminance control data input to the light emitting unit 111A. Similarly, the light emitting unit 111B irradiates the liquid crystal panel 110B with light having a luminance corresponding to the luminance control data input to the light emitting unit 111B. When the light emitting unit irradiates light on the back surface of the modulation panel (liquid crystal panel) and the modulation panel transmits light from the light emitting unit, the light emitting unit can be said to be a “backlight unit”.

  The projection lens 112A optically changes the light from the liquid crystal panel 110A, and irradiates the projection area with the light after the optical change is given. Thereby, the first image is displayed (imaged) in the projection area. Similarly, the projection lens 112B optically changes the light from the liquid crystal panel 110B, and irradiates the projection area with the light after the optical change is given. Thereby, the second image is displayed in the projection area.

  Specific examples and effects of the processing of the projection system according to the present embodiment will be described with reference to FIGS. 3 and 12 show examples of luminance distributions of various image data used in the projection system. 3 and 12, the luminance distribution of the input image data of the first projection device, the luminance distribution of the intermediate image data of the first projection device, and the luminance distribution of the output image data of the first projection device are shown. Yes. The input image data is image data input to the projection apparatus. The intermediate image data is image data generated during the process of generating output image data from input image data. The output image data is image data input to the liquid crystal panel. 3 and 12, the luminance distribution of the input image data of the second projection device, the luminance distribution of the intermediate image data of the first projection device, and the luminance distribution of the output image data of the first projection device are shown. Yes. In the lower part of FIGS. 3 and 12, the distribution of the stack luminance and the distribution of the perceived luminance are shown. FIG. 3 also shows the distribution of the projection luminance (luminance of the projection image; luminance in the projection area; luminance on the projection plane) of the first frame.

  The stack luminance is the luminance when stack projection is performed, and is the sum of the luminance of the first projection device and the luminance of the second projection device. 3 and 12, the sum of the upper-level luminance distribution and the middle-level luminance distribution is shown as the stack luminance distribution. 3 and 12, the perceptual brightness when an image based on the output image data is displayed is shown as the perceptual brightness. When the first frame and the second frame are sequentially displayed at a sufficiently high frame rate, the average of the luminance of the first frame and the luminance of the second frame is perceived by the user. Therefore, in FIGS. 3 and 12, the average of the stack luminance distribution corresponding to the output image data of the first frame and the stack luminance distribution corresponding to the output image data of the second frame is shown as the distribution of perceptual luminance. ing.

  FIG. 3 shows a specific example of the process of the projection system according to the present embodiment, and FIG. 12 shows a specific example of the process of the conventional projection system. 3 and 12 show an example in which image processing for generating high-frequency emphasized frame data as image data of the first frame and generating high-frequency suppression frame data as image data of the second frame is performed.

A specific example of the processing of the conventional projection system will be described with reference to FIG. In the example of FIG. 12, the same input image data is input to each of the first projection device and the second projection device. And first
Stack projection is performed using the projection device and the second projection device.

  In the example of FIG. 12, each projection device generates high-frequency emphasized frame data as intermediate image data of the first frame and high-frequency suppression frame data as intermediate image data of the second frame from the input image data. In the example of FIG. 12, the high frequency emphasized frame data includes saturated pixels. In FIG. 12, the brightness | luminance of a broken line part is a brightness | luminance of a saturated pixel. In the example of FIG. 12, high-frequency emphasized frame data is used as output image data. However, when the high-frequency emphasized frame data includes saturated pixels, the gradation value of the saturated pixel is changed to a gradation value inside a predetermined range. Restriction processing to restrict is performed. Therefore, in the example of FIG. 12, the luminance distribution obtained by removing the hatched portion from the luminance distribution of the high-frequency emphasized frame data is obtained as the luminance distribution of the output image data of the first frame. The hatched portion corresponds to the amount of change in the gradation value due to the restriction process. When the high-frequency suppression frame data is used as the output image data of the second frame, although the gradation value of the first frame is changed by the restriction process, the image quality of the composite image is deteriorated (failed). Therefore, in the example of FIG. 12, the output image data of the second frame is generated by adding the change amount of the gradation value by the restriction process to the gradation value of the high-frequency suppression frame data.

  As shown in FIG. 12, the luminance distribution approaches the luminance distribution of the input image data by the restriction process. In the example of FIG. 12, the same luminance distribution as the luminance distribution of the input image data is obtained as the luminance distribution of the output image data of the first frame. Furthermore, as shown in FIG. 12, the luminance distribution approaches the luminance distribution of the input image data also by adding the amount of change in the gradation value due to the limiting process to the gradation value of the high-frequency suppression frame data. In the example of FIG. 12, the same luminance distribution as that of the input image data is obtained as the luminance distribution of the output image data of the second frame. As a result, as the hold period (the period during which the same frame image is continuously displayed), the same (or close) period as the hold period before dividing the frame of the input image data into the first frame and the second frame is realized. The effect of improving visibility cannot be obtained.

  On the other hand, according to the projection system of the present embodiment, various problems including the above problem described with reference to FIG. 12 can be solved, and the image quality of the composite image can be improved. A specific example of the processing of the projection system of the present embodiment will be described with reference to FIG.

  In the present embodiment, as shown in FIG. 3, the first projection device generates high-frequency emphasized frame data as intermediate image data corresponding to the first frame and the first projection device from the input image data. In the example of FIG. 3, the high frequency emphasis frame data includes saturated pixels. Saturated pixels are detected by the saturation determination unit 103. When the saturation pixel is detected, the asymmetry processing unit 104 uses the intermediate image data (high frequency enhancement frame) corresponding to the first frame and the first projection device as the intermediate image data corresponding to the first frame and the second projection device. Image data different from (data) is generated. Specifically, as shown in FIG. 3, image data in which each gradation value is an offset value is generated as intermediate image data corresponding to the first frame and the second projection device.

  In this embodiment, as shown in FIG. 3, the inter-apparatus distribution unit 105 generates output image data corresponding to the first frame and the first projection device by performing a restriction process on the high-frequency emphasized frame data. Then, the inter-apparatus distribution unit 105 adds the change amount of the gradation value due to the limiting process to the intermediate image data (offset value) corresponding to the first frame and the second projection apparatus, and thereby the first frame and the second frame. Output image data corresponding to the projection device is generated.

  Further, as shown in FIG. 3, the first projection device generates high-frequency suppression frame data as output image data corresponding to the second frame, the first projection device, and the second projection device. That is, in the present embodiment, the gradation value of the high frequency suppression frame data is not changed.

As described above, in this embodiment, the change in the gradation value due to the restriction process on the image data corresponding to the first frame and the first projection device is not supplemented by the image data of the second frame, and the first frame and Supplemented with image data corresponding to the second projection device. Thereby, the image quality of the composite image can be improved with high accuracy. For example, it is possible to suppress deterioration in image quality due to a change in gradation value due to the restriction process. Further, since the hold period of the high-frequency component of the input image data can be halved, the magnitude of hold blur can be halved. As a result, the moving image visibility can be improved.

  Furthermore, in this embodiment, when image data different from the high frequency emphasis frame data is adopted as the intermediate image data corresponding to the first frame and the second projection device, the light emission luminance of the light emitting units 111A and 111B is the reference The luminous intensity is increased to twice the luminance. As a result, as shown in FIG. 3, it is possible to realize a luminance that is twice the luminance of the output image data of the first frame as the projection luminance of the first frame. As a result, it is possible to suppress the luminance variation of the composite image. Specifically, as the stack luminance distribution corresponding to the first frame and the projection luminance distribution, a distribution equivalent to the stack luminance distribution corresponding to the first frame and the luminance distribution of the intermediate image data is obtained in FIG. be able to. As a result, it is possible to realize high-luminance display (projection) by stack projection while maintaining image reproducibility.

  As shown in FIG. 3, the stack luminance corresponding to the distribution of the first frame and the projected luminance is set to an offset value compared to the stack luminance corresponding to the luminance distribution of the first frame and the intermediate image data in FIG. Only the corresponding brightness is high. However, in the projection apparatus, the luminance of black is not close to 0, and relatively black is expressed by the contrast of the image. In other words, unlike the direct-view display device, the projection device is not required to display absolute black (black whose luminance is substantially 0). Therefore, if the luminance distribution is the same between the images, there is no sense of hindrance in image quality even if the luminance of black differs between the images.

  In this embodiment, it is assumed that the luminance characteristic of the first image is the same as the luminance characteristic of the second image, but the present invention is not limited to this. Various adjustments may be made in at least one of the first projection device and the second projection device so that the luminance characteristics of the first image and the luminance characteristics of the second image approach each other. For example, the projection device may be provided with a brightness sensor that detects projection brightness, light emission brightness of the light emitting unit, light brightness from the modulation panel, light brightness from the projection lens, and the like. Then, based on the detection value of the brightness sensor, the projection apparatus may automatically adjust its processing characteristics so that its own brightness characteristics approach the brightness characteristics of other projection apparatuses. The processing characteristics include a correspondence characteristic between the gradation value and the modulation rate, a correspondence characteristic between the luminance control data and the light emission luminance of the light emitting unit, and the like.

<Example 2>
Embodiment 2 of the present invention will be described below. In the following, points (configuration, processing, etc.) different from those in the first embodiment will be described in detail, and descriptions of the same points as those in the first embodiment will be omitted. In this embodiment, the same input image data is input to each of the plurality of projection apparatuses. Then, output image data for itself is generated in each of the plurality of projection apparatuses. In this embodiment, image data is not transmitted between the projection apparatuses, and only information regarding the role of the projection apparatus is transmitted between the projection apparatuses.

  FIG. 4 is a block diagram illustrating a configuration example of the projection system according to the present embodiment. 4, the same reference numerals as those in the first embodiment are assigned to the same functional units as those in the first embodiment (FIG. 1). As shown in FIG. 4, the projection system according to the present embodiment includes a distributor 213, a projection apparatus A, and a projection apparatus B. The distributor 213 receives input image data (first original image data). The distributor 213 outputs the input image data to the projection apparatuses A and B.

The frame rate conversion units 201A and 201B have the same functions as the frame rate conversion unit 101 of the first embodiment. The high frequency emphasis / high frequency suppression units 202A and 202B have the same functions as the high frequency emphasis / high frequency suppression unit 102 of the first embodiment. The luminance restoration units 206A and 206B have the same function as the luminance restoration unit 106 of the first embodiment. The reference luminance determining units 207A and 207B have the same function as the reference luminance determining unit 107 of the first embodiment.

  The asymmetry processing unit 204A generates one of the second original image data and the third original image data according to the determination result from the saturation determination unit 103, and outputs the generated image data to the inter-apparatus distribution unit 205A. Further, the asymmetry processing unit 204A instructs the asymmetry processing unit 204B to generate the other of the second original image data and the third original image data according to the determination result from the saturation determination unit 103. The asymmetry processing unit 204B generates image data which is the other of the second original image data and the third original image data in response to an instruction from the asymmetry processing unit 204A, and distributes the generated image data to the inter-device distribution unit Output to 205B. The generation method of the second original image data, the generation method of the third original image data, and the like are the same as those in the first embodiment.

  When the projection apparatus A plays the same role as the first projection apparatus of the first embodiment and the projection apparatus B plays the same role as the second projection apparatus of the first embodiment, the asymmetry processing unit 204A Original image data is generated, and the third original image data is output to the inter-apparatus distribution unit 205A. Then, the asymmetry processing unit 204A instructs the asymmetry processing unit 204B to generate the second original image data. When the determination result from the saturation determination unit 103 is “with saturated pixels”, the asymmetry processing unit 204A issues an instruction to generate second original image data in which the gradation value of each pixel is an offset value. To the asymmetry processing unit 204B. Since this instruction is an instruction related to the restriction process, it can be said to be “restriction information”. Thereafter, the asymmetry processing unit 204B generates second original image data in response to an instruction from the asymmetry processing unit 204A, and outputs the second original image data to the inter-device distribution unit 205B.

  When the projection apparatus A plays the same role as the second projection apparatus and the projection apparatus B plays the same role as the first projection apparatus, the asymmetry processing unit 204A generates the second original image data, and The binary image data is output to the inter-apparatus distribution unit 205A. Then, the asymmetry processing unit 204A instructs the asymmetry processing unit 204B to generate the third original image data. Thereafter, the asymmetry processing unit 204B generates third original image data in response to an instruction from the asymmetry processing unit 204A, and outputs the third original image data to the inter-device distribution unit 205B.

  The inter-apparatus distribution unit 205A generates one of limited frame data and compensation frame data, and outputs the generated image data. The inter-apparatus distribution unit 205B generates the other of the limited frame data and the compensation frame data, and outputs the generated image data. Specifically, when the projection apparatus A plays the same role as the first projection apparatus and the projection apparatus B plays the same role as the second projection apparatus, the inter-apparatus distribution unit 205A generates restricted frame data. The inter-apparatus distribution unit 205B generates compensation frame data. When the projection apparatus A plays the same role as the second projection apparatus and the projection apparatus B plays the same role as the first projection apparatus, the inter-apparatus distribution unit 205A generates compensation frame data, and the inter-apparatus distribution unit 205B generates restricted frame data. The method for generating restricted frame data, the method for generating compensation frame data, and the like are the same as those in the first embodiment. In the present embodiment, high frequency emphasis frame data is generated in each of the projection apparatus A and the projection apparatus B. Therefore, each of the projection apparatus A and the projection apparatus B can generate both the limited frame data and the compensation frame data using the high-frequency emphasized frame data generated by itself.

Note that information indicating the amount of change in the gradation value due to the restriction process may be output from a projection apparatus that serves as the first projection apparatus to a projection apparatus that serves as the second projection apparatus. This information can also be called “restriction information”. And in the projection apparatus which plays the role of a 2nd projection apparatus, compensation frame data may be produced | generated using the information which shows the variation | change_quantity of the gradation value by a restriction | limiting process, without using high frequency emphasis frame data. Thereby, in the projection apparatus that plays the role of the second projection apparatus, it is not necessary to determine the amount of change in the gradation value due to the restriction process from the high-frequency emphasized frame data, so that the processing load can be reduced.

  The changeover switches 208A and 208B switch the image data transmission path, the luminance control data transmission path, and the like so that the same data as in the first embodiment is input to the synchronization forming units 109A and 109B.

Specifically, the changeover switch 208A switches the transmission path so that the following processing is realized.

A process for guiding the image data output from the inter-apparatus distribution unit 205A to the synchronization forming unit 109A in the first frame period. In the second frame period, the high frequency output from the high frequency emphasis / high frequency suppression unit 202A. Processing for deriving region-suppressed frame data to synchronization forming unit 109A. Processing for deriving luminance control data output from luminance restoring unit 206A to synchronization forming unit 109A in the first frame period. Reference in second frame period. Processing for guiding the brightness control data output from the brightness determining unit 207A to the synchronization forming unit 109A

The changeover switch 208B switches the transmission path so that the following processing is realized.

A process for guiding the image data output from the inter-apparatus distribution unit 205B to the synchronization forming unit 109B during the first frame period. The high frequency output from the high frequency emphasis / high frequency suppression unit 202B during the second frame period. Processing for deriving region-suppressed frame data to synchronization forming unit 109B. Processing for deriving luminance control data output from luminance restoring unit 206B to synchronization forming unit 109B in the first frame period. Reference in second frame period. Processing for guiding the luminance control data output from the luminance determining unit 207B to the synchronization forming unit 109B

  According to the configuration described above, the same image as the first image of the first embodiment can be displayed on one of the projection apparatus A and the projection apparatus B, and the first embodiment can be displayed on the other of the projection apparatus A and the projection apparatus B. The same image as the second image can be displayed. Therefore, also in the present embodiment, as in the first embodiment, the image quality of the composite image can be improved with high accuracy.

  Further, in this embodiment, image data is not transmitted between the projection apparatuses, and only data (instruction; information) having a small data size is transmitted between the projection apparatuses. Therefore, in this embodiment, the data transmission time between the projection apparatuses can be shortened, and the time required for displaying the composite image can be shortened. In this embodiment, since the data size of data transmitted between the projection apparatuses is small, it is not necessary to use a high-speed interface (interface capable of high-speed data communication) for data transmission between the projection apparatuses. By shortening the transmission time, the data size of data (data for the transmission time) that needs to be temporarily held for the synchronization process can be reduced, and the storage unit included in the synchronization forming unit The storage capacity can be reduced. That is, an inexpensive storage unit having a small storage capacity can be used as the storage unit included in the synchronization forming unit. Therefore, the cost of each projection apparatus can be reduced.

  As shown in FIG. 5, different signal sources 1 and 2 may be used instead of the distributor 213 in FIG. In the example of FIG. 5, the signal source 1 outputs image data to the projection apparatus A, and the signal source 2 outputs image data to the projection apparatus B in synchronization with the output of the image data from the signal source 1. The image data output from the signal sources 1 and 2 may or may not be input image data (first original image data). For example, the first image data and the second image data are generated in advance, one of the first image data and the second image data is recorded in advance in the signal source 1, and the other of the first image data and the second image data is the signal source. 2 may be recorded in advance. Then, the signal source 1 may output one of the first image data and the second image data, and the signal source 2 may output the other of the first image data and the second image data. According to such a configuration, it is possible to display a high-quality composite image even if the projection apparatus does not have a function of generating the first image data and the second image data. And the cost of a projection apparatus can be reduced significantly.

<Example 3>
Embodiment 3 of the present invention will be described below. In the following, differences (configuration, processing, etc.) from the first and second embodiments will be described in detail, and the description of the same points as the first and second embodiments will be omitted. In the present embodiment, as in the first embodiment, an example in which input image data is input to only one projector will be described. However, the processing of the present embodiment can also be implemented with a configuration in which input image data is input to each projection apparatus. In the present embodiment, the plurality of projection devices include two or more projection devices having different displayable luminance ranges (dynamic ranges). It can be said that the dynamic range is “a gradation range that is a range of gradation values that can be displayed”.

  A display device having a very wide dynamic range, an image format in which a very wide dynamic range is defined (data format of image data), and the like have been proposed. The very wide dynamic range is called “HDR (High Dynamic Range)” or the like. In the present embodiment, an example in which a non-HDR compatible projection apparatus is used as the projection apparatus A and an HDR compatible projection apparatus is used as the projection apparatus B will be described. Projectors that do not support HDR are projectors that have a narrow dynamic range, and projectors that support HDR are projectors that have a wide dynamic range.

  FIG. 6 is a block diagram illustrating a configuration example of the projection system according to the present embodiment. In FIG. 6, the same functional parts as those in the first embodiment (FIG. 1) are denoted by the same reference numerals as those in the first embodiment. As shown in FIG. 6, the projection system according to the present embodiment includes a projection apparatus A and a projection apparatus B.

  The dynamic range acquisition unit 313 acquires range information regarding the dynamic range for each of the plurality of projection apparatuses (range information acquisition process). In the present embodiment, the dynamic range acquisition unit 313 stores range information of the projection apparatus A in advance, and acquires range information of the projection apparatus B from the projection apparatus B. In addition, the acquisition method of range information is not specifically limited. For example, range information may be acquired by specifying a dynamic range by a user. Range information may be acquired from the Internet. When the dynamic range of the projection apparatus can be changed, the dynamic range may be increased, the dynamic range may be reduced, and the like as appropriate.

  The saturation determination unit 303 determines whether there is a saturation pixel in the high frequency emphasis frame data output from the high frequency emphasis / high frequency suppression unit 102. In this embodiment, the presence / absence of a saturated pixel is determined based on the high frequency emphasis frame data and the range information acquired by the dynamic range acquisition unit 313. Specifically, a pixel having a gradation value outside the dynamic range (gradation range) indicated by the range information is detected as a saturated pixel. This process is performed for each of the plurality of projection apparatuses. For HDR compatible projection devices, it is almost impossible to detect saturated pixels.

  Note that the predetermined range used for determining whether or not the pixel is a saturated pixel may be different from the dynamic range. Further, the above processing of the saturation determination unit 303 may be performed only for some projection apparatuses. For example, the above process may be performed only for the projection apparatus that plays the role of the first projection apparatus of the first embodiment. In the present embodiment, based on the range information of each projection apparatus, the projection apparatus that plays the role of the first projection apparatus of Embodiment 1 and the projection apparatus that plays the role of the second projection apparatus of Embodiment 1 are selected. . In addition, the selection method of a projection apparatus is not specifically limited. The projection device that plays the role of the first projection device of Embodiment 1 and the projection device that plays the role of the second projection device of Embodiment 1 may be determined in advance.

  The inter-apparatus distribution unit 305 selects a first projection apparatus and a second projection apparatus from a plurality of projection apparatuses based on each range information acquired by the dynamic range acquisition unit 313 (apparatus selection process). In the present embodiment, the inter-apparatus distribution unit 305 selects a projection apparatus that is not the projection apparatus having the widest dynamic range among the plurality of projection apparatuses as the first projection apparatus. Then, the inter-apparatus distribution unit 305 selects, as the second projection device, a projection device having a dynamic range wider than the gradation range of the first projection device among the plurality of projection devices. In this embodiment, the dynamic range of the projection apparatus A is narrower than the dynamic range of the projection apparatus B. Therefore, the projection apparatus A is selected as the first projection apparatus, and the projection apparatus B is selected as the second projection apparatus.

  The inter-device distribution unit 305 generates restriction frame data by performing restriction processing on the high frequency emphasized frame data output from the high frequency emphasis / high frequency suppression unit 102. In the present embodiment, as the limiting process, a process of limiting the gradation value outside the dynamic range of the first projection apparatus to the gradation value inside the dynamic range of the first projection apparatus is performed. Note that the predetermined range used in the restriction process may be different from the dynamic range.

  Further, the inter-apparatus distribution unit 305 compensates for the gradation value of the high-frequency emphasis frame data output from the high-frequency emphasis / high-frequency suppression unit 102 by adding the change amount of the tone value due to the restriction process to the tone value of the high-frequency emphasis frame data. Generate frame data. That is, in this embodiment, image data equal to the pre-restriction image data (frame data of the pre-restriction image data) is always used as the second original image data. Specifically, the high frequency emphasis frame data is always used as the second original image data.

  The reference luminance determining units 307A and 307B have the same function as the reference luminance determining unit 107 of the first embodiment. However, in this embodiment, the reference luminance determining unit 307A directly outputs the luminance control data to the synchronization forming unit 109A, and the reference luminance determining unit 307B directly outputs the luminance control data to the synchronization forming unit 109B.

The changeover switch 308 has the same function as the changeover switch 108 of the first embodiment. However, in this embodiment, the changeover switch 308 does not switch the transmission path of the luminance control data, but switches the transmission path of the image data so that the following processing is realized.

A process for guiding the limited frame data output from the inter-apparatus distribution unit 305 during the first frame period to the synchronization forming unit of the first projection apparatus. The output from the inter-apparatus distribution unit 305 during the first frame period. Processing for guiding the compensation frame data to the synchronization forming unit of the second projection apparatus. In the period of the second frame, the high-frequency suppression frame data output from the high-frequency emphasis / high-frequency suppression unit 102 is synchronized with the first projection apparatus. Processing leading to the forming unit and the synchronization forming unit of the second projection device

Specific examples and effects of the processing of the projection system according to the present embodiment will be described with reference to FIG. In this embodiment, as shown in FIG. 7, input image data is input only to the first projection apparatus (projection apparatus A). Then, the first projection device generates output image data for each of the first projection device and the second projection device (projection device B). The output image data of the second projection device is transmitted from the first projection device to the second projection device and displayed by the second projection device.

  In FIG. 7, “DR1” indicates the dynamic range of the first projector, and “DR2” indicates the dynamic range of the second projector. As shown in FIG. 7, in the present embodiment, the dynamic range DR1 of the first projection device is narrower than the dynamic range DR2 of the second projection device.

  In the present embodiment, the first projection device generates high-frequency emphasized frame data as intermediate image data from the input image data. Specifically, as shown in FIG. 7, high-frequency emphasized frame data is used as intermediate image data corresponding to the first frame and the first projection device, and intermediate image data corresponding to the first frame and the second projection device. Is generated. In the example of FIG. 7, the high-frequency emphasis frame data includes saturated pixels having gradation values outside the dynamic range of the first projection device. However, since the dynamic range of the second projector is wide, the high-frequency emphasized frame data does not include saturated pixels having gradation values outside the dynamic range of the second projector.

  In this embodiment, as shown in FIG. 7, the inter-apparatus distribution unit 305 generates output image data corresponding to the first frame and the first projection device by performing restriction processing on the high-frequency emphasized frame data. Then, the inter-apparatus distribution unit 305 generates output image data corresponding to the first frame and the second projection apparatus by adding the change amount of the gradation value by the restriction process to the high frequency emphasis frame data. In addition, as illustrated in FIG. 7, the first projection device generates high-frequency suppression frame data as output image data corresponding to the second frame, the first projection device, and the second projection device.

  Since the dynamic range of the second projector is wide, the gradation value after the addition is unlikely to be a gradation value outside the dynamic range of the second projector. Even if the gradation value after the addition exceeds the upper limit gradation value (the upper limit of the gradation value) of the dynamic range of the second projector, the difference between the gradation value after the addition and the upper limit gradation value is It is smaller than the change amount of the gradation value by the restriction process. Similarly, even if the gradation value after the addition falls below the lower limit gradation value (lower limit of the gradation value) of the dynamic range of the second projector, the difference between the gradation value after the addition and the lower limit gradation value is the same. The difference is smaller than the change amount of the gradation value due to the limiting process.

  As described above, also in the present embodiment, the change in the gradation value due to the restriction process on the image data corresponding to the first frame and the first projection device is not supplemented by the image data of the second frame, and the first frame and Supplemented with image data corresponding to the second projection device. Thereby, the image quality of the composite image can be improved with high accuracy.

  Further, in this embodiment, by using a projection device with a wide dynamic range as the second projection device, the high-frequency emphasized frame data can always be used as the second original image data. Thereby, the brightness | luminance fluctuation | variation of a synthesized image, etc. can be suppressed, without changing the light emission brightness of a light emission part. Specifically, as the distribution of the stack luminance corresponding to the first frame and the luminance distribution of the output image data, the distribution equivalent to the distribution of the stack luminance corresponding to the first frame and the luminance distribution of the intermediate image data in FIG. Can be obtained. As a result, it is possible to realize high-luminance display (projection) by stack projection while maintaining image reproducibility. Further, in the present embodiment, the offset values of the first and second embodiments are not used, and thus the luminance does not increase due to the offset value. Therefore, as the stack luminance corresponding to the first frame and the luminance distribution of the output image data, a luminance equivalent to the stack luminance corresponding to the first frame and the luminance distribution of the intermediate image data in FIG. 12 can be obtained.

<Example 4>
Embodiment 4 of the present invention will be described below. In the following, points (configuration, processing, etc.) different from those of the first to third embodiments will be described in detail, and the description of the same points as those of the first to third embodiments will be omitted. In the present embodiment, as in the first embodiment, an example in which input image data is input to only one projector will be described. However, the processing of the present embodiment can also be implemented with a configuration in which input image data is input to each projection apparatus. In this embodiment, a plurality of image data is input as input image data. The plurality of input image data are a plurality of image data generated on the assumption that they are displayed by a plurality of projection devices.

  A projection method called “projection mapping” has been proposed as a projection method utilizing the high degree of freedom in image display of the projection apparatus. In projection mapping, for example, projection is performed using the surface of a building as a projection plane. Projection methods that take advantage of the high degree of freedom in image display of the projection apparatus are considered to be actively proposed in the future.

  As an example, consider a case where two images based on two different image data are combined and displayed. When a direct-view display device is used, two image data are combined using a combining method called “alpha blend”. In alpha blending, for example, a weighted average of two image data is generated as composite image data. However, in the method of synthesizing image data, gradation, spatial resolution, and the like are deteriorated from those of image data before synthesis. For example, when two image data are combined with a weight (blend rate) of 0.5: 0.5, gradation, spatial resolution, and the like are halved.

  Therefore, the image I1 which is one of the two images to be synthesized is displayed by the projection device P1, and the image I1 which is the other of the two images is displayed by being superimposed on the image I1 by the projection device P2. According to such a method, it is possible to suppress the above-described decrease in gradation, spatial resolution, and the like. The present embodiment is an example assuming such display (projection).

  FIG. 8 is a block diagram illustrating a configuration example of the projection system according to the present embodiment. In FIG. 8, the same reference numerals as those in the first and third embodiments are assigned to the same functional units as those in the first and third embodiments (FIGS. 1 and 6). As shown in FIG. 8, the projection system according to the present embodiment includes a first projection device and a second projection device.

  The frame rate conversion unit 401 has the same function as the frame rate conversion unit 101 of the first embodiment. However, in this embodiment, the frame rate conversion unit 401 performs the same processing as the processing of the frame rate conversion unit 101 for each of a plurality of input image data. In this embodiment, two input image data ID1 and ID2 are input as a plurality of input image data. When the input image data is still image data, the frame rate conversion unit 401 outputs the still image data as frame data of converted moving image data. For example, when the input image data ID1 is moving image data and the input image data ID2 is still image data, the frame rate conversion unit 401 uses the frame rate of the converted moving image data generated from the moving image data ID1. Still image data ID2 is repeatedly output.

The layer information acquisition unit 413 acquires layer information (image information) related to the input image data for each of the plurality of input image data (image information acquisition process). In this embodiment, information indicating whether the input image data is still image data or moving image data is acquired as the layer information. In addition, the acquisition method of layer information is not specifically limited. For example, information specified by the user may be acquired as range information. Layer information may be extracted from metadata (header information) added to input image data. The layer information is not limited to the above information. For example, information indicating the priority order of input image data (type information indicating whether the image data is foreground image data or background image data, a blend rate, etc.) may be used as the layer information.

  The high frequency emphasis / high frequency suppression unit 402 determines whether to perform enhancement processing and suppression processing for each of the plurality of input image data based on the layer information acquired by the layer information acquisition unit 413. For the input image data determined to “execute enhancement processing and suppression processing”, the high frequency enhancement / high frequency suppression unit 402 uses the high frequency enhanced frame data and the high frequency from the frame data output from the frame rate conversion unit 401. Generate suppression frame data. Then, the high frequency emphasis / high frequency suppression unit 402 outputs the high frequency emphasis frame data to the saturation determination unit 403 and the inter-device distribution unit 405, and outputs the high frequency suppression frame data to the changeover switch 408. For the input image data determined to “do not execute enhancement processing and suppression processing”, the high frequency enhancement / high frequency suppression unit 402 uses the frame data output from the frame rate conversion unit 401 as it is as the saturation determination unit 403. The data is output to the inter-apparatus distribution unit 405. Note that since no saturated pixel exists in the frame data output from the frame rate conversion unit 401, the frame data output from the frame rate conversion unit 401 may not be output to the saturation determination unit 403.

  In this embodiment, when the input image data is moving image data, it is determined that “enhancement processing and suppression processing are executed”, and when the input image data is still image data, “enhancement processing and suppression processing is executed. It is judged as “No”. The determination method is not particularly limited. For example, when the input image data is foreground image data, it is determined that “enhancement processing and suppression processing are executed”, and when the input image data is background image data, “enhancement processing and suppression processing is executed. It may be determined that “no”. When the blend rate of the input image data is equal to or greater than the threshold, it is determined that “enhancement processing and suppression processing are executed”, and when the blend rate of the input image data is less than the threshold, “execute enhancement processing and suppression processing” is performed. It may be determined that “no”.

  Although details will be described later, in this embodiment, input image data determined to “execute enhancement processing and suppression processing” is used as the first original image data, and it is determined that “enhancement processing and suppression processing are not performed”. The inputted image data is used as the second original image data. Therefore, “determination of whether to perform enhancement processing and suppression processing” is “image selection for selecting first original image data and second original image data from a plurality of input image data based on a plurality of layer information” It can also be said to be “processing”.

  Note that the image selection process may not be performed. For example, input image data used as the first original image data and input image data used as the second original image data may be determined in advance. Specifically, the first projection device has two input terminals T1 and T2, and the image data input to the input terminal T1 is used as the first original image data, and the image input to the input terminal T2 is used. Data may be used as the second original image data. In such a configuration, for example, moving image data is input to the input terminal T1, and still image data is input to the input terminal T2.

  The saturation determination unit 403 saturates the frame data for each frame data output from the high frequency emphasis / high frequency suppression unit 402 (high frequency emphasis frame data or frame data not subjected to emphasis processing and suppression processing). It is determined whether or not a pixel exists. Then, saturation determination section 403 outputs the result of the determination to inter-apparatus distribution section 405. Saturated pixels can be present in high-frequency emphasized frame data, and are not present in unprocessed frame data (frame data that has not been subjected to enhancement processing and suppression processing). For this reason, “with saturated pixels” or “without saturated pixels” is obtained as a determination result for high-frequency emphasized frame data, and “without saturated pixels” is obtained as a determination result for unprocessed frame data. In this embodiment, the unprocessed frame data is still image data input to the first projection device.

  Based on the determination result of the saturation determination unit 403, the inter-apparatus distribution unit 405 generates restriction frame data by performing restriction processing on the high frequency emphasis frame data output from the high frequency emphasis / high frequency suppression unit 402. . Further, the inter-apparatus distribution unit 405 adds the amount of change in the gradation value due to the restriction process to the gradation value of the unprocessed frame data output from the high-frequency emphasis / high-frequency suppression unit 102, thereby obtaining a compensation frame. Generate data. The limited frame data is output to the changeover switch 408, and the compensation frame data is output to the synchronization forming unit 109B. In this embodiment, the compensation frame data is guided to the synchronization forming unit 109B in both the first frame period and the second frame period.

  The third original image data is a target of restriction processing. Therefore, in this embodiment, the high frequency emphasis frame data is used as the third original image data. The first original image data is image data that is the source of the third original image data. Therefore, in this embodiment, input image data determined to “execute enhancement processing and suppression processing” is used as the first original image data. The second original image data is a target of addition of the change amount of the gradation value by the restriction process. For this reason, in this embodiment, input image data determined as “not performing enhancement processing and suppression processing” is used as second original image data. Specifically, unprocessed frame data obtained from input image data determined to “not execute enhancement processing and suppression processing” is used as second original image data. In this embodiment, still image data input to the first projection device is used as second original image data.

The changeover switch 408 has the same function as the changeover switch 108 of the first embodiment. However, in the present embodiment, the changeover switch 408 does not switch the transmission path of the luminance control data, but switches the transmission path of the image data so that the following processing is realized.

A process of guiding the limited frame data output from the inter-apparatus distribution unit 405 to the synchronization forming unit 109A in the first frame period. A process of outputting from the high frequency emphasis / high frequency suppression unit 402 in the second frame period. Processing for guiding high frequency suppression frame data to the synchronization forming unit 109A

  It may be determined that “enhancement processing and suppression processing are executed” for both input image data ID1 and input image data ID2. In that case, for example, high-frequency emphasized frame data in which saturated pixels are present among the two high-frequency emphasized frame data generated from the two input image data ID1 and ID2 is used as the third original image data. . Of the two high-frequency emphasized frame data, high-frequency emphasized frame data having no saturated pixel is used as the second original image data. In the period of the first frame, the limited frame data is guided to the synchronization forming unit 109A, and the compensation frame data is guided to the synchronization forming unit 109B. Further, in the period of the second frame, one of the two high-frequency suppression frame data generated from the two input image data ID1 and ID2 is guided to the synchronization forming unit 109A, and the other of the two high-frequency suppression frame data is synchronized. Guided to forming part 109B.

  When saturated pixels exist in both of the two high-frequency emphasized frame data, among the two high-frequency emphasized frame data, the high-frequency emphasized frame data in which the saturation amount of the saturated pixels is relatively large is the third element. It may be used as image data. Of the two high-frequency emphasized frame data, high-frequency emphasized frame data in which the saturation amount of saturated pixels is relatively small may be used as the second original image data. When the gradation value of the saturated pixel is larger than the upper limit gradation value of the predetermined range, “saturation amount” means “difference between the upper limit gradation value and the gradation value of the saturation pixel”. When the gradation value of the saturated pixel is smaller than the lower limit gradation value of the predetermined range, the “saturation amount” means “the difference between the lower limit gradation value and the gradation value of the saturated pixel”.

  Specific examples and effects of the processing of the projection system according to the present embodiment will be described with reference to FIG. In this embodiment, as shown in FIG. 9, input image data ID1 and ID2 are input to the first projection device. Then, output image data is generated for each of the first projection device and the second projection device by the first projection device. The output image data of the second projection device is transmitted from the first projection device to the second projection device and displayed by the second projection device.

  Here, an example in which the input image data ID1 is moving image data and the input image data ID2 is still image data will be described. In the present embodiment, the first projection device uses the input image data ID1 as the first original image data, and uses the input image data ID2 as the second original image data. In FIG. 9, the stack luminance distribution corresponding to the luminance distribution of the input image data ID1 and the luminance distribution of the input image data ID2 is the sum of the luminance distribution of the input image data ID1 and the luminance distribution of the input image data ID2. Hereinafter, “the sum of the luminance of the input image data ID1 and the luminance of the input image data ID2” will be referred to as “assumed luminance”.

  The enhancement process and the compression process are performed in order to improve the moving image visibility. Therefore, in the present embodiment, the first projection device performs enhancement processing and compression processing only on the input image data ID1 that is moving image data. Accordingly, as shown in FIG. 9, high-frequency emphasized frame data is generated as intermediate image data corresponding to the first frame and the first projection device. Then, high-frequency suppression frame data is generated as output image data corresponding to the second frame and the first projection device. Further, as the intermediate image data corresponding to the first frame and the second projection device and the output image data corresponding to the second frame and the second projection device, the input image data ID2 which is still image data is employed.

  In this embodiment, as shown in FIG. 9, the inter-apparatus distribution unit 405 generates output image data corresponding to the first frame and the first projection device by performing restriction processing on the high-frequency emphasized frame data. Then, the inter-apparatus distribution unit 405 generates output image data corresponding to the first frame and the second projection apparatus by adding the change amount of the gradation value due to the restriction process to the input image data ID2. Since there is no saturated pixel in the input image data ID2, the above addition can reduce the saturation amount, the number of saturated pixels, and the like while suppressing deterioration in image quality (deterioration in image quality of the composite image) due to the restriction process. .

  As described above, also in the present embodiment, the change in the gradation value due to the restriction process on the image data corresponding to the first frame and the first projection device is not supplemented by the image data of the second frame, and the first frame and Supplemented with image data corresponding to the second projection device. Thereby, the image quality of the composite image can be improved with high accuracy. Also in this embodiment, it is possible to suppress the luminance fluctuation of the composite image. Specifically, as shown in FIG. 9, a distribution equivalent to the distribution of assumed luminance can be obtained as the distribution of perceived luminance. As a result, it is possible to realize high-luminance display (projection) by stack projection while maintaining image reproducibility. Further, in the present embodiment, the offset values of the first and second embodiments are not used, and thus the luminance does not increase due to the offset value. Therefore, the brightness equivalent to the assumed brightness can be obtained as the perceived brightness.

<Example 5>
Embodiment 5 of the present invention will be described below. In the following, points (configuration, processing, etc.) different from those of the first to fourth embodiments will be described in detail, and the description of the same points as those of the first to fourth embodiments will be omitted. In the present embodiment, as in the second embodiment, an example in which input image data is input to each projection apparatus will be described. However, the processing of the present embodiment can be performed even in a configuration in which input image data is input to only one projector. In the present embodiment, as in the fourth embodiment, an example in which input image data differs between projection apparatuses will be described. However, the processing of this embodiment can be performed even when the input image data is the same between the projection apparatuses.

  In the first to fourth embodiments, the example in which image processing including frame rate conversion processing, enhancement processing, suppression processing, restriction processing, and the like is performed as the predetermined image processing has been described. In the present embodiment, an example will be described in which a multiplication process is performed in which each tone value of input image data is multiplied by a predetermined gain value, instead of the frame rate conversion process, the enhancement process, and the suppression process. A general display device has a function of adjusting the brightness of the display image, the contrast of the display image, and the like according to the user's preference. Such adjustment is realized, for example, by multiplying image data by a predetermined gain value in accordance with an instruction from the user. Since the gradation value of the image data has an upper limit and a lower limit, the state after adjustment (brightness, contrast, etc.) also has an upper limit and a lower limit. Therefore, the gradation value larger than the upper limit is restricted to the upper limit, and the gradation value smaller than the lower limit is restricted to the lower limit. Alternatively, the gain value is limited so that the gradation value does not exceed the upper limit and the gradation value does not fall below the lower limit.

  FIG. 10 is a block diagram illustrating a configuration example of the projection system according to the present embodiment. 10, the same reference numerals as those in the first and third embodiments are assigned to the same functional units as those in the first and third embodiments (FIGS. 1 and 6). As shown in FIG. 10, the projection system according to the present embodiment includes a first projection device and a second projection device.

  The signal expansion unit 502A generates first expanded image data by multiplying each gradation value of the input image data ID1 by a predetermined gain value (multiplication processing). Similarly, the signal expansion unit 502B generates second expanded image data by multiplying each gradation value of the input image data ID2 by a predetermined gain value.

  Note that the gain value multiplied by each gradation value of the input image data ID1 may be the same as or different from the gain value multiplied by each gradation value of the input image data ID2. The gain value may be a fixed value determined in advance by the manufacturer, or may be a value that is appropriately changed according to an instruction from the user, a use environment of the projection system, and the like. The multiplication process for the input image data ID2 may be omitted. Each of the input image data ID1 and the input image data ID2 may be moving image data or still image data.

  Note that it is preferable to use, as the input image data ID1, image data in which saturated pixels due to multiplication processing are likely to occur as input image data ID1, and image data in which saturation pixels due to multiplication processing are unlikely to occur. For example, image data having many pixels having gradation values far from the center of a predetermined range is used as the input image data ID1, and image data having few pixels having gradation values far from the center of the predetermined range is used as the input image data ID2. It is preferable to use it. Image data with relatively high importance may be used as input image data ID1, and image data with relatively low importance may be used as input image data ID2.

  The inter-apparatus distribution unit 505 generates restriction image data by performing restriction processing on the first decompressed image data. The method for generating restricted image data is the same as the method for generating restricted frame data in the first embodiment. Then, the inter-apparatus distribution unit 505 outputs the restricted image data to the synchronization forming unit 109A. Further, the inter-apparatus distribution unit 505 notifies the addition unit 513 of the change in the gradation value due to the restriction processing that has been performed. It can be said that “the change amount of the gradation value by the restriction process” is “restriction information”.

  Based on the restriction information, the adder 513 generates compensation image data that compensates for the change in gradation value due to the restriction process. The method for generating compensation image data is the same as the method for generating compensation frame data in the first embodiment. Specifically, the adding unit 513 generates compensated image data by adding the amount of change in the tone value resulting from the restriction process to the tone value of the second decompressed image data. Then, the adding unit 513 outputs the compensation image data to the synchronization forming unit 109B.

  Specific examples and effects of the processing of the projection system according to the present embodiment will be described with reference to FIG. As shown in FIG. 11, in this embodiment, input image data is input to each of the first projection device and the second projection device. In this embodiment, the input image data ID1 of the first projection device is different from the input image data ID2 of the second projection device.

  In this embodiment, as shown in FIG. 10, the signal decompression unit 502A multiplies each tone value of the input image data ID1 by a gain value, thereby obtaining the first decompression as intermediate image data corresponding to the first projection device. Generate image frame data. Further, the signal expansion unit 502B generates second expanded image frame data as intermediate image data corresponding to the second projection device by multiplying each gradation value of the input image data ID2 by a gain value. In the example of FIG. 10, the first decompressed image data includes saturated pixels, and the second decompressed image data does not include saturated pixels.

  In this embodiment, as shown in FIG. 10, the inter-apparatus distribution unit 505 generates output image data corresponding to the first projection device by performing a restriction process on the first decompressed image data. Then, the adding unit 513 generates output image data corresponding to the second projection device by adding the change amount of the gradation value by the restriction process to the second decompressed image data. By the above addition, it can be expected that the saturation amount, the number of saturated pixels, and the like are reduced while suppressing the image quality deterioration (deterioration of the image quality of the composite image) due to the restriction process.

  In addition, each hardware part of Examples 1-5 may be individual hardware, and may not be so. The functions of two or more functional units may be realized by common hardware. Each of a plurality of functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Each functional unit may be realized by hardware or not. For example, the apparatus may include a processor and a memory in which a control program is stored. The functions of at least some of the functional units included in the apparatus may be realized by the processor reading and executing the control program from the memory.

  In addition, Examples 1-5 are an example to the last, and the structure obtained by changing suitably and changing the structure of Examples 1-5 within the range of the summary of this invention is also contained in this invention. Configurations obtained by appropriately combining the configurations of Examples 1 to 5 are also included in the present invention.

<Other examples>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  105, 205A, 205B, 305, 405: inter-apparatus distribution unit 513: addition unit

Claims (20)

A first projection device that displays a first image based on the first image data in the projection area by performing projection on the projection area of the projection surface;
A second projection device that displays a second image based on second image data in the projection area by performing projection on the projection area;
An image processing device for generating image data for a projection system having a plurality of projection devices including:
The first image data is image data obtained by performing predetermined image processing on the first original image data,
The predetermined image processing includes a limiting process of limiting a gradation value outside a predetermined range of gradation values to a gradation value inside the predetermined range;
The image processing apparatus includes:
Obtaining means for obtaining restriction information on the restriction processing performed;
Generation means for generating, as the second image data, image data corresponding to a change in gradation value by the restriction process based on the restriction information obtained by the obtaining means;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the second image data is image data that compensates for a change in gradation value due to the restriction process. Processing means for generating the first image data by performing the predetermined image processing on the first original image data;
The acquisition unit acquires the restriction information by determining a change amount of a gradation value by the restriction process based on the predetermined image processing performed by the processing unit. The image processing apparatus according to 1 or 2. For each of the plurality of projection devices, range information acquisition means for acquiring range information regarding a gradation range that is a range of gradation values that can be displayed;
Among the plurality of projection devices, a projection device that is not the projection device having the widest gradation range is selected as the first projection device, and among the plurality of projection devices, the gradation range is a floor of the first projection device. Device selection means for selecting a projection device wider than the adjustment range as the second projection device;
The image processing apparatus according to claim 1, further comprising: The image processing apparatus according to claim 1, wherein the predetermined range is a range of gradation values that can be displayed by the first projection apparatus. The said generation means produces | generates the said 2nd image data by adding the variation | change_quantity of the gradation value by the said limitation process to the gradation value of 2nd original image data. The image processing apparatus according to any one of the above. The generation means is configured such that a gradation value of a pixel corresponding to a restricted pixel whose gradation value is restricted to a value inside the predetermined range by the restriction process indicates a change amount of the gradation value by the restriction process. Image data which is a gradation value obtained by adding to the gradation value of the original image data, and in which the gradation value of a pixel corresponding to a pixel other than the restricted pixel is the gradation value of the second original image data The image processing apparatus according to claim 6, wherein the second image data is generated as the second image data. Image information acquisition means for acquiring image information related to image data for each of a plurality of image data;
Image selecting means for selecting the first original image data and the second original image data from the plurality of image data based on a plurality of pieces of image information corresponding to the plurality of image data;
The image processing apparatus according to claim 6, further comprising: The image processing apparatus according to claim 6, wherein the second original image data is equal to the first image data before the restriction process is performed. The image processing apparatus according to claim 6, wherein the second original image data is different from the first image data before the restriction process is performed. The image processing apparatus according to claim 10, wherein a gradation value of each pixel of the second original image data is a predetermined value. The image processing apparatus according to claim 10, wherein the second original image data is still image data. The generating means includes
The second image data corresponding to the first image data including the restricted pixel whose gradation value is restricted to a value inside the predetermined range by the restriction process, the second original image data, and the gradation by the restriction process With the amount of change in value,
The image processing apparatus according to claim 6, wherein the second original image data is adopted as second image data corresponding to the first image data not including the restriction pixel. The generating means includes
The second image data corresponding to the first image data including the restricted pixel whose gradation value is restricted to a value inside the predetermined range by the restriction process, the second original image data, and the gradation by the restriction process With the amount of change in value,
The image processing apparatus according to claim 11, wherein the first image data before the restriction process is applied as the second image data corresponding to the first image data not including the restriction pixel. The first projection device includes first light emitting means, and first modulation means for modulating light from the first light emitting means based on the first image data,
The second projection device includes second light emitting means, and second modulation means for modulating light from the second light emitting means based on the second image data,
The image processing apparatus further includes control means for controlling at least one of the light emission luminance of the first light emission means and the light emission luminance of the second light emission means,
The control means includes
In the display period of the first image data including the restricted pixels, the first light emission means and the second light emission means have a luminance that is twice as high as a predetermined luminance. Controlling at least one of the light emission brightness of the light emission means and the light emission brightness of the second light emission means;
In the display period of the first image data that does not include the restricted pixels, the first light emitting unit has a predetermined luminance so that the sum of the light emitting luminance of the first light emitting unit and the light emitting luminance of the second light emitting unit becomes the predetermined luminance. The image processing apparatus according to claim 14, wherein at least one of light emission luminance and light emission luminance of the second light emitting unit is controlled. The predetermined image processing includes
N frames (N is an integer greater than or equal to 2) are generated from each frame of the first original image data that is moving image data, and the frame rate of the first original image data includes a plurality of generated frames. Frame rate conversion processing for generating converted video data that is video data having a frame rate N times the
An enhancement process that emphasizes a spatial high-frequency component for a first frame that is one of an odd-numbered frame and an even-numbered frame among a plurality of frames of the converted moving image data;
A suppression process for suppressing a spatial high-frequency component for the second frame that is the other of the odd-numbered frame and the even-numbered frame among the plurality of frames of the converted moving image data; and
The image processing apparatus according to claim 1, further comprising the restriction process on the image data of the first frame after the enhancement process is performed. The predetermined image processing includes
A multiplication process for multiplying each gradation value of the first original image data by a predetermined gain value; and
The image processing apparatus according to claim 1, further comprising the restriction process on the image data after the multiplication process is performed. A first projection device that displays a first image based on the first image data in the projection area by performing projection on the projection area of the projection surface;
A second projection device that displays a second image based on second image data in the projection area by performing projection on the projection area;
An image processing method for generating image data for a projection system having a plurality of projection devices including:
The first image data is image data obtained by performing predetermined image processing on the original image data,
The predetermined image processing includes a limiting process of limiting a gradation value outside a predetermined range of gradation values to a gradation value inside the predetermined range;
The image processing method includes:
Obtaining restriction information regarding the restriction processing that has been performed;
Generating, as the second image data, image data corresponding to a change in gradation value by the restriction process based on the obtained restriction information;
An image processing method comprising: The image processing method according to claim 18, wherein the second image data is image data that compensates for a change in gradation value due to the restriction process.   A program for causing a computer to execute each step of the image processing method according to claim 18 or 19.

Images