WO2018097114A1

WO2018097114A1 - Image processing device, image processing method, and program

Info

Publication number: WO2018097114A1
Application number: PCT/JP2017/041739
Authority: WO
Inventors: 将史大矢
Original assignee: キヤノン株式会社
Priority date: 2016-11-28
Filing date: 2017-11-21
Publication date: 2018-05-31

Abstract

An image processing device, which generates image data for generating one image by superimposing a first output image output by a first image output device and a second output image output by a second image output device, is characterized by having a first acquisition means for acquiring input image data, a second acquisition means for acquiring first output image data to be output to a first image output device, and a generation means for generating second output image data to be output to the second image output device on the basis of the input image data and the first output image data and in that the sharpness of the image represented by the second output image data is in accordance with the sharpness of the image represented by the first output image data.

Description

Image processing apparatus, image processing method, and program

The present invention relates to an image processing technique for reproducing one image by combining images output from a plurality of devices.

In recent years, there are increasing opportunities to handle images with higher definition and a wider dynamic range in imaging with a digital camera or the like and CG (computer graphics) rendering. Similarly, the demand for image reproduction as described above by an output device such as a printer is increasing. In response to this demand, the use of projectors, large-format printers, and the like that can reproduce the color, gradation, texture, and the like of real objects (subjects) and CG objects in powerful sizes is expected. JP-A-2010-103863 discloses a technique for reproducing one image by superimposing images output from a plurality of devices. In JP-A-2010-103863, color reproduction in a dark region of an image reproduced by using an image projection apparatus superimposed on a printed matter formed by an image forming apparatus, compared to the case of using only the image projection apparatus. Techniques for expanding the scope are disclosed.

JP 2010-103863 A

As described above, when images output from a plurality of devices are superimposed, images with different resolutions may be superimposed. In this case, there is a problem that the sharpness of the image reproduced by superimposing the images output from each device is lower than the sharpness of the input image. This problem will be described by taking as an example a case where an image output by the image projection apparatus and an image output by the image forming apparatus are superimposed. Since the number of pixels in the image projected by the image projector is constant, the resolution of the image that can be output in inverse proportion to the increase in the image size is reduced. For this reason, when the same image data is input to the image forming apparatus and the image projecting apparatus and is output after being extended, the resolution of the image output by the image projecting apparatus is lower than the resolution of the image output by the image forming apparatus. End up. Due to the reduction in resolution, the information held at the resolution of the input image data cannot be expressed and the sharpness is lowered. In this case, if the image output from each device is superimposed, the sharpness of the image output by the image projector decreases, so that the sharpness of the image reproduced by superimposing the image output from each device is reduced. The degree is lower than the sharpness of the input image.

The present invention provides image processing for suppressing a decrease in the sharpness of a superimposed image with respect to an input image, which occurs when a single image is generated by superimposing images output from a plurality of devices based on the input image. The purpose is to do.

In order to solve the above problems, an image processing apparatus according to the present invention outputs a first output image output from a first image output apparatus based on an input image, and an output from a second image output apparatus based on the input image. An image processing device that generates image data to be output to the second image output device in order to generate one image by superimposing the second output image with a higher resolution than the first output image. A first acquisition unit configured to acquire input image data representing the input image; and a second acquisition unit configured to acquire first output image data generated based on the input image data and output to the first image output device. First generation means for generating second output image data to be output to the second image output device based on the acquisition means, the input image data, and the first output image data, and Second Sharpness of the image represented by the image data is characterized in that depending on the sharpness of the first output image data represents an image.

According to the present invention, it is possible to suppress a decrease in the sharpness of a superimposed image with respect to an input image, which occurs when an image output from a plurality of devices is superimposed based on the input image to generate one image.

The block diagram which shows an example of a function structure of the image processing apparatus 1 Flow chart for processing executed by image processing apparatus 1 Flow chart for processing executed by image processing apparatus 1 The figure which shows an example of the 1st color conversion LUT105 for producing | generating projection image data A block diagram showing an example of a hardware configuration of the image processing apparatus 1 Flowchart for processing (S203) for calculating degree of sharpness reduction Flowchart for processing (S203) for calculating degree of sharpness reduction The figure which shows an example of LUT113 for determining a correction coefficient according to ambient light information The figure which shows an example of the 2nd color conversion LUT108 for producing | generating formation image data The block diagram which shows an example of a function structure of the image processing apparatus 1 The block diagram which shows an example of a function structure of the image processing apparatus 1 The block diagram which shows an example of a function structure of the image processing apparatus 1 A diagram schematically showing resolution conversion processing based on variable magnification A diagram schematically showing resolution conversion processing based on variable magnification The figure which showed typically an example of the process which calculates the fall degree of sharpness The figure which showed an example of the processing which emphasizes the sharpness of an image typically Schematic diagram schematically showing the relationship between the image to be reproduced and the image output by the image forming apparatus Flowchart for processing (S203) for calculating degree of sharpness reduction Schematic diagram showing an example of a high-pass filter according to the variable magnification Schematic diagram showing an example of a high-pass filter according to the variable magnification Schematic diagram showing an example of a high-pass filter according to the variable magnification The block diagram which shows an example of a function structure of the image processing apparatus 1 Schematic diagram schematically showing the relationship between the image to be reproduced and the image output by the image projection device The figure which shows an example of UI screen for receiving the input by a user The block diagram which shows the function structure of the image processing apparatus 1 The block diagram which shows the function structure of the image processing apparatus 1 The block diagram which shows the function structure of the image processing apparatus 1 The figure which shows an example of the chart for measuring the output characteristic of the image forming apparatus 3

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, about the same structure, the same code | symbol is attached | subjected and demonstrated.

[Example 1]
<Relationship between the image to be reproduced and the image output by the device>
First, the relationship between the image to be reproduced and the image output from the image forming apparatus and the relationship between the image to be reproduced and the image output from the image projection apparatus will be described with reference to FIGS. Note that the resolution in this embodiment is also expressed as a measure of the fineness of expression in a projected or formed image, and uses dpi (dot per inch) as a unit. Further, the sharpness in this embodiment is a measure of the clarity of a fine portion (high resolution portion) of an image. For example, the sharpness can be expressed by a response function MTF (modulation transfer function) represented by a rate of decrease in output contrast with respect to input contrast at each resolution. When the contrast reduction rate is low and the response is good, the sharpness is high, and when the contrast reduction rate is high and the response is bad, the sharpness is low.

FIG. 14A shows input image data (also referred to as first image data) representing an image to be reproduced. FIG. 14B shows formed image data (also referred to as third image data) representing an image having the same size as the input image data (enlargement ratio 1.0), and FIG. 14C shows twice the input image data ( The formation image data showing the image of the size of the enlargement ratio 2.0) is shown. The formed image data is image data input by the image forming apparatus to form an image on a recording medium. In FIG. 14, an input image data representing an image having a resolution of 300 dpi and a size of 8192 pixels in the horizontal direction and 4320 pixels in the vertical direction, and an image forming apparatus capable of forming an image with a resolution of 1200 dpi based on the input image data. An example of the formed image is shown. The image forming apparatus forms an image by scanning a recording head that ejects ink on a recording medium a plurality of times. Since the distance between the recording medium and the recording head is constant regardless of the size of the image to be formed, the resolution of the image formed by the image forming apparatus is constant. For example, as shown in FIG. 14, when the above-described image forming apparatus outputs an image with a resolution of 300 dpi at a resolution of 1200 dpi, the number of pixels of the image represented by the formed image data is set both vertically and horizontally with respect to the number of pixels of the image represented by the input image data. 4 times. When an image is formed with an image size twice that of the image represented by the input image data, the number of pixels of the image represented by the formed image data is doubled both vertically and horizontally with respect to the number of pixels of the image represented by the input image data. As described above, the resolution of an image output (formed) by the image forming apparatus is made constant.

18A shows the input image data representing the image to be reproduced, and FIG. 18B shows the size of the image represented by the input image data when projected with the number of pixels of the image projection apparatus (enlargement ratio 1.0). 3 shows projection image data (also referred to as second image data) representing an image having an “actual size” size. Further, FIG. 18C shows projection image data representing an image when the size of the image to be projected is twice the “same size” size (enlargement ratio 2.0). The projection image data is image data input by the image projection apparatus to project an image. In FIG. 18, the input image data is the same as in FIG. 14, and the number of pixels of the image output from the image projection apparatus is 4096 pixels in the horizontal direction and 2160 pixels in the vertical direction. Unlike an image forming apparatus, an image projection apparatus generates an image by driving a display element such as a liquid crystal panel for each pixel, and displays the image by projecting the generated image with a projection lens. For this reason, the number of pixels of an image to be output is determined by a display element such as a liquid crystal panel held in advance, and the number of pixels to be output cannot be increased or decreased as in the image forming apparatus. As a result, as shown in FIG. 18, the image was projected in the case of projecting the image at the same size based on the input image data representing the image having the resolution of 300 dpi and in the case of projecting the image at the size of 2 times. The number of pixels in the image is constant. Therefore, the resolution of the image decreases in inverse proportion to the increase in the size of the image to be projected.

As described above, since the image forming apparatus can increase or decrease the number of pixels of the image to be output by the scanning method of the recording head, it is possible to form an image with a constant resolution regardless of the image size. On the other hand, in the image projection apparatus, the number of pixels of the output image is determined by a display element such as a liquid crystal panel, so that the resolution of the image decreases in inverse proportion to the enlargement of the image size caused by extending and projecting the input image. Resulting in. In other words, the output image from the image forming apparatus has a constant sharpness regardless of the size of the reproduction target image, whereas the output image from the image projection apparatus increases in pixel size as the size of the reproduction target image increases. The sharpness when viewed from the position is lowered. In the present embodiment, one image is reproduced by superimposing the image projected by the image projection apparatus and the image formed by the image forming apparatus. At that time, the reduction in the sharpness of the image projected by the image projection apparatus as described above is compensated by enhancing the sharpness of the image formed by the image forming apparatus. Details will be described below.

<Hardware Configuration of Image Processing Apparatus 1>
FIG. 4 is a hardware configuration example of the image processing apparatus 1 in the present embodiment. The image processing apparatus 1 is a computer, for example, and includes a CPU 1401, a ROM 1402, and a RAM 1403. The CPU 1401 executes an OS (Operating System) and various programs stored in the ROM 1402, HDD (Hard Disk Drive) 1412, and the like using the RAM 1403 as a work memory. The CPU 1401 controls each component via the system bus 1408. Note that the processing according to the flowchart to be described later is executed by the CPU 1401 after the program code stored in the ROM 1402, the HDD 1412, or the like is expanded in the RAM 1403. A display 5 is connected to a VC (video card) 1404. A general-purpose I / F (interface) 1405 is connected to an input device 1410 such as a mouse and a keyboard, the image projection apparatus 2, and the image forming apparatus 3 via a serial bus 1409. A SATA (Serial ATA) I / F 1406 is connected to a general-purpose drive 1413 for reading and writing the HDD 1412 and various recording media via a serial bus 1411. A NIC (network interface card) 1407 inputs / outputs information to / from an external device. The CPU 1401 uses various recording media mounted on the HDD 1412 or the general-purpose drive 1413 as a storage location for various data. The CPU 1401 displays a UI (user interface) provided by the program on the display 5 and receives an input such as a user instruction accepted via the input device 1410.

<Functional Configuration of Image Processing Apparatus 1>
FIG. 1 is a block diagram illustrating a functional configuration of the image processing apparatus 1. In FIG. 1, 1 is an image processing apparatus, 2 is an image projection apparatus (projector), 3 is an image forming apparatus (printer), and 4 is illumination that determines ambient light when observing a superimposed image. The superimposed image indicates an image in which the projection image 502 projected by the image projection device 2 is superimposed on the formation image 501 formed on the recording medium by the image forming device 3. The image processing apparatus 1 can be implemented by a printer driver installed in a general personal computer, for example. In that case, each part of the image processing apparatus 1 described below is realized by a computer executing a predetermined program. As another configuration, for example, the image processing device 1 may include the image projection device 2 and the image forming device 3.

The image processing apparatus 1 and the image projection apparatus 2, and the image processing apparatus 1 and the image forming apparatus 3 are connected by an interface or a circuit. The image processing apparatus 1 includes a first input terminal 101, a second input terminal 102, an acquisition unit 103, a first generation unit 104, a first color conversion LUT 105, a calculation unit 106, and a second generation unit 107. , A second color conversion LUT 108, a first output terminal 109, and a second output terminal. The acquisition unit 103 acquires input image data representing an image to be reproduced via the first input terminal 101. Further, the projection state of the image projection device 2 is acquired via the second input terminal 102. The projection state will be described later. The first generation unit 104 refers to the first color conversion LUT 105 based on the input image data described above, and generates image data (projection image data) to be input to the image projection device 2. The calculation unit 106 acquires the input image data and the projection image data, and converts the resolution of the image represented by the input image data and the resolution of the image represented by the projection image data according to the projection state. Further, the degree of reduction in the sharpness of the image represented by the projection image data with respect to the image represented by the input image data is calculated based on the input image data representing the image whose resolution has been converted and the projection image data. The second generation unit 107 enhances the sharpness of the image represented by the input image data based on the input image data and the degree of sharpness reduction. Further, based on input image data representing an image with enhanced sharpness, the second color conversion LUT 108 is referred to, and image data (formed image data) to be input to the image forming apparatus 3 is generated. The projection image data generated by the first generation unit 104 is output to the image projection apparatus 2 via the first output terminal 109, and the formed image data generated by the second generation unit 107 is output by the second output terminal 110. And output to the image forming apparatus 3.

<Configuration and Operation of Image Projecting Device 2>
The image projection apparatus 2 has a projection optical unit (not shown). The projection optical unit includes a lamp that is a light source, a liquid crystal driving device that drives a liquid crystal panel based on input projection image data, and a projection lens. The light from the lamp is decomposed into R, G, and B light by the optical system and guided to the liquid crystal panel. The light guided to each liquid crystal panel is modulated in luminance by each liquid crystal panel, and an image is projected onto a printed matter formed by the image forming apparatus 3 by a projection lens.

<Configuration and Operation of Image Forming Apparatus 3>
The image forming apparatus 3 records ink dots on the recording medium by moving a recording head (not shown) vertically and horizontally relative to the recording medium based on the formed image data generated by the image processing apparatus 1. , Form an image. In the present embodiment, the image forming apparatus 3 uses an ink jet printer, but other types of printers such as an electrophotographic method may be used.

<Processing content of image processing apparatus 1>
Next, processing contents of the image processing apparatus 1 having the above-described functional configuration will be described with reference to the flowchart of FIG. 2A. Hereinafter, each step (process) is represented by adding S before the reference numeral.

In S201, the acquisition unit 103 acquires input image data via the first input terminal 101. The input image data is 3-channel color image data in which 8-bit RGB values are recorded in each pixel. The image represented by the input image data has a higher resolution than the image projected by the image projection device 2. That is, the image projected by the image projection device 2 has a lower resolution than the image represented by the input image data. The input image data acquired by the input image data acquisition unit 103 is sent to the first generation unit 104, the calculation unit 106, and the second generation unit 107. Further, the projection state is acquired via the second input terminal. The projection state is an image determined from the distance (projection distance) between the image projection apparatus 2 and the formed image 501 that is the projection target when the image projection apparatus 2 projects the projection image 502 and the relationship between the projection lens and the liquid crystal panel. It is a horn. The projection distance D is acquired as 4-bit data converted into meters (m) and the angle of view θ is converted into radians. The projection state is acquired by receiving an input from the user via the UI screen illustrated in FIG. 19 or directly acquired from the image projection apparatus 2 by connecting the image projection apparatus 2 and the image processing apparatus 1.

In S202, the first generation unit 105 converts the resolution of the image represented by the input image data acquired by the acquisition unit 103 based on the number of pixels of the image output from the image projection device 2. A known bicubic method is used for resolution conversion, but other resolution conversion methods such as a bilinear method may be used. Further, based on input image data representing an image whose resolution has been converted, projection image data is generated by referring to a second color conversion LUT 105 held in advance. The first color conversion LUT 105 referred to is shown in FIG. As shown in FIG. 3, the correspondence between the signal value (RGB value) recorded for each pixel of the input image data and the signal value (RGB value) recorded for each pixel of the projection image data is maintained. The first color conversion LUT 105 described above is created in advance by projecting a chart with a known input signal value recorded in the projection image data and measuring the color of the projected image. The generated projection image data is three-channel color image data in which 8-bit RGB values are recorded in each pixel, as in the case of the input image data. When the projection image data is generated, it is sent to the image projection device 2 via the first output terminal 109. It is also sent to the calculation unit 106.

In S203, the calculation unit 106 acquires the input image data and the projection state acquired in S201, and the projection image data generated in S202. Further, based on the acquired projection state, an enlargement ratio E of the size of the image projected by the image projection apparatus 2 with respect to the size of the image represented by the input image data is calculated. Here, the enlargement ratio is a magnification for enlarging the image. Based on the calculated enlargement ratio E, the resolution of the image represented by the projection image data and the resolution of the image represented by the input image data are converted. After converting the resolution, the degree of reduction in the sharpness of the image represented by the projection image data with respect to the image represented by the input image data is calculated. The calculated degree of sharpness reduction is sent to the second generator. Hereinafter, the detailed processing of S203 will be described using the flowchart shown in FIG. 5A.

In step S2031, input image data, projection image data, and a projection state are acquired. In S2032, based on the acquired projection state (D and θ) and the projection state (projection distance D ₀ and angle of view θ ₀ ) in which the size of the image represented by the input image data and the size of the projection image 502 are equal. Then, the enlargement ratio E is calculated using the following equation (1).

E = D / D ₀ + θ / θ ₀ (1)

Note that the projection state (D ₀ and θ ₀ ) having the same magnification as described above is determined in advance by the following method and held in the calculation unit 106. Projecting projection image data generated based on input image data whose image size is known in advance, and searching for a projection state in which the image size of the projected image 502 projected matches the size of the image represented by the input image data Determined by. Note that a calculation formula may be constructed based on the characteristics of the projection lens and the liquid crystal panel included in the image projection apparatus 2, and the calculation formula may be used. Note that, in S2031, to acquire only the projection distance D as the projection state, at S2032, it may calculate the enlargement ratio E by dividing _{D 0} from the projection distance D.

In S2033, the resolution of the image represented by the input image data and the resolution of the image represented by the projection image data are converted based on the enlargement ratio E calculated in S2032 and the resolution _{Rf of} the image formed by the image forming apparatus 3. The resolution conversion of the image represented by the projection image data is performed by using the following formula (2) based on the enlargement ratio E, the resolution R _f , and the resolution R _p of the image projected by the image projection device 2. The scaling factor _Fp is calculated.

_Fp = _Rf / ( _Rp / E) ... Formula (2)

The calculated scaling factor F _p based, resolution-converting each pixel of the projection image data in F _p pieces. A known nearest neighbor method is used for this resolution conversion. FIG. 11A shows an example of resolution conversion of the image represented by the projection image data when the scaling factor F _p = 3 of the projection image data. Since the nearest neighbor method is used, the pixel value before conversion is duplicated and can be converted into projection image data representing an image simulating the projected image 502. For resolution conversion, other methods such as the bicubic method may be used instead of the nearest neighbor method.

Similarly, the resolution conversion of the image represented by the input image data is performed by first using the following formula (3) based on the enlargement ratio E, the resolution R _f , and the resolution R _in of the image represented by the input image data. to calculate the scaling factor _{F in} the.

F _in = R _f / (R _in / E) (3)

The calculated scaling ratio F _in the group, to resolution conversion of each pixel of the input image data to F _in pieces. A known bicubic method is used for the resolution conversion. FIG. 11B shows an example of resolution conversion of the image represented by the input image data when the scaling factor F _in = 1.5 of the input image data. Conversion to input image data for generating formed image data representing an image simulating the formed image 501 by using a bicubic method for converting resolution by performing interpolation processing based on the neighboring pixel values before conversion. Is possible. Note that the resolutions R _f and R _p described above are acquired by user input or directly by connecting the image projection apparatus 2 or the image forming apparatus 3 and the image processing apparatus 1. Both the resolutions R _f and R _p are preferably the highest resolution that each device can output.

In step S2034, the pixel value of the input image data representing the image whose resolution is converted in step S2033 and the pixel value of the projection image data are subtracted, and the difference obtained by the subtraction process is set as the sharpness reduction degree. The degree of reduction in sharpness is calculated as 3-channel color image data in which 8-bit RGB values are recorded, as in the case of input image data and projection image data. The subtraction process is performed independently on each of the R, G, and B channels for each pixel of each image. The pixel value of the R channel of the input image data is I_R _{x, y} (x is the pixel position in the horizontal direction and y is the pixel position in the vertical direction), and the pixel value of the R plane of the projection image data is P_R _{x, y} . Further, the pixel value of the R plane having the degree of reduction in sharpness is set as Q_R _{x, y,} and is calculated from the following equation (4).

Q_R _{x, y} = (I_R _{x, y} -P_R _{x, y} ) (4)

FIG. 12 schematically shows an example in which the degree of reduction in sharpness is calculated from input image data and projection image data. Since the G and B planes are the same process, description thereof is omitted. The above calculation is calculated for each pixel in the R channel, and then processed in the order of G and B. Since I_R _{x, y} and P_R _{x, y} are 8-bit data representing 0 to 255 _, Q_R _{x, y} output in the above calculation is 9-bit data representing -255 to +255. In addition, although the example which processes sequentially is shown above, it is not limited to said example. For example, the calculation for each channel may be processed in parallel. The calculated degree of reduction in sharpness (3-channel color image data in which 9-bit RGB values are recorded in each pixel) is sent to the second generation unit 107.

In S204, the second generation unit 107 enhances the sharpness of the image represented by the input image data whose resolution has been converted in S2033 based on the degree of reduction in sharpness calculated in S203. The sharpness enhancement is realized by adding the pixel value of the input image data subjected to resolution conversion and the pixel value of the image data representing the degree of reduction in sharpness. By reducing the sharpness of the formed image 501 that is superimposed on the projection image 502 by reducing the sharpness of the image projection device 2 by projecting the projection image 502, it is possible to suppress a decrease in the sharpness of the superimposed image. The addition process is performed independently on each of the R, G, and B channels for each pixel of each image, as in the subtraction process of S2034. The pixel value of the R channel of the input image data is I_R _{x, y} (x is the pixel position in the horizontal direction, y is the pixel position in the vertical direction), and the pixel value of the R channel of the degree of reduction in sharpness is Q_R _{x, y} . . Further, the pixel value of the R channel of the input image data after the addition processing is set as I_R ′ _{x, y} and is calculated from the following equation (5).

I_R′x _{, y} = I_Rx _{, y} + Q_Rx _{, y} Expression (5)

FIG. 13 schematically shows an example in which input image data representing an image with enhanced sharpness is calculated based on the input image data and image data representing the degree of sharpness reduction. If I_R _{x, y} is less than 0 as a result of the calculation, the input image data representing an image in which sharpness is emphasized is represented by 0 to 255 by clipping to 0 if it is 0, 256 or more. Bit data. Since the G and B channels are the same process, the description thereof is omitted. The above calculation is calculated for each pixel in the R channel and then sequentially processed with G and B, but may be parallel processing.

Further, based on the input image data representing the image with enhanced sharpness, the second image conversion LUT 108 held in advance is referred to generate the formed image data. The second color conversion LUT 108 referred to is shown in FIG. As shown in FIG. 7, the correspondence between the signal value (RGB value) recorded for each pixel of the input image data and the signal value (RGB value) recorded for each pixel of the formed image data is maintained. The second color conversion LUT 108 described above is created in advance by forming an image on a recording medium based on a chart with known input signal values recorded in the formed image data, and measuring the color of the formed image. deep. The formed image data to be generated is 3-channel color image data in which RGB values of 8 bits are recorded in each pixel, as in the case of input image data. The generated formed image data is sent to the image forming apparatus 3 via the second output terminal 110. Thus, a series of processes for generating the projection image data and the formed image data is completed.

By performing the processing control described above, it is possible to compensate for the reduction in the sharpness of the projection image 502 with respect to the image to be reproduced by enhancing the sharpness of the formed image 501. As a result, it is possible to suppress a reduction in sharpness in a superimposed image obtained by superimposing the projection image 502 projected by the projection device 2 on a printed matter (formed image 501) formed by the image forming device 3.

<Modification>
In this embodiment, the input image data and the projection image data are subjected to resolution conversion processing according to the projection state, and the image projection apparatus 2 projects the image to be reproduced based on the input image data and the projection image data. An example of calculating the degree of reduction in sharpness of an image to be displayed has been shown. However, the method for calculating the degree of reduction in sharpness is not limited to the above example. For example, the degree of reduction in the sharpness of the projected image 502 with respect to the image to be reproduced by arithmetic processing based on the resolution R _f of the image formed by the image forming apparatus 3 and the resolution R _{p of} the image projected by the image projecting apparatus 2 May be calculated. The detailed processing of S203 in the calculation processing of the sharpness reduction degree will be described with reference to the flowchart shown in FIG.

Since S2031 and S2032 are the same as those of the first embodiment described above, description thereof is omitted. In S2035, only the input image data is converted based on the enlargement ratio E calculated in S2032 and the resolution _{Rf of} the image formed by the image forming apparatus 3. Since the resolution conversion for the input image data is the same as that in the first embodiment, description thereof is omitted.

In S2036, similarly to S2033 described above, it calculates the scaling factor _{F p} using equation (2), to generate a high pass filter matrix size based on the magnification ratio _{F p} calculated, filters. FIG. 16 shows an example of a high-pass filter applied when the scaling factor F _p is 3, 5, or 9. As shown in FIG. 16, a high-pass filter having a matrix of F _p × F _p is generated, and the input image data subjected to resolution conversion in S2035 is subjected to filter processing, and the processing result is set as a degree of reduction in sharpness.

First, by converting the image represented by the input image data into the resolution of the image formed by the image forming apparatus 3, the sharpness of the image that can be reproduced by the image forming apparatus 3 is extracted from the sharpness of the image to be reproduced. Then, by performing a filtering process using a high-pass filter based on the scaling factor F _p to sharpness extracted above, and calculates the sharpness is lost when expressed by the image projection apparatus 2. By executing a series of processing, the sharpness that can be reproduced by the image forming apparatus 3 and cannot be reproduced by the image projection apparatus 2 among the sharpnesses of the image to be reproduced is calculated as the above-described reduction degree of the sharpness. be able to.

In the above-described modification, a high-pass filter is generated and filter processing is performed using the generated high-pass filter. However, a plurality of types of filters may be held in advance. In this case, a filter for filter processing is acquired from a plurality of types of filters stored in advance according to the above-described scaling factor and the frequency band in which the sharpness that can be calculated based on the scaling factor is reduced.

In the present embodiment, the configuration example using one image projection device 2 is shown, but the configuration using two or more image projection devices 2 may be used. For example, as shown in FIG. 10, n image projection apparatuses 2a to 2c and one image forming apparatus 3 are used, and n image projection apparatuses 2 project n projection images at the same position. A superimposed image expression system (stack projection) may be constructed. By using two or more image projection apparatuses 2, it is possible to superimpose two or more projection images on the formed image and further expand the luminance range that can be represented by the superimposed image to the higher luminance side.

Further, for example, as shown in FIG. 9, four image projection apparatuses 2a to 2d and one image forming apparatus 3 are used to divide the input image data into 2 × 2, and the divided input image data Projection image data generated based on the above is projected from the image projection apparatuses 2a to 2d. With the above configuration, a superimposed image expression system (multi-projection) that superimposes a 2 × 2 projection image on a formed image may be constructed. Each image projection device 2a to 2d reproduces each area of the image represented by the input image data, thereby superimposing a projection image having a higher resolution or a larger size than the projection image projected by one image projection device 2. It becomes possible to do.

In this embodiment, the resolution of the image represented by the projection image data is matched with the resolution of the formed image 501, and the degree of reduction in sharpness is calculated on the basis of the projection image data subjected to resolution conversion and the input image data. An example was given. However, the method of generating projection image data used for calculating the degree of reduction in sharpness is not limited to the above example. FIG. 17 shows a functional configuration of the image processing apparatus 1 for calculating the degree of reduction in sharpness in the projected image 502. As illustrated in FIG. 17, the image processing apparatus 1 includes an imaging device 6 that captures a projection image 502 and a third input terminal 112 that acquires data from the imaging device 6. Image data obtained by imaging the projection image 502 by the imaging device 6 may be used as projection image data used for calculating the degree of reduction in sharpness. As described above, by using the projection image data obtained by capturing the actual projection image 502, it is possible to cope with a change in the relationship between the projection image data and the projection image 502 due to the deterioration of the image projection apparatus 2 over time. It becomes possible.

In this embodiment, an example in which the projection distance is acquired as the projection state, the enlargement ratio E is calculated based on the projection distance, and the degree of reduction in sharpness is calculated has been described. However, the projection state is not limited to the above example. For example, a known trapezoidal distortion correction (keystone correction) is previously applied to the projected image 502 in order to suppress trapezoidalization of the projected image 502 due to an angle φ formed by the optical axis of the image projected from the image projection apparatus 2 and the formed image 501. ) May be applied. In the above case, it is necessary to calculate the degree of reduction in sharpness based on the projection image 502 that has been corrected for trapezoidal distortion by the angle φ formed. In the following, an example of generating projection image data in consideration of φ being a projection state will be shown.

First, the first generation unit 104 acquires the angle φ formed as the projection state. Furthermore, a known affine transformation parameter (trapezoidal distortion correction coefficient) for transforming input image data into a trapezoidal image is held for each φ, and the input image data is converted into a trapezoidal image using the affine transformation parameters corresponding to the obtained φ. Is converted into image data. The affine transformation parameter includes, for example, a horizontal movement amount, a vertical movement amount, a horizontal scaling factor, and a vertical scaling factor for each pixel position of the input image data in accordance with φ. A known bicubic method is used for resolution conversion when converting to trapezoidal image data. Thereafter, the image data representing the trapezoidal image is inversely transformed using the affine transformation parameters to return to the same rectangle as the input image data. When performing inverse transformation, a known nearest neighbor method is used. By using the nearest neighbor method that replicates neighboring pixels at the time of inverse conversion, it becomes possible to generate projection image data that assumes sharpness lost when converting to image data representing a trapezoidal image. . Furthermore, based on the reversely converted input image data, the first color conversion LUT 105 is referred to generate projection image data. Since the processing after generating the projection image data in consideration of φ being the projection state is the same, the description is omitted. Although the above describes the example of correcting the deformation of the input image data in accordance with the angle φ formed by the formed image 501 and the optical axis of the projected image 502, the present invention is not limited to the above example. For example, there is a case where distortion caused by an angle formed between the liquid crystal panel and the projection lens, or distortion of the projection lens caused when the refraction of the projection lens moves away from the ideal state from the central part to the peripheral part. In these cases, the affine transformation parameters corresponding to the respective corrections may be held, and the conversion process similar to the above-described trapezoidal distortion correction may be performed.

Also, the brightness of the image may be reduced due to a decrease in the amount of light generated in the peripheral portion as compared with the central portion of the projected image 502. For example, in order to correct the variation in the amount of light at each position of the projection image 502 described above, the input image data may be corrected in advance. In the above case, it is necessary to calculate the degree of reduction in sharpness based on input image data that has been subjected to correction processing. An example in which projection image data is generated in consideration of α and β which are projection states will be shown below.

First, the first generation unit 104 acquires a light amount reduction amount α and a coefficient β for adjusting the light amount of the entire image for each pixel position of the projection image 502 as a projection state. The obtained α _{x, y} and β _{x, y} and the R channel pixel value I_R _{x, y} (x is the pixel position in the horizontal direction, y is the pixel position in the vertical direction) of the input image data, and the following equation (6): Based on the above, the pixel value I_R ′ _{x, y} of the corrected input image data is calculated.

I_R ′ _{x, y} = (I_R _{x, y} + α _{x, y} ) × β _{x, y} (6)

The light amount reduction amount α for each pixel position of the projected image 502 and the coefficient β for adjusting the light amount of the entire image are obtained by projecting an image based on input image data whose signal value is known, and measuring the projected image. To determine in advance. Based on the corrected input image data, the first color conversion LUT 105 is referred to generate projection image data. Since the processing after generating the projection image data in consideration of the projection states α and β is the same, the description is omitted.

In this embodiment, the sharpness of the input image data is enhanced by adding the pixel value of the input image data and the pixel value of the image data representing the degree of reduction of the sharpness. The enhancement process is not limited to the above example. For example, a correction value corresponding to the value of the degree of sharpness reduction may be separately stored, and this correction value may be added to the pixel value of the input image data. Further, a gamma (γ) value corresponding to the degree of reduction in sharpness may be stored in advance, and the sharpness of input image data may be enhanced by a known γ correction process using the stored γ value. . Alternatively, a plurality of known edge enhancement filters with different enhancement levels for emphasizing fine portions of an image may be held, and the edge enhancement filters may be used properly according to the degree of sharpness reduction.

In this embodiment, the projection image data is generated based on the input image data, and then the formation image data is generated. However, the processing of this embodiment is not limited to the above example. For example, projection image data is generated based on input image data in advance and stored in the HDD 1412 or the like. Input image data and projection image data generated in advance are acquired, and formation image data is generated based on the input image data and the projection image data.

[Example 2]
In the first embodiment, the example in which the formed image data is generated by enhancing the sharpness of the image represented by the input image data based on the degree of reduction in the sharpness calculated from the projection image data and the input image data has been described. However, the luminance range that can be expressed by the formed image 501 changes according to the ambient light determined by the illumination 4. In other words, the formed image 501 is an image representing an image recorded by reflecting the irradiated light. Therefore, when the illumination light is small (dark), the luminance range that can be expressed by the formed image 501 is narrow, and conversely, when the illumination light is large (bright), the luminance range that can be expressed by the formed image 501 tends to be wide. is there. Therefore, in the present embodiment, as described above, the enhancement degree of the process for enhancing the sharpness is controlled in consideration of the luminance range of the formed image 501 that changes according to the ambient light. As a result, it is possible to reduce fluctuations in the effect of suppressing the reduction in sharpness due to ambient light. An example of realizing the above processing will be described mainly with respect to differences from the first embodiment.

<Functional Configuration of Image Processing Apparatus 1>
A functional configuration of the image processing apparatus 1 is shown in FIG. The second generation unit 107 acquires ambient light information via the fourth output terminal 111. The ambient light information is 4-bit data representing the intensity of the ambient light irradiated on the superimposed image. The ambient light information is directly acquired by input by the user or by connecting the illumination 4 and the image processing apparatus 1. The acquisition of ambient light information is not limited to the above example. For example, the light intensity information of a presumed superimposed image observation scene (outdoor clear sky, outdoor cloudy sky, indoor spotlight lighting, indoor office lighting, etc.) is held, Light intensity information may be acquired. Since the configuration other than the above is the same as that of the first embodiment, the description thereof is omitted.

<Processing content of image processing apparatus 1>
As processing contents in the second embodiment, S204 different from the first embodiment is described. In S204, the degree of reduction in sharpness calculated in S203 is corrected based on the ambient light information acquired from the fourth input terminal 111, and the sharpness of the image represented by the input image data is determined by the corrected degree of reduction in sharpness. Emphasize. Further, based on the input image data representing the image with enhanced sharpness, the formation image data is generated by referring to the second color conversion LUT 108 held in advance as in the first embodiment. The enhancement process is realized by adding a sharpness reduction component corrected with a correction coefficient corresponding to the ambient light information to the pixel value of the input image data subjected to resolution conversion. As described above, the formed image 501 is an image representing an image recorded by reflecting the irradiated light. Therefore, when the ambient light is small, the luminance range that can be expressed by the formed image 501 tends to be narrow, and conversely, when the ambient light is large, the luminance range that can be expressed by the formed image 501 tends to be wide. Therefore, when the ambient light is small compared to the case where there is a lot of ambient light, the projection image 502 cannot be fully expressed, and a correction that emphasizes the sharpness component expressed only by the formed image 501 is performed, thereby changing the ambient light. It is possible to reduce fluctuations in the effect of suppressing the reduction in sharpness. Detailed processing contents will be described below.

First, the correction coefficient Z is determined from the ambient light information with reference to the LUT 113 that holds the correspondence relationship between the ambient light information and the correction coefficient held in advance. An example of the LUT 113 is shown in FIG. The LUT 113 measures the luminance range that can be expressed by the formed image 501 for each observation environment in which ambient light is changed in advance, and determines the luminance range according to the ratio of the luminance range. Similar to S2033, the addition processing is performed independently on each of the R, G, and B channels for each pixel of each image. The pixel value of the R plane of the input image data is I_R _{x, y} (x is the pixel position in the horizontal direction, y is the pixel position in the vertical direction), the pixel value of the R plane of the degree of sharpness reduction is Q_R _{x, y} , and emphasis Let the pixel value of the R plane of the input image data after processing be I_R ′ _{x, y} . Further, the correction coefficient Z is used to calculate from the following equation (7).

I_R ′ _{x, y} = I_R _{x, y} + (Q_R _{x, y} ) × Z (7)

As described above, by controlling the enhancement degree of the enhancement processing on the input image data according to the ambient light, it is possible to reduce the fluctuation of the effect of suppressing the sharpness degradation due to the ambient light.

<Modification>
In this embodiment, an example in which the conversion LUT 113 that holds the correspondence between the ambient light information and the correction coefficient is shown, but the present invention is not limited to the above example. For example, a calculation formula for predicting the correspondence between the ambient light information and the correction coefficient may be constructed, and the correction coefficient may be calculated according to the ambient light information input using the above calculation formula.

In the present embodiment, an example is shown in which one piece of data is used as the ambient light information, but the present invention is not limited to the above example. For example, based on the ambient light information for each region of the formed image 501, two-dimensional ambient light information similar to the input image data is acquired, and enhancement processing according to the ambient light information is performed for each region (pixel) of the input image data. The degree of emphasis may be controlled.

[Example 3]
In the first embodiment, the example in which the formed image data is generated by enhancing the sharpness of the image represented by the input image data based on the degree of reduction in the sharpness calculated from the projection image data and the input image data has been described. As described above, the resolution R _p of the projected image, the resolution varies depending on the magnification of the image. Therefore, for example, when displaying by reducing the input image (if the enlargement ratio E is below 1) may resolution R _p of the projected image is higher than the resolution R _f of the formed image. Therefore, in this embodiment, the degree of reduction in sharpness occurring in the formed image is predicted based on the input image and the enlargement ratio E, and the sharpness of the projection image is emphasized based on the predicted degree of reduction in sharpness. An example of realizing the above processing will be described mainly with respect to differences from the first embodiment. In this embodiment, since the enlargement ratio E is less than 1, the enlargement ratio E is referred to as a reduction ratio S. Here, the reduction ratio is a magnification when the image is reduced.

<Functional Configuration of Image Processing Apparatus 1>
A functional configuration of the image processing apparatus 1 is shown in FIG. In the configuration of the image processing apparatus 1 in this embodiment, the image processing apparatus 1 is connected to the image projection apparatus 2 via the second output terminal 110, and the image processing apparatus 1 is connected to the image forming apparatus via the first output terminal 109. 3 is connected. The first generation unit 104 refers to the first color conversion LUT 105 based on the input image data, and generates image data (formed image data) to be input to the image forming apparatus 3. The calculation unit 106 acquires the input image data and the formed image data, and converts the resolution of the image represented by the input image data and the resolution of the image represented by the formed image data according to the projection state. Further, the degree of reduction in the sharpness of the image represented by the formed image data relative to the image represented by the input image data is calculated based on the input image data representing the image whose resolution has been converted and the formed image data. The second generation unit 107 enhances the sharpness of the image represented by the input image data based on the input image data and the degree of sharpness reduction. Further, based on input image data representing an image with enhanced sharpness, the second color conversion LUT 108 is referred to, and image data (projected image data) to be input to the image projection device 3 is generated. The formed image data generated by the first generation unit 104 is output to the image forming apparatus 3 via the first output terminal 109, and the projection image data generated by the second generation unit 107 is output by the second output terminal 110. Is output to the image projection apparatus 2.

<Processing content of image processing apparatus 1>
Next, processing contents of the image processing apparatus 1 having the above-described functional configuration will be described with reference to the flowchart of FIG. 2B. Since S201 is the same as that of the first embodiment, a description thereof will be omitted, and S202 ′, S203 ′, and S204 ′ will be described.

In S <b> 202 ′, the first generation unit 105 converts the resolution of the image represented by the input image data acquired by the acquisition unit 103 based on the number of pixels of the image output from the image forming apparatus 3. A known bicubic method is used for resolution conversion, but other resolution conversion methods such as a bilinear method may be used. Further, based on the input image data representing the image whose resolution has been converted, the first color conversion LUT 105 that is held in advance is referred to, and formed image data is generated. The first color conversion LUT 105 referred to is the same as the second color conversion LUT 108 in the first embodiment, and a description thereof will be omitted. In S203 ', the calculation unit 106 acquires the input image data and the projection state acquired in S201, and the formation image data generated in S202'. Furthermore, based on the acquired projection state, a reduction ratio S of the size of the image projected by the image projection apparatus 2 with respect to the size of the image represented by the input image data is calculated. Based on the calculated reduction ratio S, the resolution of the image represented by the formed image data and the resolution of the image represented by the input image data are converted. After converting the resolution, the degree of reduction in the sharpness of the image represented by the projection image data with respect to the image represented by the input image data is calculated. The calculated degree of sharpness reduction is sent to the second generator 107. In S <b> 204 ′, the second generation unit 107 enhances the sharpness of the image represented by the input image data whose resolution is converted in S <b> 2033 ′, based on the degree of sharpness reduction calculated in S <b> 203 ′. Sharpness enhancement is performed by addition processing as in the first embodiment. Further, based on the input image data representing the image with enhanced sharpness, the second color conversion LUT 108 held in advance is referred to generate projection image data. Since the second color conversion LUT 108 referred to is the same as the first color conversion LUT 105 in the first embodiment, the description thereof is omitted.

Hereinafter, the detailed processing of S203 ′ will be described using the flowchart shown in FIG. 5B. In S2031 ′, the calculation unit 106 acquires input image data, projection image data, and a projection state, as in the first embodiment. In S2032 ′, the calculation unit 106 obtains the obtained projection state (D and θ) and the projection state in which the size of the image represented by the input image data and the size of the projection image are equal (projection distance D ₀ and angle of view θ). ₀ )), the reduction ratio S is calculated using the following equation (8).
S = D / D ₀ + θ / θ ₀ (8)

'In the calculation unit 106, S2032' S2033 and the reduction ratio S calculated at, on the basis of the resolution R _p of the image projected by the image projection apparatus 2, represents the resolution and the formed image data of an image represented by the input image data is Convert image resolution. The resolution conversion of the image represented by the formed image data is performed by using the following equation (9) based on the reduction ratio S, the resolution R _p , and the resolution R _f of the image formed by the image forming apparatus 3, and the scaling ratio of the formed image data F _f is calculated.
F _f = (R _p / S) / R _f Formula (9)

Based on the calculated scaling factor F _f , the resolution of each pixel of the formed image data is converted to F _f . A known nearest neighbor method is used for this resolution conversion. For resolution conversion, other methods such as the bicubic method may be used instead of the nearest neighbor method. Similarly, the resolution conversion of the image represented by the input image data is performed by first using the following expression (10) based on the reduction ratio S and the resolution R _p and the resolution R _in of the image represented by the input image data. to calculate the scaling factor _{F in} the.
F _in = (R _in / E) / R _p (10)

The calculated scaling ratio F _in the group, to resolution conversion of each pixel of the input image data to F _in pieces. A known bicubic method is used for the resolution conversion. Note that the resolutions R _f and R _p described above are acquired by user input or directly by connecting the image projection apparatus 2 or the image forming apparatus 3 and the image processing apparatus 1. Both the resolutions R _f and R _p are preferably the highest resolution that each device can output.

In S2034 ′, the calculation unit 106 subtracts the pixel value of the input image data representing the image whose resolution has been converted in S2033 ′ and the pixel value of the formed image data, and reduces the difference obtained by the subtraction process to reduce the sharpness. The degree. The method for calculating the degree of reduction in sharpness and generating an image in which the sharpness of the input image is emphasized is the same as that in the first embodiment, and a description thereof will be omitted.

By performing the processing control described above, it is possible to compensate for the reduction in the sharpness of the formed image with respect to the image to be reproduced (decrease in the contrast of the fine portions) by enhancing the sharpness of the projected image. As a result, it is possible to suppress a reduction in sharpness in a superimposed image obtained by superimposing a projection image projected by the image projection device 2 on a formed image formed by the image forming device 3.

<Modification>
In the present embodiment, an example in which a reduction in sharpness generated in a superimposed image of a projection image by the image projection device 2 and a formation image by the image forming device 3 has been described. However, the combination of apparatuses for generating a superimposed image is not limited to the above example. When a superimposed image is generated by superimposing images output by two image output devices having different expressible resolutions, the sharpness reduction generated in the output image of the device having a low expressible resolution is higher. Any combination may be used as long as it can be compensated by the output image of the apparatus. For example, as shown in FIG. 21, a plurality of image projecting apparatuses having different expressible resolutions are used, and a first image projecting apparatus having a lower resolution is obtained by a projection image of the second image projecting apparatus 2b having a higher expressible resolution. You may supplement the fall of the sharpness of the projection image by 2a.

In the above-described embodiments, an example in which an image projecting apparatus that projects an image and an image forming apparatus that forms an image on a recording medium is used as an apparatus that outputs an image. The above-described processing can be applied to other image output apparatuses. The device combination may be a combination of two or more image output devices capable of generating a superimposed image. For example, an image display device such as a liquid crystal display or an organic EL display may be used. As an example of generating a superimposed image using an image display device, an image is formed on a recording medium that transmits light, such as an OHP sheet, by the image forming device, and the OHP sheet on which the image is formed is placed on the image display device. Put it on. As described above, the above-described processing can be applied to image superposition when one is a formed image formed by the image forming apparatus and the other is a display image displayed by the image display apparatus.

[Example 4]
In the embodiment described above, the sharpness reduction that occurs in the output image of the image output device whose resolution that can be expressed is reduced according to the image size of the superimposed image can be reduced by the output image of the image output device that can express higher resolution. An example to supplement was shown. However, the reduction of the sharpness of the superimposed image is not limited to the reduction of the sharpness according to the size of the superimposed image. The reduction in image sharpness also occurs according to the output characteristics of the image output apparatus. For example, an image formed by an image forming apparatus is caused by a deviation in the landing position of a color material (ink), bleeding (mechanical dot gain) when the color material is fixed on a recording medium, optical blur (optical dot gain), and the like. It is known that the sharpness is lower than that of the input image. In this embodiment, an example will be described in which, in addition to a reduction in sharpness corresponding to the size of the superimposed image, a reduction in sharpness corresponding to the output characteristics of the image output apparatus is also suppressed. In the present embodiment, the output characteristics (characteristics of the sharpness of the formed image) of the image forming apparatus 3 are measured in advance, and a filter (hereinafter referred to as a compensation filter) that is opposite to the measured characteristics in frequency space is created. Keep it. A filter created in advance is convolved with the input image data. By executing the processing in the third embodiment based on the input image data subjected to the convolution processing, in addition to the reduction in the sharpness according to the size of the superimposed image, the sharpness according to the output characteristics of the image output device The decrease is also suppressed. An example of realizing the above processing will be described mainly with respect to differences from the third embodiment.

<Functional Configuration and Processing Contents of Image Processing Apparatus 1>
FIG. 22 shows a functional configuration of the image processing apparatus 1 according to the fourth embodiment. In addition to the configuration of the third embodiment, a compensation filter 113 having a reverse characteristic with respect to the output characteristic of the image forming apparatus 3 is provided in advance. Since the processing flow is the same as that of the third embodiment except for S202 ′, the description is omitted, and S202 ′ different from that of the third embodiment will be described. In S202 ′, the first generation unit 105 converts the resolution of the image represented by the input image data acquired by the acquisition unit 103 based on the number of pixels of the image output from the image forming apparatus 3. Furthermore, a convolution process is performed on the input image data whose resolution has been converted, according to the output characteristics of the image forming apparatus 3 provided in advance. A method for creating a compensation filter created in advance will be described below.

<Compensation filter creation method>
The compensation filter is created by printing a chart including a plurality of sine wave pattern images and uniform pattern images having different frequencies as shown in FIG. 23 on a recording medium and measuring the printed chart. Details of the method for creating the compensation filter will be described below.

First, the reflectance distribution of the output chart is acquired using a known image acquisition device (scanner, camera, microscope, etc.). Using the acquired reflectance distribution, a frequency response value fi (u) that is an output characteristic of the image forming apparatus 3 is calculated by the following equation (11). u is the frequency of the sine wave, and Max (u) and Min (u) are the maximum reflectance and the minimum reflectance of the image that change according to the frequency u, respectively. Furthermore, in Expression (11), White and Black are the reflectances of the uniform pattern, respectively.
fi (u) = MTF (u) = C (u) / C ′ (11)
C (u) = (Max (u) −Min (u)) / (Max (u) + Min (u))
C ′ = (White-Black) / (White + Black)

Next, the frequency characteristic Rx of the compensation filter is calculated using the acquired frequency response value fi (u) and the following equation (12).
Rx (u) = 1 / fi (u) (12)

A known inverse Fourier transform is performed on the above Rx, and a filter calculated by the inverse Fourier transform is used as a compensation filter. Note that when the above compensation filter is used to compensate up to a high-frequency component, noise and brightness fluctuations occur. Therefore, it is desirable that the compensation intensity (enhancement degree) is lower than less than 4 Cycle / mm when the sensitivity is 4 Cycle / mm or more, which has a low sensitivity in the known visual characteristics.

By performing the processing control described above, it is possible to generate input image data to compensate for the reduction in sharpness that occurs according to the output characteristics when forming a formed image. As a result, in addition to a reduction in sharpness according to the size of the superimposed image, a reduction in sharpness according to the output characteristics of the image output device can be suppressed.

<Modification>
In this embodiment, a sharpness reduction due to the output characteristics of the image formed by the image forming apparatus 3 (color material landing position deviation, bleeding, optical blur, etc.) is predicted in advance, and the input image is enhanced. An example is shown. However, the apparatus in which the sharpness reduction according to the output characteristics occurs is not limited to the image forming apparatus 3. Also in the image projector 2, the sharpness is lowered by the optical blurring of the projection lens. Also in an image display device such as a display, an optical blur is generated by the liquid crystal panel, and the sharpness is lowered. The processing of the present embodiment can also be applied to sharpness reduction according to the output characteristics of these image output devices. In the present embodiment, an example in which compensation according to output characteristics is applied to any one of a plurality of devices is shown. However, compensation processing according to the output characteristics of each device used for generating a superimposed image is performed. A configuration to perform is desirable.

In the present embodiment, an example is shown in which one filter having characteristics opposite to the output characteristics of the image formed by the image forming apparatus 3 is held in advance and the sharpness of the input image is enhanced. However, the output characteristics described above vary depending on image printing conditions (recording medium, ink type, number of passes, carriage speed, scanning direction, halftone processing). Therefore, it is desirable to hold a plurality of inverse characteristic filters corresponding to the above printing conditions and perform the switching process according to the printing conditions. In addition, without having a plurality of inverse characteristic filters, one inverse characteristic filter and a filter correction coefficient for each printing condition are provided, and a plurality of inverse characteristic filters are generated by switching the filter correction coefficient according to the printing conditions. May be.

In this embodiment, the sharpness enhancement process is performed on the input image data using the inverse characteristic filter. However, the image processing procedure including the inverse characteristic filter is not limited to the above example. For example, after generating output image data to be output to each image output device by the processing of the first embodiment, an inverse characteristic filtering process based on the output characteristics of each image output device may be performed.

In this embodiment, the example in which the output characteristic is calculated using the equation (11) has been described. However, the output characteristic calculation method is not limited to the above example. In the expression (11), when the average brightness of the output image changes according to the frequency u of the sine wave, the response value in the dark part is excessive with respect to the bright part. Therefore, when the average brightness of the output image changes, the following formula (13) is used.
fi (u) = MTF (u) = (Max (u) −Min (u)) / (White-Black) (13)

Note that Max (u), Min (u), White, and Black are described as reflectance, but luminance, density, and RGB values of the device may be used. Further, the chart for acquiring the output characteristics of the output image is not limited to the example of FIG. A rectangular wave pattern may be used instead of the sine wave pattern as long as the responsiveness for each frequency can be calculated. At this time, the CTF value calculated by applying Expression (11) to the rectangular wave pattern is used as the frequency characteristic fi (u). Alternatively, the CTF value may be converted into the MTF value using a known Coltman correction equation without using the frequency characteristic.

In this embodiment, an example in which an inverse characteristic filter is generated and held in advance has been described. However, an input unit is provided for the user to input the reflectance distribution of the chart, and the inverse characteristic is determined according to the input reflectance distribution. A filter may be generated.

The present invention is not limited to the above embodiment, and various changes and modifications can be made without departing from the spirit and scope of the present invention. Therefore, in order to make the scope of the present invention public, the following claims are attached.

This application claims priority based on Japanese Patent Application No. 2016-229696 filed on Nov. 28, 2016 and Japanese Patent Application No. 2017-161799 filed on Aug. 25, 2017. All the descriptions are incorporated herein.

Claims

A first output image output from the first image output device based on the input image; a second output image output from the second image output device based on the input image and having a higher resolution than the first output image; , For generating one image by generating image data to be output to the second image output device,
First acquisition means for acquiring input image data representing the input image;
Second acquisition means for acquiring first output image data generated based on the input image data and output to the first image output device;
First generation means for generating second output image data to be output to the second image output device based on the input image data and the first output image data;
The image processing apparatus characterized in that the sharpness of the image represented by the second output image data depends on the sharpness of the image represented by the first output image data.
2. The image processing apparatus according to claim 1, wherein at least one of the first image output apparatus and the second image output apparatus is a projector.
3. The image processing apparatus according to claim 2, wherein the first image output device is a projector, and the second image output device is a printer.
3. The image processing apparatus according to claim 2, wherein both the first image output apparatus and the second image output apparatus are projectors.
2. The image processing apparatus according to claim 1, wherein one of the first image output apparatus and the second image output apparatus is a printer and the other is a display.
A second generating unit configured to generate the first output image data based on the input image data;
The image processing apparatus according to claim 1, wherein the second acquisition unit acquires the first output image data generated by the second generation unit.
The second generation means converts the resolution of the input image data according to the number of pixels of the image output by the first image output device, and the first output is based on the input image data whose resolution has been converted. The image processing apparatus according to claim 6, wherein the image data is generated.
There is further provided calculation means for calculating a degree of reduction in the sharpness of the image represented by the first output image data with respect to the sharpness of the image represented by the input image data based on the input image data and the first output image data. And
The said 1st production | generation means produces | generates the said 2nd output image data based on the fall degree of the said sharpness calculated by the said calculation means, The any one of Claim 1 thru | or 7 characterized by the above-mentioned. An image processing apparatus according to 1.
A third obtaining unit for obtaining a first magnification of the size of the first output image with respect to the size of the image represented by the input image data;
The calculating means calculates a second magnification for converting the resolution of the input image data and the first output image data based on the first magnification, and the input image data based on the second magnification. And the first output image data are converted in resolution, and the reduction degree of the sharpness is calculated based on the input image data and the first output image data subjected to the resolution conversion. The image processing apparatus according to 8.
The calculation unit calculates a difference between a pixel value included in the input image data and a pixel value included in the first output image data as the reduction degree of the sharpness. An image processing apparatus according to 1.
The said calculating means calculates the fall degree of the said sharpness by applying the high-pass filter according to the said 2nd magnification with respect to the said input image data. Image processing apparatus.
The first generation means emphasizes the sharpness of an image represented by the input image data based on the input image data and the degree of reduction of the sharpness, and the input image data representing an image with enhanced sharpness. The image processing apparatus according to claim 8, wherein the second output image data is generated based on the image data.
The first generation means enhances the sharpness of an image represented by the input image data by adding a value representing the degree of reduction in the sharpness to a pixel value of the input image data. The image processing apparatus according to claim 12.
The first generation means enhances the sharpness of an image represented by the input image data by performing γ correction on the input image data using a γ value corresponding to the degree of reduction in the sharpness. The image processing apparatus according to claim 12.
The first generation means performs edge enhancement on the input image data using a filter according to the degree of reduction in the sharpness, thereby enhancing the sharpness of the image represented by the input image data. The image processing apparatus according to claim 12, characterized in that:
A fourth acquisition means for acquiring the intensity of ambient light in an environment in which the image is superimposed;
The image processing apparatus according to claim 1, wherein the first generation unit further generates the second output image data based on the intensity of the ambient light.
A fifth acquisition means for acquiring an output characteristic when outputting an image of at least one of the first image output device and the second image output device;
The image processing apparatus according to claim 1, wherein the first acquisition unit acquires the input image data corrected using a filter corresponding to the output characteristic. .
A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 17.
A first output image output from the first image output device based on the input image; a second output image output from the second image output device based on the input image and having a higher resolution than the first output image; Are image processing methods for generating image data to be output to the second image output device in order to generate one image by superimposing
A first acquisition step of acquiring input image data representing the input image;
A second acquisition step of acquiring first output image data generated based on the input image data and output to the first image output device;
A first generation step of generating second output image data to be output to the second image output device based on the input image data and the first output image data;
An image processing method, wherein the sharpness of an image represented by the second output image data is in accordance with the sharpness of an image represented by the first output image data.