JP2022091477A - Image projection device, method for controlling image projection device, and program - Google Patents

Abstract

To provide an image projection device that can reduce blurring of fine lines in a chart while continuing movement control of a projection position.SOLUTION: An image projection device comprises: a light source; an acquisition unit that acquires a source image; a creation unit that creates, from images of frames F1, F2, F3 of the source image, images of a plurality of sub-frames Sf1-1, Sf1-2, Sf2-1, Sf2-2, Sf3-1, Sf3-2; a chart composition unit that composes the images of predetermined sub-frames Sf1-1, Sf2-1, Sf3-1 among the images of the plurality of sub-frames with a chart image; a light modulation unit that has pixels arranged in matrix, and sequentially displays the images of the plurality of sub-frames including the images of the sub-frames with which the chart image is composed to modulate light from the light source; and a shift unit that shifts a projection position of the modulated light for each of the images of the sub-frames.SELECTED DRAWING: Figure 4

Description

The present invention relates to an image projection device, a control method for the image projection device, and a program.

Conventionally, an image projection device that performs pixel shift (pixel shift), which is a technique for increasing the resolution of a displayed image by moving the position of the projected image in time, has been known. In such an image projection device, an image having a resolution exceeding the resolution of the display panel can be projected and displayed by projecting an image in a time-sequential manner at a position where the pixel positions on the projected surface are displaced by approximately half a pixel.

Further, there has been proposed an image projection device that changes whether or not the position of a projected image can be moved in time according to conditions. For example, Patent Document 1 discloses an image projection device that controls the position (projection position) of a projected image so as not to move when the resolution of the image to be displayed is equal to or lower than the resolution of the light modulation element. It is said that the image projection device controls to move the projection position when the resolution of the image to be displayed is higher than the resolution of the light modulation element. Further, in Patent Document 2, when the degree of movement of the image is equal to or less than the threshold value, the position of the projected image is controlled to be moved, and when the degree of movement of the image is larger than the threshold value, the projection position is not moved. An image projection device to be controlled is disclosed.

Patent No. 5569084 Japanese Unexamined Patent Publication No. 2017-225032

The time-sequential drive as described above generates images of a plurality of subframes from the image to be displayed (input image), switches the images of these subframes in time-sequential manner, and sets the projection position of the image on the projected surface. This is done by shifting the image by approximately half a pixel. In the image of each subframe, the image is subjected to resolution conversion processing in order to represent the input image with a display element having fewer pixels. However, due to this resolution conversion process and the movement of the projection position for each subframe, a phenomenon in which fine lines are blurred in the projected image on the projected surface may occur.

Here, in the image projection device, an image including a chart is projected, and focus adjustment, projection position adjustment, and the like are often performed while checking the appearance of the chart. If the fine lines of the chart are blurred in such a projected image, it will be difficult to make accurate adjustments. For example, in focus adjustment, if the fine lines of the chart are blurred regardless of the focus position, it becomes difficult to find the best focus position.

For example, in the technique disclosed in Patent Document 1, the presence / absence of movement control of the projection position is switched according to the resolution of the image to be displayed. In this method, if the chart is displayed while the high-resolution image to be displayed is displayed, the movement of the projection position is continued and the fine lines of the chart are blurred. Further, if the chart is switched to a low resolution chart at the time of adjustment, switching between the presence / absence of movement control of the projection position occurs when straddling the display of an image having a resolution higher than the resolution of the display panel and an image having a resolution lower than the resolution of the display panel. In switching the presence / absence of movement control of the projection position, since the display element and the moving element are moved in synchronization, it takes time for the synchronization operation to stabilize, and the operability is impaired. In particular, there are many cases where the display and non-display of the chart are frequently switched when adjusting the projected image, and if the user has to wait for the operation to stabilize each time, the operability is significantly impaired.

Further, in the technique disclosed in Patent Document 2, the presence or absence of movement of the projection position is switched according to the degree of movement of the image. In this method, when the chart is displayed on the moving image to be displayed, the movement control of the projection position is continued, and the fine lines of the chart are blurred. Further, when switching between the display of the moving image and the chart covering the entire screen, the presence / absence of the movement control of the projection position is switched. As described above, when the presence / absence of the movement control of the projection position is switched, it takes time for stable operation, so that the operability is impaired.

Therefore, one embodiment of the present invention aims to alleviate at least a part of such a problem. In one embodiment of the present invention, there is provided an image projection device, a control method for the image projection device, and a program capable of reducing blurring of fine lines in a chart while continuing control of movement of the projection position.

The image projection device according to one embodiment of the present invention includes a light source, an acquisition unit for acquiring a source image, a generation unit for generating a plurality of subframe images from an image of each frame of the source image, and the plurality of units. A chart compositing unit that synthesizes a chart image with a predetermined subframe image among the subframe images, and the plurality of subframes including a subframe image in which pixels are arranged in a matrix and the chart image is synthesized. It is provided with an optical modulation unit that sequentially displays the images of the above and modulates the light from the light source, and a shift unit that shifts the projection position of the modulated light for each image of the subframe.

According to one embodiment of the present invention, blurring of fine lines in the chart can be reduced while the movement control of the projection position is continued.

It is a functional block diagram of the image projection apparatus which concerns on Example 1. FIG. It is a figure for demonstrating the pixel shift projection. It is a flowchart which shows the flow of the projection processing which concerns on Example 1. The timing of signal processing and various images according to the first embodiment are shown. The timing of signal processing and various images according to the second embodiment are shown. It is a functional block diagram of the image projection apparatus which concerns on Example 3. FIG. It is a flowchart which shows the flow of the projection processing which concerns on Example 3. FIG. The timing of signal processing and various images according to the third embodiment are shown. It is a figure for demonstrating the behavior of the image projection apparatus which concerns on Example 4. FIG.

Hereinafter, exemplary examples for carrying out the present invention will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, relative positions of the components, etc. described in the following examples are arbitrary and can be changed according to the configuration of the device to which the present invention is applied or various conditions. Also, in the drawings, the same reference numerals are used between the drawings to indicate elements that are the same or functionally similar.

(Example 1)
Hereinafter, with reference to FIGS. 1 to 4, as the image projection device according to the first embodiment of the present invention, an image projection that obtains a display resolution equal to or higher than the resolution of the display panel by a mechanism for moving the position of the projected image in time. The device will be described. FIG. 1 is a block diagram showing a configuration of an image projection device 100 according to this embodiment. The image projection device 100 includes a light source 101, a panel 102, a shift element 103, a projection lens 104, an acquisition unit 105, an image processing unit 106, a subframe generation unit 107, a chart composition unit 108, a drive unit 109, and a control unit 110. It is provided.

The light source 101 emits light. As a typical light source, a laser diode, an LED (LIGHT Emitting Diode), a high-pressure mercury lamp, or the like is used. The panel 102 functions as an example of the optical modulation unit. A plurality of pixels are arranged in a matrix on the panel 102, and the light emitted from the light source 101 passing through the panel 102 can be modulated by controlling the transmittance and the reflectance of each pixel.

The shift element 103 has a glass plate whose inclination can be controlled, and by controlling the inclination angle of the glass plate, the optical axis of the light transmitted through the glass plate can be shifted. The tilt angle of the glass plate may be controlled by using an arbitrary actuator such as an electromagnetic coil. The shift element 103 is arranged between the panel 102 and the projection lens 104, and the projected image formed on the projected surface via the projection lens 104 by shifting the optical axis of the image light modulated by the panel 102. The position of can be moved by about half a pixel. Therefore, the shift element 103 can move the position of the image projected on the projected surface to a plurality of positions by shifting the optical axis of the image light.

Here, the pixel shift projection using the shift element 103 will be described with reference to FIG. FIG. 2 is a diagram for explaining pixel shift projection. In this embodiment, as pixel shift projection, the operation of pixel shift projection called 2-way shift, in which two subframes are generated from one frame and the images of the two subframes are shifted to two different positions and projected, will be described. do. FIG. 2 shows an example of the projected image 201 of the first subframe projected at the first projection position and the projected image 202 of the second subframe projected at the second projection position. Here, the first projection position and the second projection position are diagonally offset by half a pixel from each other, and the movement of the projection position between the first projection position and the second projection position is the image light using the shift element 103. It is controlled by shifting the optical axis of.

In the 2-way shift operation, first, two subframe images are generated based on the image of one frame to be displayed. As a method for generating an image of two subframes based on an image of one frame, any known generation method related to the pixel shift operation may be used.

Next, the panel 102 sequentially displays the images of these two subframes in one frame, and operates the shift element 103 at the timing when the images of the subframes are switched to set the projection position for each pixel in the vertical and horizontal directions. Shift by half a pixel. Here, the pixel P1 of the projection image 201 at the first projection position and the pixel P2 of the projection image 202 at the second projection position are positioned diagonally half a pixel apart from each other on the projected surface, but on the panel 102. Then corresponds to the same pixel. By this operation, the projected image is visually recognized by overlapping the projected images 201 and 202 of each subframe as shown in FIG. 2, and the spatial number of pixels of the image on the projected surface is effectively calculated from the number of pixels of the panel. Can be increased.

Returning to FIG. 1 again, the image projection device 100 will be described. The acquisition unit 105 acquires a source image from the outside of the device or the inside of the device. Here, the source image is, for example, an input image acquired from an external device such as a personal computer (PC: Personal Computer), a disk player, or a streaming device, or an initial image stored in a storage unit (not shown) inside the device. May include. The initial image may be arbitrarily set according to a desired configuration, and may be, for example, a monochromatic image or the like.

The image processing unit 106 performs predetermined image processing on the acquired source image. The image processing unit 106 can adjust, for example, brightness, contrast, and the like. The image processing performed by the image processing unit 106 may include any known image processing depending on the desired configuration. For example, the image processing unit 106 may perform image scaling processing on the source image according to the driving method of the panel 102 or the shift element 103. For example, if the resolution of the source image is not four times the resolution of the display panel, the image processing unit 106 scales up or downs the source image to obtain the resolution of the source image four times the resolution of the display panel. Adjust to. As the method for scaling the image, any known method may be used. Further, in this embodiment, in order to perform the 2-way shift operation, the resolution of the source image is adjusted to 4 times the resolution of the display panel, but the magnification is changed according to the driving method of the panel 102 and the shift element 103. May be done.

The subframe generation unit 107 generates a plurality of subframe images from the image of each frame of the image processed by the image processing unit 106. Here, the subframe generation unit 107 generates an image of the subframe according to the number of projection positions moved by using the shift element 103. Therefore, in this embodiment, the subframe generation unit 107 generates two subframe images corresponding to the number of projection positions from the image of each frame of the image processed by the image processing unit 106. The subframe generation unit 107 performs resolution conversion processing so as to generate a subframe image having a panel resolution or less from a high resolution image when generating a plurality of subframe images from the image of each frame. As a method for generating an image of two subframes based on a source image of one frame related to the resolution conversion process, any known generation method related to the pixel shift operation may be used.

The chart synthesizing unit 108 superimposes and synthesizes a chart image showing a predetermined chart on a predetermined subframe image among the subframe images generated by the subframe generation unit 107. In this embodiment, the chart synthesis unit 108 is located after the subframe generation unit 107. Therefore, the chart synthesizing unit 108 synthesizes a chart image that is not affected by the resolution conversion process only with the image of a predetermined subframe, for example, the projected image 201 of the first subframe corresponding to the first projection position in FIG. can do.

The drive unit 109 drives the panel 102 based on the image of the subframe generated by the subframe generation unit 107 and the image of the subframe in which the chart image is combined by the chart composition unit 108. More specifically, the driving unit 109 sequentially displays the image of the subframe in which the chart image is synthesized by the chart synthesizing unit 108 and the image of the subframe other than the image of the subframe on the panel 102. The order in which the subframe images are displayed may be arbitrarily set according to the desired configuration. However, it is assumed that the subframe image and the projection position correspond to each other.

The control unit 110 controls the image processing unit 106, the subframe generation unit 107, the chart composition unit 108, the drive unit 109, the shift element 103, and the like. For example, the control unit 110 can control the timing of synthesizing the chart image by the chart synthesizing unit 108, the timing of driving the panel 102 by the driving unit 109, the timing of driving the shift element 103, and the like. In particular, since the control unit 110 shifts the projection position of the projected image for each subframe image, the drive timing of the panel 102 and the drive timing of the shift element 103 can be synchronized. Further, the control unit 110 can also control the position of the region for synthesizing the chart image by, for example, the chart synthesizing unit 108. Further, the control unit 110 may control the type of the chart image according to the instruction regarding various adjustment processes such as the focus adjustment process by the user, and the chart composition unit 108 synthesizes the chart according to the control by the control unit 110. The image may be determined.

The acquisition unit 105, the image processing unit 106, the subframe generation unit 107, the chart composition unit 108, the drive unit 109, and the control unit 110 execute a software module in which any processor is stored in a storage unit (not shown). It may be realized by. Here, the processor may be a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Further, the processor may be, for example, a GPU (Graphic Processing Unit), an FPGA (Field-Programmable Gate Array), or the like. Further, these components may be configured by a circuit or the like that performs a specific function such as an ASIC (Application Specific Integrated Circuit).

Here, a 2-way shift operation and a chart composition process, which are pixel shift operations using the image projection device 100 having the configuration according to the present embodiment, will be described. In the image projection apparatus 100 according to the present embodiment, an example in which an input image having a resolution four times the resolution of the panel 102 is processed as a source image will be described.

The subframe generation unit 107 is based on each frame of the image processed by the image processing unit 106, and is a subframe image corresponding to the first projection position related to the 2-way shift operation and a subframe corresponding to the second projection position. Generate each image of. Here, for example, the image of the subframe corresponding to the first projection position corresponds to the projection image 201, and the image of the subframe corresponding to the second projection position corresponds to the projection image 202. In this embodiment, assuming that the resolution of the panel 102 is 1920 × 1080, an image having a resolution of 3840 × 2160 and a frame rate of 60 Hz is input from the image processing unit 106 to the subframe generation unit 107. In this case, the subframe generation unit 107 selects the first subframe image and the second subframe image having a resolution of 1920 × 1080 from the image of one frame of the image having a resolution of 3840 × 2160 and a frame rate of 60 Hz. Generate. The subframe generation unit 107 generates a signal having a resolution of 1920 × 1080 and a frame rate of 120 Hz in chronological order.

The drive unit 109 modulates the light from the light source 101 passing through the panel 102 by displaying the images of the first and second subframes generated by the subframe generation unit 107 on the panel 102 in chronological order. Further, the control unit 110 is driven so as to project the image of the first subframe to the first projection position on the projected surface and the image of the second subframe to the second projection position on the projected surface. The drive of the panel 102 by the unit 109 and the drive of the shift element 103 are synchronized. As a result, the projected images of the subframes are overlapped and visually recognized in a state of being displaced by approximately half a pixel, and an image having a resolution exceeding the resolution of the panel 102 can be displayed on the projected surface.

Here, as described above, the phenomenon of fine lines blurring on the projected surface may occur due to the resolution conversion process when generating the subframe image and the movement of the projection position for each subframe image. Therefore, when the focus adjustment or the projection position adjustment using the chart is performed while performing such a pixel shift operation, the fine lines of the chart may be blurred and it may be difficult to perform appropriate adjustment.

Therefore, in this embodiment, the chart synthesizing unit 108 synthesizes the chart image only with the image of a predetermined subframe among the images of a plurality of subframes. As a result, the chart image is combined with the subframe image after the resolution conversion process is performed, so that the influence of blurring due to the resolution conversion process on the chart can be eliminated. Further, since the chart image is combined only with the image of a predetermined subframe, the projected image including the chart is projected only at one projection position. Therefore, it is possible to eliminate the influence of blurring due to the movement of the projection position with respect to the chart. Therefore, in the image projection apparatus according to the present embodiment, it is possible to reduce the blurring of the fine lines of the chart while continuing the movement control of the projection position.

Next, with reference to FIG. 3, a flow of a series of projection processes according to this embodiment will be described. FIG. 3 is a flowchart showing the flow of projection processing according to this embodiment. When the image projection process is started, the process proceeds to step S301.

In step S301, the acquisition unit 105 acquires the source image. As described above, the acquisition unit 105 may acquire an input image such as a moving image desired by the user from an external device as a source image, or may acquire an initial image from a storage unit inside the device. .. For example, the acquisition unit 105 acquires an input image when the input image is input to the image projection device 100 from the external device, and when the input image is not input to the image projection device 100 from the external device, the acquisition unit 105 acquires the input image. , The initial image can be acquired from the storage unit inside the device.

In step S302, the image processing unit 106 performs predetermined image processing such as brightness and contrast adjustment on the acquired image. Further, the image processing unit 106 may perform image scaling processing on the source image according to the driving method of the panel 102 or the shift element 103.

In step S303, the subframe generation unit 107 generates images of the first and second subframes from the image of each frame of the image processed by the image processing unit 106. The subframe generation unit 107 performs resolution conversion processing so as to generate a subframe image having a panel resolution or less from a high resolution image when generating two subframe images from the image of each frame.

In step S304, the chart composition unit 108 superimposes and synthesizes a chart image showing a predetermined chart on the image of the predetermined subframe among the images of the subframe generated by the subframe generation unit 107. In this embodiment, the chart synthesizing unit 108 synthesizes the chart image with the image of the first subframe among the images of the first and second subframes. The image of the predetermined subframe is not limited to the image of the first subframe, and may be any one of a plurality of subframe images generated by the subframe generation unit 107, for example, the second. It may be a subframe image of.

In step S305, the drive unit 109 causes the panel 102 to sequentially display the image of the first subframe in which the chart image is combined by the chart composition unit 108 and the image of the second subframe. Further, in order to shift the projection position for each subframe image, the control unit 110 synchronizes the drive timing of the panel 102 with the drive timing of the shift element 103. As a result, the light emitted from the light source 101 is modulated according to the subframe image displayed on the panel 102, and the projected image corresponding to the modulated light is projected by the shift element 103 for each subframe image. Is projected.

Here, the signal processing described in this embodiment will be described with reference to FIG. FIG. 4 shows the timing of signal processing and various images. In FIG. 4, the flow of time is represented in the direction from left to right on the drawing. FIG. 4 shows a vertical sync signal 401, a source image frame 402, a subframe 403, a chart composition process 404, a projected image 405, and a drive signal 406.

The vertical synchronization signal 401 represents a vertical synchronization signal (VSync) of the image signal acquired by the acquisition unit 105. The frame 402 of the source image represents each frame F1, F2, F3 of the image acquired by the acquisition unit 105. Here, as an example, the format of the source image is set to a resolution of 3840 × 2160 and a frame rate of 60 Hz.

The subframe 403 is a subframe Sf1-1, Sf1-2, Sf2-1, Sf2-2, Sf3-1, Sf3-2 generated from the images of the frames F1, F2, F3 by the subframe generation unit 107. Represents. Here, the images of the subframes Sf1-1 and Sf1-2 show the images of the subframes generated from the images of the frame F1. Similarly, the image of the subframes Sf2-1 and Sf2-2 shows the image of the subframe generated from the image of the frame F2, and the image of the subframes Sf3-1 and Sf3-2 is generated from the image of the frame F3. The image of the subframe is shown.

In this embodiment, it is assumed that the panel resolution is 1920 × 1080. In order to represent an image with a panel having a resolution lower than the resolution of the source image, the subframe generation unit 107 performs resolution conversion processing at the time of subframe generation. Further, in order to increase the number of spatial pixels and increase the resolution by moving the positions of the projected images in chronological order, the subframe generation unit 107 uses the positions of the respective projected images (first projection in FIG. 2). Subframe images corresponding to the position (position and second projection position) are output alternately. The white-painted subframes Sf1-1, Sf2-1, and Sf3-1 in the subframe 403 are for the first projection position, and the black-painted subframes Sf1-2, Sf2-2, and Sf3-2 are for the first projection position. Corresponds to the subframe image for the second projection position. By the above processing, the subframe generation unit 107 outputs an image signal having a resolution of 1920 × 1080 and a frame rate of 120 Hz.

The chart composition process 404 shows the chart composition process performed by the chart composition unit 108 for the image of the corresponding subframe. As shown in the chart composition process 404, the chart composition unit 108 has a predetermined subframe Sf1-1, Sf2-1, Sf3 for a plurality of subframe images generated from the images of each frame F1, F2, F3. The chart image is combined only with the image of -1. The timing of synthesizing the chart images and the type of the chart images to be combined may be controlled by the control unit 110. Here, the type of the chart image may be arbitrarily set according to a desired configuration. In this embodiment, as an example of the chart image, a rectangle including a cross and a rectangular thin line and a monochrome rectangular area as a background is included. The image of is used.

The projected image 405 shows an example of a light-modulated image (projected image) with light modulated by the panel 102 as a result of the above processing. In this embodiment, as shown in the projected image 405, the subframes Sf1-1, Sf2-1, and Sf3-1 in which the chart is superimposed on the image based on the input image and the subframe of only the image based on the input image are subframes. Images of Sf1-2, Sf2-2, and Sf3-2 are projected alternately.

The drive signal 406 represents a drive signal of the shift element 103 controlled by the control unit 110. In the drive signal 406, the Low signal indicates a signal for controlling the projection position to the first projection position, and the High signal indicates a signal for controlling the projection position to the second projection position. The drive signal 406 may be, for example, a drive signal of an actuator that drives the glass plate in the shift element 103.

By synchronizing the display of the subframe image on the panel 102 and the drive of the shift element 103 in this way, the projected image including the chart is projected only at the first projection position. Further, the chart synthesizing unit 108 synthesizes a chart image with the subframe image after the subframe generation unit 107 has performed the resolution conversion process. Therefore, the chart composition unit 108 does not perform the resolution conversion process. Therefore, the chart in the projected image whose resolution is increased by the pixel shift operation is not affected by the line bleeding due to the pixel position movement due to the movement of the shift element 103 and the line bleeding due to the resolution conversion. As a result, it is possible to simultaneously display a chart for adjustment without blurring of fine lines on the projected surface while continuing the pixel shift operation with respect to the source image to be originally displayed and maintaining a high-resolution display.

As described above, the image projection device 100 according to the present embodiment includes a light source 101, an acquisition unit 105, a subframe generation unit 107, a chart composition unit 108, a panel 102, and a shift element 103. The acquisition unit 105 acquires the source image. The subframe generation unit 107 functions as an example of a generation unit that generates a plurality of subframe images from the image of each frame of the source image. The chart synthesizing unit 108 synthesizes a chart image with an image of a predetermined subframe among the images of a plurality of subframes. The panel 102 is an example of an optical modulation unit that modulates the light from the light source 101 by sequentially displaying a plurality of subframe images including a subframe image in which pixels are arranged in a matrix and a chart image is combined. Function. The shift element 103 functions as an example of a shift unit that shifts the projection position of the modulated light for each subframe image.

Due to such a configuration, according to the image projection apparatus 100 according to the present embodiment, the chart in the projected image whose resolution has been increased by the pixel shift operation has line bleeding due to the movement of the pixel position due to the movement of the shift element 103 and line bleeding. Not affected by line bleeding due to resolution conversion. As a result, it is possible to reduce the blurring of the fine lines in the chart while continuing the movement control of the projection position. Therefore, the user can clearly confirm the shape of the chart in the projected image, and can appropriately perform various adjustment processes of the image projection device 100.

In this embodiment, a 2-way shift operation in which two subframes are generated from an image of one frame of the source image has been described, but the pixel shift operation is not limited to this. The method for controlling the pixel shift operation according to the present embodiment is, for example, a 4-way shift operation in which an image of one frame is generated into four subframes and the images of the four subframes are shifted to four different positions and projected. It can also be applied to other known pixel shift operations. In this case, similarly to the projection process described above, for example, the chart image may be combined only with the image of one subframe out of the images of the four subframes. The number of subframes generated in the pixel shift operation and the number of projection positions to be moved are not limited to two or four, and may be changed according to a desired configuration.

Further, the format of the source image shown in this embodiment (resolution is 3840 × 2160, frame rate is 60 Hz) is also an example, and the format of the source image is not limited to this. For example, the resolution may be 4096 x 2160, 5120 x 2880, 7680 x 4320, or the like. The frame rate may be 24 Hz, 30 Hz, or the like. Although the frame rate is expressed as the sampling frequency of the video and Hz is used as the unit, fps (frame per second) may be used as the unit of the frame rate.

Further, as a pixel shift method, a method of shifting the pixels by controlling the inclination of the glass plate has been described. However, the pixel shift method is not limited to this, and other pixel shift methods such as a method of shifting the panel 102 itself by half a pixel or a method of shifting the optical axis by using a birefringent element may be adopted. In these cases, the shift element 103 may include an actuator, a birefringence element, or the like for shifting the panel 102 itself. The above-mentioned various modifications can be applied to the following examples.

(Example 2)
Hereinafter, the image projection apparatus according to the second embodiment of the present invention will be described with reference to FIG. Since this embodiment performs the same operation as that of the first embodiment except for the operation by the chart synthesizer, the description other than the operation by the chart synthesizer will be omitted. Further, since the configuration of the image projection apparatus according to the present embodiment is the same as the configuration of the image projection apparatus 100 according to the first embodiment, the description thereof will be omitted using the same reference numerals.

In step S304, the chart synthesizing unit 108 according to the present embodiment synthesizes a chart image with a predetermined subframe image among a plurality of subframe images, and a pattern image different from the chart image with another subframe image. To synthesize. Here, the pattern image may be, for example, an image showing a background excluding thin lines in a chart image, and may be, for example, a monochrome rectangular image.

FIG. 5 shows the timing of signal processing and various images according to this embodiment. In FIG. 5, the flow of time is represented in the direction from left to right on the drawing. In FIG. 5, the vertical synchronization signal 401, the frame 402 of the source image, the subframe 403, and the drive signal 406 are the same as those described in the first embodiment, and thus the description thereof will be omitted.

The compositing process 504 shows the compositing process performed by the chart compositing unit 108 on the subframe image generated by the subframe generating unit 107. Similar to the first embodiment, the chart synthesizing unit 108 according to the present embodiment synthesizes a chart image with the images of the first subframes Sf1-1, Sf2-1, and Sf3-1. Further, the chart synthesizing unit 108 according to the present embodiment synthesizes a pattern image with the image of the second subframe Sf1-2, Sf2-2, Sf3-2 as shown in the synthesizing process 504. Here, for the sake of simplicity, there are two types of images to be combined, but the types of images to be combined depend on the number of subframe images generated by the subframe generation unit 107 from the images of each frame of the source image. May be variable.

The projected image 505 shows an example of a light-modulated image (projected image) with light modulated by the panel 102 as a result of the above processing. In this embodiment, as shown in the projected image 505, the subframe Sf1-1, Sf2-1, and Sf3-1 on which the chart image including the thin line is superimposed and the subframe on which the black-painted pattern image is superimposed are superimposed. Images of Sf1-2, Sf2-2, and Sf3-2 appear alternately. By switching these sequentially in time and projecting them on the projected surface, the chart including the fine lines is displayed without overlapping with the image based on the source image as the background in the projected image whose resolution is increased by the pixel shift operation. To.

As described above, in the image projection apparatus 100 according to the present embodiment, the chart synthesizing unit 108 synthesizes a chart image with an image of a predetermined subframe among the images of a plurality of subframes. Further, the chart compositing unit 108 synthesizes a pattern image different from the chart image with the subframe image other than the predetermined subframe image among the images of the plurality of subframes. With such a configuration, it is possible to simultaneously display the adjustment chart without blurring of fine lines on the projected surface while continuing the pixel shift operation for the source image to be originally displayed and maintaining the high resolution display. Further, since the thin lines of the chart do not overlap with the image based on the source image as the background, the thin lines of the chart are more easily visible to the user.

(Example 3)
Hereinafter, the image projection apparatus according to the third embodiment of the present invention will be described with reference to FIGS. 6 to 8. In this embodiment, since the same operation as that of the first embodiment is performed except for the operation by the OSD (On Screen Display) synthesis unit and the chart composition unit, which will be described later, the description other than the operation by the OSD synthesis unit and the chart composition unit will be described. Omit. Further, since the configuration of the image projection apparatus according to the present embodiment is the same as the configuration of the image projection apparatus 100 according to the first embodiment except for the OSD synthesis unit described later, the same configuration will be described using the same reference numerals. Is omitted.

FIG. 6 is a block diagram showing the configuration of the image projection device 600 according to the present embodiment. The image projection device 600 is provided with an OSD synthesis unit 601 in addition to the configuration of the image projection device 100 according to the first embodiment. In this embodiment, the OSD synthesis unit 601 can be located after the image processing unit 106 and before the subframe generation unit 107.

The OSD compositing unit 601 superimposes and synthesizes a predetermined OSD image such as a menu image or a pattern image on the image processed by the image processing unit 106. In particular, in this embodiment, the OSD compositing unit 601 superimposes and synthesizes the background image corresponding to the pattern image described in the second embodiment on the image processed by the image processing unit 106.

Hereinafter, a flow of a series of projection processes according to this embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart showing the flow of projection processing according to this embodiment. Regarding the flow of the projection process according to the present embodiment, the same processing as the flow of the projection process according to the first embodiment will be omitted by using the same reference numerals.

In this embodiment, when the image processing unit 106 performs predetermined image processing on the acquired image in step S302, the processing shifts to step S703. In step S703, the OSD compositing unit 601 superimposes and synthesizes the background image corresponding to the pattern image described in the second embodiment as a predetermined OSD image on the image processed by the image processing unit 106.

In step S704, the subframe generation unit 107 generates a plurality of subframe images from the image output by the OSD synthesis unit 601. In this embodiment, the subframe image generation process (step S704) by the subframe generation unit 107 is located after the OSD composition process (step S703) by the OSD composition unit 601. Therefore, since the OSD compositing unit 601 superimposes a predetermined OSD image on the image processed by the image processing unit 106, the image of each subframe generated by the subframe generation unit 107 is also represented. The OSD image is superimposed. Therefore, the subframe generation unit 107 generates a plurality of subframe images on which the background image is superimposed from the image of each frame of the image on which the background image is superimposed by the OSD synthesis unit 601.

In step S705, the chart synthesizing unit 108 superimposes a predetermined chart image on the image of the predetermined subframe among the images of the plurality of subframes generated by the subframe generation unit 107, as in the first embodiment. Synthesize. Since the background image is already superimposed on the subframe image according to this embodiment as described above, the chart image synthesized by the chart synthesizing unit 108 may be composed of only thin lines for adjustment processing. good. Here, the control unit 110 can control the chart superimposition position of the chart compositing unit 108 so that at least a part of the chart image overlaps the display area of the OSD image superimposed by the OSD compositing unit 601. Since the subsequent processing is the same as the processing according to the first embodiment, the description thereof will be omitted.

With such a configuration, the chart image is synthesized by the chart synthesizer 108 while including the background image for the chart image for all the subframe images based on the composite process of the OSD synthesizer 601 to blur the fine lines. It is possible to generate a projected image for no adjustment. As a result, it is possible to simultaneously display a chart for adjustment in which fine lines do not blur on the projected surface while continuing the pixel shift operation for the source image that should be displayed and maintaining a high-resolution display. Further, since the thin lines of the chart do not overlap with the image based on the source image as the background, the thin lines of the chart are more easily visible to the user.

Here, the signal processing described in this embodiment will be described with reference to FIG. FIG. 8 shows the timing of signal processing and various images according to this embodiment. In FIG. 8, the flow of time is represented in the direction from left to right on the drawing. FIG. 8 shows a vertical sync signal 401, a source image frame 402, an OSD composite image 803, a subframe 804, a subframe image 805, a chart composite process 806, a projection image 807, and a drive signal 406. Since the vertical synchronization signal 401, the frame 402 of the source image, and the drive signal 406 are the same as those described with reference to FIG. 4 in the first embodiment, the description thereof will be omitted using the same reference numerals. ..

The OSD composite image 803 shows an example of an OSD composite image in which the OSD composite image 601 synthesizes a background image, which is a predetermined OSD image, with an image processed by the image processing unit 106. Here, the OSD compositing unit 601 synthesizes a black rectangular background image with the image processed by the image processing unit 106. Since the OSD compositing process is performed before the subframe image generation process, the OSD compositing process is performed on the image having a resolution of 3840 × 2160.

The subframe 804 contains each subframe Sf1-1, Sf1-2, Sf2-1, Sf2-2, Sf3-1, Sf3-2 generated from the image of each frame of the OSD composite image by the subframe generation unit 107. Represents. Here, the images of the subframes Sf1-1 and Sf1-2 show the image of the subframe generated from the image of the frame F1 in which the background image is combined. Similarly, the images of the subframes Sf2-1 and Sf2-2 show the image of the subframe generated from the image of the frame F2 in which the background image is combined. Further, the images of the subframes Sf3-1 and Sf3-2 show the image of the subframe generated from the image of the frame F3 in which the background image is combined.

In this embodiment, it is assumed that the panel resolution is 1920 × 1080. In order to represent an image with a panel having a resolution lower than the resolution of the source image, the subframe generation unit 107 performs resolution conversion processing at the time of subframe generation. Further, in order to increase the number of spatial pixels and increase the resolution by moving the positions of the projected images in chronological order, the subframe generation unit 107 uses the positions of the respective projected images (first projection in FIG. 2). Subframe images corresponding to the position (position and second projection position) are output alternately. The white-painted subframes Sf1-1, Sf2-1, and Sf3-1 in the subframe 804 are for the first projection position, and the black-painted subframes Sf1-2, Sf2-2, and Sf3-2 are for the first projection position. Corresponds to the subframe image for the second projection position. By the above processing, the subframe generation unit 107 outputs an image signal having a resolution of 1920 × 1080 and a frame rate of 120 Hz.

Here, the resolution of the image of each subframe is converted into a resolution of 1920 × 1080 in order to make it a resolution that can be displayed on the panel 102. Further, the frame rate of the image of each subframe is doubled in order to display the image of the subframe in synchronization with the movement by the shift element 103.

The subframe image 805 shows an example of the image of each subframe Sf1-1, Sf1-2, Sf2-1, Sf2-2, Sf3-1, Sf3-2 shown in the subframe 804. In this embodiment, since the subframe image is generated by the subframe generation unit 107 after the background image is superimposed by the OSD composition unit 601, each sub is generated regardless of the order of the subframes as shown in the subframe image 805. The frame image contains a background image.

The chart composition process 806 shows the chart composition process performed by the chart composition unit 108 for the image of the corresponding subframe. As shown in the chart compositing process 806, the chart compositing unit 108 makes a predetermined subframe image for a plurality of subframe images generated from the images of each frame F1, F2, F3 in which the background image is synthesized. Only composite chart images. Here, the chart image is combined only with the images of the subframes Sf1-1, Sf2-1, and Sf3-1.

The timing of synthesizing the chart images and the type of the chart to be combined may be controlled by the control unit 110. Here, the type of the chart image may be arbitrarily set according to a desired configuration, and in this embodiment, an image including a cross and a rectangular thin line is used as an example of the chart image. Further, the control unit 110 is a chart composition unit so that the position where the chart image is synthesized by the chart composition unit 108 overlaps the region including the background image in the images of the subframes Sf1-1, Sf2-1, Sf3-1. 108 can be controlled.

Here, since the chart image compositing process is performed after the subframe image generation process, the chart image composited with the subframe image is not affected by the line blurring due to the resolution conversion. Further, by synthesizing the chart image only with the image of a predetermined subframe, the chart can be displayed only at one of the plurality of projection positions moved by the shift element 103.

The projected image 807 shows an example of an image of each subframe after the composition processing of the chart image. In this embodiment, as shown in the projected image 807, the subframe Sf1-1, Sf2-1, and Sf3-1 on which the background image and the chart image are superimposed and the subframe Sf1 on which only the background image is superimposed are superimposed. -2, Sf2-2, Sf3-2 images are projected alternately. By sequentially switching these images in time and projecting them on the projected surface, the chart including the fine lines does not overlap with the image based on the source image as the background in the projected image whose resolution is increased by the pixel shift operation. Is displayed.

As described above, the image projection apparatus 600 according to the present embodiment further includes an OSD synthesis unit 601. The OSD compositing unit 601 functions as an example of a background compositing unit that synthesizes a background image with an image of each frame. The subframe generation unit 107 generates a plurality of subframe images from the image of each frame in which the background image is combined. Further, the chart synthesizing unit 108 synthesizes a chart image with an image of a predetermined subframe among the images of a plurality of subframes.

By the above processing, the image projection device 600 according to the present embodiment is for adjusting the source image to be originally displayed so that the fine lines do not blur while continuing the pixel shift operation and maintaining the high resolution display. Chart can be displayed on the projected surface at the same time. Further, for example, by superimposing the chart on the background image composed of a monochrome rectangular image, the display area of the chart can be prevented from interfering with the image based on the source image. By doing so, the thin lines of the chart do not overlap with the image based on the source image as the background, so that the user can more easily see the thin lines of the chart.

The process according to the second embodiment may be applied to the image projection apparatus 600 according to the present embodiment. In this case, the pattern image that the chart synthesizing unit 108 synthesizes with the subframe image other than the predetermined subframe image can be an image different from the background image. For example, the pattern image may be a rectangular image or the like having a color different from that of the background image.

(Example 4)
Hereinafter, the behavior of the image projection apparatus according to the fourth embodiment of the present invention will be described with reference to FIG. FIG. 9 is a diagram for explaining the behavior of the image projection device according to the present embodiment. FIG. 9 shows an image projection device 901, an operation unit 902, a remote controller 903, a projection image 904, an OSD 905, and a chart 906 according to the present embodiment.

The image projection device 901 may be the image projection device according to any one of the first to third embodiments. The image projection device 901 receives operation information from an operation unit 902 provided on the housing, a remote controller 903 provided outside the housing, and the like.

The projected image 904 is an example of a projected image projected on the projected surface by the image projection device 901. When a predetermined operation is performed on the image projection device 901 by the user via the operation unit 902 or the remote controller 903, the image projection device 901 is controlled according to the operation content. The image projection device 901 can display information indicating the operation on the projected image by using the OSD905 in order to make the control contents easy for the user to understand. Here, it is assumed that the user performs a focus adjustment operation using the operation unit 902 or the remote controller 903. The OSD905 is an example of an OSD indicating that focus adjustment is performed.

The image projection device 901 displays the chart 906 on the projected surface in response to the operation instruction of the focus adjustment by the user. The display method of the chart 906 may be according to any one of Examples 1 to 3.

By doing so, the chart synthesizing unit 108 can switch whether or not to synthesize the chart image according to the instruction of the user. In addition, the image projection device 901 reduces the influence of resolution conversion and projection position movement control while maintaining the display state of the high-resolution image that should be originally displayed according to the content of the user's operation instruction, and blurring of fine lines. It is possible to display a chart with reduced resolution.

In this embodiment, the focus adjustment execution instruction is given as the operation instruction by the user. However, the user's instruction regarding the adjustment process of the image projection device is not limited to this. For example, the user's instruction may be an execution instruction such as position adjustment, shape adjustment, registration adjustment, etc. of the projected image. Here, the position adjustment of the projected image may include the adjustment of the position of the entire projected image, the alignment of the projected image using a plurality of image projection devices, and the like. Further, the shape correction of the projected image may include correction of a trapezoid called a keystone, correction of curvature, and the like.

In Examples 1 to 4 above, an example in which pixel shift projection processing is performed as projection processing has been described, but it is possible to switch between a projection mode in which pixel shift projection processing is performed and a projection mode in which general image projection processing is performed. An image projection device may be configured. When the projection mode for performing pixel shift projection is selected, the image projection method according to the first to fourth embodiments may be applied.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Although the present invention has been described above with reference to the examples, the present invention is not limited to the above examples. The present invention also includes inventions modified to the extent not contrary to the gist of the present invention, and inventions equivalent to the present invention. In addition, the above-mentioned Examples and Modifications can be appropriately combined as long as they do not contradict the gist of the present invention.

100: Image projection device, 101: Light source, 102: Panel (optical modulation unit), 103: Shift element (shift unit), 105: Acquisition unit, 107: Subframe generation unit (generation unit), 108: Chart composition unit

Claims (8)

Light source and
The acquisition unit that acquires the source image,
A generator that generates a plurality of subframe images from the image of each frame of the source image,
A chart compositing unit that synthesizes a chart image with an image of a predetermined subframe among the images of the plurality of subframes, and a chart compositing unit.
A light modulation unit that modulates the light from the light source by sequentially displaying the images of the plurality of subframes including the subframe image in which the pixels are arranged in a matrix and the chart image is combined.
A shift unit that shifts the projection position of the modulated light for each subframe image,
An image projection device. The image according to claim 1, wherein the chart synthesizing unit synthesizes a pattern image different from the chart image with an image of a subframe other than the image of the predetermined subframe among the images of the plurality of subframes. Projection device. A background compositing unit for compositing a background image with the image of each frame is further provided.
The image projection device according to claim 1 or 2, wherein the generation unit generates an image of the plurality of subframes from an image of each frame in which the background image is combined. The image projection device according to any one of claims 1 to 3, wherein the source image is either an initial image included in the image projection device or an input image input to the image projection device. The image projection device according to any one of claims 1 to 4, wherein the chart synthesizing unit switches whether or not to synthesize the chart image according to a user's instruction. The image projection apparatus according to claim 5, wherein the user's instruction includes at least one execution instruction of focus adjustment, position adjustment of a projected image, shape adjustment of a projected image, and registration adjustment. To get the source image and
To generate multiple subframe images from the image of each frame of the source image,
Combining a chart image with an image of a predetermined subframe among the images of the plurality of subframes,
To modulate the light from the light source by sequentially displaying the images of the plurality of subframes including the image of the subframe in which the chart image is synthesized by the pixels in a matrix.
By shifting the projection position of the modulated light for each image of the subframe,
A method of controlling an image projection device, including. A program that causes the image projection device to execute each step of the control method according to claim 7, when executed by a computer included in the image projection device.

Images