WO2016132446A1

WO2016132446A1 - Picture projection apparatus and shaking correction method

Info

Publication number: WO2016132446A1
Application number: PCT/JP2015/054229
Authority: WO
Inventors: 中出　真弓; 慎一郎廣岡
Original assignee: 日立マクセル株式会社
Priority date: 2015-02-17
Filing date: 2015-02-17
Publication date: 2016-08-25

Abstract

An ultra-short focus projector has the problem of noticeable image distortion due to screen shaking caused by even a slight wind or the like, interfering with a presentation or the like. Hitherto, shaking such as a constant wave-like motion of the screen has not been taken into consideration. Thus, in a projected-picture shaking correction method for a picture projection apparatus, picture-signal shaking correction is performed by: projecting a projection picture signal obtained by superposing a pattern image having singular points at correction points on a picture signal; capturing an image of the projected picture; detecting the pattern image and position information about the singular points from the captured picture; and calculating a correction parameter by comparing position information that is the correction point standard with the detected position information. As a result, even in a state where the screen that is a projection destination of the picture projection apparatus is shaking, the shaking of the picture projected by the picture projection apparatus can be corrected and the shaking can be reduced.

Description

Image projection apparatus and shake correction method

The present invention relates to a video projection device that projects image data onto a screen, and more particularly to a video projection device and a shake correction method for correcting a shake of a video.

The small and lightweight video projection device has an essential function for correcting the projected video according to the positional relationship between the screen and the video projection device because of its usage.

There is JP-A-2005-148298 (Patent Document 1) as background art in this technical field. In Patent Document 1, a test pattern image is displayed on a screen with invisible light in a predetermined cycle, an image of the test pattern displayed on the screen is acquired, and image distortion correction processing is performed based on the test pattern image. It is described.

JP 2005-148298 A

In recent years, due to the limitation of installation location, ultra-short focus projectors with a short focal length that can obtain a large screen even if the distance between the screen and the projector is short have come to be used frequently. In particular, in the ultra-short focus projector, there is a problem that image distortion due to screen shaking due to a slight wind or the like is remarkable, which hinders presentation and the like.

In Patent Document 1, there is a description that always corrects image distortion with invisible light, but there is no description of specific control, and correction of distortion of the entire screen that occurs mainly during installation is possible, but it seems that the screen always undulates. It does not take into account the tremors.

An object of the present invention is to provide a video projection apparatus and a shake correction method that can constantly monitor the shake of the screen and reduce the shake of the projected video.

In order to solve the above problems, for example, the configuration described in the claims is adopted. The present invention includes a plurality of means for solving the above-described problems. To give an example, the present invention relates to a method for correcting a shake of a projected video of a video projection device, and a pattern image having a singular point at a correction point in a video signal. Project the superimposed video signal for projection, capture the projected image, detect the position information of the pattern image and singular point from the captured image, and compare the position information that serves as the reference for the correction point with the detected position information Then, the correction parameter is calculated, and the shake correction of the video signal is performed.

According to the present invention, even when the screen that is the projection destination of the video projection device is shaking, it is possible to correct the shaking of the image projected by the video projection device and reduce the shaking.

1 is a block diagram illustrating an overall configuration of a video projection device in Embodiment 1. FIG. FIG. 6 is an explanatory diagram of dividing an image into blocks according to the first embodiment. FIG. 6 is a diagram illustrating an output image after shake correction in the first embodiment. It is a figure which shows the projection pattern image of the video projection apparatus in Example 1. FIG. It is a figure which shows the picked-up image of the projection pattern image in Example 1. FIG. FIG. 3 is a process flow diagram of the video projection apparatus in Embodiment 1. It is a block diagram which shows the whole structure of the video projection apparatus in Example 2. FIG. FIG. 10 is a diagram illustrating a captured image captured by the pattern image capturing apparatus according to the second embodiment. FIG. 10 is a diagram illustrating a captured image captured by a pattern image capturing unit according to the second embodiment. FIG. 10 is a diagram illustrating setting of a reference correction position in the second embodiment. It is a block diagram which shows the whole structure of the video projection apparatus in Example 3. FIG. It is a figure which shows the locus | trajectory of the correction point in Example 3. FIG. It is a block diagram which shows the whole structure of the video projection apparatus in Example 4. FIG. FIG. 10 is a block diagram of a shake correction unit in Embodiment 4. FIG. 12 is a timing diagram of video projection apparatus input video, projection, and imaging in Example 4. It is a block diagram which shows the whole structure of the video projection apparatus in Example 5. FIG. It is a figure which shows the pattern image in Example 5. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol is used for the same element and the overlapping description is abbreviate | omitted.

In the present embodiment, a video projection apparatus that detects the fluctuation of the pattern and corrects the projected video will be described.

FIG. 1 is a block diagram showing the overall configuration of the video projection apparatus in the present embodiment. In FIG. 1, 110 is a video projection device, 100 is a video input terminal, 111 is a video signal processing unit, 112 is a shake correction unit, 113 is a pattern image unit, 114 is a projection video composition unit, 115 is a projection unit, and 123. Is an imaging unit, 122 is an imaging pattern detection unit, 121 is a control unit, 130 is a position information storage unit, and 117 is a projection screen. Hereinafter, each block will be described.

The video input terminal 100 is connected to a device that outputs a video signal, such as a personal computer or a media player, and the video signal is input from the video input terminal 100.

The video signal processing unit 111 converts the video signal input from the video input terminal 100 into a video signal that can be processed by the shake correction unit 112 at the subsequent stage. For example, when the input video signal is an analog signal, conversion processing to a digital signal is performed. For example, when a component signal is input, video signal processing is performed as necessary, such as conversion to an RBG signal.

The shake correction unit 112 applies correction parameters set by the control unit 121, which will be described later, to the signal output from the video signal processing unit 111 when the video projected by the video projection device 110 is shaken. Perform shake correction accordingly.

The shake correction is performed, for example, by dividing one frame image output from the video signal processing unit 111 into a plurality of blocks, and the control unit 121 uses the coordinates of the movement destination of the vertex for each block as a correction parameter. The shake correction unit 112 converts the image in the block into a trapezoid, rotates or moves the position so that the coordinates specified by the correction parameter for each block, and synthesizes the image of each block to correct the shake. Generate an image.

Fig. 2 shows an example of block division of one frame image. FIG. 2 is an example in which one frame image output from the video signal processing unit 111 is divided into 3 × 3, 300 is an image of one frame, 310 is a display position of a pattern image described later, and P11 and P12. , P13, P14, P21, P22, P23, P24, P31, P32, P33, P34, P41, P42, P43, and P44 indicate the vertex coordinates of each block of the pattern image, and are used as shake correction correction points. For example, the shake correction parameter is given as a movement amount having the coordinates or direction of the movement destination of each correction point.

FIG. 3 shows an example of an output image after the shake correction unit 112 corrects the shake. FIG. 3 shows an example in which the shake correction unit 112 converts an image for each block. P11a, P12a, P21a, and P22a indicate the vertex coordinates of the upper left block set by the shake correction parameter, and 300a indicates the shake correction. The subsequent one-frame shake correction image is shown. For example, if the coordinates of the movement destination of the block having P11, P12, P21, and P22 as vertices are P11a, P12a, P21a, and P22a, the shake correcting unit 112 can convert the quadrilateral P11a, P11a, Conversion to P12a, P21a, and P22a is performed. In other blocks as well, the shake correction unit 112 converts images sequentially in the same manner, and connects all the converted blocks to generate a shake correction image of one frame.

In addition, in order to perform shake correction, it may be necessary to arrange the image outside the display position before shaking. If the display position before shaking is set to an image size that can be projected by the projection unit 115, the image needs to be arranged outside the position where the projection unit 115 can project due to shaking, and the image to be displayed may be interrupted and cannot be displayed. . Therefore, it is possible to prevent the image from being interrupted by reducing the image size of the input video signal in advance. In consideration of these, the shake correction unit 112 may reduce the image data simultaneously with the shake correction.

Further, although the boundary lines and block vertices in FIG. 2 and FIG. 3 are illustrated for explanation, they are not drawn on an actual image. The number of block divisions is not limited to 3 × 3, and other division numbers may be used in accordance with the state of shaking and the performance of correction.

The pattern image unit 113 stores a pattern image for detecting the shaking of the projected video. The pattern image is a pattern image having a singular point at a correction point. For example, the image may be any image in which the shake correction unit 112 can detect the vertices of a block that divides the video. A lattice may be used, and the vertex position of a block may be a dot or cross pattern instead of a line. In addition, when projecting a pattern image with visible light, the pattern image does not use the pattern that overlaps the shake correction image output from the shake correction unit 112, but uses only the pattern arranged outside the shake correction image. To do.

FIG. 4 shows an example of a pattern image projected with visible light. Reference numeral 210 denotes an entire image area projected by the video projection apparatus 110, and reference numeral 211 denotes an area for displaying a shake correction image. A pattern image 212 is arranged inside the entire image area 210 and outside the shake image area 211. . In the pattern image 212, for example, black and white lines are alternately displayed in the horizontal x direction and the vertical y direction, and the black lines are rectangles having vertices C1, C2, C3, and C4, and B11, B12, B21, and B22. , B31, B32, B41, B42. B21 and B22, B31 and B32 have the same coordinate position in the y direction, and B11 and B41, and B12 and B42 have the same coordinate position in the x direction. C1, C2, C3, C4, B11, B12, B21, B22, B31, B32, B41, and B42 correspond to the shake correction points at which the shake correction unit 112 divides the video. That is, B ** can be detected as a vertex of the block because the black line is interrupted, and C ** can be similarly detected as a vertex of the block because it is a corner.

In this pattern image example, there is no pattern image corresponding to the shake correction points P22, P23, P32, and P33 in FIG. 2, but the movement amount of each shake correction point from the shake correction point that can be detected by the control unit 121 described later. Correction can be performed by calculating. FIG. 4 shows an example in which the shake correction unit divides the image into 3 × 3 blocks for correction. However, the number of divisions is not limited to this as long as it corresponds to the block of the shake correction unit 112. However, further fine division can increase the amount of processing, but more accurate correction can be performed.

Further, the pattern image is not limited to the one shown in the figure as long as the position can be detected, and may be a black and white inverted pattern or a pattern using colors other than black and white.

In FIG. 1, the projection video synthesis unit 114 synthesizes the shake correction image output from the shake correction unit 112 and the pattern image of the pattern image unit 113 to generate and output a projection video signal.

The projection unit 115 includes a light source, a light modulation unit, a projection lens, and the like, modulates the light output from the light source by the light modulation unit, and displays an image on the projection screen 117 with the projection lens. The light modulation unit may be any component that can modulate light by changing the light transmittance or reflectance, and includes, for example, a liquid crystal panel, a liquid crystal panel and a dichroic mirror, a digital mirror device, and a color wheel.

The projection screen 117 is not limited to the projector screen, and may be, for example, a wall or a curtain as long as it can project an image.

The imaging unit 123 captures an image projected by the projection unit 115 onto the projection screen, and outputs it as an captured image signal. The imaging unit 123 is configured by appropriately using, for example, a lens group including a zoom lens and a focus lens, an image sensor including an iris, a shutter, an imaging element such as a CCD or a CMOS, an amplifier, an AD converter, and the like. The optical image received at is photoelectrically converted, and the signal is subjected to separation processing, demosaicing processing, and the like to generate a captured video signal.

The imaging pattern detection unit 122 detects a pattern image from the captured video signal output from the imaging unit 123, and detects a correction point in the pattern image.

FIG. 5 shows an example of a captured image of one frame of the captured video signal from the imaging unit 123 when the video projection device 110 projects a video on which the pattern image shown in FIG. 4 is superimposed. In FIG. 5, 400 is the entire range of the captured image, 212b is the captured pattern image, C1b, C2b, C3b, and C4b are the apex portions of the captured pattern image, B11b, B12b, B21b, B22b, B31b, B32b, B41b , B42b indicates a discontinuous portion of the captured pattern image. Each vertex part and the discontinuous part correspond to each vertex part and each discontinuous part of the pattern image shown in FIG.

The imaging pattern detection unit 122 detects, for example, C1b, C2b, C3b, C4b, B11b, B12b, B21b, B22b, B31b, B32b, B41b, and B42b as correction points, and the position information of each correction point is the position at the time of installation. Output as information.

The control unit 121 compares the position of the correction point detected by the imaging pattern detection unit 122 with the position of the reference correction point stored in the position information storage unit 130, and determines the amount of shift of the correction point from the pattern image shift. It is converted into a quantity and set in the shake correction unit 112 as a shake correction parameter. Here, the reference correction point is a correction point obtained for the first time immediately after the start of shake correction, for example, and the control unit 121 stores the position information detected in the captured pattern image at the time of the first correction in the position information storage unit 130. Store. When there is no reference position information in the position information storage unit 130, the control unit 121 sets a correction parameter having a shake correction amount of zero in the control unit 121 during the initial correction. In this way, by setting the first correction point as the reference position, the shake correction can be performed regardless of the positions of the projection screen 117 and the imaging unit 123. In addition, it is good also as a correction point in arbitrary past time points. In addition, since there is a possibility that the position of the projection screen 117 is greatly shaken in one acquisition of the position, a calibration period is provided, and an intermediate position or an average position of correction points acquired in a plurality of past frames. May be used as a reference correction point. In this case, correction to a more appropriate position is possible.

Further, when a pattern image is projected with visible light and correction points can be provided only at the peripheral part, for example, correction points P21, P22, P32 and P33 in FIG. Correction coordinates are calculated from the points and set in the shake correction unit 112. As a method for calculating the position of the correction point that cannot be detected, for example, the movement amount is calculated by weighting according to the distance to the correction point for calculating the movement amount of the correction point in the peripheral portion. The control unit 121 sets the correction coordinates between the detected correction point and the calculated correction point as a correction parameter for the shake correction unit 112.

With the video projection device 110 described above, even when the projected video moves due to the shaking of the video projection destination, correction can always be made and a projection video without shaking can be obtained.

Next, the processing flow of the control unit 121 and the shake correction unit 112 of the video projection device 110 will be described below.

FIG. 6 is a processing flow of the video projection apparatus in the present embodiment. In FIG. 6, the video projection apparatus 110 initializes correction parameters and reference positions in step S1001, for example, when the power is turned on or when shake correction is started. Next, the shake correction unit 112 acquires the video signal input in step S1002, and in step S1003, performs shake correction according to the shake correction parameter and outputs a shake correction image. Here, when the shake correction parameter is in the initialized state, an image similar to an image not subjected to shake correction is output as a shake correction image. In step S1004, the projection image synthesizing unit 114 synthesizes the shake correction image and the pattern image. In step S1005, the composite image from the projection video composition unit 114 is converted into a projected image projected by the projection unit 115 as necessary, and video projection is performed by the projection unit. The image projected in step S1006 is imaged by the imaging unit. In step S1007, a pattern image is detected from the captured image, and further, a correction point position is detected. In step S1008, it is determined whether position information serving as a reference for calculating the amount of shaking is set. If the reference position information is not set, the correction point position acquired in step S1009 is set as the reference position information, and the correction end determination in the next step S1012 is performed. If the reference position information has been set in step S1008, in step S1010, the correction point position information is compared with the reference position information, and a shake correction parameter is calculated. Further, if necessary, the fluctuation amount of the center correction point position is calculated from the fluctuation correction parameters of the peripheral portion. In step S1011, the shake correction parameter is set in the shake correction unit 112. In step S1012, the end of the correction is determined. If the correction is not finished, the process returns to the video signal acquisition in step S1002. If it is determined in step S1012 that the correction has been completed, the shake correction is terminated.

As described above, in this embodiment, the image projection apparatus projects the image even when the screen is shaken by measuring the amount of shake on the screen and correcting the image to be projected at any time so as not to shake. It can correct the shaking of the image and reduce the shaking.

This embodiment describes a video projection apparatus that uses initial correction data as reference correction data when distortion of a projected image at the time of initial setting is combined with shake correction.

FIG. 7 is a block diagram showing the overall configuration of the video projection apparatus in the present embodiment. The same parts as those in FIG. In FIG. 7, reference numeral 140 denotes an image pickup apparatus. For example, an image sensor including a lens group including a zoom lens and a focus lens, an image pickup device such as an iris, a shutter, a CCD or a CMOS, an amplifier, and an AD converter are used as appropriate. The optical image received by the image sensor is photoelectrically converted, the signal is subjected to separation processing and demosaicing processing, etc., and a video signal is generated and output.

The imaging device 140 is a position where the distortion of the projected image at the time of initial setting of the video projection device 500 is to be corrected, for example, when a video is projected from the video projection device 500 onto the projection screen 117 and a presentation is given. The image projected by the image projection device 500 is imaged at the center of the position. The imaging device 140 sends the captured video to the video projection device 500 as a video signal. Here, the video signal is not limited to a moving image, and may be one piece of still image data.

FIG. 8 shows a captured image obtained by projecting the pattern image shown in FIG. In FIG. 8, 600 is the entire range of the captured image, 212d is the captured pattern image, C1d, C2d, C3d, and C4d are the apex portions of the captured pattern image, B11d, B12d, B21d, B22d, B31d, B32d, B41d, B42d indicates a portion where the captured pattern image is interrupted. Each vertex part and the discontinuous part correspond to each vertex part and each discontinuous part of the pattern image shown in FIG. That is, the imaging device 140 is a means for inputting a distortion image of a projection image when the video projection device 500 is installed.

FIG. 9 shows a captured image obtained by projecting the pattern image shown in FIG. In FIG. 9, 610 is the entire range of the captured image, 212e is the captured pattern image, C1e, C2e, C3e, and C4e are the apex portions of the captured pattern image, B11e, B12e, B21e, B22e, B31e, B32e, B41e, B42e indicates a portion where the captured pattern image is interrupted. Each vertex part and the discontinuous part correspond to each vertex part and each discontinuous part of the pattern image shown in FIG.

The control unit 121 causes the imaging pattern detection unit 122 to first process the pattern image from the imaging device 140, and then cause the pattern image from the imaging unit 123 to be analyzed.
In accordance with an instruction from the control unit 121, the imaging pattern detection unit 122 interrupts the pattern images from the vertexes C1d, C2d, C3d, and C4d of the pattern image that are correction points at the time of installation from the image data 600 acquired from the imaging device 140. The positions of the existing parts B11d, B12d, B21d, B22d, B31d, B32d, B41d, and B42d are detected, and position information is output. The control unit 121 stores the position information output by the imaging pattern detection unit 122 as installation position information.

Next, the imaging pattern detection unit 122 uses the pattern data vertexes C1e, C2e, C3e, and C4d, which are correction points at the time of installation, from the image data 610 acquired from the imaging unit 123, and a portion B11e where the image pattern is interrupted. The positions of B12e, B21d, B22d, B31e, B32e, B41e, and B42e are detected, and position information is output. The control unit 121 stores the position information acquired from the imaging pattern detection unit 122 as pre-reference position information.

The control unit 121 calculates reference position information from the installation position information and the pre-reference position information. As a calculation method, for example, a point obtained by moving the same difference distance in the direction opposite to the installation position with respect to the position of each correction point in the pre-reference position information is set as a reference correction point.

Fig. 10 shows an example of setting the reference correction position. In FIG. 10, reference numeral 620 denotes an entire image space where shake correction is performed, and P11e, P12e, P13e, P14e, P21e, P24e, P31e, P34e, P41e, P42e, P43e, and P44e denote reference position correction points. Each reference position correction point is calculated from the position information of the correction point 212d shown in FIG. 8 and the correction point position information 212e shown in FIG. 9, and is stored in the position information storage unit 130 as reference position information. Since the subsequent operation is the same as that of the first embodiment, the description thereof is omitted.

In addition, when correcting the distortion in the screen by the initial setting, the approximate line is calculated from the measurement position of the peripheral portion of the captured image of the imaging device 140 by matching the timing of imaging of the imaging device 140 and the imaging unit 123. A reference correction point may be created in consideration of the amount of positional deviation from a straight line. In this case, in addition to the trapezoidal distortion caused by the installation position, for example, a projection image without distortion can be obtained even in a projection place where an image such as a wave curtain is distorted.

Further, the pattern image captured by the imaging unit 123 and the pattern image captured by the imaging device 140 may differ depending on the imaging distance and the imaging size. For example, the size of the pattern image captured by the imaging device 140 may be different. Can be realized by converting to a size of the pattern image captured by the imaging unit 123 and correcting the pattern image.

The example in which the images captured by the imaging device 140 and the imaging unit 123 are used one by one has been described above. However, an average value of position information detected from a plurality of pieces of image data or a representative value may be used. In this case, although calibration takes time, more accurate shake correction can be realized.
The imaging device 140 is not limited to a dedicated imaging device, and may be any device that has an imaging function and can output image data or a video signal.

Further, if the imaging unit 123 of the video projection device 500 can move to an arbitrary place different from the video device 500, the imaging unit 140 may be used as the imaging unit 123. For example, the imaging unit 123 has a memory for storing a still image, captures an image of the front of the projection screen 117 at the time of installation, and stores the image in the still image storage memory. Then, the imaging unit 123 is moved to a position where the video of the video projection device 500 is to be corrected, and the video on the projection screen 117 is captured. When the imaging unit 123 is connected to the video projection device, a pattern image may be detected from the stored still image.

As described above, according to the present embodiment, means for inputting a distorted image of a projected image at the time of installation of the video projection device is provided, and the distorted image is reflected in reference position information, for example, a projection destination and a video. Even in a state where the projection apparatus is not parallel, it is possible to obtain a projection image that is free from trapezoidal distortion and is free from shaking or reduced.

In this embodiment, an image projection apparatus that predicts a shake amount from a change locus of the shake amount and performs shake correction will be described.

FIG. 11 is a block diagram showing the overall configuration of the video projection apparatus 700 in the present embodiment. The same parts as those in FIG. In FIG. 11, 710 is a position information storage unit, 711 is a trajectory analysis unit, and the control unit 121 stores position information of correction points detected by the imaging pattern detection unit 122 in the position information storage unit 710. The trajectory analysis unit 711 analyzes the position information stored in the position information storage unit 710 and calculates the predicted movement amount and movement direction.

Fig. 12 shows an example of the locus of correction point fluctuation. FIG. 12 shows an example of movement of the position of the correction point detected by the imaging pattern detection unit 122. It shows that a certain correction point moves from the position m0 to m1, m2, and m3 every frame. In this way, the trajectory analysis unit 711 calculates the predicted position of the next correction point when it is determined that the projection screen 117 is periodically swaying, for example, the correction point is moving in a circular motion. . The trajectory analysis unit 711 can easily predict the periodic shaking described above, but cannot predict when the prediction data is small or the projection screen does not move, or the predicted position is It will cause a large shift and a large shaking. In order to prevent this, the trajectory analysis unit 711 outputs an index of the probability of the predicted position at the same time, and the control unit 121 determines whether or not the predicted position information of the trajectory analysis unit 711 is adopted. Good.

As described above, the present embodiment predicts shaking and performs the predicted correction on the image before projection, thereby reducing the time lag of shaking correction that occurs from projection until imaging. Correction becomes possible.

In this embodiment, a video projection apparatus that changes the feedback frequency in accordance with the speed of shaking will be described. This embodiment is particularly effective when the shaking is fast.

FIG. 13 is a block diagram showing the overall configuration of the video projection apparatus 800 in this embodiment. The same parts as those in FIG. In FIG. 13, reference numeral 810 denotes a shaking speed analysis unit, which outputs a result of analyzing the shaking speed from the movement amount of the position information accumulated by the position information accumulation unit 710. When the control unit 121 determines from the result of the analysis by the shake speed analysis unit 810 that the speed of the shake is fast, the control unit 121 sets the shake correction unit to perform the shake correction at a period earlier than the period of the input video signal. Control.

FIG. 14 shows a block diagram of the shake correction unit 112. In FIG. 14, 820 is a video storage unit, 821 is a video storage unit, 822 is an image conversion unit, 823 is a corrected image storage unit, and 824 is a corrected image output unit.

The video storage unit 820 stores the video signal output from the video signal processing unit 111 in the video storage 821. Here, the video storage unit 821 has a capacity for storing video signals of a plurality of frames, and the video storage unit 820 stores the video signal in the storage location of the video storage unit 821 specified by the control unit 121.

The image conversion unit 822 takes out necessary image data from the video storage unit 821, performs image data enlargement / reduction, rotation, and movement processing according to the correction parameters set by the shake correction unit, and stores corrected image data. Stored in the unit 823. The corrected image output unit 824 sequentially reads out image data from the corrected image storage unit 823 after the image conversion unit 822 stores an image for one frame in the corrected image storage unit 823, and outputs it as a shake correction video signal.

In the control unit 121, for example, the image correction unit 822 speeds up the shake correction by making the read-out cycle of the image conversion unit 822 faster than the cycle in which the video storage unit 820 stores the video signal in the video storage unit 821.

FIG. 15 shows an example of the input video of the video projection device 800 and the timing of projection and imaging.
The input video of the video projection device 800 is sent image data in the order of video 1, video 2 and video 3 in the cycle specified by the device outputting the video signal. For example, the video storage unit 821 has an area in which a video signal can be stored for two frames. Data of video 1 is recorded in the first frame, and is held in the first frame until video 3 which is a video signal of the next frame is input.

In FIG. 15, the control unit 112 projects the shake correction image corrected by the shake correction unit in the period in which the data of the video 1 is held in the first frame of the video storage unit, and images the image pickup unit. This shows that the cycle of detecting the pattern image from the captured image and setting the correction parameter in the shake correction unit is performed three times.

Note that the correction cycle of the present embodiment is an example, and is not limited to this. The cycle for the input signal may be set according to the speed of shaking and the cycle of the input video signal. If it is determined that the shaking is slow and high-speed correction is not necessary, correction may be performed in accordance with the period of the input video signal.

As described above, in this embodiment, the speed of shaking is analyzed from the amount of movement of the position information accumulated by the position information accumulating unit, and when it is determined that the speed of shaking is fast, the period of the input video signal is However, by performing shake correction at an early cycle, the followability to the shake is improved, and an image without shake is obtained.

In the present embodiment, a video projection apparatus that corrects shaking accurately without projecting a shaking image by projecting a pattern image with invisible light will be described.

FIG. 16 is a block diagram showing the overall configuration of the video projection apparatus 900 in the present embodiment. The same parts as those in FIG. In FIG. 16, 911 is a pattern image unit for invisible light, a projection unit 910 is a projection unit capable of projecting invisible light, and 920 is an imaging unit capable of imaging invisible light.

FIG. 17 shows an example of a pattern image of invisible light. Reference numeral 950 denotes an entire video area projected by the projection unit 910, and reference numeral 951 denotes a video area when there is no shaking, D11, D12, D13, D14, D21, D22, D23, D24, D31, D32, D33, D34, D41, D42, D43, and D44 represent, for example, dot pattern images. Since the pattern image only needs to know the dot pattern position, it is only necessary that the brightness and darkness of the brightness other than the dot pattern portion and the dot pattern portion be different. Further, the shape of the pattern is not limited to the dot, but may be a square or lattice image pattern. Since the pattern image is projected with non-visible light, it is also arranged at the center of the screen, for example, D22, D23, D32, and D33.

Projection unit 910 is configured to project visible light and invisible light simultaneously. For example, in the case of a projection method using one liquid crystal panel, the pixels of the liquid crystal panel are provided with a filter that transmits only invisible light, for example, specific infrared light in addition to red, blue and green. The image signal that has been subjected to shake correction is displayed in the portion where the red, blue, and green filters are arranged, and the pattern image is displayed in the non-visible light filter portion. For example, in the case of a projection method that divides a light source, it may be a method in which four liquid crystal panels are combined by a dichroic mirror by splitting into invisible light, for example, specific infrared light in addition to red, blue and green. . The image signal corrected for shaking is displayed on the liquid crystal panel irradiated with red blue green light, and the pattern image is displayed on the liquid crystal panel irradiated with invisible light. In the method of projecting by splitting light, the light source is separated into red, blue, green, and infrared light by a dichroic mirror and transmitted through a liquid crystal panel provided in each. At this time, the image signal generated by the shake correction unit is displayed on the liquid crystal panel that transmits light emitted from the red, blue, and green light sources, and the pattern image is transmitted through the liquid crystal panel for infrared light.

The imaging unit 920 is configured by appropriately using an image sensor, an amplifier, an AD converter, or the like that includes, for example, an imaging element such as a lens group, an iris, a shutter, a CCD or a CMOS, and photoelectrically converts an optical image received by the image sensor. For example, a filter that transmits only infrared light is disposed in the image sensor. The imaging unit 920 captures an image projected by the image projection apparatus 900 displayed on the projection screen 117 and extracts a pattern image portion projected with infrared light.

The imaging pattern detection unit 122 detects the position of the pattern image and outputs the position information of each pattern as in the first embodiment.

In the control unit 121, as in the first embodiment, the shake correction parameter is calculated from the position information of each correction point and set in the shake correction unit. Here, in this embodiment, since the pattern image is arranged even in the central portion of the projection screen and all the correction points can be measured, the processing for calculating the correction point in the central portion from the correction points in the peripheral portion is not necessary.

As described above, according to the present embodiment, it is possible to perform shake correction with high accuracy without showing an extra image to the viewer. In addition, since correction points can be displayed at the center, the amount of calculation can be reduced and correction can be performed with high accuracy.

It should be noted that the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

110, 500, 700, 800, 900: video projection device,
111: Video signal processing unit, 112: Shake correction unit, 113, 911: Pattern image unit,
114: Projection video composition unit, 115, 910: Projection unit, 117: Screen,
121: control unit, 122: imaging pattern detection unit, 123, 920: imaging unit,
130: position information storage unit, 140: imaging device, 710: position information storage unit,
711: locus analysis unit, 810: shaking speed analysis unit, 820: video storage unit,
821: Video storage unit, 822: Video conversion unit, 823: Correction image storage unit,
824: A corrected image output unit.

Claims

A shake correction unit that generates a shake correction video signal obtained by performing shake correction on the video signal based on the correction parameter;
A projection video synthesizing unit that outputs a projection video signal by superimposing a pattern image having a singular point at a correction point on the shake correction video signal output by the shake correction unit;
A projection unit that projects the projection video signal output by the projection video synthesis unit, an imaging unit that captures the projected video, and outputs a captured video;
An imaging pattern detection unit for detecting a pattern image and positional information of the singular point from the captured video output by the imaging unit;
A position information storage unit that stores position information serving as a reference for the correction point;
A control unit that calculates a shake correction amount and generates a correction parameter of the shake correction unit;
Have
The video projection apparatus, wherein the control unit calculates the correction parameter by comparing position information stored in the position information storage unit with position information detected by the imaging pattern detection unit.
The video projection device according to claim 1,
The reference position information is information in which position information detected by the imaging pattern detection unit in the past is stored in the position information storage unit.
The video projection device according to claim 2,
The video projection apparatus according to claim 1, wherein the information stored in the past position information storage unit is information stored in the position information storage unit obtained for the first time after starting shake correction.
The video projection device according to claim 1,
The image projection apparatus characterized in that the reference position information is a plurality of intermediate positions or average positions of position information detected by the imaging pattern detection unit in the past.
The video projection device according to claim 1,
Providing a means for inputting a distortion image of a projection image at the time of installation of the video projection device;
An image projection apparatus, wherein the distortion image is reflected in the reference position information.
The video projection device according to claim 1, wherein
The location information storage unit is a location information storage unit that stores location information of a plurality of frames,
The control unit predicts a shake direction of a video signal from regularity of positional information of the plurality of frames, and sets a correction parameter for correcting the predicted shake in the shake correction unit.
The video projection device according to claim 1, wherein
The location information storage unit is a location information storage unit that stores location information of a plurality of frames,
The control unit analyzes the shaking speed from the amount of movement of the position information accumulated by the position information accumulating unit, and when it is determined that the shaking speed is fast, the shaking correction is performed at a period earlier than the period of the input video signal. An image projection apparatus that controls the shake correction unit so as to perform.
The video projection device according to claim 1, wherein
The pattern image is a pattern image for invisible light, the projection unit is a projection unit capable of projecting invisible light, and the imaging unit is an imaging unit capable of imaging invisible light. Projection device.
A method of correcting a shake of a projected image of a video projection device,
Project a video signal for projection with a pattern image with a singular point at the correction point superimposed on the video signal,
Capture the projected image,
Detecting the position information of the pattern image and the singular point from the captured video,
A shake correction method comprising: comparing position information serving as a reference for the correction point with the detected position information to calculate a correction parameter, and performing shake correction of the video signal.
The shake correction method according to claim 9,
The shake correction method, wherein the reference position information is the detected position information in the past.
The shake correction method according to claim 9,
A shake correction method characterized by constantly performing shake correction of the video signal.
The shake correction method according to claim 9,
The shake correction method, wherein the pattern image is displayed with visible light outside the display area of the video signal.
The shake correction method according to claim 9,
A shake correction method, wherein the reference position information is set as a distortion image of a projected image when the video projector is installed, and the shake correction is performed based on the reference position.
The shake correction method according to claim 9,
A shake correction method comprising: predicting a shake direction of a video signal from regularity of a plurality of frames of the position information, and performing shake correction based on the predicted shake.
The shake correction method according to claim 9,
Analyzing the speed of shaking from the amount of movement of a plurality of frames of the position information, if it is judged that the speed of shaking is fast, the shaking correction is characterized in that the shaking correction is performed at a period earlier than the period of the input video signal Method.