WO2020170663A1

WO2020170663A1 - Projecting device, and control method and control program for same

Info

Publication number: WO2020170663A1
Application number: PCT/JP2020/001352
Authority: WO
Inventors: 正樹荒井; 晶啓石塚
Original assignee: 富士フイルム株式会社
Priority date: 2019-02-19
Filing date: 2020-01-16
Publication date: 2020-08-27
Also published as: JP7270025B2; JPWO2020170663A1

Abstract

Provided are a projecting device, and a control method and a control program for the same, with which it is possible for a projected image to be corrected in accordance with a projection plane, without being affected by a ghost of a captured image.　A projector (100): successively selects region groups G1, G2 in a specific area R obtained by joining together the region groups G1, G2 comprising regions R1, R2, set in a portion of a projection area AR; projects test light onto the selected region group G1, G2; with the test light being projected, acquires a first test light image, which is a captured image of the region group G1, G2 onto which the test light is being projected, using an image capturing element (6); and generates correction data for correcting at least one of the color or the luminance of an image to be projected onto a screen SC, on the basis of a characteristic difference between the first test light image and the test light, for each region group G1, G2.

Description

Projection apparatus, control method thereof, and control program

The present invention relates to a projection device, a control method thereof, and a control program.

A system that combines an imaging device and a projection device has been proposed. For example, in Patent Document 1, a half mirror that transmits the light from the projection unit to the screen and guides the light from the screen to the imaging unit, and a lens arranged between the half mirror and the projection unit, A projector having this half mirror and a lens arranged between the image pickup unit is described.

Patent Document 2 describes a projector that includes a projection unit and a color imaging element that images a projection surface on which light is projected from the projection unit, in the same housing. In this projector, a captured image obtained by capturing an image of the projection surface with a color image sensor is divided into a plurality of areas, and the color of the projection surface corresponding to the area should be detected based on the image of each area and projected. Image color correction is performed.

Japanese Patent Laid-Open No. 2016-149618 Japanese Patent Laid-Open No. 2006-349792

As described in Patent Document 2, in the case of correcting the projection image according to the state of the projection surface, in order to detect the state of the projection surface (color or pattern presence or absence), a reference image is used. It is necessary to capture an image of the projection surface in the state of being projected on the projection surface. However, in the system described in Patent Document 1 in which imaging and projection are performed using a common optical system, strong light from a light source causes the optical system to project when an image is projected on the projection surface. It will pass. If the projection surface is imaged in this state, strong light for projection may be mixed into the imaging element, and a ghost may occur in the captured image. If a ghost occurs in the captured image, the state of the projection surface cannot be recognized accurately, and the projection image cannot be corrected with high accuracy.

Note that even if the projection optical system and the imaging optical system are not made common, if they are arranged close to each other, a ghost may similarly occur in the captured image.

The present invention has been made in view of the above circumstances, and provides a projection device and a control method and a control program therefor capable of correcting a projection image in accordance with a projection surface without being affected by a ghost of a captured image. The purpose is to

A projection device of the present invention is configured such that an optical system for projecting an image from a display unit onto a projection target object, an image sensor for capturing a projection range of the image by the optical system, and a part or the whole of the projection range. Also, a test light projection control for sequentially selecting the plurality of region groups in a specific range obtained by joining a plurality of region groups including a plurality of regions and projecting test light from the optical system onto the selected region group. And a test light image acquisition unit that acquires a first test light image, which is a captured image of the area group onto which the test light is projected, from the imaging device in a state where the test light is projected, Correction data for correcting at least one of color and brightness of the image to be projected on the projection target is generated based on the characteristic difference between the first test light image for each area group and the test light. And a correction data generating section for performing the correction.

A control method of a projection device according to the present invention includes an optical system for projecting an image from a display unit onto a projection target, an image sensor for capturing a projection range of the image by the optical system, and a part or the whole of the projection range. A test for sequentially selecting the plurality of area groups in a specific range where a plurality of area groups made up of a plurality of areas are set, and projecting a test light from the optical system onto the selected area group. A light projection control step, and a test light image acquisition step of acquiring a first test light image, which is a captured image of the area group onto which the test light is projected, from the image sensor in the state where the test light is projected. And a correction for correcting at least one of the color and the brightness of the image to be projected on the projection target based on the characteristic difference between the first test light image for each area group and the test light. A correction data generating step of generating data.

A control program for a projection apparatus according to the present invention is a control program for a projection apparatus including an optical system that projects an image from a display unit onto a projection target and an image sensor that captures a projection range of the image by the optical system. Then, set to a part or the whole of the projection range, sequentially select the plurality of region groups in a specific range that is a combination of a plurality of region groups consisting of a plurality of regions, for the selected region group And a test light projection control step of projecting test light from the optical system, and, in a state where the test light is projected, a first test light image which is a captured image of the area group onto which the test light is projected. Based on a characteristic difference between the test light image acquisition step of acquiring from the image sensor, the first test light image for each of the area groups, and the test light, the color of the image to be projected on the projection target or And a correction data generating step of generating correction data for correcting at least one of the brightness and the correction data generating step.

According to the present invention, it is possible to provide a projection apparatus, a control method and a control program therefor capable of correcting a projection image that matches a projection surface without being affected by a ghost of a captured image.

FIG. 1 is a schematic diagram showing an external configuration of a projector 100 that is an embodiment of a projection device of the present invention. FIG. 2 is a schematic diagram showing an internal configuration of projector 100 shown in FIG. 1. It is a schematic diagram which shows the structural example of the display part 1 shown in FIG. It is a schematic diagram which shows an example of the screen SC. It is a figure which shows the modification of the setting of the specific range R. It is a schematic diagram which shows the example of arrangement|positioning of the area|region of the specific range R. FIG. 9 is a schematic diagram showing another example of an arrangement example of regions in a specific range R. It is a schematic diagram which shows another example of the example of arrangement|positioning of the area|region of the specific range R. FIG. 4 is a functional block diagram of a system control unit 8 shown in FIG. 3. 6 is a flowchart for explaining correction data generation processing by projector 100 shown in FIG. 1. 9 is a flowchart for explaining a modified example of the correction data generation process by the projector 100 shown in FIG. 1. It is a schematic diagram which shows the example in which two specific ranges R are set. 9 is a flowchart for explaining a modified example of the correction data generation process by the projector 100 shown in FIG. 1.

FIG. 1 is a schematic diagram showing an external configuration of a projector 100 that is an embodiment of a projection device of the present invention. The projector 100 includes a main body 100A and an optical unit 100B supported by the main body 100A. The optical unit 100B may be configured to be detachable from the main body 100A.

FIG. 2 is a schematic diagram showing an internal configuration of the projector 100 shown in FIG. FIG. 3 is a schematic diagram showing a configuration example of the display unit 1 shown in FIG. The projector 100 is configured to project an image on the screen SC, which is a projection target, and at least image a projection range, which is a range on which the image on the screen SC is projected.

As shown in FIG. 2, the projector 100 includes a display unit 1, a projection optical system 2, a common optical system 3 forming an optical system, an optical member 4, an image pickup optical system 5, an image pickup device 6, and a system. And a control unit 8. The display unit 1 and the system control unit 8 are housed in the main body 100A. The projection optical system 2, the common optical system 3, the optical member 4, the imaging optical system 5, and the imaging element 6 are housed in the optical unit 100B.

The display unit 1 displays an image for projection based on the input image data. As shown in FIG. 3, the display unit 1 includes a light source unit 40 and a light modulation element 44.

The light source unit 40 includes a light source 41 that emits white light, a color wheel 42, and an illumination optical system 43. The light source 41 is configured to include a light emitting element such as a laser or an LED (Light Emitting Diode). The color wheel 42 is arranged between the light source 41 and the illumination optical system 43. The color wheel 42 is a disk-shaped member, and is provided with an R filter that transmits red light, a G filter that transmits green light, and a B filter that transmits blue light along the circumferential direction thereof. The color wheel 42 is rotated around an axis, and white light emitted from the light source 41 is time-divided into red light, green light, and blue light, which are guided to the illumination optical system 43. The light emitted from the illumination optical system 43 enters the light modulation element 44.

The light modulation element 44 spatially modulates the light emitted from the illumination optical system 43 based on the image data, and emits the spatially modulated light to the projection optical system 2.

The display unit 1 shown in FIG. 3 is an example in which a DMD (Digital Micromirror Device) is used as the light modulation element 44. It is also possible to use a (Systems) element, a liquid crystal display element, or the like.

The display unit 1 may display an image using a self-luminous organic EL (electro-luminescence) display element and cause the displayed image to enter the projection optical system 2. Alternatively, a device that displays an image by scanning laser light may be used.

The projection optical system 2 is a system to which light from the display unit 1 is incident, and is composed of, for example, a relay optical system including at least one lens. The light that has passed through the projection optical system 2 enters the optical member 4, is reflected by the optical member 4, and enters the common optical system 3.

The common optical system 3 projects the light that has passed through the projection optical system 2 onto the screen SC and forms an image of a subject on the screen SC side, and is configured by, for example, a relay optical system.

In the example of FIG. 1, the common optical system 3 includes a lens group 31 including at least one lens, an optical member 32, a lens group 33 including at least one lens, and a lens group 34 including at least one lens. Equipped with. The lens group 31, the optical member 32, the lens group 33, and the lens group 34 are arranged on the optical path in this order from the screen SC side.

The optical member 32 is a member for bending the optical path of the common optical system 3, and for example, a half mirror, a beam splitter, or a polarizing member is used.

The light from the display unit 1 that has passed through the projection optical system 2 is reflected by the optical member 4 and enters the lens group 34. The lens group 34 forms an intermediate image formed by the light from the display unit 1 at a position IG between the

lens groups

33 and 34. The image forming position of the intermediate image is not limited to this.

The intermediate image passes through the lens group 33, enters the optical member 32, is reflected by the optical member 32, and enters the lens group 31. The intermediate image incident on the lens group 31 is projected toward the screen SC and becomes a projected image. A range where the light is projected from the common optical system 3 on the screen SC is called a projection range of the screen SC.

The subject light that has entered the lens group 31 from the screen SC side passes through the lens group 31, is reflected by the optical member 32, and enters the lens group 33. The lens group 33 forms an intermediate image formed by the subject light at a position IG between the lens group 33 and the lens group 34.

This intermediate image passes through the lens group 34, enters the optical member 4, passes through the optical member 4, and enters the imaging optical system 5.

The image pickup optical system 5 is for forming an intermediate image formed at the position IG on the image pickup element 6. The image pickup optical system 5 is arranged in front of the image pickup device 6 and focuses the subject light transmitted through the optical member 4 to form an image on the image pickup device 6. The imaging optical system 5 is configured to include at least one lens.

As the image sensor 6, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor is used.

The optical member 4 is composed of, for example, a half mirror, a beam splitter, or a polarizing member. The optical member 4 reflects the light from the display unit 1 that has passed through the projection optical system 2 and guides it to the common optical system 3, and also makes the subject light imaged at the position IG by the lens group 33 of the common optical system 3. The intermediate image based on the image is guided to the image pickup device 6 through the image pickup optical system 5. In this way, the image pickup device 6 picks up an image projection range of the common optical system 3 through the common optical system 3 for projecting the image from the display unit 1 on the screen SC.

The system control unit 8 controls the light incident on the projection optical system 2 from the display unit 1 and the imaging control of a subject by the image sensor 6, and includes a processor and a ROM (Read Only Memory) that constitutes a storage unit. And a RAM (Random Access Memory).

As a processor, a programmable logic device (CPU), which is a general-purpose processor that executes programs to perform various processes, that can change the circuit configuration after manufacturing such as CPU (Central Processing Unit) and FPGA (Field Programmable Gate Array) ( Included is a dedicated electrical circuit that is a processor having a circuit configuration specifically designed to execute a specific process such as Programmable Logic Device (PLD) or ASIC (Application Specific Integrated Circuit).

More specifically, the structure of these processors is an electric circuit in which circuit elements such as semiconductor elements are combined.

The processor of the system control unit 8 may be configured by one of the various processors described above, or may be a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA). Combination).

FIG. 4 is a schematic diagram showing an example of the screen SC. In FIG. 4, the projection range AR of the light projected from the common optical system 3 is shown by a broken line. Further, in FIG. 4, a state in which test light (for example, white light) is projected on the entire projection range AR by the projector 100 (in other words, a white image is displayed on the display unit 1 and the white image is projected on the projection range AR). A captured image IM obtained by capturing an image of the projection range AR by the image sensor 6 is shown.

As shown in FIG. 4, the captured image IM includes a ghost generation area GS in which a ghost is generated due to the influence of the test light projected on the projection range AR.

The captured image IM and the projection range AR shown in FIG. 4 correspond to each other. Specifically, each divided area when the projection range AR is evenly divided by the total number of pixels of the captured image IM corresponds to each pixel of the captured image IM. Then, in the projection range AR, there is a region GSa corresponding to the ghost occurrence region GS of the captured image IM. The ghost that occurs in the ghost occurrence region GS of the captured image IM is generated by the light being projected from the common optical system 3 to this region GSa. That is, it can be said that the area GSa is the cause of the ghost.

In the projector 100, the range surrounding the area GSa in the projection range AR is preset as the specific range R. Below, the range other than the specific range R in the projection range AR is called the non-specific range NR. Note that the specific range R is a range surrounding the region GSa and wider than the region GSa in the example of FIG. 4, but is not limited to this. For example, the region GSa may be set as it is as the specific range R. Further, as shown in FIG. 5, a partial range of the area GSa may be set as the specific range R. Since the specific range R is determined based on the position of the ghost occurrence area GS corresponding to the area GSa, it can be said to be a range based on the ghost occurrence area GS.

Further, in the projector 100, the specific range R is set as a range obtained by joining a plurality of area groups including a plurality of areas.

FIG. 6 is a schematic diagram showing an arrangement example of regions in the specific range R. The specific range R shown in FIG. 6 includes a region group G1 made up of eight regions R1 arranged two-dimensionally in a first direction X and a second direction Y orthogonal to the first direction X, a first direction X and a first region X. It is a range obtained by joining together a region group G2 made up of eight regions R2 arranged two-dimensionally in the two directions Y. The regions R1 and R2 are arranged in a checkered pattern.

FIG. 7 is a schematic diagram showing another example of the arrangement of the areas of the specific range R. The specific range R shown in FIG. 7 is a range obtained by joining a region group G1 made up of eight regions R1 arranged two-dimensionally and a region group G2 made up of eight regions R2 arranged two-dimensionally. The fact is the same as in FIG. However, unlike FIG. 6, four regions R2 arranged in the second direction Y and four regions R1 arranged in the second direction Y are arranged alternately in the first direction X. There is.

FIG. 8 is a schematic diagram showing still another example of the arrangement example of the areas of the specific range R. The specific range R shown in FIG. 8 includes a region group G1 made up of four regions R1 arranged in a grid pattern, a region group G2 made up of four regions R2 arranged in a grid pattern, and 4 arranged in a grid pattern. The range is a range obtained by joining a region group G3 including one region R3 and a region group G4 including four regions R4 arranged in a grid pattern. It should be noted that the configuration example of the specific range R described so far is an example, and any configuration can be adopted as long as it has a plurality of region groups including a plurality of regions.

FIG. 9 is a functional block diagram of the system control unit 8 shown in FIG. The processor of the system control unit 8 functions as a test light projection control unit 82, a test light image acquisition unit 83, a correction data generation unit 84, and an image projection control unit 85 by executing a program including a control program of the projection device. To do.

The test light projection control unit 82 controls the display unit 1 to project the test light of a predetermined color (for example, white) on the specific range R set in the projection range AR of the screen SC, and to control the screen SC. The control for projecting the test light in the non-specific range NR of the projection range AR is performed.

For example, when projecting the test light only in an arbitrary range of the projection range AR, the test light projection control unit 82 projects the test light only in this arbitrary range in the projection range AR, and this arbitrary range in the projection range AR. The image data for test is generated so that the test light is not projected to the area other than the range of, and the image data for test is input to the light modulation element 44. As a result, the test light is projected only on an arbitrary range in the projection range AR.

When projecting the test light on the specific range R, the test light projection controller 82 includes k (k is a natural number of 2 or more) region groups Gn (n is 1, 2,...・K) is sequentially selected, and control is performed to project the test light on the selected region group Gn. That is, the control is performed such that the test light is projected with the timing shifted for each region group Gn.

The test light image acquisition unit 83 causes the image sensor 6 to capture an image of the projection range AR in a state where the test light is projected onto each region group Gn of the specific range R. Then, the first test light image, which is the captured image of the portion of the region group Gn onto which the test light is projected, is acquired from the image sensor 6 among the captured images obtained by this imaging.

Further, the test light image acquisition unit 83 causes the imaging element 6 to capture an image of the projection range AR while the test light is projected on the non-specific range NR. Then, the second test light image, which is a picked-up image of the portion of the non-specific range NR onto which the test light is projected, is obtained from the image pickup element 6 among the picked-up images obtained by this image pickup.

The correction data generation unit 84 generates a stitched image obtained by stitching the k first test light images and the second test light images acquired by the test light image acquisition unit 83. The stitched image corresponds to a captured image obtained by projecting the test light on the entire projection range AR and capturing the projection range AR.

Then, the correction data generation unit 84 causes the test light projection control unit 82 to project the stitched image and the first test light image and the second test light image onto the entire projection range AR. The correction data for correcting the color or the brightness of the image to be projected on the screen SC is generated based on the characteristic difference between the test image consisting of (i.e., the white image projected on the entire projection range AR). The generated correction data is stored in the ROM of the system control unit 8.

The characteristic difference between the stitched image and the test image is specifically the difference in color or brightness between the images.

The correction data generation unit 84 compares, for example, the test image and the stitched image, recognizes an area where the color difference is equal to or larger than a predetermined value as a pattern within the projection range AR, and projects the image on the pattern portion. In this case, color correction data for making this pattern hard to see is generated. Alternatively, the correction data generation unit 84 compares the test image and the stitched image, recognizes an area where the difference in brightness is equal to or more than a predetermined value as a pattern within the projection range AR, and projects the image on the pattern portion. In this case, the brightness correction data for making this pattern hard to see is generated. The correction data generation unit 84 may generate both the brightness correction data and the color correction data and store them in the ROM.

The image projection controller 85 corrects the color and/or the brightness of the image data to be input to the light modulation element 44 based on the correction data stored in the ROM, and the corrected image data is corrected by the light modulation element 44. The image based on the corrected image data is projected on the screen SC. By the correction processing by the image projection control unit 85, even if there is a pattern on the screen SC, the image can be visually recognized in the state where the pattern is erased.

FIG. 10 is a flowchart for explaining correction data generation processing by the projector 100 shown in FIG. First, the test light projection controller 82 projects the test light on the non-specific range NR (step S1).

In a state where the test light is projected on the non-projection range NR in step S1, the test light image acquisition unit 83 controls the image sensor 6 to capture an image of the screen SC, and the captured image output from the image sensor 6 is controlled. A captured image IM(NR), which is an image of a portion of the non-specific range NR, is acquired (step S2).

Next, the test light projection control unit 82 sets n to 1 (step S3), selects the region group Gn of the specific range R, and projects the test light on the selected region group Gn (step S4).

In a state where the test light is projected on the area group Gn in step S4, the test light image acquisition unit 83 controls the image sensor 6 to capture an image of the screen SC, and among the captured images output from the image sensor 6. The captured image IM(Gn), which is the image of the portion of the region group Gn selected in step S4, is acquired (step S5).

After that, if n=k is not satisfied (step S6: NO), the test light projection control unit 82 increments the value of n by 1 and performs the processes of steps S4 and S5. If so (step S6: YES), the process proceeds to step S8.

In step S8, the correction data generation unit 84 joins the k captured images IM(Gn) obtained in the process of step S5 with the captured image IM(NR) obtained in step S2. Generate a combined image. Then, the correction data generation unit 84 generates correction data based on the characteristic difference between this stitched image and the test image based on the test light projected on the entire projection range AR by the processing of step S1 and step S4. To do.

As described above, in the projector 100, the test light is sequentially projected onto each region group Gn in the specific range R of the projection range AR, and the sequentially projected test lights are imaged by the image sensor 6 to obtain k pieces. A first test light image of is obtained. When the test light is projected on a part of the specific range R, the possibility that strong light from the display unit 1 leaks into the image sensor 6 is reduced as compared with the case where the test light is projected on the entire specific range R. it can. Therefore, by repeatedly performing the process of projecting the test light on only a part of the specific range R and acquiring the first test light image, it is possible to suppress the occurrence of a ghost caused by projecting the test light on the specific range R. You can

On the other hand, for the non-specific range NR, image pickup is performed with the test light projected onto the entire area, and the second test light image data is acquired. The non-specific range NR is a range in which even if the test light is projected on the whole, it does not affect the generation of the ghost. Therefore, it is possible to obtain the second test light image data that is not affected by the ghost by only performing the test light and the imaging once.

Then, by joining the k pieces of the first test light image data and the second test light image data thus obtained, the projection range AR in the state where the test light is projected on the entire projection range AR The captured image of can be obtained in a state in which the ghost is suppressed. Therefore, the correction data can be generated with high accuracy, and the visibility of the projected image can be improved even when the color, brightness, or the like in the projection range AR is not uniform.

Further, as shown in FIGS. 6 to 8, the plurality of region groups forming the specific range R are each formed of a plurality of regions. Therefore, for example, the number of times of this processing can be reduced as compared with the case where the processing such as projecting the test light onto that area and imaging in that state are repeated for every area that constitutes the specific range R. Further, since the test light is projected and imaged only once in the non-specific range NR, the processing can be completed faster than in the specific range R. As a result, the time required to generate the correction data can be shortened, the time until the image desired by the user can be projected can be shortened, and the satisfaction of the product can be improved.

In the configuration example of the specific range R shown in FIG. 8, the four regions forming each region group are arranged separately. In such a configuration, the test light projected on an arbitrary area included in the area group and the test light projected on another area included in the area group physically pass through the optical system. become. Therefore, generation of a ghost when the test light is projected onto each area group can be suppressed more than in the configurations of FIGS. 6 and 7. Therefore, the state of the specific range R can be recognized more accurately.

In the configuration example of the specific range R shown in FIG. 6, although the regions in each region group are not separated from each other, the contact portion between the regions is smaller than that in the configuration example of FIG. 7. Therefore, generation of a ghost when projecting the test light on each area group can be suppressed more than in the configuration example of FIG. 7.

FIG. 11 is a flowchart for explaining a modified example of the correction data generation process by the projector 100 shown in FIG. The flowchart shown in FIG. 11 is the same as FIG. 10 except that step S1 is changed to step S1a, step S2 is changed to step S2a, and step S8 is changed to step S8a and step S8b. 11, the same processes as those in FIG. 10 are designated by the same reference numerals and the description thereof will be omitted. First, the test light projection controller 82 projects the test light on the entire projection range AR (step S1a).

In the state where the test light is projected on the projection area AR in step S1a, the test light image acquisition unit 83 controls the image sensor 6 to capture an image of the screen SC, and of the captured images output from the image sensor 6, A captured image IM(AR) which is an image of a portion of the projection range AR of is acquired (step S2a). The captured image IM(AR) constitutes a third test light image. After step S2a, the processes of steps S3 to S7 are performed, and when the determination in step S6 is YES, the process of step S8a is performed.

In step S8a, the correction data generation unit 84 corrects the captured image IM(Gn) acquired in step S5 based on the image of the specific range R in the captured image IM(AR) acquired in step S2a. Specifically, the contour correction of the seamed image of the captured image IM(Gn) is performed based on the binarized image obtained by binarizing the image of the specific range R in the captured image IM(AR).

After that, the correction data generation unit 84 tests the image obtained by stitching the image of the non-specific range NR in the captured image IM(AR) and the stitched image of the captured image IM(Gn) after the correction in step S8a. The correction data is generated based on the characteristic difference from the light (step S8b). By doing so, the accuracy of generating the correction data can be further improved.

Up to this point, an example has been described in which there is one specific range R set in the projection range AR, but multiple specific ranges R may be set. For example, when the captured image IM of the projection range AR in the state where the test light is projected on the projection range AR includes a plurality of ghost occurrence regions, the specific range R is set in correspondence with each of the plurality of ghost occurrence regions. Will be.

FIG. 12 is a schematic diagram showing an example in which two specific ranges R are set. The projection range AR shown in FIG. 12 includes an area GSa1 corresponding to one ghost occurrence area GS1 in the captured image IM and an area GSa2 corresponding to another ghost occurrence area GS2 in the captured image IM. Therefore, two specific ranges R including the specific range R including the region GSa1 and the specific range R including the region GSa2 are set as the projection range AR.

As described above, when a plurality of specific ranges R are set, the processes of steps S3 to S7 in the flowchart shown in FIG. 10 may be sequentially performed for each of the plurality of specific ranges R. , Steps S3 to S7 may be performed in parallel for each of the plurality of specific ranges R.

By performing the processing of steps S3 to S7 for each of the plurality of specific ranges R in parallel, it is possible to shorten the time until the correction data is generated. On the other hand, when the processes of step S3 to step S7 are sequentially performed on each of the plurality of specific ranges R, the occurrence of ghost can be further suppressed and the accuracy of the correction data can be improved.

Further, as shown in FIG. 12, when a plurality of specific ranges R are set, a ghost occurrence area corresponding to each of the plurality of specific ranges R (in the example of FIG. 12, the ghost occurrence area GS1 and the ghost occurrence area GS1 It is preferable to determine the number of region groups in each specific range R based on the intensity of the ghost generated in the generation region GS2).

For example, when the intensity of the ghost generated in the ghost occurrence region GS1 is higher than the intensity of the ghost generated in the ghost occurrence region GS2, the number of region groups forming the specific range R based on the ghost occurrence region GS1 is set to , The number of regions forming the specific range R based on the ghost occurrence region GS2. For example, the specific range R based on the ghost generation area GS1 has the configuration shown in FIG. 9, and the specific range R based on the ghost generation area GS2 has the configuration shown in FIG.

As described above, with respect to the specific range R corresponding to the ghost generation region having high ghost intensity, the number of regions is increased to reduce the range in which the test light is projected at one projection of the test light. Therefore, it is possible to effectively suppress the generation of a ghost in the ghost generation area corresponding to the specific range R.

The size of the ghost occurrence area in the captured image IM described above may change depending on the brightness or color of the test light projected on the projection range AR. For example, when the test light has a high brightness, ghosts are more likely to occur than when the test light has a low brightness, so that the size of the ghost occurrence region becomes large. Further, if the color of the test light is different, the size may change even in the same ghost generation region due to the difference in the wavelength of the light.

On the other hand, the projector 100 can adjust the brightness or color (saturation) of an image when projecting the image. For example, the user starts the projector 100, operates a button or the like to specify the brightness or color of the projected image to a desired value, and then performs an image projection start operation. Therefore, the brightness or color of the test light used to generate the correction data also needs to be the brightness or color specified by the user. As described above, when the brightness or color of the test light changes, the position and size of the ghost occurrence area may also change. Therefore, it is preferable to store the information of the specific range R in the ROM of the system control unit 8 in association with all the intensities or colors that can be set as the test light.

FIG. 13 is a flowchart for explaining a modified example of the correction data generation process by the projector 100 shown in FIG. The flowchart shown in FIG. 13 is the same as FIG. 10 except that steps S11 to S13 are added before step S1. 13, the same processes as those in FIG. 10 are designated by the same reference numerals, and the description thereof will be omitted.

First, the test light projection control unit 82 acquires setting information of the brightness or color of the image designated by the user (step S11). Next, the test light projection control unit 82 reads out information of the specific range R corresponding to the intensity or color designated by the user based on this setting information, and sets this specific range R as the projection range AR (step S12). Next, the test light projection controller 82 sets the intensity or color of the test light used after step S1 to the same as the brightness or color designated by the user based on this setting information (step S13). After that, the processing from step S1 onward is started.

As described above, by setting the optimum specific range R in accordance with the projection condition (luminance or color) of the image from the projector 100, the accuracy of the correction data can be increased.

In the description so far, it is assumed that the specific range R is set as a part of the projection range AR based on one or more ghost occurrence areas. However, the entire projection range AR may be set as the specific range R. The operation of the system controller 8 in this case is such that steps S1 and S2 are deleted in FIG. 10, and in step S8, correction is performed based on the characteristic difference between the image obtained by stitching the captured images IM(Gn) and the test light. The data will be generated.

Even in this case, in the state where the test light is projected on each area group, the strong light from the display unit 1 leaks into the image sensor 6 as compared with the case where the test light is projected on the entire projection range AR. The possibility can be reduced. Therefore, it is possible to prevent a ghost from occurring in the captured image and improve the accuracy of the correction data.

Further, in this case, in the ROM of the system control unit 8, the number of ghost occurrence areas that occur in the captured image IM and the ghosts that occur in each ghost occurrence area for all the brightness or colors that can be set as test light. The information of the strength of is stored. Then, when generating the correction data, the information on the number of ghost occurrence areas and the ghost intensity corresponding to the intensity or color of the image designated by the user is acquired, and the specific range R is determined based on the information. It is advisable to change the number of regions to be configured.

For example, when the number of ghost occurrence areas exceeds a threshold value or when the ghost intensity exceeds a threshold value, the number of area groups is set to be smaller than when the number of ghost occurrence areas is less than or equal to the threshold value or when the ghost intensity is less than or equal to the threshold value. increase. With this, it is possible to suppress the generation of ghosts regardless of the image projection condition, and it is possible to improve the accuracy of generating the correction data.

In the projector 100, the image sensor 6 images the projection range AR through the common optical system 3. However, the present invention is a projector (in other words, projection optics) in which the image pickup device 6 images the projection range AR through an optical system different from the common optical system 3 (an optical system arranged close to the common optical system 3 and outside). It is also applicable to a projector in which the system and the imaging optical system are provided independently.

As described above, the following items are disclosed in this specification.

(1)
An optical system that projects an image from the display unit onto a projection target,
An image sensor for capturing the projection range of the image by the optical system,
A plurality of region groups, which are set to a part or the whole of the projection range, are sequentially selected in a specific range obtained by joining a plurality of region groups, and a test light is selected for the selected region group. A test light projection control unit for projecting from the optical system,
In a state where the test light is projected, a test light image acquisition unit that acquires a first test light image, which is a captured image of the area group onto which the test light is projected, from the image sensor,
Based on the characteristic difference between the first test light image for each area group and the test light, correction data for performing correction of at least one of the color or the brightness of the image to be projected on the projection target object. And a correction data generation unit that generates the projection data.

(2)
The projection device according to (1),
The plurality of regions forming the region group are two-dimensionally arranged in a first direction and a second direction orthogonal to the first direction, and the region adjacent to the first direction and the second A projection device in which the area of the other area group is arranged next to the direction.

(3)
The projection device according to (1),
The projection device in which the plurality of regions forming the region group are arranged apart from each other.

(4)
The projection device according to any one of (1) to (3),
The specific range may include a ghost due to the test light in a captured image of the projection range obtained by capturing the image with the image sensor with the test light projected onto the entire projection range from the optical system. A projection device that is a range based on a ghost generation area and is a part of the projection range.

(5)
(4) The projection device as described above,
The projection device, wherein the specific range is at least a part of a region corresponding to the ghost generation region or a range wider than the region.

(6)
The projection apparatus according to (4) or (5),
The test light projection control unit further projects the test light from the optical system onto a non-specific range other than the specific range of the projection range,
The test light image acquisition unit further captures the second test light image which is a captured image of the non-specific range of the projection range in a state where the test light is projected on the non-specific range. Get from the element,
A projection device in which the correction data generation unit generates the correction data based on a characteristic difference between the test light and the first test light image and the second test light image for each area group.

(7)
The projection apparatus according to (4) or (5),
The test light projection control unit further projects the test light from the optical system onto the entire projection range,
The test light image acquisition unit further acquires a third test light image, which is a captured image of the projection range, from the image sensor in a state where the test light is projected on the entire projection range,
The correction data generation unit corrects the first test light image for each of the region groups based on an image of the specific range of the third test light image, and corrects the first test light image. A projection device that generates the correction data based on a characteristic difference between an image other than the specific range of the optical image and the third test light image and the test light.

(8)
The projection device according to any one of (4) to (7),
When there are a plurality of ghost occurrence areas, the specific range is set corresponding to each of the plurality of ghost occurrence areas, and the number of the area groups set for each of the specific ranges is the specific range. A projection apparatus that is determined based on the intensity of the ghost that can occur in the ghost generation area corresponding to.

(9)
The projection apparatus according to any one of (4) to (8),
When there are a plurality of ghost occurrence regions, the specific range is set corresponding to each of the plurality of ghost occurrence regions,
The test light projection control unit is a projection device which projects the test light onto the plurality of specific ranges in parallel.

(10)
The projection device according to any one of (4) to (9),
The information of the specific range is provided with a storage unit that stores the different brightness or color of the test light in association with each other,
The test light projection control unit sets the brightness or color of the test light projected on the specific range to the brightness or color of the designated image, and sets the specific range corresponding to the set intensity or color to the specified range. A projection device that sets the projection range.

(11)
The projection apparatus according to any one of (1) to (10),
The image pickup device is a projection device that picks up the projection range of the image through the optical system.

(12)
An optical system that projects an image from the display unit onto a projection target,
An image sensor for capturing the projection range of the image by the optical system,
A plurality of region groups, which are set to a part or the whole of the projection range, are sequentially selected in a specific range obtained by joining a plurality of region groups, and a test light is selected for the selected region group. And a test light projection control step for projecting from the optical system,
In a state where the test light is projected, a test light image acquisition step of acquiring from the image sensor a first test light image that is a captured image of the area group onto which the test light is projected,
Based on the characteristic difference between the first test light image for each area group and the test light, correction data for performing correction of at least one of the color or the brightness of the image to be projected on the projection target object. A method of controlling a projection device, comprising: a correction data generation step of generating the correction data.

(13)
A method for controlling a projection device according to (12),
The plurality of regions forming the region group are two-dimensionally arranged in a first direction and a second direction orthogonal to the first direction, and the region adjacent to the first direction and the second A method for controlling a projection device, wherein the area of another area group is arranged next to the direction.

(14)
A method for controlling a projection device according to (12),
A method of controlling a projection device, wherein the plurality of regions forming the region group are arranged apart from each other.

(15)
A method for controlling a projection device according to any one of (12) to (14), comprising:
The specific range may include a ghost due to the test light in a captured image of the projection range obtained by capturing the image with the image sensor with the test light projected onto the entire projection range from the optical system. A method for controlling a projection device, which is a range based on a ghost generation area and which is a part of the projection range.

(16)
A method for controlling a projection device according to (15), comprising:
The method for controlling a projection device, wherein the specific range is at least a part of a region corresponding to the ghost generation region or a range wider than the region.

(17)
A method for controlling a projection device according to (15) or (16),
The test light projection control step further projects the test light from the optical system onto a non-specific range of the projection range excluding the specific range,
The test light image acquisition step further captures a second test light image that is a captured image of the non-specific range of the projection range in a state where the test light is projected on the non-specific range. Get from the element,
The correction data generating step is a control of a projection device that generates the correction data based on a characteristic difference between the test light and the first test light image and the second test light image for each area group. Method.

(18)
A method for controlling a projection device according to (15) or (16),
The test light projection control step further projects the test light from the optical system onto the entire projection range,
The test light image acquisition step, further, in a state in which the test light is projected on the entire projection range, to obtain a third test light image is a captured image of the projection range from the image sensor,
The correction data generation step corrects the first test light image for each of the area groups based on an image of the specific range of the third test light image, and corrects the first test light image. A method for controlling a projection device, wherein the correction data is generated based on a characteristic difference between an image other than the specific range of the optical image and the third test light image and the test light.

(19)
A method for controlling a projection device according to any one of (15) to (18),
When there are a plurality of ghost occurrence areas, the specific range is set corresponding to each of the plurality of ghost occurrence areas, and the number of the area groups set for each of the specific ranges is the specific range. A method for controlling a projection device, which is determined based on the intensity of the ghost that can occur in the ghost generation region corresponding to.

(20)
A method for controlling a projection device according to any one of (15) to (19), comprising:
When there are a plurality of ghost occurrence regions, the specific range is set corresponding to each of the plurality of ghost occurrence regions,
The test light projection control step is a method for controlling a projection device, wherein the test light is projected onto the plurality of specific ranges in parallel.

(21)
A method for controlling a projection device according to any one of (15) to (20),
Information of the specific range is stored in the storage unit in association with different brightness or color of the test light,
The test light projection control step sets the brightness or color of the test light projected on the specific range to the brightness or color of the specified image, and sets the specific range corresponding to the set intensity or color to the specified range. A method for controlling a projection device that sets a projection range.

(22)
A method for controlling a projection device according to any one of (12) to (21),
The said image pick-up element is a control method of the projection apparatus which images the said projection range of the said image through the said optical system.

(23)
A control program for a projection device, comprising: an optical system that projects an image from a display unit onto a projection target; and an image sensor that captures a projection range of the image by the optical system,
A plurality of region groups, which are set to a part or the whole of the projection range, are sequentially selected in a specific range obtained by joining a plurality of region groups, and a test light is selected for the selected region group. And a test light projection control step for projecting from the optical system,
In a state where the test light is projected, a test light image acquisition step of acquiring from the image sensor a first test light image that is a captured image of the area group onto which the test light is projected,
Based on the characteristic difference between the first test light image for each area group and the test light, correction data for performing correction of at least one of the color or the brightness of the image to be projected on the projection target object. A control program of a projection device for causing a computer to execute a correction data generation step to be generated.

Although various embodiments have been described with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It is obvious to those skilled in the art that various changes or modifications can be conceived within the scope described in the claims, and naturally, these also belong to the technical scope of the present invention. Understood. Further, the constituent elements in the above-described embodiments may be arbitrarily combined without departing from the spirit of the invention.

The present application is based on the Japanese patent application (Japanese Patent Application No. 2019-027525) filed on February 19, 2019, the contents of which are incorporated by reference in the present application.

100 projector 100A body part 100B optical unit 1 display part 40 light source unit 41 light source 42 color wheel 43 illumination optical system 44 light modulation element 2 projection optical system 3 common

optical system

31, 33, 34 lens group 32 optical member IG position SC screen 4 Optical member 5 Imaging optical system 6 Imaging device 8 System control unit 82 Test light projection control unit 83 Test light image acquisition unit 84 Correction data generation unit 85 Image projection control unit AR Projection range R Specific range R1, R2 Regions G1, G2, G3 , G4 region group NR non-specific range IM captured images GS, GS1, GS2 ghost occurrence regions GSa, GSa1, GSa2 regions

Claims

An optical system that projects an image from the display unit onto a projection target,
An image sensor for capturing the projection range of the image by the optical system;
A plurality of region groups, which are set to a part or the whole of the projection range, are sequentially selected in a specific range obtained by joining a plurality of region groups, and a test light is selected for the selected region group. A test light projection controller for projecting from the optical system,
In a state where the test light is projected, a test light image acquisition unit that acquires a first test light image, which is a captured image of the region group onto which the test light is projected, from the image sensor,
Based on the characteristic difference between the first test light image for each of the area groups and the test light, correction data for correcting at least one of the color and the brightness of the image to be projected on the projection target object. And a correction data generation unit that generates the projection data.
The projection device according to claim 1, wherein
The plurality of regions constituting the region group are two-dimensionally arranged in a first direction and a second direction orthogonal to the first direction, and the region adjacent to the first direction and the second A projection device in which the area of another area group is arranged next to the direction.
The projection device according to claim 1, wherein
The projection device in which the plurality of regions forming the region group are arranged apart from each other.
The projection device according to any one of claims 1 to 3,
The specific range may include a ghost due to the test light in a captured image of the projection range obtained by capturing the image of the test light from the optical system over the entire projection range. A projection device which is a range based on a ghost generation area and which is a part of the projection range.
The projection device according to claim 4, wherein
In the projection device, the specific range is at least a part of a region corresponding to the ghost generation region or a range wider than the region.
The projection device according to claim 4 or 5, wherein
The test light projection control unit further projects the test light from the optical system onto a non-specific range other than the specific range of the projection range,
The test light image acquisition unit further captures a second test light image which is a captured image of the non-specific range of the projection range in a state where the test light is projected on the non-specific range. Get from the element,
The said correction data production|generation part is a projection apparatus which produces|generates the said correction data based on the characteristic difference of the said test light and the said 1st test light image and said 2nd test light image for every said area group.
The projection device according to claim 4 or 5, wherein
The test light projection control unit further projects the test light from the optical system onto the entire projection range,
The test light image acquisition unit further acquires a third test light image, which is a captured image of the projection range, from the image sensor in a state where the test light is projected on the entire projection range,
The correction data generation unit corrects the first test light image for each of the region groups based on an image of the specific range of the third test light image, and the corrected first test A projection device that generates the correction data based on a characteristic difference between an image other than the specific range of the optical image and the third test light image and the test light.
The projection device according to any one of claims 4 to 7,
When there are a plurality of ghost occurrence areas, the specific range is set corresponding to each of the plurality of ghost occurrence areas, and the number of the area groups set for each specific range is the specific range. A projection apparatus that is determined based on the intensity of the ghost that can occur in the ghost generation area corresponding to.
The projection device according to any one of claims 4 to 8, wherein:
When there are a plurality of ghost occurrence regions, the specific range is set corresponding to each of the plurality of ghost occurrence regions,
The test light projection control unit is a projection device that projects the test light onto the plurality of specific ranges in parallel.
The projection device according to any one of claims 4 to 9,
A storage unit that stores the information in the specific range in association with different brightnesses or colors of the test light,
The test light projection control unit sets the brightness or color of the test light projected on the specific range to the brightness or color of the specified image, and sets the specific range corresponding to the set intensity or color to the specified range. A projection device that sets the projection range.
The projection device according to any one of claims 1 to 10, wherein
The image pickup device is a projection device that picks up the projection range of the image through the optical system.
An optical system that projects an image from the display unit onto a projection target,
An image sensor for capturing the projection range of the image by the optical system;
A plurality of region groups, which are set to a part or the whole of the projection range, are sequentially selected in a specific range obtained by joining a plurality of region groups, and a test light is selected for the selected region group. And a test light projection control step of projecting from the optical system,
In a state where the test light is projected, a test light image acquisition step of acquiring from the image sensor a first test light image that is a captured image of the area group onto which the test light is projected,
Based on the characteristic difference between the first test light image for each of the area groups and the test light, correction data for correcting at least one of the color and the brightness of the image to be projected on the projection target object. A method of controlling a projection device, comprising: a correction data generation step of generating the correction data.
A method for controlling a projection device according to claim 12, wherein
The plurality of regions constituting the region group are two-dimensionally arranged in a first direction and a second direction orthogonal to the first direction, and the region adjacent to the first direction and the second A method for controlling a projection device, wherein the area of another area group is arranged next to the direction.
A method for controlling a projection device according to claim 12, wherein
The method of controlling a projection device, wherein the plurality of regions forming the region group are arranged apart from each other.
A method for controlling a projection device according to any one of claims 12 to 14, wherein
The specific range may include a ghost due to the test light in a captured image of the projection range obtained by capturing the image of the test light from the optical system over the entire projection range. A method of controlling a projection device, which is a range based on a ghost generation region and which is a part of the projection range.
A method for controlling a projection device according to claim 15, wherein
The method for controlling a projection device, wherein the specific range is at least a part of a region corresponding to the ghost occurrence region or a range wider than the region.
A control method for a projection device according to claim 15 or 16, wherein
The test light projection control step further projects the test light from the optical system onto a non-specific range other than the specific range of the projection range,
In the test light image acquisition step, the second test light image, which is a captured image of the non-specific range in the projection range, is further captured in a state where the test light is projected in the non-specific range. Get from the element,
The correction data generating step is a control of a projection device that generates the correction data based on a characteristic difference between the test light and the first test light image and the second test light image for each area group. Method.
A control method for a projection device according to claim 15 or 16, wherein
The test light projection control step further projects the test light from the optical system onto the entire projection range,
The test light image acquisition step further acquires a third test light image, which is a captured image of the projection range, from the image sensor in a state where the test light is projected on the entire projection range,
The correction data generating step corrects the first test light image for each of the area groups based on an image of the specific range of the third test light image, and corrects the first test light image. A method for controlling a projection device, which generates the correction data based on a characteristic difference between an image other than the specific range of the optical image and the third test light image and the test light.
A method for controlling a projection device according to any one of claims 15 to 18, comprising:
When there are a plurality of ghost occurrence areas, the specific range is set corresponding to each of the plurality of ghost occurrence areas, and the number of the area groups set for each specific range is the specific range. A method for controlling a projection device, which is determined based on the intensity of the ghost that can occur in the ghost generation region corresponding to.
A method for controlling a projection device according to any one of claims 15 to 19, comprising:
When there are a plurality of ghost occurrence regions, the specific range is set corresponding to each of the plurality of ghost occurrence regions,
The test light projection control step is a method of controlling a projection device, wherein the test light is projected onto the plurality of specific ranges in parallel.
21. A method for controlling a projection device according to claim 15, comprising:
The information of the specific range is stored in the storage unit in association with each different brightness or color of the test light,
The test light projection control step sets the brightness or color of the test light projected on the specific range to the brightness or color of the designated image, and sets the specific range corresponding to the set intensity or color to the specific range. A method for controlling a projection device that sets a projection range.
A method for controlling a projection device according to any one of claims 12 to 21, comprising:
The said imaging element is a control method of the projection apparatus which images the said projection range of the said image through the said optical system.
A control program for a projection device, comprising: an optical system that projects an image from a display unit onto a projection target; and an image sensor that captures a projection range of the image by the optical system,
A plurality of region groups, which are set to a part or the whole of the projection range, are sequentially selected in a specific range obtained by joining a plurality of region groups, and a test light is selected for the selected region group. And a test light projection control step of projecting from the optical system,
In a state where the test light is projected, a test light image acquisition step of acquiring from the image sensor a first test light image that is a captured image of the area group onto which the test light is projected,
Based on the characteristic difference between the first test light image for each of the area groups and the test light, correction data for correcting at least one of the color and the brightness of the image to be projected on the projection target object. A control program of a projection device for causing a computer to execute a correction data generation step to be generated.