JP7228294B2 - Projector control device, projector, projection system, projection method and program - Google Patents

Description

The present invention relates to a projector control device, a projector, a projection system, a projection method, and a program.

Conventional projector optics are designed with a large aperture. This is to achieve a high resolution while minimizing the loss of light emitted from the light source. However, in such a design, the depth of field of the projector is shallow, so the projection surface is preferably a stationary flat screen.

On the other hand, in recent years, attention has been focused on information systems and artworks using projection light from a projector called projection mapping. In addition, a projector that takes such circumstances into consideration has also been proposed (for example, Patent Document 1).

JP 2019-132939 A

However, in such information systems and art works, it is often necessary to project onto a surface other than a flat screen. The shallow depth of field described in 1. above is a problem. Of course, it is conceivable to adjust the focus according to the projection position and its distance, but if you need to project simultaneously on two or more areas with a difference in distance in the depth direction, or when projecting on a fast-moving object, etc. , at least a part of which is out of focus or out of focus occurs. Even the projector according to Patent Document 1 does not raise or suggest such a problem.

In view of the above circumstances, the present invention provides a projector control device, a projector, a projection system, and a projector control device capable of projecting onto a temporally and/or spatially variable real object even with an optical system having a shallow depth of field. We decided to provide a projection method and a program.

According to one aspect of the present invention, a control device for a projector is provided. This control device includes a light emission control section and an optical system control section. The light emission control section is configured to be able to control the light emission section of the projector so that the light emission section emits light at the first operating frequency. The light emitting unit is configured to emit projection light including an image so as to be able to project onto a real object as a projection target, and the distance between the real object and the light emitting unit is temporally and/or spatially variable. The optical system controller is configured to be able to control the variable focus optical system such that the variable focus optical system changes the focal position of the projection light at the second operating frequency. The first operating frequency is 100 Hz or higher and the second operating frequency is 30 Hz or higher.

In the projector according to the present invention, both the light emitting unit and the varifocal optical system are controlled at an operating frequency higher than the perceptual frequency that humans can visually recognize. projection light can be projected onto a real object.

1 is a schematic diagram of a system according to an embodiment; FIG. FIG. 2 is a block diagram showing the hardware configuration of the control device; FIG. 3 is a functional block diagram showing functions of a control unit in the control device; [Fig. 4A] A mode in which the projection light containing the image 1 is focused on the first object; [Fig. 4B] A mode in which the projection light containing the image 2 is focused on the second object; [Fig. 4C] A mode in which the projection light containing the image 3 is focused on the third object. An embodiment in which all of the objects 51-53 are in focus from the human perspective. [Fig. 6A] A form of an input waveform (upper waveform) for resonating the focal length of the lens with a periodic waveform, and a form of transmitting the projection start timing to the projector as a pulse signal in synchronization with the resonance of the lens (Fig. 6A). [FIG. 6B] A diagram showing a mode in which the projection of the projector is imaged by the sensor. [Fig. 7A] A mode in which the focus position of the projection light is on the front side, [Fig. 7B] A mode in which the focus position of the projection light is on the back side, [Fig. 7C] A mode in which the focus position of the projection light is in the middle Aspect. A mode in which the focal position of the projection light matches everything. Activity diagram showing the flow of the projection method. FIG. 10 is a schematic diagram of a projector according to a modified example;

Embodiments of the present invention will be described below with reference to the drawings. Various features shown in the embodiments shown below can be combined with each other. In particular, in this specification, the term "unit" can include, for example, a combination of hardware resources implemented by circuits in a broad sense and software information processing that can be specifically realized by these hardware resources. . In addition, although various information is handled in this embodiment, this information is represented by the level of the signal value as a binary bit aggregate composed of 0 or 1, and communication and calculation are performed on a circuit in a broad sense. can be

A circuit in a broad sense is a circuit implemented by appropriately combining at least circuits, circuits, processors, memories, and the like. Application Specific Integrated Circuits (ASICs), programmable logic devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and field It includes a programmable gate array (Field Programmable Gate Array: FPGA).

1. Overall Configuration FIG. 1 is a diagram showing an overview of the configuration of a projection system 1 according to this embodiment. Section 1 describes the overall configuration of the projection system 1 .

1-1. Projection system 1
The projection system 1 includes a projector 1 a , a variable focus optical system 23 and a control device 3 . In this embodiment, it is considered that projection mapping is performed by projecting the projection light L onto the targets 51 to 53 using such a projection system 1 . Here, the objects 51 to 53 are assumed to be real objects. That is, since the HUD is a virtual image, only the person can see the image, but the projection mapping according to the present embodiment projects the projection light L onto the real object, so the image can be seen by a plurality of people. These components are further described below.

1-2. Projector 1a
The projector 1a includes a light emitting unit 22, and the light emitting unit 22 is configured to emit projection light L including an image onto a real object, which is a projection target, so that the distance between the real object and the light emitting unit 22 varies with time and time. /or spatially variable. That is, the real object to be projected may move, or the projector 1a may move. Although the external appearance and internal configuration of the projector 1a are not particularly limited, it should be noted that the projector 1a employs a so-called high-speed projector having a high refresh rate (first operating frequency CR1). sea bream. The first operating frequency CR1 is 100 Hz or higher, preferably 250 Hz or higher, more preferably 500 Hz or 1000 Hz. Specifically, for example, 650,675,700,725,750,775,800,825,850,875,900,925,950,975,1000,1025,1050,1075,1100,1125,1150,1175,1200,1225,1250, 1275,1300,1325,1350,1375,1400,1425,1450,1475,1500,1525,1550,1575,1600,1625,1650,1675,1700,1725,1750,1775,1800,1825,1850,1875, 1900, 1925, 1950, 1975, 2000 Hz, and may be in the range between any two of the values exemplified here.

1-3. sensor 21
The sensor 21 is, for example, a camera (visible light, infrared light, etc. can be appropriately adopted) configured to be able to acquire information of the outside world as an image. In particular, the sensor 21 preferably employs a sensor with a high operating frequency (frame rate) for so-called high-speed vision. The frame rate is, for example, 100 fps or higher, preferably 250 fps or higher, more preferably 500 fps or 1000 fps. Specifically, for example, 650,675,700,725,750,775,800,825,850,875,900,925,950,975,1000,1025,1050,1075,1100,1125,1150,1175,1200,1225,1250, 1275,1300,1325,1350,1375,1400,1425,1450,1475,1500,1525,1550,1575,1600,1625,1650,1675,1700,1725,1750,1775,1800,1825,1850,1875, 1900, 1925, 1950, 1975, 2000 fps, and may be in the range between any two of the values exemplified here.

The sensor 21 functions, for example, as a first sensor. The first sensor is configured to obtain first information. Here, the first information is information including at least one of depth positions, shapes, and projected modes of the objects 51 to 53 onto which the projection light L is projected. This will be detailed later.

Moreover, the sensor 21 functions, for example, as a second sensor. A second sensor is configured to obtain the second information. Here, the second information is information including the positions within the angle of view of the objects 51 to 53 on which the projection light L is projected. This will be detailed later.

It is assumed that the angle of view position of the projector 1a and the angle of view position of the sensor 21 have been matched by calibration in advance. Also, although not shown, by adopting a coaxial optical system, it may be implemented as a calibration-free system.

Moreover, the sensor 21 is not limited to a camera, and a measurement sensor or the like capable of acquiring a three-dimensional shape may be adopted as the sensor 21, or a plurality of sensors 21 having different functions may be adopted. Furthermore, depth measurement based on Depth from Defocus using variable focus projection of the present technology may be performed. Depth from Defocus is a technique for obtaining three-dimensional depth information from a two-dimensional image, and is a technique for estimating depth using image blur. In this case, the distance from the sensor 21 to the objects 51 to 53 (real objects) is measured by the depth from defocus, and the varifocal optical system is used so that the projection light L is focused on the objects 51 to 53 located at the measured distances. System 23 may be controlled.

1-4. Variable focus optical system 23
The varifocal optical system 23 is configured such that the focal position FP of the projection light L is variable. Note that the varifocal optical system 23 may be configured to control the focal position FP of the projection light L by changing the shape of the boundary between two types of media (for example, liquids) having different refractive indices. Alternatively, a part of the lens group may be physically moved in parallel as in a compact digital camera or the like. It may be provided with an elastic film enclosed therein, or may be one capable of electrically controlling the refractive index distribution of the medium itself, or may be one utilizing liquid crystal phase control. Furthermore, a fixed optical system or the like (not shown here) may be arranged on the front and back as required.

With such a configuration, the focal position FP can be changed at high speed at an operating frequency (second operating frequency CR2) matching the refresh rate of the projector 1a, which is a high-speed projector. For example, the second operating frequency CR2 is 30 Hz or higher, preferably 100 Hz or higher, more preferably 500 Hz or 1000 Hz. Specifically, for example, 250,300,350,400,450,500,550,600,650,700,750,800,850,900,950,1000,1050,1100,1150,1200,1250,1300,1350,1400,1450, 1500, 1550, 1600, 1650, 1700, 1750, 1800, 1850, 1900, 1950, 2000 Hz, and may be in the range between any two of the values exemplified here.

1-5. Control device 3
FIG. 2 is a block diagram showing the hardware configuration of the control device 3. As shown in FIG. FIG. 3 is a functional block diagram showing functions of the control unit 33 in the control device 3. As shown in FIG. The control device 3 has a communication section 31 , a storage section 32 and a control section 33 , and these constituent elements are electrically connected via a communication bus 30 inside the control device 3 . Regarding the controller 33 in particular, the controller 3 includes a light emission controller 332 and an optical system controller 333 . Each component will be further described below.

<Communication unit 31>
The communication unit 31 preferably uses wired communication means such as USB, IEEE1394, Thunderbolt, wired LAN network communication, etc., but wireless LAN network communication, mobile communication such as LTE/3G, Bluetooth (registered trademark) communication, etc., can be used as necessary. may be included. That is, it is more preferable to implement as a set of these communication means.

For example, the communication unit 31 is preferably configured to be able to communicate with the projector 1a, the sensor 21, and the varifocal optical system 23 according to a predetermined high-speed communication standard. Specifically, the communication unit 31 is configured to be capable of transmitting an image of the projection light L emitted by the light emitting unit 22 of the projector 1a. The communication unit 31 is configured to be able to receive the first information and the second information acquired by the sensor 21 . The communication unit 31 is configured to be able to transmit a control signal for changing the focal position FP of the varifocal optical system 23 .

<Storage unit 32>
The storage unit 32 stores various information defined by the above description. This is, for example, a storage device such as a solid state drive (SSD), or a random access memory (Random Access Memory: RAM) or the like. A combination of these may also be used.

In particular, the storage unit 32 stores the first information and the second information acquired by the sensor 21 and received by the communication unit 31 . The storage unit 32 also stores a sensor control program for controlling the sensor control unit 331 in the control unit 33 to receive the first information and the second information from the sensor 21 . The storage unit 32 stores a light emission control program for the light emission control unit 332 (described later) in the control unit 33 to control the light emission unit 22 so that the light emission unit 22 emits light at the first operating frequency CR1. The storage unit 32 stores an optical system control program for controlling the varifocal optical system 23 so that the varifocal optical system 23 changes the focal position FP of the projection light L at the second operating frequency CR2.

In addition, the storage unit 32 also stores various programs related to the control device 3 that are executed by the calculation unit 334 in the control unit 33 .

<Control unit 33>
The control unit 33 processes and controls overall operations related to the control device 3 . The control unit 33 is, for example, a central processing unit (CPU) (not shown). The control unit 33 implements various functions related to the control device 3 by reading a predetermined program stored in the storage unit 32 . Specifically, it corresponds to a sensor control function, a light emission control function, an optical system control function, an arithmetic function, and the like. That is, by specifically realizing information processing by software (stored in the storage unit 32) by hardware (control unit 33), the sensor control unit 331, the light emission control unit 332, the optical system control unit 333, and a computing unit 334 . That is, the program causes the computer to function as each unit of the control device 3 .

Although FIG. 2 shows a single controller 33, the present invention is not limited to this, and a plurality of controllers 33 may be provided for each function. A combination thereof may also be used. The sensor control section 331, the light emission control section 332, the optical system control section 333, and the calculation section 334 will be described in further detail below.

[Sensor control unit 331]
In the sensor control unit 331, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). The sensor control unit 331 controls to receive the first information and the second information from the sensor 21 . Alternatively, the operating frequency of the sensor 21 may be adjusted, or the first and second information acquired by the sensor 21 may be used to generate a 3D projection image model.

[Light emission control unit 332]
In the light emission control unit 332, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). The light emission control section 332 is configured to be able to control the light emission section 22 in the projector 1a so that the light emission section 22 emits light at the first operating frequency CR1. That is, the light emission control section 332 generates a control signal for controlling the light emission section 22, and this is transmitted to the light emission section 22 in the projector 1a via the communication section 31. FIG. Then, the light emitting unit 22 emits light so as to project the projection light L onto the targets 51 to 53 based on the control signal. In addition, the light emitting unit 22 is configured to emit the projection light L so as to be able to project onto a real object as a projection target, and the distance between the real object and the light emitting unit 22 is temporally and/or spatially variable. That is, the real object to be projected may move, or the projector 1a (light emitting unit 22) may move. The light emission controller 332 is configured to dynamically control the intensity of the projection light according to the surrounding environment. This will be described in more detail together with the optical system control section 333 below.

[Optical system control unit 333]
In the optical system control unit 333, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). The optical system controller 333 is configured to be able to control the varifocal optical system 23 so that the varifocal optical system 23 changes the focal position FP of the projection light L at the second operating frequency CR2. That is, the optical system control section 333 generates a control signal for controlling the focal position FP of the varifocal optical system 23 , which is transmitted to the varifocal optical system 23 via the communication section 31 . Based on this control signal, the varifocal optical system 23 determines the focal position FP so that the objects 51 to 53 and the like are in focus. With respect to such control signals, the variable focus optical system 23 is configured to determine the focal position FP with, for example, a step waveform or a periodic waveform (such as a sine wave) as input.

Here, the optical system control unit 333 controls the varifocal optical system 23 so as to dynamically change the focal position FP to define a plurality of focal positions FP. After that, the light emission control section 332 controls the light emission section 22 so that the projection light L is emitted at the timing when each of the plurality of focal positions FP is focused. More specifically, the light emission control unit 332 causes the first and second projection planes (for example, the target 51 and the target 52, etc.) corresponding to different focal positions FP to project different images at the timing when they are in focus. The light emitting part 22 is controlled so that the light L is emitted.

4A shows a mode in which the focus position FP of the projection light L containing the image 1 is aligned with the target 51, and FIG. 4B shows a mode in which the focus position FP of the projection light L containing the image 2 is aligned with the target 52. FIG. 4C shows a mode in which the focus position FP of the projection light L containing the image 3 is aligned with the object 53 . Further, FIG. 5 shows an embodiment in which all of the objects 51-53 are in focus from the human perspective.

For example, at a certain timing, the varifocal optical system 23 adjusts the focal position FP to the target 51 so that an image is projected onto the target 51, and the light emitting unit 22 emits light in consideration of the angle of view position (FIG. 4A). At another timing, the varifocal optical system 23 adjusts the focal position FP to the target 52 so that an image is projected onto the target 52, and the light emitting unit 22 emits light in consideration of the angle of view position (FIG. 4B). At yet another timing, the varifocal optical system 23 adjusts the focal position FP to the target 53 so that an image is projected onto the target 53, and the light emitting unit 22 emits light in consideration of the angle of view position (FIG. 4C).

Regarding this, if the projection onto the objects 51 to 53 is sequentially switched at a frequency higher than the frequency that human vision can perceive (generally 20 Hz or less), the distance difference in the depth direction as seen from the human eye will increase. This has the advantageous effect of appearing to be projected simultaneously onto some two or more regions of object 51-53 (FIG. 5). Further, the control unit 33 (for example, the light emission control unit 332 and the optical system control unit 333, etc.) controls each of the targets 51 to 53 so that an image with large out-of-focus, medium out-of-focus, and no out-of-focus blur is projected. (The order of the degrees of out-of-focus blurring is arbitrary) and may be positively controlled. Such control has the beneficial effects of appealing to human sensitivity and enhancing human learning.

FIG. 6A shows an aspect of an input waveform (upper waveform) for causing the focal length of the lens to resonate with a periodic waveform, and an aspect of transmitting the projection start timing to the projector 1a as a pulse signal in synchronization with the resonance of the lens. (lower waveform) and FIG. FIG. 6B is a diagram showing a mode in which the sensor 21 captures the projection of the projector 1a.

In FIG. 6B, the time between imaging frames is set to 1 ms, and projection is performed on six planes with different depths. Here, it can be confirmed that an image in focus is captured on each plane. As described above, since the time between imaging frames is 1 ms, it looks to humans as if they were photographed at the same time on each plane.

7A shows a mode in which the focus position FP of the projection light L is on the near side, FIG. 7B shows a mode in which the focus position FP of the projection light L is on the back side, and FIG. 7C shows a mode in which the projection light L is focused on the back side. It shows a mode in which the focus position FP of L is aligned with the center. Furthermore, FIG. 8 shows a mode in which the focal position FP of the projection light L is perfectly aligned.

Here, six planes are installed between 0.3 and 1.5 m in depth. A sine wave with a period of 12 ms is used as an input waveform, and the image of each plane is updated at 166 fps. 7A-7C do not use the technique of the present invention, so only one depth is in focus. In contrast, FIG. 8 uses the technique according to the present embodiment, so all planes are in focus. As can be seen here, by making the input waveform sinusoidal, it is possible to increase the number of depths at which the object (plane) is in focus.

Furthermore, the light emission control section 332 may be configured to control the position or size of the image occupied by the projection light L with respect to the entire angle of view based on the second information. As described above, the second information includes the positions within the angle of view of the objects 51 to 53 on which the projection light L is projected. Based on this information, a desired image can be projected at an appropriate angle of view position.

That is, sensor feedback using the sensor 21 is performed. As described above, since the operating frequencies of the sensor 21, the light emitting unit 22, and the varifocal optical system 23 are all high, when the positions of the targets 51 to 53 dynamically change ) is also valid. More broadly speaking, there is an advantageous effect that projection mapping can be performed not only on the targets 51 to 53 but also on any dynamic target by following them.

Further, the optical system control section 333 may control the varifocal optical system 23 so that the focal position FP is corrected based on the first information. I will supplement this. As described above, the first information includes at least one of the depth position, shape, and projected mode of the targets 51 to 53 onto which the projection light L is projected. That is, by implementing sensor feedback based on these pieces of information that are sequentially acquired by the sensor 21, an appropriate focal position FP at that timing is determined. In particular, it is an effective configuration when the objects 51-53 are dynamic.

Further, the optical system control section 333 may control the varifocal optical system 23 so as to dynamically change the angle of view of the projection light L based on the focal position FP. I will supplement this. Due to the physical properties of projection, even if the projection light L is emitted so as to include the same image, depending on the depth of the objects 51 to 53, the size projected onto each object and its brightness will differ. Although it is possible to adjust these by the light emission control unit 332, there is a problem that the resolution becomes lower especially as the distance increases. Therefore, the optical system control unit 333 uses the varifocal optical system 23 or an optical system dedicated to controlling the angle of view (not shown here), based on the depth of the objects 51 to 53 (in other words, the focal position FP), By dynamically changing the angle of view, the problem of resolution can be resolved.

It should be noted that preferably, the light emission control unit 332 performs control so that the reflected luminance of the projection light L is substantially uniform regardless of the focal position FP.

Specifically, the light emission controller 332 may be configured to dynamically control the intensity of the projection light L based on the focal position FP. Specifically, for example, an object 51 relatively far from the projection system 1 is projected with a relatively strong light intensity, and an object 53 relatively close to the projection system 1 is projected with a relatively weak light intensity. do it. Alternatively, the light emission control section 332 may be configured to dynamically control the emission time of the projection light L based on the focal position FP. Human eyes observe different luminances depending on the light emission time. For example, an object 51 far from the projection system 1 is projected with a relatively long light emission time, and conversely, an object 53 near the projection system 1 is projected with a long light emission time. Projection should be performed with a relatively short light emission time.

[Calculation unit 334]
The calculation unit 334 is a unit in which information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). The calculation unit 334 is configured to be capable of performing various calculations such as image processing and conversion on the projection image, for example. These are not particularly limited, and can be appropriately implemented as necessary.

2. Projection Method Using Projection System 1 In Section 2, an example of a projection method using the projection system 1 will be described. FIG. 9 is an activity diagram showing the flow of the projection method. Here, it is assumed that the positions of the targets 51 to 53 shown in FIG. 1 etc. are variable in time series, and the aim is to simultaneously project appropriate images on the respective targets 51 to 53 as seen from the human eye. and Each activity in FIG. 9 will be described below.

[from here]
(Activity A11)
The sensor 21 acquires information on the targets 51-53. This is for example the first information or the second information. Also, the sensor control section 331 receives these information via the communication section 31 . Further, the sensor control unit 331 generates a 3D model of projection images projected onto the objects 51 to 53 having depth based on at least one of the first and second information.

(Activity A21)
Subsequently, the light emission control unit 332 generates a two-dimensional projection image to be actually projected after considering the depth from the 3D model of the projection image generated by the sensor control unit 331 . Here, an image projected onto the target 51 is generated at a view angle position corresponding to the target 51 . The image projected onto the target 52 or target 53 is not displayed at this timing.

(Activity A22)
Subsequently, the light emission control unit 332 causes the light emitting unit 22 to emit projection light L including the projection image generated in activity A21.

(Activity A31)
On the other hand, in parallel with activity A21, the optical system control unit 333 calculates an appropriate focal position FP for the object 51 based on the first information.

(Activity A32)
Subsequently, the optical system controller 333 controls the focal position FP of the varifocal optical system 23 .

(Activity A23)
An image labeled 1 is projected onto the object 51 in focus, which can be seen by the human eye.

Thereafter, similar processing (activities A11, A21, A22, A23, A31, A32) is executed for the targets 52 and 53 every few frames. As a result, when viewed from the human eye, images that are in focus are simultaneously projected onto the objects 51 to 53 that are different in the depth direction. In addition, sensor feedback from the sensor 21 can cope with situations in which the positions of the objects 51 to 53 dynamically change.
[to this point]

The above is just an example. For example, the objects 51 to 53 may not be dynamic, the object may be single and dynamic, the object may be single and have unevenness, or the object may be curved. may be present. More broadly speaking, according to this projection method, the projection system 1 (projector 1a) is used to project images onto at least one of a dynamic object, an object having unevenness in the depth direction, and two or more objects having different depth directions. , projection mapping can be performed by projecting the projection light L containing the image. That is, since the HUD is a virtual image, only the person can see the image, but the projection mapping according to the present embodiment projects the projection light L onto the real object, so the image can be seen by a plurality of people. In addition, the projection system 1 is mounted on a moving object such as an automobile, an animal including humans, an airship such as an airship or a drone, or a ship, and the projection system 1 is projected onto the surrounding environment from the moving object that is running, flying, or traveling. , the projection mapping according to the present embodiment can be performed.

3. Modifications Section 3 describes a modification of the projection system 1 according to this embodiment. That is, the following aspects may be adopted.

In this embodiment, a single controller 33 in a single controller 3 executes various processes. may be used. Especially in such a case, it should be noted that it is important to synchronize the time series of each component. Of course, a plurality of control units 33 that execute exclusive processing may be used for a single control device 3 .

In the present embodiment, the position of the image included in the projection light L emitted by the light emitting unit 22 is changed when the angle of view positions of the objects 51 to 53 are different. It may be implemented so as to optically control the optical axis direction of the projector 1a.

This embodiment relates to the projection system 1 including the projector 1a (light emitting unit 22), the sensor 21, the varifocal optical system 23, and the control device 3, but instead of this, the sensor 21 and the light emitting A high-speed and high-performance projector 1a (see FIG. 10) that integrally includes the unit 22, the varifocal optical system 23, and the control unit 33 may be implemented.

That is, the projector 1 a includes a light emitting section 22 , a variable focus optical system 23 , a light emission control section 332 and an optical system control section 333 . The varifocal optical system 23 is configured such that the focal position FP of the projection light L is variable. The light emission control section 332 is configured to be able to control the light emission section 22 so that the light emission section 22 emits light at the first operating frequency CR1. The optical system controller 333 is configured to be able to control the varifocal optical system 23 so that the varifocal optical system 23 changes the focal position FP of the projection light L at the second operating frequency CR2. The first operating frequency CR1 is 100 Hz or higher and the second operating frequency CR2 is 30 Hz or higher.

4. Conclusion As described above, according to the present embodiment, even with an optical system having a shallow depth of field, it is possible to implement a projector capable of simultaneously projecting onto two or more areas having a distance difference in the depth direction. can be done.

It may be provided in each aspect described below.
In the control device, the optical system control section controls the variable focus optical system so as to dynamically change the focal position to define a plurality of focal positions; The light emitting unit is controlled so that the projection light is emitted at the timing when the focal position of the projection light is focused on each focal position.
In the control device, the light emission control unit controls the light emission unit so that the projection light including the different images is emitted at the timing onto the first and second projection planes corresponding to the different focal positions. that controls the
In the control device, the varifocal optical system is configured to determine the focal position by inputting a step waveform or a periodic waveform.
In the control device, the optical system control section controls the variable focus optical system so as to dynamically change the angle of view of the projection light based on the focal position.
In the control device, the light emission control section is configured to dynamically control the intensity of the projection light based on the focal position.
In the control device, the light emission control section is configured to dynamically control the light emission time of the projection light based on the focal position.
In the control device, the light emission control section controls the reflected luminance of the projection light so that it is substantially uniform regardless of the focal position.
In the control device, the varifocal optical system is configured to control the focal position of the projection light by changing the shape of a boundary portion between two types of media having different refractive indices.
In the control device, the light emission control section is configured to dynamically control the intensity of the projection light according to the surrounding environment.
A projector, comprising: a light emitting unit; a variable focus optical system; A distance between the real object and the light emitting unit is temporally and/or spatially variable, and the variable focus optical system is configured to vary the focal position of the projection light.
The projector further comprises a first sensor, wherein the first sensor is configured to obtain first information, wherein the first information is an image of an object onto which the projection light is projected. Information including at least one of a depth position, a shape, and a projected aspect, and the optical system control unit controls the variable focus optical system so that the focal position is corrected based on the first information. what to do.
The projector further comprising a second sensor, wherein the second sensor is configured to obtain second information, wherein the second information is an image of an object onto which the projection light is projected. The information including the position within the angle of view, and the light emission control unit is configured to control the position or size of the image occupying the entire angle of view of the projection light based on the second information. What will be done.
A projection system comprising a projector, a varifocal optical system, and the control device, wherein the projector comprises a light emitting unit, and the light emitting unit can project projection light including an image onto a real object as a projection target. The optical system is configured to emit light, the distance between the real object and the light emitting unit is temporally and/or spatially variable, and the variable focus optical system is configured to vary the focal position of the projection light.
A projection method, wherein the projector or the projection system is used to project an image onto at least one of a dynamic object, an object having unevenness in the depth direction, and two or more objects having different depth directions. A method of performing projection mapping by projecting light.
A program for causing a computer to function as each part of the control device.
Of course, this is not the only case.

Finally, while various embodiments of the invention have been described, these have been presented by way of example and are not intended to limit the scope of the invention. The novel embodiment can be embodied in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. The embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

1: Projection system 1a: Projector 21: Sensor 22: Light emitting unit 23: Variable focus optical system 3: Control device 30: Communication bus 31: Communication unit 32: Storage unit 33: Control unit 331: Sensor control unit 332: Light emission control unit 333: Optical system control unit 334: Calculation unit 51: Target 52: Target 53: Target CR1: First operating frequency CR2: Second operating frequency FP: Focal position L: Projection light

Claims (15)

A control device for a projector,
A light emission control unit and an optical system control unit,
The light emission control section is configured to be able to control the light emission section of the projector so that the light emission section emits light at a first operating frequency, wherein the light emission section emits projection light including an image onto a real object to be projected. configured to emit light so as to be projectable on the body, the distance between the real object and the light emitting unit being temporally and/or spatially variable,
The optical system control unit is configured to be able to control the variable focus optical system so that the variable focus optical system changes the focal position of the projection light at a second operating frequency,
the first operating frequency is 100 Hz or higher and the second operating frequency is 30 Hz or higher;
The optical system controller controls the varifocal optical system so as to dynamically change the angle of view of the projection light based on the focal position. A control device for a projector,
A light emission control unit and an optical system control unit,
The light emission control section is configured to be able to control the light emission section of the projector so that the light emission section emits light at a first operating frequency, wherein the light emission section emits projection light including an image onto a real object to be projected. configured to emit light so as to be projectable on the body, the distance between the real object and the light emitting unit being temporally and/or spatially variable,
The optical system control unit is configured to be able to control the variable focus optical system so that the variable focus optical system changes the focal position of the projection light at a second operating frequency,
the first operating frequency is 100 Hz or higher and the second operating frequency is 30 Hz or higher;
The light emission control unit is configured to dynamically control the light emission time of the projection light based on the focal position . In the control device according to claim 1 or claim 2 ,
The optical system control unit controls the variable focus optical system so as to dynamically change the focus position to define a plurality of focus positions,
The light emission control section controls the light emission section so that the projection light is emitted at timing when each of the plurality of focal positions is focused. In the control device according to claim 3 ,
The light emission control unit controls the light emission unit so that the projection light including different images is emitted at the timing onto the first and second projection planes corresponding to the focal positions different from each other. In the control device according to any one of claims 1 to 4 ,
The variable focus optical system is configured to determine the focal position with a step waveform or periodic waveform as an input. In the control device according to any one of claims 1 to 5,
The light emission control unit is configured to dynamically control the intensity of the projection light based on the focal position. In the control device according to any one of claims 1 to 6 ,
The light emission control unit controls the reflection brightness of the projection light so that it becomes substantially uniform regardless of the focal position. In the control device according to any one of claims 1 to 7 ,
The varifocal optical system is configured to control the focal position of the projection light by changing the shape of the boundary between two types of media having different refractive indices. In the control device according to any one of claims 1 to 8 ,
The light emission control unit is configured to dynamically control the intensity of the projection light according to the surrounding environment. being a projector,
A light emitting unit, a variable focus optical system, and the control device according to any one of claims 1 to 9 ,
The light emitting unit is configured to emit projection light including an image so as to be projectable onto a real object that is a projection target, and the distance between the real object and the light emitting unit is temporally and/or spatially variable,
The variable focus optical system is configured to change the focal position of the projection light. The projector according to claim 10 , wherein
further comprising a first sensor;
the first sensor configured to obtain first information;
Here, the first information is information including at least one of a depth position, a shape, and a projected mode of an object on which the projection light is projected,
The optical system control section controls the variable focus optical system so that the focal position is corrected based on the first information. The projector according to claim 10 or claim 11 ,
further comprising a second sensor;
the second sensor is configured to obtain second information;
Here, the second information is information including the position within the angle of view of the object on which the projection light is projected,
The light emission control unit is configured to control the position or size of the image with respect to the entire angle of view of the projection light based on the second information. A projection system,
A projector, a variable focus optical system, and the control device according to any one of claims 1 to 9 ,
The projector includes a light emitting unit, and the light emitting unit is configured to emit projection light including an image onto a real object that is a projection target, and the distance between the real object and the light emitting unit varies temporally and/or. or spatially variable,
The variable focus optical system is configured to change the focal position of the projection light. A projection method,
Using the projector according to any one of claims 10 to 12 or the projection system according to claim 13 , a dynamic object, an object having unevenness in the depth direction, and two different depth directions A method of performing projection mapping by projecting projection light containing an image onto at least one of the above objects. a program,
A device for causing a computer to function as each part of the control device according to any one of claims 1 to 9 .

Images