CN116830566A

CN116830566A - Projection system and method with pixel shift

Info

Publication number: CN116830566A
Application number: CN202280012130.XA
Authority: CN
Inventors: J·P·佩蒂埃拉; M·J·理查兹
Original assignee: Dolby Laboratories Licensing Corp
Current assignee: Dolby Laboratories Licensing Corp
Priority date: 2021-01-29
Filing date: 2022-01-27
Publication date: 2023-09-29

Abstract

A projection system and method thereof comprising: a light source configured to emit light; a spatial light modulator configured to receive light and generate modulated light; a lens configured to spatially fourier transform the modulated light; a filter including an aperture, the filter configured to transmit at least one diffraction order of the modulated light fourier transformed by the lens and to block a remaining portion of the modulated light; and a controller configured to: for each of a plurality of subcycles, the projection system is caused to project an image through the filter, and the image is offset by a fraction of the pixel distance between each of the plurality of subcycles.

Description

Projection system and method with pixel shift

Cross Reference to Related Applications

The present application claims priority from U.S. provisional application No. 63/143,149 and european patent application No. 21154272.5, both filed on 1-month 29 of 2021, each of which is incorporated herein by reference in its entirety.

Background

1. Technical field

The present application relates generally to projection systems and projection methods.

2. Background art

Digital projection systems typically utilize a light source and an optical system to project an image onto a surface or screen. The optical system may include components such as mirrors, lenses, waveguides, optical fibers, beam splitters, diffusers, spatial Light Modulators (SLMs), and the like. The contrast of the projector indicates the brightest output of the projector relative to the darkest output of the projector. The contrast ratio is a quantitative measure of contrast, defined as the ratio of the brightness of the brightest output of the projector to the brightness of the darkest output of the projector. This definition of contrast ratio is also referred to as "static", "native" or "sequential" contrast ratio.

Some projection systems are based on SLMs that implement spatial amplitude modulation, such as Digital Micromirror Device (DMD) chips. The DMD may utilize a two-dimensional array of mirrors that can be controlled to create an image. If it is desired to project an image having a resolution higher than that of the DMD (e.g., an image having a number of pixels greater than the number of mirrors in the DMD), a pixel shifting technique may be used. In one comparative example of a pixel shifting technique (sometimes referred to as "duty"), the system may be controlled to cause the modulator to effectively shift fractional pixels in a set pattern to create the appearance of displaying additional pixels.

Disclosure of Invention

Various aspects of the present disclosure relate to devices, systems, and methods for projection displays, particularly for use with high contrast projection architectures.

In one exemplary aspect of the present disclosure, there is provided a projection system including: a light source configured to emit light; a spatial light modulator configured to receive light and generate modulated light; a lens configured to spatially fourier transform the modulated light; a filter including an aperture, the filter configured to transmit at least one diffraction order of the modulated light fourier transformed by the lens and to block a remaining portion of the modulated light; and a controller configured to: for each of a plurality of subcycles, the projection system is caused to project an image through the filter, and the image is offset by a fraction of the pixel distance between each of the plurality of subcycles.

In another exemplary aspect of the present disclosure, there is provided a projection method including: emitting light by a light source of the projection system; receiving light by a spatial light modulator of the projection system and generating modulated light; performing a spatial fourier transform on the modulated light by a lens of the projection system; transmitting, by a filter of the projection system comprising an aperture, at least one diffraction order of the modulated light after fourier transforming by the lens and blocking a remaining portion of the modulated light; for each of a plurality of subcycles, causing, by a controller of the projection system, the projection system to project an image through the aperture; and shifting, by the controller, the image by a portion of the pixel distance between each of the plurality of subcycles.

In this way, various aspects of the present disclosure provide for the display of images with high dynamic range, high contrast ratio, and high resolution, and provide significant improvements at least in the technical fields of image projection, holography, signal processing, and the like.

Drawings

These and other more detailed and specific features of the various embodiments are more fully disclosed in the following description, with reference to the accompanying drawings, in which:

FIGS. 1A-1B illustrate views of an exemplary spatial light modulator according to aspects of the present disclosure;

FIG. 2 illustrates an exemplary pixel shifting operation in accordance with aspects of the present disclosure;

FIG. 3 illustrates a block diagram of an exemplary projection system in accordance with aspects of the present disclosure;

FIG. 4 illustrates an exemplary projection optical system in accordance with aspects of the present disclosure;

FIG. 5 illustrates an exemplary illumination diffraction pattern according to a comparative example;

FIG. 6 illustrates an exemplary illumination diffraction pattern in accordance with aspects of the present disclosure;

7A-7C illustrate exemplary pixel shaping effects in accordance with aspects of the present disclosure;

FIG. 8 illustrates an exemplary process flow of an exemplary pixel shifting method in accordance with aspects of the present disclosure;

9A-9C illustrate exemplary projection images in accordance with aspects of the present disclosure; and

fig. 10A-10C illustrate exemplary projection images in accordance with aspects of the present disclosure.

Detailed Description

The present disclosure and aspects thereof may be embodied in various forms including: hardware, devices or circuits controlled by computer implemented methods, computer program products, computer systems and networks, user interfaces and application programming interfaces; and hardware implemented methods, signal processing circuits, memory arrays, application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs), and the like. The foregoing summary is intended merely to give a general idea of various aspects of the disclosure and is not intended to limit the scope of the disclosure in any way.

In the following description, numerous details are set forth, such as optical device configurations, timings, operations, etc., to provide an understanding of one or more aspects of the present disclosure. It will be apparent to one skilled in the art that these specific details are merely exemplary and are not intended to limit the scope of the application.

Further, while the present disclosure focuses primarily on examples of using various circuits in a digital projection system, it should be understood that this is merely one example of an implementation. It should further be appreciated that the disclosed systems and methods may be used in any device in which projection of light is desired; such as cinema projection systems, consumer-grade projection systems and other commercial projection systems, heads-up displays, virtual reality displays, and the like.

Pixel offset

The optics of an SLM-based projection system can be broadly classified into two parts: optics on the illumination side (i.e., optically upstream of the SLM) and optics on the projection side (i.e., optically downstream of the SLM). The SLM itself comprises a plurality of modulating elements arranged, for example, in a two-dimensional array. Each modulating element receives light from the illumination optics and transmits the light to the projection optics. In some examples, the SLM may be implemented as a DMD; this will be discussed in more detail below. However, in general, DMDs include a two-dimensional array of reflective elements (micromirrors or simply "mirrors") that selectively reflect or reject light toward projection optics based on the position of each reflective element.

Fig. 1A-1B illustrate various views of an exemplary DMD 100 in accordance with aspects of the present disclosure. In particular, fig. 1A illustrates a plan view of DMD 100, and fig. 1B illustrates a partial cross-sectional view of DMD 100 taken along line I-B illustrated in fig. 1A. DMD 100 includes a plurality of square micromirrors 202 arranged in a two-dimensional rectangular array on substrate 104. In some examples, DMD 100 may be a Digital Light Processor (DLP) device. Each micromirror 102 may correspond to one pixel of the final projected image and may be configured to tilt about an axis of rotation 108 (which is shown for a particular subset of the micromirrors 102) due to static electricity or other actuation. Each micromirror 102 has a width 112 and is arranged to be spaced apart by a gap having a width 110. The micro mirrors 102 may be formed of or coated with any highly reflective material, such as aluminum or silver, to specularly reflect light. The gaps between the micromirrors 102 may be absorptive such that input light entering the gaps is absorbed by the substrate 104.

Although fig. 1A only explicitly shows some representative micromirrors 102, in practice DMD 100 may include many more individual micromirrors. The resolution of DMD 100 refers to the number of micromirrors in both the horizontal and vertical directions. In some examples, the resolution may be 2K (2048×1080), 4K (4096×2160), 1080p (1920×1080), consumer level 4K (3840×2160), and the like. Further, in some examples, the micromirrors 102 may be rectangular and arranged in a rectangular array; may be hexagonal and arranged in a hexagonal array or the like. Further, while fig. 1A illustrates the rotation axis 108 extending in an oblique direction, in some embodiments the rotation axis 108 may extend vertically or horizontally.

As can be seen in fig. 1B, each micromirror 102 may be coupled to the substrate 104 by a yoke 114 that is rotatably coupled to the micromirror 102. The substrate 104 includes a plurality of electrodes 116. Although only two electrodes 116 are visible per micromirror 102 in the cross-sectional view of FIG. 1B, each micromirror 102 may actually include additional electrodes. Although not specifically illustrated in fig. 1B, DMD 100 may further include spacer layers, support layers, hinge members for controlling the height or orientation of micromirrors 102, and the like. The substrate 204 may include electronic circuitry associated with the DMD 100, such as CMOS transistors, memory elements, and the like.

Depending on the particular operation and control of the electrodes 116, each micromirror 102 can be switched between an "on" position, an "off" position, and an unactuated or neutral position. If the micromirror 102 is in the on position, it is actuated to an angle of, for example, -12 ° (i.e., rotated 12 ° counterclockwise relative to the neutral position) to specularly reflect the input light 106 into the on-state light 118. If the micromirror 102 is in the closed position, it is actuated to an angle of, for example, +12° (i.e., rotated 12 ° clockwise relative to the neutral position) to specularly reflect the input light 106 into the off-state light 120. The off-state light 120 may be directed toward a light collector that absorbs the off-state light 120. In some examples, the micromirrors 102 may be unactuated and parallel to the substrate 104. The particular angles illustrated in fig. 1A-1B and described herein are merely exemplary and not limiting. In some embodiments, the open position angle and the closed position angle may be between ±12 degrees and ±13 degrees, respectively (inclusive).

In some embodiments, the resolution of DMD 100 may be lower than the desired resolution of the projected image. For example, it may be desirable to project an image with a resolution of 4K, but DMDs with a resolution of 4K may have limited or no commercial availability, high costs, etc. In such an embodiment, the DMD 100, which has a relatively low resolution, may be controlled to effectively display additional pixels in the projected image. For example, a 2K mirror array may be controlled to display a 4K projected image, or a 1080p mirror array may be controlled to display a consumer 4K projected image. This control may be achieved using pixel shifting techniques.

An exemplary pixel shifting technique (wobulation) uses some method (e.g., optical method) to effectively shift the DMD 100 by fractional pixels in a set pattern to display additional pixels. An example of such a pixel shifting technique is illustrated in fig. 2. In the pixel shifting technique of fig. 2, the frame display period T (typically one divided by the projector frame rate) is divided into four sub-periods, each of duration T/4. At time t ₀ (which corresponds to the start of the first sub-period and thus to the start of the frame display period T), the image is projected onto the screen. In fig. 2, only a 2 x 2 subset of the pixels in the first resulting image 201 are shown; however, the resolution of the first resulting image 201 actually corresponds to the relatively low resolution of the DMD 100. At time t ₁ At (which corresponds to the beginning of the second sub-period) the image is shifted half a pixel to the right, thereby generating a second resulting image 202 on the screen. At time t ₂ At (which corresponds to the beginning of the third sub-period) the image is shifted down by half a pixel, thereby generating a third resulting image 203 on the screen. At time t ₃ At (which corresponds to the beginning of the fourth sub-period), the image is shifted half a pixel to the left, thereby generating a fourth resulting image 204 on the screen. At the end of the frame display period T, the image may be shifted up by half a pixel, corresponding to the original position where the display of the next frame starts. As can be seen from fig. 2, each offset increases the effective display resolution. The direction of the offset is not limited to the column direction and the row direction (i.e., up, down, left, and right) of the pixel array, but may be in an oblique direction (e.g., diagonal). Further, although FIG. 2 illustrates a pixel shifting technique that shifts images in two dimensions In some embodiments, the pixel shift may occur in one dimension only in a back-and-forth fashion.

Such pixel shifting techniques, when taken alone, may not be sufficient to properly reproduce the desired high resolution image. For example, a pixel at a lower resolution is four times larger than a pixel that would be used to display a real image at a higher resolution. Thus, while this technique allows more pixel data to be displayed when taken alone, the overlap prevents a complete rendering of the high resolution image. In the most extreme example where the image for display is a full resolution black/white checkerboard pattern, the above technique will show a monotonic gray field when taken alone, rather than a checkerboard, due to the overlap between each adjacent white pixel and black pixel in the checkerboard. To counteract the effects of the overlap, the design of the various physical components of the projector system itself may be modified.

Projector system

Some projection systems are based on SLMs that implement spatial amplitude modulation. In such a system, the light source may provide a light field that achieves the brightest level that can be reproduced on the image, and the light is attenuated or discarded in order to create the desired scene level. Some high contrast examples of projection systems based on such architectures use semi-collimated illumination systems and fourier diaphragms in the projection optics to improve contrast. Examples of projectors or other display systems that include or involve fourier planes and apertures have been described in commonly owned patents and patent applications including WIPO publication No. 2019/195182 entitled "Systems and Methods for Digital Laser Projection with Increased Contrast Using Fourier Filter [ systems and methods for digital laser projection using fourier filters to increase contrast ]," the contents of which are incorporated herein by reference in their entirety.

Fig. 3 illustrates an exemplary high contrast projection system 300 in accordance with aspects of the present disclosure. In particular, fig. 3 illustrates a projection system 300 comprising: a light source 301 configured to emit first light 302; illumination optics 303 configured to receive first light 302 and redirect or otherwise modify the first light, thereby generating second light 304; a DMD 305 configured to receive the second light 304 and selectively redirect and/or modulate the second light into third light 306; first projection optics 307 configured to receive third light 306 and redirect or otherwise modify the third light, thereby generating fourth light 308; a filter 309 configured to filter the fourth light 308, thereby generating a fifth light 310; and second projection optics 311 configured to receive fifth light 311 and project the fifth light as sixth light 312 onto screen 313. DMD 305 may be the same as or similar to DMD 200 illustrated in fig. 2A-2B. The first projection optics 307 may include at least one lens configured to spatially fourier transform the third light 306 onto a plane (also referred to as a fourier plane). The filter 309 may be a fourier aperture (also referred to as a fourier filter); i.e. an aperture at or near the plane forming the fourier transform of the object. The gap of the micromirror and DMD 305 may cooperate to form a two-dimensional grating that diffracts the input light. Thus, modulated light propagating away from DMD 305 may form multiple diffraction orders in the far field region of DMD 305 or at the focal plane of the lens, which may be observed as Fraunhofer diffraction patterns. Each diffraction order corresponds to one light beam propagating away from DMD 305 in a unique corresponding direction. By design, a majority of the optical power of the modulated light from DMD 305 may be in the zero diffraction order.

Although fig. 3 illustrates the first projection optics 307, the filter 309, and the second projection optics 311 as separate entities, in some embodiments, the filter 309 may be incorporated as part of a larger optical system that includes the first projection optics 307 and the second projection optics 311. The various elements of projection system 300 may be operated by or under the control of controller 314; the controller is, for example, one or more processors of the projection system 300, such as a Central Processing Unit (CPU). As illustrated in fig. 3, the light source 301 and DMD 305 are controlled by a controller 314. In some implementations, the controller 314 may additionally or alternatively control other components of the projection system 300, including but not limited to the illumination optics 303, the first projection optics 307, and/or the second projection optics 311. In one particular example, the controller 314 may control components of the illumination optics 303 to ensure that the second light 304 is incident on the DMD 205 at the proper location and/or angle.

In a 3D or 3D-capable projection embodiment, the physical projector may include two projection systems 300 arranged side-by-side, each individual projection system 300 projecting an image corresponding to one eye of a viewer. Alternatively, the physical projector may utilize a combined projection system 300 to project individual images corresponding to the eyes of the viewer.

In practical embodiments, projection system 300 may include fewer optical components or may include additional optical components such as mirrors, lenses, waveguides, optical fibers, beam splitters, diffusers, and the like. In addition to the screen 313, the components illustrated in fig. 3 may be integrated into a housing to provide a projection device. Such projection devices may include additional components such as memory, input/output ports, communication circuitry, power supplies, and the like.

The light source 301 may be, for example, a laser light source, an LED, or the like. Generally, the light source 301 is any light emitter that emits coherent light. In some aspects of the present disclosure, the light source 301 may include a plurality of individual light emitters, each corresponding to a different wavelength or band of wavelengths. The light source 301 emits light in response to an image signal supplied from the controller 314. The image signal includes image data corresponding to a plurality of frames to be displayed consecutively. The image signal may originate from an external source in a streaming or cloud-based manner, from an internal memory of the projection system 300, such as a hard disk, from a removable medium operatively connectable to the projection system 300, or from a combination thereof.

Although fig. 3 illustrates a generally linear optical path, in practice the optical path is generally more complex. For example, in projection system 300, second light 304 from illumination optics 303 is diverted to DMD chip(s) 305 at a fixed angle determined by the diverting angle of the DMD mirrors.

As described above, the design of the first projection optics 307, the filter 309, and/or the second projection optics 311 may be modified in order to counteract the effect of the overlap caused by the pixel shift. Preferably, the configuration of the filter 309 is specifically selected to counteract this effect. To illustrate the effect of filter configuration on an image, fig. 4 shows an exploded view of an exemplary projection lens system 400 in accordance with aspects of the present disclosure. Projection lens system 400 is one example of a combination of first projection optics 307, filter 309, and second projection optics 311 illustrated in fig. 3. In some aspects of the present disclosure, the performance of the complete projection lens system 400 meets the Digital Cinema Initiative (DCI) image specification; for example, DCI Digital Cinema System Specification (DCSS) release 1.3 or an updated release.

Projection lens system 400 includes a first projection optic 401 (also referred to as a fourier portion or fourier lens assembly) configured to form a fourier transform of an object at its exit pupil, a filter 402, and a second projection optic 403 (also referred to as a zoom portion or zoom lens assembly), as will be described in more detail below. The first projection optics 401, the filter 402, and the second projection optics 403 may correspond to the first projection optics 307, the filter 309, and the second projection optics 311, respectively, illustrated in fig. 3. As used herein, a "fourier portion" or "fourier lens assembly" refers to an optical system that spatially fourier transforms modulated light (e.g., light from DMD 305) by focusing the modulated light onto a fourier plane. The first projection optics 401 and the filter 402 together operate as a fourier lens with a spatial filter, which fourier lens can also be used as a stationary projection lens. The spatial fourier transform applied by the first projection optics 401 converts the propagation angle of each diffraction order of the modulated light into a corresponding spatial position on the fourier plane. The first projection optics 401 is thus able to achieve selection of desired diffraction orders and rejection of undesired diffraction orders by spatial filtering at the fourier plane. The spatial fourier transform of the modulated light at the fourier plane is equivalent to the fraunhofer diffraction pattern of the modulated light. Both the first projection optics 401 and the second projection optics may comprise a plurality of individual lens elements.

To allow access to the fourier aperture when the projection system 400 is set up and/or to allow the filters 402 to be interchangeable, the projection lens system 400 may have a modular design. In such an example, the first projection optics 401 may be provided with a first attachment portion 404 and the second projection optics 403 may be provided with a second attachment portion 405. The first attachment portion 404 and the second attachment portion 405 may include complementary mating fasteners such as screws, threads, pins, slots, and the like. In other embodiments, the housings for the first projection optics 401 and the second projection optics 403 may be integral.

The filter 402 is configured to block a portion of the light in the projection lens system 200 and transmit a portion of the light (e.g., transmit modulated light corresponding to at least one diffraction order). In some embodiments, the diffraction orders transmitted by the filter 402 may be an overall diffraction order (i.e., an overall transmitted order); alternatively, the diffraction orders transmitted by the filter 402 may be converging diffraction orders (i.e., a portion of one order and a complementary portion from the other order or orders). As illustrated in fig. 4, the filter 402 has square openings with a side length of, for example, 6 mm. Fig. 4 also illustrates an optical axis 410 of the projection lens system 400. After assembly, the first projection optics 401 and the second projection optics are substantially coaxial with each other and with the optical axis 410. In some implementations (e.g., depending on the illumination angle), the filter 402 is further substantially coaxial with the optical axis 410.

Projection lens system 400 may include or be associated with one or more non-optical elements including a heat sink device such as a heat sink (or cooling fins), one or more adhesives (or fasteners), and the like. In some implementations, the filter 402 may block and thus absorb about 15% or more of the incident light, and thus the heat sink or cooling fins may be positioned and configured to properly dissipate heat from the filter 402. In some embodiments, the filter 402 is thermally isolated from other portions of the projection lens system 400.

Although fig. 4 illustrates the opening of the filter 402 as having a square shape, the present disclosure is not so limited. In some embodiments, the filter 402 may have differently shaped openings, such as circular, oval, rounded square, rectangular, rounded rectangular, hexagonal, rounded hexagonal, pincushion, etc., so long as the shape is sized to cover the appropriate angular passband. A suitable angular passband may be about 190% to 210%, for example, about 200%, of the size of the diffraction order in the fourier plane (e.g., zero order, first order, second order, etc.). In some embodiments, the size and shape of the opening of the filter 402 may be specifically selected such that operation of the projector is enhanced when used with a pixel shifting method. For example, because filter 402 acts as a Fourier filter, it can have an effect on which angular frequency components from the DMD are displayed on the screen. Thus, modification of the opening of the filter 402 may result in a change in the size and/or shape of the pixels displayed on the screen. In one example, the size of the pixels may be effectively reduced such that the pixels are smaller on the screen than the projected size of the physical mirror of the DMD. This, when combined with pixel shifting techniques, can create higher resolution images. To achieve this, the light source of the projection system (e.g., light source 301 illustrated in FIG. 3) is selected to have sufficiently high coherence and sufficiently low etendue to effectively employ Fourier filtering. Thus, the light source may be a laser light source.

Fourier enhanced pixel offset

Fig. 5 to 9C illustrate the effect of the size and shape of the filter aperture. Fig. 5 illustrates the effect of pixel shifting without a filter (or with a filter having a large aperture) assuming that the illumination from the light source is sufficient. In fig. 5, an aperture 501 is superimposed on a plurality of diffraction orders 502. The aperture 501 may correspond to a projection lens aperture (e.g., of the projection lens in the first projection optics 307 illustrated in fig. 3), or, in the case where a fourier filter with a large aperture is used, to an opening of a fourier filter (e.g., filter 309 and/or filter 402 illustrated in fig. 3-4). All or most of the blocked diffraction orders 502 are illustrated using dashed lines, while all or most of the transmitted diffraction orders 502 are illustrated using solid lines.

Fig. 5 corresponds to a particular example, where the wavelength of light emitted from the light source is 545nm, the illumination angle of light incident on the DMD (e.g., the angle measured by input light 106 illustrated in fig. 1B relative to the normal to the micromirrors in the unactuated or neutral position) is 24 °, the open position angle of the mirrors of the DMD (e.g., the angle measured by the surface of micromirror 102 in the open state illustrated in fig. 1B relative to the plane of substrate 104) is 12 °, the closed position angle of the mirrors of the DMD (e.g., the angle measured by the surface of micromirror 102 in the closed state illustrated in fig. 1B relative to the plane of substrate 104) is-12 °, the width of the mirrors of the DMD (e.g., width 112 illustrated in fig. 1A) is 10.80 μm, and the gap between the mirrors of the DMD (e.g., width 114 illustrated in fig. 1A) is 0.75 μm.

Fig. 6 illustrates the effect of pixel shifting by a filter of a particular size (e.g., its diameter or height and/or width, or area) with a square aperture that is 200% of the size of the diffraction order (e.g., the diameter or height and/or width, or area corresponding to the zero or higher order diffraction order) assuming sufficient illumination from the light source. In fig. 6, an aperture 601 is superimposed on a plurality of diffraction orders 602. The aperture 601 may correspond to an opening of a fourier filter (e.g., filter 309 and/or filter 402 illustrated in fig. 3-4). Blocked diffraction orders 602 are illustrated using dashed lines, while transmitted diffraction orders 602 are illustrated using solid lines. As shown in fig. 6, the aperture 601 passes light from a diffraction order 602, which is illustrated using a solid line and corresponds to the zero-order diffraction order.

FIG. 6 corresponds to the same projection system parameters as FIG. 5; that is, corresponding to the specific example of fig. 5, where the wavelength of the light emitted from the light source is 545nm, the illumination angle of the light incident on the DMD (e.g., the angle measured by the input light 106 illustrated in fig. 1B with respect to the normal of the micromirror in the unactuated or neutral position) is 24 °, the open position angle of the mirrors of the DMD (e.g., the angle measured by the surface of the micromirror 102 illustrated in fig. 1B with respect to the plane of the substrate 104) is 12 °, the closed position angle of the mirrors of the DMD (e.g., the angle measured by the surface of the micromirror 102 illustrated in fig. 1B with respect to the plane of the substrate 104) is-12 °, the width of the mirrors of the DMD (e.g., the width 112 illustrated in fig. 1A) is 10.80 μm, and the gap between the mirrors of the DMD (e.g., the width 114 illustrated in fig. 1A) is 0.75 μm.

In fig. 5, because many diffraction orders (corresponding to a large portion of the DMD angular fourier spectrum) are captured by the lens, many high spatial frequency details (e.g., mirror edges) are reproduced on the screen, resulting in square pixels corresponding to the size of the mirrors in the physical device. In fig. 6, the filter corresponds to a main lobe that captures the mirror diffraction (i.e., sinc function) cloud. Because the passband of the filter in fig. 6 is reduced compared to the system in fig. 5, high angular frequencies will not be transmitted. As the passband of the filter becomes smaller, sharp details such as the edges of the mirror will be lost and at a particular point shown in fig. 6, only the image content frequency is allowed to pass. If the size of the opening of the filter is further reduced, the filter will start to low pass the image content, which may cause blurring of the image on the screen. Fig. 7A to 7C illustrate the effect of a specific size of the opening.

Fig. 7A to 7C each illustrate a single pixel projected by a projection system having the same characteristics and components, except that the size of the filter aperture is different in each illustration. Further, each of fig. 7A to 7C is presented in the same scale. In fig. 7A, the filter has a square aperture with a size of 600% of the diffraction order; in fig. 7B, the filter has a square aperture with a size of 200% of the diffraction order; and in fig. 7C, the filter has a square aperture with a size of 100% of the diffraction order. In contrast to fig. 7A, fig. 7B does not exhibit undesirably high spatial frequency details. Fig. 7B is brighter in the middle and darker towards the corner of the mirror than fig. 7C. Fig. 7B illustrates the point at which image data (without other content) passes through the filter. The smaller pixels of fig. 7B have less overlap when combined with the pixel shifting technique. Thus, in projection systems that include fourier filters and that utilize pixel shifting techniques, the filters preferably have square filters with dimensions of about 190% to 210% of the diffraction order, and most preferably have square filters with dimensions of about 200% of the diffraction order.

Fig. 8 illustrates one particular embodiment of a pixel shifting technique in a projection system having a fourier filter (such as projection system 300 illustrated in fig. 3). The method 800 of fig. 8 may be performed by the controller 314 of fig. 3 and may be implemented using hardware, software, firmware, or a combination thereof. In some examples, method 800 is implemented as instructions stored in a non-transitory computer readable medium (such as a hard disk, or other storage medium contained in or associated with projection system 300).

In method 800, a series of images are displayed using image data comprising a series of frames. The image data is divided into a plurality of frame periods, each frame period corresponding to a duration T of a frame; for example, 60Hz shows a frame period T of (1/60) seconds. At operation 801, a frame period is divided into N sub-periods, where N is an integer greater than 1. Preferably, N is four to implement a pixel shift pattern similar to that illustrated in fig. 2; however, in other embodiments, N may be six or other numbers other than four. At operation 802, a counter I is initialized to 1. Subsequently, at operation 803, for the I-th sub-period, an image is projected through the filter aperture. Operation 803 may include sub-operations such as causing a light source of the projection system to emit light, controlling a spatial light modulator (e.g., DMD 305 of fig. 3) to modulate the light and form an image, etc. The image is held for a duration of T/N. In the example of a 60Hz display using four subcycles per frame, the subcycle duration is (1/240) seconds.

At operation 804, at the end of the sub-period, the counter I is compared with N to determine if the sub-period is the last sub-period of the frame. If counter I is not equal to N, then counter I is incremented by 1 at operation 805 and the pixel is shifted at operation 806. In the example where four sub-periods are provided per frame and the pixel offset follows the square pattern in fig. 2, this corresponds to an offset of half a pixel. Alternatively, four sub-periods may be provided per frame, and the pixel offset may follow a diamond or rectangular pattern; six subcycles may be provided per frame, and the pixel offset may follow a rectangular or hexagonal pattern; three sub-periods may be provided per frame and the pixel offset may follow a triangle pattern; two sub-periods may be provided per frame, and the pixel shift may follow a linear (back and forth) pattern, etc. In some embodiments, the number of sub-periods may be tens (or greater), and the pixel offset may approximate a circular pattern or complex shape. Thereafter, operation 803 is repeated in the next sub-period until counter I equals N. At this point, at operation 807, the frame is incremented and the method 800 returns to operation 807. Operations 802 through 807 are repeated for the duration of the image display and may continue until the endpoint of the media content has been reached, a pause or stop instruction is issued, etc.

Fig. 9A to 10C illustrate the effect of the specific size of the opening when the pixel shift technique is combined with the filter. Fig. 9A to 9C illustrate results of a first image test using a checkerboard pattern, and fig. 10A to 10C illustrate results of a second image test using a resolution map.

Fig. 9A shows an input image for the first image test, which is a full resolution 4K checkerboard pattern shown in close-up. Fig. 9B shows the result (i.e., output image) of a projection system that implements the pixel shift technique of four sub-periods (as shown in fig. 2) but does not include a fourier aperture (such as filter 309 of fig. 2 or filter 402 of fig. 4). Fig. 9C shows the result of a projection system implementing a four sub-period pixel shifting technique and including a fourier aperture with an opening of 200% of the diffraction order in size. In fig. 9B, the black squares of the checkerboard are indistinguishable from the white squares, because all squares appear to be essentially the same light gray. However, in fig. 9C, the squares of the checkerboard are distinguishable from each other. Although the modulation amplitude is not as high as when the projection system uses a native full resolution modulator, the combination of pixel offset with a specific size of aperture allows the use of a lower resolution modulator to reproduce high resolution details. As can be seen by comparing fig. 9B and 9C, the pixel shift-only technique is unable to reproduce such details without further implementation of the aperture.

Fig. 10A shows an input image for a second image test, which is a renormalized quantization resolution map shown in close-up. Fig. 10B shows the results of a projection system implementing a four sub-period pixel shifting technique and including a fourier aperture with an opening sized 800% of the diffraction order. Fig. 10C shows the results of a projection system implementing a four sub-period pixel shifting technique and including a fourier aperture with an opening sized 200% of the diffraction order. In practice, the large opening for the projected image in fig. 10B is similar to no filter at all, as can be seen by comparing the artifacts in fig. 10B with the artifacts similar in fig. 9B. Therefore, in fig. 10B, the lines in the resolution pattern are not resolved. However, in fig. 10C, these lines are resolved even in the close-up view.

Effects of

By comparing fig. 9C and 10C with fig. 9B and 10B and in view of fig. 7B and 7C, it can be seen that the effect of pixel offset and aperture opening of a particular size (and in particular, the ability to correctly render high resolution images using a lower resolution modulator) is unexpectedly greater than just the sum of the pixel offset alone and the fourier aperture alone.

Systems, methods, and devices according to the present disclosure may employ any one or more of the following configurations.

(1) A projection system, comprising: a light source configured to emit light; a spatial light modulator configured to receive the light and generate modulated light; a lens configured to spatially fourier transform the modulated light; a filter comprising an aperture, the filter configured to transmit at least one diffraction order of the modulated light fourier transformed by the lens and to block a remaining portion of the modulated light; and a controller configured to: for each of a plurality of subcycles, causing the projection system to project an image through the filter, and shifting the image by a fractional pixel distance between each of the plurality of subcycles.

(2) The projection system of (1), wherein the aperture has a size between 190% and 210% of the size of the diffraction order of the modulated light.

(3) The projection system of (2), wherein the aperture has a size of 200% of a size of a diffraction order of the modulated light.

(4) The projection system according to any one of (1) to (3), wherein the aperture has a square, circular, or elliptical shape.

(5) The projection system of any one of (1) to (4), wherein the plurality of sub-periods is four sub-periods.

(6) The projection system of (5), wherein the partial pixel distance is equal to a half pixel distance.

(7) The projection system of (6), wherein the controller is configured to: the method includes shifting the image by a distance of one half pixel in a first direction between a first sub-period and a second sub-period, shifting the image by a distance of one half pixel in a second direction perpendicular to the first direction between the second sub-period and a third sub-period, shifting the image by a distance of one half pixel in a third direction perpendicular to the second direction and opposite to the first direction between the third sub-period and a fourth sub-period, and shifting the image by a distance of one half pixel in a fourth direction perpendicular to the third direction and opposite to the second direction after the fourth sub-period.

(8) The projection system of any one of (1) to (7), wherein the controller is configured to repeatedly cause the projection system to project the image and offset the image for a plurality of image frames.

(9) The projection system according to any one of (1) to (8), wherein the spatial light modulator is a digital micromirror device.

(10) The projection system of any one of (1) to (9), wherein the light source comprises at least one laser.

(11) A projection method, comprising: emitting light by a light source of the projection system; receiving the light by a spatial light modulator of the projection system and generating modulated light; performing a spatial fourier transform on the modulated light by a lens of the projection system; transmitting, by a filter of the projection system comprising an aperture, at least one diffraction order of the modulated light after fourier transformation by the lens and blocking a remaining portion of the modulated light; for each of a plurality of subcycles, causing, by a controller of the projection system, the projection system to project an image through the filter; and shifting, by the controller, the image by a partial pixel distance between each of the plurality of subcycles.

(12) The projection method according to (11), wherein the size of the aperture is between 190% and 210% of the size of the diffraction order of the modulated light.

(13) The projection method according to (12), wherein the size of the aperture is 200% of the size of the diffraction order of the modulated light.

(14) The projection method according to any one of (11) to (13), wherein the aperture has a square, circular, or elliptical shape.

(15) The projection method according to any one of (11) to (14), wherein the plurality of sub-periods is four sub-periods.

(16) The projection method according to (15), wherein the partial pixel distance is equal to a distance of half a pixel.

(17) The projection method of (16), wherein the operation of shifting pixels comprises: the method includes shifting the image by a distance of one half pixel in a first direction between a first sub-period and a second sub-period, shifting the image by a distance of one half pixel in a second direction perpendicular to the first direction between the second sub-period and a third sub-period, shifting the image by a distance of one half pixel in a third direction perpendicular to the second direction and opposite to the first direction between the third sub-period and a fourth sub-period, and shifting the image by a distance of one half pixel in a fourth direction perpendicular to the third direction and opposite to the second direction after the fourth sub-period.

(18) The projection method according to any one of (11) to (17), comprising: repeating, by the controller, the projecting of the image and shifting pixels of the image for a plurality of frames of the image data.

(19) The projection method according to any one of (11) to (18), wherein the light source includes at least one laser.

(20) A non-transitory computer readable medium storing instructions that, when executed by an electronic processor of a projection device, cause the projection device to perform operations comprising the method of any of (11) to (19).

With respect to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, while the steps of such processes, etc. have been described as occurring in a particular ordered sequence, such processes may be practiced with the described steps performed in an order different than that described herein. It is further understood that certain steps may be performed concurrently, other steps may be added, or certain steps described herein may be omitted. In other words, the process descriptions herein are provided for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claims.

Accordingly, it is to be understood that the above description is intended to be illustrative, and not restrictive. Many embodiments and applications other than the examples provided will be apparent from a reading of the above description. The scope should be determined not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that the technology discussed herein will evolve in the future, and that the disclosed systems and methods will be incorporated into such future embodiments. In summary, it should be understood that the application is capable of modification and variation.

All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the art described herein unless an explicit indication to the contrary is made herein. In particular, the use of singular articles such as "a," "the," "said," and the like should be understood to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

The Abstract of the disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. This Abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing detailed description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the detailed description, with each claim standing on its own as a separately claimed subject matter.

Claims

1. A projection system, comprising:

a light source comprising at least one laser configured to emit an input laser light;

a spatial light modulator configured to receive the input laser light and generate modulated laser light, wherein the modulated laser light includes a plurality of diffraction orders generated by the input laser light being diffracted by the spatial light modulator;

a lens configured to fourier transform the modulated laser light and focus the modulated laser light fourier-transformed by the lens onto a fourier plane;

a filter for spatially filtering the modulated laser light fourier transformed by the lens in the fourier plane, the filter configured to transmit at least one diffraction order of the modulated laser light fourier transformed by the lens and block a remaining portion of the modulated laser light fourier transformed by the lens; and

a controller for controlling the spatial light modulator, the controller configured to:

for each of a plurality of sub-periods, causing the spatial light modulator to project an image through the lens and the filter, and

The spatial light modulator is offset by a fraction of a pixel distance between each of the plurality of sub-periods.

2. The projection system of claim 1, wherein the filter is configured to transmit at least a zero-order diffraction order of the modulated laser light after fourier transformation by the lens.

3. The projection system of claim 1 or 2, wherein the filter comprises an aperture for passing the at least one diffraction order transmitted by the filter, wherein the aperture has a size between 190% and 210% of the size of the zero diffraction order of the modulated laser light in the fourier plane.

4. A projection system according to claim 3, wherein the aperture has a size of 200% of the size of the zero diffraction order of the modulated laser light in the fourier plane.

5. The projection system of any of claims 1-4, wherein the aperture is square in shape.

6. The projection system of any of claims 1-5, wherein the plurality of subcycles is four subcycles.

7. The projection system of any of claims 1-6, wherein the partial pixel distance is equal to a distance of half a pixel.

8. The projection system of any of claims 1-7, wherein the controller is configured to cause the spatial light modulator to:

between the first sub-period and the second sub-period, the position of the image to be projected by the spatial light modulator is shifted in the first direction by a distance of half a pixel,

between the second and third sub-periods, the position of the image to be projected by the spatial light modulator is shifted by a distance of half a pixel in a second direction perpendicular to the first direction,

between the third and fourth sub-periods, the position of the image to be projected by the spatial light modulator is shifted by a distance of half a pixel in a third direction perpendicular to the second direction and opposite to the first direction, and

after the fourth sub-period, the position of the image to be projected by the spatial light modulator is shifted by a distance of half a pixel in a fourth direction perpendicular to the third direction and opposite to the second direction.

9. The projection system of any of claims 1 to 8, wherein the controller is configured to repeatedly cause the spatial light modulator to project an image and a position of an image to be projected by the spatial light modulator to be offset by a partial pixel distance for a plurality of image frames.

10. The projection system of any of claims 1 to 9, wherein the spatial light modulator is a digital micromirror device.

11. A projection method, comprising:

emitting an input laser light by a light source of the projection system comprising at least one laser;

receiving the input laser light by a spatial light modulator of the projection system and generating modulated laser light, wherein the modulated laser light includes a plurality of diffraction orders generated by the input laser light being diffracted by the spatial light modulator;

fourier transforming the modulated laser light by a lens of the projection system and focusing the modulated laser light fourier transformed by the lens onto a fourier plane;

transmitting, by a filter of the projection system for spatially filtering the modulated laser light fourier transformed by the lens in the fourier plane, at least one diffraction order of the modulated laser light fourier transformed by the lens and blocking a remaining portion of the modulated light fourier transformed by the lens; and

by a controller of the projection system for controlling the spatial light modulator,

For each of a plurality of subcycles, causing the spatial light modulator to project an image through the lens and the filter; and is also provided with

12. The projection method of claim 11, wherein transmitting at least one diffraction order of the modulated laser light fourier transformed by the lens comprises transmitting at least a zero-order diffraction order of the modulated laser light fourier transformed by the lens.

13. The projection method according to claim 11 or 12, wherein the filter comprises an aperture for passing the at least one diffraction order transmitted by the filter, wherein the aperture has a size of between 190% and 210% of the size of the zero diffraction order of the modulated laser light in the fourier plane.

14. The projection method of claim 13, wherein the aperture has a size of 200% of the size of the zero diffraction order of the modulated laser light in the fourier plane.

15. The projection method according to any one of claims 11 to 14, wherein the aperture is square in shape.

16. The projection method of any of claims 11 to 15, wherein the plurality of sub-periods is four sub-periods.

17. The projection method according to any one of claims 11 to 16, wherein the partial pixel distance is equal to a distance of half a pixel.

18. The projection method of any of claims 11 to 17, wherein, between each of the plurality of sub-periods, shifting the position of the image to be projected by the spatial light modulator by a partial pixel distance comprises causing the spatial light modulator to:

19. The projection method according to any one of claims 11 to 18, comprising:

the spatial light modulator projects an image and the position of the image to be projected by the spatial light modulator is offset by a partial pixel distance repeatedly for a plurality of image frames by the controller.

20. The projection method of any of claims 11 to 19, wherein the spatial light modulator is a digital micromirror device.

21. A non-transitory computer readable medium storing instructions that, when executed by an electronic processor of a projection device, cause the projection device to perform operations comprising the method of any of claims 11-20.