WO2024196742A1

WO2024196742A1 - Display levels for dual modulation systems

Info

Publication number: WO2024196742A1
Application number: PCT/US2024/020099
Authority: WO
Inventors: Jerome D. Shields; Trevor Davies; Douglas J. Gorny; Barret Lippey; Christopher John ORLICK; Martin J. Richards; Jon Scott Miller
Original assignee: Dolby Laboratories Licensing Corp
Current assignee: Dolby Laboratories Licensing Corp
Priority date: 2023-03-20
Filing date: 2024-03-15
Publication date: 2024-09-26
Anticipated expiration: 2025-09-20

Abstract

Systems and methods for creating display levels using different bit sequences on dual modulation digital micromirror devices. One example provides a multi-modulation display system comprising a light source, a first modulator, a second modulator, and a controller. The first modulator includes a first plurality of mirrors to modulate light from the light source. The second modulator includes a second plurality of mirrors to modulate light from the first modulator. The controller includes an electronic processor and a memory. The electronic processor performs instructions including sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a first sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first sub-frame period of time and during a second sub-frame period of time.

Description

DISPLAY LEVELS FOR DUAL MODULATION SYSTEMS

BACKGROUND

Cross-Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Application No. 63/596,879 filed November 7, 2023 and U.S. Provisional Application No. 63/453,423 filed March 20, 2023, each of which is incorporated by reference in their entireties.

Field of the Disclosure

[0002] This application relates generally to bit sequences for dual modulation display systems and, particularly, to creating display levels using different bit sequences on dual modulation digital micromirror devices.

Description of Related Art

[0003] Digital projection systems typically utilize a light source and an optical system to project an image onto a surface or screen. The optical system includes components such as mirrors, lenses, waveguides, optical fibers, beam splitters, diffusers, digital micromirror devices (DMDs) spatial light modulators (SLMs), phase light modulators (PLMs), and the like. A bit sequencing pattern is used to control periods of time that the DMD micromirrors are on, and periods of time that the DMD micromirrors are off. Modulators may switch multiple times within a single frame.

BRIEF SUMMARY OF THE DISCLOSURE

[0004] Various aspects of the present disclosure relate to devices, systems, and methods for projection display. One example embodiment provides a multi-modulation display system comprising a light source, a first modulator, a second modulator, and a controller. The first modulator is illuminated by the light source and includes a first plurality of mirrors to modulate light from the light source. The second modulator is illuminated by light from the first modulator and includes a second plurality of mirrors to modulate light from the first modulator. The controller includes an electronic processor and a memory. The memory comprises processor-readable instructions such that, when the electronic processor reads the processor- readable instructions, the electronic processor performs instructions including sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a first sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first sub- frame period of time and during a second sub-frame period of time different from the first subframe period of time.

[0005] Another example embodiment provides a processor-implemented method for controlling a multi-modulation display system comprising a light source, a first modulator including a first plurality of mirrors, a second modulator including a second plurality of mirrors, and a processor controlling the first modulator and the second modulator. The method includes sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light from the light source during a first sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light from the first modulator during the first sub-frame period of time and during a second sub-frame period of time different from the first sub-frame period of time.

[0006] Another example embodiment provides a multi-modulation display system comprising a light source, a first modulator, a second modulator, and a controller. The first modulator is illuminated by the light source and includes a first plurality of mirrors to modulate light from the light source. The second modulator is illuminated by light from the first modulator and includes a second plurality of mirrors to modulate light from the first modulator. The controller includes an electronic processor and a memory. The memory comprises processor-readable instructions such that, when the electronic processor reads the processor- readable instructions, the electronic processor performs instructions including sending, during a first frame, control signals to the first modulator and the second modulator such that a plurality of first sub-frame periods sum to generate a first portion of a display level, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of second sub-frame periods sum to generate a second portion of the display level. When averaged over a first plurality of frames, the first portions of the display level generate an average first portions level. When averaged over a second plurality of frames, the second portions of the display level generate an average second portions level. When summed, the average first portions level and the average second portions level generate the display level.

[0007] Another example embodiment provides a method for generating display levels in a dual-modulation system comprising sending, during a first frame, control signals to the first modulator and the second modulator such that a plurality of first sub-frame periods sum to generate a first portion of a display level, wherein the first modulator is illuminated by a light source, and wherein the second modulator is illuminated by light from the first modulator, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of second sub-frame periods sum to generate a second portion of the display level. When averaged over a first plurality of frames, the first portions of the display level generate an average first portions level. When averaged over a second plurality of frames, the second portions of the display level generate an average second portions level. When summed, the average first portions level and the average second portions level generate the display level.

[0008] Another example embodiment provides a non-transitory computer-readable medium storing instructions that, when executed by a processor of a projection system, cause the projection system to perform operations comprising sending, during a first frame, control signals to the first modulator and the second modulator such that a plurality of first sub-frame periods sum to generate a first portion of a display level, wherein the first modulator is illuminated by a light source, and wherein the second modulator is illuminated by light from the first modulator, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of second sub-frame periods sum to generate a second portion of the display level. When averaged over a first plurality of frames, the first portions of the display level generate an average first portions level. When averaged over a second plurality of frames, the second portions of the display level generate an average second portions level. When summed, the average first portions level and the average second portions level generate the display level.

[0009] In this manner, various aspects of the present disclosure provide for the display of images having a high dynamic range and high resolution, and effect improvements in at least the technical fields of image projection, holography, signal processing, and the like.

DESCRIPTION OF THE DRAWINGS

[0010] These and other more detailed and specific features of various embodiments are more fully disclosed in the following description, reference being had to the accompanying drawings, in which:

[0011] FIG. 1 illustrates a block diagram of an exemplary image projector display system according to various aspects of the present disclosure.

[0012] FIG. 2A illustrates a plan view of an exemplary spatial light modulator for use with various aspects of the present disclosure.

[0013] FIG. 2B illustrates a cross-sectional view taken along the line I-B of FIG. 2 A.

[0014] FIG. 3 illustrates a high-level diagram of a switching scheme that matches a bitsequence repeating pattern. [0015] FIG. 4A illustrates an example bit sequence for a pre-modulator.

[0016] FIG. 4B illustrates an example bit sequence for a primary modulator.

[0017] FIG. 4C illustrates the transmittance resulting from the bit sequences of FIGS. 4A and 4B.

[0018] FIG. 5 A illustrates another example bit sequence for a pre-modulator.

[0019] FIG. 5B illustrates another example bit sequence for a primary modulator.

[0020] FIG. 5C illustrates the transmittance resulting from the bit sequences of FIGS. 5 A and 5B.

[0021] FIG. 6A illustrates another example bit sequence for a pre-modulator.

[0022] FIG. 6B illustrates another example bit sequence for a primary modulator.

[0023] FIG. 6C illustrates the transmittance resulting from the bit sequences of FIGS. 6A and 6B.

[0024] FIG. 7 illustrates bit sequence levels for various PQ code values.

[0025] FIG. 8 illustrates an example of achieved display levels with two layers of precision.

[0026] FIG. 9 illustrates an example of achieved display levels with four layers of precision.

[0027] FIG. 10 illustrates a block diagram of a method for generating display levels.

DETAILED DESCRIPTION

[0028] This disclosure and aspects thereof can be embodied in various forms, including hardware, devices, or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces, and application programming interfaces; as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and the like. The foregoing summary is intended solely to give a general idea of various aspects of the present disclosure, and does not limit the scope of the disclosure in any way.

[0029] In the following description, numerous details are set forth, such as optical device configurations, timings, operations, and the like, in order to provide an understanding of one or more aspects of the present disclosure. It will be readily apparent to one skilled in the art that these specific details are merely exemplary and not intended to limit the scope of this application.

[0030] Moreover, while the present disclosure focuses mainly on examples in which the various circuits are used in digital projection systems, it will be understood that this is merely one example of an implementation. It will further be understood that the disclosed systems and methods can be used in any device in which there is a need to project light; for example, cinema, consumer, and other commercial projection systems, heads-up displays, virtual reality displays, and the like.

Projector Systems

[0031] FIG. 1 illustrates one possible embodiment of a suitable image projector display system. In the illustrated embodiment, the projector display system is constructed as a dual/multi-modulator projection system 100. The projection system 100 employs a light source 102 that supplies the projector system with a desired illumination such that a final projected image will be sufficiently bright for the intended viewers of the projected image. Light source 102 may comprise any suitable light source, such as, but not limited to, Xenon lamps, laser(s), coherent light sources, and partially-coherent light sources. Additionally, optical systems described herein may implement optical fibers to transfer light from the light source 102 to optics within the optical system.

[0032] Light 104 from the light source 102 may illuminate a first modulator 106 (for example, a pre-modulator) that may, in turn, illuminate a second modulator 110 (for example, a primary modulator) via a set of optional optical components 108. Light from the second modulator 110 may be projected by a projection lens 112 (or other suitable optical components) to form a final projected image upon a screen 114. In some instances, the projection lens 112 includes an optical filter for filtering the light from the second modulator 110, as described below in more detail. The first modulator 106 and the second modulator 110 may be controlled by a controller 116. The controller 116 may receive input image and/or video data and may perform certain image processing algorithms, gamut mapping algorithms or other such suitable processing upon the input image/video data and output control/data signals to the first modulator 106 and the second modulator 110 in order to achieve a desired final projected image on the screen 114. In addition, in some projector systems, it may be possible, depending on the light source, to modulate light source 102 (control line not shown) in order to achieve additional control of the image quality of the final projected image. [0033] Light recycling module 103 is depicted in FIG. 1 as a dotted box that may be placed in the light path from the light source 102 to the first modulator 106. It may be appreciated that light recycling may be inserted into the projector system at various points in the projector system. For example, light recycling may be placed between the first and second modulators. In addition, light recycling may be placed at more than one point in the optical path of the display system.

[0034] Although the embodiment of FIG. 1 depicts a single light channel, it will be appreciated that the first and second modulators may be replicated for each of a series of color channels within the projector such that each color channel includes two optically offset reflective modulators. The series of color channels may comprise a red channel, a green channel, and a blue channel. The light source may comprise, for example, a plurality of colored laser light sources. In one embodiment, the light sources may be modulated either globally (in brightness) and/or spatially (locally) dimmed according to signals (not shown) from a controller (e.g., 116).

[0035] The intermediate signals to the second modulator may be, for example, based on a light field simulation comprising a point spread function of light reflected by the first modulator and the offset. For example, the intermediate signals to the second modulator may be based on a point spread function of light reflected by the first modulator in each channel and the offset in each channel. The offset in the channels may be the same, or the offset of at least two channels is different and the intermediate signals to the second modulator in each channel is based on at least one of the offset and differences in offset between channels.

[0036] While the embodiment of FIG. 1 is presented in the context of a dual, multimodulation projection system, it should be appreciated that the techniques and methods of the present application will find application in other dual, multi-modulation display systems. For example, a dual modulation display system comprising a backlight, a first modulator (e.g., LCD or the like), and a second modulator (e.g., LCD or the like) may employ suitable optical components and image processing methods and techniques to affect the performance and efficiencies discussed herein in the context of the projection systems.

[0037] The first modulator 106 and the second modulator 110 may be configured as digital micromirror devices (DMDs) composed of a plurality of mirrors used to adjust the angle of incidence of light. To illustrate the effects of the angle of incidence and the DMD mirrors, FIGS. 2A-2B show an exemplary DMD 200 in accordance with various aspects of the present disclosure. In particular, FIG. 2A illustrates a plan view of the DMD 200, and FIG. 2B illustrates partial cross-sectional view of the DMD 200 taken along line LB illustrated in FIG. 2A. The DMD 200 includes a plurality of square micromirrors 202 arranged in a two-dimensional rectangular array on a substrate 204. Each micromirror 202 may correspond to one pixel of the eventual projection image, and may be configured to tilt about a rotation axis 208, shown for one particular subset of the micromirrors 202, by electrostatic or other type of actuation. The individual micromirrors 202 have a width 212 and are arranged with gaps of width 210 therebetween. The micromirrors 202 may be formed of or coated with any highly reflective material, such as aluminum or silver, to thereby specularly reflect light. The gaps between the micromirrors 202 may be absorptive, such that input light which enters a gap is absorbed by the substrate 204.

[0038] While FIG. 2A expressly shows only some representative micromirrors 202, in practice the DMD 200 may include many more individual micromirrors in a number equal to a resolution of the projection system 100. In some examples, the resolution may be 2K (2048x 1080), 4K (4096x2160), 1080p (1920x 1080), consumer 4K (3840x2160), and the like. Moreover, in some examples the micromirrors 202 may be rectangular and arranged in the rectangular array; hexagonal and arranged in a hexagonal array, and the like. Moreover, while FIG. 2A illustrates the rotation axis 208 extending in an oblique direction, in some implementations the rotation axis 208 may extend vertically or horizontally.

[0039] As can be seen in FIG. 2B, each micromirror 202 may be connected to the substrate 204 by a yoke 214, which is rotatably connected to the micromirror 202. The substrate 204 includes a plurality of electrodes 216. While only two electrodes 216 per micromirror 202 are visible in the cross-sectional view of FIG. 2B, each micromirror 202 may in practice include additional electrodes. While not particularly illustrated in FIG. 2B, the DMD 200 may further include spacer layers, support layers, hinge components to control the height or orientation of the micromirror 202, and the like. The substrate 204 may include electronic circuitry associated with the DMD 200, such as complementary metal-oxide semiconductor (CMOS) transistors, memory elements, and the like.

[0040] Depending on the particular operation and control of the electrodes 216, the individual micromirrors 202 may be switched between an “on” position, an “off’ position, and an unactuated or neutral position. If a micromirror 202 is in the on position, it is actuated to an angle of (for example ) -12° (that is, rotated counterclockwise by 12° relative to the neutral position) to specularly reflect input light 206 into on-state light 218. If a micromirror 202 is in the off position, it is actuated to an angle of (for example) +12° (that is, rotated clockwise by 12° relative to the neutral position) to specularly reflect the input light 406 into off-state light 220. The off-state light 220 may be directed toward a light dump that absorbs the off-state light 220. In some instances, a micromirror 202 may be unactuated and lie parallel to the substrate 204. The particular angles illustrated in FIGS. 2A-2B and described here are merely exemplary and not limiting. In some implementations, the on- and off-position angles may be between ±11 and ±13 degrees (inclusive), respectively. In other implementations, the on- and off-position angles may be between ±10 and ±18 degrees (inclusive), respectively.

[0041] While examples described herein primarily refer to DMDs, the first modulator 106 and/or the second modulator 110 may be different suitable modulation devices, such as MEMS arrays or other devices including a plurality of analog mirrors or digital mirrors.

Bit Sequencing

[0042] Normally, a typical DMD employs a single bit sequence per frame to obtain a certain bit per pixel (e.g., 16 bit/pixel) modulation. “Bit sequence” as used herein refers to how the micromirrors 202 are turned on and off. A frame period is partitioned into bit-times, or bitplanes, such that pixels may be on or off for each bit-time. In many embodiments, the bit sequence may be modified such that the higher order bits are spread across the frame period, and therefore, they may be repeated multiple times. For example, in one embodiment, the top bits (e.g., the top 12 bits of 16 bits) may be repeated for each subframe. This would allow a pattern with the top 12 bits to repeat - e.g., 16 times (in a 1/16 subframe subdivision embodiment). The lower significant bits would remain unaffected (e.g., spread across the entire frame period).

[0043] In one embodiment, it may be possible to have the beam steering device (e.g., mirrors or other elements) - e.g., the first modulator 106 - switch quickly and/or at a desired rate (e.g., 10-100 microseconds). This may be desirable so that the first modulator 106 may switch in a “dark” time between sequence repeats. During such a dark time, the display system may not be outputting any light to be rendered and/or projected. This may help to avoid noticeable and/or undesired visual effects. The fraction of time a pixel is on determines the level that is made. Bit-times are short enough that the bit on-times are not seen distinctly, and the level seen is the average on-time for the frame period. The frame rate for bit-sequencing may be different than the video frame rate. For example, the bit-sequencing frame rate for a 24 fps video could be four times that rate, at 96 fps. Additionally, the number of levels that can be made by bit sequencing is limited by the portioning of the frame period into bit-times. The shortest duration bit-time determines the precision of the levels.

[0044] FIG. 3 depicts a high level switching scheme 300 that matches the bit-sequence repeating pattern. Pattern 302 depicts the time periods (e.g., 302a and 302b) during which the bit sequence is repeating. Between these time periods is a period of dark time 303 between sequence repeats. As may be seen in pattern 304, the first modulator is able to switch multiple times (e.g., 304a, 304b, . . ., 304n) during the entire period and the dark time.

[0045] Dual modulation DMDs, such as the first modulator 106 and the second modulator 110, may use synchronized bit sequencing to make display levels. In some instances, the bit sequencing uses the same bit sequences for both the first modulator 106 and the second modulator 110 such that they are both on or off at the same time. In other instances, different bit sequences are used such that both the first modulator 106 and the second modulator 110 are on, both the first modulator 106 and the second modulator 110 are off, or only one of the first modulator 106 and the second modulator 110 are on at any time.

[0046] When the bit sequencing uses the same bit sequences for both the first modulator 106 and the second modulator 110 such that they are both on or off at the same time, the fraction of a frame time that both the first modulator 106 and the second modulator 110 are on determines the display level that is created. When both the first modulator 106 and the second modulator 110 are off, the contribution to the display level is non-zero, but is instead a very small display level that may be significant only when both the first modulator 106 and the second modulator 110 are off for the whole frame time. For example, let F be the display level and let both on time be the fraction of a frame time that both the first modulator 106 and the second modulator 110 are on. Let Y on be the display level when both the first modulator 106 and the second modulator 110 are on. Let Y off be. the display level when a single modulator is off such that, with dual modulation, both the first modulator 106 and the second modulator 110 are off and create a display level of Y off. It should be understood that to simplify these examples, Y off is chosen to be the same for both modulators, and that Eq. 1 and Eq. 2 would be appropriately modified if Y off were different for the two modulators. Accordingly, the created display level for a particular bit sequence is provided by Equation 1 :

Y = both_on_time * Y_on + (1 — both mjime') * Y_off² [Eq. 1]

[0047] The second term, (1-both on time) *Y off , is likely insignificant compared to the first term, except for when establishing a black level when the first term is zero.

[0048] FIGS. 4A-4C illustrate an example bit sequence that partitions a frame into ten equal-sized sub-intervals. In other embodiments, the bit sequences may have significantly more sub-intervals or sub-intervals of different sizes. FIG. 4A provides the bit sequence for the first modulator 106 over the frame. FIG. 4B provides the bit sequence for the second modulator 110 over the frame. FIG. 4C provides the dual-modulation transmittance level over the frame. As referred to herein, transmittance level may be interpreted as normalized display level. [0049] In the example of FIGS. 4A-4C, the same bit sequence is used for both the first modulator 106 and the second modulator 110. The value of Y on is 1 (full transmittance), and the value of Y off is 1/3000 (minimal transmittance). These values are merely examples. In particular, the Y off value may vary based on the contrast of the modulators (e.g. 1/contrast). For examples described herein, the first modulator 106 and the second modulator 110 have a contrast value of 3000: 1. The bit sequence has the first modulator 106 and the second modulator 110 on at the sub-intervals 3 and 8. Accordingly, the both on time value is 2/10. Solving for Equation 1 it is found that the display level is:

The second term (1-both on time) *Y off is a value visually insignificant compared to the first term, and can be ignored. Given that the bit sequence is partitioned into ten equal-sized subintervals, the display levels increase in tenths (0/10, 1/10, 2/10. . . 10/10) and only provide as many levels as the number of partitioned sub-intervals plus one. These display levels may be referred to as “coarse levels”.

[0050] Due to the limitations of switching the micromirrors 202 on and off, both on time may be a value from a limited set of discrete values including zero and some smallest value. The smallest value and the precision of other small values may not be sufficient for making all levels needed for a display. The other needed values may be made using other complimentary schemes such as dithering.

[0051] Additionally, bit sequencing may be modified to create additional levels. Rather than limiting the set of bit sequences to those having both the first modulator 106 and the second modulator 110 on or off at the same time, bit sequences may be included that have a single modulator on while the other modulator is off. For example, let one on time be the fraction of a frame time that only one modulator is on. The display level Y for a particular bit sequence is then provided by Equation 2:

Y = both_on_time * Y_on + one_on_time * Y_off + (1 — both_on_time — one_on_time) * Y_off² [Eq. 2]

[0052] When both on time is low, the second term one on time *Y off is not insignificant compared to both on time *Y on, and one on time may be manipulated to create many more bit sequences that make many more display levels. This relationship between both on time*Y on and one on time*Y off may be considered a course level with a fine adjustment, where the fine adjustment is used to make more precise display levels than simply using the course level. The fine adjustment may henceforth be referred to as “fine levels”.

[0053] FIGS. 5A-5C illustrate another example bit sequence that partitions a frame into ten equal-sized sub-intervals. In other embodiments, the bit sequences may have significantly more sub-intervals or sub-intervals of different sizes. FIG. 5A provides the bit sequence for the first modulator 106 over the frame. FIG. 5B provides the bit sequence for the second modulator 110 over the frame. FIG. 5C provides the transmittance level over the frame.

[0054] In the example of FIGS. 5A-5C, a different bit sequence is used for the first modulator 106 than the second modulator 110. The value of Y on is 1, and the value of Y off is 1/3000. A first bit sequence has the first modulator 106 on from 2.5 to 3.5 and 7.5 to 8.5 (or on at the sub-intervals 3 and 8). A second bit sequence has the second modulator 110 on at the subintervals 3, 5, 6, and 8. Accordingly, the both on time is 2/10 and the one on time is 2/10.

Solving for Equation 2, it is found that the display level is:

[0055] In this example, both the second term one on time*Y off and the third term (1- both on time-one one time) *Y off are visually insignificant compared to the first term both on time *Y on.

[0056] These fine adjustments have additional benefits when the desired display levels are less than the display level made by both on time set at its minimum non-zero discrete level. To make these lowest display levels with the fine adjustment, both on time is set to zero and one on time is manipulated to create smaller display levels. Fine adjustments may also make other needed levels such as those between both on time first and second non-zero levels, or between other greater but low coarse levels. As coarse levels increase, their precision approaches that needed for a display, and the utility of a fine adjustment diminishes.

[0057] FIGS. 6A-6C illustrate another example bit sequence that partitions a frame into ten equal-sized sub-intervals. In other embodiments, the bit sequences may have significantly more sub-intervals or sub-intervals of different sizes. FIG. 6A provides the bit sequence for the first modulator 106 over the frame. FIG. 6B provides the bit sequence for the second modulator 110 over the frame. FIG. 6C provides the transmittance level over the frame.

[0058] In the example of FIGS. 6A-6C, a different bit sequence is used for the first modulator 106 than the second modulator 110. The value of Y on is 1, and the value of Y off is 1/3000. A first bit sequence maintains the first modulator 106 off, and accordingly both on time is set to zero. A second bit sequence has the second modulator 110 on at the subintervals 5 and 6. Accordingly, one on time is 2/10. Solving for Equation 2, it is found that the display level is:

[0059] In this example, as both on time*Y on is zero, the second term one on time*Y off is visually significant, creating a useful dark level. With the first modulator 106 off for the entire frame, the second modulator 110 may be manipulated to create eleven useful dark discrete display levels.

[0060] The eleven dark levels are created by manipulating the second modulator 110 with the first modulator 106 being maintained off. These fine adjustment levels are visually significant, but are much less than the first non-zero coarse level when both the first modulator 106 and the second modulator 110 are on, even if the modulators are on for only one subinterval. The eleven fine adjustment levels alone do not completely fill the gap between coarse level zero and the first non-zero level. To have the fine adjustment fill-in the gaps between coarse levels, the coarse levels may be spaced more closely together.

[0061] As one example, consider a bit sequence for a display that is one bit sequencing method among many. Let the bit sequence be similar to that as previously shown in FIGS. 4A- 4C, 5A-5C, and 6A-6C, but the time frame is portioned into 2¹² sub-intervals such that the coarse levels that are made correspond to the values of a 12-bit number. Accordingly, the smallest coarse level has a value of l/(2¹²). Another example may make the same 2¹² levels or other similarly precise levels by using a coarser bit sequencing method complimented by other methods such as dither, as described below in more detail.

[0062] FIG. 7 shows perceptual quantizer (PQ) signal inputs for the display levels Y (in nits) that should be displayed, and the coarse discrete levels that would be displayed by the 12- bit bit sequencing method if the Y value corresponding to the maximum 12-bit number were 108 nits. In the example of FIG. 7, the lowest 200 PQ values are shown. The number of discrete PQ levels is much greater than the number of coarse 12-bit linear levels (indicated by “X”s in FIG. 7). Assuming the same Y off level of the previous examples, the maximum fine adjustment that can be made is sufficient to fill in the gaps between all the coarse levels, and the number of fine levels is sufficient to make all PQ levels. By controlling only one modulator during the off-time of the other, nearly 2¹² fine levels are added to each coarse level. If the second modulator 110 (e.g., the primary modulator) is controlled on for the entire time the first modulator 106 (e.g., the pre-modulator) is maintained off, a maximum fine adjustment level is achieved (indicated by “O”s in FIG. 7). If the maximum fine level added to the coarse level is greater than the next greater coarse level, then the gaps between coarse levels can be filled to make all PQ levels.

[0063] For dual-modulation DMD display, such as that provided by projection system 100, light from the first modulator 106 is cast onto the second modulator 110, but not at pixel -to- pixel precision. Rather, light from a pixel of the first modulator 106 (e.g., a premodulator pixel) is cast onto an adjacent group of pixels of the second modulator 110 (e.g., primary pixels).

When making a pixel level with both the first modulator 106 and the second modulator 110 on or off at the same, there are existing methods to reduce errors from this condition. However, when making a pixel level with the first modulator 106 off and the second modulator 110 on, an adjacent pixel with the first modulator 106 on may cast light onto the primary pixel that relies on the first modulator 106 being off, corrupting the expected level. This condition may occur whenever pixel levels are being made with the first modulator 106 off, particularly when creating dark levels that must be created only with the first modulator 106 off.

[0064] To avoid this corruption, the frame time may be partitioned into two intervals, one for coarse levels created with both the first modulator 106 and the second modulator 110 on or off at the same time, and another for fine levels made with only either the first modulator 106 or the second modulator 110 on. These two intervals may be further divided into sub-intervals for two bit sequences, one for the coarse levels and another for the fine levels. During the fine levels frame interval, all premodulator pixels are off, preventing any premodulator pixel from casting light onto a primary pixel that relies on the premodulator pixel being off. The fine levels frame interval is henceforth referred to as the premodulator off time.

[0065] Dedicating a fraction of the frame interval as a premodulator off time provides for making fine levels while preventing some spatial artifacts. However, during the premodulator off time, the peak brightness of the display is reduced by the fraction of the frame time that is the premodulator off time. Accordingly, the duration of the premodulator off time may be limited to achieve the necessary level precision while maintaining a sufficient peak brightness level.

Dithering Techniques

[0066] Dithering is intentionally-applied noise used to randomize quantization error and may also be used to create more display levels. While embodiments described herein primarily refer to the use of DMDs, the use of dithering is not restricted to displays using DMDs as dithering operates on frames made by any display technology. Dithering utilizes discrete levels that are created by some other means, such as the previously-described bit sequencing for DMDs. Dithering alternately shows those discrete levels such that their average over time is an in-between level, providing additional precision to the discrete levels created by bit sequencing.

[0067] While dithering and bit-sequencing both show alternately discrete levels that average over time to some other level, they both operate over different time periods. For example, while bit-sequencing may show different discrete levels within a frame period to make a level for the frame that is in-between the discrete levels, dithering may show frames with discrete levels such that many frames average to levels that are in-between the discrete levels.

[0068] To make a more precise level in-between two consecutive discrete levels, a dither method may use any number of frames to average those levels. As one example, if 64 frames are used and each frame can be shown at a lesser discrete level and a greater discrete level, then one of 64 more precise levels may be made by showing some of the 64 frames at the lesser discrete level, and the remainder of the 64 frames at the greater discrete level. While 64 frames are used as an example, any number of frames may be used for the dithering method.

[0069] However, if too many frames are shown at differing discrete levels, the dithering may be visible. Accordingly, to reduce the visibility, pixels in a local area of the frame that alternately show the same two discrete levels to make the same intermediate level may operate at different phases, mixing the showing of the discrete levels to minimize visibility.

[0070] Additionally, bit sequencing for DMDs and dithering may be used together as complimentary methods of creating or generating display levels. For example, a bit sequence may result in 2¹² discrete display levels that are then used by a dithering method that averages those levels over 64 frames. The dithering method adds 6 bits of precision to the 12 bits of the bit-sequencing, resulting in a total of 2¹⁸ display levels. The bit sequencing creates coarse levels. The dithering creates more precise intermediate levels in between the coarse levels created by bit sequencing.

[0071] To combine the bit sequencing technique and the dithering technique, in one example, the frame time is partitioned into two parts: (i) a longer interval for making bright levels by operating the first modulator 106 and the second modulator 110 on and off at the same time, and (ii) a shorter premodulator off interval for making dimmer levels by operating the first modulator 106 off while the second modulator 110 is on or off. The longer interval and the shorter interval may also separately be combined with dithering to increase precision, resulting in four layers of precision. [0072] FIGS. 8-9 illustrate example display levels of a dual-modulation DMD display with levels created using the four layers of precision. Only lower levels of precision are provided that require the most precision. In FIG. 8, 14-bit coarse linear levels are created during the frame interval when the first modulator 106 and the second modulator 110 are operated both on or off at the same time. Finer precision is created with 25 levels of dither for each coarse level. In FIG. 9, the 2 -bit premodulator off linear levels are made during the frame interval when the first modulator 106 is off and the second modulator 110 is on or off. FIG. 9 illustrates the precision achieved with 64 levels of dither for each premodulator off linear level, creating all PQ levels.

[0073] In the example of FIG. 9, the Y range and level precision for the display are achieved by four layers of increasing precision. The layers of precision include 14 bits of coarse linear levels with 25 levels of dither, and 2 bits of premodulator off linear levels with 64 levels of dither. These values are merely an example, and may be selected based on the physical limitations of devices, light efficiency, artifacts, and other performance characteristics.

[0074] One particular implementation of the bit-sequencing technique combined with the dithering technique in a dual -modulation projection system, such as the projection system 100, is illustrated in FIG. 10. The method 1000 of FIG. 10 may be performed by the controller 116 of FIG. 1, and may be implemented using hardware, software, firmware, or combinations thereof. In some examples, the method 1000 is implemented as instructions stored in a non-transitory computer-readable medium, such as a hard disk, or other storage medium contained in or associated with the projection system 100. Additionally, while the method 1000 provides a particular order of operations, in some instances, the steps of the method 1000 may be performed in a different order.

[0075] At step 1002, the method 1000 includes sending, during a first frame, control signals to the first modulator 106 and the second modulator 110 such that a plurality of first subframe periods sum to generate a first portion of a display level. For example, the micromirrors 202 of the first modulator 106 and the second modulator 110 are controlled to modulate light during first sub-frame periods of time at a first display level.

[0076] At step 1004, the method 1000 includes sending, during the first frame, control signals to the first modulator 106 and the second modulator 110 such that a plurality of second sub-frame periods sum to generate a second portion of the display level. For example, the micromirrors 202 of the first modulator 106 and the second modulator 110 are controlled to modulate light during second sub-frame periods of time at a second display level. In some instances, during the second sub-frame periods of time, the first modulator 106 is controlled to an off state. Accordingly, the second modulator 110 modulates light during a sub-frame time period in which the first modulator 106 may not modulate light.

[0077] At step 1006, the method 1000 includes repeating steps 1002 and 1004 for a plurality of image frames. In this manner, when averaged over the plurality of image frames, the first portion of the display level and the second portion of the display level combine to form the display level. For example, when averaged over a first plurality of frames, the first portions of the display level generate an average first portions level. When averaged over a second plurality of frames, the second portions of the display level generate an average second portions level. When summed, the average first portions level and the average second portions level generate the display level.

[0078] In this manner, the first modulator 106 and the second modulator 110 may be controlled on and off separately during different sub-frame periods, providing two levels of precision in creating display levels. Additionally, the display levels created by the first modulator 106 and the second modulator 110 may be averaged over a plurality of frames, providing two additional levels of precision in creating display levels.

[0079] Systems, methods, and devices in accordance with the present disclosure may take any one or more of the following configurations.

[0080] (1) A multi-modulation display system, the display system comprising: a light source; a first modulator, the first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source; a second modulator, the second modulate being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator; and a controller including an electronic processor and a memory, the memory comprising processor- readable instructions such that, when the electronic processor reads the processor-readable instructions, the electronic processor performs instructions including: sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a first sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first sub-frame period of time and during a second sub-frame period of time different from the first sub-frame period of time. [0081] (2) The display system according to (1), wherein the first modulator is one of either a DMD and a MEMS array, and wherein the second modulator is one of either a DMD and a MEMS array.

[0082] (3) The display system according to any one of (1) or (2), wherein the first subframe period of time is approximately l/(2¹² ) of a frame.

[0083] (4) The display system according to any one of (1) to (3), wherein the first subframe period includes a longer bit time interval than the second sub-frame period..

[0084] (5) The display system according to any one of (1) to (4), wherein a contrast value of the first modulator and a contrast value of the second modulator varies over an image frame.

[0085] (6) The display system according to any one of (1) to (5), wherein the electronic processor performs instructions further including: sending control signals to the first modulator such that the first plurality of mirrors are controlled to an off state during the second sub-frame period of time.

[0086] (7) The display system according to any one of (1) to (6), wherein the electronic processor performs instructions further including: sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a third sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the third sub-frame period of time and during a fourth sub-frame period of time different from the third sub-frame period of time.

[0087] (8) A processor-implemented method for controlling a multi-modulation display system comprising a light source, a first modulator including a first plurality of mirrors, a second modulator including a second plurality of mirrors, and a processor controlling the first modulator and the second modulator, the method comprising: sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light from the light source during a first sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light from the first modulator during the first sub-frame period of time and during a second sub-frame period of time different from the first sub-frame period of time.

[0088] (9) A multi-modulation display system, the system comprising: a light source; a first modulator, the first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source; a second modulator, the second modulate being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator; and a controller including an electronic processor and a memory, the memory comprising processor- readable instructions such that, when the electronic processor reads the processor-readable instructions, the electronic processor performs instructions including: sending, during a first frame, control signals to the first modulator and the second modulator such that a plurality of first sub-frame periods sum to generate a first portion of a display level, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of second sub-frame periods sum to generate a second portion of the display level, wherein, when averaged over a first plurality of frames, the first portions of the display level generate an average first portions level, wherein, when averaged over a second plurality of frames, the second portions of the display level generate an average second portions level, and wherein, when summed, the average first portions level and the average second portions level generate the display level.

[0089] (10) The display system according to (9), wherein the first modulator is one of either a DMD and a MEMS array, and wherein the second modulator is one of either a DMD and a MEMS array.

[0090] (11) The display system according to any one of (9) to (10), wherein the plurality of frames includes at least 64 frames.

[0091] (12) The display system according to any one of (9) to (11), wherein the first plurality of frames and the second plurality of frames include a different number of frames.

[0092] (13) The display system according to any one of (9) to (12), wherein each of the first sub-frame periods includes a longer bit time interval than each of the second sub-frame periods.

[0093] (14) The display system according to any one of (9) to (13), wherein the electronic processor performs instructions further including: sending, during the first frame, control signals to the first modulator such that the first plurality of mirrors are controlled to an off state during each of the second sub-frame periods.

[0094] (15) The display system according to any one of (9) to (14), wherein the electronic processor performs instructions further including: sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of third subframe periods sum to generate the first portion of the display level, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of fourth sub-frame periods sum to generate the second portion of the display level.

[0095] (16) A method for generating display levels in a dual-modulation system, the method comprising: sending, during a first frame, control signals to the first modulator and the second modulator such that a plurality of first sub-frame periods sum to generate a first portion of a display level, wherein the first modulator is illuminated by a light source, and wherein the second modulator is illuminated by light from the first modulator; and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of second sub-frame periods sum to generate a second portion of the display level, wherein, when averaged over a first plurality of frames, the first portions of the display level generate an average first portions level, wherein, when averaged over a second plurality of frames, the second portions of the display level generate an average second portions level, and wherein, when summed, the average first portions level and the average second portions level generate the display level.

[0096] (17) The method according to (16), wherein the plurality of frames includes at least 64 frames.

[0097] (18) The method according to any one of (16) to (17), wherein each of the first sub-frame periods includes a longer bit time interval than each of the second sub-frame periods.

[0098] (19) The method according to any one of (16) to (18), further comprising: sending, during the first frame, control signals to the first modulator such that the first plurality of mirrors are controlled to an off state during each of the second sub-frame periods.

[0099] (20) The method according to any one of (16) to (19), further comprising: sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of third sub-frame periods sum to generate the first portion of the display level, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of fourth sub-frame periods sum to generate the second portion of the display level.

[0100] (21) A non-transitory computer-readable medium storing instructions that, when executed by a processor of a projection system, cause the projection system to perform operations comprising the method according to any one of (16) to (20).

[0101] With regard to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the claims.

[0102] Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent upon reading the above description. The scope should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technologies discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the application is capable of modification and variation.

[0103] All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the technologies described herein unless an explicit indication to the contrary in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

[0104] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.

Claims

CLAIMS What is claimed is:

1. A multi-modulation display system, the display system comprising: a light source; a first modulator, the first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source; a second modulator, the second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator; and a controller including an electronic processor and a memory, the memory comprising processor-readable instructions such that, when the electronic processor reads the processor- readable instructions, the electronic processor performs instructions including: sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a first sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first sub-frame period of time and during a second sub-frame period of time different from the first sub-frame period of time.

2. The display system of claim 1, wherein the first modulator is one of either a DMD and a MEMS array, and wherein the second modulator is one of either a DMD and a MEMS array.

3. The display system of claim 1 or 2, wherein the first sub-frame period of time is approximately l/(2¹² ) of a frame.

4. The display system of any of claims 1 to 3, wherein the first sub-frame period includes a longer bit time interval than the second sub-frame period.

5. The display system of any of claims 1 to 4, wherein a contrast value of the first modulator and a contrast value of the second modulator varies over an image frame.

6. The display system of any of claims 1 to 5, wherein the electronic processor performs instructions further including: sending control signals to the first modulator such that the first plurality of mirrors are controlled to an off state during the second sub-frame period of time.

7. The display system of any of claims 1 to 6, wherein the electronic processor performs instructions further including: sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a third sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the third sub-frame period of time and during a fourth sub-frame period of time different from the third sub-frame period of time.

8. A multi-modulation display system, the display system comprising: a light source; a first modulator, the first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source; a second modulator, the second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator; and a controller including an electronic processor and a memory, the memory comprising processor-readable instructions such that, when the electronic processor reads the processor- readable instructions, the electronic processor performs instructions including: sending, during a first frame, control signals to the first modulator and the second modulator such that a plurality of first sub-frame periods sum to generate a first portion of a display level, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of second sub-frame periods sum to generate a second portion of the display level, wherein, when averaged over a first plurality of frames, the first portions of the display level generate an average first portions level, wherein, when averaged over a second plurality of frames, the second portions of the display level generate an average second portions level, and wherein, when summed, the average first portions level and the average second portions level generate the display level.

9. The display system of claim 8, wherein the first modulator is one of either a DMD and a MEMS array, and wherein the second modulator is one of either a DMD and a MEMS array.

10. The display system of claim 8 or 9, wherein the plurality of frames includes at least 64 frames.

11. The display system of any of claims 8 to 10, wherein the first plurality of frames and the second plurality of frames include a different number of frames.

12. The display system of any of claims 8 to 11, wherein each of the first sub-frame periods includes a longer bit time interval than each of the second sub-frame periods.

13. The display system of any of claims 8 to 12, wherein the electronic processor performs instructions further including: sending, during the first frame, control signals to the first modulator such that the first plurality of mirrors are controlled to an off state during each of the second sub-frame periods.

14. The display system of any of claims 8 to 13, wherein the electronic processor performs instructions further including: sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of third sub-frame periods sum to generate the first portion of the display level, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of fourth sub-frame periods sum to generate the second portion of the display level.

15. A method for generating display levels in a dual-modulation system, the method comprising: sending, during a first frame, control signals to the first modulator and the second modulator such that a plurality of first sub-frame periods sum to generate a first portion of a display level, wherein the first modulator is illuminated by a light source, and wherein the second modulator is illuminated by light from the first modulator; and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of second sub-frame periods sum to generate a second portion of the display level, wherein, when averaged over a first plurality of frames, the first portions of the display level generate an average first portions level, wherein, when averaged over a second plurality of frames, the second portions of the display level generate an average second portions level, and wherein, when summed, the average first portions level and the average second portions level generate the display level.

16. The method of claim 15, wherein the plurality of frames includes at least 64 frames.

17. The method of claim 15 or 16, wherein each of the first sub-frame periods includes a longer bit time interval than each of the second sub-frame periods.

18. The method of any of claims 15 to 17, further comprising: sending, during the first frame, control signals to the first modulator such that the first plurality of mirrors are controlled to an off state during each of the second sub-frame periods.

19. The method of any of claims 15 to 18, further comprising: sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of third sub-frame periods sum to generate the first portion of the display level, and sending, during the first frame, control signals to the first modulator and the second modulator such that a plurality of fourth sub-frame periods sum to generate the second portion of the display level.

20. A processor-implemented method for controlling a multi-modulation display system comprising a light source, a first modulator including a first plurality of mirrors, a second modulator including a second plurality of mirrors, and a processor controlling the first modulator and the second modulator, the method comprising: sending control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light from the light source during a first sub-frame period of time, and sending control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light from the first modulator during the first sub-frame period of time and during a second sub-frame period of time different from the first sub-frame period of time.

21. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a projection system, cause the projection system to perform operations comprising the method according to any of claims 15 to 20.