EP3054676B1 - Appareil de projection d'un élément d'image - Google Patents

Appareil de projection d'un élément d'image Download PDF

Info

Publication number
EP3054676B1
EP3054676B1 EP15154275.0A EP15154275A EP3054676B1 EP 3054676 B1 EP3054676 B1 EP 3054676B1 EP 15154275 A EP15154275 A EP 15154275A EP 3054676 B1 EP3054676 B1 EP 3054676B1
Authority
EP
European Patent Office
Prior art keywords
sample
time
light generation
point
actual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP15154275.0A
Other languages
German (de)
English (en)
Other versions
EP3054676A1 (fr
Inventor
Joerg REITTERER
Franz Fidler
Ferdinand Saint Julien-Wallsee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TriLite Technologies GmbH
Original Assignee
TriLite Technologies GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by TriLite Technologies GmbH filed Critical TriLite Technologies GmbH
Priority to EP15154275.0A priority Critical patent/EP3054676B1/fr
Priority to US15/018,901 priority patent/US9686519B2/en
Publication of EP3054676A1 publication Critical patent/EP3054676A1/fr
Application granted granted Critical
Publication of EP3054676B1 publication Critical patent/EP3054676B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141—Constructional details thereof
    • H04N9/317—Convergence or focusing systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3129—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM] scanning a light beam on the display screen
    • H04N9/3135—Driving therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H04N9/3185—Geometric adjustment, e.g. keystone or convergence

Definitions

  • the present invention relates to an apparatus for projecting a picture element composed of at least a first component based on a first sample of a picture signal and a second component based on a second sample of the picture signal.
  • MEMS micro-electro-mechanical systems
  • the light generation units, collimation optics, and beam combiners have to be aligned very carefully and precisely with respect to each other to achieve a perfect coaxial match of the beams since an adjustment after the assembly of the components is not possible.
  • multi-color projectors are complicated to manufacture, and a re-adjustment or calibration of the individual light generation units in case of mismatch is not possible.
  • US 2013/0162960 A1 discloses an image display device, in which green, red and blue lasers are projected onto a moving mirror to obtain an image in an image area. To calibrate the image display device, the display timings of the red and blue lasers are adjusted in such a way that the components of the red and blue lasers match the green laser component in the image area.
  • the invention provides for an apparatus of the aforementioned type, which is distinguished by:
  • Said actual first and second positions are derived from both an estimated second position onto which the second light generation unit would project the second component of the picture element upon receiving the second sample at the first point in time at its input and from an estimated first position onto which the first light generation unit would project the first component of the picture element upon receiving the first sample at the first point in time at its input, i.e., the reference position can be, for example, a (weighted) mean position based on the estimated first and second position.
  • each light generation unit projects a trajectory of its components of the picture elements onto the projection area, e.g., a projection plane.
  • the light generation units are mutually displaced and/or not calibrated, they each project trajectories which are mutually displaced, which leads to a picture of picture elements whose components, e.g., the color components, are offset from one another, i.e., un- or decalibrated.
  • the "red" component can be at a small spatial distance on the projection screen from the "green” component of one and the same picture element, yielding an unclear, blurred picture.
  • the components of at least one light generation unit are thus (re-)calibrated or shifted towards the components of another light generation unit.
  • Time-shifting all picture element components of at least one of the light generation units in that way leads to a picture with locally better matching (color) components.
  • a (preferably weighted median) reference trajectory can be chosen based on the trajectories of the first and the second light generation units, which means that, for example, the components of both the "red” and the “green” light generation units are time-shifted such that they meet "in the middle". This is especially favorable if there are three or more light generation units, such that their trajectories lie around the reference trajectory.
  • the invention thus provides for a projection apparatus which can be produced cheaply and efficiently since the light generation units do not have to be aligned in a hardware-based manner but can be calibrated by means of a software function or table providing said time-shifting of samples of the picture signal fed to the light generation units.
  • said actual first and second positions are derived from a reference position lying on a reference trajectory.
  • various ways of choosing a reference position or reference trajectory for said time-shifting of the samples are possible.
  • the apparatus of the invention is capable of projecting full-color picture elements composed of three components of different primary colors, i.e., the picture element is composed of said first component, said second component, and a third component based on a third sample of the picture signal, and the apparatus comprises three light generation units corresponding to such different colors.
  • the processor is further configured to output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area, wherein the fourth point in time is determined such that the distance between the actual first and third positions and the distance between the actual second and third positions are reduced.
  • the reference trajectory can be chosen in different ways.
  • the first, second, and third positions can be derived from a reference position lying on a reference trajectory, e.g., an arbitrary chosen, predetermined reference trajectory.
  • the processor can alternatively be further configured to output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area, said actual third position being derived from the actual second position.
  • the "green” trajectory is considered as the reference trajectory and the "red” and “blue” trajectories are shifted towards the "green” trajectory.
  • said actual first position and said actual second position can be further derived from an estimated third position onto which the third light generation unit would project the third component of the picture element upon receiving the third sample at the first point in time at its input
  • the processor can be further configured to output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area, said actual third position being derived from the estimated first position, the estimated second position, and the estimated third position.
  • a reference trajectory (or simply a reference point) is chosen by calculating a "mean” position towards which the trajectories are shifted, i.e., towards the middle of the "red”, “green”, and “blue” estimated position.
  • At least one of the second and third points in time is determined such that the distance between the actual first and second positions is minimized, i.e., the actual first position and the actual second position are derived such that they lie as close together as possible. Since, as mentioned above, the trajectories of the first and second light generation unit do not generally coincide, also the actual first and second positions do not coincide. Thus, by matching the first and second positions as closely as possible, the picture elements can be displayed as sharp as possible.
  • the actual first position and the actual second position are derived such that they lie on a line which is substantially parallel to one of the axes about which the mirror pivots.
  • the components can be projected on a (virtual) vertical grid pattern on the projection area.
  • the first and second actual position will still not match globally, at least their horizontal distances can be regular.
  • the second point in time can be shifted by an amount corresponding to an angular velocity of the mirror at the second point in time.
  • the mirror has an especially low velocity on its turning points (extrema of the trajectory), the picture to be displayed will usually become distorted on the "edges" of the picture.
  • the first sample is not projected by the first light generation unit if the angular velocity of the mirror at the second point in time is below a threshold. This results in the edges of the projected picture being "cut off", i.e., the projected picture is not blurred on its edges.
  • the mirror is preferably a MEMS mirror.
  • MEMS mirrors can be driven to pivot about two axes, e.g., resonant about a first axis and non-resonant about a second axis, to "sweep" the projection area in a line-like manner. This simplifies the control and calibration as described considerably.
  • the concept of the invention works for all kinds of light projection devices, however there are two preferred embodiments of the invention.
  • the light generation units are configured to emit light beams which are substantially parallel to each other and strike the mirror at mutual distances. This is especially advantageous since the mutual displacement of the components on the projection area can be compensated by the time-shifting scheme according to the invention. Furthermore, beam combiners between the light generation units and the mirror can be avoided.
  • the light generation units are configured to emit light beams which substantially cross at the mirror. This is especially advantageous since the size of the pivoting reflective surface can be minimized, which increases the resonance frequencies of the mirror and hence the resolution of the projected image.
  • the processor can be configured to interpolate an intensity of the first sample based on the estimated second position between the original intensity of the first sample and an intensity of the further sample.
  • the intensity of the "red” component should be high on the reference position and low at a neighboring reference position.
  • the "red” component actually strikes the projection screen between these reference positions and is thus interpolated between the intense and the light intensity values.
  • the reference position is dependent on the (estimated or actual) position of, e.g., the "green” component.
  • This linear interpolation is especially useful when the actual first positions and the actual second position lie on a vertical grid pattern since no horizontal displacement has to be corrected. Also, even in general cases it gives a quick interpolation between the sample's own intensity and the intensity of, e.g., the nearest sample.
  • the above-stated formula thus interpolates the intensity of the first sample based on its distance to the reference point above it and below it.
  • One of these reference points corresponds to the position where the component should actually be, and the other reference point corresponds to a position where another (neighboring) component should be.
  • the interpolated intensity is obtained by means of an irregular bilinear interpolation using the original intensity of the first sample and intensities corresponding to three further samples. This interpolation accounts for cases in which the component is projected generally between two reference trajectories and thus adjusts the intensity of the sample based on its original intensity and on the intensity of three further (neighboring) samples.
  • mappings between a point in time at which a sample is received by the processor and a point in time at which a sample is output by the processor can be stored in a memory connected to the processor, e.g., in form of functions or tables, and the processor uses said mappings, i.e., accesses said memory, when outputting said samples.
  • a method for setting-up said mapping function or table by means of a calibration run is disclosed.
  • an auxiliary table of points in time and corresponding estimated positions is created on the basis of which the above-mentioned function or table mapping first points in time to second points in time (or third or fourth points in time) can be set up.
  • the processor is configured to, before it starts its actual operation as defined above, output - as said "calibration run” - several first and second samples at points in time which are distributed over one whole mirror period to the first and second light generation unit, respectively, and store the positions onto which the light generation units project the corresponding components as said estimated first and second positions, together with each corresponding output time with respect to the mirror period, in the memory, yielding said auxiliary table.
  • the auxiliary table can then be used to find the nearest matching positions of different trajectories, or a trajectory and the predetermined reference trajectory, respectively, from the corresponding times, of which nearest matching positions the final time mapping function or table can then be set-up and stored in the memory for access by the processor during operation.
  • the invention also provides for a projection device which can re-calibrate said auxiliary table. This is especially useful since the calibration can be done, e.g., on a calibration screen at a close distance to the projection device, whereas the actual projection would be performed onto a different screen in a different distance to the projection device, without having to re-perform said calibration run to set up the auxiliary table.
  • Figs. 1 and 2 show a projection device 1 for projecting a multi-colored light spot, i.e., picture element 2, via a mirror 3 onto a projection area 4, e.g., a screen ( Fig. 1 ) or the human eye ( Fig. 2 ), in a distance d 1 .
  • the picture element 2 is composed of three individual components 5 1 , 5 2 , 5 3 .
  • the projection device 1 comprises three light generation units 6 1 , 6 2 , 6 3 - each, e.g., in the form of one and more light emitting diodes (LEDs) or laser diodes - which emit light beams 7 1 , 7 2 , 7 3 according to the primary colors red (R), green (G), and blue (B).
  • LEDs light emitting diodes
  • B blue
  • the projection device 1 could also only contain two (or more than three) light generation units 6 1 , 6 2 for purposes other than mixing colors according to the RGB color model, thereby emitting a picture element 2 composed of two (or more) components 5 1 , 5 2 .
  • the projection device 1 further comprises a carrier 8, onto which the mirror 3 and the light generation units 6 1 , 6 2 , 6 3 are mounted. Furthermore, an optical element such as a fast axis collimation (FAC) and/or a slow axis collimation (SAC) lens 11 can be mounted on the carrier 8.
  • the carrier 8 can be a single piece or composed of separate elements and can, e.g., include a subcarrier or substrate for mounting the light generation units 6 1 , 6 2 , 6 3 .
  • the light generation units are configured to emit light beams 7 1 , 7 2 , 7 3 which are substantially parallel to each other and strike the mirror 3 at mutual distances. If the light beams 7 1 , 7 2 , 7 3 were to be emitted at same times, this would also cause the components 5 1 , 5 2 , 5 3 of a single picture element 2 to be projected onto mutually displaced positions on the projection area 4.
  • the light generation units 6 1 , 6 2 , 6 3 are configured to emit light beams 7 1 , 7 2 , 7 3 which substantially cross at the mirror 3.
  • a smaller reflective surface 22 of the mirror 3 can be used, but the components 5 1 , 5 2 , 5 3 of a single picture element 2 would still be projected onto mutually displaced positions on the projection area 4.
  • Either of the embodiments of Figs. 1 and 2 can be used to project the picture element 2 onto a projection area 4 in the form of a screen or a human eye.
  • Fig. 3 shows the controlling of the light generation units 6 1 , 6 2 , 6 3 by means of a processor 9.
  • the processor 9 receives a picture signal 10 at its input 12.
  • Each sample s 1,i , s 2,i , s 3,i corresponds to one of the components 5 1 , 5 2 , 5 3 of the picture element 2 at the time t i , called 2 i , and will thus be denoted by 5 1,i , 5 2,i , 5 3,i from hereon.
  • the processor 9 outputs the samples s 1,i , s 2,i , s 3,i received at its input 12 to outputs 13 1 , 13 2 , 13 3 connected to corresponding inputs 14 1 , 14 2 , 14 3 of the three light generation units 6 1 , 6 2 , 6 3 .
  • Fig. 4 shows such a micro-electro-mechanical system (MEMS) mirror 3.
  • MEMS micro-electro-mechanical system
  • actuation principle of the MEMS mirror 3 e.g., electrostatic, electromagnetic, piezoelectric, electrothermal, or magnetostrictive.
  • an electromagnetic actuation by feeding a current to a coil 16 of the mirror 3 lying in a magnetic field B, a movement about the two axes 17, 18 is induced to pivot the mirror plate 22.
  • the light beam 7 1 is deflected in varying directions.
  • a MEMS mirrors 3 allows a fast pivoting movement about the axis 17 if the mirror 3 is driven at a resonance frequency corresponding to the axis 17.
  • the mirror 3 is driven at a lower (non-resonant) frequency about the other axis 18 to allow a line-like "scanning” or “sweeping” movement of the light beam 7 1 over the projection area 4, as will be described below in detail by means of Fig. 5 .
  • the least common multiple of the two pivoting periods around the two axes 17, 18 constitutes a mirror period p m , after which the mirror 3 resumes the same alignment as before.
  • Fig. 6 shows a realistic scanning pattern of three non-congruent trajectories 19 1 , 19 2 , 19 3 on the projection area 4 caused by deflecting the three light beams 7 1 , 7 2 , 7 3 onto the projection area 4. Due to the mutual displacement of the light generation units 6 1 , 6 2 , 6 3 , also the trajectories 19 1 , 19 2 , 19 3 are mutually displaced.
  • the components 5 1,i , 5 2,i , 5 3,i of a single picture element 2 i are thus spatially dispersed over different positions 20 1,i , 20 2,i , 20 3,i (depicted by squares, circles, and triangles, respectively, in the figures), yielding a picture on the projection area 4 which is misaligned or blurred in its color components.
  • Fig. 7 shows only two trajectories 19 1 , 19 2 which are mutually displaced both in a horizontal and in a vertical direction; it goes without saying that all explanations given herein for two trajectories and two colors apply mutatis mutandis to the case of three trajectories and three colors.
  • the processor 9 performs a "time-shifting" or "time-mapping" function on the samples s 1,i , s 2,i , s 3,i received at its input 12 for outputting them at different instances of time within a mirror period p m such that the components 5 1,i , 5 2,i , 5 3,i of a picture element 2 i are projected in closer spatial proximity to each other.
  • a "theoretical" or “estimated” reference trajectory 19 r is introduced, see Figs. 8 and 9 .
  • one of the two trajectories 19 1 , 19 2 is declared as the reference trajectory 19 r (trajectory 19 2 in this example).
  • a sample s 2,i is output to the light generation unit 6 2 by the processor 9 at the time t i (also called “first” point in time in the following), whereupon the component 5 2,i is actually projected onto the position p 2,a (also called actual "second” position in the following and serves as a reference position p r in this case).
  • the processor 9 actually outputs the sample s 1,i - according to its time-shifting function - not at the first time t i , but at another point in time t n in the course of the trajectory 19 1 , i.e., within the mirror period p m , called "second" point in time t n in the following, so that the sample s 1,i is projected onto the actual first position p 1,a .
  • the distance between the actual first and second positions (p 1,a , p 2,a ) is reduced with respect to the distance between the estimated first and second positions (p 1,e , p 2,e ).
  • the trajectory 19 2 is considered as the reference trajectory 19 r , it can thus be said that the second point in time t n and therefore the first actual position p 1,a are "derived" from the actual second position p 2,a (the reference position p r ).
  • the actual second position p 2,a is equal to an "estimated" second position p 2,e .
  • the processor 9 can access a corresponding function or table, e.g., when outputting said sample s 1,i , s 2,i , s 3,i , the function or table mapping first points in time t i to second points in time t n which can be stored in a memory 21 connected to the processor 9. More precisely, this function or table is a mapping between a point in time t i at which a sample is received by the processor 9 and a point in time t n , t m , t k at which a sample is output by the processor 9.
  • Said function or table can either be generated analytically by means of known mutual displacements or known aging effects or can be generated by means of a calibration run as follows.
  • the processor outputs several first and second samples s 1,i , s 1,i+1 , s 2,i , s 2,i+1 at points in time t i , t i+1 , ... distributed over one whole mirror period p m to the first and second light generation unit 6 1 , 6 2 , respectively.
  • Said points in time t i , t i+1 , ... are preferably spaced at regular intervals which are short enough to obtain a large number of samples over the mirror period.
  • Each output sample will then yield a component 5 1,i , 5 2,i on the projection area 4 at the distance d 1 from the projection device 1. The sum of the projected components thus draws the trajectories 19 1 , 19 2 on the projection area 4.
  • the processor In a second step, stores said positions drawn on the projection area 4, i.e., onto which the light generation units 6 1 , 6 2 have projected the components 5 1,i , 5 2,i , as estimated first and second positions p 1,e , p 2,e , together with each corresponding output time t i , t i+1 with respect to the mirror period p m , in the memory 21 in form of an auxiliary function or table.
  • said auxiliary table relates estimated positions p 1,e , p 2,e to their corresponding output times t i .
  • each estimated first position p 1,e of time t i is positionally matched to the nearest estimated second position p 2,e (generally: matched to a nearest reference position p r ) of time t n , as retrieved from the auxiliary table (generally: as retrieved from a predetermined auxiliary table), which results in a mapping of time t i to time t n . All time mappings obtained in this way yield a final mapping table or function to be stored in memory 21.
  • the final function or table mapping the first points in time t i to the second points in time t n can then be freshly created by determining mapping each first estimated position p 1,e towards the respective estimated second position p 2,e .
  • the reference trajectory 19 r can be chosen as a (e.g., weighted) mean trajectory lying between the trajectories 19 1 and 19 2
  • the reference position p r is chosed as a (e.g., weighted) mean position lying between the estimated first and second positions p 1,e and p 2,e .
  • each "second" (shifted) time t n is dependent on both the estimated second position p 2,e and the estimated first position p 1,e , i.e., on the (mean) reference trajectory 19 r .
  • the above time-shifting can be performed by the processor 9 mutatis mutandis. That is, the processor 9 outputs the second sample s 2,i at a "third" point in time t m within the same mirror period p m to the second light generation unit 6 2 , at which third point in time t m the second light generation unit 6 2 projects, via the pivoting mirror 3, its second component 5 2,i of the picture element 2 i onto the "actual" second position p 2,a on the projection area 4.
  • the actual second position p 2,a is "derived” from the estimated second position p 2,e and the estimated first position p 1,e , i.e., from the (mean) reference trajectory 19 r .
  • the reference trajectory 19 r can be predetermined, i.e., not dependent on the estimated first and second positions p 1,e p 2,e .
  • the reference trajectories 19 r can, for example, be chosen to be formed by straight parallel lines, such that the reference positions p r cover the projection area 4 in a regular pattern.
  • Fig. 12 shows the application of the described calibration scheme to all three light generation units 6 1 , 6 2 , 6 3 .
  • either one of the trajectories 19 1 , 19 2 , 19 3 can be considered as the reference trajectory 19 r or, e.g., a mean reference trajectory 19 r is taken, whereupon the actual positions p 1,a , p 2,a , p 3,a of the components 5 1,i , 5 2,i , 5 3,i stemming from the samples s 1,i , s 2,i , s 3,i at output times t n , t m , t k are each derived from the estimated positions p 1,e , p 2,e , p 3,e at their common input time t i , i.e., from the (mean) reference trajectory 19 r .
  • the corresponding function/s or table/s can be stored in the memory 21.
  • Fig. 13a shows the projection screen 4, onto which the first components 5 1,i have been projected in a calibrated manner as described, wherein the second components 5 2,i have been declared reference points on the reference trajectory 19 r .
  • Fig. 14a shows the corresponding samples s 1,i , s 2,i of the input picture signal 10 in the two component channels G in and R in for the color components "green” and "red". To emphasize the mapping, successive incoming samples s 1,i , s 2,i have different intensities I 1,i , I 2,i . For example, this could be used for a test signal used for the calibration technique described above.
  • Fig. 15a shows the mapped samples s 1,i , s 2,i appearing at the outputs 13 1 , 13 2 of the processor 9, i.e., after the time-shifting function, for the two component channels G out , ("green") and R out ("red”). It can be seen that, e.g., sample s 1,i at input time t i has been shifted to the output time t n and there spatially forms the picture element 2 i with closer proximity of the components 5 1,i and 5 2,i .
  • Fig. 15a it can be seen that samples s 1,i , s 2,i at the "edge" of the projection area 4 can be deleted, see the exemplary gap 24 in the upper diagram of Fig. 15a .
  • the first reason for this is that the angular velocity of the mirror 3 becomes too slow at its turning-points, such that the components 5 1,i , 5 2,1 cannot be projected accurately enough.
  • the second reason is that due to the mutual displacement there may be no overlap at all of the trajectories 19 1 , 19 2 in the edge area.
  • a first sample s 1,i and/or a second sample s 2,i is not projected by the respective light generation unit 6 1 , 6 2 if the angular velocity of the mirror 3 at the second (output) points in time t n , t m is below a given threshold. Only components 5 1,i , 5 2,i within the frame 23 on the projection area 4 are thus actually projected.
  • Figs. 14a and 15a show a variant of the described calibration scheme wherein the actual first positions p 1,a and the actual second positions p 2,a were derived such that they lie on lines 25 which are substantially parallel to one of the axes 17, 18 about which the mirror 3 pivots.
  • the intervals between the output times t n can be chosen to be irregular.
  • second points in time t n of a sample s 1,i are shifted by an amount corresponding to an angular velocity of the mirror 3 at the second point in time t n .
  • the non-linear, sine-like movement of the mirror 3 can be compensated and a perfect alignment along regularly spaced lines 25 ( Fig. 13b ) can be achieved.
  • components 5 1,i , 5 2,i could be skipped to account for the edge of the projection area 4.
  • all components 5 1,i , 5 2,i could be "squeezed” such that they all fit within the defined frame 23 on the projection area 4.
  • the frame 23 can be chosen so that the angular velocity of the mirror 3 within the frame 23 is always above a given threshold.
  • Figs. 16 and 17 show a further calibration technique performed with time-shifted components 5 1,i as described above.
  • a component 5 1,i should be at its reference position p r,i (in this case the actual second position p 2,a , in general a position derived from at least the estimated second position p 2,e ).
  • the intensity I i carried by the sample s 1,i should actually occur at the reference position p r,i .
  • the intensity I i,new of the sample s 1,i is interpolated between its original intensity I i corresponding to its reference position p r,i and an intensity I j which should occur at a neighboring reference position p r,j .
  • the latter intensity I j is carried by a sample s i,j (j ⁇ i) which should be at the reference position p r,j .
  • Fig. 17 corresponds to the calibration scheme of Fig. 13a .
  • the same principles as described above apply, but this time there is also an offset in the horizontal direction.
  • two more intensities I ⁇ , I p can be considered to perform a so-called irregular bilinear interpolation, which accounts for components 5 1,i , 5 2,i forming a non-regular pattern on the projection area 4.
  • a weighted average is determined by means of a distance d i between the first actual position p 1,a and a reference position p r,i derived from the estimated second position p 2,e , and distances d j , d o , d p between the first actual position p 1,a and three further reference positions p r,j , P r,o , P r,p corresponding to the further samples s 1,j , s 1,o , s 1,p .
  • a bilinear interpolation or any other interpolation whose mathematical outline is known in the state of the art can be performed, possibly also using the weighted average of more than four intensities.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Optics & Photonics (AREA)
  • Mechanical Optical Scanning Systems (AREA)

Claims (13)

  1. Appareil de projection d'un élément d'image (2i) composé d'au moins une première composante (51,i) basée sur un premier échantillon (s1,i) d'un signal d'image (10) et d'une deuxième composante (52,i) basée sur un deuxième échantillon (s2,i) du signal d'image (10), l'appareil (1) comprenant
    un support (8),
    un miroir (3), monté sur le support (8), étant configuré pour pivoter autour de deux axes (17, 18) et ayant une période de miroir (pm),
    au moins une première et une deuxième unité de génération de lumière (61, 62), chaque unité de génération de lumière (61, 62) étant montée sur le support (8), dirigée au miroir (3), et ayant une entrée (141, 142) par l'intermédiaire de laquelle elle peut être commandée, et
    un processeur (9) ayant une entrée (12) et une pluralité de sorties (131, 132), où chaque sortie (131, 132) est connectée à l'une des entrées (141, 142) des unités de génération de lumière (61, 62),
    le processeur (9) étant configuré pour
    recevoir, au niveau de son entrée (12), le premier échantillon (s1,i) et le deuxième échantillon (s2,i) à un premier point dans le temps (ti) dans la période de miroir (pm),
    sortir le premier échantillon (s1,i) à un deuxième point dans le temps (tn) dans la même période de miroir (pm) vers la première unité de génération de lumière (61), auquel deuxième point dans le temps (tn) la première unité de génération de lumière (61) projette, par l'intermédiaire du miroir (3) pivotant, la première composante (51,i) de l'élément d'image (2i) sur une première position effective (p1,a) sur une zone de projection (4),
    sortir le deuxième échantillon (s2,i) à un troisième point dans le temps (tm) dans la même période de miroir (pm) vers la deuxième unité de génération de lumière (62), auquel troisième point dans le temps (tm) la deuxième unité de génération de lumière (62) projette, par l'intermédiaire du miroir pivotant (3), la deuxième composante (52,i) de l'élément d'image (2i) sur une deuxième position effective (p2,a) sur la zone de projection (4),
    où au moins un parmi les deuxième et troisième points dans le temps (tn, tm) est déterminé de telle manière que la distance entre les première et deuxième positions effectives (p1,a, p2,a) est réduite,
    caractérisé en ce que lesdites première et deuxième positions effectives (p1,a, p2,a) sont dérivées à la fois à partir d'une deuxième position estimée (p2,e) sur laquelle la deuxième unité de génération de lumière (62) projetterais la deuxième composante (52,i) de l'élément d'image (2i) lors de la réception du deuxième échantillon (s2,i) au premier point dans le temps (ti) au niveau de son entrée (142), et à partir d'une première position estimée (p1,e) sur laquelle la première unité de génération de lumière (61) projetterait la première composante (51,i) de l'élément d'image (2i) lors de la réception du premier échantillon (s1,i) au premier point dans le temps (ti) au niveau de son entrée (141).
  2. Appareil selon la revendication 1, dans lequel lesdites première et deuxièmes positions effectives (p1,a, p2,a) sont dérivées d'une position de référence (pr) se situant sur une trajectoire de référence (19r).
  3. Appareil selon la revendication 1 ou 2, destiné à la projection d'un élément d'image (2i) composé de la première composante (51,i), de la deuxième composante (52,i), et d'une troisième composante (53,i), basée sur un troisième échantillon (s3,i) du signal d'image (10), l'appareil (1) comprenant trois unités de génération de lumière (61, 62, 63) correspondant à différentes couleurs, le processeur (9) étant en outre configuré pour
    sortir le troisième échantillon (s3,i) à un quatrième point dans le temps (tk) dans la même période de miroir (pm) vers la troisième unité de génération de lumière (63), auquel quatrième point dans le temps (tk) la troisième unité de génération de lumière (63) projette, par l'intermédiaire du miroir pivotant (3), la troisième composante (53,i) de l'élément d'image (2i) sur une troisième position effective (p3,a) sur la zone de projection (4),
    où le quatrième point dans le temps (tk) est déterminé de telle manière que la distance entre les première et troisième positions effectives (p1,a, p3,a) et la distance entre les deuxième et troisième positions effectives (p2,a, p3,a) soient réduites.
  4. Appareil selon l'une quelconque des revendications 1 à 3, dans lequel au moins un parmi les deuxième et troisième points dans le temps (tn, tm) est déterminé de telle manière que la distance entre les première et deuxième positions effectives (p1,a, p2,a) est minimisée.
  5. Appareil selon l'une quelconque des revendications 2 à 4, dans lequel la première position effective (p1,a) et la deuxième position effective (p2,a) sont dérivées de telle manière qu'elles se situent sur une ligne (25) qui est essentiellement parallèle à l'un des axes (17, 18) autour duquel le miroir (3) pivote.
  6. Appareil selon l'une quelconque des revendications 1 à 5, dans lequel le deuxième point dans le temps (tn) est décalé d'une grandeur correspondant à une vitesse angulaire du miroir (3) au deuxième point dans le temps (tn).
  7. Appareil selon l'une quelconque des revendications 1 à 6, dans lequel le premier échantillon (s1,i) n'est pas projeté par la première unité de génération de lumière (61) si la vitesse angulaire du miroir (3) au deuxième point dans le temps (tn) est en dessous d'un seuil.
  8. Appareil selon l'une quelconque des revendications 1 à 7, dans lequel le processeur (9) est en outre configuré pour interpoler une intensité (Ii,new) du premier échantillon (s1,i) basée sur la deuxième position estimée (p2,e) entre l'intensité d'origine (Ii) du premier échantillon (s1,i) et une intensité (Ij) d'un autre échantillon (s1,j).
  9. Appareil selon la revendication 8, dans lequel l'intensité interpolée est donnée par I i , new = I i d j d i + d j + I j d i d i + d j
    Figure imgb0013
     où
    Ii est l'intensité d'origine (Ii) du premier échantillon (s1,i),
    Ij est un intensité (Ij) de l'autre échantillon (s1,j),
    di est une distance entre la première position effective (p1,a) et une position de référence (pr,i) dérivée de la deuxième position estimée (p2,e), et
    dj est une distance entre la première position effective (p1,a) et une autre position de référence (pr,j) correspondant à l'autre échantillon (s1,j).
  10. Appareil selon la revendication 8, dans lequel l'intensité interpolée est obtenue au moyen d'une interpolation bilinéaire irrégulière en utilisant l'intensité d'origine (Ii) du premier échantillon (s1,i) et des intensités (Ij, Iο, Ip) correspondant aux trois autres échantillons (s1,j, s1,o, s1,p).
  11. Appareil selon l'une quelconque des revendications 1 à 10, dans lequel une mise en correspondance entre un point dans le temps (ti) auquel un échantillon est reçu par le processeur (9) et un point dans le temps (tn, tm, tk) auquel un échantillon est sorti par le processeur (9) est stocké dans une mémoire (21) connectée au processeur (9), et dans lequel le processeur (9) utilise ladite mise en correspondance lors de la sortie dudit échantillon (s1,i, s2,i, s3,i).
  12. Appareil selon l'une quelconque des revendications 1 à 11, dans lequel le processeur est configuré pour
    avant que le processeur ne reçoive le premier échantillon (s1,i) au premier point dans le temps (ti),
    sortir plusieurs premier et deuxième échantillons (s1,i, s2,i) à des points dans le temps (ti) distribués sur une période de miroir (pm) complète vers respectivement la première et la deuxième unité de génération de lumière (61, 62), et
    stocker les positions sur lesquelles les unités de génération de lumière (61, 62) projettent les composantes correspondantes (51,i, 52,i) sous forme desdites première et deuxième positions estimées (p1,e, p2,e), ensemble avec chaque temps de sortie correspondant (ti) par rapport à la période de miroir (pm) dans la mémoire (21) .
  13. Appareil selon la revendication 12, dans lequel le processeur (9) est configuré pour calculer de nouvelles première et deuxième positions estimées (p1,e, p2e) selon x d 2 = x d 1 d 2 d 1 + x ˜ 1 − d 2 d 1
    Figure imgb0014
     et y d 2 = y d 1 d 2 d 1 + y ˜ 1 − d 2 d 1 ,
    Figure imgb0015
     où
    xd1 est la composante horizontale de la première ou de la deuxième position estimée stockée (p1,e, p2,e);
    yd1 est la composante verticale de la première ou de la deuxième position estimée stockée (p1,e, p2,e);
    xd2 est la composante horizontale de la première ou de la deuxième position estimée nouvelle (p1,e, p2,e);
    yd2 est la composante verticale de la première ou de la deuxième position estimée nouvelle (p1,e, p2,e)
    x̃ est un décalage horizontal entre les unités de génération de lumière (61, 62) ;
    ỹ est un décalage vertical entre les unités de génération de lumière (61, 62) ;
    d1 est la distance à laquelle la zone de projection (4) se trouvait durant l'obtention de la première ou de la deuxième position estimée stockée (p1,e, p2,e); et
    d2 est la distance de la zone de projection (4) à laquelle les nouvelles première ou deuxième positions estimées (p1,e, p2,e) doivent être projetées.
EP15154275.0A 2015-02-09 2015-02-09 Appareil de projection d'un élément d'image Active EP3054676B1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP15154275.0A EP3054676B1 (fr) 2015-02-09 2015-02-09 Appareil de projection d'un élément d'image
US15/018,901 US9686519B2 (en) 2015-02-09 2016-02-09 Apparatus for projecting a picture element

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP15154275.0A EP3054676B1 (fr) 2015-02-09 2015-02-09 Appareil de projection d'un élément d'image

Publications (2)

Publication Number Publication Date
EP3054676A1 EP3054676A1 (fr) 2016-08-10
EP3054676B1 true EP3054676B1 (fr) 2019-04-03

Family

ID=52462836

Family Applications (1)

Application Number Title Priority Date Filing Date
EP15154275.0A Active EP3054676B1 (fr) 2015-02-09 2015-02-09 Appareil de projection d'un élément d'image

Country Status (2)

Country Link
US (1) US9686519B2 (fr)
EP (1) EP3054676B1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11895442B2 (en) 2020-03-30 2024-02-06 University Of Fukui Light beam emission device and light beam projection device equipped therewith

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6884322B2 (ja) * 2016-10-31 2021-06-09 国立大学法人福井大学 ２次元光走査ミラー装置の製造方法
GB2579801B (en) 2018-12-13 2021-04-14 Exalos Ag Superluminescent diode module
EP3890315B1 (fr) * 2020-04-02 2022-11-09 Trilite Technologies GmbH Projecteur et procédé de projection d'une image
US20230015702A1 (en) * 2021-07-13 2023-01-19 Google Llc Display system with variable beam expansion for multiple lasers
JP2023038676A (ja) 2021-09-07 2023-03-17 セーレンＫｓｔ株式会社 光導波路素子及び光源モジュール

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002286904A (ja) * 2001-03-27 2002-10-03 Seiko Epson Corp 光学部品およびこれを用いたプロジェクタ
JP2007140010A (ja) * 2005-11-17 2007-06-07 Seiko Epson Corp 画像表示装置
US8634024B2 (en) * 2009-06-15 2014-01-21 Microvision, Inc. Asynchronous scanning display projection
JP2011039326A (ja) * 2009-08-12 2011-02-24 Funai Electric Co Ltd 画像表示装置
JP2012058371A (ja) * 2010-09-07 2012-03-22 Funai Electric Co Ltd 画像表示装置
JP2014194495A (ja) * 2013-03-29 2014-10-09 Funai Electric Co Ltd プロジェクタ装置、ヘッドアップディスプレイ装置及びプロジェクタ装置の制御方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11895442B2 (en) 2020-03-30 2024-02-06 University Of Fukui Light beam emission device and light beam projection device equipped therewith

Also Published As

Publication number Publication date
US20160234469A1 (en) 2016-08-11
EP3054676A1 (fr) 2016-08-10
US9686519B2 (en) 2017-06-20

Similar Documents

Publication Publication Date Title
US9686519B2 (en) Apparatus for projecting a picture element
KR101264009B1 (ko) 디스플레이 및 다른 애플리케이션용 2차원 다각형 스캐너에 기초한 빔 스캐닝
US10944945B2 (en) Projection device and projection method, projection module, electronic device, and program
KR101713366B1 (ko) 이미지 투사 방법 및 장치
CN111566537A (zh) 用于hmds的扫描mems显示系统中的几何复用的rgb激光
US9052521B2 (en) Optical component calibration system for laser-based display device
JP7062952B2 (ja) 投影装置、投影モジュールおよび電子機器
US10187620B2 (en) Display device
US20090195644A1 (en) Image forming apparatus
US8582191B2 (en) Positioning sensing and position servo control
US20180192018A1 (en) Projection device and projection method, projection module, electronic device, and program
EP2711916A1 (fr) La correction d'une distorsion trapézoïdale d'images dans un appareil de projection laser
EP1328773B1 (fr) Detection et correction du desalignement de lasers multiples
JP5234514B2 (ja) 画像形成装置
US20120327373A1 (en) Scan-type image display device
US20140002804A1 (en) Laser Scanning Projection System with Reduced Speckle Contrast and Speckle Contrast Reducing Method
US10110866B1 (en) Scanning laser projectors with improved short projection distance image quality
US7817323B2 (en) Optical structure and an imaging system using the same
JP2005242036A (ja) 画像投射装置、および画像投射装置の制御方法
KR102210710B1 (ko) 레이저 디스플레이 장치 및 레이저 디스플레이 장치의 동작방법
EP3890315B1 (fr) Projecteur et procédé de projection d'une image
US11589441B2 (en) Laser projection device, method for operating same, and laser projector
WO2017145859A1 (fr) Dispositif de source de lumière laser et dispositif d'affichage à balayage laser

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20170209

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 9/31 20060101AFI20180814BHEP

INTG Intention to grant announced

Effective date: 20180914

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1117229

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190415

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602015027411

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190403

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1117229

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190803

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190703

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190704

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190703

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190803

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602015027411

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

26N No opposition filed

Effective date: 20200106

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200229

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200209

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200209

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200229

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190403

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230217

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230221

Year of fee payment: 9

Ref country code: DE

Payment date: 20230216

Year of fee payment: 9

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230907