WO2007100258A1 - Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use - Google Patents

Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use Download PDF

Info

Publication number
WO2007100258A1
WO2007100258A1 PCT/NO2007/000081 NO2007000081W WO2007100258A1 WO 2007100258 A1 WO2007100258 A1 WO 2007100258A1 NO 2007000081 W NO2007000081 W NO 2007000081W WO 2007100258 A1 WO2007100258 A1 WO 2007100258A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
disclosed
projector
display screen
detection points
Prior art date
Application number
PCT/NO2007/000081
Other languages
French (fr)
Inventor
Knut Krogstad
Original Assignee
3 D Perception As
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 3 D Perception As filed Critical 3 D Perception As
Priority to US12/280,828 priority Critical patent/US8610778B2/en
Priority to EP07715962.2A priority patent/EP1989887A4/en
Priority to CA2644117A priority patent/CA2644117C/en
Publication of WO2007100258A1 publication Critical patent/WO2007100258A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/74—Projection arrangements for image reproduction, e.g. using eidophor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B21/00—Projectors or projection-type viewers; Accessories therefor
    • G03B21/14—Details
    • G03B21/53—Means for automatic focusing, e.g. to compensate thermal effects
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B21/00—Projectors or projection-type viewers; Accessories therefor
    • G03B21/54—Accessories
    • G03B21/56—Projection screens
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H04N9/3182—Colour adjustment, e.g. white balance, shading or gamut
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H04N9/3185—Geometric adjustment, e.g. keystone or convergence

Definitions

  • the present invention relates to a method and a device for use in calibrating a projector 5 image display towards a display screen as disclosed in the preamble of appended method claims 1, 12 and 18, and appended device claims 46, 55 and 61. Furthermore, the invention relates to a display screen for displaying an image from at least one projector and for calibrating image geometry, as disclosed in the preamble of appended claim 88.
  • US Patent 6222593 describes a system for coordinating a plurality of projectors so that together they form one image.
  • EP Publications 1065890 and 0703714 describe a system including sensors for optimally adapting a projected image in relation to the display screen.
  • US Patent Publication US 2002/0180727 describes the adaptation of a plurality of projectors to 5 %how one image using, inter alia, edge blending.
  • the object of the present invention is to provide a technical solution that simplifies the calibration work and the equipment required, compared to the method and equipment used today.
  • the display screen mentioned above is characterised, according to the invention, in that the display screen is equipped with a plurality of discrete light detection points spaced from, on or in the front surface of the screen, or located immediately below the front surface, and where each light detection point or groups thereof are associated with a light detector, as set forth in appended claim 88.
  • Fig. 1 shows a projector projecting an image towards a display screen, and where detection points are located on the screen.
  • Fig. 2a is a block diagram showing an example of a circuit set-up for implementation of the invention.
  • Fig. 2b is a block diagram showing an example of a circuit set-up for implementation of image processing before storing to a storage medium.
  • Fig. 2c is a block diagram showing an example of a circuit set-up for implementation of feedback to one or more sources.
  • Figs. 3 a - 3j show non-limiting examples of test images that can be used for calibration.
  • Fig. 4a shows an enlargement of a plurality of image pixels in a projected image
  • Fig. 4b shows an indication of a detector point which detects several such image pixels simultaneously.
  • Figs. 5a - 5j show alternative embodiments of detector points in connection with a display screen.
  • Fig. 6 shows the display of partly overlapping images on a display screen using at least two projectors.
  • Fig. 7a is a schematic illustration of measured light as a function of time.
  • Figs. 7b and 7d are rough schematic illustrations of the gradual colour transition from white to black
  • Figs. 7c and 7e are a graphical presentation of brightness L as a function of respectively x with reference to Fig. 7b and of y with reference to Fig. 7d.
  • Fig. 8 illustrates the possibility of mechanical adjustment of the projector position and/or orientation based on measured deviations.
  • Fig. 9 indicates aspects of the invention where two projectors are to be adjustable to cover the same screen area.
  • the invention is especially useful in those cases where it is desirable to position a plurality of projectors that project composite images towards a curved screen with a view to obtaining a continuous image that covers a larger area, with higher resolution and with a greater light quantity than would have been obtained by projecting from a single projector.
  • Digital video projector systems are basically made for projecting onto flat canvases or screens.
  • a curved screen optionally a screen curved in both the longitudinal and the transverse direction, will result in a distorted image and a visible seam when the image is projected on the screen.
  • a lighter area where the images from the projectors overlap.
  • Today 3D perception AS supplies products that contain: necessary image processing to adjust the image geometry so as to neutralise the distortion on a curved screen; edge blend technology, i.e., so-called edge blending, to reduce the brightness in the transitional area between images from juxtaposed projectors; and compensation for colour and light unevenness in the images.
  • 3D perception AS has developed software which enables a user to digitally model screen geometry and projector position in the room, and to define what type of images are to be shown and what the result should look like from a given position in the room.
  • the digital model of the projection geometry is used by the software to compute necessary compensation for geometric distortion, edge blending, and compensation for brightness. Compensation data is transferred to the projector or image processor, which applies the compensation to the images before they are displayed.
  • a further calibration must therefore be made, which in today's products from 3D perception AS is carried out as a manual post-calibration based on visual inspection of the image, and a software-assisted shifting of parts of the image to compensate for such errors.
  • Fig. 1 shows a projector 1; 21 which projects an image 2 or a sequence of images towards a display screen 3.
  • the display screen is in addition equipped with a plurality of discrete light detection points 4, 5, 6, 7, 8 on or in the front surface of the screen, or located immediately below the front surface, each light detection point or groups thereof being associated with a light detector.
  • Fig. 5a shows that the light detection points 4 - 8 are associated with a respective, discrete light sensor 9 which is positioned almost flush with the front surface of the screen 3.
  • the light detection point may optionally also contain a light emitter 9'.
  • Fig. 5b shows that the detection point, optionally all the detection points or groups of detection points are associated with a light sensor 10 via optical fibre 10". Also in this case, a light emitter 10' can, if desired, be integrated in connection with the sensor 10.
  • Fig. 5c shows that the detection point can be placed on the front surface 3' of the screen 3, i.e., that the sensor, indicated by the reference numeral 11 in this figure, is, for example, attached with an adhesive 11 " to the front surface 3 ' .
  • the reference numeral 11 ' indicates an optional light emitter. It is also conceivable that such light detection points are attachable to the front of the display screen individually or in groups. This may be done by using fixing devices (not shown) that can be hooked onto the screen by means of mounting brackets, suspended using thread, wire or connecting cable, or that very fine tacks are used that can be pressed into the screen surface 3 ' .
  • Fig. 5d shows that all the detection points or groups of detection points, for example, the points 4, 5 are associated with a common light sensor 13 via optical fibres 14, 15.
  • An optional light emitter is indicated by 13'.
  • One advantage of this solution is that only one single light sensor is required, and furthermore only one single light emitter, if such is desired.
  • Fig. 5e indicates the possibility of locating at least one of the detection points 4 - 8 in the form of a light sensor 16 placed on the front of the display screen and at a distance therefrom. This distance does not necessarily need to be great, but it will be understood that any measurement errors can be compensated with respect to the distance of the light sensor from the screen surface.
  • An optional light emitter is indicated by the reference numeral 16' in this figure.
  • the light sensor 16 / light emitter 16' may either be mounted in a wired bar (not shown) or be mounted using signal wires 17, 17'.
  • Figs. 5f - 5j are a variant of what has been shown in Figs. 5a - 5e.
  • the front of the display screen is provided with a cover layer 3" which is partially translucent.
  • the light sensor 9 and the light emitter 9' can thus be positioned in the screen structure so that they are hidden under the cover layer when the screen is viewed from the front, whilst it is ensured that light respectively to and from the light sensor 9 and the emitter 9' passes through the cover layer 3".
  • Fig. 5g shows that the optical fibre 10" is located underneath the cover layer 3" in the screen structure 3, whilst it is ensured that light respectively to and from the sensor 10 and the emitter 10' passes through the cover layer 3" and via the optical fibre 10".
  • Fig. 5h where one light detector 13 and light emitter 13' is used for at least two optical fibres 14, 15. In this instance, the optical fibres are terminated underneath the cover layer 3 " .
  • Fig. 5i shows that the detection point can be placed on the front surface 3' of the screen 3, but where the front surface in this case optionally has a cover layer 3".
  • a cover layer 3" is, as mentioned, appropriate, for example, with the solutions shown, for instance, in Figs. 5f - 5h.
  • the sensor also designated by the reference numeral 11 here, can be attached, for example with adhesive 11", to the cover layer 3" on the front surface 3' of the display screen 3.
  • the reference numeral 11' indicates, as previously mentioned, an optional light emitter.
  • a light sensor 12 optionally with a light emitter 12', may be used that is fastened to the cover layer 3" and is oriented towards the cover layer 3" and thus towards the rear of the display screen 3.
  • This light sensor 12 / light emitter 12' is expediently secured to the cover layer in a detachable manner using an adhesive 12". It is also conceivable that such light detection points could be attachable to the front of the display screen individually or in groups.
  • fixing devices (not shown) that can be hooked on the s screen by means of mounting brackets, suspended using thread, wire or connecting cable, or that very fine tacks are used that can be pressed into the surface 3' of the screen and/or the cover layer 3".
  • Fig. 5j is a variant of the solution shown for the light sensor 12 and the light emitter 12' o in Fig. 5i and an alternative solution to that shown in Fig. 5e.
  • the possibility of locating at least one of the detection points 4 - 8 in the form of light sensor 16 placed on the front of the display screen and at a distance therefrom is indicated. This distance does not necessarily need to be great, but it will be understood that any measurement errors can be compensated with respect to the distance of the light sensor from the s screen surface.
  • An optional light emitter has been indicated by the reference numeral 16'in this figure.
  • the light sensor 16 / light emitter 16' can either be mounted in a wired bar (not shown) or be mounted using signal wires 17, 17'.
  • the light sensor 16 /light emitter 16' here is oriented towards the cover layer 3" and thus towards the back of the screen 3 for when image projection against the display screen 3 is done from the 0 rear.
  • the cover layer 3" which is shown in Figs. 5f - 5j may, for example, consist of at least one layer of paint, a sheet material, a foil or film or other suitable cover layer which has the property of being partially translucent.
  • the screen structure in this case is partially translucent, for example so that the front of the screen appears matt, it may optionally be possible to dispense with the cover layer 3", in which case the light sensor 12 / light emitter 12' is secured 0 directly on the front surface 3' as indicated in Fig. 5c, by using, for example, adhesive 12".
  • the same approach would entail turning the light sensor 16 / light emitter 16' 180°.
  • such light measurement can thus be carried out either through 5 discrete light sensors, for example, phototransistors, mounted in holes in the screen, mounted behind the screen, light being guided to the sensor via an optical fibre, mounted directly on the screen surface, mounted in holes in the screen in order via optical fibres to guide light from the detection point to a central sensor or sensor group, or mounted at a distance from the front surface of the screen.
  • 5 discrete light sensors for example, phototransistors, mounted in holes in the screen, mounted behind the screen, light being guided to the sensor via an optical fibre, mounted directly on the screen surface, mounted in holes in the screen in order via optical fibres to guide light from the detection point to a central sensor or sensor group, or mounted at a distance from the front surface of the screen.
  • each light detection point is known, so that light response in a known limited area or point can be measured. Any small deviations in the position of the light detection points will ultimately result in a corresponding deviation in image position, although this can probably be averaged out over a larger area.
  • a system with sensors mounted in the screen can, as mentioned, also conceivably be supplemented by light sources or light emitters 9', 10', 11', 12', 13', 16' as an additional reference for the user, e.g., in the form of light emitting diodes which mark the position of the horizon in a simulator projection theatre, or additional markings which indicate a grid or the like for facilitating the rough adjustment of projectors, for example, also after a maintenance operation. Furthermore, additional markings could be used for verifying the end result.
  • the light sensors selected from the type 9 - 13 and 16 and which are associated with the detection points 4 - 8, will be connected to a data collecting unit 18 which via cable 19 is connected to a processor 20 which, inter alia, will serve as an image processing unit. Communication from the sensors to the processor can take place by a fixed connection or wirelessly. It is also possible that the processor 20 is constituted of or is a part of the projector's own processor. In Fig. 2a, the reference numeral 22 indicates an image source.
  • the object of the light detection points is primarily to be able to determine which image pixel or pixels of the projected image 2 hit the respective detection point 4 - 8 on the screen. Although as an example just five detection points 4 - 8 are shown, it will be understood that fewer or more detection points are possible, without thereby departing from the inventive idea, and that these points may be positioned differently than in Figs. 2a - 2b, 6, 8 and 9.
  • the use of the light detection points when projecting at least one image towards the display screen also permits correlation of brightness and/or colours in the projected image by comparing measurements at the detection points, thereby making it possible to adjust relevant projector parameters correspondingly in the projector 21 or in the processor 20.
  • the processor will contain a correlator 20'.
  • Fig 2a indicates in broken lines supplementary processor 23, projector 24 and image source 25, these being conceivable if, for example, two partly overlapping images are to be displayed, as illustrated in Fig. 6. However, it is possible that in some cases the processor 20 and the source 22 may be the same if two projectors are used. Additional signals from other light detection points are delivered to such a processor 20 and/or 23 via, for example, cable 26. This will be explained in some more detail in connection with Fig. 6.
  • Fig. 2b it is indicated that the light sensors associated with the detection points 4 - 8 are connected to a data collecting unit 18 which via cable 19 is connected to an image processing unit 27.
  • An image source 28 is present, as is a storage medium 28' with the same image data or with a sequence of test images.
  • the storage medium 28' is located in a player 31 which delivers image data directly to the projector 21. Measurement errors that are detected by the detection points 4 - 8 pass to the unit 18 which via the connection 19 delivers detection data to a signal processing unit 27, which on the basis of these data establishes correction data.
  • An image source 28 is adapted to deliver images to the unit 27, thereby enabling the images to be corrected in the unit 27 to the form in which they will later be displayed from the projector 21 when the storage medium 28' has been removed and replaced by a new, corrected storage medium 30'.
  • the output from the unit 27 is delivered to a video recorder 29 which is adapted to record on a storage medium 30, for example, a CD-ROM or the like, the signal- processed video that comes from the source 28.
  • Said storage medium with corrected video, now designated medium 30' can then be put in the player 31 which thus delivers a corrected and adjusted video to the projector 21, a corrected image thus being displayed on the screen without the necessity of any additional corrections.
  • the light sensors selected from the type 9 - 13 and 16 and which are associated with the detection points 4 - 8 will be connected to a data collecting unit 18 which via a cable 19 is connected to a source 36.
  • This feedback to one or more sources such as the sources 36, 37 will be able to function if the source or sources are, for example, a 3D graphic generator or generators. If there are several projectors, such as projectors 21 ; 24, it will be necessary to have a corresponding number of sources, such as sources 36; 37.
  • Several types of graphic generators will be capable of compensating wholly or partly for computed measurement deviations, either by adjusting the computed field of view, by correcting for screen geometry or by adjusting edge blending and/or colours.
  • a commonly available 3D graphic generator will be able to serve as image processing unit and compensate for any orientation of the projector towards a flat screen. This can be done by correcting the computed field of view, so- called view frustum. More advanced 3D graphic generators could be used for nonlinear adaptation of image geometry towards curved screens, i.e., by introducing image processing into the image generator shown in connection with, for example, Figs. 2a and 2b and as otherwise described below. Communication from the sensors to the source or sources 36; 37 may take place by fixed connection or wirelessly.
  • the images that are successively projected on the screen are advantageously wholly or partly different or changing images.
  • the image or images used have a typical test image configuration. Rough examples of such test images partly to find where light detection points are in relation to the projected image 2 are shown in Figs. 3a - 3j.
  • the projected test image appears as a sectioned or divided image on the display screen, and the position of respective detection points is thus defined with relation to an x, y position or a group of x, y positions in the sectioned image.
  • test image pattern changes from one test image to the next, like, for example, the changes indicated in Fig. 3a to 3e inclusive.
  • Fig. 3f it will be seen that there is a vertical stripe in the 5 image and which in a subsequent image could, for example, have shifted to the left or to the right.
  • the exemplary respective patterns in Figs. 3g - 3j could change individually to more easily determine the correlation between test image pixels and detection point.
  • a set of test images can be used that illuminate the screen and the sensors associated with light detection points 4 - 8, e.g., with white lines on a black background that are projected on different points in the projectors' image surface, in order to then measure when such a line hits each light detection point.
  • a pattern of Q coding for example binary coding, of areas of the screen, so as to determine the position of many sensors with few measurements.
  • Q coding for example binary coding
  • Fig. 7a indicates measured light at a detection point as a function of time, hi the time interval indicated by tN, light is measured from image pixel in image N.
  • the measurements of the position of the light detection points will for each such point within a projector's coverage, give a pair of coordinates (x, y) in the projector's image coordinates which indicate what pixel is closest to the centre of the given light detection point.
  • the coding may be analogue and related to analogue parameters such as brightness or colour in a detected image pixel.
  • analogue measurement a test image can, for example, be shown with a colour gradient. This can, for example, take place at gradually varying intensity from black in one image edge to white in the opposite image edge.
  • the position can be measured by measuring the brightness or colour that hits the detection point.
  • the position can be determined in x, y.
  • Fig. 7b with associated brightness diagram in Fig. 7c, and as an alternative in Fig. 7d with associated brightness diagram in Fig. 7e, there is a rough outline of how gradual transition from white through the colour scale to black may appear as a test image.
  • Figs. 7b and 7d By using several test images with a colour gradient as shown in Figs. 7b and 7d, it is possible to determine the position of a measuring point by looking at the analogue light quantity that is detected at the respective light detection point.
  • By using combinations of gradients for example, from right to left, followed by a gradient from left to right, it is possible to compensate for measurement inaccuracies in analogue measurement.
  • Fig. 4a shows an enlarged section of a projected image where pixels designated 38 - 46 are shown.
  • Fig. 4b shows that one detector point 4; 5; 6; 7; 8 captures the pixels 39, 42, 43 and 45 completely, whilst the pixels 38, 40, 41, 44 and 46 are only partly captured.
  • a centre of mass computation of detected image pixels must be carried out to compute the position (centre) of the detection point in relation to the image pixels' hit in the detection points of the display screen.
  • This centre of mass computation may, for example, be made on the basis of the brightness of the image pixels and/or their colour or overall colour.
  • the detection points can also be used to control the focussing of the projectors. If more pixels give a "hit" towards a detection point than expected, this may be due to the projector not being correctly focussed.
  • This information can be used in its simplest form to give an operator a warning that the lens should be focussed, or in a more advanced system it can give feedback to an autofocus mechanism which adjusts motorised focus in the projector 1; 21; 24 based on these measurements.
  • the fact that a light detection point is hit by two or more image pixels at the same time, as indicated in Fig. 4b, may be due to either the image resolution (number of pixels per surface unit), or that the projected image is not completely focussed, or that the area of the light detection point is so large that it will cover several image pixels.
  • the accuracy in finding (x, y) will be equal to the pixel resolution (i.e., (x, y) is given in integer pixel coordinates). If the detection point is given a larger coverage area, as indicated in Fig. 4b, more pixels will be able to stimulate the sensor that is integrated in the detection point. This can then be used to enhance the precision of the position determination.
  • (x, y) can be computed with greater precision than integer pixels.
  • the object is with the aid of the processor 18 to compute the expected position of the individual detection points based on parameters of a theoretical model which at least include the x, y, z parameters of the projector and the x, y or x, y, z parameters of the detection points in relation to the position and orientation of the projector.
  • the processor will subsequently try to find the position of the projector in relation to the theoretical model based on the measured position of the detection points, after which the processor will cause deviations in the computation to be minimised in order to define the position of the projector in relation to the individual detection points.
  • Today 3D perception AS uses a three-dimensional model of the projection theatre which includes the position of projectors in six degrees of freedom, and screen geometry.
  • the projector may have variable setting of zoom, focus and lens change. More advanced parameters, as for instance pre-measured or computed lens distortion, effect of mirrors or the like may also be used in addition.
  • the error function is found by comparing computed and measured positions of each sensor.
  • the sum of the error function E(x proj , y proj , z proj , ⁇ proj , ⁇ proj , ⁇ proj , zoom proJ , ...) ⁇ Err i
  • an error function is used that is based on deviations between measured and computed coordinates represented in the projectors' image coordinates.
  • an error function that is based on deviations in physical position (x, y) or (x, y, z) within the projection theatre, optionally deviations in angles seen from eye point.
  • Tests involving the measurement of position on a screen indicate that some parameters are strongly connected and/or extremely sensitive to measuring noise.
  • the new position can be used for theoretical computation of necessary compensation for geometry and edge blending. Since the projection theatre, optics and measurements are not ideal, there will still be some residual errors which lead to the result not being perfect. I.e., EITJ is > 0 for some measuring points. This residual error can be compensated by using an extra correction, where residual error is known for each measured detection point/ "sensor" (x;, yj).
  • Linear or second order interpolation is used between the measuring points.
  • Fig. 6 shows two juxtaposed projectors 21 ; 24 which project respective images 47; 48 towards a common display screen 49 to give a composite image.
  • a plurality of light detection points 50 - 64 are shown. It will be seen that the two images 47, 48 have an overlapping area 65. In this area, the image pixels which appear are basically the same from each image, where these hit the screen 49. To prevent the brightness in this area from being too great, a per se known edge blending must be made in the area.
  • the light detection points 56, 57 and 58 are in this common edge area 65. These light detection points can, inter alia, be used to compare and adjust the adjacent images relative to each other with regard to image edge intensity and image pixels which represent corresponding picture elements in the composite image, so that there will be an exact overlapping. In this way, the adjacent, calibrated projector's parameters are also determined in relation to the display screen's respective detection points. Although it might be ideal to use, for example, two or more common light detection points in the edge area 65, satisfactory results will also be obtainable using just one detection point in the area.
  • the light detection points can be used for calibration of colour and light between projectors and/or within each image. Colour similarity and light quantity between projectors is a known problem which must also be taken into account during the construction of a projection theatre, and it is therefore obvious to consider this.
  • To be able to perform a good colour and/or light calibration it may be necessary to have an accurate sensor connected to the individual detection point, and it may also be necessary to calibrate such sensors in relation to each other after calibration of the position of the detection points. If the screen is to be surface-treated, e.g., painted, after calibration of position, image geometry, brightness and colour or colour combination, it will hardly be possible to make accurate measurements afterwards. In such a case, surface-located detection points will have to be considered.
  • Fig. 8 illustrates mechanical or electromechanical adjustment of the position and/or orientation of a projector 1; 21; 24 in relation to a display screen 65 based on measured deviations. It can be predetermined which image pixels in the projected image 66 are to hit the individual detection points 67 - 71 on the screen. The use of the detection points in this way will thus cause the corners of the projected image to come into the right place in that the projector is adjusted as regards lateral position, vertical position relative to the screen 65, and distance from the screen 65. In connection with this, the focus setting of the projector, for example, may also be adjusted. Strictly speaking, in this case the detection point 69 will not be necessary, but can nevertheless be used to optimise the setting before other image corrections are made, as explained above.
  • the present invention thus distinguishes itself in many ways, but especially by its simplicity, and it can be used for at least one of: adjustment of the image geometry of the image; adjustment of projection distortions in the image; adjustment of the edge areas of the image relative to the edges of the display screen; adjustment of edge transition or transitions to at least one adjacent projected image on the display screen; correction of colours in the image; correction of brightness in the image.
  • Fig. 9 outlines a solution where two projectors 1; 21 and 24 are to be adjustable to cover the same area on the display screen 65.
  • the images 66, 66' that shown on the screen 65 projected by the respective projector appear as far as possible as identical, so that overlapping identical images will appear as identical, or that alternation between the displays of the two projectors will not disturb the visual impression.
  • more detection points may optionally be used on the screen than the five that are shown as an example.
  • the adjustment methodology is in any case the same as shown and explained above in the description of the invention.
  • the solution that is shown in connection with Fig. 9 could be used where there is a need for redundancy or stand-by solution, i.e., that, for example, the projector 24 takes over the function of the projector 1; 21 if the last-mentioned has a malfunction.
  • the use may be in connection with a doubling of brightness, i.e., that both projectors are switched on at the same time and show the same image or that one of the projectors shows an image that gradually fades in, whilst the other projector shows an image that gradually fades out and vice versa.
  • the use may be related to forming "stereo-vision" or 3-dimensional vision, where the projectors have light filters mounted thereon, for example, colour filters or polarisation filters, and where the viewer of the screen 65 wears glasses which correspond to the projectors' light filters.
  • a further utilisation may be related to the use of the screen for two different purposes where it is advantageous or expedient to use separate projectors, for example, on one and the same display screen, a video display from one projector combined with computer graphics from the second projector.
  • other uses for providing special effects are also conceivable.

Abstract

A method and a device for use in calibrating a projector image display towards a display screen, wherein the display screen is equipped with a plurality of discrete light detection points on or in the front surface of the screen, or located immediately below the front surface, wherein each light detection point or groups thereof are associated with a light detector, wherein at least one image is projected towards the display screen, and wherein it is determined which image pixel or pixels of the projected image hit the respective detection point on the screen, and/or wherein brightness and/or colours in the projected image are correlated by comparing measurements at the detection points and relevant projector parameters are adjusted correspondingly.

Description

Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use
The present invention relates to a method and a device for use in calibrating a projector 5 image display towards a display screen as disclosed in the preamble of appended method claims 1, 12 and 18, and appended device claims 46, 55 and 61. Furthermore, the invention relates to a display screen for displaying an image from at least one projector and for calibrating image geometry, as disclosed in the preamble of appended claim 88. Q
As an illustration of the prior art, reference will be made to, inter alia, Norwegian Patent 310490 which is related to technology for image processing comprising geometry correction and edge adjustment, so-called edge blending, between images from juxtaposed projectors which project images towards a common display screen. As a s further illustration of the prior art, reference is made to, inter alia, US Patents 4974073, 5136390 and 6115022. These patents comprise primarily methods and apparatus for processing images in order to obtain seamless, multi-channel projection on a curved screen. These patents are only remotely related to processes for calibrating a system.
Q As further illustration of the prior art, reference is made to US Patent 6222593 which describes a system for coordinating a plurality of projectors so that together they form one image. EP Publications 1065890 and 0703714 describe a system including sensors for optimally adapting a projected image in relation to the display screen. US Patent Publication US 2002/0180727 describes the adaptation of a plurality of projectors to 5 %how one image using, inter alia, edge blending.
In recent years, a great deal of research work related to automatic calibration of multichannel projection systems has been carried out. Commercial systems based on the use of cameras are known from as early as 1985, and a common feature of these solutions is 0 that a digital camera has been used to read a projected image on the screen, so as to compute necessary compensation of the image. In its simplest form, a camera is placed in the observer position, whilst in more advanced forms several cameras are used, optionally also a camera mounted on a motorised arm. Recently, several products for such camera-based calibration have been launched. Some work has been done' on 5 research based on directing a digital camera towards the screen in addition to the projectors, so as to measure image position and adjust correction. As an example of such prior art, reference can be made to the web sites http://www.cs.unc.edu/Research/stc/publications/Raskar Vis99.pdf. http://www.cs.unc.edu/~marc/pubs/RaiiICPR2Q04.pdf httpV/www.olvmpusamerica.com/visionplex/product presentation.asp A video clip illustrating such a concept is also found on http://www.scalabledisplay.com/02 EEB mpegl 081004.mpg
The known solutions are technically complex, costly and awkward to use.
The object of the present invention is to provide a technical solution that simplifies the calibration work and the equipment required, compared to the method and equipment used today.
According to the invention, the method mentioned above, according to its aspects, is characterised by the features that are set forth in respective claims 1, 12 and 18.
Additional embodiments of the said aspects of the method are set forth in respective, appended subsidiary claims 2 - 11, 13 - 17 and 19 - 45.
The device mentioned above, according to its aspects, is characterised by the features set forth in appended claims 46, 55 and 61.
Additional embodiments of the said aspects of the device are set forth in appended subsidiary claims 47 - 54, 56 - 60 and 62 - 87.
The display screen mentioned above is characterised, according to the invention, in that the display screen is equipped with a plurality of discrete light detection points spaced from, on or in the front surface of the screen, or located immediately below the front surface, and where each light detection point or groups thereof are associated with a light detector, as set forth in appended claim 88.
Additional embodiments of the display screen are set forth in respective subsidiary claims 89 - 100.
The invention will now be described in more detail in relation to non-limiting exemplary embodiments. Fig. 1 shows a projector projecting an image towards a display screen, and where detection points are located on the screen.
Fig. 2a is a block diagram showing an example of a circuit set-up for implementation of the invention.
Fig. 2b is a block diagram showing an example of a circuit set-up for implementation of image processing before storing to a storage medium.
Fig. 2c is a block diagram showing an example of a circuit set-up for implementation of feedback to one or more sources.
Figs. 3 a - 3j show non-limiting examples of test images that can be used for calibration.
Fig. 4a shows an enlargement of a plurality of image pixels in a projected image, and Fig. 4b shows an indication of a detector point which detects several such image pixels simultaneously.
Figs. 5a - 5j show alternative embodiments of detector points in connection with a display screen.
Fig. 6 shows the display of partly overlapping images on a display screen using at least two projectors.
Fig. 7a is a schematic illustration of measured light as a function of time.
Figs. 7b and 7d are rough schematic illustrations of the gradual colour transition from white to black, and Figs. 7c and 7e are a graphical presentation of brightness L as a function of respectively x with reference to Fig. 7b and of y with reference to Fig. 7d.
Fig. 8 illustrates the possibility of mechanical adjustment of the projector position and/or orientation based on measured deviations.
Fig. 9 indicates aspects of the invention where two projectors are to be adjustable to cover the same screen area. The invention is especially useful in those cases where it is desirable to position a plurality of projectors that project composite images towards a curved screen with a view to obtaining a continuous image that covers a larger area, with higher resolution and with a greater light quantity than would have been obtained by projecting from a single projector.
Digital video projector systems are basically made for projecting onto flat canvases or screens. The use of a curved screen, optionally a screen curved in both the longitudinal and the transverse direction, will result in a distorted image and a visible seam when the image is projected on the screen. In the transition between the images there will also be a lighter area where the images from the projectors overlap. In this connection, reference is made to Fig. 6.
To avoid these unwanted effects it is desirable to process the images inside, or before the image information is delivered to, the projector so that these effects can be neutralised. Today 3D perception AS supplies products that contain: necessary image processing to adjust the image geometry so as to neutralise the distortion on a curved screen; edge blend technology, i.e., so-called edge blending, to reduce the brightness in the transitional area between images from juxtaposed projectors; and compensation for colour and light unevenness in the images.
A primary challenge in the installation of such a system is the adaptation of the image processing to the physical position and set-up of the projectors. For this purpose, 3D perception AS has developed software which enables a user to digitally model screen geometry and projector position in the room, and to define what type of images are to be shown and what the result should look like from a given position in the room.
The digital model of the projection geometry is used by the software to compute necessary compensation for geometric distortion, edge blending, and compensation for brightness. Compensation data is transferred to the projector or image processor, which applies the compensation to the images before they are displayed.
Since there will always be finite tolerances in how a projector is constructed, and in the exact position in which it is placed, the computed compensation will not be of a sufficient standard to obtain a completely seamless image. A small displacement in, e.g., optical components in the projector, or in the orientation of the projector will result in a large displacement in the image on the screen, with the result that the image is not perceived as continuous.
A further calibration must therefore be made, which in today's products from 3D perception AS is carried out as a manual post-calibration based on visual inspection of the image, and a software-assisted shifting of parts of the image to compensate for such errors.
Fig. 1 shows a projector 1; 21 which projects an image 2 or a sequence of images towards a display screen 3. The display screen is in addition equipped with a plurality of discrete light detection points 4, 5, 6, 7, 8 on or in the front surface of the screen, or located immediately below the front surface, each light detection point or groups thereof being associated with a light detector.
Fig. 5a shows that the light detection points 4 - 8 are associated with a respective, discrete light sensor 9 which is positioned almost flush with the front surface of the screen 3. The light detection point may optionally also contain a light emitter 9'.
Fig. 5b shows that the detection point, optionally all the detection points or groups of detection points are associated with a light sensor 10 via optical fibre 10". Also in this case, a light emitter 10' can, if desired, be integrated in connection with the sensor 10.
Fig. 5c shows that the detection point can be placed on the front surface 3' of the screen 3, i.e., that the sensor, indicated by the reference numeral 11 in this figure, is, for example, attached with an adhesive 11 " to the front surface 3 ' . The reference numeral 11 ' indicates an optional light emitter. It is also conceivable that such light detection points are attachable to the front of the display screen individually or in groups. This may be done by using fixing devices (not shown) that can be hooked onto the screen by means of mounting brackets, suspended using thread, wire or connecting cable, or that very fine tacks are used that can be pressed into the screen surface 3 ' .
Fig. 5d shows that all the detection points or groups of detection points, for example, the points 4, 5 are associated with a common light sensor 13 via optical fibres 14, 15. An optional light emitter is indicated by 13'. One advantage of this solution is that only one single light sensor is required, and furthermore only one single light emitter, if such is desired. Fig. 5e indicates the possibility of locating at least one of the detection points 4 - 8 in the form of a light sensor 16 placed on the front of the display screen and at a distance therefrom. This distance does not necessarily need to be great, but it will be understood that any measurement errors can be compensated with respect to the distance of the light sensor from the screen surface. An optional light emitter is indicated by the reference numeral 16' in this figure. The light sensor 16 / light emitter 16' may either be mounted in a wired bar (not shown) or be mounted using signal wires 17, 17'.
Figs. 5f - 5j are a variant of what has been shown in Figs. 5a - 5e. In the solutions shown in Figs. 5f - 5j, the front of the display screen is provided with a cover layer 3" which is partially translucent.
As shown in Fig. 5f, the light sensor 9 and the light emitter 9' can thus be positioned in the screen structure so that they are hidden under the cover layer when the screen is viewed from the front, whilst it is ensured that light respectively to and from the light sensor 9 and the emitter 9' passes through the cover layer 3".
The same considerations apply to that shown in Figs. 5g and 5h.
Fig. 5g shows that the optical fibre 10" is located underneath the cover layer 3" in the screen structure 3, whilst it is ensured that light respectively to and from the sensor 10 and the emitter 10' passes through the cover layer 3" and via the optical fibre 10". The same happens as shown in Fig. 5h, where one light detector 13 and light emitter 13' is used for at least two optical fibres 14, 15. In this instance, the optical fibres are terminated underneath the cover layer 3 " .
Fig. 5i shows that the detection point can be placed on the front surface 3' of the screen 3, but where the front surface in this case optionally has a cover layer 3". Such a cover layer 3" is, as mentioned, appropriate, for example, with the solutions shown, for instance, in Figs. 5f - 5h. hi this case, the sensor, also designated by the reference numeral 11 here, can be attached, for example with adhesive 11", to the cover layer 3" on the front surface 3' of the display screen 3. The reference numeral 11' indicates, as previously mentioned, an optional light emitter. If the display screen is of such a structure that image projection expediently takes place towards the rear of the screen, which should be regarded as a realistic alternative, a light sensor 12, optionally with a light emitter 12', may be used that is fastened to the cover layer 3" and is oriented towards the cover layer 3" and thus towards the rear of the display screen 3. This light sensor 12 / light emitter 12' is expediently secured to the cover layer in a detachable manner using an adhesive 12". It is also conceivable that such light detection points could be attachable to the front of the display screen individually or in groups. This may optionally be done using fixing devices (not shown) that can be hooked on the s screen by means of mounting brackets, suspended using thread, wire or connecting cable, or that very fine tacks are used that can be pressed into the surface 3' of the screen and/or the cover layer 3".
Fig. 5j is a variant of the solution shown for the light sensor 12 and the light emitter 12' o in Fig. 5i and an alternative solution to that shown in Fig. 5e. Here, the possibility of locating at least one of the detection points 4 - 8 in the form of light sensor 16 placed on the front of the display screen and at a distance therefrom is indicated. This distance does not necessarily need to be great, but it will be understood that any measurement errors can be compensated with respect to the distance of the light sensor from the s screen surface. An optional light emitter has been indicated by the reference numeral 16'in this figure. The light sensor 16 / light emitter 16' can either be mounted in a wired bar (not shown) or be mounted using signal wires 17, 17'. The light sensor 16 /light emitter 16' here is oriented towards the cover layer 3" and thus towards the back of the screen 3 for when image projection against the display screen 3 is done from the 0 rear.
The cover layer 3" which is shown in Figs. 5f - 5j may, for example, consist of at least one layer of paint, a sheet material, a foil or film or other suitable cover layer which has the property of being partially translucent. This means for that shown in Figs. 5i and 5j, s with respect to the light sensor 12 / light emitter 12' and light sensor 16 / light emitter 16", that the material in the screen structure 3 also here should be wholly or partially translucent. If the screen structure in this case is partially translucent, for example so that the front of the screen appears matt, it may optionally be possible to dispense with the cover layer 3", in which case the light sensor 12 / light emitter 12' is secured 0 directly on the front surface 3' as indicated in Fig. 5c, by using, for example, adhesive 12". For the solution shown in Fig. 5e, the same approach would entail turning the light sensor 16 / light emitter 16' 180°.
As shown in Figs. 5a - 5j, such light measurement can thus be carried out either through 5 discrete light sensors, for example, phototransistors, mounted in holes in the screen, mounted behind the screen, light being guided to the sensor via an optical fibre, mounted directly on the screen surface, mounted in holes in the screen in order via optical fibres to guide light from the detection point to a central sensor or sensor group, or mounted at a distance from the front surface of the screen.
To obtain a good result, it is important that the physical position of each light detection point is known, so that light response in a known limited area or point can be measured. Any small deviations in the position of the light detection points will ultimately result in a corresponding deviation in image position, although this can probably be averaged out over a larger area.
When a projection system is in normal operation, it is desirable that the light detection points with associated light sensors on or in the screen are visible to the least possible extent. The right choice of sensor at the relevant detection point will make it realistic to be able to paint over the screen with a thin layer of paint without the sensor function being significantly affected, but so that the detection points / the sensors are almost invisible.
A system with sensors mounted in the screen can, as mentioned, also conceivably be supplemented by light sources or light emitters 9', 10', 11', 12', 13', 16' as an additional reference for the user, e.g., in the form of light emitting diodes which mark the position of the horizon in a simulator projection theatre, or additional markings which indicate a grid or the like for facilitating the rough adjustment of projectors, for example, also after a maintenance operation. Furthermore, additional markings could be used for verifying the end result.
As shown in Fig. 2a, the light sensors selected from the type 9 - 13 and 16 and which are associated with the detection points 4 - 8, will be connected to a data collecting unit 18 which via cable 19 is connected to a processor 20 which, inter alia, will serve as an image processing unit. Communication from the sensors to the processor can take place by a fixed connection or wirelessly. It is also possible that the processor 20 is constituted of or is a part of the projector's own processor. In Fig. 2a, the reference numeral 22 indicates an image source.
The object of the light detection points is primarily to be able to determine which image pixel or pixels of the projected image 2 hit the respective detection point 4 - 8 on the screen. Although as an example just five detection points 4 - 8 are shown, it will be understood that fewer or more detection points are possible, without thereby departing from the inventive idea, and that these points may be positioned differently than in Figs. 2a - 2b, 6, 8 and 9.
Furthermore, it is possible that just one of the solutions shown in Figs. 5a - 5d may be used, optionally that two or more solutions are used simultaneously.
The use of the light detection points when projecting at least one image towards the display screen also permits correlation of brightness and/or colours in the projected image by comparing measurements at the detection points, thereby making it possible to adjust relevant projector parameters correspondingly in the projector 21 or in the processor 20. For this purpose, the processor will contain a correlator 20'.
Fig 2a indicates in broken lines supplementary processor 23, projector 24 and image source 25, these being conceivable if, for example, two partly overlapping images are to be displayed, as illustrated in Fig. 6. However, it is possible that in some cases the processor 20 and the source 22 may be the same if two projectors are used. Additional signals from other light detection points are delivered to such a processor 20 and/or 23 via, for example, cable 26. This will be explained in some more detail in connection with Fig. 6.
In Fig. 2b it is indicated that the light sensors associated with the detection points 4 - 8 are connected to a data collecting unit 18 which via cable 19 is connected to an image processing unit 27. An image source 28 is present, as is a storage medium 28' with the same image data or with a sequence of test images. The storage medium 28' is located in a player 31 which delivers image data directly to the projector 21. Measurement errors that are detected by the detection points 4 - 8 pass to the unit 18 which via the connection 19 delivers detection data to a signal processing unit 27, which on the basis of these data establishes correction data. An image source 28 is adapted to deliver images to the unit 27, thereby enabling the images to be corrected in the unit 27 to the form in which they will later be displayed from the projector 21 when the storage medium 28' has been removed and replaced by a new, corrected storage medium 30'. The output from the unit 27 is delivered to a video recorder 29 which is adapted to record on a storage medium 30, for example, a CD-ROM or the like, the signal- processed video that comes from the source 28. Said storage medium with corrected video, now designated medium 30', can then be put in the player 31 which thus delivers a corrected and adjusted video to the projector 21, a corrected image thus being displayed on the screen without the necessity of any additional corrections. This is a solution which is especially valuable for those cases where the alignment between projector and screen remains constantly unchanged, as is typical in a projection theatre. If there is a further projector 23, it may be appropriate to provide a further signal processing unit 32, source 33, recorder 34 and player 35. As shown for Fig. 2a, communication from the sensors to the signal processing unit 27; 32 may take place by a fixed connection or wirelessly. It is also conceivable that the player 31; 35 is not necessarily separate, but in integrated in the projector 21; 24.
As shown in Fig. 2c, the light sensors selected from the type 9 - 13 and 16 and which are associated with the detection points 4 - 8 will be connected to a data collecting unit 18 which via a cable 19 is connected to a source 36. This feedback to one or more sources such as the sources 36, 37 will be able to function if the source or sources are, for example, a 3D graphic generator or generators. If there are several projectors, such as projectors 21 ; 24, it will be necessary to have a corresponding number of sources, such as sources 36; 37. Several types of graphic generators will be capable of compensating wholly or partly for computed measurement deviations, either by adjusting the computed field of view, by correcting for screen geometry or by adjusting edge blending and/or colours. A commonly available 3D graphic generator will be able to serve as image processing unit and compensate for any orientation of the projector towards a flat screen. This can be done by correcting the computed field of view, so- called view frustum. More advanced 3D graphic generators could be used for nonlinear adaptation of image geometry towards curved screens, i.e., by introducing image processing into the image generator shown in connection with, for example, Figs. 2a and 2b and as otherwise described below. Communication from the sensors to the source or sources 36; 37 may take place by fixed connection or wirelessly.
The images that are successively projected on the screen are advantageously wholly or partly different or changing images.
It is also quite conceivable that the image or images used have a typical test image configuration. Rough examples of such test images partly to find where light detection points are in relation to the projected image 2 are shown in Figs. 3a - 3j. In the first instance, an attempt is made to find what area of the image the detector points are in by alternating between different images, and by such alternation gradually obtaining determination of the coordinates in the projected test image for the light detection point in question. As can be seen from Figs. 3a - 3j, the projected test image appears as a sectioned or divided image on the display screen, and the position of respective detection points is thus defined with relation to an x, y position or a group of x, y positions in the sectioned image. In this connection, it is conceivable that the test image pattern changes from one test image to the next, like, for example, the changes indicated in Fig. 3a to 3e inclusive. In Fig. 3f, it will be seen that there is a vertical stripe in the 5 image and which in a subsequent image could, for example, have shifted to the left or to the right. It is of course also possible that the exemplary respective patterns in Figs. 3g - 3j could change individually to more easily determine the correlation between test image pixels and detection point.
Q To perform measurement of an image, a set of test images can be used that illuminate the screen and the sensors associated with light detection points 4 - 8, e.g., with white lines on a black background that are projected on different points in the projectors' image surface, in order to then measure when such a line hits each light detection point. By finding one hitting point in the horizontal direction and one in the vertical direction, s it will be possible to determine the position of each light detection point seen from the projector's viewpoint, or put another way, decide which picture element in the projector hits the light detection point.
As an alternative to using horizontal and vertical lines, it is possible to use a pattern of Q coding, for example binary coding, of areas of the screen, so as to determine the position of many sensors with few measurements. A person of skill in the art will understand that if at least two successive image measurements are used, this will cause the image pixel or pixels detected in a respective light detection point to define respective image pixel coding for the determined image pixel position in the projected 5 image, which means that the position of each detection point is associated with at least one pixel address. Such coding may appear as a binary code. This is illustrated in more detail in connection with Fig. 7a which indicates measured light at a detection point as a function of time, hi the time interval indicated by tN, light is measured from image pixel in image N. In the time interval indicated by tN+1, light is measured from image Q N+l, etc. When the light measurement is determined to be above a set threshold, as indicated by the horizontal broken line, it can be viewed as a binary "1", whereas when it is below the threshold it can be viewed as a binary "0". The sequence of measurements may thus be regarded as a binary pattern or a binary code, which can be used to identify a position in the projected image. 5
The measurements of the position of the light detection points will for each such point within a projector's coverage, give a pair of coordinates (x, y) in the projector's image coordinates which indicate what pixel is closest to the centre of the given light detection point.
As an alternative or supplement to digital coding, the coding may be analogue and related to analogue parameters such as brightness or colour in a detected image pixel. When performing analogue measurement, a test image can, for example, be shown with a colour gradient. This can, for example, take place at gradually varying intensity from black in one image edge to white in the opposite image edge. In such an image, the position can be measured by measuring the brightness or colour that hits the detection point. By using several successive test images with gradients in different directions, or by using different colours to code different gradients, the position can be determined in x, y.
In Fig. 7b, with associated brightness diagram in Fig. 7c, and as an alternative in Fig. 7d with associated brightness diagram in Fig. 7e, there is a rough outline of how gradual transition from white through the colour scale to black may appear as a test image. By using several test images with a colour gradient as shown in Figs. 7b and 7d, it is possible to determine the position of a measuring point by looking at the analogue light quantity that is detected at the respective light detection point. By using combinations of gradients, for example, from right to left, followed by a gradient from left to right, it is possible to compensate for measurement inaccuracies in analogue measurement. As an alternative or supplement, it is also possible to perform a measurement of image content in ordinary projected images, i.e., non-typical test images. In this case, the choice is made to correlate measured analogue values with the signal level in different parts of the image in question at given points in time, so as to be able to determine the position of each measuring point during normal operation of the system. With sufficient variation in the images over time, it is thus possible to determine the position of the measuring point as the point in the image which gives best correlation between signal level and measured light quantity.
Fig. 4a shows an enlarged section of a projected image where pixels designated 38 - 46 are shown. Fig. 4b shows that one detector point 4; 5; 6; 7; 8 captures the pixels 39, 42, 43 and 45 completely, whilst the pixels 38, 40, 41, 44 and 46 are only partly captured. In such a case where the detection point is caused to be hit wholly or partly by two or more image pixels, a centre of mass computation of detected image pixels must be carried out to compute the position (centre) of the detection point in relation to the image pixels' hit in the detection points of the display screen. This centre of mass computation may, for example, be made on the basis of the brightness of the image pixels and/or their colour or overall colour. The detection points can also be used to control the focussing of the projectors. If more pixels give a "hit" towards a detection point than expected, this may be due to the projector not being correctly focussed. This information can be used in its simplest form to give an operator a warning that the lens should be focussed, or in a more advanced system it can give feedback to an autofocus mechanism which adjusts motorised focus in the projector 1; 21; 24 based on these measurements.
The fact that a light detection point is hit by two or more image pixels at the same time, as indicated in Fig. 4b, may be due to either the image resolution (number of pixels per surface unit), or that the projected image is not completely focussed, or that the area of the light detection point is so large that it will cover several image pixels.
If image pixels are only evaluated directly on the basis of a digital coding principle, there is a risk that the measurement accuracy will not be of a sufficient standard if the detection point has such a dimension that it captures several image pixels. In such a case, analogue weighting of several successive measurements at the detection point, optionally in addition to digital coding, could give better measurement data.
In a simple system with a physically small detection point, the accuracy in finding (x, y) will be equal to the pixel resolution (i.e., (x, y) is given in integer pixel coordinates). If the detection point is given a larger coverage area, as indicated in Fig. 4b, more pixels will be able to stimulate the sensor that is integrated in the detection point. This can then be used to enhance the precision of the position determination. Through the suggested said centre of mass computation of the pixels that "hit" the sensor, optionally by taking into account the light level and/or colour level detected for each pixel, (x, y) can be computed with greater precision than integer pixels.
Where, for example, laser projectors or CRT (cathode ray tube) projectors are used, it is possible by measuring brightness to adjust for, for example, convergence between the primary colours red, blue and green, or other primary colours. Such adjustment by correlation of the actual colours that are projected from a projector is essential where parameters and/or image processing must be carried to get the different sub-images to hit the same point on the display screen. According to a possible preferred solution, according to the invention, the object is with the aid of the processor 18 to compute the expected position of the individual detection points based on parameters of a theoretical model which at least include the x, y, z parameters of the projector and the x, y or x, y, z parameters of the detection points in relation to the position and orientation of the projector. The processor will subsequently try to find the position of the projector in relation to the theoretical model based on the measured position of the detection points, after which the processor will cause deviations in the computation to be minimised in order to define the position of the projector in relation to the individual detection points.
Here, it is also possible to conceive of the construction of a catalogue of measurement deviations associated with each detection point, and that a correction parameter is introduced which is specific to each detection point so as to cause shifting of a desired picture element or image pixel to as close to said detection point as possible.
The principle, in addition to measured (x, y) coordinate pairs, for computing an expected position of each light detection point and associated sensor on the basis of said theoretical model of the projector position in relation to screen and sensors, will now be described in more detail. Based on this explanation, a person of skill in the art will understand how this may be done in practice.
Today 3D perception AS uses a three-dimensional model of the projection theatre which includes the position of projectors in six degrees of freedom, and screen geometry. In addition, the projector may have variable setting of zoom, focus and lens change. More advanced parameters, as for instance pre-measured or computed lens distortion, effect of mirrors or the like may also be used in addition.
On the basis of the model's theoretical values for these parameters, it is possible to compute an estimate of where each sensor will be in the image coordinate system of the projector. These estimates can then be compared with the measured (x, y) values.
By parameterising the computation of theoretically computed (Xj , y; ) as a function of the model's parameters to a function of each detection point ("sensor") (i):
(xi> yi) = f(xproj> y proj' Zproj> ζ pro}' 1I proj > ζ proj > ZOOm proj > • • •> Xsensor,i> )* sensor.i' Zsemor,i) W This function can be used as a basis on which to compute an approximation of the exact values for the parameters of projector and screen. The computation may be done using the least square method, in that an error function is found Err, = {/(..., xsensoφ ysensorJ, zseι,sorJ) - (x,, y,))2 (2)
The error function is found by comparing computed and measured positions of each sensor. The sum of the error function E(xproj, yproj, zproj, ξproj, ηproj, ζproj, zoomproJ, ...) = ∑Erri(...) (3)
is sought to be minimised by solving the equation set below for each of the parameters in EO-
υXproj dE
= 0 (4) dyproj etc...
The solution to the equation set gives a set of new parameters for the projector's position, which can subsequently be used in the model.
In the example an error function is used that is based on deviations between measured and computed coordinates represented in the projectors' image coordinates. As an alternative which may give better results, it is possible to use, for example, an error function that is based on deviations in physical position (x, y) or (x, y, z) within the projection theatre, optionally deviations in angles seen from eye point.
Tests involving the measurement of position on a screen indicate that some parameters are strongly connected and/or extremely sensitive to measuring noise. By way of example, mention may be made of the connection between zoom and the distance of the projector from the screen, which cannot be separated when projecting on a flat screen, and which may only be separated slightly when projecting on a curved screen. Within the scope of the invention, and having regard to the equipment used for the installation in question, it may in certain cases be advantageous to introduce limitations and/or weighting of these parameters to ensure a good result. It may also be necessary to allow some parameters to be kept constant in the solution of the equation set and/or allow some parameters to be weighted differently.
After an optimal adjustment of the projector position and orientation has been determined, the new position can be used for theoretical computation of necessary compensation for geometry and edge blending. Since the projection theatre, optics and measurements are not ideal, there will still be some residual errors which lead to the result not being perfect. I.e., EITJ is > 0 for some measuring points. This residual error can be compensated by using an extra correction, where residual error is known for each measured detection point/ "sensor" (x;, yj).
For each measuring point the extra correction (displacement of geometry) that is to be added is
Figure imgf000017_0001
Linear or second order interpolation is used between the measuring points.
Fig. 6 shows two juxtaposed projectors 21 ; 24 which project respective images 47; 48 towards a common display screen 49 to give a composite image. In addition, a plurality of light detection points 50 - 64 are shown. It will be seen that the two images 47, 48 have an overlapping area 65. In this area, the image pixels which appear are basically the same from each image, where these hit the screen 49. To prevent the brightness in this area from being too great, a per se known edge blending must be made in the area.
However, it will be seen that the light detection points 56, 57 and 58 are in this common edge area 65. These light detection points can, inter alia, be used to compare and adjust the adjacent images relative to each other with regard to image edge intensity and image pixels which represent corresponding picture elements in the composite image, so that there will be an exact overlapping. In this way, the adjacent, calibrated projector's parameters are also determined in relation to the display screen's respective detection points. Although it might be ideal to use, for example, two or more common light detection points in the edge area 65, satisfactory results will also be obtainable using just one detection point in the area.
In order that a later human visual check can be made to see that the image geometry seems to be correct, it is possible in a final image or sequence of images to introduce point references into the image and see that these correspond with the detection points. In order to accomplish this, at least a majority of the detection points must be equipped with a light emitter, see 9'; 10'; 11 '; 13 '; 16' in Fig. 5, to visually view an adjusted test image after a completed calibration operation.
The use of a plurality of light detection points 4 - 8; 50 - 64 spread across the screen 3; 49 permits correlation of brightness and/or colours in the projected image by comparing measurements at the detection points, and on the basis of the comparisons adjusting related projecting parameters in the corresponding projector.
It is also conceivable that the light detection points can be used for calibration of colour and light between projectors and/or within each image. Colour similarity and light quantity between projectors is a known problem which must also be taken into account during the construction of a projection theatre, and it is therefore obvious to consider this. To be able to perform a good colour and/or light calibration it may be necessary to have an accurate sensor connected to the individual detection point, and it may also be necessary to calibrate such sensors in relation to each other after calibration of the position of the detection points. If the screen is to be surface-treated, e.g., painted, after calibration of position, image geometry, brightness and colour or colour combination, it will hardly be possible to make accurate measurements afterwards. In such a case, surface-located detection points will have to be considered.
Fig. 8 illustrates mechanical or electromechanical adjustment of the position and/or orientation of a projector 1; 21; 24 in relation to a display screen 65 based on measured deviations. It can be predetermined which image pixels in the projected image 66 are to hit the individual detection points 67 - 71 on the screen. The use of the detection points in this way will thus cause the corners of the projected image to come into the right place in that the projector is adjusted as regards lateral position, vertical position relative to the screen 65, and distance from the screen 65. In connection with this, the focus setting of the projector, for example, may also be adjusted. Strictly speaking, in this case the detection point 69 will not be necessary, but can nevertheless be used to optimise the setting before other image corrections are made, as explained above.
The present invention thus distinguishes itself in many ways, but especially by its simplicity, and it can be used for at least one of: adjustment of the image geometry of the image; adjustment of projection distortions in the image; adjustment of the edge areas of the image relative to the edges of the display screen; adjustment of edge transition or transitions to at least one adjacent projected image on the display screen; correction of colours in the image; correction of brightness in the image.
Fig. 9 outlines a solution where two projectors 1; 21 and 24 are to be adjustable to cover the same area on the display screen 65. In such a case, it is important that the images 66, 66' that shown on the screen 65 projected by the respective projector appear as far as possible as identical, so that overlapping identical images will appear as identical, or that alternation between the displays of the two projectors will not disturb the visual impression. This means that at the detection points 67 - 71 the same image pixels appear in the same images. To deal with the projection distortions that must be compensated by adjustments, more detection points may optionally be used on the screen than the five that are shown as an example. However, the adjustment methodology is in any case the same as shown and explained above in the description of the invention. The solution that is shown in connection with Fig. 9 could be used where there is a need for redundancy or stand-by solution, i.e., that, for example, the projector 24 takes over the function of the projector 1; 21 if the last-mentioned has a malfunction. In some cases, the use may be in connection with a doubling of brightness, i.e., that both projectors are switched on at the same time and show the same image or that one of the projectors shows an image that gradually fades in, whilst the other projector shows an image that gradually fades out and vice versa. In other cases, the use may be related to forming "stereo-vision" or 3-dimensional vision, where the projectors have light filters mounted thereon, for example, colour filters or polarisation filters, and where the viewer of the screen 65 wears glasses which correspond to the projectors' light filters. A further utilisation may be related to the use of the screen for two different purposes where it is advantageous or expedient to use separate projectors, for example, on one and the same display screen, a video display from one projector combined with computer graphics from the second projector. Of course, other uses for providing special effects are also conceivable.

Claims

P a t e n t c l a i m s
1.
A method for use in calibrating a projector image display towards a display screen, comprising providing the display screen with a plurality of discrete light detection points, and projecting at least one image towards the display screen, characterised in that each light detection point or groups thereof are associated with a light detector; and - that it is determined which image pixel or pixels of the projected image hit the respective detection point on the screen.
2.
A method as disclosed in claim 1 characterised in that said discrete light detection points are placed immediately below the front surface of the display screen.
3.
A method as disclosed in claim 2, characterised by providing the front surface of the screen with a partially translucent cover layer which covers said discrete light detection points.
4.
A method as disclosed in claim 3 characterised in that the cover layer consists of at least one layer of paint, a sheet, a foil or a film.
5.
A method as disclosed in claim 1, characterised in that said discrete light detection points are placed at a distance from, on or in the front surface of the screen.
6.
A method as disclosed in claims 1, 2, 3, 4 or 5, characterised also by adjusting relevant image parameters or projector parameters so that a desired image pixel or desired image pixels hit a respective detection point.
7.
A method as disclosed in claim 1 or 2, characterised by - delivering image adjustment parameters to an image signal processor for storage of adjusted images on a storage medium, said storage medium being adapted to subsequently deliver to the projector adjusted image information for display.
8.
A method as disclosed in any one of claims 1 - 6, characterised by
- a) delivering signals from the detection points to a signal processing unit;
- b) receiving in the signal processing unit image signals from a source; - c) adjusting the received image signals; and
- d) delivering the adjusted image signals to the projector.
9.
A method as disclosed in any one of claims 1 - 6, characterised by
- a) delivering signals from the detection points to an image source;
- b) adjusting the image signals in the image source by using the detection point signals; and
- c) delivering the adjusted image signals to the projector.
10.
A method as disclosed in claim 8, characterised by
- repeating steps a) - d) until optimal image adjustment is achieved.
1 1.
A method as disclosed in claim 9, characterised by
- repeating steps a) - c) until optimal image adjustment is achieved.
12. A method for use in calibrating a projector image display towards a display screen, comprising providing the display screen with a plurality of discrete light detection points, wherein at least three of the light detection points are positioned close to the edges or corners of the display screen, and projecting at least one image towards the display screen, characterised by - physically shifting the projector by adjusting the physical position of the projector with respect to at least one of: lateral direction, vertical direction, orientation and distance to the screen, by simultaneously correlating measured image pixel information at the detection points with theoretical image pixel information for these detection points until the measurement deviation has been minimised as a consequence of said adjustment; and
- subsequently fixing the position of the projector.
13.
A method as disclosed in claim 12, characterised in that said discrete light detection points are placed immediately below the front surface of the display screen.
14.
A method as disclosed in claim 13, characterised by providing the front surface of the screen with a partially translucent cover layer which covers said discrete light detection points.
15.
A method as disclosed in claim 14, characterised in that the cover layer consists of at least one layer of paint, a sheet, a foil or a film.
16. A method as disclosed in claim 12, characterised in that said discrete light detection points are placed at a distance from, on or in the front surface of the screen.
17.
A method as disclosed in any one of claims 1 - 16, characterised in that groups of light detection points are associated with a light detector.
18.
A method for use in calibrating a projector image display towards a display screen, comprising providing the display screen with a plurality of discrete light detection points, and projecting at least one image towards the display screen, characterised by
- associating a light detector with each light detection point or groups of light detection points;
- correlating brightness and/or colours in the projected image by comparing measurements at the detection points; and
- adjusting relevant projection parameters or image parameters correspondingly.
19.
A method as disclosed in claim 18, characterised in that said discrete light detection points are placed immediately below the front surface of the display screen.
20.
A method as disclosed in claim 19, characterised by providing the front surface of the screen with a partially translucent cover layer which covers said discrete light detection points.
21.
A method as disclosed in claim 20, characterised in that the cover layer consists of at least one layer of paint, a sheet, a foil or a film.
22. A method as disclosed in claim 18, characterised in that said discrete light detection points are placed at a distance from, on or in the front surface of the screen.
23.
A method as disclosed in any one of claims 18 - 23, characterised in that groups of light detection points are associated with a light detector.
24.
A method as disclosed in any one of claims 1 - 23, characterised in - that there is successively projected on the screen wholly or partly different or changing images.
25.
A method as disclosed in any one of claims 1 - 24, characterised in
- that said image or images are at least one dedicated test image.
26.
A method as disclosed in claim 25, characterised in - that said projected test image appears as a sectioned or divided image on the display screen, and that the position of the respective detection point is defined with relation to the x, y position or group of x, y positions in the sectioned image.
27.
A method as disclosed in one of claims 1 - 26, characterised in
- that at least two successive image measurements cause the detected image pixel or pixels to define respective image pixel coding so that the position of each detection point is associated with at least one pixel address.
28.
A method as disclosed in claim 27, characterised in - that the coding appears as a binary code.
29.
A method as disclosed in claim 27, characterised in
- that the coding is analogue and is related to analogue parameters such as brightness or colour in a detected image pixel or detected image pixels.
30.
A method as disclosed in any one of claims 1 - 29, characterised in - that a detection point is caused to be hit wholly or partly by two or more image pixels, and that a centre of mass computation of the detected image pixels is carried out to compute the position of the detection point in relation to the image pixels' hit on the display screen and/or to adjust the image pixels with respect to at least one of the following parameters: brightness, colour and overall colour.
31.
A method as disclosed in claim 30, characterised in
- that the centre of mass computation is done on the basis of the brightness and/or colour or overall colour of the image pixels.
32.
A method as disclosed in any one of claims 1 - 29, characterised in
- that for the adjustment of convergence of a plurality of primary colours emitted from a projector, for example, of the laser projector type or the cathode ray tube type, the brightness of the individual colours is measured at at least one detection point.
33.
A method as disclosed in one of claims 1 - 32, characterised by
- computing the expected position of the individual detection points based on the
5 parameters of a theoretical model which at least include the x, y, z parameters of the projector and the x, y or x, y, z parameters of the detection points in relation to the position and orientation of the projector;
- seeking and finding the position of the projector in relation to the theoretical model based on the measured position of the detection points; and Q - causing deviations in the computation to be minimised in order to define the position of the projector in relation to the individual detection points.
34.
A method as disclosed in any one of preceding claims 1 - 33, s characterised by
- constructing a catalogue of measurement deviations associated with each detection point; and
- introducing a correction parameter that is specific to each detection point so as to cause shifting of desired at least one picture element or image pixel to as close to said 0 detection point as possible.
35.
A method as disclosed in any one of preceding claims 1 - 17, and claims 24 - 34 when subsidiary to any one of claims 1 - 17, 5 characterised by
- correlating brightness and/or colours in the projected image by comparing measurements at the detection points, and by adjusting related projection parameters in the projector or image parameters correspondingly.
Q
36.
A method as disclosed in claim 31 or 32, characterised in
- that the centre of mass computation is used to cause optimal focussing towards the display screen of the test image projected from the projector.
S
37.
A method as disclosed in any one of preceding claims 1 - 36, characterised in - that each detection point is associated with a respective discrete light sensor.
38.
A method as disclosed in any one of preceding claims 1 - 36, characterised in
- that all the detection points or groups of detection points are associated with a common light sensor via optical fibres.
39. A method as disclosed in claim 37 or 38, characterised in
- that the light sensor or sensors and the projector communicate with a common processor.
40. A method as disclosed in claim 39, characterised in
- that the light sensor or sensors communicate with the processor via wireless communication.
41. A method as disclosed in any one of preceding claims 1 - 40, wherein the projector cooperates with at least one simultaneously operative juxtaposed projector for projecting a composite image on the display screen, characterised in
- that the parameters of the adjacent projectors in relation to the respective detection points of the display screen are compared and adjusted in relation to each other with respect to image intensity and image pixels that represent corresponding picture elements in the composite image;
- allowing the composite image to consist of two wholly or partly overlapping images;
- comparing the position of corresponding picture elements in an overlapping area formed by the images projected from the respective projector by detection at at least one detection point in the overlapping area; and
- adjusting the image parameters in the images from the respective projector to give the images identical or almost identical pixel positions in the overlapping area which forms the composite image.
42.
A method as disclosed in any one of preceding claims 1 - 40, wherein the projector is intended to substitute or be substituted by at least one juxtaposed projector for providing substitute image display on the display screen, characterised in
- that the parameters of the adjacent projectors in relation to the respective detection points of the display screen are compared and adjusted relative to each other with respect to image intensity and image pixels that represent corresponding picture elements in the display area in question; - comparing the position of corresponding picture elements in the display area formed by the images projected from the respective projector by detection at at least one detection point in the display area; and
- adjusting the image parameters in the images from the respective projector to give the images identical or almost identical pixel positions in the display area.
43.
A method as disclosed in any one of preceding claims 1 - 40, wherein the projector cooperates with at least one juxtaposed projector for projecting a composite image on the display screen, characterised by
- comparing and adjusting the parameters of the adjacent, calibrated projectors in relation to the respective detection points of the display screen relative to each other with respect to image edge intensity and image pixels that represent corresponding picture elements in the composite image; - comparing the position of corresponding picture elements in the display area formed by the images projected from the respective projector by detection at at least one detection point in the display area that is the same for respective images; and
- adjusting the parameters of the projectors in relation to each other with respect to image edge intensity and image pixels that represent said corresponding picture elements in the composite image.
44.
A method as disclosed in any one of preceding claims 1 - 43, characterised in - that at least a majority of the detection points are equipped with a light emitter in order to visually view an adjusted test image after a completed calibration operation.
45.
A method as disclosed in any one of the preceding claims, characterised in that
- it is used for at least one of: adjustment of the image geometry of the image;
5 adjustment of projection distortions in the image; adjustment of the edge areas of the image in relation to the display screen edges; adjustment of edge transition or transitions, so-called edge blending, to at least one adjacent projected image on the display screen; adjustment of wholly or partly overlapping display images; correction of colours in the image; correction of brightness in the image; and adjustment of the o projector's position and orientation.
46.
A device for use in calibrating a projector image display towards a display screen, wherein the display screen is provided with a plurality of discrete light detection points, s characterised in
- that each light detection point or groups of light detection points are connected to a light detector;
- that the light detector or detectors and the projector are connected to at least one processor, or that the light detector or detectors are connected to the projector's 0 processor;
- and that the processor or processors are arranged to determine which image pixel or pixels of the projected image hit the respective detection point on the screen.
47. 5 A device as disclosed in claim 46, characterised in that said discrete light detection points are located immediately below the front surface of the display screen.
48.
A device as disclosed in claim 47, characterised in that the front surface of the screen is o provided with a partially translucent cover layer which covers said discrete light detection points.
49.
A device as disclosed in claim 48, characterised in that said cover layer is formed of at 5 least one layer of paint, a sheet, a foil or a film.
50.
A device as disclosed in claim 46, characterised in that said discrete light detection points are located at a distance from, on or in the front surface of the screen.
51.
A method as disclosed in any one of claims 46 - 50, characterised in that
- the processor or processors are adapted to adjust relevant image parameters or projector parameters, so that a desired image pixel or pixels hit a respective detection point.
52.
A method as disclosed in any one of claims 46 - 51, characterised in that - the device comprises a storage medium for storing adjusted images, said storage medium being adapted to subsequently deliver to the projector adjusted image information for display.
53. A device as disclosed in any one of claims 46 - 51 , characterised in
- that the detection points are connected to a signal processing unit;
- that the signal processing unit, on an input side, is connected to an image signal source, the signal processing unit being adapted to adjust the received image signals; and
- that the signal processing unit has an output for delivering adjusted image signals to the projector.
54. A device as disclosed in any one of claims 46 - 51, characterised in
- that the detection points deliver image signals to an image source;
- that the image source is adapted to adjust the image signals by using the detection point signals; and - that the image source output is connected to the projector to deliver the adjusted image signals to the projector.
55.
A device for use in calibrating a projector image display towards a display screen, wherein the display screen is provided with a plurality of discrete light detection points, and wherein at least three of these light detection points are positioned close to the edges or corners of the display screen, and
- that the projector is adapted to project at least one image towards the display screen, characterised in
- that the projector is physically shiftable in order to adjust its physical position with respect to at least one of: lateral direction, vertical direction, orientation and distance to the screen; and
- that a correlating means is associated with the light detection points and the projector to correlate measured image pixel information in the detection points with theoretical image pixel information for these detection points until the measurement deviation has been minimised as a result of said adjustment and subsequent fixing.
56.
A device as disclosed in claim 55, characterised in that said discrete light detection points are located immediately below the front surface of the screen.
57.
A device as disclosed in claim 56, characterised in that the front surface of the screen is provided with a partially translucent cover layer which covers said discrete light detection points.
58.
A device as disclosed in claim 57, characterised in that said cover layer is formed of at least one layer of paint, a sheet, a foil or a film.
59. A device as disclosed in claim 55, characterised in that said discrete light detection points are located at a distance from, on or in the front surface of the screen.
60.
A device as disclosed in any one of claims 55 - 59, characterised in that groups of light detection points are associated with a light detector.
61.
A device for use in calibrating a projector image display towards a display screen, wherein the display screen is provided with a plurality of discrete light detection points, characterised in - that each light detection point or groups of light detection points are connected to a light detector;
- that the light detector or detectors and the projector are connected to at least one processor, or that the light detector or detectors are connected to the projector's processor; and - that the processor or processors are adapted to correlate brightness and/or colours in the projected image by comparing measurements at the detection points, and adjusting relevant projector parameters correspondingly.
62. A device as disclosed in claim 61, characterised in that said discrete light detection points are located immediately below the front surface of the display screen.
63.
A device as disclosed in claim 62, characterised in that the front surface of the screen is provided with a partially translucent cover layer which covers said discrete light detection points.
64.
A device as disclosed in claim 63, characterised in that said cover layer is formed of at least one layer of paint, a sheet, a foil or a film.
65.
A device as disclosed in claim 61, characterised in that said discrete light detection points are located at a distance from, on or in the front surface of the screen.
66.
A device as disclosed in any one of claims 62 - 66, characterised in that groups of light detection points are associated with a light detector.
67.
A device as disclosed in one of claims 61 - 66, characterised in - that the images which are successively projected on the display screen from the projector appear as wholly or partly different or changing images.
68. A device as disclosed in one of claims 61 - 67, characterised in
- that said image or images are at least one dedicated test image;
- that said projected test image appears as a sectioned or divided image on the display screen; and
- that the position of the respective detection point defined has a relation to the x, y position or group of x, y positions in the sectioned image.
69.
A device as disclosed in one of claims 61 - 68, characterised in
- that the processor or processors are adapted to give the position of each detection point at least one pixel address, at least two successive image measurements causing the detected image pixel or pixels to define respective pixel coding.
70.
A device as disclosed in claim 69, characterised in - that the coding is a binary code.
71.
A device as disclosed in claim 69, characterised in
- that the coding is analogue and is related to analogue parameters such as brightness or colour in a detected image pixel or detected image pixels.
72.
A device as disclosed in any one of claims 46 - 71, characterised in - that the processor or processors are adapted to make a centre of mass computation of the detected image pixels when the detection point is hit wholly or partly by two or more image pixels in order to compute the position of the detection point in relation to the image pixels' hit on the display screen and/or to adjust the projected image pixels with respect to at least one of the following parameters: brightness, colour and overall colour.
73.
A device as disclosed in claim 72, characterised in
- that the centre of mass computation carried out by the processor or processors is a function of the brightness and/or colour and/or overall colour of the image pixels.
74.
A device as disclosed in any one of claims 46 - 73, characterised in
- that the processor or processors are adapted to make an adjustment of convergence of a plurality of primary colours emitted from a projector, for example, of the laser projector type or the cathode ray tube type, based on measurement of the brightness of the individual colours at at least one detection point.
75. A device as disclosed in any one of claims 46 - 74, characterised in
- that the processor or processors, on the basis of the parameters of a theoretical model which at least include the x, y, z parameters of the projector and the x, y or x, y, z parameters of the detection points in relation to the position and orientation of the projector, are adapted to compute the expected position of the individual detection points;
- that the processor or processors, on the basis of the measured position of the detection points, are adapted to seek and find the position of the projector in relation to the theoretical model; and - that the processor or processors are adapted to minimise deviations in the computation in order to define the position of the projector in relation to the individual detection points on the display screen.
76. A device as disclosed in any one of preceding claims 46 - 75, characterised in that
- the processor or processors are adapted to construct a catalogue of measurement deviations associated with each detection point, and to introduce a correction parameter that is specific to each detection point so as to cause shifting of desired at least one picture element or image pixel to as close to said detection point as possible.
77.
A device as disclosed in any one of preceding claims 46 - 76, characterised in
- that the processor or processors contain a correlator that is adapted to correlate brightness and/or colours in the projected image by comparing measurements at the detection points, and where the output signals from the correlator cause an adjustment of related projection parameters in the projector or image parameters.
78. A device as disclosed in claim 72 or 73, characterised in
- that the centre of mass computation is used to cause optimal focussing towards the display screen of the test image projected from the projector.
79. A device as disclosed in any one of preceding claims 46 - 78, characterised in
- that each detection point is associated with a respective discrete light sensor.
80. A device as disclosed in any one of preceding claims 46 - 79, characterised in
- that all the detection points or groups of detection points are associated with a common light sensor via optical fibres.
81.
A device as disclosed in claim 79 or 80, characterised in
- that the light sensor or sensors and the projector are connected to or communicate with a common processor, or that the light sensor or sensors are directly connected to the projector's processor.
82.
A device as disclosed in claim 81, characterised in
- that the light sensor or sensors communicate with the processor via wireless communication.
83.
A device as disclosed in any one of preceding claims 46 - 82, wherein the projector is adapted to cooperate with at least one simultaneously operative, juxtaposed projector for projecting a composite image on the display screen, characterised in
- that the composite image consists of two wholly or partly overlapping images;
- that an image processing device is arranged to compare the parameters of the adjacent projectors in relation to the respective detection points of the display screen, and to adjust the images relative to each other with respect to image intensity and image pixels that represent corresponding picture elements in the composite image;
- that a signal processor is adapted to compare the position of corresponding picture elements in an overlapping area formed by the images projected from the respective projector by detection at at least one detection point in the overlapping area; and
- that the image processing device causes an adjustment of the image parameters in the images from the respective projector to give the images identical or almost identical pixel positions in the overlapping area which forms the composite image.
84.
A device as disclosed in any one of preceding claims 46 - 82, wherein the projector is intended to substitute or be substituted by at least one juxtaposed projector for providing substitute image display on the display screen, characterised in
- that the image processing device is arranged to compare the parameters of the adjacent projectors in relation to the respective detection points of the display screen and to adjust these parameters relative to each other with respect to image intensity and image pixels that represent corresponding picture elements in the display area in question;
- that a signal processor is adapted to compare the position of corresponding picture elements in the display area formed by the images projected from the respective projector by detection at at least one detection point in said display area; and - that the image processing unit is adapted to adjust the image parameters in the images from the respective projector to give the images identical or almost identical pixel positions in the display area.
85. A device as disclosed in any one of preceding claims 46 - 84, wherein the projector cooperates with at least one juxtaposed projector for projecting a composite image on the display screen, characterised in
- that in the overlapping area between at least two images which form a composite image there is placed at least one detection point which is common to said at least two images; - that the processor or processors are adapted to compare the parameters of the adjacent projectors in relation to the respective detection points of the display screen and adjust these parameters relative to each other with respect to image edge intensity and image pixels that represent identical picture elements in the composite image.
86.
A device as disclosed in any one of preceding claims 46 - 85, characterised in
- that at least a majority of the detection points are equipped with a light emitter in order to visually view an adjusted test image after a completed calibration operation.
87.
A device as disclosed in any one of preceding claims 46 - 86, characterised in that
- it is configured for at least one of: adjustment of the image geometry of the image; adjustment of projection distortions in the image; adjustment of the edge areas of the image in relation to the display screen edges; adjustment of edge transition or transitions, so-called edge blending, to at least one adjacent projected image on the display screen; adjustment of wholly or partly overlapping display images; correction of colours in the image; and correction of brightness in the image.
88.
A display screen for displaying an image from at least one projector and for calibrating image geometry and brightness, characterised in that - the display screen is equipped with a plurality of discrete light detection points at a distance from, on or in the front surface of the screen or located immediately below the front surface; and
- that each light detection point or groups thereof are associated with a light detector.
89.
A display screen as disclosed in claim 88, characterised in
- that each light detection point is associated with a respective discrete light sensor.
90.
A display screen as disclosed in claim 88, characterised in
- that all the detection points or groups of detection points are associated with a common light sensor via optical fibres.
91.
A display screen as disclosed in claim 88, 89 or 90, characterised in - that the light sensor or sensors and the projector are connected to or communicate with a common processor, or that the light sensor or sensors are directly connected to the projector's processor.
92. A display screen as disclosed in any one of preceding claims 88 - 91, characterised in
- that in the overlapping area between at least two images projected from at least two respective, juxtaposed projectors, and where these images form a composite image, there is placed a least one detection point that is common to said at least two images, which detection points are adapted for use, by a connected processor, to compare parameters of the adjacent, calibrated projectors in relation to the respective detection points of the display screen and adjust these parameters in relation to each other with respect to image edge intensity and image pixels that represent identical detected picture elements in the composite image.
93.
A display screen as disclosed in any one of preceding claims 88 - 92, characterised in
- that at least a majority of the detection points are equipped with a light emitter in order to visually view an adjusted test image after a completed calibration operation.
94.
A display screen as disclosed in any one of claims 88 - 93, characterised in that the light detection points are selectable from the group consisting of:
- discrete light sensors adhesible to the front surface of the display screen or attachable to the display screen individually or in groups; - discrete light sensors that communicate with respective optical fibres extending from the rear of the screen towards the front thereof;
- discrete light sensors that communicate with respective optical fibres extending from the rear of the screen towards the front thereof and are covered there by a partially translucent cover layer on the front surface of the screen;
- optical fibres that extend from the rear of the screen towards the front thereof and are covered there by a partially translucent cover layer on the front surface of the screen, and where the optical fibres are associated with a respective light sensor or a group of optical fibres is associated with a common light sensor; - discrete light sensors located immediately below the front surface of the screen and covered by a partially translucent cover layer on the front surface of the screen;
- optical fibres which extend from the front of the screen backwards through the screen to a common light sensor; and
- light sensors suspended on or in front of the screen at a distance from its front surface.
95.
A display screen as disclosed in claim 94, characterised in that said cover layer covers the whole of the screen's front surface in addition to said discrete the light detection points.
96.
A display screen as disclosed in claim 95, characterised in that said cover layer is constituted of at least one layer of paint, a sheet, a foil or a film.
97.
A display screen as disclosed in claims 88 - 93, characterised in that the light detection points, when an image is projected against the rear of the display screen, consist of
- discrete light sensors adhesible to the front surface of the display screen or attachable to the display screen individually or in groups; and - light sensors suspended on or in front of the screen at a distance from its front surface.
98.
A display screen as disclosed in claim 97, characterised in that the display screen is partially translucent.
99.
A display screen as disclosed in claim 98, characterised in that the display screen is made partially translucent by using a cover layer that covers the front surface of the screen, and where the cover layer is constituted of at least one layer of paint, a sheet, foil or film.
100.
A display screen as disclosed in any one of preceding claims 88 - 99, characterised in that - in addition to ordinary image display, it is used for at least one of: adjustment of the image geometry of the image; adjustment of projection distortions in the image; adjustment of the edge areas of the image in relation to the display screen edges; adjustment of edge transition or transitions, so-called edge blending, to at least one adjacent projected image on the display screen; adjustment of wholly or partly overlapping display images; correction of colours in the image; and correction of brightness in the image.
PCT/NO2007/000081 2006-02-28 2007-02-28 Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use WO2007100258A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US12/280,828 US8610778B2 (en) 2006-02-28 2007-02-28 Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use
EP07715962.2A EP1989887A4 (en) 2006-02-28 2007-02-28 Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use
CA2644117A CA2644117C (en) 2006-02-28 2007-02-28 Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
NO20060984A NO330155B1 (en) 2006-02-28 2006-02-28 Method and apparatus for use in calibrating a projector's image display to a display screen, and display screen for such use.
NO20060984 2006-02-28

Publications (1)

Publication Number Publication Date
WO2007100258A1 true WO2007100258A1 (en) 2007-09-07

Family

ID=38459299

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/NO2007/000081 WO2007100258A1 (en) 2006-02-28 2007-02-28 Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use

Country Status (5)

Country Link
US (1) US8610778B2 (en)
EP (1) EP1989887A4 (en)
CA (1) CA2644117C (en)
NO (1) NO330155B1 (en)
WO (1) WO2007100258A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011143684A1 (en) * 2010-05-21 2011-11-24 Isiqiri Interface Technologies Gmbh Projection device, which comprises a projector, a projection surface, and a data processing system, and method for operating said projection device
EP2472874A3 (en) * 2010-12-29 2012-09-12 Delphi Technologies, Inc. Seamless mosaic projection system and method of aligning the same
CN104360567A (en) * 2014-10-11 2015-02-18 苏州合欣美电子科技有限公司 Projection brightness adjustment method of adaptive projection system
WO2018154108A1 (en) * 2017-02-27 2018-08-30 Zactrack Gmbh Method for calibrating a rotatable and pivotable piece of technical stage equipment
AT522320B1 (en) * 2019-05-07 2020-10-15 Profactor Gmbh Calibration procedure for a projector

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7954954B2 (en) * 2007-07-31 2011-06-07 Hewlett-Packard Development Company, L.P. System and method of projecting an image using a plurality of projectors
EP2346251B1 (en) * 2008-09-28 2017-02-22 Shenzhen Aoto Electronics Co., Ltd. Method and system for monitoring led display screen operation
JP2010152267A (en) * 2008-12-26 2010-07-08 Seiko Epson Corp Projector
WO2010095952A1 (en) * 2009-02-19 2010-08-26 3D Perception As Method and device for measuring at least one of light intensity and colour in at least one modulated image
US20100287513A1 (en) * 2009-05-05 2010-11-11 Microsoft Corporation Multi-device gesture interactivity
US20100321408A1 (en) * 2009-06-19 2010-12-23 Sean Miceli Viewpoint Compensation for Curved Display Surfaces in Projector-Based Display Systems
US8602565B2 (en) * 2009-10-19 2013-12-10 Harris Technology, Llc Projected display
US8531485B2 (en) * 2009-10-29 2013-09-10 Immersion Corporation Systems and methods for compensating for visual distortion caused by surface features on a display
FR2959023B1 (en) * 2010-04-20 2012-05-25 Thales Sa ASSEMBLY MULTI-PROJECTOR VISUALIZATION SYSTEM
JP2013182142A (en) * 2012-03-02 2013-09-12 Mitsubishi Electric Corp Multi-screen display device
US9338447B1 (en) * 2012-03-14 2016-05-10 Amazon Technologies, Inc. Calibrating devices by selecting images having a target having fiducial features
JP6172495B2 (en) * 2012-12-28 2017-08-02 株式会社リコー Calibration apparatus, apparatus, projector, three-dimensional scanner, calibration method, method, program, and storage medium
JP6217110B2 (en) * 2013-03-29 2017-10-25 セイコーエプソン株式会社 Projector and adjustment method
CN104635408B (en) * 2015-03-11 2016-04-20 青岛海信电器股份有限公司 A kind of method of projection arrangement, optical projection system and adjustment projection systems project brightness
JPWO2017002298A1 (en) * 2015-06-29 2018-04-19 パナソニックＩｐマネジメント株式会社 Screen device and video projection system
USD823372S1 (en) * 2015-07-01 2018-07-17 Cj Cgv Co., Ltd. Screen frame
JP6556539B2 (en) * 2015-07-15 2019-08-07 国立大学法人東北大学 Screen and method of manufacturing screen
US9693029B1 (en) * 2015-12-22 2017-06-27 Microvision, Inc. System and method for feedback control in scanning projectors
EP3264359A1 (en) 2016-06-28 2018-01-03 Dassault Systèmes A computer-implemented method of calibrating a camera
EP3264360A1 (en) * 2016-06-28 2018-01-03 Dassault Systèmes Dynamical camera calibration
USD826974S1 (en) * 2017-02-03 2018-08-28 Nanolumens Acquisition, Inc. Display screen or portion thereof with graphical user interface
DE102017010683B4 (en) * 2017-11-17 2019-08-14 domeprojection.com GmbH Method for automatic restoration of a measured state of a projection system
TWI701950B (en) * 2019-05-16 2020-08-11 鈺緯科技開發股份有限公司 A display auto calibration device and the calibration method thereof
CN111951745A (en) * 2019-05-16 2020-11-17 钰纬科技开发股份有限公司 Image adjusting device of display and adjusting method thereof
CN110149506A (en) * 2019-06-28 2019-08-20 浙江水晶光电科技股份有限公司 Image projection detection device and image projection detection method
CN112261392B (en) * 2019-07-22 2022-08-09 中强光电股份有限公司 Projection system and image uniformity compensation method thereof
TWI750773B (en) * 2020-08-19 2021-12-21 大陸商寶宸（廈門）光學科技有限公司 Projecting system and method thereof and projecting displaying system
CN113012458A (en) * 2021-02-07 2021-06-22 启迪云控(上海)汽车科技有限公司 Multi-point road side perception target state monitoring management method and system
CN113759652B (en) * 2021-09-02 2023-05-16 深圳市炫之风文化创意有限公司 Visual immersion type stereoscopic projection equipment and projection method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4999703A (en) * 1988-12-23 1991-03-12 Hughes Aircraft Company Automatic image correction method and apparatus for projectors utilizing cathode ray tubes
US6219011B1 (en) * 1996-09-17 2001-04-17 Comview Graphics, Ltd. Electro-optical display apparatus
EP1385335A1 (en) * 2002-07-23 2004-01-28 NEC Viewtechnology, Ltd. Image projector with image-feedback control
EP1457963A1 (en) * 2003-03-10 2004-09-15 Hewlett-Packard Development Company, L.P. Enhanced contrast projection screen and projected control image

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4485394A (en) * 1982-09-27 1984-11-27 General Electric Company Automatic convergence and gray scale correction for television _receivers and projection television systems
US4684996A (en) * 1986-08-25 1987-08-04 Eastman Kodak Company Video projector with optical feedback
DE3911363B4 (en) 1989-04-07 2005-02-03 Freytag Von Loringhoven, Andreas Process for the preparation of fragrances to be enriched with fragrances or perfume and fragrance adding agents for carrying out the process
EP0703714A3 (en) 1994-09-20 1997-09-03 Matsushita Electric Ind Co Ltd Image display position detecting apparatus and image correction apparatus
JP3735158B2 (en) * 1996-06-06 2006-01-18 オリンパス株式会社 Image projection system and image processing apparatus
JPH10333088A (en) * 1997-05-28 1998-12-18 Canon Inc Display method of projected picture and projective picture display device
US5872593A (en) * 1997-06-04 1999-02-16 Sony Corporation Apparatus and method for calibrating video displays about points within the viewable area
JP3714163B2 (en) * 1997-12-12 2005-11-09 株式会社日立製作所 Video display system
US6671003B1 (en) 1999-06-30 2003-12-30 Thomson Licensing S.A. Automated calibration in a projection display apparatus
JP4309549B2 (en) * 2000-03-17 2009-08-05 オリンパス株式会社 Multi-display device, multi-display system, and adjustment method for multi-display device
US20020180727A1 (en) * 2000-11-22 2002-12-05 Guckenberger Ronald James Shadow buffer control module method and software construct for adjusting per pixel raster images attributes to screen space and projector features for digital warp, intensity transforms, color matching, soft-edge blending, and filtering for multiple projectors and laser projectors
JP3939141B2 (en) * 2001-12-05 2007-07-04 オリンパス株式会社 Projection type image display system and color correction method thereof
EP1607795A1 (en) * 2003-03-26 2005-12-21 Matsushita Electric Industrial Co., Ltd. Video display system
US6888562B2 (en) * 2003-03-26 2005-05-03 Broadcom Corporation Integral eye-path alignment on telephony and computer video devices using a pinhole image sensing device
JP2005024618A (en) 2003-06-30 2005-01-27 Nec Viewtechnology Ltd Projector having tilt angle measuring instrument
US7001023B2 (en) * 2003-08-06 2006-02-21 Mitsubishi Electric Research Laboratories, Inc. Method and system for calibrating projectors to arbitrarily shaped surfaces with discrete optical sensors mounted at the surfaces

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4999703A (en) * 1988-12-23 1991-03-12 Hughes Aircraft Company Automatic image correction method and apparatus for projectors utilizing cathode ray tubes
US6219011B1 (en) * 1996-09-17 2001-04-17 Comview Graphics, Ltd. Electro-optical display apparatus
EP1385335A1 (en) * 2002-07-23 2004-01-28 NEC Viewtechnology, Ltd. Image projector with image-feedback control
EP1457963A1 (en) * 2003-03-10 2004-09-15 Hewlett-Packard Development Company, L.P. Enhanced contrast projection screen and projected control image

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP1989887A4 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011143684A1 (en) * 2010-05-21 2011-11-24 Isiqiri Interface Technologies Gmbh Projection device, which comprises a projector, a projection surface, and a data processing system, and method for operating said projection device
EP2472874A3 (en) * 2010-12-29 2012-09-12 Delphi Technologies, Inc. Seamless mosaic projection system and method of aligning the same
CN104360567A (en) * 2014-10-11 2015-02-18 苏州合欣美电子科技有限公司 Projection brightness adjustment method of adaptive projection system
WO2018154108A1 (en) * 2017-02-27 2018-08-30 Zactrack Gmbh Method for calibrating a rotatable and pivotable piece of technical stage equipment
CN110326364A (en) * 2017-02-27 2019-10-11 扎克特拉克有限责任公司 Method for calibrating rotatable and pivotable technology stage setting
AU2018223167B2 (en) * 2017-02-27 2020-01-23 Zactrack Gmbh Method for calibrating a rotatable and pivotable piece of technical stage equipment
US10564034B2 (en) 2017-02-27 2020-02-18 Zactrack Gmbh Method for calibrating a rotatable and pivotable piece of technical stage equipment
CN110326364B (en) * 2017-02-27 2021-07-23 扎克特拉克有限责任公司 Method for calibrating a rotatable and pivotable technical stage device
AT522320B1 (en) * 2019-05-07 2020-10-15 Profactor Gmbh Calibration procedure for a projector
AT522320A4 (en) * 2019-05-07 2020-10-15 Profactor Gmbh Calibration procedure for a projector

Also Published As

Publication number Publication date
NO330155B1 (en) 2011-02-28
US8610778B2 (en) 2013-12-17
CA2644117A1 (en) 2007-09-07
US20090091623A1 (en) 2009-04-09
NO20060984L (en) 2007-08-29
EP1989887A4 (en) 2013-08-07
EP1989887A1 (en) 2008-11-12
CA2644117C (en) 2015-08-18

Similar Documents

Publication Publication Date Title
CA2644117C (en) Method and device for use in calibration of a projector image display towards a display screen, and a display screen for such use
US11930304B2 (en) System and method for providing improved display quality by display adjustment and image processing using optical feedback
US7652251B1 (en) Registration methods for fusing corresponding infrared and visible light images
US10750141B2 (en) Automatic calibration projection system and method
EP1540948B1 (en) Method and system for determining correspondence between locations on display surface having arbitrary shape and pixels in output image of projector
US6558006B2 (en) Image projection display apparatus using plural projectors and projected image compensation apparatus
US6561651B1 (en) Multiple display apparatus
US9910507B2 (en) Image display apparatus and pointing method for same
US20130215132A1 (en) System for reproducing virtual objects
CN108989777A (en) Projection device, the control method of projection device and non-transitory storage medium
US20090041201A1 (en) Alignment apparatus for imaging system
CN110244282A (en) A kind of multicamera system and laser radar association system and its combined calibrating method
CN107728410B (en) Image distortion correction method for laser projector and laser projector
US20090086927A1 (en) Alignment apparatus for imaging system using reflective element
WO2005015123A3 (en) Inclination angle detection device and inclination angle detection method
CN104657970B (en) A kind of scaling method and calibration system of full-automatic binocular endoscope
JP2017034576A (en) Imaging system, imaging device and image processing apparatus
US20220148223A1 (en) Lens Calibration System
CN106773080A (en) 3 d display device and display methods
CN106908081A (en) Laser level detecting system and method based on ccd video camera
US5446536A (en) System of detecting optical distortion of a light-transmitting plate-like member
US20220006989A1 (en) Method for Automatically Restoring a Calibrated State of a Projection System
EP2105790B1 (en) Automated geometry correction system for rear projection system
GB2260188A (en) Target acquisition training apparatus
CN114565676A (en) Infrared camera calibration device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2644117

Country of ref document: CA

Ref document number: 12280828

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2007715962

Country of ref document: EP