CN108076332A

CN108076332A - The system and method for digital black level mixing

Info

Publication number: CN108076332A
Application number: CN201711025499.1A
Authority: CN
Inventors: 凯文·莫尔; 尼克·瓦西尔卡; 彼得·安东尼·凡内尔; 安德鲁·道格拉斯·丹尼森; 瑞恩·托马斯·莫里斯; 丹尼尔·汤姆森·厄克特
Original assignee: Christie Digital Systems USA Inc
Current assignee: Christie Digital Systems USA Inc
Priority date: 2016-11-11
Filing date: 2017-10-27
Publication date: 2018-05-25
Anticipated expiration: 2037-10-27
Also published as: CN108076332B

Abstract

The present invention relates to projecting apparatus, and in particular to for the system and method for digital black level mixing.The present invention includes a system, determines by giving the picture point on projector to screen/object and the correspondence of the point of the video modulator on other projecting apparatus, therefore some correspondences are associated with one or more overlapping regions.Once it is determined that correspond to the point at the video modulator of one or more overlapping regions, can control overlapping in and overlapping around point on or surrounding video modulator pixel, to determine the luminance difference between one or more overlapping regions and other regions of image, and hereafter reduce the luminance difference between one or more overlapping regions and other regions of image.Even if only projecting black image, projecting apparatus can also be controlled to make overlapping region less apparent.Since overlapping region cannot become darker, so mitigating overlapping by adding some light to Non-overlapping Domain in various ways.

Description

The system and method for digital black level mixing

[technical field]

This specification relates in general to projecting apparatus, the system and method more particularly to being mixed for digital black level.

[background technology]

Even if projecting apparatus when projecting " black ", still can project light (such as grey, depth of some other colors on the screen Grey, white etc.).When multiple projecting apparatus are overlapped on the screen, overlapping region becomes " black " brighter than adjacent area.Weight Therefore folded region disperses project content；For example, when the content of projection is largely black scenes in overlapping region (for example, night Aerial fireworks) when, there is the polygonal region brighter than darker area around.

A solution of the prior art is from the saturating of projecting apparatus by fixed geometry neutral-density filter component Light path frustum is movable into and out in mirror, to attempt to create not scattered optics mixing for dark scene, this can be used for night Between simulator and other situations with dark scene.However, such solution will not make overheated filter and change Can not work well for being significantly larger than the projecting apparatus of 8000 lumens in the case of shape, unless they it is quite big and Away from camera lens, and can so be difficult to arrange.Although this filter can also be made of heat proof material, these materials may It is difficult to, it may be possible to expensive and/or may have poor optical property.

[content of the invention]

The present invention provides what a kind of digital black level for being used for multiple projecting apparatus with overlapping view field mixed to be System and method, digital black level mixing are happened at around overlapping view field.Particularly, even if only projecting black image, also may be used To control projecting apparatus so that overlapping region is less apparent.Since overlapping region cannot become darker (attached without physics/optics Part), so mitigating overlapping by adding some light to Non-overlapping Domain in various ways.In fact, by using electricity/number Solution (rather than light/physical solution) can adapt to match somebody with somebody due to projecting apparatus/screen using technique described herein The variation put and/or be aligned and the variation of overlapping region occurred.Particularly, this specification include a system, determine by Picture point pass corresponding with the point at the video modulator on other projecting apparatus on given projector to screen/object System, therefore some correspondences are associated with one or more overlapping regions.Once it is determined that correspond to one or more overlappings Point at the video modulator in region, can control overlapping in and overlapping around point on or surrounding video modulator picture Element, to determine the luminance difference between one or more overlapping regions and other regions of image, and hereafter reduce one or Luminance difference between multiple overlapping regions and other regions of image.

In addition, it in the present specification, will describe to project using various exemplary terms.For example, projecting apparatus is by image tune The pixel of device processed is projected to as pixel region on object and/or screen；Such pixel region is in one or more and senses Video modulator at device (including but not limited to the optical sensor on camera sensor/image and/or object and/or screen) In the visual field of pixel, it can be used for being formed on " point " (for example, its video modulator) in projecting apparatus and object and/or screen " point " and/or one or more sensors (such as the optical sensor on camera sensor/imager, object and/or screen It is first-class) in point correspondence.Utilize these correspondences, it may be determined that between the arbitrary point of two projecting apparatus and/or pixel pair Correspondence, correspondence is the point correspondence between projecting apparatus, pixel correspondence and/or projection between projecting apparatus Region correspondence between instrument.In addition, herein, term " pixel " for represent by projecting apparatus video modulator (and/ Or the least unit of the image data of original image content) formed image data least unit, term " region " and/or " pixel region " is by the region for the projector image being used to indicate at object and/or screen and/or pixel, term " overlapping region " The region of the projected image of overlapping will be used to indicate, and is corresponded on the bottom between pixel and/or sub-pixel and/or region Relation will use term " point ".In fact, correspondence here is specifically described as determining between points, although can also be Such correspondence is determined between point and/or region and/or pixel and/or sub-pixel.

Therefore, in the present specification, phrase " determining the correspondence between point " can include but is not limited to " determine region Between correspondence ", " determining the correspondence between pixel ", " determining the correspondence between sub-pixel " and/or determine Correspondence between its any variation, including determining the corresponding pass between point and region, point and pixel, region and pixel etc. System.

In the present specification, element can be described as " being configured as " execution one or more functions or be arranged to Such function.In general, the element that is configured as performing or be arranged to perform function is able to carry out the function or is suitable for Performing the function, to be either adapted to perform the function either operable to perform the function or can otherwise hold The row function.

It should be appreciated that for the purpose of this specification, " at least one in X, Y and Z " and " one or more in X, Y and Z It is a " language only X, only Y, only Z or two or more projects X, Y and Z can be interpreted (for example, XYZ, XY, YZ, XZ Deng) any combinations.Similar logic can be applied to " at least one ... " and " one or more ... " language it is any go out Two or more projects in existing.

The one side of this specification provides a kind of system, including：Multiple projecting apparatus, each projecting apparatus include image tune Device processed, the multiple projecting apparatus is oriented the corresponding projector image that projection is formed by corresponding video modulator, multiple Projector image is overlapped in one or more overlapping regions；At least one sensor is configured as obtaining the multiple projection At least part of digital picture in instrument image；And computing device, it is configured as including for the multiple projecting apparatus The each given projecting apparatus of given video modulator：Given projector image is determined using at least one sensor It is corresponding right between subpoint and the respective point of the corresponding video modulator of other projecting apparatus in the multiple projecting apparatus It should be related to, corresponding correspondence represents the corresponding image for corresponding to other projecting apparatus of one or more overlapping regions The corresponding point of modulator；Determine the corresponding brightness of one or more of overlapping regions；And control the given image The pixel of each of modulator and corresponding video modulator, at least adjacent edge pixel correspond to one or more of The edge of overlapping region is with based on the brightness in the corresponding projector image region of corresponding brightness increase.

In some embodiments, at least for the edge pixel at the edge for corresponding to one or more of overlapping regions Realize the correspondence between the subpoint and corresponding point.

In some embodiments, corresponding correspondence includes and the given image modulator and described corresponding Video modulator in the periphery of each the corresponding associated correspondence of dead zone pixel.

In some embodiments, the computing device is configured to control one or more of overlapping regions Outside and at least neighbouring edge pixel corresponding with the edge of one or more of overlapping regions the given image tune At least a portion of the pixel of each in device processed and corresponding video modulator, to increase corresponding projecting apparatus Brightness in image-region is to match the corresponding bright of one or more of overlapping regions.

In some embodiments, the computing device be configured to control the given video modulator and Each at least neighbouring corresponding with the edge of one or more of overlapping regions in corresponding video modulator The pixel of edge pixel, it is bright in corresponding projector image region to be increased according to the respective distance with the edge pixel Degree.

In some embodiments, the computing device is configured to be controlled according to one or more of following Make each in the given image modulator and corresponding video modulator it is at least neighbouring with it is one or more of heavy The pixel of the corresponding edge pixel in edge in folded region：The nearest correspondence overlapping region edge of the projector image of projection；Base Weighting in the nearest correspondence overlapping region edge of the one or more of the projector image of projection；And the projection from projection The integrated brightness of each in one or more nearest correspondence overlapping region edges of instrument image.

In some embodiments, the computing device is configured to：It determines to be formed one or more of heavy The corresponding orientation of the given projector image in folded region and the image pixel of other projector images；And according to institute State image pixel corresponding orientation further control it is each in the given image modulator and corresponding video modulator A pixel.

In some embodiments, the computing device is configured to：It determines to be formed one or more of heavy The corresponding projected size of the given projector image in folded region and the image pixel of other projector images；And root The given image modulator and corresponding video modulator are further controlled according to the corresponding projected size of described image pixel In the pixel of each brightness.

In some embodiments, the computing device is configured to：It determines to be formed one or more of heavy The corresponding projected color of the given projector image in folded region and the image pixel of other projector images；And root The given image modulator and corresponding video modulator are further controlled according to the corresponding projected color of described image pixel In the pixel of each.

The another aspect of this specification provides a method, including：In including following system：Computing device；It is more A projecting apparatus, each projecting apparatus include video modulator, and the multiple projecting apparatus is oriented projection by corresponding image modulation The corresponding projector image that device is formed, multiple projector images are overlapped in one or more overlapping regions；And at least one A sensor is configured as obtaining the more of the multiple projecting apparatus for each given projecting apparatus for including given image modulator At least part of digital picture of a projector image：The throwing of given projector image is determined using at least one sensor Corresponding correspondence between shadow point and the respective point of the corresponding video modulator of other projecting apparatus in the multiple projecting apparatus Relation, corresponding correspondence represent the corresponding image tune for corresponding to other projecting apparatus of one or more overlapping regions The corresponding point of device processed；Determine the corresponding brightness of one or more of overlapping regions；And control the given image tune The pixel of each of device processed and corresponding video modulator, at least adjacent edge pixel correspond to one or more of heavy The edge in folded region is with based on the brightness in the corresponding projector image region of corresponding brightness increase.

In some embodiments, the method is further included control outside one or more of overlapping regions and The given image modulator of at least neighbouring edge pixel corresponding with the edge of one or more of overlapping regions and institute At least a portion of the pixel of each in corresponding video modulator is stated, to increase corresponding projector image region In brightness to match the corresponding bright of one or more of overlapping regions.

In some embodiments, the method is further included controls the given video modulator and described corresponding Video modulator in at least neighbouring edge pixel corresponding with the edge of one or more of overlapping regions of each Pixel, to increase the brightness in corresponding projector image region according to the respective distance with the edge pixel.

In some embodiments, the method is further included controlled according to one or more of following described in Determine at least neighbouring and one or more of overlapping regions of each in video modulator and corresponding video modulator The pixel of the corresponding edge pixel in edge：The nearest correspondence overlapping region edge of the projector image of projection；Based on projection The weighting at one or more nearest correspondence overlapping region edges of projector image；And the projector image from projection The integrated brightness of each in one or more nearest correspondence overlapping region edges.

In some embodiments, the method is further included determines to form the institute of one or more of overlapping regions State the corresponding orientation of given projector image and the image pixel of other projector images；And according to described image pixel Corresponding orientation further control the pixel of each in the given image modulator and corresponding video modulator.

In some embodiments, the method is further included determines to form the institute of one or more of overlapping regions State the corresponding projected size of given projector image and the image pixel of other projector images；And according to described image The corresponding projected size of pixel further controls each in the given image modulator and corresponding video modulator Pixel brightness.

In some embodiments, the method is further included determines to form the institute of one or more of overlapping regions State the corresponding projected color of given projector image and the image pixel of other projector images；And according to described image The corresponding projected color of pixel further controls each in the given image modulator and corresponding video modulator Pixel.

The another aspect of this specification provides a kind of computer-readable medium for storing computer program, wherein the calculating The execution of machine program is used for：It communicates with multiple projecting apparatus in computing device, each includes：Computing device；Multiple projecting apparatus, each Projecting apparatus include video modulator, the multiple projecting apparatus be oriented projection formed by corresponding video modulator it is corresponding Projector image, multiple projector images are overlapped in one or more overlapping regions；And at least one sensor, it is configured To be directed to multiple projector images of the multiple projecting apparatus of each given projecting apparatus acquisition including given image modulator At least part of digital picture：Using at least one sensor determine the subpoint of given projector image with it is the multiple Corresponding correspondence between the respective point of the corresponding video modulator of other projecting apparatus in projecting apparatus, it is corresponding Correspondence represents the corresponding point for corresponding to the corresponding video modulator of other projecting apparatus of one or more overlapping regions； Determine the corresponding brightness of one or more of overlapping regions；And control the given image modulator and corresponding image The pixel of each of modulator, at least adjacent edge pixel correspond to the edge of one or more of overlapping regions with base Increase the brightness in corresponding projector image region in corresponding brightness.The computer-readable medium can include non-temporary When computer-readable medium.

[description of the drawings]

Various embodiments described herein and more clearly show how they are carried out in order to better understand, it is existing Example refer to the attached drawing is only being used as, in attached drawing：

Fig. 1 depicts the optical projection system according to non-limiting embodiment.

Fig. 2 is depicted to be mixed according to what non-limiting embodiment can use in the system of fig. 1 for digital black level The system of conjunction.

The projecting apparatus that Fig. 3 depicts the system of Fig. 2 according to non-limiting embodiment is projected image onto on screen Details.

Fig. 4 depicts the method mixed according to the digital black level of non-limiting embodiment.

Fig. 5 depicts the profile of the image on the screen for projecting to Fig. 2 according to non-limiting embodiment, including mirror image The profile in pond.

Fig. 6 A depict the subpoint for being used to determine the first projecting apparatus and other projections according to non-limiting embodiment The back projection technique of correspondence between the point of the video modulator of instrument, such as apeture lens model is used in overlapping region The technology shown.

Fig. 6 B depict the back projection technique of Fig. 6 A of two video modulators of use according to non-limiting embodiment

Fig. 7 depicts the staircase chart of two projecting apparatus according to non-limiting embodiments, including in overlay chart picture Visible ray and/or the corresponding overlapping region of brighter seam.

Fig. 8 depict according to the brightness of two projecting apparatus of non-limiting embodiment for outside overlapping region at least The staircase chart of one part of pixel and raised Fig. 7.

Fig. 9 is depicted uses overlay region according to the brightness of two projecting apparatus of non-limiting embodiment in mixed pattern The staircase chart of Fig. 7 when overseas pixel raises.

Figure 10 depicts the example of the overlay chart picture formed according to non-limiting embodiment using the mixed mode of Fig. 8.

Figure 11 depicts the overlapping image pixel of two projecting apparatus according to non-limiting embodiment, such as in screen On.

Figure 12 depicts the overlapping image pixel according to Figure 11 of non-limiting embodiment, and some of which edge pixel is dashed forward Go out to show degree of overlapping.

Figure 13 shows the overlapping image pixel from two projecting apparatus according to non-limiting embodiments, has The image pixel size changed at least one image.

Figure 14 depict according to the overlapping image pixel of Figure 13 of non-limiting embodiment and with for controlling brightness Overlapping region edge distance description.

Figure 15 depicts three overlay chart pictures being overlapped according to non-limiting embodiment.

Figure 16 depicts the mirrored pool in the overlay chart picture according to non-limiting embodiment.

Figure 17 depicts the digital black level hybrid system including optical sensor according to non-limiting embodiment, and It can be used in the system of fig. 1.

Figure 18 depicts the method mixed according to the digital black level of non-limiting embodiment.

[specific embodiment]

The invention will be further described below in conjunction with the accompanying drawings, for the structure and principle of this device people professional to this It is very clearly.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not used to limit this Invention.

Fig. 1 depicts system 100, including：Display device 101 (is interchangeably referred to as equipment 101) below；Content plays Device 103；Computing device 105；With projecting apparatus 107.In general, equipment 101 and content player 103 and optional computing device 105 Communication, and content player 103 communicates with projecting apparatus 107.

As shown in Figure 1, equipment 101 and content player 103 are combined into an equipment 108, however in other embodiment party In formula, equipment 101 and content player 103 can be separated equipment.Computing device 105 is configurable to generate geometric data 109s, include at least define such as projecting apparatus 107 and projecting apparatus 107 be projected in it is several between object (such as screen) above The data of what relation.Equipment 101 can from geometric data 109s generate present image data 110, such as by present for by The conventional images data (not shown) that projecting apparatus 107 projects.In Fig. 1, the solid line of connection component show image stream and/or its Between video data, and the chain-dotted line that computing device 105 is connected to equipment 101 and/or equipment 108 shows geometric data Stream between 109s and object data 109o.Object data 109o, which generally includes projection, the object of the image from projecting apparatus 107 The model of the position and direction of body, and object can be provided with and/or by using one or more cameras by computing device 105 determine.

When equipment 101 and content player 103 separate, image data 110 is transmitted to content player by equipment 101 103, content player is handled and/or " broadcast by generating the data for projection 112 of the processing and the projection that are suitable for projecting apparatus 107 Put " image data 110.For example, image data 110 can include but is not limited to avi file, a series of JPG files, PNG files Deng.Data for projection 112 can include but is not limited to HDMI data, VGA data and/or transmission of video data.When equipment 101 and interior When appearance player 103 is combined in equipment 108, data for projection 112 (for example, video data) can be presented in equipment 108 in real time, Without generating image data 110.Under any circumstance, data for projection 112 is transmitted to projecting apparatus 107 by content player 103, Wherein using data for projection 112 projecting apparatus 107 is controlled to project image onto on such as three-dimensional body.

Equipment 101 generally includes to be configurable to generate and/or image composer of the image as image data 110 is presented And/or renderer, such as computing device, server etc..Such image data 110 can include but is not limited to static image, Video etc..In addition, although being not shown, equipment 101 can include image composer and/or memory communication and/or It communicates, memory storage can generate the data of image data 110 from it.Alternatively, equipment 101 can use for generating Algorithm of image etc. generates image data 110.

Content player 103 includes the player that is configured as " playing " and/or present image data 110；For example, when figure When including video data as data 110, content player 103 is configured as by exporting the projection number projected by projecting apparatus 107 Video data is played and/or presents according to 112.Therefore, content player 103 can include, but are not limited to video player, regard Frequency processing equipment, computing device, server etc..However, equipment 101 and content player 103 are combined as setting as described above, working as When standby 108, the presentation of image data 110 can be eliminated, and data for projection 112 is presented without generating image data in equipment 108 110。

Computing device 105 includes projecting apparatus (including projecting apparatus 107), camera (not shown in figure 1) and computing device Any suitable combination, computing device are configured to automatically determine the geometric data 109s of projecting apparatus 107 and optionally automatic Ground determines the one or more of object data 109o.The nonrestrictive of computing device 105 is described below with reference to Fig. 2 to Figure 16 Embodiment.

Projecting apparatus of the projecting apparatus 107 including being configured to projection lenses data 112, including but not limited to digital projector, Cinematographic projector, the LCOS (liquid crystal covers silicon) based on projecting apparatus, DMD (digital multimirror device) based on projecting apparatus etc..Though in addition, A projecting apparatus 107 is so depicted only, but system 100 can include multiple projecting apparatus 107, each projecting apparatus 107 is configured To project the corresponding data for projection for the part for including larger tiling image for example to be projected.No matter make in projecting apparatus 107 How is technology, it is assumed that projecting apparatus 107 and/or other projecting apparatus described herein include video modulator, including multiple Independent pixel modulator；For example, when projecting apparatus includes DMD projecting apparatus, video modulator includes multiple digital micro-mirrors, wherein one A micro mirror is used for each pixel of image to be projected.

System 100 as depicted further includes one or more 2D (" two dimension ") warping apparatus and/or module 113, such as On projecting apparatus 107 (warping apparatus of even now can reside in content player and/or as self-contained unit).Work as presence When, data for projection 112 can be by 113 warpage of warping apparatus, such as passes through the picture in mobile and/or adjustment data for projection 112 Element, adjustment project to the data for projection 112 on the including but not limited to object of screen, object etc. for projecting apparatus 107.However, When computing device 105 determines geometric data 109s and it is communicated with equipment 101 (and/or equipment 108), warpage module 113 can be untapped, eliminate optionally and/or from system 100.In fact, it is represented using warpage module 113 according to existing Have how technology handles image, and by the computing device 105 for providing (and/or device 108) geometric data of device 101 109s To eliminate the presence of warpage module 113.However, in some embodiments, such as when for generating the virtual of the object of image When model inaccurately matches physical objects, warpage module 113 can be used for carrying out small change in image projection to physical objects Become.

Although each in equipment 101, content player 103, computing device 105 and projecting apparatus 107 is depicted as not Same component, but in other embodiments, in equipment 101, content player 103, computing device 105 and projecting apparatus 107 One or more corresponding portions, and can implement in same equipment (for example, equipment 108) and/or process resource can To share therebetween.For example, although it is not shown, system 100 includes one or more controllers, one or more processors, one A or multiple memories and one or more communication interfaces, such as equipment 101, content player 103, computing device 105 With the controller of each in projecting apparatus 107, memory and communication interface and/or in equipment 101, content player 103, meter Calculate controller, memory and the communication interface shared between equipment 105 and projecting apparatus 107.In fact, in general, system 100 Component represent the difference in functionality of optical projection system as shown in the figure, wherein：The geometric data of projecting apparatus 107 can be automatically determined 109s.In some embodiments, system 100 include for when projecting apparatus 107 moves projection mapping to three-dimensional body and/or The component and/or function and/or image of update geometric data 109s or 109o are projected to screen and/or object thereon Body moves.

Next Fig. 2 is paid attention to, which depict the systems 200 mixed for digital black level.In fact, computing device 105 It can include the one or more assemblies of system 200, in addition, the component of system 100 can include the group of system 200 as needed Part.System 200 includes：Computing device 201 (is interchangeably referred to as equipment 201) below；Multiple projecting apparatus 207-1,207-2, 207-3 (projecting apparatus 207 is interchangeably referred to as, and usually, as projecting apparatus 207)；At least one camera 214；And Object, image including but not limited to from projecting apparatus 207 will project to screen 215 thereon.In fact, assume each projection Instrument 207 is installed compared with screen 215 so that in the image projection from projecting apparatus 207 to screen 215.Moreover, it is assumed that camera 214 can be used to determine projecting apparatus 207 compared with the geometry of screen 215 and/or the geometry of screen 215；Particularly, At least one camera 214 is arranged to the digital picture of capture screen 215, and particularly projecting apparatus 207 projects to the figure on screen 215 Picture.Although equipment 201 is shown as communicating with projecting apparatus 207 and camera 214, system 200 can include being communicatively located in Other computing devices and/or communication equipment between 207/ camera 214 of equipment 201 and projecting apparatus, such as vision signal is generated And/or the equipment for being manipulated to projecting apparatus 207).(for example, within system 100, wherein computing device 105 and display device 101 and/ Or the content player 103 to communicate with player 107 communicates)

However, although current embodiment, in other embodiments, Ke Yiyong are described on screen 215 Object including three-dimensional body replaces screen, and projecting apparatus 207 can will be in projection mapping to object.

Each projecting apparatus 207 includes video modulator (not shown in Fig. 2), and other projecting apparatus 207 is oriented Corresponding image 216-1,216-2,216-3 (image 216 being interchangeably referred to as, and usually, as image 216) are thrown Shadow is formed on screen 215 by the corresponding video modulator of projecting apparatus 207, in one or more overlapping regions with screen The corresponding image 216 of 215 overlappings.For example, with reference to figure 3, describe projecting apparatus 207, screen 215 and image 216, projecting apparatus 207-1 projected image 216-1, projecting apparatus 207-2 projected images 216-2 and projecting apparatus 207-3 projected images 216-3.In addition, Image 216-1,216-2 are overlapped in the 217-1 of overlapping region, and image 216-1,216-3 are overlapped in the 217-2 of overlapping region, figure Picture 216-2,216-3 are overlapped -3 in overlapping region 217, and image 216-1,216-2,216-3 are in the 217-4 of overlapping region Overlapping.Overlapping region can interchangeably be referred to as overlapping region 217, and generally referred to as overlapping region 217.

Fig. 2 is returned to, in general, projecting apparatus 107 can include any one or more projecting apparatus 207, computing device 105 can be with Including computing device 201, projecting apparatus 207 and at least one camera 214, and it is any in equipment 101 and content player 103 One at least a portion that can include computing device 201, for example, when in equipment 101, content player 103 and computing device Between 105 during shared resource.

In addition, depicted in an arrangement that three projecting apparatus 207, but system 200 can include at least two projecting apparatus 207, throw 207 project overlapping images 216 of shadow instrument or more than three projecting apparatus 207, two of which or more projected image 216 are overlapped.It is real On border, although as shown in the figure, all three images 216 are overlapped and (obtain four overlapping regions 217), in other embodiment In, only adjacent image 216 and/or only a part image 216 is overlapped.

Equipment 201 can include any suitable computing device, include but not limited to graphics processing unit (GPU), figure Processing equipment, graphics processing engine, video processing equipment, personal computer (PC), server etc., and generally include controller 220th, memory 222 and communication interface 224 (being interchangeably referred to as interface 224 below) and optionally any input equipment and aobvious Show any suitable combination of equipment.

Interface 224 include being configured as needed in a manner of wiredly and/or wirelessly with projecting apparatus 207 and camera 214 (with And any equipment in equipment 101, content player 103, computing device 105 and equipment 108) communicate it is any suitable Wired or wireless communication interface.

Controller 220 can include processor and/or multiple processors, include but not limited to one or more central processings Device (CPU) and/or one or more processing units；No matter which kind of mode, controller 220 includes at hardware element and/or hardware Manage device.In fact, in some embodiments, controller 220 can include ASIC (application-specific integrated circuit) and/or special configuration It is the FPGA (field programmable gate array) for the attitude data (such as projecting apparatus 207) at least determining projecting apparatus.Therefore, equipment 201 be preferably not universal computing device, and is configured specifically for the equipment for realizing specific digital black level mixed function.Example Such as, equipment 201 and/or controller 220 can specifically include being configured as the meter for realizing optional network specific digit black level mixed function Calculation machine can perform engine.

Memory 222 can include non-volatile memory cells (such as Erasable Programmable Read Only Memory EPROM (" EEPROM "), flash memory) and volatile memory cell (such as random access memory (" RAM ")).Realize described herein set The programming instruction of standby 201 functional introduction is typically permanently maintained in memory 222, and is used for controller 220, is controlled Device 220 is during programming instruction as execution suitably with volatile memory.Those skilled in the art recognize, store Device 222 is the example for the computer-readable medium that can store the program instruction that can be performed on controller 220.In addition, storage Device 222 is also memory cell and/or memory module and/or the example of nonvolatile memory.

Particularly, the storage of memory 222 is using 230, when being handled by controller 220, using 230 make controller 220 and/ Or computing device 201 can：It determines to project to object by the given projecting apparatus 207 on object using at least one camera 214 The corresponding figure of the subpoint of given image 216 on (for example, screen 215) and other projecting apparatus 207 of multiple projecting apparatus 207 As modulator corresponding point between corresponding correspondence, corresponding correspondence represents to correspond to one or more overlappings The corresponding pixel of the video modulator of other projecting apparatus 207 in region 217；Determine the phase of one or more overlapping region 217 Answer brightness；And control at least neighbouring and one or more of each of given image modulator and corresponding video modulator The pixel of the corresponding edge pixel in edge of overlapping region 217 is bright in correspondence image region to be increased based on corresponding brightness Degree.

In other embodiments, as shown in the figure, memory 222 also stores the model 240 and/or perspective view of screen 215 Any other object of picture can use camera 214 and projecting apparatus 207 to determine, for example, by control 207 grade of projecting apparatus by The image projection of frame, structured light patterns etc. determines its geometry on screen 215 from the image obtained by camera 214 Shape；However, in other embodiments, the given mathematical model for example provided with screen 215 can be used in memory 222 Place provides model 240.In general, object data 109o can include model 240, and can be optionally provided to as described above The component of system 100.

In addition, as shown in the figure, memory 222 also storage system 200 geometric data 250, such as projecting apparatus 207 compared with The geometric position of screen 215 and/or posture.It is similar with model 240, camera 214 and projecting apparatus 207 can be used, such as is passed through The image projection such as frame, structured light patterns is determined that geometric data 250 projects by control projecting apparatus 207 grade on screen 215 Image, and determine its geometry from the image obtained by camera 214.Data 109p can include geometric data 250.

Anyway, by using the model of object image to be projected and the geometric data 250 of system 200, equipment 201 can be additionally configured to determine the image tune of projecting apparatus 207 of the subpoint of given image 216 with generating given image 216 The phase of the subpoint and other projecting apparatus 207 of multiple projecting apparatus 207 of mapping and given image 216 between the point of device processed Mapping between the corresponding point for the video modulator answered, such as when the subpoint of given image 216 is in overlapping region 217 When.

Notice that Fig. 4, Fig. 4 depict the method mixed for digital black level according to non-limiting embodiment now 400 flow chart.For the explanation of assisted method 400, it will be assumed that method 400 is performed using system 200, specifically by equipment 201 controller 220 performs method, such as when the processing of controller 220 is using 230.In fact, method 400 is can to configure A kind of mode of system 200 and/or equipment 201 and/or controller 220.In addition, will cause to the discussion of method 400 below pair Equipment 201 and system 200 and its various assemblies are further understood.It will be appreciated, however, that system 200 and/or equipment 201 and/ Or controller 220 and/or method 400 can be variation, and need not accurately as described herein in the such work of discussion Make, and this variation is in the range of present embodiment.

Anyway, it is emphasized that unless otherwise indicated, method 400 need not be carried out according to shown exact sequence； And various frames can be equally performed parallel rather than successively；Because the method 400 element be referred to herein as " frame " without It is " step ".It should again be understood, however, that method 400 can also be realized in the variation of system 200.Although in addition, calculating is set Standby 201 are described as realizing and/or perform each frame of method 400, but it is to be understood that each frame of method 400 is at It ought to be carried out with 230 controller 220.

It is also understood that method 400 is performed for each given projecting apparatus 207 of multiple projecting apparatus 207, and therefore Each projecting apparatus 207 can be directed to and/or according to the order of each projecting apparatus 207 and/or concurrently to each of projecting apparatus 207 Section/region/scan line/pixel-parallel performs method 400.

In frame 401, controller 220 determines to project by the given projecting apparatus 207 on object using at least one camera 214 The phase of the subpoint and other projecting apparatus 207 of multiple projecting apparatus 207 of given image 216 on to object (such as screen 215) Corresponding correspondence between the corresponding point for the video modulator answered, corresponding correspondence represent to correspond to one or more The corresponding point of the video modulator of other projecting apparatus 207 of a overlapping region 217.

In frame 403, controller 220 determines the corresponding bright of one or more overlapping regions 217.

In frame 405, controller 220 controls the pixel of each of given image modulator and corresponding video modulator, The edge that at least adjacent edge pixel corresponds to one or more overlapping regions 217 increases corresponding diagram to be based on corresponding brightness As the brightness in region.

In some embodiments, at least for the corresponding edge pixel in the edge of one or more overlapping regions, The correspondence between subpoint and corresponding point can occur.Although for example, in some embodiments, can be mainly And/or at least correspondence is determined for the point of edge pixel, other points can be directed to and determine correspondence, including one or more The point of " outer " of a overlapping region " interior " and overlapping region.In general, can control correspond to it is adjacent with the edge of overlapping region The pixel in the region of the projected image in " falling " region falls the brightness in region with control.

The non-limiting embodiment of method 400 is described now with reference to Fig. 5 to Figure 16.

Method 400 usually assumes that at least one camera 214 can be used for capture screen 215 (and/or projector image 216 Any object) digital picture, can be used for determining correspondence between camera 214 and screen 215.In some embodiments In, screen 215 (its specific markers that can include the physical edge of screen 215 and/or at least be temporarily placed on screen 215) The digital picture of mark obtain to build camera to the correspondence of screen by least one camera 214, with screen 215 Three-dimensional (3D) model (for example, model 240) is related, to obtain 2D (two dimension) cameras to the correspondence of 3D screens.Equipment 201 can Projecting apparatus 207 to be controlled to project to the structured light patterns (including but not limited to frame etc.) of each projecting apparatus 207 on screen 215, And at least one camera 214 captures the digital picture of the structured light pattern at screen 215, is used by equipment 201 with true 2D cameras are determined to the correspondence of 2D projecting apparatus.2D cameras are to 3D screens correspondence and 2D cameras to 2D projecting apparatus correspondences Available for determining 2D projecting apparatus to 3D screen correspondences.Although in fact, depict only a camera 214, system 200 It can include multiple cameras, and can be that each camera determines such correspondence；In addition, and not all multiple cameras all It needs to obtain the whole digital picture of screen 215；In fact, multiple cameras can aim at the different piece of screen 215.

These various correspondences can be used for determining the geometrical relationship in system 200, include but not limited to projecting apparatus 207 Physical location and direction；Such correspondence and/or geometrical relationship can be stored as geometric data 250.Geometric data 250 etc. It can be used for any further definite；Alternately, projecting apparatus 207 and screen can be determined using best fit multinomial etc. Geometric data 250 and/or geometrical relationship between curtain 215.It no matter in this way, can be in the every of each projecting apparatus 207 A correspondence is determined between subpoint on each point and screen 215 of a video modulator；In other words, such side Method can determine that each projector pixel will fall the position on screen 215, as the projected pixel region in image 216.

Next Fig. 5 is will be noted that, which schematically depicts projecting apparatus 207-X, 207-Y (each similar to projecting apparatus 207), screen 215 and camera 214-Ai, 214-Bj (being each similar to camera 214).Projecting apparatus 207-X, 207-Y represent Fig. 2's Two in projecting apparatus 207, camera 214-Ai, 214-Bj represent at least one camera 214 of Fig. 2.Image 216 edge (and/ Or profile) can for example one be determined by " shooting forward " (for example, from each projecting apparatus 207-X, 207-Y to screen 215) The grid (having the edge of image 216, described in Figure 5 with dotted line) of sampled point, is corresponded to each image 216, and made Determine it to the correspondence projected position (for example, subpoint) on screen 215 at least one camera 214-Ai, 214-Bj.Due to The position of the pixel of each video modulator in each projecting apparatus 207-X, 207-Y has been determined and has been stored as data 250, institute It can determine that the edge for corresponding to associated image 216 is (most with the pixel of the video modulator of each projecting apparatus 207-X, 207-Y Pipe is it is appreciated that the exact position of each pixel need not be stored as data 250, but data 250 can be including can be true Surely the data of the position of at least pixel on object and/or screen 215 are projected to).In this way, the throwing for the first projecting apparatus 207 Whether shadow pixel region (for example, projecting to the region on object and/or screen 215) can be assessed as them close to overlay region Domain 217, therefore may map to the pixel in the video modulator of another projecting apparatus 207.

It is also understood that in Figure 5, the edge of image 216 includes inward flange and outer edge, between inward flange and outer edge Region correspond to the dead zone of the pixel of respective image modulator for forming a part for image 216 but being always off.For example, In DMD video modulators and projecting apparatus, " dead " DMD speculums surround the effective coverage to form image (such as modulating light), " dead " DMD speculums are referred to as " mirrored pool " (POM).POM is always at closed state and/or fixed intermediate state, can be equivalent In closed state (for example, POM is so as to be equal to the mode reflected light of the light of closed state), but for by DMD projecting apparatus Any image of projection provides " black " border of non-state light；Such " black " frame can have in overlapping region 217 In contribute to the brightness of light.For DMD, POM can be up to about 20 pixels, but the DMD projecting apparatus that can be provided from manufacturer carries It is further appreciated that in the information of confession.

In some embodiments, can in the following manner using optional degree of approach checking process eliminating and/or It avoids " false positive "：207 mesh point of projecting apparatus is projected into screen 215；It determines on the screen 215 of adjacent forward direction mesh point most Distance D greatly；And store the position of the advance grid of maximum range D and point.In addition, any other screen 3D points can be by true Screen 3D points are determined whether in the maximum range D of any these advance screens 3D points, which are carried out " close to inspection It looks into "；If it is, screen 3D points are " close enough " for the projected image of projecting apparatus 207, to consider to be " projected " into throwing Shadow instrument 207 (as described below)；If it is not, then screen 3D points are insufficient to close to the projected image of projecting apparatus 207 consider " to throw Shadow " arrives projecting apparatus 207.

No matter which kind of mode, the rear-projection that can make projector image and/or image wheel can be carried out using whole image The technology of wide mutually poststack, corresponds only to the pixel of overlapping region and/or corresponds only to edge and/or profile (to may include dead zone And/or POM edges) pixel.This technology is used to determine which of each video modulator of each projecting apparatus 207 pixel Corresponding to one or more overlapping regions 217.Subpoint can be used in " 2D pixel spaces " (such as the picture in video modulator Element) in perform as determine.For example, in addition to 2D projecting apparatus to 3D screen correspondences, 3D screens to 2D projecting apparatus pair It should be related to and can also be used for determining 2D projecting apparatus 207-X to 2D projecting apparatus 207-Y (for example, by the way that " X " projecting apparatus 207 will be given Subpoint is transformed to the subpoint 207 of another " Y " projecting apparatus).In other words, such as in the frame 401 of method 400, controller 220 It determines to project to the given image 216 on screen 215 by the given projecting apparatus 207 on object using at least one camera 214 Subpoint and the corresponding video modulator of other projecting apparatus 207 of multiple projecting apparatus 207 corresponding point between it is corresponding Correspondence, corresponding correspondence represents the image for corresponding to other projecting apparatus 207 of one or more overlapping regions 217 The corresponding point of modulator.

It in some embodiments, can be by from projecting apparatus 207-X to camera 214-Ai to screen 215 to camera 214- Bj implements frame 401 to the mapping (it can be identical with camera 207-Ai or another cameras) of the pixel space of projecting apparatus 207-Y. The technology will be described, system 200 includes multiple camera A1, A2 ... An is (for example, in other words, camera 214-Ai can be represented " n " a camera 214), camera 214 and multiple camera B1, B2 are each similar to ... Bm (for example, in other words, camera 214- Bj can represent " m " a camera 214), each camera is similar to camera 214, although in some embodiments, can use few To a camera 214；No matter whether each camera is directed to screen 215.Under any circumstance, for example, for projecting apparatus 207-X Any pixel on (such as projecting apparatus 207-1), by projected pixel area maps to " can seeing " its each camera 214- Ai (A1, A2 ..An), is then mapped to screen 215, such as (and is assumed using previously stored correspondence also for every A camera determines such correspondence).Then by screen result be mapped to each camera (B1, B2 ... Bm- it is some of with Camera A1, A2 ... An is identical), the region of screen 215 is can see, then maps them into the figure of projecting apparatus 207-Y As the position of modulator.This may not cause the mapping of single position；On the contrary, the result from each camera can be used Weighted average determines the mapping of single position, is weighted by associated with the corresponding camera of pixel data " confidence level " value (for example, how related to the prototype structure light pattern by camera 214 and/or camera capture based on location of pixels).

Alternatively, geometry and the projecting apparatus calculated and the screen being stored in data 250 and/or model 240 can be used Curtain position determines from projecting apparatus 207-X to screen 215 to projecting apparatus 207-Y mapping.This can use single " overall situation " to throw Shadow instrument/screen position and/or each camera projecting apparatus/screen position of use (can obtain different from the data of each camera Geometric consequence) it performs, this can reuse weighted average to merge.

In general, such technology obtains the subpoint for making given projecting apparatus 207 and the video modulator of another projecting apparatus Correspondence between point, and can further obtain the correspondence for all overlapping regions 217；When equipment 201 When processing capacity is suitable, all subpoints can be handled, but in order to reduce such processing capacity, as defined As, the subpoint at the edge corresponding to image 216 and/or overlapping region 217 can be used only, for example, being checked from close Assessment and/or edge walking algorithm.

Next Fig. 6 A will be paid attention to, which depict the simple versions of back projection technique, it is assumed that for the small of projecting apparatus 207 Apeture lens model (lens and/or optical element for example, Fig. 6 A do not give particulars in back projection technique), although it is contemplated that projection The model of the influence of instrument lens and/or optical device is in the range of the present embodiment so that rear-projection will not occur along straight line.Nothing By how, the result correspondence of subpoint P1i is shown in Fig. 6 A, that schematically shows compared with the every of screen 215 Video modulator 617-1,617-2,617-3 of a projecting apparatus 207 (are interchangeably referred to as video modulator 617, and general Ground, as video modulator 617) side view；And as shown in the figure, each video modulator 617 be depicted as it is parallel and with Screen 215 is roughly equidistant, only for providing such description for the sake of clear, and each video modulator 617 can with The other angles positioning of screen 215.

As shown in the figure, the point p1i of video modulator 617-1 corresponds to subpoint P1i (its in projector image 216-1 It is depicted as, with certain depth, only indicating its position at screen 215 in fig. 6.Anyway, use is predetermined Geometrical relationship between projecting apparatus 207 and screen 215, and optionally, selected based on it includes the edges as image 216-1 Subpoint P1i is selected, controller 220 determines may map to the point p2i at the video modulator 617-2 of projecting apparatus point P1i and can To be mapped to the point p3i at the video modulator 617-3 of subpoint P1i.In other words, each in point p1i, p2i, p3i Correspond to the point of overlapping region 217 (not showing in fig. 6).Accordingly, it is determined that the corresponding pass between subpoint P1i and point p2i System, and by being determined from subpoint P1i to each " rear-projection " in point p2i, p3i between subpoint P1i and point p3i Correspondence.When considering the lens of projecting apparatus 207, rear-projection access may not be straight line.

This point can be repeated for other subpoints in image 216-1, for example, the side corresponding to overlapping region 217 Edge and/or the chance with the edge corresponding to overlapping region 217, to obtain the subpoint of image 216-1 and each image tune The point of device 617-2,617-3 processed；In addition, some projecting apparatus points of image 216-1 will correspond to only video modulator 617-2, The point of one in 617-3.It in other embodiment, can be to all view fields in image 216-1 and/or image The subset of view field in 216-1 repeats this point, rather than just the projected area at the edge corresponding to overlapping region 217 Domain.

For example, even for the subpoint outside overlapping region, it can also be in video modulator 617-1 and video modulator Mapping, at least the video modulator 617- for " near " subpoint as video modulator 617-2 are determined between the point of 617-2 1 subpoint (for example, video modulator 617-2 projecting edge in set a distance).However, this be mapped to energy The point of video modulator 617-1 enough outside the border of video modulator 617-2.In other words, video modulator 617-1 Point be mapped to point outside the physical space by the video modulator 617-2 points projected can have negative value (etc.) with (assuming that being had by the physical space of the video modulator 617-2 points projected in the associated coordinate spaces of video modulator 617-2 Positive value).Therefore, even if subpoint P1i is inconsistent (for example, P1i is any overlapping with the point projected by video modulator 617-2 Outside region), identified geometrical relationship still determines that point p2i is mapped to the coordinate space defined by video modulator 617-2 Position, even if the point beyond video modulator 617-2 border (although the precision of the mapping can be according to point P1i and image Modulator 617-2 projection point distance and reduce).

Next it will be noted that Fig. 6 B, which depict use shown in the simple version of video modulator 617-1,617-2 Another example of back projection technique, the shape modulation device 617-1,617-2 depict the six of each video modulator 617-1,617-2 A pixel, although each video modulator 617-1,617-2 generally include thousands of pixel, such as each video modulator 617-1, Shown in ellipse in 617-2.In addition, although video modulator 617-3 is not shown in fig. 6b, it still may have. As shown in Figure 6A, for simplicity, the influence of lens and/or optical device to background technology is had ignored.It is also supposed that each figure As at least pixel of the top and rightmost in modulator 617-1,617-2 is edge pixel.

Fig. 6 B are further depicted by the video modulator 617-1 projector image 216-1 generated and by video modulator 617-2 The projector image 216-2 of generation.Although not describing projector image 216-3, it still may have.

In addition, in fig. 6b, the overlapping region between image 616-1,616-2 includes the pixel of a pixel, such as projection Shown in the PR1i of region.It although, in other embodiments, may in overlapping region only there are one pixel in overlapping region There is dozens of to thousands of pixel.

Fig. 6 B also show the projected pixel region for example mentioned in method 400, can include corresponding to projecting apparatus The view field of the pixel of image.However, such view field can also include subpoint and/or can map back projecting apparatus Video modulator etc. image any region and/or point.

It is also shown in fig. 6b for the correspondence of projected pixel region PR1i.Specifically, video modulator 617-1 Pixel p x1i correspond to such as projector image 216-1 in projected pixel region PR1i.Use predetermined projecting apparatus Geometrical relationship between 207 and screen 215, and optionally, select to project based on it includes the edges as image 216-1 Pixel region PR1i, controller 220 determine to map to the pixel of projector pixel region PR1i at video modulator 617-2 px2i.In other words, each in pixel p x1i, px2i corresponds to the picture of overlapping region (for example, overlapping region 217-1) Element.Therefore, by determining projected pixel region PR1i and pixel from projected pixel region PR1i to " rear-projection " of pixel p x2i Correspondence between px2i.When considering the lens and/or optical element of projecting apparatus 207, rear-projection access may not be straight line.

In addition, the more than one pixel at video modulator 617-2 may map to projected pixel region PR1i；For example, Two or more pixels at video modulator 617-2 are mapped to (and/or part mapping arrives) projected pixel region PR1i。

Furthermore, it is possible to a part (for example, part of pixel) for pixel is mapped to projected pixel region PR1i；At this In a little embodiments, whether entire pixel, which is included in, can depend on the portion for being mapped to projected pixel region PR1i in overlapping region The size divided, as described in more detail below.

In further embodiment, the point at pixel p x1i, px2i is mapped on the PR1i of projected pixel region Or neighbouring point, such as the angle of pixel px1i, px2i etc..

It in other words, it is determined the subpoint of given image and the corresponding figure of other projecting apparatus 207 of multiple projecting apparatus 207 As modulator corresponding point between corresponding correspondence, can include between view field and one or more pixels Correspondence between the one or more points of correspondence, view field and pixel etc..

Pixel is converted into be converted into pixel a little and/or by point in addition it is possible to use becoming to bring, and further determines that picture Overlapping degree (and/or point corresponding with pixel and/or point overlap) between element is (for example, the pixel in overlapping region Percentage etc.).For example, the point (such as angle point) of pixel can be converted, to determine the region of the pixel corresponding to overlapping region. When finding correspondence for multiple points in pixel, the point in pixel can be used for sub-pixel antialiasing.Therefore, scheming In the example of 6B, by the sub- point (example for finding projected pixel region PR1i and pixel p x1i, px2i and/or pixel p x1i, px2i Such as, the point in pixel p x1i, px2i) between correspondence, it may be determined that pixel p x1i, px2i in overlapping region 217 Part and the size of the part of each in pixel p x1i, px2i in overlapping region.

It is in any case possible to all view fields in image 216-1 are repeated with determining for this correspondence, example Edge such as corresponding to overlapping region 217 and/or the chance with the edge corresponding to overlapping region 217, to obtain image Correspondence between the projecting apparatus region of 216-1 and the corresponding pixel of video modulator 617-2,617-3；In addition, image It some projecting apparatus regions of 216-1 will be corresponding to the pixel of one only in video modulator 617-2,617-3.

Next Fig. 7, staircase chart and its overlapping which depict the image from projecting apparatus A and projecting apparatus B are paid attention to.Each Projecting apparatus A, B can correspond to two projecting apparatus 207.Particularly, the staircase chart of Fig. 7 (with one or more lumens, is appointed including light The unit etc. of anticipating measurements) with the graph in distance (for example, in " x " direction or " y " direction).Distance can correspond to screen 215 On distance (for example, in projector space, object space and/or screen space can also be referred to as (for example, when being projected Object when being screen)) or the video modulator of projecting apparatus 207 at distance (for example, pixel space) (in fact, from projection To the conversion of pixel space, they are substantially isomorphism in space).In the figure 7, projecting apparatus A export a certain amount of light (for example, In the region labeled as " projecting apparatus A "), and some other amounts of projecting apparatus B outputs (such as labeled as " projecting apparatus B " In region) and in overlapping region (for example, labeled as the region of " overlapping of projecting apparatus A, B "), from projecting apparatus A and throwing The light overlapping of shadow instrument B, approximately linear addition.In overlapping region, the contribution of each projecting apparatus A, B is by from " projecting apparatus A " Each corresponding dotted line for extending to overlapping region in " projecting apparatus B " region indicates that wherein the light in overlapping region is total Amount is expressed as solid line on every dotted line.

Thus, for example, it can correspond to corresponding to the region of Fig. 7 of the light from only one projecting apparatus A, B in image 216 Two, and overlapping region can correspond to overlapping region 217.In general, in this example, the output from projecting apparatus B is than throwing The output of shadow instrument A is brighter (for example, their corresponding height are compared with " light " axis)

It is readily apparent that the steep vertical variation of the brightness of overlapping region either side can cause the video in total figure picture Band, the image of each projecting apparatus of projecting apparatus A, B including projecting " black ".Therefore, depict can be with application method 400 by Fig. 7 Come the general considerations solved.

In addition, by using one or more of camera 214, the digital photometry to communicate with computing device 201 and/or make Determine that the brightness of overlapping region (for example, at frame 403 of method 400) (therefore can use input equipment with manual technique Brightness is input to equipment 201).Alternatively, the brightness of each projecting apparatus A, B can be determined respectively, by by the bright of projecting apparatus A, B Du Xiang Calais determines the brightness of overlapping region.

According to correspondence (such as box 401 of application method 400) definite as described above, can control at least with again The pixel (such as box 405 of application method 400) of the video modulator of region adjacent each projecting apparatus A, B is folded to be based on weight The corresponding bright in region is folded to increase brightness.

For example, with reference to figure 8, which depict the staircase charts similar with Fig. 7, can use uniform black level hybrid technology The brightness of all areas of the image outside overlapping region is improved, makes it similar to the brightness of overlapping region；Due to definite pair It should be related to, can particularly control the pixel (for example, at frame 405 of method 400) outside overlapping region to increase each projection The brightness (for example, corresponding pixel using corresponding projecting apparatus A and projecting apparatus B) of instrument A, B.For each projecting apparatus A, B, this A little corresponding brightness increases are shown in Figure 8 for the increase 801- of the light of the increase 801-A and projecting apparatus B of the light of projecting apparatus A B each increases 801-A, 801-B outside overlapping region, but adjacent thereto.As shown in the figure, each increase 801-A, 801-B The opposite edges of respective image are extended to from the edge of overlapping region.

Since the output of projecting apparatus B is brighter than the output of projecting apparatus A, so increasing more than projecting apparatus B at projecting apparatus A Light, therefore increase 801-B and be less than and increase 801-A (for example, by their corresponding height compared with " light " axis).In addition, The height of the increase 801-B of projecting apparatus B is similar to the height initially exported of projecting apparatus A shown in Fig. 7；Similarly, projecting apparatus A Height 801-A increase similar to projecting apparatus B shown in Fig. 7 the height initially exported.

For example, in this embodiment, controller 220 can control the given throwing outside one or more overlapping regions The part of each in the given image modulator of shadow instrument 207 and the corresponding video modulator of other projecting apparatus 207 and/ Or whole pixels and at least adjacent edge pixel corresponding with the edge of one or more overlapping regions, to increase corresponding figure As the brightness in region to match the corresponding bright of one or more overlapping regions.

In fig. 8, the brightness of the whole image outside overlapping region increases to the brightness of overlapping region；However, in other realities It applies in mode, only the part increase of the image outside overlapping region, this can be referred to as " mixing ".This hybrid technology is retouched It is painted in the staircase chart of Fig. 9, which is substantially similar to Fig. 8.Particularly, in hybrid technology, the projection adjacent with overlapping region The pixel of each video modulator of instrument A, B is controlled (for example, at frame 405 of method 400) with basis and edge pixel Respective distance increases the brightness in correspondence image region.For each projecting apparatus A, B, these corresponding brightness increases are in fig.9 The variation 901-B of the light of the variation 901-A and projecting apparatus B of the light of projecting apparatus A is shown as, each changes 901-A, 901-B Outside overlapping region, but it is adjacent thereto.As shown in the figure, each in variation 901-A, 901-B is from the side of overlapping region Edge extends, and reduces as the distance apart from the edge of overlapping region increases.In Fig. 9 further respectively will variation 901-A, 901-B is labeled as " Mixed Zone 1 " and " Mixed Zone 2 ".Therefore, Fig. 9 depicts the black level in a part for overlay chart picture Mixing.

In other words, brightness is gradually reduced from overlapping region.As shown in the figure, the decline of brightness is linear, however may Generation is other kinds of to drop, and includes but not limited to " S curve ".The distance that mixing occurs can be by user and/or based on image Content it is heuristic determine.In addition, Figure 10 depicts how in overlapping region to go out by Mixed Zone 1 and Mixed Zone 2 The view 1001 of existing this mixing, wherein the brightness of the adjacent area outside from overlapping region to overlapping region continuously decreases.

The aspect of the usual plotting methods 400 of Fig. 5 to Figure 10；Hereafter some specific embodiments of providing method 400.

Brightness

As shown in Figure 7 to 9, a projecting apparatus 207 can be brighter than another.However, when each projecting apparatus 207 projects During image 216 corresponding with black image, since light is seldom, so the absolute brightness of the black level of measuring projector 207 It may be challenging.The measuring apparatus of very sensitive (and expensive) can be used, but in installation projecting apparatus system And/or when needing after some months or several years to need that these projecting apparatus systems are readjusted or recalibrated, these Equipment possibly can not use.In which case it is possible to use at least one camera 214 determines image 216 and/or overlay region The brightness (such as at frame 403) in domain 217.

Although at least one camera 214 may or may not be enough to measure absolute brightness, various technologies can be used To overcome this point.In a kind of technology, the gain of at least one camera 214 can be increased.In another technology, Ke Yi The brightness of image 216 and/or overlapping region 217 is measured at the light output of various higher levels, then can be come using extrapolation true Determine the brightness at the other light output of lower level.

In another technology again, there can be the region of similar luminance with average expectancy (being based on system model).For example, tool The image (such as black image) for having black level can have the brightness that can be can detect by least one camera 214, but bright Degree can be close to the essential noise of at least one camera 214.Therefore, as long as large area does not include its of other possible brightness His region, at least one camera 214 can determine the average brightness in the relatively large region of brightness than interested zonule, with More accurately measured.For example, from system model, the edge of various overlapping regions 217 is known, thus may determine that keeping away Exempt from the average brightness in the average large area between different zones (e.g., including area-of-interest).

In addition, in again other embodiment, user can be with estimated brightness value, and obtained black level is presented and mixes It closes to show result.In other words, in some embodiments, equipment 201 can receive defeated at input equipment (not shown) Enter, instruction controller 220 controls the region of the image 216 of adjacent overlapping region 217 to increase brightness.This control can be with Different patterns occurs, this can be selected using menu system etc., such as cause the whole image outside overlapping region 217 Increase in the 216 increased first mode of brightness (for example, as shown in Figure 8))；Or only make the image 216 outside overlapping region 217 Brightness a part of increased second mode in (for example, as shown in Figure 9).In other words, user can manually (one by one, Common and combination thereof) brightness of image 216 outside overlapping region 217 is adjusted, and controller 220 can be configured as It identifies and controls the image-region 216 outside overlapping region 217.

For example, as shown in Figure 7 and Figure 8, projecting apparatus B is corrected to the increase 801-B of light of the brightness of overlapping region about Equal to, by the original intensity of the projecting apparatus A light generated, and projecting apparatus A being corrected to the light of the brightness of overlapping region in Fig. 7 Increase 801-A to be approximately equal in Fig. 7 by the original intensity of the projecting apparatus B light generated (for example, brightness in overlapping region is about etc. In the addition of the brightness of each projecting apparatus A, B；Therefore, in order to which projecting apparatus B is made to reach the level of overlapping region, it is big to increase 801-B It is approximately equal to the original intensity of projecting apparatus A, vice versa).Therefore, as user estimated projection instrument A (for example, independently of projecting apparatus B) Luminance level when, and control device 201 further by outside overlapping region projecting apparatus B generate image 216 brightness It improves, the projection outer edge of the projecting apparatus A adjacent with overlapping region usually disappears (the dotted line end of every lower section in Fig. 7 and Fig. 8 Only).

In fact, this occurs when even when the not yet luminance level of estimated projection instrument B (because it effectively falls in equation Outside, therefore later, the region of the image 216 only outside overlapping region increases).However, it is possible to it iteratively controls by projecting apparatus B This increase of the brightness of the image 216 of generation is to form the better estimation to brightness：When by the projecting apparatus B outside overlapping region When the region of the image 216 of generation is too bright (such as the transition from overlapping region to adjacent area is that vision is apparent), estimation drop It is low；And when the region of the image 216 generated by the projecting apparatus B outside overlapping region is too dark (for example, from overlapping region to adjacent The transformation in region is that vision is apparent), then improve estimation.

However, user can replace at least one camera 214 in the following manner：- estimated using at least one camera 214 Count luminance level；It is calculated using the estimation from least one camera 214 to perform black level, and adjusts projecting apparatus A, B (example Such as projecting apparatus 207) luminance level, such as outside overlapping region；Result is captured at least one camera 214 so which to be determined Region is too light or too dark；And estimated according to these results modifications, and repeat.It in other words, can be by by overlapping region 217 Brightness and the brightness of adjacent area (for example, corresponding to the region for increasing by 801 and/or change 901) are compared to iteratively hold Row method 400.Similarly, whether typical line detection algorithms can be used to determine the camera image of capture still comprising expression weight Whether the line and/or edge at folded edge disappear.

Partly overlap pixel

In some projecting apparatus, such as the projecting apparatus based on DMD, the brightness of single pixel is controllable between the two states 's：Under conducting state and closed state, although luminance level can be controlled using following technology：But be not limited to, shake and Bit plane.In other words, the pixel in this projecting apparatus is individual unit, and a part for such pixel cannot be compared with Another part of same pixel is adjusted.Therefore, when the brightness of pixel changes, the brightness of entire pixel changes.When by this Pixel overlapping in the image that the projecting apparatus of sample generates but it is not exactly the same when, some view fields from projected pixel can be with Outside overlapping region, and other view fields from same projection pixel can be in overlapping region.It is shown in Figure 11 Such case, the part which depict projector image pixels from two DMD projecting apparatus is (for example, DMD-A, DMD-A The part of image is described with dotted line and the part of the image of DMD-B, DMD-B is described with solid line), by digital pixel (example Such as, color) it is transformed to projected light.However, although describe the example below, the example below on the projecting apparatus based on DMD The whole brightness that can be applied to wherein pixel is the part of controllable other kinds of projecting apparatus rather than pixel.

Projected pixel 1101-A is the projected pixel of the image of the DMD-A adjacent with overlapping region, and projected pixel 1101-B is the projected pixel of the image of the DMD-B adjacent with overlapping region.Each in projected pixel 1101-A, 1101-B It is shielded the projected pixel in their overlapping regions between projected pixel 1101-A, 1101-B to be distinguished.Also describe The nominal example brightness of each projected pixel 1101-A, 1101-B.For example, all projected pixel 1101-A are nominally 10 The brightness of unit, and all projected pixel 1101-B are nominally the brightness of 20 units；Therefore, in overlapping region, brightness It would be about 30 units.

Therefore, in fig. 11, any one projected pixel 1101-A, 1101-B in DMD-A, DMD-B all not by Control mixes for black level, therefore the overlapping region of discribed image seems more than the adjacent Non-overlapping Domain of each image It is bright.According to resolution ratio, the distance of projecting apparatus to screen, camera lens etc., the projected pixel of projecting apparatus A, a B may be with another throwings The projected pixel of shadow instrument A, B are inconsistent.Therefore, this can be produced in terms of the pixel of each DMD adjacent with overlapping region is controlled Raw problem.For example, the only partially pixels Chong Die with DMD-B, and being only from of some projected pixels 1101-A from DMD-A Only some projected pixels 1101-B of DMD-B is only partially Chong Die with the pixel in DMD-A.In other words, along from DMD-B Image the edge line Chong Die with the image from DMD-A, the projected pixel of projected pixel and DMD-A from DMD-B is not Alignment.In fact, the respective edges of the projected pixel from DMD-B may pass through the centre of the projected pixel of DMD-A.This may Mean according to method 400, the view field of the DMD-A projected pixel Chong Die with DMD-B projected pixels should not be endowed volume Outer light (because overlapping has been bright), and according to method 400 (making its brightness closer to overlaid intensity), not with DMD-B Other view fields of the identical DMD-A projected pixels of projected pixel overlapping should give additional light.However, cannot with it is identical Another regions of single DMD projected pixels differently control a region of single DMD projected pixels：Light can be added to Or it is not added to entire projected pixel.

However, it is possible to use shake and/or bit plane control DMD pixels to given luminance level, for example, can control System is added to the additional light quantity of each projected pixel 1101-A, 1101-B, for example, with the overlapping degree in every pixel basis into than Example.

This is shown in FIG. 12, which is substantially similar to Figure 11, wherein identical element has identical number.Especially Ground, projected pixel 1101-A, 1101-B increases to the brightness of 30 units, unless they only partly overlap；So increase of brightness Amount is related with degree of overlapping.Therefore, depending on overlapping degree, by three projected pixels partly overlapping with the projected pixel of DMD-B 1101-A controls are the value of less than 30 units (such as 25 units, 28 units and 23 units), can be in method 400 Frame 401 at determine；And according to overlapping degree by two projected pixel 1101-B partly overlapping with the projected pixel of DMD-A It controls as the value of less than 30 units (such as 29 units and 24 units), can be determined at the frame 401 of method 400.

But according to projecting apparatus resolution ratio, screen size, viewing distance and pixel size, the unique portion of Figure 12 is overlapped Projected pixel may still for beholder it is visible (for example, when image 216 resolution ratio be less than human visual system " depending on During nethike embrane resolution ratio ").Therefore, the wherein one side projected pixel line Chong Die with the projected pixel of another projecting apparatus can seem It is brighter than opposite side, the object line of beholder may be caused.Various forms of shakes can be used or introduce some noises/random Property is further reduced and/or mixes and/or hide the brightness of these lines so that and line is not solid brightness, but with distance change, So that the line is less visible for beholder.

Pixel size variation

Next it will be noted that Figure 13, a part of image pixel from two projecting apparatus is depicted again and (such as is thrown Image element) (for example, being labeled as the image of first projecting apparatus of " projecting apparatus X " and from the second projection labeled as " projecting apparatus Y " The image of instrument), however, the projected pixel of each projecting apparatus X, Y have different sizes, in addition, size can change (such as Plain size variance).Projected pixel from projecting apparatus X is not only of different sizes with the projected pixel of projecting apparatus Y, but also even if in list In a projecting apparatus X, Y, the synthesis " size " of " projected pixel " on object/screen is (for example, such as by the image from projecting apparatus The size in the region of the bright object (screen) of the illumination of the pixel projection of modulator), from one on identical video modulator Projected pixel changes to next projected pixel, due to the projection of object/screen and/or lens etc. and/or the angle of curvature.

Therefore, although not describing：The line of each image in Figure 13 can also be curved, on curved screen And/or when being projected by fish eye lens.

Under any circumstance, as shown in the figure, the size of the projected pixel 1301 of projecting apparatus Y is less than the projection image of projecting apparatus Y The size of element 1303, however other projected pixels of projecting apparatus Y can be less than projected pixel 1301.

Since the area of each projected pixel in each image is reduced, because same amount of light concentrates on smaller region In, thus each projected pixel in each image integrated luminosity it is similar in the case of, the brightness of projected pixel increases.It is similar Ground increases, luminance-reduction with the area of projected pixel.Thus, for example, the brightness of projected pixel 1301 will be above projected pixel 1303 brightness.

Therefore, when application process 400 in these cases, it may be considered that projected pixel size.For example, in frame 401, it can Mathematically to determine the position of each in four angles of projected pixel so that it can determine the size of projected pixel, and because This determines its relative luminance and how the additional optical of addition to be used to increase corresponding projected pixel at the frame 405 of method 400 Brightness.In in these embodiments some, in order to reduce the use of process resource, it may be determined that only one sample projection image Plain grid it is correspondingly sized, and may be inserted into the correspondingly sized of projected pixel therebetween.

In other situations, it may be considered that the luminance difference of view field.For example, the pixel intensity of image may also Article/the shelter being subject between light bulb, optical device and/or the projecting apparatus of projecting apparatus and the object of projector instrument image Influence.In fact, due to light bulb and/or optical element, projection centre can be slightly brighter than projecting edge (particularly corner).It is bright This species diversity of degree and/or variation can be by receiving (such as from manufacturer etc.) for the given product based on projecting apparatus Typical case's " decline " of the brightness of board/model/brand etc. come explain and/or by project white or grey (or similar) image simultaneously The brightness (such as using camera and/or photometer) on image is measured to measure the brightness change on single projector.Alternatively, it throws Shadow instrument can with projection of color images (for example, individually in corresponding planes of color, such as red, green and blueness), and It can determine the brightness of each plane.Alternatively, " mask " that blocks can be provided (for example, such mask can use tricks The draftings such as calculation machine software).It is anyway possible to this brightness data and/or mask are supplied to device 201, when in method 400 In determine how change image-region brightness when, this brightness data and/or mask can be used.

The brightness of distance based on overlapping region

Next concern Figure 14 is similar to the size variation of Figure 13, the wherein projected pixel of projecting apparatus Y.In addition, Figure 14 The embodiment of method 400 is depicted, wherein the throwing adjacent with overlapping region can be controlled using the distance to overlapping region The brightness of image element.Particularly, similar to the embodiment of Fig. 9, when " inclination " and/or when brightness is changed from overlapping region, Light can be added to projected pixel based on their distances to the edge of overlapping region so that closer to the edge of overlapping region Projected pixel brightness ratio further away from the edge of overlapping region projected pixel increase.In order to determine with edge away from From, can using include but not limited to KD trees (k Wei Shu) data structure rationally find nearest region and/or point soon And/or pixel.

For example, it in fig. 14, determines from the center of each of the projected pixel 1401,1403 of projecting apparatus X to projecting apparatus Y Image edge corresponding distance, using projecting apparatus Y projected pixel corner as the datum mark at edge (although can be with Use other points of the projected pixel of projecting apparatus Y), the distance indicated by corresponding arrow.The increasing of the brightness of projected pixel 1403 The brightness of projected pixel 1401 can be more than by adding because projected pixel 1403 than projected pixel 1401 closer to the figure of projecting apparatus Y The edge of picture.

When more than two images overlapping from more than two projecting apparatus, similar technology can be used so that deposit In more than one overlapping region.For example, Figure 15 is set forth below, which depict (be labeled as " projecting apparatus X " simultaneously from projecting apparatus X With the image of dotted lines), projecting apparatus Y (labeled as " projecting apparatus Y " image and with depicted as solid lines) and projecting apparatus Z (be labeled as The image of " projecting apparatus Z " is simultaneously drawn with thick dashed line) three images part, image overlapping (for example, as shown in Figure 2).

In these embodiments, some overlapping regions have two images of overlapping (with overlapping region 217-1,217- 2nd, 217-3 is overlapped), and some overlapping regions have three images Chong Die with overlapping region 217-4.Therefore, it is added to every The amount of brightness of a region (for example, outside overlapping region) and its projecting apparatus becomes slightly more complex.Particularly, it is to be added additional All projecting apparatus X, Y, Z in overlapping region of light between separate.It can uniform Ground Split or based on which projection image Element is most bright weighting (such as based on size) segmentation, and most possibly generates additional light.

In other words, " distance of overlapping " becomes much more complex, and can be with method of adjustment 400, to determine with being more than one The adjacent each pixel in a overlapping region：Nearest overlapping region edge, the brightness of each overlapping region and/or most bright region (that is, which slope should be followed for given projected pixel).It is adjacent with more than one overlapping region to determine how increase The brightness of given projected pixel has several possibilities：

A. brightness is increased based on " most powerful " edge：In the art, for given projected pixel, such as project Pixel 1501, determine nearest overlapping region edge (for example, for speed, the technology can use be stored in during KD trees are formed Edge and/or using KD trees " bucket ")；The increase of required light determines that the brightness direction of given projected pixel is each corresponding Nearest edge brightness (for example, the pixel for leaving edge, brightness rise but not necessarily until edge pixel it is bright Degree)；And use maximum increase.It is nearest with the instruction of two arrows and projected pixel 1,501 two for example, as shown in figure 15 Edge, and the brightness of projected pixel 1501 is increased to the greater in two increases.

In addition, when " left side " edge is brighter, but range pixel 1501 is farther, the brightness of pixel 1501 can be compared with " right side " Edge and raise, whichever increases maximum.For example, can according to function maxima (increase by 1, increase by 2 ...) and determine " most have Influence power " border determines multiple increased maximums (increasing by 1, increase 2....), and each edge increases by one.B. it is based on The increase brightness of " accumulation edge " technology；Nearest overlay region is determined for given projected pixel (such as projected pixel 1501) The technology (such as " most powerful " technology) at domain edge；It determines the brightness of given projected pixel being increased to each corresponding most near side (ns) The increase of light needed for the brightness of edge；And using increased and.For example, can from function summation (increase by 1, increase by 2 ...) It determines cumulative rises, increases and (increase by 1, increase 2...) addition, each one increase in edge by multiple.It will be appreciated, however, that It can determine that the brightness of given pixel increases using other functions similar to summation () and/or maximum ().

Mirrored pool

As described above, in DMD projecting apparatus, and with reference to figure 16, around each DMD, exist " mirrored pool " (POM).It is special , Figure 16 is not similar to Figure 11, however, the 3rd image from the 3rd DMD-C the 3rd image of (be labeled as " DMD-C ") with The image overlapping of DMD-A and DMD-B.As shown in figure 16, the shadow region of the edge of each image DMD-A, DMD-B, DMD-C Corresponding to the POM of each image, therefore labeled as POM-A, POM-B, POM-C.In other words, each POM is included along cannot Several pixel/speculums at the edge of controlled DMD pixels/speculum, they always send minimum light (black level Light), and cannot plus or minus light.When projecting relatively bright content (for example, being determined using threshold value and/or by comparing Generic pixel brightness and relative threshold difference) when, any neighbouring non-POM projected pixels are all brighter, therefore can usually ignore POM projector pixels.However, when POM projected pixels form a part for overlapping region and/or (are shown black in adjacent area Color or close to black) when, the brightness of POM projected pixels is considered in implementation 400, even if their brightness cannot be changed.

For example, whenever method 400 causes additional light being ideally added to the timing really of POM projected pixels, which must The non-POM projected pixels that must be alternatively added in the region.For example, in figure 16, the projected pixel 1601 of the image of DMD-A And projection POM-B pixels it is Chong Die with all three images overlapping region 1603 it is adjacent.Projected pixel 1601 can be controlled To generate additional light, but the POM-B projected pixel Chong Die with projected pixel 1601 cannot be controlled to generate additional light.Cause This, in method 400, at frame 405, increases the brightness of projected pixel 1601, to generate near for overlapping region 1603 All required additional light that projected pixel 1601 and POM-B projected pixels determine.

However, further understand, for the shadow casting technique in addition to DMD, POM may be not present and/or POM sizes can be with For 0, this can simplify techniques described herein.

Colored black

It was hypothesized that each projecting apparatus 207 (s) generated the black level of similar color in the past.However, in practice, no With projecting apparatus can generate the black level of color change, but very little.For example, a projecting apparatus can generate slightly red The light of black level, another projecting apparatus can generate the light of the black level slightly for blueness.Therefore, when additional light is added to During the projected pixel of neighbouring overlapping region, color transition is still visible.In these embodiments, can with amending method 400 with This color distortion is at least considered in two ways:

A. light (such as light of uniform color) can be added to overlapping region, rather than just adjacent " ramping up " Region.It, can be at least in the color of its perception of Edge position control so that the side of overlapping region using the light of the addition in overlapping region Edge approaches and/or similar to the color of adjacent Non-overlapping Domain and/or same.In addition, the color across overlapping region can To deviate from edge mixing.For example, in this technology, due to adding the light of uniform color, so ambiguously control area Color；But come " wash-off " these regions using the light of uniform color.

B., colourama can be added to the region adjacent with overlapping region so that addition brightness and color, and will throw The natural colour of shadow instrument neutralizes and/or is mixed into the color of overlapping region.By this technology, we will increase brightness, because Tend to be overlapped for it and change the color on slope to realize final color (as we reach that in overlapping region In).

In other words, in two kinds of technologies, the color of one or more overlapping regions and adjacent area is changed to color Matching and/or color mixing.

It as described below, can further amending method 400.

It is anti-aliasing

Along the edge (adaptive) of overlapping region, can apply it is anti-aliasing, to help to minimize the notification capabilities at edge, Similar to the technology shown in Figure 12.

It is adaptive to solve

The addition light calculated on each projecting apparatus needed for each pixel may be very slow.Therefore, in some embodiments, exist Method 400 is performed in each n-th projected pixel to determine how the increase brightness at frame 405, and is determined using interpolation How the brightness of remaining projected pixel is increased.This technology can reduce the use of process resource and improve answering for method 400 Use speed.In addition, this interpolation can be adaptive：For example, when in region, the difference between n-th pixel it is too big and/ Or during higher than threshold value, (for example, increasing the definite brightness of the N number of projected pixels of kth * with the N number of projected pixels of (k+1) * really Fixed brightness is compared (for example, " next " pixel, no matter " from left to right " and/or " from top to bottom ")), it may be determined that area The brightness of each N/2 projected pixels increases (although this can reduce the implementation speed of method 400) in domain, then can make again Use interpolation.If deviation remains above threshold value, brightness increase of each N/4 projected pixels in region etc. can be determined.

It (is drawn by number) overlapping region

In order to implement method 400 shown in Fig. 8, the brightness of the wherein entire part of the image outside overlapping region increases to weight The brightness in folded region increases instead of the brightness for calculating each projected pixel, it can be assumed that the first projected pixel in scan line Definite brightness increase (assuming that method 400 is applied to each row image pixel by pixel) is identical in corresponding scan line, until Until reaching overlapping region.Then, method 400 is used to determine the brightness of next scan line and/or next Non-overlapping Domain Increase, until reaching next overlapping region.This technology can be similar to the filling algorithm that floods, and can be colloquially called and " compile Code oil painting " technology, because the projected pixel in given area outside overlapping region increases to identical brightness.

However, in this technology, it is as shown in figure 13, each to project when pixel size and/or the brightness of compensation projection Pixel may have different brightness to increase.In order to which the brightness of approximate each projected pixel increases, and it is " high in order to minimize Expensive " (in terms of process resource) rear-projection calculating, the brightness of the projected pixel of mean size can be determined with application method 400, It then can be according to its actual correspondingly sized brightness for adjusting corresponding projected pixel pari passu.

The embodiment for being related at least one camera 214 is heretofore described, camera capture such as screen 215 Object digital picture.It will be appreciated, however, that at least one camera 214 includes being configured as obtaining projecting on object The sensor of at least part of digital picture (and/or picture) of multiple images 216 (includes but not limited to CCD, LCOS equipment Deng).Alternatively, image 216 can be referred to as projector image 216 with by them with being obtained by the sensor of at least one camera 214 Digital picture taken etc. distinguishes.

In addition, at least one camera 214 and/or its sensor not necessarily obtain the digital picture of whole object, even The part of object；On the contrary, at least one camera 214 and/or its sensor are configured as obtaining the multiple throwings projected on object At least part of digital picture of shadow instrument image 216.For example, in some embodiments, projector image 216 can use Infrared light and/or ultraviolet light and/or the light of (and/or sightless) frequency that cannot be watched by human visual system project (such as the mankind and/or be configured as system to be imaged with people's similar mode to object).In addition, at least one camera 214 and/or its sensor can be configured as the digital picture for obtaining identical " invisible " light rather than " visible ray ".At this In a little embodiments, at least one camera 214 and/or the acquisition of its sensor project on object but are not necessarily the multiple of object At least part of digital picture of projector image 216.

Therefore, in these embodiments, method 400 is adapted so that controller 220 is configured as：Use at least one biography Sensor (a for example, at least camera 214) come determine the subpoint of given projector image 216 and multiple projecting apparatus 207 other Corresponding correspondence between the corresponding point of the corresponding video modulator of projecting apparatus 207, corresponding correspondence represent Corresponding to the corresponding point of the video modulator of another projecting apparatus 207 of one or more overlapping regions 217；Determine one or more The corresponding brightness of a overlapping region 217；And control the picture of each of given image modulator and each video modulator Element, at least adjacent edge pixel correspond to the edge of one or more overlapping regions 217 to be based on corresponding brightness increase pair Answer the brightness in projector image region.

In addition, other kinds of sensor is configured as obtaining at least part of number in multiple projector images 216 Word image.For example, Figure 17 next will be paid close attention to, which depict the systems 1700 substantially similar to system 200, have similar member Part has similar number.Assuming that the system of Fig. 1 includes optical sensor as described below, it can be with the side similar with system 200 Formula is used together system 1700 with the system of Fig. 1.

On the contrary, system 1700 does not include camera and/or camera sensor, (and/or camera and/or camera pass with system 200 Sensor is optional).On the contrary, in system 1700, multiple optical sensors 1714 are built-in in screen 215 and positioned at screens One or more near 215.Each optical sensor 1714 communicates with computing device 201, and be configured as sensing light and Instruction is provided to computing device 201 when sensing light.In addition, depositing 1603 reservoirs 222 includes optical sensor position data 1750, It stores the position of each optical sensor 1714.Therefore, when optical sensor 1714 detects light, such as projector image is worked as One or more of 216 when projecting on screen 215, and optical sensor 214 provides such instruction to computing device 201, The 1714 its corresponding position of optical sensor for detecting light is associated.In this way, optical sensor 1714 obtains multiple At least part of digital picture of projector image 216.

For example, each optical sensor 1714 includes but not limited in photodiode, phototransistor, phototransistor etc. One or more, and be configured as the detection light in the region than the pixel arrangement bigger of camera 214, such as about 1mm²Or on smaller region.However, the size of the optical sensor 1714 in Figure 17 is not necessarily to scale, and only by corresponding Circle schematically show.In addition, depicted in an arrangement that some positions of optical sensor 1714, but optical sensor 1714 can be at Other positions.Similarly, depicted in an arrangement that a certain number of optical sensors 1714, but present embodiment include it is more or Less optical sensor 1714.In fact, the apparent position of the projector image 216 at screen 215 is known embodiment party In formula, the configuration of optical sensor 1714 can be selected so that：When screen 215 is plane, each projector image 216 by At least three optical sensors 1714 or at least four optical sensors 2174 detect to consider measurement error；And work as screen 215 not When being plane (such as bending and/or three-dimensional), each projector image 216 is detected by least six optical sensors 1714.So And in a specific embodiment, at least each projector image 1714 uses optical sensor 174, for redundancy and considers Measurement error.

In addition, in some embodiments, optical sensor 1714 is embedded in screen 215 and/or projector image 216 projects It arrives in object thereon, such as projects in the surface of screen.In these embodiments, it is stored in optical sensor position data The position of optical sensor 1714 in 1750 can be such as being supplied to screen 215 from the manufacturer screen 215.

In other embodiments, optical sensor 1714 is placed with and/or projector image 216 adjacent with screen 215 The object projected to, such as the array as interconnection optical sensor；In fact, in these embodiments, once technical staff The position of the projector image 216 on screen 215 is visually determined, then technical staff etc. can be interim by optical sensor 1715 It is located near the surface of screen 215.In these embodiments, the light sensing being stored in optical sensor position data 1750 The position of device 1714 can be provided by technical staff using such as survey tool, the camera calibrated etc..

Anyway, compared with by camera 214 and/or the digital picture and/or digital picture of the generation of its optical sensor, by The digital picture for the projector image 216 that optical sensor 1714 and optical sensor position data 1750 provide has low-down point Resolution.For example, the digital picture of the projector image 216 provided by optical sensor 1714 can be each digital picture as little as 3- 8 pixels.

In addition, it is contemplated that the position of optical sensor 1714, structure light image etc., can be used to determine 1714 He of optical sensor Correspondence between the point of video modulator 617.In other words, the correspondence of 2D optical sensors to 2D projecting apparatus is to determine , it is used for determining projecting apparatus to the correspondence of projecting apparatus (for example, representing to correspond to one or more overlay regions again in turn The corresponding correspondence put accordingly of the video modulator of the projecting apparatus 207 in domain 217).These correspondences can be applied to With reference to the back projection technique of Fig. 6 A descriptions.

In fact, determine to be positioned corresponding to the point of optical sensor position by using interpolation technique and/or extrapolation technique Between overlapping region in point mapping and/or correspondence, such correspondence can be by video modulator 617-1's Point determines that these points are located in overlapping region and correspond to and do not correspond to the subpoint at optical sensor 1714.

In fact, this interpolation and/or extrapolation can also be including camera 214 and 2D cameras to 2D projecting apparatus correspondences Embodiment in occur.For example, in these embodiments, it is true only for the subset of the point (such as pixel) of camera sensor 2D cameras are determined to the correspondence of 2D projecting apparatus so that back projection technique is only used for one of point p1i, p2i, p3i in overlapping region A subset, and can determine the overlay region being located between the point used in rear-projection using interpositioning and/or extrapolation technique The mapping of point in domain and/or correspondence.

In addition, it may be adapted to consider to be configured as obtaining multiple projector images 217 at least using 230 and method 400 The one or more sensors of the digital picture of a part, this one or more sensors include one or more of camera 214 A camera sensor and optical sensor 1714.In fact, in some embodiments, the camera sensor and light of camera 214 pass The combination of sensor 1714 is used for back projection technique as described herein to perform mapping.

In fact, being returning briefly to Figure 17, the storage of memory 222, should when being handled by controller 220 using 230a Using enabling controller 220 and/or computing device 201：At least using being configured as obtaining multiple projector images 216 The sensor of at least part of digital picture determines the subpoint of given projector image 216 and multiple projecting apparatus 207 Correspondence between the corresponding point of the corresponding video modulator of other projecting apparatus 207, corresponding correspondence expression pair It should be in the corresponding pixel of the video modulator of other projecting apparatus 207 of one or more overlapping regions 217；Determine one or more The corresponding bright of a overlapping region 217；And control each of given image modulator and corresponding video modulator extremely The pixel of few neighbouring edge pixel corresponding with the edge of one or more overlapping regions 217 is increased with being based on corresponding brightness Brightness in corresponding projector image region.In some embodiments, using 230a further such that controller 220 and/or Computing device 201 can：The subset of corresponding correspondence is determined using interpositioning and/or extrapolation technique.

Figure 18 is paid attention to now, and which depict the methods mixed for digital black level according to non-limiting embodiment 1800 flow chart.For the explanation of assisted method 1800, it will be assumed that method 1800 uses system 200 and/or system 1700, tool It is performed by the controller 220 of equipment 201 body, such as when the processing of controller 220 is using 230a.In fact, method 1800 is System 200 and/or system 1700 and/or equipment 201 and/or a kind of mode of controller 220 can be configured.In addition, method 1800 following discussion will be so that further understand equipment 201, system 200 and/or system 1700 and its various assemblies.However, It should be appreciated that system 200 and/or system 1700 and/or equipment 201 and/or controller 220 and/or method 1800 can change, And discussion that need not be as described herein in accurately works, and this variation is within the scope of the invention.

Anyway, it is emphasized that unless otherwise indicated, method 1800 need not according to shown exact sequence into Row；And various frames can be equally performed parallel rather than successively；Because the element of the method 1800 is referred to herein as " frame " Rather than " step ".It should again be understood, however, that method 1800 can also be realized in the variation of system 200.It although in addition, will Computing device 201 is described as realizing and/or performs each frame of method 1800, but it is to be understood that each frame of method 1800 All carried out using processing using the controller 220 of 230a.

It is also understood that for each given execution method 1800 of projecting apparatus 207 of multiple projecting apparatus 207, and because This can be directed to each projecting apparatus 207 and/or the order according to each projecting apparatus 207 and/or each area for projecting apparatus 207 Section/region/scan line/pixel-parallel performs method 1800.

It is further assumed that multiple projector images 216 are overlapped in one or more overlapping regions 217 in method 1800.

In frame 1801, controller 220 is using at least one sensor (for example, being configured as obtaining multiple projector images At least one sensor of 216 at least part of digital picture) determine that the given projecting apparatus 207 by object projects to The subpoint of given projector image 216 on object (such as screen 215) and other projecting apparatus of multiple projecting apparatus 207 The correspondence put accordingly of 207 corresponding video modulator, corresponding correspondence represent to correspond to one or more The corresponding point of the video modulator of other projecting apparatus 207 of overlapping region 217.Frame 1801 is generally similar to the frame of method 400 401, however frame 1801 is independent of camera 214, and at least one sensor of frame 1801 include it is following in one：Camera 214 camera sensor and optical sensor 1714.

Optional frame 1802 (if frame 1802 is represented by dotted lines, then it represents that be optional), controller 220 uses interpolation One or more in technology and extrapolation technique determine corresponding corresponding subset, such as by using as described on Fig. 6 A Definite corresponding correspondence.It, can be real with linking frame 1801 and/or after frame 1801 in addition, in some embodiments Apply frame 1802.

In frame 1803, controller 220 determines the corresponding bright of one or more overlapping regions 217.The generally class of frame 1803 It is similar to the frame 403 of method 400.

In frame 1805, controller 220 controls the pixel of each of given image modulator and corresponding video modulator, At least adjacent edge pixel corresponds to the edge of one or more overlapping regions 217 increases corresponding throw to be based on corresponding brightness Brightness in shadow instrument image-region.Frame 1805 is generally similar to the frame 405 of method 400.

In addition, method 1800 in addition as in this specification on Fig. 7 to Figure 16 it is described carry out.

In general, this specification is directed to use in the corresponding pixel and object (such as screen) of projector image modulator It maps to find the method and system of the overlapping region of projector image between view field, and controls prominent in overlay region The region in overseas portion is to minimize visible transition therebetween.

It will be understood by those skilled in the art that in some embodiments, equipment 101,105,108,201 and content player 103 function can use the hardware of pre-programmed or firmware components (for example, application-specific integrated circuit (ASIC)), electric erasable that can compile Journey read-only memory (EEPROM) etc.) or other associated components implement.In other embodiment is existing, calculating can be used to set Standby to realize the function of equipment 101,105,108,201 and content player 103, which may have access to storage for calculating The code memory (not shown) of the computer readable program code of the operation of equipment.Computer readable program code can store On computer readable storage medium, which is (for example, removable floppy disk, CD- by these components ROM, ROM, fixed disk, usb driver) it is directly fixed, tangible and readable.Moreover, it will be appreciated that computer-readable journey Sequence can be stored as the computer program product for including computer usable medium.In addition, permanent storage appliance can include meter Calculation machine readable program code.It is also understood that computer readable program code and/or computer usable medium can be including non-temporary When property computer readable program code and/or non-transitory computer usable medium.Alternately, calculating can remotely be stored Machine readable program code, but can via modem or by transmission medium be connected to network (include but not limited to because Special net) other interface equipments these components can be transmitted.Transmission medium can be with right and wrong move media (for example, optics and/or number Word and/or analog communication line) or move media (for example, microwave, infrared, Free Space Optics or other transmission plans) or its Combination.

Claims

1. a kind of system of digital black level mixing, including：

Multiple projecting apparatus, each projecting apparatus include video modulator, and the multiple projecting apparatus is oriented projection by scheming accordingly As the corresponding projector image that modulator is formed, multiple projector images are overlapped in one or more overlapping regions；

At least one sensor is configured as obtaining at least part of digital picture in the multiple projector image；With And

Computing device is configured as each given projection for including given video modulator for the multiple projecting apparatus Instrument：

Using at least one sensor determine the subpoint of given projector image with it is other in the multiple projecting apparatus Corresponding correspondence between the respective point of the corresponding video modulator of projecting apparatus, corresponding correspondence expression pair It should be in the corresponding point of the corresponding video modulator of other projecting apparatus of one or more overlapping regions；

Determine the corresponding brightness of one or more of overlapping regions；And

Control the pixel of each of the given image modulator and corresponding video modulator, at least adjacent edge pixel Increase corresponding projector image region corresponding to the edge of one or more of overlapping regions to be based on corresponding brightness In brightness.

2. the system of digital black level mixing according to claim 1, wherein, it is at least one or more for corresponding to The edge pixel at the edge of a overlapping region realizes the correspondence between the subpoint and corresponding point.

3. the system of digital black level mixing according to claim 1, wherein, corresponding correspondence includes and institute State the corresponding dead zone pixel phase of the periphery of each in given image modulator and corresponding video modulator Associated correspondence.

4. the system of digital black level mixing according to claim 1, wherein, the computing device is configured as control institute It states outside one or more overlapping regions and at least neighbouring edge corresponding with the edge of one or more of overlapping regions At least a portion of the pixel of each in the given image modulator of pixel and corresponding video modulator, with Increase the brightness in corresponding projector image region to match the corresponding bright of one or more of overlapping regions.

5. the system of digital black level mixing according to claim 1, wherein, the computing device is configured to Control each in the given video modulator and corresponding video modulator it is at least neighbouring with it is one Or the pixel of the corresponding edge pixel in edge of multiple overlapping regions, to be increased according to the respective distance with the edge pixel Brightness in corresponding projector image region.

6. the system of digital black level mixing according to claim 1, wherein, the computing device is configured to Each in the given image modulator and corresponding video modulator is controlled according to one or more of following The pixel of at least neighbouring edge pixel corresponding with the edge of one or more of overlapping regions：The projector image of projection Nearest correspondence overlapping region edge；One or more nearest correspondence overlapping region edges of projector image based on projection Weighting；And each in one or more nearest correspondence overlapping region edges of the projector image from projection Integrated brightness.

7. the system of digital black level mixing according to claim 1, wherein, the computing device is further configured For：It determines to form the image of the given projector image of one or more of overlapping regions and other projector images The corresponding orientation of pixel；And the given image modulator is further controlled according to the corresponding orientation of described image pixel With the pixel of each in corresponding video modulator.

8. the system of digital black level mixing according to claim 1, wherein, the computing device is further configured For：It determines to form the image of the given projector image of one or more of overlapping regions and other projector images The corresponding projected size of pixel；And the given figure is further controlled according to the corresponding projected size of described image pixel As the brightness of the pixel of each in modulator and corresponding video modulator.

9. the system of digital black level mixing according to claim 1, wherein, the computing device is further configured For：It determines to form the image of the given projector image of one or more of overlapping regions and other projector images The corresponding projected color of pixel；And the given figure is further controlled according to the corresponding projected color of described image pixel As the pixel of each in modulator and corresponding video modulator.

10. a kind of method of digital black level mixing, including：

In including following system：Computing device；Multiple projecting apparatus, each projecting apparatus includes video modulator, the multiple Projecting apparatus is oriented the corresponding projector image that projection is formed by corresponding video modulator, and multiple projector images are one It is overlapped in a or multiple overlapping regions；And at least one sensor, being configured as being directed to includes the every of given image modulator A given projecting apparatus obtains at least part of digital picture of multiple projector images of the multiple projecting apparatus：

11. the method for digital black level according to claim 10 mixing, wherein, at least for correspond to it is one or The edge pixel at the edge of multiple overlapping regions realizes the correspondence between the subpoint and corresponding point.

12. the method for digital black level according to claim 10 mixing, wherein, corresponding correspondence include with The corresponding dead zone pixel of the periphery of each in the given image modulator and corresponding video modulator Associated correspondence.

13. the method for digital black level mixing according to claim 10, further comprises controlling one or more of It is given outside overlapping region and described at least neighbouring edge pixel corresponding with the edge of one or more of overlapping regions Determine at least a portion of the pixel of each in video modulator and corresponding video modulator, it is described corresponding to increase Projector image region in brightness to match the corresponding bright of one or more of overlapping regions.

14. the method for digital black level mixing according to claim 10, further comprises controlling the given image At least neighbouring side with one or more of overlapping regions of each in modulator and corresponding video modulator The pixel of the corresponding edge pixel of edge, to increase corresponding projector image area according to the respective distance with the edge pixel Brightness in domain.

15. the method for digital black level according to claim 10 mixing, further comprise according to one in following or It is multiple come control in the given image modulator and corresponding video modulator each it is at least neighbouring with it is one Or the pixel of the corresponding edge pixel in edge of multiple overlapping regions：The nearest correspondence overlapping region of the projector image of projection Edge；The weighting at one or more nearest correspondence overlapping region edges of the projector image based on projection；And to haul oneself willingly into The integrated brightness of each in one or more nearest correspondence overlapping region edges of the projector image of shadow.

16. the method for digital black level according to claim 10 mixing, further comprise determining that be formed it is one or The corresponding orientation of the given projector image of multiple overlapping regions and the image pixel of other projector images；And It is further controlled in the given image modulator and corresponding video modulator according to the corresponding orientation of described image pixel The pixel of each.

17. the method for digital black level according to claim 10 mixing, further comprise determining that be formed it is one or The corresponding projected size of the given projector image of multiple overlapping regions and the image pixel of other projector images； And the given image modulator and corresponding image are further controlled according to the corresponding projected size of described image pixel The brightness of the pixel of each in modulator.

18. the method for digital black level according to claim 10 mixing, further comprise determining that be formed it is one or The corresponding projected color of the given projector image of multiple overlapping regions and the image pixel of other projector images； And the given image modulator and corresponding image are further controlled according to the corresponding projected color of described image pixel The pixel of each in modulator.

19. a kind of non-transitory computer-readable medium for storing computer program, wherein the computer program performs use In：

It communicates with multiple projecting apparatus in computing device, each includes：Computing device；Multiple projecting apparatus, each projecting apparatus include figure As modulator, the multiple projecting apparatus is oriented the corresponding projector image that projection is formed by corresponding video modulator, Multiple projector images are overlapped in one or more overlapping regions；And at least one sensor, being configured as being directed to includes Each given projecting apparatus of given image modulator obtains at least part of of multiple projector images of the multiple projecting apparatus Digital picture：