Classifications

• • G—PHYSICS
  • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
  • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
  • G03B15/00—Special procedures for taking photographs; Apparatus therefor
  • G03B15/08—Trick photography
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/222—Studio circuitry; Studio devices; Studio equipment
  • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
  • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/222—Studio circuitry; Studio devices; Studio equipment
  • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
  • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
  • H04N5/2723—Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement

Definitions

• the present invention generally relates to video processing.
• the invention relates particularly, though not exclusively, to marking an object in a video camera footage and modifying the video camera footage.
• Another application of video processing is adapting advertisements in international sports events. For instance, national laws may ban broadcasts with cigarette advertisements or advertisers wish to tailor their advertisements to different markets e.g. by using different texts and illustrations depending on target culture, language and business opportunities.
• a fixed advertisement may be specially illuminated such that it can be discerned by means of a camera that operates with infrared light.
• a unit of two cameras is then used to record the event such that both cameras have identical view, with one camera capturing the visible picture and another camera capturing a marking video using infrared light range in order to enable detecting of advertisements and their positions in the visible picture.
• the advertisements are then detected from the marking video and replaced with customized advertisements to the extent desired.
• the illumination of the advertisements and their surroundings should be controlled such that there would be no excess unwanted infrared light or reflections of infrared light. While this may be possible indoors or generally in the absence of sunshine, there are often strong IR-rich light beams which may interfere with the detection of the advertisements. If the detection of an advertisement fails, it may be that no advertisement location is found or even worse, and more likely, the advertisement area may leak out to cover the picture of the actual event. While the advertisers may be unhappy if their paid advertisements fail to appear in a televised sports event, the audience may be even more disappointed if the opponent of a boxing match is covered by an advertisement that should only loom behind the boxer. Basically, any objects reflecting sufficient amount of infrared light, such as the skin, eyeballs, teeth, or clothes of a person might confuse the advertisement detection, if there is an excess of infrared light in the range captured by the marking camera.
• a method for marking an object in a video camera image comprising temporally optically alternating a marking surface of an object according to an alternating scheme, capturing successive images of a scene covering the marking surface, detecting temporal optical alternation in the successive images and detecting the marking surface from the detected temporal optical alternation in the successive images.
• the surface may be temporally optically alternated such that the surface appears to have a constant light intensity when seen by a human eye and by a camera that uses visible light, whilst the image detector can identify the surface h baasseerdi n onn t thhee t tiimmee v vaarrvyiinnng i illlluummiinnaattiioonn.
• the marking of the object may comprise illuminating the object with periodical invisible or visible light at instants synchronized with image capture periods of the video image footage.
• a high speed television camera may be used to capture the video camera footage and the successive images of the scene.
• the frame rate of the video camera footage may be reduced from a higher original rate.
• the successive images of the scene and the video camera footage may be captured using separate objectives.
• the successive images of the scene and the video camera footage may be captured using a common objective and a beam splitter.
• the marking surface may be temporally optically alternated by means of a particular sequence.
• the sequence may be used to identify different marking surfaces.
• different marking surfaces may be distinguished by using combinations of different independent temporal optical alterations.
• the different independent temporal optical alterations may involve changes in polarity, frequency, or frequency distribution.
• the marking surface may comprise a particular visual pattern.
• the pattern may comprise inner edges starting from rectangle corners. Where only part of the marking surface is visible, any visible inner edges together with the outer edges of the rectangle may be extrapolated and corner places may be estimated to be on edge intersections.
• the method of the first aspect may provide significant advantages over prior known technologies. For instance, in chroma keying technique marking surfaces are colored with a predetermined color. The marking color cannot be used elsewhere in the scene in order to enable recognizing an object (such as the marking surface) by their color. This technique restricts the use of colors in the scene and its use is therefore mostly limited to studio environment. Using static invisible light to distinguish marking surfaces enables normal appearance of all visible colors in the scene. However, even when invisible light or invisible components of light are used, controlled lighting conditions are still required. For example, sunlight or reflections of sunlight may easily pose problems, because sunlight contains a strong infrared component. On the contrary, using temporal optical alternating of the marking surface may not similarly restrict the lighting conditions, because there is no high frequency oscillating light in nature. In terms of real-life illumination, significantly enhanced object recognition reliability may be achieved. Moreover, the use of different alternating sequences may further enable simultaneous identifying of different marking surfaces. Using sequences to construct certain visual patterns on marking surfaces may further enhance the object recognition reliability.
• an apparatus for detecting a marking surface in video camera footage by using successively captured images of a scene covering the marking surface comprising: a memory comprising instructions; and a processor configured to operate according to the instructions and accordingly to: detect temporal optical alternation in the successive images; and detect visible parts of the marking surface in the video camera footage from the identified temporal optical alternation in the successive images corresponding to a temporal optical alternating scheme.
• the scheme may be temporally repeated.
• the apparatus may be further configured to cause the temporal optical alternating of the marking surface.
• the detecting of the visible parts of the marking surface from the identified temporal optical alternation corresponding to a temporal optical alternating scheme may be based on detecting the presence of the temporal optical alternation from a particular series of captured images of the scene.
• the marking surface may be provided on an object.
• the object may be visible or invisible to human eye.
• An invisible object may be made of transparent material.
• the marking surface may be temporally optically alternated such that the marking surface appears to have a substantially constant light intensity when seen by a human eye and by a camera that uses visible light, whilst the image detector can detect the surface based on the temporal optical alternation of the marking surface that may result, for instance, from illumination by alternating light.
• the temporal optical alternating of the marking surface may comprise illuminating the object with periodical invisible or visible light synchronized with video camera image capturing.
• a high-speed television camera may be used to record the video camera footage and also to provide the successively captured images of the scene.
• the frame rate of the production video may be reduced from the frame rate at which the high-speed television camera operates.
• the successively captured images of the scene may be captured using a separate objective or using a common objective with capturing of the video camera footage and a beam splitter.
• the marking surface may be temporally optically alternated by means of a particular sequence. The sequence may be used to identify different marking surfaces.
• different marking surfaces may be distinguished by using combinations of different independent temporal optical alterations. The different independent temporal optical alterations may involve changing of polarity, frequency, or frequency distribution.
• the marking surface may comprise a particular visual pattern.
• the pattern may comprise inner edges starting from rectangle corners. Where only part of the marking surface is visible, any visible inner edges together with the outer edges of the rectangle may be extrapolated and corner places may be estimated to be on edge intersections.
• a method for detecting a marking surface in video camera footage by using successively captured images of a scene covering the marking surface comprising: detecting temporal optical alternation in the successive images; and detecting visible parts of the marking surface in the video camera footage from the identified temporal optical alternation in the successive images corresponding to a temporal optical alternating scheme.
• the method of the third aspect may further comprise temporal optical alternating of the marking surface.
• the marking surface may be temporally optically alternated such that the marking surface appears to have a substantially constant light intensity when seen by a human eye and by a camera that uses visible light.
• the detecting of the visible parts of the marking surface from the identified temporal optical alternation corresponding to a temporal optical alternating scheme may be based on detecting the presence of the temporal optical alternation from a particular series of captured images of the scene.
• the temporal optical alternating of the marking surface may comprise illuminating the object with periodical invisible or visible light synchronized with video camera image capturing.
• the marking surface may be temporally optically alternated using a particular sequence.
• the sequence may be used to identify different marking surfaces.
• the different marking surfaces may be distinguished by using combinations of different independent temporal optical alterations.
• the different independent temporal optical alterations may comprise changes of polarity, frequency, or frequency distribution.
• the marking surface may comprise a particular visual pattern.
• the pattern may comprise inner edges starting from rectangle corners.
• a system comprising a controller according to the second aspect and any one of the following: a marking surface alternator configured to temporally optically alternate the marking surface; a first camera configured to capture the video camera footage; a second camera configured to successively capture images of a scene covering the marking surface; and the marking surface.
• the marking surface alternator may comprise controllable illumination equipment.
• a computer program stored on a computer readable memory medium for controlling an apparatus detecting a marking surface in video camera footage by using successively captured images of a scene covering the marking surface comprising: computer executable program code configured to enable the apparatus, when executed by the apparatus, to detect temporal optical alternation in the successive images; and computer executable program code configured to enable the apparatus, when executed by the apparatus, to detect the marking surface from the identified temporal optical alternation in the successive images corresponding to the temporal optical alternating of the marking surface.
• the computer program stored on a computer readable memory medium according to the fifth aspect may further comprise computer executable program code configured to enable the apparatus, when executed by the apparatus, to perform any method according to any embodiment of the third aspect.
• FIG. 1 presents a schematic diagram representing a system according to an embodiment of the invention
• FIG. 2 presents a schematic diagram representing a system according to another embodiment of the invention.
• FIG. 3 presents a block diagram representing an imaging system according to an embodiment of the invention
• FIG. 4 presents an exemplary image taken of a particular scene with a high speed video camera, when a marking surface is not illuminated;
• FIG. 5 presents an exemplary image taken of a particular scene with the high speed video camera, when the marking surface is illuminated
• FIG. 6 presents a difference image based on the difference of images FIG. 4 and FIG. 5;
• FIG. 7 presents a sample image to be inserted in video stream
• FIG. 8 presents a modified sample image of Fig. 7 scaled, positioned and masked according to the difference image of Fig. 6;
• FIG. 9 presents a modified video image, where the modified sample image of FIG. 8 is inserted into original video image FIG 5;
• FIG. 10 presents a guiding pattern for marking an object according to an embodiment of the invention.
• Fig. 11 presents a block diagram of a unit for detecting a marking surface from signals received from a camera unit.
• Fig. 1 presents a schematic diagram representing a basic system 100 according to an embodiment of the invention.
• the basic system comprises an object with a marking surface 1 , a lamp 2 adapted for illuminating the marking surface, a camera unit 3 for capturing video image of a scene that here comprises the marking surface and for detecting the marking surface, and an obstacle 4 that partly masks or visually blocks the marking surface 1 from the camera unit 3.
• the camera unit 3 comprises as different functional blocks a video image camera 5 and an image detection camera 6.
• the video image camera 5 and the image detection camera 6 may comprise two separate optical light capture elements or a common light capture element.
• the basic system 100 further comprises a controller 1000 for controlling the operation of the camera unit and possibly for synchronizing the operation of the camera unit 3 and the lamp 2 by steering the timing of either or both of the camera unit 3 and of the lamp 2.
• controller 1000 for controlling the operation of the camera unit and possibly for synchronizing the operation of the camera unit 3 and the lamp 2 by steering the timing of either or both of the camera unit 3 and of the lamp 2.
• the controller 1000 may generally control the operation of the camera unit 3 and of the lamp 2. In this description, these operations are generally described as they occur without separate references to the source of corresponding instructions from the controller 1000. This simplification is intended to make this description easier to read. However, it should be appreciated that there may be a separated or integrated controller 1000 that may consist of one or more parts (e.g. distributed to different camera units and/or lamps 2) and that actually decides upon various operations.
• Fig. 1 facilitates the understanding of particular embodiments of the invention.
• Fig. 1 does not, however, limit the scope of the invention.
• various elements of Fig. 1 may be implemented differently.
• the lamp 2 is used to periodically alter the optical appearance of the marking surface.
• the optical appearance of the marking surface may alternatively be manipulated to change its reflectance, transparency, or emission.
• Each of the reflectance, transparency or emission may be altered in terms of intensity, frequency range, polarization angle, or any combination thereof.
• Liquid Chrystal Displays employ electric altering of polarization of liquid crystals in an electric field such that the reflectance may rapidly change.
• Micro-Electro-Mechanical Systems can be used to periodically move reflective or refractive optical elements (e.g. mirrors, prisms, and lenses) such that the appearance of the marking surface periodically alternates in a manner which enables automatic detection.
• reflective or refractive optical elements e.g. mirrors, prisms, and lenses
• the image detection camera 6 is correspondingly adapted to record frames which enable detecting corresponding differences.
• the image detection camera 6 may have a polarization filter.
• the lamp may be built in an object that carries the marking surface.
• the lamp 2 may comprise a plurality of Light Emitting Diodes (LEDs) and the marking surface may comprise a transparent or translucent cover such as a textile.
• the cover may present desired content such as one or more advertisements.
• the marking surface 1 may comprise a fixed advertisement of rectangular form.
• the video image camera 5 may be a video camera or television camera such as, for example, a National Television System Committee (NTSC) standard compliant camera.
• the video image camera 5 is generally capable of producing video footage at a production video frame rate.
• the production video frame rate may be, for instance, 30 frames per second (fps).
• the video image camera 5 is a high-speed video camera configured to capture video images at a rate of 60 fps, twice the frame rate of desired production video.
• a video image camera 5 operates additionally as an image detection camera 6 that takes at least two measurements during each production video frame interval.
• a marking surface is identified on the basis of different light intensity between the measurements, all other objects remaining at about constant light intensity.
• the areas of the object visible in the image frames are then determined by combining this information with a known relationship between the picture coordinates of video image camera 5 and those of the image detection camera 6.
• the image corresponding to the visible parts of the object may then be modified in a predetermined manner.
• the image detection camera 6 is configured to capture video images at a frame rate higher than the production video frame rate, say at four or six times the frame rate of the production video frame rate.
• the heightened frame rate of the image detection camera 6 may be utilized for various needs.
• a higher frame rate can be used to perform more image detections per one production video image frame so that any marking surfaces can be more reliably detected and/or motion in the video image frames may be better compensated when there is more information to compensate detected motion.
• Such compensation information is usable for compensating any shift in the position, orientation or scale of a marking surface 1 or of any obstacles 4 masking the marking surface 1.
• the higher frame rate can be used to distinguish different marking surfaces based on different optical appearance alternating patterns.
• a large number of different marking surfaces may be relatively simply distinguished without necessarily modeling the locations of the marking surfaces and the current scene of the camera unit 3. It is appreciated that it is possible to similarly provide different patterns for alternating the appearance of the marking surface in other embodiments of the invention and in particular in the second and third embodiments.
• FIG. 2 presents a schematic diagram representing a system 200 according to a second embodiment of the invention, in which the camera unit 3 comprises as separate and parallel light capture elements a video image camera 5 and an image detection camera 6.
• the video image camera 5 is configured to capture production video footage for optional further processing.
• the image detection camera 6 is configured to capture image detection frames for identifying objects that are illuminated with light that pulsates at a predetermined rate on the frequency area in which the image detection camera 6 operates.
• the system 200 of Fig. 2 is subjected to a parallax offset that may be compensated.
• the compensation may employ, for instance, computational estimating and compensating of the location of the marking surface detected by the image detection camera 6 before or on adapting the captured video footage.
• the image detection camera 6 is configured to capture images at the production video frame rate and using visible light, but with a time offset from the video image camera 5.
• the time offset may range between the exposure time of the video image camera (e.g. 1/200 s) and a half cycle of the video image camera.
• This variant may be constructed using two normal and similar video image cameras so that common spare parts can be used for each camera and sourcing may be simplified.
• the timing of the video image camera 5 and of the image detection camera 6 can brought very close to each other so that pairs of video image frames can be captured within instantaneous time and fast moving objects cause less errors to the detection of the marking surfaces than if the time difference is larger.
• This variant may require, however, particular synchronization between the camera unit 3 and the pulsating illumination of the marking surface in order to change the appearance of the marking surface 1 between the image capture by the video image camera 5 and by the image detection camera 6.
• FIG. 3 presents a block diagram representing an imaging system 300 according to a third embodiment of the invention.
• the camera unit 3 has a common objective 31 for both a video image camera element 36 and an image detection camera element 37.
• light arriving from the scene passes through the objective 31 and travels as a light beam 32 to a beam splitter 33.
• the beam splitter 33 divides the light beam 32 into two beams, first sub-beam 34 and a second sub-beam 35.
• the first sub-beam comprises visible light from the scene and is directed to the video image camera element 36 that functionally corresponds to the video image camera 5.
• the second sub-beam 35 carries light to the image detection camera element 37.
• the beam splitter may be wave length selective i.e. the first and second beams may have light of different frequency ranges from a common scene. In this case, the second sub- beam 35 may comprise light that is invisible to human eye.
• the image detection camera element 37 may be configured to operate at a frame rate that is a multiple of the frame rate of the video image camera element 36. Alternatively, if the image detection camera element 37 is configured to operate in a common frequency area with the video image camera element 36, the image detection camera element 37 may be configured to operate at a common frame rate with the video image camera element 36 in a manner similar to that explained for the variant of the second embodiment.
• the lamp 2 is configured to radiate e.g. 30 Hz pulsed infrared light behind the advertisement so that the radiation permeates the entire area of the advertisement.
• the camera unit 3 and the lamp 2 may be synchronized so that the lamp is off and on when alternating video images are captured by the video image camera 5 as subsequent video image frames.
• the video image frames captured with the lamp 2 switched off are here denoted as odd frames and illustrated in Fig. 4 while the video image frames captured with the lamp 2 switched on are denoted as even frames and illustrated in Fig. 5.
• the marking surface appears in high-speed detection (whether in visible or invisible frequency area) alike the obstacle 4.
• Figs. 4 and 5 present consecutive images as seen by the image detection camera (in a simplified manner).
• the term image detection camera refers to any equipment for this purpose.
• the image detection camera may be a high-speed video image camera 5 shown in Fig. 1 as well as a separate image detection camera shown in Fig. 2, an image detection camera element 37 shown in Fig. 3.
• the marking surface is identified by means of a pulsating visible light. In this case, ambient and constantly visible light is reflected by the obstacle 4 which thus appears in high-speed image detection as an unchanged shape 44 in Figs. 4 and 5. It is understood that any other details in the image whether in front of or around the marking surface 1 would appear unchanged by the alternating appearance of the marking surface 1.
• the marking surface 1 appears in Fig. 4 rather similarly to the obstacle 4 as a dark rectangle 41 that is merged with the shape 41 (of the obstacle 4) when the lamp 2 is off. Notice that in this document dark areas are represented by dotted areas. When the lamp 2 is on, the marking surface appears as a light radiating source and is represented as a light rectangle 51.
• the processing of video images may be based on common video processing techniques. For instance, a multichannel color image can be converted into a single channel intensity image if needed. There are also well known techniques for converting a multichannel color image into a single channel intensity image, hence these techniques are not described in detail here. The technique for conversion may be chosen such that it maximizes the marking surface contrast in consecutive images.
• an absolute intensity difference of two consecutive images is computed.
• those areas where illumination has changed have a high value (light) and other areas have low value (dark).
• a new image 61 results that practically corresponds to visible parts of intentionally created marking surfaces.
• the new image 61 clearly illustrates a rectangular area 62 that corresponds to the visible parts of the marking surface 1.
• An image of alternative content 71 such as an advertisement image shown in FIG. 7 is stored in the controller 1000 or in a separate computer.
• a rectangular form of the marking surface image 62 is identified in the new image 61 of FIG. 6 and the location of the alternative content is determined by using pattern recognition algorithms. This determination may employ particular patters in the new image 61 as exemplified by Fig. 10 and further described with reference thereto.
• the alternative content is scaled and positioned according to the identified rectangle. Using the difference image presented in FIG. 6, the scaled and positioned alternative content is masked so that only visible parts of the marking surface 1 contain respective parts of the scaled and positioned alternative content. Resulting image is presented in FIG. 8. The image of Fig. 8 is then added into corresponding original odd image shown in FIG.
• the lamp 2 only has to illuminate the marking surface during every second period in which image detection is carried out.
• the illumination may only cover a part of the duration of the illuminated image detections, if all or substantial portion of the image is captured simultaneously such that reduced illumination time will not distort the form of the detected image.
• the effect of the illumination is reduced but may yet suffice.
• the illumination may be thus reduced under such an ambient light in which a reduced illumination suffices to reliably identify the marking surface.
• the timing of the illumination by the lamp 2 has to be synchronized with the capturing periods of the high-speed detection.
• the lamp 2 is configured to illuminate the marking surface periodically such that the lamp is on and off alternating cycles at the image detection frame rate (e.g. on 1/60 seconds and then off 1/60 seconds) in a recurring manner.
• the lamp 2 illuminates subsequent frames in differing manner unless the illumination cycle starts after one half of the highspeed image capturing period has lapsed.
• sufficient synchronization may be achieved e.g. by recording video image from a scene with a marking surface and automatically detecting whether the rectangle of Fig. 6 is identifiable. If not, the recording is time-shifted by a suitable time e.g. by one half of the exposure period of the image detection.
• the illumination of the lamp 2 lasts one image detection cycle and starts exactly in the middle of a image capture period of one video image frame (and thus ends exactly in the middle of the following image detection capture period).
• the rectangle visible in Fig. 6 will not appear.
• the timing of the video image camera and image detection camera is shifted by one half of the image detection cycle, say by 1/200 s i.e. by 5 ms.
• the illumination by lamp 2 starts simultaneously with image capture period of one video detection frame and stops simultaneously with the end of that image detection frame or with the start of the subsequent image detection frame that is then the frame without illumination.
• the differential image shown in Fig. 6 has a clearly identifiable rectangle.
• it is easy to detect which one of the two image detection frames is the one without illumination by the lamp 2 in order to suitably determine which of the image detection frames shall be used as basis for combining the alternative content 71.
• the lamp 2 and the camera are initially temporally aligned such that the illumination period of the lamp 2 starts one quarter after the start of an image detection frame, and the illumination period lasts for one image detection cycle (e.g. 1/60 s).
• the rectangle in Fig. 6 is identifiable, but not necessarily sufficiently well.
• the image detection and camera operation may then be delayed by one half of the detection image capture period.
• the illuminated part will still cover one quarter of the image capture period in one frame and three quarters in adjacent image detection frames.
• the rectangle is not clearer to identify.
• a second similar delay is caused to the operation of the video image camera and to the image detection camera.
• This second delay shifts the image detection so that the illumination of the marking surface starts one quarter of the detection period before one image detection frame starts.
• the illumination should completely cover the image capture period in one frame but not in its adjacent frames and the rectangle in Fig. 6 should be clearly identifiable.
• the timing of the high-speed image detection may also be automatically adjusted in real time. Additional high-speed image detection equipment may be provided to test different time offsets. In such an embodiment, it may be possible that different marking surfaces are illuminated with independent lamps 2 at random time offsets. It may even be possible to distinguish from a common scene different marking surfaces based on their different time offsets without necessarily synchronizing the operation of the different lamps, based on likely timing difference incurred by random time offsets.
• a synchronization circuitry for synchronizing the lamp 2 or a plurality of lamps 2 with the camera and highspeed image detection.
• image detection may operate at a rate that is a multiple of the video image frame rate. It should be appreciated that a high frame rate in the image detection may be advantageous for accurate locating of visible parts of the marking surface 1 when obstructed by fast-moving obstacles.
• the image detection employs image detection frames captured at equal or lower rate than that of video image capture. If visible light is used in the image detection, the even and odd frames may refer to different successive video image frames. For instance, the appearance of the marking surface 1 may be alternated between each subsequent video image frame. In this case, the appearance would effectively alternate on frequency of 15 Hz and the marking surface 1 might appear as blinking in the production video.
• the image detection may use a frame rate such as 1/3 of the video image frame rate. In this case, some image detection frames coincide with video frames while some other image detection frames do not coincide with video frames. Hence, it is possible to maintain the appearance of the marking surface 1 stable in these coinciding frames so that the production video should not appear blinking to viewers.
• the image detection may take place in any frequency.
• FIG. 10 presents an example of a recognition pattern 101 on the marking surface 1.
• the pattern 101 comprises two patterns drawn with and without dotting in FIG. 10.
• the two patterns may be altematingly illuminated in opposite phases or one of the patterns may be intermittently illuminated while the remainder is not. When one pattern is illuminated, the remaining one is not.
• This allows a good detection of not only the outer edges 11 , but also the inner edges 12 to 15.
• the inner edges are aligned so that they all lead to corner points of the rectangular object. If large parts of the object are occluded, the corners of the marking surface can be estimated by extrapolating known edge parts. If for example only quarter of the marking surface is visible, i.e.
• the part between points 16 and 17, extrapolating lines 14 and 15 together with the known outer edge between points 16 and 17 defines the location of the left side corner points.
• Right side corner points are also found based on symmetry of the pattern 101 so that the location of the whole marking surface becomes known.
• the angles of the inner edges are selected so that they divide the opposite outer edge into known fractions. For example, the inner edge 12 divides the bottom edge in parts ⁇ A and 3 A, and the inner edges 13 and 15 divide opposite edges in half. Since perspective distortion appears as linear transformation, extrapolation of edges into corner points and known points in outer edges of rectangular object also stays valid.
• Fig. 11 shows a controller unit 1000 for detecting a marking surface from signals received from a camera unit.
• the controller unit 1000 may be configured to operate as the controller 3.
• the controller unit 1000 may be based on personal computer architecture. For instance, the controller unit 1000 may be based on a laptop computer or on a desktop computer, or on a networked server such as an internet server.
• the controller unit 1000 comprises a processor 110 such as a central processing unit or a digital signal processor, and functionally connected to the processor a memory 120, an input/output element 150, a user interface 160, a power supply 170, and an optional synchronizing circuitry 180.
• the memory 120 comprises a work memory 130 and a persistent memory 140 comprising an operating system 141 and software 142.
• the operating system 141 provides basic functionalities to the controller unit 1000 while the software 142 comprises particular instructions for controlling the processor 110, when executed by the processor, to different operations needed to implement particular embodiments of the invention.
• the memory 120 need not be a singular unit.
• the persistent memory 140 and the work memory 130 need not be contained in a common module.
• either or both of the persistent memory 140 and work memory 130 may comprise two or more different units which may also differ from one another by their types.
• the controller unit 1000 need not always contain the software 142, but may be delivered without the software and configured to receive the software from an external source such as a data network or transferrable memory medium.
• the controller unit comprises the software 142 on some memory medium when operating according to an embodiment of the invention.
• the input/output element 150 comprises equipment for exchanging information with other elements such as the lamp 2 and the camera unit 3.
• the input/output element 150 may comprise, for instance, a network interface device, a digital / analog converter, a digital input bank, or any combination of thereof.
• the user interface 160 may comprise information presentation and/or input devices in order to enable a user to control and/or monitor the operation of the controller unit 1000.
• the power supply may comprise a battery, a mains transformer, a direct current transformer, a direct current connector (e.g. a Universal Serial Bus, USB, or a Firewire connector), or any combination thereof.
• the function of the power supply is to power the circuitries of the controller unit 1000.
• the synchronizing circuitry 180 may be provided as a separate unit that may or may not be functionally connected to other parts of the controller 1000.
• the synchronizing unit may comprise a comparator (not shown) configured to compare timing of the lamp 2 and of the camera unit 3 and to responsively control the timing of either or both of the lamp 2 and of the camera 3.
• the controller may further comprise a separate processor or be configured to use its processor 110 or other circuitry to modify the captured video footage in the visible areas of the marking surface. Such modification may involve overlaying or superimposing alternative content on the visible parts, highlighting the visible parts (e.g. by increasing luminosity.alternating luminosity and/or modifying different chrominance components for the visible parts), or any image manipulation known in the field of video processing.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Business, Economics & Management (AREA)
• Marketing (AREA)
• Studio Devices (AREA)

Applications Claiming Priority (4)

Publications (1)

Family

ID=40404568

Family Applications (1)

Country Status (2)

Cited By (7)

Citations (6)

• 2009
  • 2009-02-09 FI FI20090040A patent/FI20090040A0/sv not_active Application Discontinuation
  • 2009-10-27 WO PCT/FI2009/000093 patent/WO2010049578A1/en active Application Filing

Patent Citations (6)

Also Published As

Similar Documents

Legal Events