Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/222—Studio circuitry; Studio devices; Studio equipment
  • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
  • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
  • H04N5/275—Generation of keying signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/222—Studio circuitry; Studio devices; Studio equipment
  • H04N5/2224—Studio circuitry; Studio devices; Studio equipment related to virtual studio applications
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/222—Studio circuitry; Studio devices; Studio equipment
  • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
  • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
  • H04N5/2723—Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement

Definitions

• the present invention relates to modifying the content of a television image by inserting additional graphics and substitutive content into specific areas of the television image.
• Inserting substitutive content into a television image requires naturally information about the location of the area to be modified in the image.
• This information can be generated, for example, by using a marking surface within the vicinity of the target to be substituted in the television image, the marking surface being identifiable on the basis of radiation from it differing from the other radiation from the area shot by the camera. By detecting this differing radiation by specific detectors, the coordinates of the target to be modified in the television image can be determined.
• This kind of marking of the target to be modified is described in more detail in the applicant's earlier patent application WO 01/58147.
• the coordinates of the target to be modified are known it is possible, for example, to modify the target to be modified separately for different spectator groups formed by using certain grouping criteria, e.g. nationality.
• WO 01/58147 discloses generating a mask image determining the areas of the tar- get area really visible in the image.
• this information can be transmitted together with the actual television signal for enabling modification of the target to be modified in the television image at a later stage of the signal transmission chain.
• this approach less signal transmission capacity is required in comparison to the case of modifying the signal already at the beginning of the signal transmission chain and transmitting all modified signals together.
• the principle of this approach is disclosed in the applicants' earlier patent application WO 03/021957.
• the purpose of the present invention is to provide a method enabling insertion of substitutive content to a certain portion of a television image at a later stage of the image signal transmission chain while saving unchanged in the final television image the desired appearance of graphics added at an early stage of the signal transmission chain, the method preferably requiring as low data transmission capacity as possible.
• the method of the present invention for modifying the content of a target area in a television image comprises generating an original image signal by means of a camera, processing the original image signal by adding graphics into the image, determining the target area in the image, transmitting further the processed image signal and the target area determining data, and inserting, at a later stage of the signal transmission chain, substitutive content to the target area according to the target area determining data.
• the target area to be modified can be, for example, an advertising billboard visible in the image captured by the camera.
• Said modification can comprise e.g. adjusting the content in the area of the bill- board in the image finally visible for the spectators separately for different spectator groups.
• the graphics can comprise any additive graphical content, e.g. different kinds of figures, letters or the like.
• marking surfaces described in WO 01/58147 can be used.
• the possible mask area i.e. an area within the target area which is not visible in the television image and to where thus no sub- stitutive content should be inserted, can be determined.
• a later stage of the signal transmission chain means any stage after the initial trans- mission of the processed image signal and the target area determining data.
• the substitutive content insertion can be carried out separately in each receiving country.
• Generating and transmitting further the processed image signal and the target area determining data can, in their turn, be performed e.g. already at the scene where the original image signal is generated.
• the graphics is added into the television image according to a predetermined graphics percentage of coverage; the method further comprises determining the shared area of the target area and the additional graphics area in the television image, and transmitting further, together with the processed image signal and the target area determining data, the shared area determining data and data proportional to said graphics percentage of coverage; and, to the shared area, i.e to the intersection of the target area and the additional graphics, the substitutive content is inserted according to the residual percentage of coverage not covered by the added graphics .
• the percentage of coverage means here the proportion of the additive content at each point of the image. In other words, the percentage of coverage determines which proportion of the image at each pixel or at a group of adjacent pixels consists of the added graphics, the rest consisting of the earlier content at the pixel at issue.
• the shared area of the target area and the additional graphics in the television im- age is the area within the target area including added graphics. Transmitting the shared area determining data and the data proportional to the graphics percentage of coverage can be performed, for example, by transmitting the original graphics and a control sig- nal having been used in adjusting the graphics percentage of coverage.
• the data proportional to the graphics percentage of coverage can be presented e.g. as the residual percentage not covered by the graphics.
• Inserting the substitutive content to the shared area according to the residual percentage of coverage not covered by the added graphics means that the graphics percentage of coverage is maintained. For ex- ample, if the predetermined graphics percentage of coverage is 80% there is a percentage of coverage of 20% to be covered by the original and the substitutive contents in the shared area. It is possible that also the desired percentage of coverage of the substitutive content is below 100 %. It means that the residual percentage of coverage in the shared area not covered by the added graphics is made to consist of both the original image content and the substitutive content. This requires transmitting the original graphics together with the processed image signal. The desired percentage of coverage of the substitutive content can be determined already at the transmitting stage and sent further together with the other data.
• the key feature of the present invention is the possibility to add partially transparent graphics also to the target area to be modified, though maintaining the predetermined graphics percentage of coverage also after insertion of the substitutive content to the target area. This is a great improvement to the prior art techniques restricted to insertions with a full coverage, i.e. without any transparency.
• the graphics is added to the shared area with a full coverage independently from the predetermined graphics percentage of coverage. By proceeding this way, transmitting the original graphics is not required. This is a clear advantage from the data transmission capacity point of view. Replacing the content of the target area of an image processed this way by a substitutive content according to said residual percentage of coverage always produces a result image where the shared area comprises the graph- ics with the predetermined percentage of coverage, the rest of the coverage being composed of the substitutive content .
• the target area determining data, the shared area determining data and the data proportional to the graphics percentage of coverage are transmitted in an auxiliary image signal, the image content of which being arranged according to said data.
• the image content is arranged according to the geometries and locations of the target area and the shared area and contains also information about the graphics percentage of coverage.
• An auxiliary image signal is a form of data which is straightforward and easy to process in the signal processing systems of the television signal transmission chain. Also information concerning the possible mask area as well as informa- tion according to the intended substitutive content percentage of coverage is preferably carried along the auxiliary image signal.
• the auxiliary image signal can contain, for example, a one-color image, the intensity of which in the different areas, i.e. in the target area, the shared area, and in the possible mask area, being adjusted according to the intended percentages of coverage. It is also preferable that the coordinates of the corners or some other reference points of the target area in the television image are transmitted together with the auxiliary image signal.
• This kind of auxiliary image signal carrying all information needed in the later insertion of the substitutive content makes the insertion itself a very simple procedure requiring no highly sophisticated data handling systems. Naturally, it is also possible that the substitutive content percentage of coverage is determined not until at a later stage of the signal transmission chain.
• the image area of the auxiliary image signal is preferably limited to the target area only in order to further decrease the required data transmitting capacity.
• the information about the location of this kind of limited-size target area in the television is transmitted along the auxiliary image signal as image coordinates of selected points of the target area .
• Figure 1 illustrates the principle of generating the original television image and determining the location of the target area in the image to be modified.
• Figure 2 represents, as a block diagram, the steps of the embodiment at the transmitting end and the receiving end of the signal transmission chain, respectively .
• FIG. 1 illustrates the starting point of the exemplary embodiment of the present invention.
• a television camera 1 shoots a view including an advertising billboard 2, the content of which in the final television image is desired to be modified. Between the camera and the billboard there is a person 3 making a part of the billboard invisible to the camera. For marking the area of the billboard in the image captured by the camera and thus for giving information needed for determining the area of it in the television image, i.e. the target area to be modified, there is a radiation source 4 behind the billboard emitting infrared radiation 5 to the whole area of the billboard.
• the radiation could be any type of radiation sufficiently differing from the other radiation at the venue.
• the billboard is arranged to re-direct this radiation to the front side of the billboard.
• the billboard area appears as a planar infrared source.
• the infrared radiation from the bill- board area is detected by a special detector 6 attached to the camera . From the data captured by the detector, when the relationship between the coordinates of the detector and camera images is known, it is possible to determine the target area in the tele- vision image. Further, it is possible to determine the mask formed by the person hiding part of the billboard.
• the contents of the mask image also comprises information about the intended percentage of coverage of the substitutive content, which in this case is 100 % except the mask area, for which it natu- rally is 0 %.
• the mask image is, in this embodiment, a single-color image, the intensities of its different areas being selected to be proportional to those percentages of coverage.
• the original television image 7 captured by the camera and supplied in an original television signal PGMl to the transmitting stage 8 of the signal transmitting chain includes the billboard 2, part of it being hidden behind the person 3 having been stood between the billboard and the cam- era at the original venue.
• a graphics control signal GRAPEMKEY To the transmitting stage is supplied also a graphics control signal GRAPEMKEY.
• this graphics control signal determines the graphics 9, also supplied to the transmitting stage, to be added to the right side of the image area with a percentage of coverage of 70%.
• the third signal supplied to the transmitting stage is a first auxiliary image signal AUXl containing the mask image 10 described above.
• the image area of the mask image is limited to the target area only in order to save the data transmission capacity.
• the target area is divided into two sub-areas 11a, lib for enabling e.g. insertion of two different advertisements as the substitutive content to the target area.
• the identification information of the sub-areas are also carried along the first auxiliary image signal AUXl.
• the mask image 10, the graphics 9 and the graphic control signal GRAPH_KEY are combined to determine the shared area 13 where the added graphics area overlaps with the target area 11.
• the graphics when adding the graphics to the image according to the actual graphics control signal, in this shared area the graphics is set to have a percentage of coverage of 100%, independently from the percentage determined by the graphics control signal GRAF_KEY.
• two signals are transmitted further: a processed televi- sion signal PGM2 including the graphics 9 in the image
• the desired substitutive content 15a, 15b is inserted into the image 7 of television signal PGM2 according to the information carried along the second auxiliary image signal AUX2. This means that the substitutive content is inserted into the target area 11 determined by the mask image 10 and said target area coordinates.
• the insertion is carried out with percentages of coverage determined by the intensities of the different areas of the image of the second auxiliary image signal.
• the substitutive content is added with a full percentage of coverage.
• no substitutive content is inserted to the mask area 12.
• the earlier mentioned shared area percentage of coverage of 30% is used for the shared area 13 fully covered by the added graphics in the processed television signal PGM2.
• the shared area 13 of the televi- sion image 7 of the third television signal PGM3 produced by the receiving stage 14 consists of the added graphics with a percentage of coverage of 70 % and the substitutive content with a percentage of coverage of 30 %.
• the content of the final televi- sion image in the output signal PGM3 is exactly what it would be if the substitutive content would have been at the site of the advertising billboard at the original venue .
• the borders between the different television image areas described above are preferably softened by utilization of a transition zone within which the image content changes from that of one area to that of the other.
• the original image content is set to be replaced by the substitutive one within a finite-width transition zone. Abrupt change without any softening could cause an unrealistic impression.
• the width of the transition zone is preferably adjusted according to the movement of the objects in the television image so that with rapidly moving objects it is wider than with station- ary ones.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Business, Economics & Management (AREA)
• Marketing (AREA)
• Controls And Circuits For Display Device (AREA)
• Studio Circuits (AREA)
• Processing Or Creating Images (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
• Image Processing (AREA)

Priority Applications (7)

Applications Claiming Priority (1)

Publications (1)

Family

ID=40755260

Family Applications (1)

Country Status (7)

Cited By (3)

Families Citing this family (5)

Citations (4)

Family Cites Families (3)

• 2007
  • 2007-12-13 CN CN2007801019376A patent/CN101981912B/zh not_active Expired - Fee Related
  • 2007-12-13 WO PCT/FI2007/050683 patent/WO2009074710A1/en not_active Ceased
  • 2007-12-13 CA CA2709003A patent/CA2709003A1/en not_active Abandoned
  • 2007-12-13 EP EP07858334.1A patent/EP2232845B1/en active Active
  • 2007-12-13 US US12/808,065 patent/US20100289960A1/en not_active Abandoned
  • 2007-12-13 JP JP2010537477A patent/JP5101707B2/ja not_active Expired - Fee Related
  • 2007-12-13 KR KR1020107015491A patent/KR20100133356A/ko not_active Withdrawn

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events