CN105284122A - Device and method for inserting advertisement by using frame clustering - Google Patents

Device and method for inserting advertisement by using frame clustering Download PDF

Info

Publication number
CN105284122A
CN105284122A CN201480033739.0A CN201480033739A CN105284122A CN 105284122 A CN105284122 A CN 105284122A CN 201480033739 A CN201480033739 A CN 201480033739A CN 105284122 A CN105284122 A CN 105284122A
Authority
CN
China
Prior art keywords
region
frame
pixel
advertisement
virtual indirect
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201480033739.0A
Other languages
Chinese (zh)
Other versions
CN105284122B (en
Inventor
李秀瑸
申亨哲
韩柱贤
全柄玑
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
11th Street Co., Ltd.
SK Telecom Co Ltd
Original Assignee
SK Telecom Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR20140009195A external-priority patent/KR101503029B1/en
Priority claimed from KR1020140009196A external-priority patent/KR101561083B1/en
Priority claimed from KR1020140009193A external-priority patent/KR101573482B1/en
Priority claimed from KR1020140009194A external-priority patent/KR101564313B1/en
Priority claimed from KR1020140013842A external-priority patent/KR102135671B1/en
Application filed by SK Telecom Co Ltd filed Critical SK Telecom Co Ltd
Publication of CN105284122A publication Critical patent/CN105284122A/en
Application granted granted Critical
Publication of CN105284122B publication Critical patent/CN105284122B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/812—Monomedia components thereof involving advertisement data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/036—Insert-editing
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring

Abstract

The present invention relates to a device and a method for inserting an advertisement by using frame clustering. According to the present invention, when an advertisement insertion region is set in at least one of the frames of a video, clustering is performed by searching for the frames including the advertisement insertion region, into which an advertisement is to be inserted, and the advertisement is consistently inserted into all of the frames for which clustering has been performed, thereby enabling a more natural insertion of advertisements.

Description

For the apparatus and method by using frame cluster to carry out insertion of advertising
Technical field
The present invention relates to a kind of apparatus and method for using frame cluster (frameclustering) to carry out insertion of advertising, and more specifically, relate to such apparatus and method for using frame cluster to carry out insertion of advertising, its search package is inserted target frame containing the advertisement that region is inserted in advertisement and target frame is inserted in described advertisement and carries out cluster, then according to unified mode for by the frame insertion of advertising of cluster.
Application claims submit on January 24th, 2014 korean patent application No.10-2014-0009193, the korean patent application No.10-2014-0009194 submitted on January 24th, 2014, the korean patent application No.10-2014-0009195 submitted on January 24th, 2014, the korean patent application No.10-2014-0009196 submitted on January 24th, 2014 and the rights and interests of the applying date of korean patent application No.10-2014-0013842 submitted on February 6th, 2014, be incorporated into herein the content intact of these korean patent applications.
Background technology
Along with the development of Digital Signal Processing, the virtual ads be inserted into by virtual image in video is popularized, and himself is asserted the effective medium of advertisement by indirect advertising.Therefore, people are for be combined with indirect advertising by virtual ads and interest advertising image being inserted into naturally the virtual indirect advertising in the video previously generated just promptly increases.
Different from the situation of virtual ads, when virtual indirect advertising, naturally being inserted into by advertising image in existing video is prior problem, this is because when user identify easily artificially insertion of advertising image time, there will be the interference to content.Particularly when advertising image does not have according to unified mode to be current in whole video, great inconvenience can be caused to the user watching video, make the dispersion attention etc. of user.
Traditional advertisement is inserted technology and is only used for the technology of replacing specific pattern in continuous print image, and and unexposed and hint be used for for the structure of all frames according to unified mode insertion of advertising.
Therefore, in order to for all frames according to unified mode insertion of advertising, new technology in the urgent need to such: the advertisement that this technology search package inserts region containing advertisement is inserted target frame and described advertisement insertion target frame is carried out cluster, then clustering factor (that is, the result compared between region is inserted in the advertisement insertion region in reference frame and the advertisement in other frame) is used to carry out insertion of advertising according to unified mode.
In addition, new technology for inserting virtual indirect advertising in the urgent need to such: this technology determines the candidate region by being inserted into virtual indirect advertising, measure the exposure levels (exposurelevel) of candidate region, and measured exposure levels is supplied to user, therefore provide guidance information, make user that virtual indirect advertising can be selected while the advertising results identifying each candidate region more intuitively to insert region.
In addition, new virtual indirect advertising service technology in the urgent need to such: this virtual indirect advertising service technology can make restoring area minimize when directly indicating the mark at spot for photography place, determine to use these marks insert the region of virtual indirect advertising and eliminate these marks, and can reparation be performed.
In addition, the new virtual indirect advertising service technology in the urgent need to such: this virtual indirect advertising service technology can make restoring area minimize when replacing existing advertisement and being inserted in video image content by virtual indirect advertising, and can perform reparation.
Summary of the invention
Technical problem
An object of the present invention is search package to insert target frame containing the advertisement that region is inserted in advertisement and target frame is inserted in described advertisement and carry out cluster, then according to unified mode for by the frame insertion of advertising of cluster, therefore make advertisement more naturally to be inserted.
In addition, an object of the present invention is to use clustering factor (namely, the result compared between region is inserted in advertisement insertion region in reference frame and the advertisement in other frame) carry out insertion of advertising according to unified mode, therefore just advertisement automatically can be inserted in multiple frame by means of only advertisement being inserted in single frame.
In addition, an object of the present invention is to extract from frame only comprise advertisement insert region part and extracted part is carried out cluster, therefore realize cluster more rapidly.
In addition, an object of the present invention is the candidate region determined being inserted into virtual indirect advertising, measure the exposure levels of described candidate region, and provide measured exposure levels to user, therefore provide guidance information, make user that virtual indirect advertising can be selected while the advertising results identifying each candidate region more intuitively to insert region.
In addition, an object of the present invention is only when the level of the candidate region being inserted into virtual indirect advertising is exceeded the level being equal to or greater than predetermined value, corresponding candidate region relevant information is provided to user, therefore provide guidance information, make it possible to more promptly select virtual indirect advertising to insert region.
In addition, an object of the present invention is to provide by based on various measurement standard (size that such as virtual indirect advertising is exposed, time, angle and position to user, the speed of virtual indirect advertising movement in screen, the size in the region covered by another object, the frequency that virtual indirect advertising is covered by another object) assign weight and the total exposure level calculated, therefore provide guidance information, make it possible to select virtual indirect advertising to insert region efficiently based on the advertising results in corresponding candidate region.
In addition, an object of the present invention is to use and be inserted into mark in video image content to insert virtual indirect advertising, and reparation is performed to marked region, therefore perform process to realize the coordination with surrounding image.
In addition, an object of the present invention is only to perform reparation to not overlapping with the insertion region of the virtual indirect advertising region of marked region, therefore reduce the running cost needed for repairing.
In addition, an object of the present invention is to reduce the running cost needed for repairing, therefore make the ad margins of virtual indirect advertising maximize.
In addition, an object of the present invention is to replace existing advertisement and virtual indirect advertising is inserted in video image content, and reparation is performed to existing advertising area, therefore perform process to realize the coordination with surrounding image.
In addition, an object of the present invention is automatically to calculate the value of the borderline region pixel of the virtual indirect advertising be inserted in video image content, and perform the boundary line that process makes to insert virtual indirect advertising and can coordinate with around image.
In addition, an object of the present invention is based on the position of the borderline region pixel of the virtual indirect advertising that will be inserted in video image content and foreground pixel and the isolated distance of background pixel, after foreground pixel value and background pixel value are mixed, the value of computation bound area pixel, therefore performing the boundary line that process makes to insert virtual indirect advertising can coordinate with around image.
In addition, an object of the present invention is the value carrying out automatically computation bound area pixel with reference to the pixel value of the frame before the present frame inserting virtual indirect advertising and after it, therefore perform process make to insert virtual indirect advertising boundary line can the playback of video and around image coordinate.
Technical solution
To achieve these goals, the invention provides a kind of device for using frame cluster to carry out insertion of advertising, this device comprises: frame search unit, it is constructed to when advertisement insertion region is arranged at least one in the frame of video, and target frame is inserted in the advertisement that search package inserts region containing advertisement; Cluster cell, it is constructed to insert target frame to described advertisement and divides into groups, described advertisement is selected to insert any one in target frame as with reference to frame, region is inserted with the advertisement advertisement of inserting in target frame except described reference frame in advertisement insertion region in described reference frame compare, calculate with the difference of described reference frame as clustering factor, and perform cluster; And advertisement plug-in unit, it is constructed to when during insertion of advertising, being inserted described advertisement for all by the frame of cluster according to unified mode in by least one in the described frame of cluster.
In the case, the step inserted according to unified mode can comprise the following steps: carried out insertion of advertising by being applied to by clustering factor by each in the frame of cluster.
In the case, cluster cell can comprise: scene structure analytic unit, and its scene structure being constructed to insert advertisement region is analyzed; Camera motion analytic unit, its camera motion being constructed to insert advertisement region is analyzed; And clustering factor computing unit, it is constructed to use scenes structure and camera motion to calculate clustering factor.
In the case, clustering factor computing unit can be selected to insert advertisement and insert the frame in region as reference frame, region can be inserted with reference to the advertisement insertion region in frame with the advertisement advertisement of inserting in target frame except this reference frame to compare, and can calculate with the difference of this reference frame as clustering factor.
In the case, clustering factor computing unit can insert scene structure in target frame with reference to the scene structure in frame and camera motion and the advertisement except this reference frame and camera motion compares.
In the case, during scene structure analytic unit can obtain below item any one or more: the position in region, size, rotation and visual angle are inserted in advertisement.
In the case, camera motion analytic unit can obtain advertisement and insert the focus in region and any one in shake or more.
In the case, clustering factor computing unit can be determined for by the mass motion between the successive frame of the frame of cluster, and can calculate clustering factor based on the determination result of described mass motion.
In the case, cluster cell can only insert target frame to extract from corresponding advertisement and comprise the part that region is inserted in advertisement, and carries out cluster to extracted part.
In the case, cluster cell can carry out cluster to continuous print frame from reference frame.
In the case, frame search unit can determine the mass motion of successive frame, and can carry out search advertisements insertion target frame based on the determination result of described mass motion.
In the case, frame search unit can use the determination result of mass motion any one in former frame and a rear frame next or more middle prediction advertisement to insert the position in region, and can search advertisements insertion target frame
In the case, advertisement plug-in unit can insert region to advertisement and perform reparation.
In the case, reference frame can be when show intuitively to user advertisement insert in target frame any one or more time the frame selected by user.
In the case, reference frame can be the frame with the widest advertisement insertion region that advertisement inserts in target frame.
In addition, the invention provides a kind of method using frame cluster to carry out insertion of advertising, the method comprises the following steps: when advertisement insertion region is arranged at least one in the frame of video, target frame is inserted in the advertisement that search package inserts region containing advertisement; Insert target frame to described advertisement to divide into groups, described advertisement is selected to insert any one in target frame as with reference to frame, region is inserted with the advertisement advertisement of inserting in target frame except described reference frame in advertisement insertion region in described reference frame compare, calculate with the difference of described reference frame as clustering factor, and perform cluster; And when during insertion of advertising, being inserted described advertisement by the frame of cluster according to unified mode for all in by least one in the described frame of cluster.
In the case, insert can be carried out insertion of advertising by being applied to by clustering factor by each in the frame of cluster according to unified mode.
In the case, the step performing cluster can comprise the following steps: scene structure advertisement being inserted to region is analyzed; Camera motion advertisement being inserted to region is analyzed; And use scenes structure and camera motion calculate clustering factor.
In the case, the step calculating clustering factor can comprise the following steps: select to insert advertisement and insert the frame in region as reference frame, region is inserted with the advertisement advertisement of inserting in target frame except this reference frame in advertisement insertion region in this reference frame compare, and calculate with the difference of this reference frame as clustering factor.
In the case, calculate the step of clustering factor can comprise the following steps: insert scene structure in target frame with reference to the scene structure in frame and camera motion and the advertisement except this reference frame and camera motion compares.
In addition, to achieve these goals, the invention provides a kind of device of the insertion region guidance information for being provided for inserting virtual indirect advertising, this device comprises: candidate region determining unit, and it is constructed to the candidate region determined being inserted into virtual indirect advertising in video image content; Advertisement plug-in unit, it is constructed to process described virtual indirect advertising, and is inserted in described candidate region by treated virtual indirect advertising; Exposure measuring unit, its exposure characteristics being constructed to expose in described video image content based on described virtual indirect advertising measures the exposure levels of described virtual indirect advertising; And information provider unit, the insertion region guidance information of its described exposure levels be constructed to comprising described virtual indirect advertising is supplied to user.
In the case, information provider unit can determine whether the described exposure levels of described virtual indirect advertising exceedes Pre-set reference exposure levels, and if the described exposure levels of described virtual indirect advertising exceedes described Pre-set reference exposure levels, then information provider unit can to the described insertion region guidance information of described user's providing package containing the described exposure levels of described virtual indirect advertising.
In the case, during information provider unit can show below item any one or more: the candidate region inserting described virtual indirect advertising and the candidate region existed before inserting described virtual indirect advertising.
In the case, device for being provided for the insertion region guidance information inserting virtual indirect advertising can also comprise advertising expenditure computing unit, this advertising expenditure computing unit is constructed to the advertising expenditure calculating virtual indirect advertising based on exposure levels, and information provider unit can show described advertising expenditure to user.
In the case, advertisement plug-in unit can comprise: advertisement selecting unit, and it is constructed to select to be inserted into the virtual indirect advertising in candidate region; Advertisement processing unit, it is constructed to process described virtual indirect advertising based on preliminary treatment characteristic, and described virtual indirect advertising can be inserted in candidate region; And process advertisement plug-in unit, it is constructed to treated virtual indirect advertising to be inserted in described candidate region.
In the case, the real-time characteristic measured in real time while can being included in and exposing in virtual indirect advertising video image content of exposure characteristics and preliminary treatment characteristic.
In the case, exposure measuring unit can comprise dimension measuring unit, and this dimension measuring unit is constructed to proportionally measure exposure levels with the ratio of the size of the exposure area exposed in video image content of virtual indirect advertising and the size of whole screen.
In the case, exposure measuring unit can comprise time measuring unit, and the time that this time measuring unit is constructed to expose in video image content with virtual indirect advertising proportionally measures exposure levels.
In the case, exposure measuring unit can comprise deformation measurement unit, and the difference between this angle that deformation measurement unit is constructed to and virtual indirect advertising exposes in video image content and Pre-set reference angle measures exposure levels inversely.
In the case, exposure measuring unit can comprise support size measuring unit, and the size that this support size measuring unit is constructed to the part covered by another object with virtual indirect advertising while virtual indirect advertising just exposes in video image content measures exposure levels inversely.
In the case, exposure measuring unit can comprise frequency measurement unit, and the frequency that this frequency measurement unit is constructed to be covered by another object with virtual indirect advertising while virtual indirect advertising just exposes in video image content proportionally measures exposure levels.
In the case, exposure measuring unit can comprise support size measuring unit, and this support size measuring unit is constructed to measure the exposure levels of virtual indirect advertising near screen center.
In the case, exposure measuring unit can comprise velocity measuring unit, and this velocity measuring unit is constructed to proportionally measure exposure levels with virtual indirect advertising speed of movement in screen while virtual indirect advertising just exposes in video image content.
In the case, exposure measuring unit can calculate total exposure level by collecting based on the exposure levels measured by exposure characteristics, and advertising expenditure computing unit proportionally can calculate advertising expenditure with total exposure level.
In addition, the invention provides a kind of method being provided for inserting the insertion region guidance information of virtual indirect advertising, the method comprises the following steps: in video image content, determine the candidate region by being inserted into virtual indirect advertising; Process described virtual indirect advertising, and treated virtual indirect advertising is inserted in described candidate region; The exposure characteristics exposed in described video image content based on described virtual indirect advertising measures the exposure levels of described virtual indirect advertising; And the insertion region guidance information comprising the described exposure levels of described virtual indirect advertising is supplied to user.
In the case, the step inserting region guidance information is provided to comprise the following steps: to determine whether the exposure levels of virtual indirect advertising exceedes Pre-set reference exposure levels, and if the exposure levels of virtual indirect advertising exceedes reference exposure level, then contain the insertion region guidance information of the exposure levels of virtual indirect advertising to user's providing package.
In the case, the step inserting region guidance information is provided to comprise the following steps: to any one in the item below user's display or more: the candidate region inserting virtual indirect advertising and the candidate region existed before inserting virtual indirect advertising.
In the case, the method being provided for the insertion region guidance information inserting virtual indirect advertising can also comprise the following steps: the advertising expenditure calculating virtual indirect advertising based on exposure levels, and provides the step inserting region guidance information to comprise the following steps: show advertising expenditure to user.
In the case, to process and the step inserting virtual indirect advertising can comprise the following steps: select to be inserted into the virtual indirect advertising in candidate region; Process virtual indirect advertising based on preliminary treatment characteristic, virtual indirect advertising can be inserted in candidate region; And treated virtual indirect advertising is inserted in candidate region.
In the case, exposure characteristics can comprise the real-time characteristic and preliminary treatment characteristic measured in real time while virtual indirect advertising just exposes in video image content.
In addition, to achieve these goals, the invention provides a kind of device for insertion of advertising, this device comprises: marker recognition unit, and it is constructed to identify the mark comprised in the frame of video; Area determination unit, it is constructed to determine to mark corresponding marked region with described, will be inserted into the insertion region of virtual indirect advertising and use the mark in described marked region and described insertion region to eliminate region; Mark eliminates unit, and it is constructed to only to eliminate region to described mark and performs reparation; And advertisement plug-in unit, it is constructed to, after the described mark of reparation eliminates region, be inserted in described insertion region by described virtual indirect advertising.
In the case, described marked region can be divided into restoring area and non-restoring area by area determination unit, and described restoring area can be defined as described mark elimination region.
In the case, area determination unit can determine whether each marked pixels in described marked region is present in described insertion region, and the marked pixels be not present in described insertion region can be defined as described restoring area.
In the case, area determination unit can determine whether each marked pixels in described marked region is present in described insertion region, and the marked pixels be present in described insertion region can be defined as described non-restoring area.
In the case, area determination unit can generate the insertion borderline region comprising and form the pixel inserting border, and the part inserting borderline region can be included in restoring area.
In the case, area determination unit can determine whether form each pixel inserting zone boundary is present in from the border preset distance inserting region, the region be made up of the pixel existed in preset distance can be defined as inserting borderline region, and can by considering that inserting borderline region determines that mark eliminates region.
In the case, area determination unit can determine whether each pixel inserted in borderline region is present in marked region and inserts in region, and the pixel existed in marked region and insertion region can be included in restoring area.
In the case, mark is eliminated unit and can be comprised: separation of images unit, and it is constructed to the image corresponding with restoring area to be divided into structural images and texture image; Structure repair unit, it is constructed to by performing to repair to structural images generate treated structural images according to making to be diffused into mode in restoring area close to the pixel of the periphery of restoring area along the line of equal gray value; Textures synthesis unit, it is constructed to generate treated texture image by performing textures synthesis to texture image; And image composing unit, it is constructed to treated structural images and treated texture image to synthesize.
In the case, mark is eliminated unit and can be comprised: block segmenting unit, and its each being constructed to not comprising in the region of restoring area and restoring area is segmented into cell block; Block selected cell, it is constructed to each piece for restoring area, in the region not comprising restoring area, select the most consistent block; And block synthesis unit, it is constructed to each piece of restoring area and selected block to synthesize.
In the case, mark is eliminated unit and in internal direction, can be performed reparation from the unit area corresponding with the periphery of restoring area.
In the case, area determination unit can generate the mark boundaries region comprising the pixel forming mark boundaries, and the part in mark boundaries region can be included in restoring area.
In the case, area determination unit can determine whether each pixel forming marked region border is present in from the border preset distance of marked region, the region be made up of the pixel existed in preset distance can be defined as mark boundaries region, and can by considering that mark boundaries region determines that mark eliminates region.
In the case, area determination unit can determine whether each pixel in mark boundaries region is present in marked region and inserts in region, and the pixel be not present in marked region and insertion region can be included in restoring area.
In addition, the invention provides a kind of elimination for inserting the method for the mark of virtual indirect advertising, the method comprises the following steps: identify the mark comprised in the frame of video; Determine to mark corresponding marked region with described, will the insertion region of virtual indirect advertising is inserted into and use the mark in described marked region and described insertion region to eliminate region; Only region is eliminated to described mark and perform reparation; And after the described mark of reparation eliminates region, described virtual indirect advertising is inserted in described insertion region.
In the case, determine that marking the step eliminating region can comprise the following steps: marked region is divided into restoring area and non-restoring area, and restoring area is defined as mark elimination region.
In the case, determine that marking the step eliminating region can comprise the following steps: determine whether each marked pixels in marked region is present in insertion region, and be defined as restoring area by not being present in the marked pixels inserted in region.
In the case, determine that marking the step eliminating region can comprise the following steps: determine whether each marked pixels in marked region is present in insertion region, and be defined as non-restoring area by being present in the marked pixels inserted in region.
In the case, determine that marking the step eliminating region can comprise the following steps: generate the insertion borderline region comprising and form the pixel inserting border, and the part inserting borderline region be included in restoring area.
In the case, determine that marking the step eliminating region can comprise the following steps: determine whether form each pixel inserting zone boundary is present in from the border preset distance inserting region, the region be made up of the pixel be present in preset distance is defined as insert borderline region, and by considering that inserting borderline region determines that mark eliminates region.
In the case, determine that the step marking elimination region can comprise the following steps: determine whether each pixel inserted in borderline region is present in marked region and inserts in region, and the pixel be present in marked region and insertion region is included in restoring area.
In addition, to achieve these goals, the invention provides a kind of device for calculating the boundary value for inserting virtual indirect advertising, this device comprises: area determination unit, and it is constructed to the borderline region of the boundary pixel determining to comprise the virtual indirect advertising be inserted in frame of video; Reference pixel selected cell, it is constructed to for each borderline region pixel in described borderline region, by considering that prospect reference pixel and reference background pixel are selected in the position of described borderline region pixel from the remaining area except described borderline region; And boundary value computing unit, it is constructed to use the weight of the weight of the pixel value of the pixel value of described prospect reference pixel and described reference background pixel and described prospect reference pixel and described reference background pixel to calculate the value of described borderline region pixel.
In the case, the remaining area except described borderline region can be divided into the foreground area corresponding with described virtual indirect advertising and background area by area determination unit; And reference pixel selected cell can comprise: prospect reference pixel selected cell, its be constructed to select in the pixel of described foreground area near of described borderline region pixel as described prospect reference pixel; And reference background pixel selection unit, its be constructed to select in the pixel of described background area near of described borderline region pixel as described reference background pixel.
In the case, reference pixel selected cell can comprise: former frame reference pixel selected cell, it is constructed to for each borderline region pixel in described borderline region, by considering that former frame reference pixel is selected in the position of described borderline region pixel from the former frame of described frame of video; And a rear frame reference pixel selected cell, it is constructed to from a rear frame of described frame of video, select a rear frame reference pixel; And a rear frame reference pixel selected cell, it is constructed to from a rear frame of frame of video, select a rear frame reference pixel; And boundary value computing unit can use the weight of the pixel value of former frame reference pixel and the rear pixel value of a frame reference pixel and the weight of former frame reference pixel and a rear frame reference pixel to carry out the value of computation bound area pixel.
In the case, reference pixel selected cell can select the pixel corresponding with the position of borderline region pixel as former frame reference pixel from former frame, and can from a rear frame, select the pixel corresponding with the position of borderline region pixel as a rear frame reference pixel.
In the case, boundary value computing unit can arrange the weight of prospect reference pixel based on the distance between borderline region pixel and prospect reference pixel, and can arrange the weight of reference background pixel based on the distance between borderline region pixel and reference background pixel.
In the case, boundary value computing unit can arrange the weight of prospect reference pixel and the weight of reference background pixel inversely with the distance from borderline region pixel.
In the case, area determination unit can determine whether each pixel of frame of video is present in from boundary pixel preset distance, and the region be made up of the pixel be present in preset distance can be defined as borderline region.
In the case, area determination unit can by considering that the overall resolution of frame of video arranges preset distance.
In the case, area determination unit can arrange longer preset distance for the overall resolution of higher frame of video.
In the case, reference pixel selected cell can also comprise vertical line generation unit, and this vertical line generation unit is constructed to generate the vertical line between borderline region pixel and the boundary line of remaining area; Prospect reference pixel selected cell can select in the pixel of foreground area near borderline region pixel and corresponding with the foot of vertical line one as prospect reference pixel, and reference background pixel selection unit can select in the pixel of background area near borderline region pixel and corresponding with the foot of vertical line reference pixel as a setting.
In addition, the invention provides a kind of calculating for inserting the method for the boundary value of virtual indirect advertising, the method comprises following: the borderline region determining to comprise the boundary pixel of the virtual indirect advertising be inserted in frame of video; For each borderline region pixel in described borderline region, by considering that prospect reference pixel and reference background pixel are selected in the position of described borderline region pixel from the remaining area except described borderline region; And use the weight of the weight of the pixel value of the pixel value of described prospect reference pixel and described reference background pixel and described prospect reference pixel and described reference background pixel to calculate the value of described borderline region pixel.
In the case, the method calculating the boundary value for inserting virtual indirect advertising can also comprise the following steps: the remaining area beyond borderline region is divided into the foreground area corresponding with virtual indirect advertising and background area; And select the step of prospect reference pixel and reference background pixel to comprise the following steps: select in the pixel of foreground area near of borderline region pixel as prospect reference pixel, and select in the pixel of background area near one of borderline region pixel reference pixel as a setting.
In the case, the method calculated for the boundary value inserting virtual indirect advertising can also comprise the following steps: for each borderline region pixel in borderline region, by considering that former frame reference pixel is selected in the position of borderline region pixel from the former frame of frame of video, and a frame reference pixel after selecting from a rear frame of frame of video; And the step of the value of computation bound area pixel can comprise the following steps: use the pixel value of former frame reference pixel and the weight of the rear pixel value of a frame reference pixel and the weight of former frame reference pixel and a rear frame reference pixel to carry out the value of computation bound area pixel further.
In the case, select the step of former frame reference pixel and a rear frame reference pixel to comprise the following steps: from former frame, select the pixel corresponding with the position of borderline region pixel as former frame reference pixel, and from a rear frame, select the pixel corresponding with the position of borderline region pixel as a rear frame reference pixel.
In the case, the step of the value of computation bound area pixel can comprise the following steps: the weight arranging prospect reference pixel based on the distance between borderline region pixel and prospect reference pixel, and the weight arranging reference background pixel based on the distance between borderline region pixel and reference background pixel.
In the case, the step of the value of computation bound area pixel can comprise the following steps: arrange the weight of prospect reference pixel and the weight of reference background pixel inversely with the distance from borderline region pixel.
In the case, determine that the step of borderline region can comprise the following steps: determine whether each pixel of frame of video is present in from boundary pixel preset distance, and the region be made up of the pixel be present in preset distance is defined as borderline region.
In the case, determine that the step of borderline region can comprise the following steps: by considering that the overall resolution of frame of video arranges preset distance.
In the case, determine that the step of borderline region can comprise the following steps: the overall resolution for higher frame of video arranges longer preset distance.
In the case, the method calculating the boundary value for inserting virtual indirect advertising can also comprise the following steps: generate the vertical line between borderline region pixel and the boundary line of remaining area; Select in the pixel of foreground area near borderline region pixel and corresponding with the foot of vertical line one as prospect reference pixel, and select in the pixel of background area near borderline region pixel and corresponding with the foot of vertical line reference pixel as a setting.
Beneficial effect
According to the present invention, search package is inserted target frame containing the advertisement that region is inserted in advertisement and target frame is inserted in described advertisement and carries out cluster, then according to unified mode for by the frame insertion of advertising of cluster, therefore make advertisement more naturally to be inserted.
In addition, (namely the present invention uses clustering factor, the result compared between region is inserted in advertisement insertion region in reference frame and the advertisement in other frame) carry out insertion of advertising according to unified mode, therefore just advertisement automatically can be inserted in multiple frame by means of only advertisement being inserted in single frame.
In addition, the present invention extract from frame only comprise advertisement insert region part and extracted part is carried out cluster, therefore realize cluster more rapidly.
In addition, the present invention determines the candidate region by being inserted into virtual indirect advertising, measure the exposure levels of described candidate region, and provide measured exposure levels to user, therefore provide guidance information, make user that virtual indirect advertising can be selected while the advertising results identifying each candidate region more intuitively to insert region.
In addition, the present invention, only when the level of the candidate region being inserted into virtual indirect advertising is exceeded the level being equal to or greater than predetermined value, provides corresponding candidate region relevant information to user, therefore makes it possible to more promptly select virtual indirect advertising to insert region.
In addition, the present invention provides by based on various measurement standard (size that such as virtual indirect advertising is exposed, time, angle and position to user, the speed of virtual indirect advertising movement in screen, the size in the region covered by another object, the frequency that virtual indirect advertising is covered by another object) assign weight and the total exposure level that calculates, therefore make it possible to select virtual indirect advertising to insert region efficiently based on the advertising results in corresponding candidate region.
In addition, the present invention uses the mark be inserted in video image content to insert virtual indirect advertising, and performs reparation to marked region, therefore performs process to realize the coordination with surrounding image.
In addition, the present invention only performs reparation to not overlapping with the insertion region of the virtual indirect advertising region of marked region, therefore reduces the running cost needed for repairing.
In addition, the present invention reduces the running cost needed for reparation, therefore makes the ad margins of virtual indirect advertising maximize.
In addition, according to the present invention, replace existing advertisement and virtual indirect advertising is inserted in video image content, and reparation is performed to existing advertising area, therefore perform process to realize the coordination with surrounding image.
In addition, according to the present invention, automatically can calculate the value of the borderline region pixel of the virtual indirect advertising be inserted in video image content, and perform the boundary line that process makes to insert virtual indirect advertising and can coordinate with around image.
In addition, the present invention is based on the position of the borderline region pixel of the virtual indirect advertising be inserted in video image content and foreground pixel and the isolated distance of background pixel, after foreground pixel value and background pixel value are mixed, the value of computation bound area pixel, therefore performing the boundary line that process makes to insert virtual indirect advertising can coordinate with around image.
In addition, the present invention carrys out the value of automatically computation bound area pixel with reference to the pixel value of the frame before the present frame inserting virtual indirect advertising and after it, therefore perform process make to insert virtual indirect advertising boundary line can the playback of video and around image coordinate.
Accompanying drawing explanation
Fig. 1 illustrates that use frame cluster according to the present invention carrys out the figure of the holistic approach of insertion of advertising;
Fig. 2 is the block diagram that the use frame cluster illustrating according to the embodiment of the present invention carrys out the device of insertion of advertising;
Fig. 3 is the block diagram of the example that the cluster cell shown in Fig. 1 is shown;
Fig. 4 is the operational flowchart that the use frame cluster illustrating according to the embodiment of the present invention carrys out the method for insertion of advertising;
Fig. 5 is the operational flowchart of the example that the sorting procedure shown in Fig. 4 is shown;
Fig. 6 is the figure of the example illustrated according to reference frame of the present invention and frame cluster;
Fig. 7 illustrates that use frame cluster according to the present invention carrys out the figure of the example of the method for insertion of advertising;
Fig. 8 illustrates that use frame cluster according to the present invention carrys out the figure of another example of the method for insertion of advertising;
Fig. 9 is the block diagram of the device of the insertion region guidance information for being provided for inserting virtual indirect advertising illustrated according to the embodiment of the present invention;
Figure 10 is the block diagram of the example that the advertisement plug-in unit shown in Fig. 9 is shown;
Figure 11 is the block diagram of the example that the exposure measuring unit shown in Fig. 9 is shown;
Figure 12 is the figure of the example that the equation calculated by the exposure measuring unit shown in Fig. 9 is shown;
Figure 13 is the operational flowchart being provided for inserting the method for the insertion region guidance information of virtual indirect advertising illustrated according to the embodiment of the present invention;
Figure 14 is the operational flowchart providing the example of the step inserting region guidance information illustrated shown in Figure 13;
Figure 15 is the operational flowchart process shown in Figure 13 being shown and inserting the example of the step of virtual indirect advertising;
Figure 16 is the block diagram of the device for eliminating the mark for inserting virtual indirect advertising illustrated according to the embodiment of the present invention;
Figure 17 illustrates that the mark shown in Figure 16 eliminates the block diagram of the example of unit;
Figure 18 illustrates that the mark shown in Figure 16 eliminates the block diagram of another example of unit;
Figure 19 illustrates according to the figure comprising the example of the frame of video of mark for inserting virtual indirect advertising of the present invention;
Figure 20 illustrates according to the figure inserting the example of the frame of video of virtual indirect advertising of the present invention;
Figure 21 illustrates insert virtual indirect advertising according to of the present invention and eliminate the figure of the example of the frame of video of mark;
Figure 22 illustrates according to the figure of marked region of the present invention with the example in insertion region;
Figure 23 is the figure of the example illustrated according to restoring area of the present invention;
Figure 24 is the figure of another example illustrated according to restoring area of the present invention;
Figure 25 illustrates that elimination is according to the embodiment of the present invention for inserting the operational flowchart of the method for the mark of virtual indirect advertising;
Figure 26 is the block diagram of the virtual indirect advertising service unit illustrated according to the embodiment of the present invention;
Figure 17 is the block diagram of the example that the reparation unit shown in Figure 26 is shown;
Figure 18 is the block diagram of another example that the reparation unit shown in Figure 26 is shown;
Figure 27 is the figure of the example of the restoring area illustrated according to the embodiment of the present invention;
Figure 28 is the figure inserting the example of the video image content of virtual indirect advertising illustrated according to the embodiment of the present invention;
Figure 29 is the operational flowchart of the example of the virtual indirect advertising method of servicing illustrated according to the embodiment of the present invention;
Figure 30 is the operational flowchart of the example that the step that the execution shown in Figure 29 is repaired is shown;
Figure 31 is the operational flowchart of another example that the step that the execution shown in Figure 29 is repaired is shown;
Figure 32 is the block diagram of the device for calculating the boundary value for inserting virtual indirect advertising illustrated according to the embodiment of the present invention;
Figure 33 is the block diagram of the example that the reference pixel selected cell shown in Figure 32 is shown;
Figure 34 illustrates according to the figure inserting the example of the frame of video of virtual indirect advertising of the present invention;
Figure 35 is the figure of the example illustrated according to borderline region of the present invention, foreground area and background area;
Figure 36 is the figure that the example selecting prospect reference pixel and reference background pixel according to the present invention is shown;
Figure 37 is the figure that the example selecting former frame reference pixel and a rear frame reference pixel according to the present invention is shown;
Figure 38 illustrates that calculating is according to the embodiment of the present invention for inserting the operational flowchart of the method for the boundary value of virtual indirect advertising; And
Figure 39 is the operational flowchart of the example of the step that the selection prospect reference pixel shown in Figure 38 and reference background pixel are shown.
Embodiment
Describe the preferred embodiment of the present invention in detail below with reference to accompanying drawings.But, purport of the present invention unnecessarily ambiguous, detailed description to known function and structure may be made by omitting in the following description and drawings.In addition, it should be noted that and represent identical assembly with identical Reference numeral as much as possible in whole accompanying drawing.
The term used in following specification and claim and word should not be interpreted as to being limited property having general implication and the implication of dictionary, and implication and the concept with technical spirit according to the invention should be interpreted as based on following principle: inventor can define the concept of term rightly, to describe its invention according to the mode of the best.Therefore, whole technical spirit of the present invention is not contained because the structure shown in the execution mode that describes in this specification and accompanying drawing only corresponds to the preferred embodiment of the present invention, therefore it is intended that, can exist and can replace various equivalent and the modification of described structure when submitting the application to.In addition, term " first ", " second " etc. are used to describe different assemblies and an assembly and another assembly be distinguished from each other out, but are not intended to limit these assemblies.
The device for using frame cluster to carry out insertion of advertising according to the preferred embodiment of the present invention is provided by processor, memory cell and user interface.
Processor receives multiple continuous print frame, by described multiple continuous print Frame storage in a memory cell, and inserts the advertisement relevant to the described multiple continuous print frame of video stored in memory cell.
In addition, memory cell storage comprises various types of information of the handling procedure of processor.
In addition, user interface receives various types of information from user, and information is supplied to processor.
Fig. 1 illustrates that use frame cluster according to the present invention carrys out the figure of the holistic approach of insertion of advertising.
With reference to Fig. 1, the method that use frame cluster according to the present invention carrys out insertion of advertising performs cluster by inserting the search of target frame 710,720 and 730 performance objective to the advertisement comprising advertisement insertion region 610.
In the case, to user's display splicing (stitching) frame 700 (namely use frame cluster according to the present invention carrys out the method for insertion of advertising, advertisement is inserted the result that target frame 710,720 and 730 carries out splicing), as cluster execution result.
In the case, when user is to when splicing insertion of advertising in frame 700, use frame cluster according to the present invention carrys out the method for insertion of advertising while display advertisement has been inserted into the result 800 of splicing in frame 700, according to unified mode, advertisement being inserted into all advertisements obtained from the cluster areas 611 inserting advertisement inserts target frame 710,720 and 730, therefore obtains the frame 810,820 and 830 inserting advertisement.
As mentioned above, can it is seen that, come in the method for insertion of advertising in use frame cluster according to the present invention, target frame is inserted in the advertisement that search package inserts region containing advertisement, advertisement is inserted target frame and carry out cluster, and carried out insertion of advertising by the frame of cluster according to unified mode for all, therefore make it possible to insertion of advertising more naturally.
Fig. 2 is the block diagram that the use frame cluster illustrating according to the embodiment of the present invention carrys out the device of insertion of advertising.
With reference to Fig. 2, according to the embodiment of the present invention comprise frame search unit 210, cluster cell 220 and advertisement plug-in unit 230 for the device using frame cluster to carry out insertion of advertising.
When advertisement insertion region is arranged at least one in the frame of video, target frame is inserted in the advertisement that frame search unit 210 search package inserts region containing advertisement.
In the case, frame search unit 210 can from reference frame extract minutiae, and can use this characteristic point to insert target frame to obtain advertisement.
In the case, Kanade-Lucas-Tomasi feature tracker (KLT) can be used to carry out extract minutiae.
In the case, reference frame can be that in target frame, that user is provided with advertisement insertion region frame is inserted in advertisement.
In the case, reference frame can be when show intuitively to user advertisement insert in target frame any one or more time the frame selected by user.
In the case, reference frame can be the frame with the widest advertisement insertion region that advertisement inserts in target frame.
In the case, frame search unit 210 can determine moving integrally of continuous print frame, and can carry out search advertisements insertion target frame based on the determination result moved integrally.
In the case, in the frame that frame search unit 210 can use the determination result that moves integrally former and follow-up frame any one or more in prediction advertisement insert the position in region, and search advertisements insert target frame.
Advertisement is inserted target frame and is divided into groups by cluster cell 220, calculates and inserts the clustering factor of target frame for advertisement, and advertisement is inserted target frame and carry out cluster.
In the case, cluster cell 220 can use advertisement to insert the characteristic point of target frame to detect homography matrix (homographymatrix), and homography matrix can be used to perform cluster, make, for single frames, target frame is inserted in advertisement and be stitched together.
In the case, Kanade-Lucas-Tomasi feature tracker (KLT) can be used to carry out extract minutiae.
In the case, RANSAC process can be used to detect homography matrix.
In the case, following order is followed in RANSAC process:
1. select any four characteristic points between two frames.
2. calculate meet between these four characteristic points such as below the homography matrix H of the such relation of equation 1.
Equation 1
x'＝Hx
x ′ y ′ 1 = h 1 h 2 h 3 h 4 h 5 h 6 h 7 h 8 h 9 x y 1
x ′ = h 1 x + h 2 y + h 3 h 7 x + h 8 y + h 9 , y ′ = h 4 x + h 5 y + h 6 h 7 x + h 8 y + h 9
In the case, (x, y) is the characteristic point of the first frame, and (x', y') is the characteristic point of the second frame.In equation 1, H is homography matrix.When two dimension (2D) image, H can be expressed as 3*3 matrix.
3., for all characteristic points, calculate wherein P 1be the characteristic point of the first frame, P 2be the characteristic point of the second frame, and k is the quantity of characteristic point.In the case, although according to the known P of equation 1 2=HP 1but, may error be there is in the calculating of H, therefore for P 2and HP 1the value calculated is not identical.D krepresent P 2and HP 1between error.
4.d kthe quantity being equal to or less than the point of predetermined value is expressed as NI i.
5. perform step 1 to step 4 pre-determined number.
6. work as NI itime maximum, recalculate H by use characteristic point and obtain final homography matrix H.
As mentioned above, in RANSAC process, by repeatedly selecting arbitrary characteristics point pre-determined number to calculate H, and by NI i(that is, d kbe equal to or less than the quantity of the point of predetermined value) maximum time H be defined as homography matrix of the present invention.
In the case, cluster cell 220 can be analyzed advertisement and insert the scene structure in region, can analyze the camera motion that region is inserted in advertisement, and use scenes structure and camera motion can calculate clustering factor.
In the case, cluster cell 220 can select advertisement to insert any one in target frame as with reference to frame, region can be inserted with reference to the advertisement insertion region in frame with the advertisement advertisement of inserting in target frame except reference frame to compare, and can calculate with the difference of reference frame as clustering factor.
In the case, cluster cell 220 can insert scene structure in target frame with reference to the scene structure in frame and camera motion and the advertisement except reference frame and camera motion compares.
In the case, cluster cell 220 can obtain each advertisement insert in the position in region, size, rotation and visual angle (perspective) any one or more.
In the case, cluster cell 220 can obtain each advertisement and insert the focus in region and any one in shake or more.
In the case, cluster cell 220 can determine moving integrally between continuous print frame for by the frame of cluster, and can calculate clustering factor based on the determination result moved integrally.
In the case, cluster cell 220 can insert target frame to extract from each advertisement and only comprise the part that region is inserted in advertisement, then can perform cluster.
In the case, cluster cell 220 can carry out cluster to continuous print frame from reference frame.
When in by least one in the frame of cluster during insertion of advertising, advertisement plug-in unit 230 is carried out insertion of advertising by the frame of cluster according to unified mode for all.
In the case, complete insertion (integratedinsertion) refer to by application clustering factor advertisement is inserted into by each in the frame of cluster.
In the case, when for single frames advertisement being inserted target frame and being stitched together, advertisement plug-in unit 230 can use the inverse matrix of homography matrix to be separated to make advertisement after insertion of advertising.
In the case, advertisement plug-in unit 230 can insert region to advertisement and perform reparation.
In the case, virtual image can be inserted in advertisement insertion region by advertisement plug-in unit 230.
Fig. 3 is the block diagram of the example that the cluster cell 220 shown in Fig. 1 is shown.
Scene structure analytic unit 310, camera motion analytic unit 320 and clustering factor computing unit 330 is comprised with reference to the cluster cell 220 shown in Fig. 3, Fig. 1.
Scene structure analytic unit 310 analyzes the scene structure that region is inserted in advertisement.
In the case, scene structure analytic unit 310 can obtain each advertisement insert in the position in region, size, rotation and visual angle any one or more.
Camera motion analytic unit 320 analyzes the camera motion that region is inserted in advertisement.
In the case, camera motion analytic unit 320 can obtain each advertisement and insert the focus in region and any one in shake or more.
Clustering factor computing unit 330 use scenes structure and camera motion can calculate clustering factor.
In the case, clustering factor computing unit 330 can select advertisement to insert any one in target frame as with reference to frame, insert region with reference to the advertisement insertion region in frame with the advertisement advertisement of inserting in target frame except reference frame to compare, and can calculate with the difference of reference frame as clustering factor.
In the case, clustering factor computing unit 330 can insert scene structure in target frame with reference to the scene structure in frame and camera motion and the advertisement except reference frame and camera motion compares.
In the case, clustering factor computing unit 330 can be determined about by moving integrally between the continuous print frame of the frame of cluster, and can calculate clustering factor based on the determination result moved integrally.
Fig. 4 is the operational flowchart that the use frame cluster illustrating according to the embodiment of the present invention carrys out the method for insertion of advertising.
With reference to Fig. 4, come in the method for insertion of advertising in use frame cluster according to the embodiment of the present invention, when advertisement insertion region is arranged at least one in the frame of video, target frame is inserted in the advertisement that search package inserts region containing advertisement in step S410.
In the case, in step S410, in continuous print frame from reference frame, search advertisements can insert target frame.
In the case, in step S410, characteristic point can be detected from reference frame, and advertisement insertion target frame can be obtained by use characteristic point.
In the case, Kanade-Lucas-Tomasi feature tracker (KLT) can be used to detect characteristic point.
In the case, reference frame can be that in target frame, that user is provided with advertisement insertion region frame is inserted in advertisement.
In the case, reference frame can be that user selects and to user show intuitively advertisement insert in target frame any one or more frame.
In the case, reference frame can be the frame with the widest advertisement insertion region that advertisement inserts in target frame.
In the case, in step S410, moving integrally of continuous print frame can be determined, and search advertisements insertion target frame can be carried out based on the determination result moved integrally.
In the case, in step S410, the determination result moved integrally any one in former frame and a rear frame next or more middle prediction advertisement can be used to insert the position in region, and can search advertisements insertion target frame.
In addition, come in the method for insertion of advertising in use frame cluster according to the embodiment of the present invention, in the step s 420, target frame is inserted to advertisement and divides into groups, insert target frame for advertisement and calculate clustering factor, and advertisement is inserted target frame and carry out cluster.
In the case, in the step s 420, advertisement can be used to insert the characteristic point of target frame to detect homography matrix, and homography matrix can be used to perform cluster, make, for single frames, target frame is inserted in advertisement and be stitched together.
In the case, Kanade-Lucas-Tomasi feature tracker (KLT) can be used to carry out extract minutiae.
In the case, RANSAC process can be used to detect homography matrix.
In the case, following order is followed in RANSAC process:
1. select any four characteristic points between two frames.
2. calculate meet between these four characteristic points such as above the homography matrix H of the such relation of equation 1.
3., for all characteristic points, calculate wherein P 1be the characteristic point of the first frame, P 2be the characteristic point of the second frame, and k is the quantity of characteristic point.In the case, although according to the known P of equation 1 2=HP 1but, may error be there is in the calculating of H, therefore for P 2and HP 1the value calculated is not identical.D krepresent P 2and HP 1between error.
4.d kthe quantity being equal to or less than the point of predetermined value is expressed as NI i.
5. perform step 1 to step 4 pre-determined number.
6. work as NI itime maximum, recalculate H by use characteristic point and obtain final homography matrix H.
As mentioned above, in RANSAC process, by repeatedly selecting arbitrary characteristics point pre-determined number to calculate H, and by NI i(that is, d kbe equal to or less than the quantity of the point of predetermined value) maximum time H be defined as homography matrix of the present invention.
In the case, in the step s 420, advertisement can be analyzed and insert the scene structure in region, the camera motion that region is inserted in advertisement can be analyzed, and use scenes structure and camera motion can calculate clustering factor.
In the case, in the step s 420, advertisement can be selected to insert any one in target frame as with reference to frame, insert region with reference to the advertisement insertion region in frame with the advertisement advertisement of inserting in target frame except reference frame to compare, and can calculate with the difference of reference frame as clustering factor.
In the case, in the step s 420, scene structure in target frame can be inserted with reference to the scene structure in frame and camera motion and the advertisement except reference frame and camera motion compares.
In the case, in the step s 420, any one in the position in each advertisement insertion region, size, rotation and visual angle or more can be obtained.
In the case, in the step s 420, each advertisement can be obtained and insert the focus in region and any one in shake or more.
In the case, in the step s 420, moving integrally between continuous print frame can be determined for by the frame of cluster, and clustering factor can be calculated based on the determination result moved integrally.
In the case, in the step s 420, only each advertisement can inserting in target frame the part extracted from comprising advertisement insertion region, then can perform cluster.
In the case, in the step s 420, cluster can be carried out to continuous print frame from reference frame.
Come in the method for insertion of advertising in use frame cluster according to the embodiment of the present invention, when advertisement is inserted in by least one in the frame of cluster, in step S430, carried out insertion of advertising by the frame of cluster according to unified mode for all.
In the case, in step S430, when for single frames advertisement being inserted target frame and being stitched together, the inverse matrix of homography matrix can be used after insertion of advertising to be separated to make advertisement.
In the case, in step S430, region can be inserted to advertisement and perform reparation.
In the case, in step S430, virtual image can be inserted into advertisement and insert in region.
Fig. 5 is the operational flowchart of the example that the sorting procedure S420 shown in Fig. 4 is shown.
With reference to Fig. 5, for the sorting procedure S420 shown in Fig. 4, in step S510, analyze advertisement insert region.
In the case, in step S510, any one in the position in each advertisement insertion region, size, rotation and visual angle or more can be obtained.
In addition, for the sorting procedure S420 shown in Fig. 4, in step S520, the camera motion that region is inserted in advertisement is analyzed.
In the case, in step S520, each advertisement can be obtained and insert the focus in region and any one in shake or more.
In addition, for the sorting procedure S420 shown in Fig. 4, in step S530, use scenes structure and camera motion calculate clustering factor.
In the case, in step S530, advertisement can be selected to insert any one in target frame as with reference to frame, insert region with reference to the advertisement insertion region in frame with the advertisement advertisement of inserting in target frame except reference frame to compare, and can calculate with the difference of reference frame as clustering factor.
In the case, in step S530, scene structure in target frame can be inserted with reference to the scene structure in frame and camera motion and the advertisement except reference frame and camera motion compares.
In the case, in step S530, moving integrally between continuous print frame can be determined for by the frame of cluster, and clustering factor can be calculated based on the determination result moved integrally.
Fig. 6 is the figure of the example illustrated according to reference frame 620 of the present invention and frame cluster.
With reference to Fig. 6, come in the method for insertion of advertising in use frame cluster according to the present invention, search package can insert the advertisement insertion target frame t in region 610 containing advertisement in the video frame 0to t n-1.
In the case, come in the method for insertion of advertising in use frame cluster according to the present invention, advertisement can be inserted target frame t 0to t n-1in any one as with reference to frame 620, for performing reparation.
Such as, come in the method for insertion of advertising in use frame cluster according to the present invention, advertisement can be inserted target frame t 0to t n-1in initial advertisement insert target frame t 0as reference frame 620.
In the case, come in the method for insertion of advertising in use frame cluster according to the present invention, region 610 can be inserted with reference to the advertisement in frame 620 and insert target frame t with the advertisement except reference frame 620 1to t n-1in advertisement insert region 610 and compare, and can to calculate with the difference of reference frame as clustering factor.
Such as, come in the method for insertion of advertising in use frame cluster according to the present invention, region 610 can be inserted with reference to the advertisement in frame 620 and insert target frame t with the advertisement except reference frame 620 1to t n-1in advertisement insert region 610 compare, and can obtain advertisement insert region 610 advertisement insert target frame t 1to t n-1in relative position and as clustering factor.
Come in the method for insertion of advertising in use frame cluster according to the present invention, although do not illustrate in Fig. 6, n advertisement can be inserted target frame t 0to t n-1in, user is provided with frame that advertisement inserts region 610 as with reference to frame 620.
In addition, come in the method for insertion of advertising in use frame cluster according to the present invention, advertisement can be shown intuitively to user and insert target frame t 0to t n-1in any one or more, and frame user can selected is as with reference to frame 620.
Come in the method for insertion of advertising in use frame cluster according to the present invention, advertisement can be inserted target frame t 0to t n-1in there is frame that the widest advertisement inserts region 610 as with reference to frame 620.
Fig. 7 illustrates that use frame cluster according to the present invention carrys out the figure of the example of the method for insertion of advertising.
With reference to Fig. 7, come in the method for insertion of advertising in use frame cluster according to the present invention, target frame 710,720 and 730 is inserted to the advertisement comprising advertisement insertion region 610 and performs cluster.
In the case, come in the method for insertion of advertising in use frame cluster according to the present invention, can to user's display splicing frame 700 (that is, advertisement being inserted the result that target frame 710,720 and 730 carries out splicing), as the result performing cluster.
In the case, target frame 710,720 and 730 is inserted in advertisement can be that target frame t is inserted in the advertisement shown in Fig. 6 0to t n-1in some.
Fig. 8 illustrates that use frame cluster according to the present invention carrys out the figure of another example of the method for insertion of advertising.
With reference to Fig. 8, come in the method for insertion of advertising in use frame cluster according to the present invention, when advertisement being inserted in advertisement insertion region 610 at least one insertion by the advertisement of cluster in target frame 710,720 and 730 shown in Fig. 7, by according to unified mode advertisement being inserted into allly being inserted target frame 710,720 and 730 by the advertisement of cluster of obtaining from the cluster areas 611 inserting advertisement, obtain the frame 810,820 and 830 inserting advertisement.
In the case, come in the method for insertion of advertising in use frame cluster according to the present invention, the splicing frame 700 shown in Fig. 7 can be shown to user, and when advertisement is inserted in splicing frame 700 by user, result advertisement being inserted into and splicing in frame 700 can be shown.
The method that use frame cluster according to the present invention carrys out insertion of advertising may be implemented as the program or smart phone application that can be performed by various computer installation.In the case, program or smart phone application can be recorded in computer-readable storage medium.Computer-readable storage medium can comprise program command, data file and data structure alone or in combination.The program command recorded in storage medium can be specifically designed for the present invention and construct, or can be known or available for the those of ordinary skill in the field of computer software.The example of computer-readable storage medium comprises all types of hardware units be specially constructed as record and execution of program instructions, such as magnetizing mediums (such as hard disk, floppy disk and tape), optical medium (such as CD (CD)-read-only memory (ROM) and digital versatile disc (DVD)), magnet-optical medium (such as soft CD, ROM, random access memory (RAM) and flash memory).The example of program command comprises such as such by the code of compiler-creating machine code and uses the executable higher-level language code of interpreter by computer.These hardware units can be constructed to as the operation of one or more software module to perform operation of the present invention, and vice versa.
Apparatus and method for using frame cluster to carry out insertion of advertising according to the present invention are not limited to structure and the method for above-mentioned execution mode, but partly or entirely can being constructed to of these execution modes is optionally combined, and makes it possible to the execution mode revised according to various mode.
The apparatus and method of the insertion region guidance information for being provided for inserting virtual indirect advertising are according to the embodiment of the present invention described below.
Fig. 9 is the block diagram of the device of the insertion region guidance information for being provided for inserting virtual indirect advertising illustrated according to the embodiment of the present invention.
With reference to Fig. 9, according to the embodiment of the present invention comprise candidate region determining unit 1110, advertisement plug-in unit 1120, exposure measuring unit 1130 and information provider unit 1140 for the device being provided for the insertion region guidance information inserting virtual indirect advertising.
Candidate region determining unit 1110 determines the candidate region by being inserted into virtual indirect advertising in video image content.
Advertisement plug-in unit 1120 processes virtual indirect advertising, and is inserted in candidate region by virtual indirect advertising.
In the case, advertisement plug-in unit 1120 can select the virtual indirect advertising that will be inserted in candidate region, virtual indirect advertising can be processed based on preliminary treatment characteristic, virtual indirect advertising can be inserted in candidate region, and treated virtual indirect advertising can be inserted in candidate region.
In the case, preliminary treatment characteristic can comprise in item below any one or more: the size of the part covered by another object of the size of the part be exposed in video image content of virtual indirect advertising, the distortion occurred based on angle, virtual indirect advertising and the speed of virtual indirect advertising movement in screen.
Exposure measuring unit 1130 measures the exposure levels of virtual indirect advertising based on the exposure characteristics that virtual indirect advertising is exposed in video image content.
In the case, exposure characteristics can be included in while virtual indirect advertising just exposes in video image content by the real-time characteristic measured in real time and preliminary treatment characteristic.
In the case, during real-time characteristic can comprise below item any one or more: the frequency that the time that virtual indirect advertising exposes in video image content, virtual indirect advertising are covered by another object and virtual indirect advertising are arranged on the position in screen.
In the case, expose measuring unit 1130 and can measure the exposure levels be directly proportional to the ratio of the size of the exposure area exposed in video image content of virtual indirect advertising and the size of whole screen.
In the case, expose measuring unit 1130 and can measure the exposure levels that time of exposing in video image content to virtual indirect advertising is directly proportional.
In the case, the exposure levels that the difference between the angle that measuring unit 1130 can be measured and virtual indirect advertising exposes in video image content and Pre-set reference angle is inversely proportional to is exposed.
In the case, expose measuring unit 1130 and can measure the exposure levels be inversely proportional to the size working as the part that virtual indirect advertising is covered by another object while virtual indirect advertising just exposes in video image content.
In the case, expose measuring unit 1130 and can measure the exposure levels be directly proportional to the frequency that covered by another object of virtual indirect advertising while virtual indirect advertising just exposes in video image content.
In the case, the exposure levels near screen center that measuring unit 1130 can measure virtual indirect advertising is exposed.
In the case, expose measuring unit 1130 can measure and the virtual indirect advertising exposure levels that the speed of movement is directly proportional in screen while virtual indirect advertising just exposes in video image content.
In the case, expose measuring unit 1130 and can calculate total exposure level by collecting based on the exposure levels measured by exposure characteristics.
Information provider unit 1140 contains the insertion region guidance information of the exposure levels of virtual indirect advertising to user's providing package.
In the case, information provider unit 1140 can determine whether the exposure levels of virtual indirect advertising exceedes Pre-set reference exposure levels, and if the exposure levels of virtual indirect advertising exceedes described reference exposure level, then information provider unit 1140 can to the insertion region guidance information of user's providing package containing the exposure levels of virtual indirect advertising.
In the case, information provider unit 1140 can to user's display insert virtual indirect advertising candidate region and insert in the candidate region existed before virtual indirect advertising any one or more.
That is, by the device for being provided for the insertion region guidance information inserting virtual indirect advertising according to the embodiment of the present invention, can provide will be inserted into the candidate region of virtual indirect advertising and the exposure levels information corresponding with candidate region to user.In the case, the region being finally inserted into virtual indirect advertising is selected while user can check candidate region and exposure levels at the same time.In addition, user can select the region that will finally be inserted into virtual indirect advertising while checking the candidate region being inserted into virtual indirect advertising.
As mentioned above, when user selects a part from the multiple candidate regions that will be inserted into virtual indirect advertising, and use according to the embodiment of the present invention carry out insertion of advertising for the device being provided for the insertion region guidance information inserting virtual indirect advertising time, can select more easily insert region.
In addition, the information that the user candidate region that the device for the insertion region guidance information being provided for inserting virtual indirect advertising according to the embodiment of the present invention can be used to receive only exceed predeterminated level with exposure levels is corresponding.Such as, user can receive information corresponding to candidate region only with exposure levels more than 30%.
As mentioned above, use the device for being provided for the insertion region guidance information inserting virtual indirect advertising according to the embodiment of the present invention, user can more promptly select the region with high demonstration effect from multiple candidate regions that can be inserted into virtual indirect advertising.
Although not display in Fig. 9, but according to the embodiment of the present invention can also comprise advertising expenditure computing unit for the device being provided for the insertion region guidance information inserting virtual indirect advertising, this advertising expenditure computing unit is constructed to the advertising expenditure calculating virtual indirect advertising based on exposure levels.
In the case, information provider unit 1140 can show advertising expenditure to user.
In the case, advertising expenditure computing unit can measure the advertising expenditure be directly proportional to exposure levels of virtual indirect advertising, and can measure the advertising expenditure that the total exposure level that calculates to exposure calculating unit is directly proportional.
Figure 10 is the block diagram of the example that the advertisement plug-in unit 1120 shown in Fig. 9 is shown.
Advertisement selecting unit 1210, advertisement processing unit 1220 and process advertisement plug-in unit 1230 is comprised with reference to the advertisement plug-in unit 1120 shown in Figure 10, Fig. 9.
Advertisement selecting unit 1210 is selected to be inserted into the virtual indirect advertising in candidate region.
Advertisement processing unit 1220 processes virtual indirect advertising based on preliminary treatment characteristic, and virtual indirect advertising can be inserted in candidate region.
In the case, preliminary treatment characteristic can comprise in item below any one or more: the size of the part covered by another object of the size of the part exposed in video image content of virtual indirect advertising, the distortion occurred based on angle, virtual indirect advertising and the speed of virtual indirect advertising movement in screen.
Treated virtual indirect advertising is inserted in candidate region by process advertisement plug-in unit 1230.
Figure 11 is the block diagram of the example that the exposure measuring unit 1130 shown in Fig. 9 is shown.
Dimension measuring unit 1310, time measuring unit 1320, deformation measurement unit 1330, support size measuring unit 1340, covering frequence measuring unit 1350, location measurement unit 1360, velocity measuring unit 1370 and exposure calculating unit 1380 is comprised with reference to the exposure measuring unit 1130 shown in Figure 11, Fig. 9.
Dimension measuring unit 1310 measures the exposure levels be directly proportional to the ratio of the size of the exposure area exposed in video image content of virtual indirect advertising and the size of whole screen.
It is evident that, target light exposure level improves along with the increase of the size of the exposure area exposed in screen of virtual indirect advertising.
In the case, the size of the exposure area exposed in video image content at virtual indirect advertising cannot be measured in real time, but can carry out measuring this size based on information during indirect advertising virtual by advertisement processing unit process.
Time measuring unit 1320 measures the exposure levels be directly proportional to the time that virtual indirect advertising exposes in video image content.
It is evident that, the increase of the time that target light exposure level exposes on screen along with virtual indirect advertising and become higher.
Deformation measurement unit 1330 measures the level of virtual indirect advertising distortion based on the angle of the viewpoint of camera.
In the case, the exposure levels that the difference between the angle that deformation measurement unit 1330 can be measured and virtual indirect advertising exposes in video image content and Pre-set reference angle is inversely proportional to.
In the case, Pre-set reference angle can be the angle corresponding with the front surface of virtual indirect advertising.
Therefore, the angle that deformation level and virtual indirect advertising expose and correspond to virtual indirect advertising front surface angle between difference be directly proportional, and measure exposure levels inversely with this difference.
In the case, the angle that virtual indirect advertising exposes in video image content cannot be measured in real time, but can carry out measuring this angle based on information during indirect advertising virtual by advertisement processing unit process.
Support size measuring unit 1340 measures the exposure levels be inversely proportional to the size in the region covered by another object of virtual indirect advertising while virtual indirect advertising exposes in video image content.
It is evident that, the level that exposure levels and virtual indirect advertising are covered by another object is inversely proportional to.
In the case, the size in the region covered by another object of virtual indirect advertising while virtual indirect advertising exposes in video image content cannot be measured in real time, but can carry out measuring this size based on information during indirect advertising virtual by advertisement processing unit process.
The exposure levels that the frequency that covering frequence measuring unit 1350 is measured and virtual indirect advertising is covered by another object while virtual indirect advertising exposes in video image content is inversely proportional to.
It is evident that, the frequency that exposure levels and virtual indirect advertising are covered by another object is inversely proportional to.
Location measurement unit 1360 measures the exposure levels be directly proportional to the degree of approach of screen center to set virtual indirect advertising.
That is, closer to the virtual indirect advertising be set up in screen center can be the virtual indirect advertising with higher projecting degree.Observer is interested for comparatively outstanding virtual indirect advertising.Therefore, exposure levels is measured as and is directly proportional to the projecting degree of virtual indirect advertising.
Velocity measuring unit 1370 is measured and the virtual indirect advertising exposure levels that the speed of movement is inversely proportional in screen while virtual indirect advertising exposes in video image content.
That is, because observer is difficult to proportionally identify virtual indirect advertising with the speed of virtual indirect advertising movement in screen or become interested to it, therefore exposure levels is measured inversely with this speed.
In the case, the speed of virtual indirect advertising movement in screen while virtual indirect advertising exposes in video image content cannot be measured in real time, but can carry out measuring this speed based on information during indirect advertising virtual by advertisement processing unit process.
Exposure calculating unit 1380 calculates total exposure level by collecting the exposure levels measured based on exposure characteristics.
In the case, different weights can be applied to corresponding standard.
Such as, based on the region be exposed size measured by exposure levels have than based on the large weight of the exposure levels measured by the distortion of angle.
Figure 12 is the example of the equation calculated by the exposure measuring unit 1130 shown in Fig. 9.
With reference to Figure 12, the equation that exposure measuring unit 1130 calculates comprises coefficient a to g and variable E, S, T, A, C, F, D and V.
Variable E represents exposure levels.It is drawn by the equation shown in Figure 12.
Variable S represents the size in the region exposed in video image content of virtual indirect advertising.
In the case, S is calculated by the quantity of the pixel by exposure area divided by the quantity of the pixel of whole screen.
Variable S is proportionally calculated with exposure levels.
Variable T represents the time that virtual indirect advertising exposes in video image content.
In the case, Ke Yimiao, millisecond are unit, first-classly on the basis of frame count calculate variable T.
With variable T proportionally calculation exposure level.
The level of angular distortion when variables A represents that virtual indirect advertising just exposes based on virtual indirect advertising in video image content.
In the case, the difference between the angle exposed in video image content based on Pre-set reference angle and virtual indirect advertising calculates variables A.
In the case, Pre-set reference angle can be the corresponding angle with the front surface of virtual indirect advertising.
With variables A calculation exposure level inversely.
Variable C represents the size in the region covered by another object of the virtual indirect advertising when virtual indirect advertising just exposes in video image content.
By the quantity of the pixel comprised in the region covered by another object is calculated variable C divided by the prime number amount of the picture comprised in the region exposed in video image content of virtual indirect advertising.
With variable C calculation exposure level inversely.
Variable F represents the frequency covered by another object of the virtual indirect advertising when virtual indirect advertising just exposes in video image content.
Variable F is calculated divided by total exposure time by the time covered by another object in video image content by virtual indirect advertising
With variable F calculation exposure level inversely.
Variables D represents the distance between the position that virtual indirect advertising exposes in video image content and the center of screen.
With variables D calculation exposure level inversely.
Variable V represents the speed of virtual indirect advertising movement in screen while virtual indirect advertising just exposes in video image content.
With variable V calculation exposure level inversely.
The method being carried out calculation exposure level by the exposure measuring unit shown in Fig. 9 is not limited to the equation shown in Figure 12, but equation that can be different according to the equation shown in from Figure 12 differently carrys out calculation exposure level.
Figure 13 is the operational flowchart being provided for inserting the method for the insertion region guidance information of virtual indirect advertising illustrated according to the embodiment of the present invention.
With reference to Figure 13, in the method for the insertion region guidance information inserting virtual indirect advertising, in step S1510, determine the candidate region by being inserted into virtual indirect advertising in video image content being provided for according to the embodiment of the present invention.
In addition, in the method for the insertion region guidance information inserting virtual indirect advertising being provided for according to the embodiment of the present invention, in step S1520, processing virtual indirect advertising and be inserted in candidate region.
In the case, in step S1520, the virtual indirect advertising that will be inserted in candidate region can be selected, virtual indirect advertising can be processed based on preliminary treatment characteristic, virtual indirect advertising can be inserted in candidate region, and treated virtual indirect advertising can be inserted in candidate region.
In the case, preliminary treatment characteristic can comprise in item below any one or more: the size of the part covered by another object of the size of the part be exposed in video image content of virtual indirect advertising, the distortion occurred based on angle, virtual indirect advertising and the speed of virtual indirect advertising movement in screen.
In addition, according to the embodiment of the present invention be provided in the method for the insertion region guidance information inserting virtual indirect advertising, in step S1530, the exposure characteristics exposed in video image content based on virtual indirect advertising measures the exposure levels of virtual indirect advertising.
In the case, exposure characteristics can be included in while virtual indirect advertising just exposes in video image content by the real-time characteristic measured in real time and preliminary treatment characteristic.
In the case, during real-time characteristic can comprise below item any one or more: the frequency that the time that virtual indirect advertising exposes in video image content, virtual indirect advertising are covered by another object and virtual indirect advertising are arranged on the position in screen.
In the case, in step S1530, proportionally can measure exposure levels with the ratio of the size of the size of the exposure area exposed in video image content of virtual indirect advertising and whole screen.
In the case, in step S1530, the time that can expose in video image content with virtual indirect advertising proportionally measures exposure levels.
In the case, in step S1530, can and difference between the angle that exposes in video image content of virtual indirect advertising and Pre-set reference angle measure exposure levels inversely.
In the case, in step S1530, the size of the part that can be covered by another object with virtual indirect advertising while virtual indirect advertising just exposes in video image content measures exposure levels inversely.
In the case, in step S1530, the frequency that can be covered by another object with virtual indirect advertising while virtual indirect advertising just exposes in video image content proportionally measures exposure levels.
In the case, in step S1530, exposure levels can be measured at the immediate vicinity at screen of virtual indirect advertising.
In the case, in step S1530, proportionally can measure exposure levels with virtual indirect advertising speed of movement in screen while virtual indirect advertising just exposes in video image content.
In the case, in step S1530, total exposure level can be calculated by collecting based on the exposure levels measured by exposure characteristics.
In addition, according to the embodiment of the present invention be provided in the method for the insertion region guidance information inserting virtual indirect advertising, to the insertion region guidance information of exposure levels of user's providing package containing virtual indirect advertising in step S1540.
In the case, in step S1540, can determine whether the exposure levels of virtual indirect advertising exceedes Pre-set reference exposure levels, and if the exposure levels of virtual indirect advertising exceedes described reference exposure level, then can to the insertion region guidance information of user's providing package containing the exposure levels of virtual indirect advertising.
In the case, in step S1540, can to user's display insert virtual indirect advertising candidate region and insert in the candidate region existed before virtual indirect advertising any one or more.
Although not display, can calculate the advertising expenditure of virtual indirect advertising in Figure 13 based on exposure levels by the method being provided for the insertion region guidance information inserting virtual indirect advertising according to the embodiment of the present invention.
In the case, in step S1540, advertising expenditure can be shown to user.
Figure 14 is the operational flowchart providing the example of the step S1540 inserting region guidance information illustrated shown in Figure 13.
With reference to Figure 14, for providing the step S1540 inserting region guidance information, in step S1610, determine whether the exposure levels of virtual indirect advertising exceedes Pre-set reference exposure levels.
In the case, reference exposure level can be absolute value or relative value.
Such as, when reference exposure level is absolute value, the value corresponding with 50% of the maximum of exposure levels can be set to reference to exposure levels.
Such as, when reference exposure level is relative value, can be set to drop into value in the scope of more than 30% of the exposure levels of the virtual indirect advertising be inserted in all candidate regions with reference to exposure levels.
There is provided in the step S1540 inserting region guidance information in fig. 13, if as the result that step S1610 determines, the exposure levels of virtual indirect advertising does not exceed reference exposure level, then skip in step S1621 inserting providing of region guidance information.
That is, according to the embodiment of the present invention be provided in the method for the insertion region guidance information inserting virtual indirect advertising, can adjust reference exposure level, and the candidate region that can have low exposure levels for some is skipped to inserting providing of region guidance information.
There is provided in the step S1540 inserting region guidance information in fig. 13, if as the result that step S1610 determines, the exposure levels of virtual indirect advertising has exceeded reference exposure level, then in step S1622, display inserts the candidate region of virtual indirect advertising.
In addition, for the step S1540 providing insertion region guidance information shown in Figure 13, in step S1630, the exposure levels of virtual indirect advertising is shown.
In addition, for the step S1540 providing insertion region guidance information shown in Figure 13, in step S1640, the advertising expenditure of virtual indirect advertising is shown.
That is, can it is seen that, according to the embodiment of the present invention be provided in the method for the insertion region guidance information inserting virtual indirect advertising, adjustment reference exposure level, then only provides some candidate regions with the exposure levels exceeding reference exposure level and inserts region guidance information.
Figure 15 is the operational flowchart of the example of the step S1520 process shown in Figure 13 being shown and inserting virtual indirect advertising.
With reference to Figure 15, for the process shown in Figure 13 and the step S1520 inserting virtual indirect advertising, select to be inserted into the virtual indirect advertising in candidate region in step S1710.
In addition, in fig. 13 shown in process and insert in the step S1520 of virtual indirect advertising, process virtual indirect advertising based on preliminary treatment characteristic, make it possible in step S1720, virtual indirect advertising is inserted in candidate region.
In the case, preliminary treatment characteristic can comprise in item below any one or more: the size of the part covered by another object of the size of the part exposed in video image content of virtual indirect advertising, the distortion occurred based on angle, virtual indirect advertising and the speed of virtual indirect advertising movement in screen.
In addition, the step S1520 of virtual indirect advertising inserts for the process shown in Figure 13, in step S1730, treated virtual indirect advertising is inserted in candidate region.
Can according to the order shown in each in Figure 13 to Figure 15, according to the order contrary with aforementioned sequence or perform in Figure 13 to Figure 15 the step shown in each concurrently.
The method being provided for the insertion region guidance information inserting virtual indirect advertising according to the present invention may be implemented as the program or smart phone application that can be performed by various computer installation.In the case, program or smart phone application can be recorded in computer-readable storage medium.Computer-readable storage medium can comprise program command, data file and data structure alone or in combination.The program command recorded in storage medium can be specifically designed for the present invention and construct, or can be known or available for the those of ordinary skill in the field of computer software.The example of computer-readable storage medium comprises all types of hardware units be specially constructed as record and execution of program instructions, such as magnetizing mediums (such as hard disk, floppy disk and tape), optical medium (such as CD (CD)-read-only memory (ROM) and digital versatile disc (DVD)), magnet-optical medium (such as soft CD, ROM, random access memory (RAM) and flash memory).The example of program command comprises such as such by the code of compiler-creating machine code and uses the executable higher-level language code of interpreter by computer.These hardware units can be constructed to as the operation of one or more software module to perform operation of the present invention, and vice versa.
Apparatus and method for using frame cluster to carry out insertion of advertising according to the present invention are not limited to structure and the method for above-mentioned execution mode, but partly or entirely can being constructed to of these execution modes is optionally combined, and makes it possible to the execution mode revised according to various mode.
The apparatus and method for eliminating the mark for inserting virtual indirect advertising are according to the embodiment of the present invention described below.
Figure 16 is the block diagram of the device for eliminating the mark for inserting virtual indirect advertising illustrated according to the embodiment of the present invention.
With reference to Figure 16, according to the embodiment of the present invention comprise marker recognition unit 2110 for the device eliminating the mark for inserting virtual indirect advertising, area determination unit 2120, mark eliminate unit 2130 and advertisement plug-in unit 2140.
Marker recognition unit 2110 identifies the mark comprised in the frame of video.
In the case, marker recognition unit 2110 can use Predistribution Algorithm or from user receive input to identify the mark comprised the frame of video.
Area determination unit 2120 is determined to mark corresponding marked region with each, the mark being inserted into the insertion region of virtual indirect advertising and usage flag region and inserting region is eliminated region.
In the case, area determination unit 2120 can use Predistribution Algorithm or from user receive input to determine marked region and insert region.
In the case, marked region can be divided into restoring area and non-restoring area by area determination unit 2120, and restoring area can be defined as mark elimination region.
In the case, area determination unit 2120 can be determined whether each marked pixels in marked region is present in and insert in region, and can be defined as restoring area by not being present in the marked pixels inserted in region.
In the case, area determination unit 2120 can be determined whether each marked pixels in marked region is present in and insert in region, and can be defined as non-restoring area by being present in the marked pixels inserted in region.
That is, because non-restoring area is the mark part being covered by virtual indirect advertising and hidden, therefore only reparation can be performed to non-overlapping region.
Therefore, and perform compared with reparation on whole mark, this can reduce reparation running cost.
In the case, area determination unit 2120 can generate the insertion borderline region comprising and form the pixel inserting border, and the part inserting borderline region can be included in restoring area.
In the case, whether each pixel that area determination unit 2120 can determine to be formed the border of inserting region is present in from the border preset distance inserting region, the region be made up of the pixel be present in preset distance can be defined as inserting borderline region, and can by considering that this insertion borderline region determines that mark eliminates region.
In the case, area determination unit 2120 can determine whether each pixel inserted in borderline region is present in marked region and inserts in region, and the pixel existed in marked region and insertion region can be included in restoring area.
In the case, area determination unit 2120 can generate the mark boundaries region comprising the pixel forming mark boundaries, and the part in mark boundaries region can be included in restoring area.
In the case, area determination unit 2120 can determine whether each pixel forming marked region border is present in from the border preset distance of marked region, the region be made up of the pixel be present in preset distance can be defined as mark boundaries region, and can by considering that this mark boundaries region determines that mark eliminates region.
In the case, area determination unit 2120 can determine whether each pixel in mark boundaries region is present in marked region and inserts in region, and the pixel be not present in marked region and insertion region can be included in restoring area.
Mark is eliminated unit 2130 and is only performed reparation to mark elimination region.
In the case, mark eliminates unit 2130 can be divided into structural images and texture image by the image corresponding with restoring area, can by performing to repair to structural images generate treated structural images according to making to be diffused into mode in restoring area close to the pixel of the periphery of restoring area along the line of equal gray value, treated texture image can be generated by performing textures synthesis to texture image, and treated structural images and treated texture image can be synthesized.
In the case, structural images can be the image of the structure only comprising the object comprised in image.That is, structural images comprises the shape, color, brightness etc. of object.
In the case, texture image can be the image of the texture only comprising the object comprised in image.That is, texture image comprises the pattern, texture etc. of object.
In the case, because human vision is responsive to attribute (such as structure, texture etc.) the comparison gray value in region, therefore, when performing reparation, the reparation based on the attribute (such as structure, texture etc.) in region can produce more natural image than the reparation based on gray value.
In the case, perform the algorithm of reparation for by the line diffusion along equi intensity curve, the structure of repairing order target area is carried out connecting produce result attractive in appearance, but shortcoming is to there will be in the picture in the veined situation of region tool fuzzy.
In the case, textures synthesis refers to the texture information generating large image from the texture information of small sample image.That is, the texture information of the large image by filling restoring area is generated according to the texture information never comprising the sample image selected in the region of restoring area.
In the case, for sample image, single sample image can be selected, then can perform textures synthesis, or multiple sample image can be selected, then can perform textures synthesis, therefore generate single large image.
In the case, mark eliminates unit 2130 can be segmented into cell block by restoring area and each not comprising in the region of restoring area, the most consistent block can be selected for each piece of restoring area in the region not comprising restoring area, and each piece of restoring area and selected block can be synthesized.
In the case, the size of block can be unfixed, but can according to circumstances change.
In the case, when selecting block, the most consistent block can be selected based on the priority of assigning or weight.In the case, higher priority can be assigned for having the openness block of the height that is different from surrounding environment.In the case, weight can be assigned based on distance.
In the case, single piece can not be selected, but can use non-local means (nonlocalmeans) obtain the weight of multiple candidate blocks and, then can perform selection.Alternatively, this plot consistency can be used to replace non-local means.
In the case, mark is eliminated unit 2130 and in internal direction, can be performed reparation from the unit area corresponding with the periphery of restoring area.
Therefore, this can perform naturally repairs than the reparation in the unit area corresponding with the inside of the restoring area not being present in the image that can be referenced in surrounding environment, and efficiency is higher, this is because when target area stops or can stop when meeting non-restoring area repairing.
After reparation mark eliminates region, virtual indirect advertising is inserted into and inserts in region by advertisement plug-in unit 2140.
Figure 17 illustrates that the mark shown in Figure 16 eliminates the block diagram of the example of unit 2130.
Eliminate unit 2130 with reference to the mark shown in Figure 17, Figure 16 and comprise separation of images unit 2210, structure repair unit 2220, textures synthesis unit 2230 and image composing unit 2240.
The image corresponding with restoring area is divided into structural images and texture image by separation of images unit 2210.
In the case, structural images can be the image of the structure only comprising the object comprised in image.That is, structural images comprises the shape, color, brightness etc. of object.
In the case, texture image can be the image of the texture only comprising the object comprised in image.That is, texture image comprises the pattern, texture etc. of object.
In the case, because human vision is responsive to attribute (such as structure, texture etc.) the comparison gray value in region, therefore, when performing reparation, the reparation based on the attribute (such as structure, texture etc.) in region can produce more natural image than the reparation based on gray value.
Therefore, reparation is performed to structure and texture.Because the algorithm of the optimum for generating the structure for region is different from the algorithm of the optimum for generating the texture for region, therefore independent best repair is performed to each in the structure in region and texture, then result is synthesized single image.In the case, can perform and more naturally repair.
Structure repair unit 2220 is by performing to repair to structural images generate treated structural images according to making to be diffused into mode in restoring area close to the pixel of the periphery of restoring area along the line of equal gray value.
In the case, perform the algorithm of reparation for by the line diffusion along equi intensity curve, the structure of repairing order target area is carried out connecting produce result attractive in appearance, but shortcoming is to there will be in the picture in the veined situation of region tool fuzzy.
Therefore, reparation can not be performed to the texture information of the image corresponding with restoring area, but can to perform reparation to its structural information.
Textures synthesis unit 2230 can generate treated texture image by performing textures synthesis to texture image.
In the case, textures synthesis refers to the texture information generating large image from the texture information of small sample image.That is, the texture information of the large image by filling restoring area is generated according to the texture information never comprising the sample image selected in the region of restoring area.
In the case, for sample image, single sample image can be selected, then can perform textures synthesis, or multiple sample image can be selected, then can perform textures synthesis, therefore generate single large image.
Therefore, reparation can not be performed to the structural information of the image corresponding with restoring area, but reparation can be performed to its texture information.
Treated structural images and treated texture image are synthesized by image composing unit 2240.
Therefore, it is possible to acquisition all performs the image of reparation to both the structure of the image corresponding with restoring area and texture.
Figure 18 illustrates that the mark shown in Figure 16 eliminates the block diagram of another example of unit 2130.
Eliminate unit 2130 with reference to the mark shown in Figure 18, Figure 16 and comprise block segmenting unit 2310, block selected cell 2320 and block synthesis unit 2330.
Restoring area and each not comprising in the region of restoring area are segmented into cell block by block segmenting unit 2310.
In the case, the size of block can be unfixed, but can according to circumstances change.
Block selected cell 2320 can select the most consistent block for each piece of restoring area in the region not comprising restoring area.
In the case, when selecting block, the most consistent block can be selected based on the priority of assigning or weight.In the case, higher priority can be assigned for having the openness block of the height that is different from surrounding environment.In the case, weight can be assigned based on distance.
In the case, single piece can not be selected, but can use non-local means to obtain the weight of multiple candidate blocks and, then can perform selection.Alternatively, this plot consistency can be used to replace non-local means.
Each piece of restoring area and selected block synthesize by block synthesis unit 2330.
Figure 19 illustrates according to the figure comprising the example of the frame of video 2400 of mark 2410 for inserting virtual indirect advertising of the present invention.
According to the mark 2410 in the device identification frame of video 2400 for eliminating the mark for inserting virtual indirect advertising of the present invention.
In the case, the device for eliminating the mark for inserting virtual indirect advertising according to the present invention can use Predistribution Algorithm or carry out identification mark 2410 from the input that user receives.
In the case, frame of video 2400 is included in the mark 2410 that spot for photography directly indicates.
That is, mark 2410 is directly indicated in spot for photography, and is the instruction that easily can distinguish with surrounding environment.
Such as, marking 2410 can be the band comprising particular color or pattern.
Figure 20 illustrates according to the figure inserting the example of the frame of video 2400 of virtual indirect advertising 2510 of the present invention.
Device for eliminating the mark for inserting virtual indirect advertising according to the present invention determines the insertion region by being inserted into virtual indirect advertising 2510 based on mark 2410.
In the case, the device for eliminating the mark for inserting virtual indirect advertising according to the present invention can use Predistribution Algorithm or determine the insertion region by being inserted into virtual indirect advertising 2510 from the input that user receives.
Such as, be defined as inserting region according to the region that the central point mark with 2410 can be the shape on its summit by the device for eliminating the mark for inserting virtual indirect advertising of the present invention, and virtual indirect advertising 2510 can be inserted in insertion region.
Figure 21 illustrates insert virtual indirect advertising 2510 according to of the present invention and eliminate the figure of the example of the frame of video 2400 of mark 2410.
Once determine the insertion region by being inserted into virtual indirect advertising 2510 based on mark 2410, just using to repair according to the device for eliminating the mark for inserting virtual indirect advertising of the present invention and eliminating mark 2410 from frame of video 2400.
As mentioned above, with reference to Figure 19 to Figure 21, can find out, device for eliminating the mark for inserting virtual indirect advertising according to the present invention uses insert district field mark 2410 to insert the virtual indirect advertising 2510 determined, and eliminates mark 2410 based on the frame of video 2400 being included in the mark 2410 that spot for photography directly indicates.
Figure 22 illustrates according to marked region according to the present invention 2710 and the figure of example inserting region 2720.
Marked region 2710 is corresponding with the mark 2410 shown in Figure 19.
Insertion region 2720 is the regions by being inserted into the virtual indirect advertising 2510 shown in Figure 20.
Determine according to marked region 2710 of the present invention by the area determination unit 2120 shown in Figure 16 and insert region 2720.
In the case, the region be made up of the pixel of the mark comprising identification can be defined as marked region 2710 by the area determination unit 2120 shown in Figure 16.
In the case, can use Predistribution Algorithm or from user receive input to determine marked region 2710.
In addition, the area determination unit 2120 shown in Figure 16 can will use the mark identified and the region calculated is defined as inserting region 2720.
Such as, to insert region 2720 can be the central point of the mark having to identify is the region of the shape on its summit.
In the case, Predistribution Algorithm can be used or determine to insert region 2720 from the input that user receives.
In the case, can find out, marked region 2710 is divided into the part overlapping with inserting region 2720 and non-overlapping part.
In the case, marked region 2710 and insert region that region 2720 overlaps each other by be inserted into or the virtual indirect advertising that will insert cover and hide, only can perform reparation to non-overlapping region.
Below in Figure 23 and Figure 24, will describe in detail for only performing the method for repairing to a part for marked region 2710.
Figure 23 is the figure of the example illustrated according to restoring area 2711 of the present invention.
With reference to Figure 23, can find out, restoring area 2711 according to the present invention is parts of the marked region 2710 shown in Figure 22.
Can be determined according to restoring area 2711 of the present invention by the area determination unit 2120 shown in Figure 16.
Marked region 2710 can be divided into restoring area 2711 and non-restoring area 2712 by the area determination unit 2120 shown in Figure 16, and restoring area 2711 can be defined as mark elimination region.
In the case, area determination unit 2120 can be determined whether each marked pixels in marked region 2710 is present in and insert in region 2720, and can be defined as restoring area 2711 by not being present in the marked pixels inserted in region 2720.
In the case, area determination unit 2120 can be determined whether each marked pixels in marked region 2710 is present in and insert in region 2720, and can be defined as non-restoring area 2712 by being present in the marked pixels inserted in region 2720.
That is, because non-restoring area is the mark part being covered by virtual indirect advertising and hidden, therefore can only repair non-overlapping region.
Therefore, and perform compared with reparation on whole mark, this can reduce reparation running cost.
Figure 24 is the figure of another example illustrated according to restoring area of the present invention.
With reference to Figure 24, can find out, restoring area 2711 according to the present invention is parts of the marked region 2710 shown in Figure 22.
In addition, different from the restoring area shown in Figure 23, can find out, restoring area 2711 according to the present invention comprises the part inserting region 2720.
Can be determined according to restoring area 2711 of the present invention by the area determination unit 2120 shown in Figure 16.
Area determination unit 2120 shown in Figure 16 can generate the insertion borderline region 2725 comprising and form the pixel inserting border, and the part inserting borderline region 2725 can be included in restoring area 2711.
In the case, whether each pixel that the area determination unit 2120 shown in Figure 16 can determine to be formed the border of inserting region 2720 is present in from the border preset distance inserting region 2720, the region be made up of the pixel be present in preset distance can be defined as inserting borderline region 2725, and can by considering that inserting borderline region 2725 determines that mark eliminates region.
In the case, area determination unit 2120 shown in Figure 16 can determine whether each pixel inserted in borderline region 2725 is present in marked region 2710 and inserts in region 2720, and the pixel existed in marked region 2710 and insertion region 2720 can be included in restoring area 2711.
As mentioned above, can by carrying out restoring area 2711 according to the present invention to expand the vestige more effectively eliminated inserting the mark that the border surface in region 2720 may occur.
Although do not illustrate in Figure 24, restoring area 2711 can be expanded by the area determination unit 2120 shown in Figure 16, to perform reparation to the intended pixel outside mark boundaries.
Such as, the area determination unit 2120 shown in Figure 16 can generate the mark boundaries region comprising the pixel forming mark boundaries, and the part in mark boundaries region can be included in restoring area 2711.
In the case, area determination unit 2120 shown in Figure 16 can determine whether each pixel on the border forming marked region 2710 is present in from the border preset distance of marked region 2710, the region be made up of the pixel be present in preset distance can be defined as mark boundaries region, and can by considering that this mark boundaries region determines that mark eliminates region.
In the case, area determination unit 2120 shown in Figure 16 can determine whether each pixel in mark boundaries region is present in marked region 2710 and inserts in region 2720, and the pixel be not present in marked region 2710 and insertion region 2720 can be included in restoring area 2711.
As mentioned above, can by carrying out restoring area 2711 according to the present invention to expand the vestige more effectively eliminated inserting the mark that the border surface in region 2710 may occur.
Figure 25 illustrates that elimination is according to the embodiment of the present invention for inserting the operational flowchart of the method for the mark of virtual indirect advertising.
With reference to Figure 25, in elimination according to the embodiment of the present invention for inserting in the method for the mark of virtual indirect advertising, in step S2010, identify the mark comprised in the frame of video.
In the case, in step S2010, can use Predistribution Algorithm or from user receive input to identify the mark comprised the frame of video.
In addition, in elimination according to the embodiment of the present invention for inserting in the method for the mark of virtual indirect advertising, in step S2020, determine to mark corresponding marked region with each, the mark being inserted into the insertion region of virtual indirect advertising and usage flag region and inserting region is eliminated region.
In the case, in step S2020, can use Predistribution Algorithm or from user receive input to determine marked region and insert region.
In the case, in step S2020, marked region can be divided into restoring area and non-restoring area, and restoring area can be defined as mark elimination region.
In the case, in step S2020, can determine whether each marked pixels in marked region is present in insertion region, and restoring area can be defined as by not being present in the marked pixels inserted in region.
In the case, in step S2020, can determine whether each marked pixels in marked region is present in insertion region, and non-restoring area can be defined as by being present in the marked pixels inserted in region.
That is, because non-restoring area is the mark part being covered by virtual indirect advertising and hidden, therefore can only repair non-overlapping region.
Therefore, and perform compared with reparation on whole mark, this can reduce reparation running cost.
In the case, in step S2020, the insertion borderline region comprising and form the pixel inserting border can be generated, and the part inserting borderline region can be included in restoring area.
In the case, in step S2020, can determine whether form each pixel inserting zone boundary is present in from the border preset distance inserting region, the region be made up of the pixel be present in preset distance can be defined as inserting borderline region, and can by considering that inserting borderline region determines that mark eliminates region.
In the case, in step S2020, can determine whether each pixel inserted in borderline region is present in marked region and inserts in region, and the pixel existed in marked region and insertion region can be included in restoring area.
In the case, in step S2020, the mark boundaries region comprising the pixel forming mark boundaries can be generated, and the part in mark boundaries region can be included in restoring area.
In the case, in step S2020, can determine whether each pixel forming marked region border is present in from the border preset distance of marked region, the region be made up of the pixel be present in preset distance can be defined as mark boundaries region, and can by considering that mark boundaries region determines that mark eliminates region.
In the case, in step S2020, can determine whether each pixel in mark boundaries region is present in marked region and inserts in region, and the pixel be not present in marked region and insertion region can be included in restoring area.
In addition, in elimination according to the embodiment of the present invention for inserting in the method for the mark of virtual indirect advertising, in step S2030, only region being eliminated to mark perform reparation.
In the case, in step S2030, the image corresponding with restoring area can be divided into structural images and texture image, can by performing to repair to structural images generate treated structural images according to making to be diffused into mode in restoring area close to the pixel of the periphery of restoring area along the line of equal gray value, treated texture image can be generated by performing textures synthesis to texture image, and treated structural images and treated texture image can be synthesized.
In the case, structural images can be the image of the structure only comprising the object comprised in image.That is, structural images comprises the shape, color, brightness etc. of object.
In the case, texture image can be the image of the texture only comprising the object comprised in image.That is, texture image comprises the pattern, texture etc. of object.
In the case, because human vision is responsive to attribute (such as structure, texture etc.) the comparison gray value in region, therefore, when performing reparation, the reparation based on the attribute (such as structure, texture etc.) in region can produce more natural image than the reparation based on gray value.
In the case, perform the algorithm of reparation for by the line diffusion along equi intensity curve, the structure of repairing order target area is carried out connecting produce result attractive in appearance, but shortcoming is to there will be in the picture in the veined situation of region tool fuzzy.
In the case, textures synthesis refers to the texture information generating large image from the texture information of small sample image.That is, the texture information of the large image by filling restoring area is generated according to the texture information never comprising the sample image selected in the region of restoring area.
In the case, for sample image, single sample image can be selected, then can perform textures synthesis, or multiple sample image can be selected, then can perform textures synthesis, therefore generate single large image.
In the case, in step S2030, restoring area and each not comprising in the region of restoring area can be segmented into cell block, the most consistent block can be selected for each piece of restoring area in the region not comprising restoring area, and each piece of restoring area and the block selected can be synthesized.
In the case, the size of block can be unfixed, but can according to circumstances change.
In the case, when selecting block, the most consistent block can be selected based on the priority of assigning or weight.In the case, higher priority can be assigned for having the openness block of the height that is different from surrounding environment.In the case, weight can be assigned based on distance.
In the case, single piece can not be selected, but can use non-local means to obtain the weight of multiple candidate blocks and, then can perform selection.Alternatively, this plot consistency can be used to replace non-local means.
In the case, in step S2030, in internal direction, reparation can be performed from the unit area corresponding with the periphery of restoring area.
Therefore, this can perform naturally repairs than the reparation in the unit area corresponding with the inside of the restoring area not being present in the image that can be referenced in surrounding environment, and efficiency is higher, this is because when target area stops or can stop when meeting non-restoring area repairing.
In addition, in elimination according to the embodiment of the present invention for inserting in the method for the mark of virtual indirect advertising, in step 2040, after reparation mark eliminates region, virtual indirect advertising being inserted into and inserting in region.
According to the order shown in Figure 25, the order contrary with aforementioned sequence or the step shown in Figure 25 can be performed concurrently.
Elimination according to the present invention may be implemented as the program or smart phone application that can be performed by various computer installation for the method for the mark inserting virtual indirect advertising.In the case, program or smart phone application can be recorded in computer-readable storage medium.Computer-readable storage medium can comprise program command, data file and data structure alone or in combination.The program command recorded in storage medium can be specifically designed for the present invention and construct, or can be known or available for the those of ordinary skill in the field of computer software.The example of computer-readable storage medium comprises all types of hardware units be specially constructed as record and execution of program instructions, such as magnetizing mediums (such as hard disk, floppy disk and tape), optical medium (such as CD (CD)-read-only memory (ROM) and digital versatile disc (DVD)), magnet-optical medium (such as soft CD, ROM, random access memory (RAM) and flash memory).The example of program command comprises such as such by the code of compiler-creating machine code and uses the executable higher-level language code of interpreter by computer.These hardware units can be constructed to as the operation of one or more software module to perform operation of the present invention, and vice versa.
Apparatus and method for eliminating the mark for inserting virtual indirect advertising according to the present invention are not limited to structure and the method for above-mentioned execution mode, but partly or entirely can being constructed to of these execution modes is optionally combined, and makes it possible to the execution mode revised according to various mode.
Virtual indirect advertising service unit according to the embodiment of the present invention and method are described below.
Figure 26 is the block diagram of the virtual indirect advertising service unit illustrated according to the embodiment of the present invention.
With reference to Figure 26, virtual indirect advertising service unit according to the embodiment of the present invention comprises advertisement selecting unit 3110, restoring area generation unit 3120, repairs unit 3130 and advertisement plug-in unit 3140.
Advertisement selecting unit 3110 is selected to be inserted into the virtual indirect advertising in video image content.
Restoring area generation unit 3120 generates restoring area and non-restoring area by the existing advertisement comprised in virtual indirect advertising and video image content being compared.
In the case, restoring area generation unit 3120 can generate the target area in the region comprising existing advertisement, and can generate restoring area and non-restoring area by dividing target area.
In the case, target area is the region in the region comprising existing advertisement, and can use Predistribution Algorithm or arrange from the input that user receives.
In the case, non-restoring area is the region be completely covered when inserting virtual indirect advertising.In the case, non-restoring area can be less than or equal to the region of the virtual indirect advertising that will be inserted into.
In the case, restoring area can be the region obtained by removing non-restoring area from target area.
That is, to be covered by virtual indirect advertising due to non-restoring area and hide, therefore can only repair non-overlapping region.
Therefore, and perform compared with reparation on whole mark, this can reduce reparation running cost.
Repair unit 3130 and only reparation is performed to restoring area.
In the case, repair unit 3130 and in internal direction, reparation can be performed from the unit area corresponding with the periphery of restoring area.
Therefore, this can perform naturally repairs than the reparation in the unit area corresponding with the inside of the restoring area not being present in the image that can be referenced in surrounding environment, and efficiency is higher, this is because when target area stops or can stop when meeting non-restoring area repairing.
After reparation mark eliminates region, virtual indirect advertising is inserted in video image content by advertisement plug-in unit 3140.
Figure 27 is the figure of the example of the restoring area illustrated according to the embodiment of the present invention.
With reference to Figure 27, the video image content inserting virtual indirect advertising according to the embodiment of the present invention comprises existing advertising area 3410, target area 3420, virtual indirect advertising 3430 and non-restoring area 3440.
Generate the target area 3420 comprising existing advertising area 3410.
In the case, can use Predistribution Algorithm or from user receive input to determine target area 3420.
In the case, target area 3420 can be generated, not to be connected with the shape of existing advertising area 3410 or to comprise existing advertising area 3410 while keeping the shape of existing advertising area 3410.
Non-restoring area 3440 is the regions be completely covered when inserting virtual indirect advertising 3430.In the case, non-restoring area 3440 can be equal to or less than the region of the virtual indirect advertising be inserted into.
Restoring area can be the region obtained by removing non-restoring area 3440 from target area 3420.
With perform compared with reparation on whole target area 3420, only reparation is performed to restoring area and can reduce reparation running cost.
Figure 28 is the figure inserting the example of the video image content of virtual indirect advertising illustrated according to the embodiment of the present invention.
With reference to Figure 28, the video image content inserting virtual indirect advertising according to the embodiment of the present invention comprises restoring area 3510 and virtual indirect advertising 3520.
Restoring area 3510 is the regions generated by the existing advertisement comprised in virtual indirect advertising 3520 and video image content being compared.
In the case, restoring area 3510 and 3511 can be separated into two or more discontinuous regions.
Generate the target area comprising existing advertising area.
In the case, Predistribution Algorithm can be used or come Offered target region from the input that user receives.
In the case, target area can be generated, not to be connected with the shape of existing advertising area or to comprise existing advertising area while keeping the shape of existing advertising area.
Non-restoring area is the region be completely covered when inserting virtual indirect advertising 3520.In the case, non-restoring area can be equal to or less than the region of the virtual indirect advertising 3520 be inserted into.
Restoring area 3510 and 3511 is the regions obtained by removing non-restoring area from target area.
With perform compared with reparation on whole target area, only reparation is performed to restoring area 3510 and 3511 and can reduce reparation running cost.
Figure 29 is the operational flowchart of the example of the virtual indirect advertising method of servicing illustrated according to the embodiment of the present invention.
With reference to Figure 29, in virtual indirect advertising method of servicing according to the embodiment of the present invention, select to be inserted into the virtual indirect advertising in video image content in step S3610.
In addition, in virtual indirect advertising method of servicing according to the embodiment of the present invention, in step S3620, restoring area and non-restoring area is generated by the existing advertisement comprised in virtual indirect advertising and video image content being compared.
In the case, in the step 3120 generating restoring area and non-restoring area, the target area comprising existing advertising area can be generated, and restoring area and non-restoring area can be generated by dividing target area.
In the case, target area is the region comprising existing advertising area, and can use Predistribution Algorithm or arrange from the input that user receives.
In the case, non-restoring area is the region be completely covered when inserting virtual indirect advertising.In the case, non-restoring area can be equal to or less than the region of the virtual indirect advertising be inserted into.
In the case, restoring area can be the region obtained with removing non-restoring area from target area.
That is, to be covered by virtual indirect advertising due to non-restoring area and hide, therefore can only repair non-overlapping region.
Therefore, and perform compared with reparation on whole target area, this can reduce reparation running cost.
In the case, non-restoring area can be belong to target area and the region overlapping with the region of virtual indirect advertising.
Because the overlapping part between target area and virtual indirect advertising is covered by virtual indirect advertising, therefore it can be generated as the non-restoring area not performing reparation.
In addition, in virtual indirect advertising method of servicing according to the embodiment of the present invention, in step S3630, only reparation is performed to restoring area.
In the case, performing in the step of repairing, in internal direction, reparation can be performed from the unit area corresponding with the periphery of restoring area.
Therefore, this can perform naturally repairs than the reparation in the unit area corresponding with the inside of the restoring area not being present in the image that can be referenced in surrounding environment, and efficiency is higher, this is because when target area stops or can stop when meeting non-restoring area repairing.
In addition, in virtual indirect advertising method of servicing according to the embodiment of the present invention, in step S3640, after reparation mark eliminates region, virtual indirect advertising is inserted in video image content.
Figure 30 illustrates the operational flowchart of the example of the step that the execution shown in Figure 29 is repaired.
With reference to Figure 30, for the step that the execution shown in Figure 29 is repaired, in step S3710, the image corresponding with restoring area is divided into structural images and texture image.
In the case, structural images can be the image of the structure only comprising the object comprised in image.That is, structural images comprises the shape, color, brightness etc. of object.
In the case, texture image can be the image of the texture only comprising the object comprised in image.That is, texture image comprises the pattern, texture etc. of object.
Because human vision is responsive to attribute (such as structure, texture etc.) the comparison gray value in region, therefore, when performing reparation, the reparation based on the attribute (such as structure, texture etc.) in region can produce more natural image than the reparation based on gray value.
Therefore, reparation is performed to structure and texture.Because the algorithm of the optimum for formation zone structure is different from the algorithm of the optimum for formation zone texture, therefore independent best repair is performed to each in the structure in region and texture, then result is synthesized single image.In the case, can perform and more naturally repair.
In addition, in the step that the execution shown in Figure 29 is repaired, by performing to repair to structural images generate treated structural images according to making to be diffused into mode in restoring area close to the pixel of the periphery of restoring area along the line of equal gray value.
In the case, perform the algorithm of reparation for by the line diffusion along equi intensity curve, the structure of repairing order target area is carried out connecting produce result attractive in appearance, but shortcoming is to there will be in the picture in the veined situation of region tool fuzzy.
Therefore, reparation can not be performed to the texture information of the image corresponding with restoring area, but can to perform reparation to its structural information.
In addition, in the step that the execution shown in Figure 29 is repaired, treated texture image can be generated by performing textures synthesis to texture image.
In the case, textures synthesis refers to the texture information generating large image from the texture information of small sample image.That is, the texture information of the large image by filling restoring area is generated according to the texture information never comprising the sample image selected in the region of restoring area.
In the case, for sample image, single sample image can be selected, then can perform textures synthesis, or multiple sample image can be selected, then can perform textures synthesis, therefore generate single large image.
Therefore, reparation can not be performed to the structural information of the image corresponding with restoring area, but reparation can be performed to its texture information.
In addition, for the step that the execution shown in Figure 29 is repaired, in step S3740, treated structural images and treated texture image are synthesized.
Therefore, the image that both enough structures to the image corresponding with restoring area and texture all perform reparation can be obtained.
Figure 31 is the operational flowchart of another example that the step that the execution shown in Figure 29 is repaired is shown.
With reference to Figure 31, for the step that the execution shown in Figure 29 is repaired, in step S3810, restoring area and each not comprising in the region of restoring area are segmented into cell block.
In the case, the size of block can be unfixed, but can according to circumstances change.
For the step that the execution shown in Figure 29 is repaired, in step S3820, the most consistent block can be selected for each piece of restoring area in the region not comprising restoring area.
In the case, when selecting block, the most consistent block can be selected based on the priority of assigning or weight.In the case, higher priority can be assigned for having the openness block of the height that is different from surrounding environment.In the case, weight can be assigned based on distance.
In the case, single piece can not be selected, but can use non-local means to obtain the weight of multiple candidate blocks and, then can perform selection.Alternatively, this plot consistency can be used to replace non-local means.
For the step that the execution shown in Figure 29 is repaired, in step S3830, each piece of restoring area and selected block are synthesized.
According to the order shown in each in Figure 29, Figure 30 and 31, the order contrary with aforementioned sequence or can perform in Figure 29, Figure 30 and 31 the step shown in each concurrently.
Virtual indirect advertising method of servicing according to the present invention may be implemented as the program or smart phone application that can be performed by various computer installation.In the case, program or smart phone application can be recorded in computer-readable storage medium.Computer-readable storage medium can comprise program command, data file and data structure alone or in combination.The program command recorded in storage medium can be specifically designed for the present invention and construct, or can be known or available for the those of ordinary skill in the field of computer software.The example of computer-readable storage medium comprises all types of hardware units be specially constructed as record and execution of program instructions, such as magnetizing mediums (such as hard disk, floppy disk and tape), optical medium (such as CD (CD)-read-only memory (ROM) and digital versatile disc (DVD)), magnet-optical medium (such as soft CD, ROM, random access memory (RAM) and flash memory).The example of program command comprises such as such by the code of compiler-creating machine code and uses the executable higher-level language code of interpreter by computer.These hardware units can be constructed to as the operation of one or more software module to perform operation of the present invention, and vice versa.
Structure and the method for above-mentioned execution mode is not limited to according to virtual indirect advertising method of servicing of the present invention or device, but partly or entirely can being constructed to of these execution modes is optionally combined, and makes it possible to the execution mode revised according to various mode.
The apparatus and method for calculating the boundary value for inserting virtual indirect advertising are according to the embodiment of the present invention described below.
Figure 32 illustrates that calculating is according to the embodiment of the present invention for inserting the block diagram of the device of the boundary value of virtual indirect advertising.
With reference to Figure 32, according to the embodiment of the present invention comprise area determination unit 4110, reference pixel selected cell 4120 and boundary value computing unit 4130 for the device calculating the boundary value for inserting virtual indirect advertising.
Area determination unit 4110 determines to comprise the borderline region of the boundary pixel of the virtual indirect advertising be inserted in the frame of video.
In the case, area determination unit 4110 can determine whether each pixel of the frame of video is present in from boundary pixel preset distance, and the region be made up of the pixel be present in preset distance can be defined as borderline region.
In the case, area determination unit 4110 can by considering that the overall resolution of frame of video arranges preset distance.
In the case, area determination unit 4110 can arrange longer preset distance for the overall resolution of higher frame of video.
In the case, area determination unit 4110 proportionally can arrange preset distance with the overall resolution of frame of video.
Such as, preset distance can be set to 4 pixels when the overall resolution of frame of video is 1280*4720 by area determination unit 4110, and preset distance can be set to 6 pixels when the overall resolution of frame of video is 1920*1080.
In the case, the remaining area except borderline region can be divided into the foreground area corresponding with virtual indirect advertising and background area by area determination unit 4110.
Such as, area determination unit 4110 can will not comprise borderline region and the remaining area inserting virtual indirect advertising is defined as foreground area, and the region obtained by getting rid of borderline region and foreground area from frame of video can be defined as background area.
As mentioned above, the pixel of surrounding virtual indirect advertising insertion boundary line can be appointed as borderline region according to the device for calculating the boundary value for inserting virtual indirect advertising of the present invention, and the value of borderline region pixel can be recalculated, therefore make it possible to naturally to watch virtual indirect advertising and insert border surface.
For each borderline region pixel in borderline region, reference pixel selected cell 4120 is by considering that prospect reference pixel and reference background pixel are selected in the position of borderline region pixel from the remaining area except borderline region.
In the case, reference pixel selected cell 4120 can select in the pixel of foreground area near of borderline region pixel as prospect reference pixel.
In addition, reference pixel selected cell 4120 can select in the pixel of background area near one of borderline region pixel reference pixel as a setting.
In the case, prospect reference pixel is the pixel of foreground area, and reference background pixel is the pixel of background area.
In the case, reference pixel selected cell 4120 can generate the vertical line between borderline region pixel and the boundary line of remaining area.
Such as, reference pixel selected cell 4120 can draw the vertical line of the boundary line from borderline region pixel to foreground area.
In the case, reference pixel selected cell 4120 can select in the pixel of foreground area near borderline region pixel and one corresponding with the foot (foot) of vertical line as prospect reference pixel.
In addition, reference pixel selected cell 4120 can draw the vertical line of the boundary line from borderline region pixel to background area.
In the case, reference pixel selected cell 4120 can select in the pixel of background area near borderline region pixel and corresponding with the foot of vertical line reference pixel as a setting.
As mentioned above, for the pixel of surrounding virtual indirect advertising insertion boundary line, device for calculating the boundary value for inserting virtual indirect advertising according to the present invention recalculates the value of borderline region pixel by referring to the pixel value of background and prospect, therefore makes it possible to naturally to watch virtual indirect advertising and inserts border surface.
In the case, for each borderline region pixel in borderline region, reference pixel selected cell 4120 is by considering that former frame reference pixel is selected in the position of borderline region pixel from the former frame of frame of video, and a frame reference pixel after selecting from a rear frame of this frame of video.
As mentioned above, according to the device for calculating the boundary value for inserting virtual indirect advertising of the present invention by not only inserting the pixel of boundary line with reference to the virtual indirect advertising of encirclement, and recalculate the pixel value of present frame with reference to the pixel value of former frame/rear frame, therefore make it possible to naturally to watch virtual indirect advertising at the playback of video and insert border surface.
In the case, reference pixel selected cell 4120 can select the pixel corresponding with the position of borderline region pixel as former frame reference pixel from former frame, and can from a rear frame, select the pixel corresponding with the position of borderline region pixel as a rear frame reference pixel.
The pixel value of boundary value computing unit 4130 prospect of the application reference pixel and the weight of the pixel value of reference background pixel and the weight of prospect reference pixel and reference background pixel carry out the value of computation bound area pixel.
In the case, boundary value computing unit 4130 can as the value of the shown computation bound area pixel of equation 2 below.
Equation 2
A＝w f*A f+w b*A b
Wherein, A is the value of borderline region pixel, A ffor prospect reference pixel value, A bfor reference background pixel value, w ffor prospect reference pixel weight, and w bfor reference background pixel weight.
In the case, boundary value computing unit 4130 can arrange the weight of prospect reference pixel based on the distance between borderline region pixel and prospect reference pixel, and can arrange the weight of reference background pixel based on the distance between borderline region pixel and reference background pixel.
In the case, boundary value computing unit 4130 can arrange the weight of prospect reference pixel and the weight of reference background pixel inversely with the distance to borderline region pixel.
In the case, boundary value computing unit 4130 can arrange weight, makes the weight of prospect reference pixel and the weight sum of reference background pixel be 1.
Such as, distance between borderline region pixel and prospect reference pixel is 1, and when the distance between borderline region pixel and reference background pixel is 3, the weight of prospect reference pixel can be set to 0.75 by boundary value computing unit 4130, and the weight of reference background pixel is set to 0.25.
As mentioned above, for the pixel of surrounding virtual indirect advertising insertion boundary line, device for calculating the boundary value for inserting virtual indirect advertising according to the present invention recalculates the value of borderline region pixel, therefore, it is possible to naturally watch virtual indirect advertising to insert border surface by referring to the pixel value of background and prospect.
In the case, boundary value computing unit 4130 can use the weight of the pixel value of former frame reference pixel and the rear pixel value of a frame reference pixel and the weight of former frame reference pixel and a rear frame reference pixel to carry out the value of computation bound area pixel.
In the case, boundary value computing unit 4130 can as the value of the shown computation bound area pixel of equation 3 below.
Equation 3
A＝w t-1*A(t-1)+w t*A(t)+w t+1*A(t-1),A(t)＝w f,t*A f,t+w b,t*A b,t
Wherein, A is the value of ultimate bound area pixel, and A (t) is reference frame reference pixel value, the value of the borderline region pixel that A (t-1) is former frame, and A (t+1) is the value of the borderline region pixel of a rear frame, w tfor the weight of reference frame reference pixel, w t-1for the weight of former frame reference pixel, w t+1for the weight of a rear frame reference pixel, A f,tfor the prospect reference pixel value of reference frame, A b,tfor the reference background pixel value of reference frame, w f,tfor the prospect reference pixel weight of reference frame, and w b,tfor the reference background pixel weight of reference frame.
In the case, boundary value computing unit 4130 can arrange weight, makes the weight of former frame reference pixel, the rear weight of a frame reference pixel and the weight sum of reference frame reference pixel be 1.
Such as, distance between borderline region pixel and prospect reference pixel is 1, and when the distance between borderline region pixel and reference background pixel is 3, the weight of prospect reference pixel can be set to 0.75 by boundary value computing unit 4130, the weight of reference background pixel is set to 0.25, the weight of former frame reference pixel is set to 0.25, the weight of a rear frame reference pixel is set to 0.25, and is set to 0.5 with reference to the weight of frame reference pixel.
As mentioned above, according to the device for calculating the boundary value for inserting virtual indirect advertising of the present invention by not only inserting the pixel of boundary line with reference to the virtual indirect advertising of encirclement, and recalculate the pixel value of present frame with reference to the pixel value of former frame/rear frame, therefore, it is possible to naturally watch virtual indirect advertising at the playback of video to insert border surface.
Figure 33 is the block diagram of the example that the reference pixel selected cell 4120 shown in Figure 32 is shown.
Vertical line generation unit 4210, prospect reference pixel selected cell 4220 and reference background pixel selection unit 4230 is comprised with reference to the reference pixel selected cell 4120 shown in Figure 33, Figure 32.
Vertical line generation unit 4210 can draw the vertical line of the boundary line from borderline region pixel to foreground area.
Prospect reference pixel selected cell 4220 can select in the pixel of foreground area near borderline region pixel and corresponding with the foot of vertical line one as prospect reference pixel.
Reference background pixel selection unit 4230 can select in the pixel of background area near borderline region pixel and corresponding with the foot of vertical line reference pixel as a setting.
That is, can find out, the reference pixel selected cell 4120 shown in Figure 32 uses the foot of vertical line to select prospect reference pixel and reference background pixel.
Although do not illustrate in Figure 33, but the reference pixel selected cell 4120 shown in Figure 32 can select the pixel corresponding with the position of borderline region pixel as former frame reference pixel from former frame, and can from a rear frame, select the pixel corresponding with the position of borderline region pixel as a rear frame reference pixel.
That is, can find out, the reference pixel selected cell 4120 shown in Figure 32 not only selects prospect reference pixel and reference background pixel, and selects former frame reference pixel and a rear frame reference pixel.
Figure 34 illustrates according to the schematic diagram inserting the example of the frame of video 4300 of virtual indirect advertising 4310 of the present invention.
Device for calculating the boundary value for inserting virtual indirect advertising 4310 according to the present invention identifies the boundary pixel of virtual indirect advertising 4310 in the frame of video 4300 inserting virtual indirect advertising 4310, and determines the borderline region 4320 comprising boundary pixel.
In the case, frame of video 4300 comprises virtual indirect advertising 4310.
That is, can find out, the device for calculating the boundary value for inserting virtual indirect advertising 4310 according to the present invention calculates the pixel value around the boundary line inserting virtual indirect advertising 4310 after the step inserting virtual indirect advertising 4310.
In the case, borderline region 4320 can be to form from the pixel in the preset distance of boundary line by being present in and inserting the region of virtual indirect advertising 4310.
In the case, borderline region 4320 is intended to make user can naturally watch the boundary line inserting virtual indirect advertising 4310, and therefore can by considering that frame of video 4300 overall resolution determines the width of borderline region 4320.
Such as, when the overall resolution of frame of video 4300 is 1280*4720, borderline region 4320 has the width of 9 pixels, and when the overall resolution of frame of video 4300 is 1920*1080, borderline region 4320 has the width of 13 pixels.
Figure 35 is the schematic diagram of the example illustrated according to borderline region of the present invention, foreground area and background area.
With reference to Figure 35, can find out, Figure 35 shows the boundary line of the virtual indirect advertising 4310 shown in Figure 34 and the part expansion of borderline region 4320.
Device for calculating the boundary value for inserting virtual indirect advertising 4310 according to the present invention will belong to the region not comprising borderline region 4320 and the region corresponding with virtual indirect advertising 4310 is defined as foreground area 4415, and the region corresponding with remainder is defined as background area 4425, and this remainder is corresponding with the background inserted before virtual indirect advertising 4310.
In the case, the device for calculating the boundary value for inserting virtual indirect advertising 4310 according to the present invention selects reference pixel from the boundary line 4410 of foreground area and the boundary line 4420 of background area.
That is, can find out, device for calculating the boundary value for inserting virtual indirect advertising 4310 according to the present invention is defined as borderline region 4320 by surrounding the region inserting the boundary line of virtual indirect advertising 4310, and carrys out the pixel value in computation bound region 4320 according to the pixel forming the boundary line 4410 of foreground area and the boundary line 4420 of background area be positioned at beyond borderline region 4320.
Figure 36 is the schematic diagram that the example selecting prospect reference pixel and reference background pixel according to the present invention is shown.
With reference to Figure 36, for each borderline region pixel 4500 in borderline region, the device for calculating the boundary value for inserting virtual indirect advertising according to the present invention is by considering that the position of borderline region pixel 4500 is selected prospect reference pixel 4510 and reference background pixel 4520 respectively from foreground area boundary line 4410 and boundary line, background area 4420.
In the case, the device for calculating the boundary value for inserting virtual indirect advertising according to the present invention can select in the pixel of foreground area boundary line 4410 near of borderline region pixel 4500 as prospect reference pixel 4510.
In addition, the device for calculating the boundary value for inserting virtual indirect advertising according to the present invention can select in the pixel of boundary line, background area 4420 near one of borderline region pixel 4500 reference pixel 4520 as a setting.
In the case, the device for calculating the boundary value for inserting virtual indirect advertising according to the present invention can generate the vertical line between borderline region pixel 4500 and foreground area boundary line 4410 and the vertical line between borderline region pixel 4500 and boundary line, background area 4420.
In the case, the device for calculating the boundary value for inserting virtual indirect advertising according to the present invention can select in the pixel of foreground area boundary line 4410 near borderline region pixel 4500 and corresponding with the foot of vertical line one as prospect reference pixel 4510.
In addition, the device for calculating the boundary value for inserting virtual indirect advertising according to the present invention can select in the pixel of boundary line, background area 4420 near borderline region pixel 4500 and corresponding with the foot of vertical line reference pixel 4520 as a setting.
That is, can find out, the reference pixel selected cell 4120 shown in Figure 32 uses the foot of vertical line to select prospect reference pixel and reference background pixel.
With reference to Figure 36, can find out, the device for calculating the boundary value for inserting virtual indirect advertising according to the present invention uses the foot of vertical line to select prospect reference pixel and reference background pixel.
Figure 37 is the schematic diagram that the example selecting former frame reference pixel and a rear frame reference pixel according to the present invention is shown.
With reference to Figure 37, for the borderline region pixel 4500 in borderline region, device for calculating the boundary value for inserting virtual indirect advertising according to the present invention is by considering that former frame reference pixel 4615 is selected in the position of borderline region pixel 4500 from the former frame 4610 of the present frame 4600 of video, and a frame reference pixel 4625 after selecting from a rear frame 4620 of the present frame 4600 of video.
In the case, can from former frame 4610, select the pixel corresponding with the position of borderline region pixel 4500 as former frame reference pixel 4615 according to the device for calculating the boundary value for inserting virtual indirect advertising of the present invention, and from a rear frame 4620, select the pixel corresponding with the position of borderline region pixel 4500 as a rear frame reference pixel 4625.
Figure 38 illustrates according to the calculating of embodiment of the present invention for inserting the operational flowchart of the method for the boundary value of virtual indirect advertising.
With reference to Figure 38, according to the calculating of embodiment of the present invention for inserting in the method for the boundary value of virtual indirect advertising, in step 4710, determine to comprise the borderline region of the boundary pixel of the virtual indirect advertising be inserted in the frame of video.
In the case, in step 4710, can determine whether each pixel of the frame of video is present in from boundary pixel preset distance, the region be made up of can be defined as borderline region the pixel be present in preset distance.
In the case, in step 4710, can by considering that the overall resolution of frame of video arranges preset distance.
In the case, in step 4710, longer preset distance can be set for the overall resolution of higher frame of video.
In the case, in step 4710, with the overall resolution of frame of video, proportionally preset distance can be set.
Such as, in step 4710, preset distance can be set to 4 pixels when the overall resolution of frame of video is 1280*4720, and preset distance can be set to 6 pixels when the overall resolution of frame of video is 1920*1080.
In the case, according to the calculating of embodiment of the present invention for inserting in the method for the boundary value of virtual indirect advertising, in step S4720, the remaining area except borderline region can be defined as the foreground area corresponding with virtual indirect advertising and background area.
Such as, in step S4720, borderline region can be will do not comprised and the remaining area inserting virtual indirect advertising is defined as foreground area, and the region obtained by getting rid of borderline region and foreground area from frame of video background area can be defined as.
As mentioned above, according to the calculating of embodiment of the present invention for inserting in the method for the boundary value of virtual indirect advertising, the pixel of surrounding virtual indirect advertising insertion boundary line can be appointed as borderline region, and the value of borderline region pixel can be recalculated, therefore make it possible to naturally to watch virtual indirect advertising and insert border surface.
In addition, according to the calculating of embodiment of the present invention for inserting in the method for the boundary value of virtual indirect advertising, in step S4730, for borderline region pixel each in borderline region, by considering that prospect reference pixel and reference background pixel are selected in the position of borderline region pixel from the remaining area except borderline region.
In the case, in step S4730, can select in the pixel of foreground area near of borderline region pixel as prospect reference pixel.
In addition, in step S4730, can select in the pixel of background area near one of borderline region pixel reference pixel as a setting.
In the case, prospect reference pixel is the pixel of foreground area, and reference background pixel is the pixel of background area.
In the case, in step S4730, reference pixel selected cell 4120 can generate the vertical line between borderline region pixel and the boundary line of remaining area.
Such as, in step S4730, the vertical line of the boundary line from borderline region pixel to foreground area can be drawn.
Such as, reference pixel selected cell 4120 can draw the vertical line of the boundary line from borderline region pixel to foreground area.
In the case, in step S4730, can select in the pixel of foreground area near borderline region pixel and corresponding with the foot of vertical line one as prospect reference pixel.
In addition, in step S4730, the vertical line of the boundary line from borderline region pixel to background area can be drawn.
In the case, in step S4730, can select in the pixel of background area near borderline region pixel and corresponding with the foot of vertical line reference pixel as a setting.
As mentioned above, in calculating according to the present invention for inserting in the method for the boundary value of virtual indirect advertising, for the pixel of surrounding virtual indirect advertising insertion boundary line, pixel value by referring to background and prospect recalculates the value of borderline region pixel, therefore, it is possible to naturally watch virtual indirect advertising to insert border surface.
In the case, in step S4730, for each borderline region pixel in borderline region, by considering that former frame reference pixel is selected in the position of borderline region pixel from the former frame of frame of video, and a rear frame reference pixel can be selected from a rear frame of frame of video.
As mentioned above, in calculating according to the present invention for inserting in the method for the boundary value of virtual indirect advertising, by not only inserting the pixel of boundary line with reference to the virtual indirect advertising of encirclement, and recalculate the pixel value of present frame with reference to the pixel value of former frame/rear frame, therefore, it is possible to naturally watch virtual indirect advertising at the playback of video to insert border surface.
In the case, in step S4730, can from former frame, select the pixel corresponding with the position of borderline region pixel as former frame reference pixel, and can from a rear frame, select the pixel corresponding with the position of borderline region pixel as a rear frame reference pixel.
In addition, according to the calculating of embodiment of the present invention for inserting in the method for the boundary value of virtual indirect advertising, in step S4740, the weight of the pixel value of prospect of the application reference pixel and the pixel value of reference background pixel and prospect reference pixel and the weight of reference background pixel carry out the value of computation bound area pixel.
In the case, in step S4740, can the value of computation bound area pixel as shown in equation 2.
In the case, in step S4740, the weight of prospect reference pixel can be set based on the distance between borderline region pixel and prospect reference pixel, and the weight of reference background pixel can be set based on the distance between borderline region pixel and reference background pixel.
In the case, in step S4740, the weight of prospect reference pixel and the weight of reference background pixel can be set inversely with the distance to borderline region pixel.
In the case, in step S4740, can weight be set, makes the weight of prospect reference pixel and the weight sum of reference background pixel be 1.
Such as, in step S4740, the distance between borderline region pixel and prospect reference pixel is 1, and when the distance between borderline region pixel and reference background pixel is 3, the weight of prospect reference pixel can be set to 0.75, and the weight of reference background pixel is set to 0.25.
As mentioned above, in calculating according to the present invention for inserting in the method for the boundary value of virtual indirect advertising, for the pixel of surrounding virtual indirect advertising insertion boundary line, pixel value by referring to background and prospect recalculates the value of borderline region pixel, therefore, it is possible to naturally watch virtual indirect advertising to insert border surface.
In the case, in step S4740, the weight of the pixel value of former frame reference pixel and the rear pixel value of a frame reference pixel and the weight of former frame reference pixel and a rear frame reference pixel can be used to carry out the value of computation bound area pixel.
In the case, in step S4740, can the value of computation bound area pixel as illustrated by equation 3.
In the case, in step S4740, can weight be set, makes the weight of former frame reference pixel, the rear weight of a frame reference pixel and the weight sum of reference frame reference pixel be 1.
Such as, in step S4740, distance between borderline region pixel and prospect reference pixel is 1, and when the distance between borderline region pixel and reference background pixel is 3, the weight of prospect reference pixel can be set to 0.75, the weight of reference background pixel is set to 0.25, the weight of former frame reference pixel is set to 0.25, the weight of a rear frame reference pixel is set to 0.25, and is set to 0.5 with reference to the weight of frame reference pixel.
As mentioned above, in calculating according to the present invention for inserting in the method for the boundary value of virtual indirect advertising, not only with reference to the pixel of surrounding virtual indirect advertising insertion boundary line, and recalculate the pixel value of present frame with reference to the pixel value of former frame/rear frame, therefore, it is possible to naturally watch virtual indirect advertising at the playback of video to insert border surface.
Figure 39 is the operational flowchart of the example of the step S4730 that the selection prospect reference pixel shown in Figure 38 and reference background pixel are shown.
With reference to Figure 39, for step S4730, the vertical line of the boundary line from borderline region pixel to foreground area can be drawn in step S4810.
In addition, for step S4730, can select in step S4820 in the pixel of foreground area near borderline region pixel and corresponding with the foot of vertical line one as prospect reference pixel.
In addition, for step S4730, can select in step S4830 in the pixel of background area near borderline region pixel and corresponding with the foot of vertical line reference pixel as a setting.
Can according to the order shown in each in Figure 38 or Figure 39, according to the order contrary with aforementioned sequence or perform in Figure 38 and Figure 39 the step shown in each concurrently.
Calculating according to the present invention may be implemented as the program or smart phone application that can be performed by various computer installation for the method for the boundary value inserting virtual indirect advertising.In the case, program or smart phone application can be recorded in computer-readable storage medium.Computer-readable storage medium can comprise program command, data file and data structure alone or in combination.The program command recorded in storage medium can be specifically designed for the present invention and construct, or can be known or available for the those of ordinary skill in the field of computer software.The example of computer-readable storage medium comprises all types of hardware units be specially constructed as record and execution of program instructions, such as magnetizing mediums (such as hard disk, floppy disk and tape), optical medium (such as CD (CD)-read-only memory (ROM) and digital versatile disc (DVD)), magnet-optical medium (such as soft CD, ROM, random access memory (RAM) and flash memory).The example of program command comprises such as such by the code of compiler-creating machine code and uses the executable higher-level language code of interpreter by computer.These hardware units can be constructed to as the operation of one or more software module to perform operation of the present invention, and vice versa.
Apparatus and method for calculating the boundary value for inserting virtual indirect advertising according to the present invention are not limited to structure and the method for above-mentioned execution mode, but partly or entirely can being constructed to of these execution modes is optionally combined, and makes it possible to the execution mode revised according to various mode.
Industrial applicibility
According to the present invention, in order to be inserted in video by advertisement, the advertisement that search package inserts region containing advertisement is inserted target frame and carries out cluster to it, therefore carrys out insertion of advertising for all frames according to unified mode.Particularly when considering to insert the growing demand of technology to the natural image for virtual indirect advertising, according to of the present invention will be high for using frame cluster to carry out the utilization rate of the technology of insertion of advertising.
In addition, according to the present invention, determine the candidate region by being inserted into virtual indirect advertising, measure the exposure levels of candidate region, and measured exposure levels is supplied to user, therefore provide guidance information, make user that virtual indirect advertising can be selected while the advertising results identifying each candidate region more intuitively to insert region.Particularly when considering that people are to the just ever-increasing trend along with the increase of the scale of nearest advertising market of the interest by the image of advertising items being inserted in TV play or sports broadcast the virtual indirect advertising strengthening advertising results, the utilization rate according to the technology for being provided for the insertion region guidance information inserting virtual indirect advertising of the present invention will be high.
In addition, according to the present invention, when in spot for photography directly cue mark time, usage flag determines the region by being inserted into virtual indirect advertising, then mark is eliminated, the region being eliminated mark is restricted to the region not overlapping with the region that will be inserted into virtual indirect advertising, then performs reparation.Particularly when considering the trend that people increase along with the expansion of advertising market scale in recent years the interest by the image of advertising items being inserted in TV play or sports broadcast the virtual indirect advertising strengthening advertising results, the utilization rate according to the technology for eliminating the mark for inserting virtual indirect advertising of the present invention will be high.
In addition, according to the present invention, existing advertisement can be replaced virtual indirect advertising is inserted in video image content.In the case, reparation must be performed in the region of existing advertisement, to coordinate with the image of surrounding.Region by being performed reparation can be minimized by existing advertising area and virtual indirect advertising region being compared.Therefore, it is possible to reduce the running cost needed for repairing, and can make the ad margins of self-virtualizing indirect advertising to maximize.
In addition, according to the present invention, automatically calculate the value of the borderline region pixel of the virtual indirect advertising be inserted in video image content, and it is processed, the boundary line inserting virtual indirect advertising can be coordinated with the image of surrounding.Particularly when considering the trend that people increase along with the expansion of advertising market scale in recent years the interest by the image of advertising items being inserted in TV play or sports broadcast the virtual indirect advertising strengthening advertising results, the utilization rate according to the technology for calculating the boundary value for inserting virtual indirect advertising of the present invention will be high.

Claims (19)

1., for a device for insertion of advertising, this device comprises:
Frame search unit, it is constructed to when advertisement insertion region is arranged at least one in the frame of video, and target frame is inserted in the advertisement that search package inserts region containing advertisement;
Cluster cell, it is constructed to insert target frame to described advertisement and divides into groups, described advertisement is selected to insert any one in target frame as with reference to frame, region is inserted with the advertisement advertisement of inserting in target frame except described reference frame in advertisement insertion region in described reference frame compare, calculate with the difference of described reference frame as clustering factor, and perform cluster; And
Advertisement plug-in unit, it is constructed to when during insertion of advertising, being inserted described advertisement for all by the frame of cluster according to unified mode in by least one in the described frame of cluster.
2. device according to claim 1, wherein, described cluster cell comprises:
Scene structure analytic unit, its be constructed to obtain described advertisement insert in the position in region, size, rotation and visual angle any one or more, and scene structure to be analyzed;
Camera motion analytic unit, it is constructed to obtain described advertisement and inserts the focus in region and any one in shake or more, and analyzes the camera motion in described advertisement insertion region; And
Clustering factor computing unit, it is constructed to select to be provided with described advertisement and inserts the frame in region as described reference frame, region is inserted with the advertisement advertisement of inserting in target frame except described reference frame in advertisement insertion region in described reference frame compare, and calculate with the difference of described reference frame as described clustering factor.
3. device according to claim 2, wherein, the scene structure in described reference frame and camera motion and the described advertisement except described reference frame are inserted scene structure in target frame by described clustering factor computing unit and camera motion compares.
4. device according to claim 3, wherein, described clustering factor computing unit is determined for by the mass motion between the successive frame of the described frame of cluster, and calculates described clustering factor based on the determination result of described mass motion.
5. device according to claim 4, wherein, described cluster cell inserts target frame to extract from corresponding advertisement and only comprises the part that region is inserted in described advertisement, and carries out cluster to extracted part.
6. device according to claim 5, wherein, described cluster cell carries out cluster to continuous print frame from described reference frame.
7. device according to claim 6, wherein, described frame search unit determines the mass motion of described successive frame, and searches for described advertisement insertion target frame based on the described determination result of described mass motion.
8. device according to claim 7, wherein, described frame search unit uses the described determination result of described mass motion any one in the frame of front and back next or more middle prediction advertisement to insert the position in region, and searches for described advertisement insertion target frame.
9. a method for insertion of advertising, the method comprises the following steps:
When advertisement insertion region is arranged at least one in the frame of video, target frame is inserted in the advertisement that search package inserts region containing advertisement;
Insert target frame to described advertisement to divide into groups, described advertisement is selected to insert any one in target frame as with reference to frame, region is inserted with the advertisement advertisement of inserting in target frame except described reference frame in advertisement insertion region in described reference frame compare, calculate with the difference of described reference frame as clustering factor, and perform cluster; And
When during insertion of advertising, being inserted described advertisement by the frame of cluster according to unified mode for all in by least one in the described frame of cluster.
10., for a device for insertion of advertising, this device comprises:
Candidate region determining unit, it is constructed to the candidate region determined being inserted into virtual indirect advertising in video image content;
Advertisement plug-in unit, it is constructed to process described virtual indirect advertising, and is inserted in described candidate region by treated virtual indirect advertising;
Exposure measuring unit, its exposure characteristics being constructed to expose in described video image content based on described virtual indirect advertising measures the exposure levels of described virtual indirect advertising; And
Information provider unit, the insertion region guidance information of its described exposure levels be constructed to comprising described virtual indirect advertising is supplied to user;
Wherein, described information provider unit performs following operation:
Determine whether the described exposure levels of described virtual indirect advertising exceedes Pre-set reference exposure levels, and if the described exposure levels of described virtual indirect advertising exceedes described Pre-set reference exposure levels, then contain the described insertion region guidance information of the described exposure levels of described virtual indirect advertising to described user's providing package; And
Show any one in item below or more: the candidate region inserting described virtual indirect advertising and the candidate region existed before inserting described virtual indirect advertising.
The method of 11. 1 kinds of insertion of advertisings, the method comprises the following steps:
The candidate region by being inserted into virtual indirect advertising is determined in video image content;
Process described virtual indirect advertising, and treated virtual indirect advertising is inserted in described candidate region;
The exposure characteristics exposed in described video image content based on described virtual indirect advertising measures the exposure levels of described virtual indirect advertising; And
The insertion region guidance information comprising the described exposure levels of described virtual indirect advertising is supplied to user.
12. 1 kinds of devices for insertion of advertising, this device comprises:
Advertisement selecting unit, it is constructed to select to be inserted into the virtual indirect advertising in video image content;
Restoring area generation unit, it is constructed to generate restoring area and non-restoring area by the existing advertisement comprised in described virtual indirect advertising and described video image content being compared;
Repair unit, it is constructed to only perform reparation to described restoring area; And
Advertisement plug-in unit, it is constructed to, after the described restoring area of reparation, be inserted in described video image content by described virtual indirect advertising;
Wherein, described restoring area generation unit generates the target area comprising the region of described existing advertisement, and generates described restoring area and described non-restoring area by dividing described target area;
Wherein, described restoring area be described target area not with the region that the region of described virtual indirect advertising is overlapping; And
Wherein, described non-restoring area is the region overlapping with the region of described virtual indirect advertising of described target area.
The method of 13. 1 kinds of insertion of advertisings, the method comprises the following steps:
Select to be inserted into the virtual indirect advertising in video image content;
Restoring area and non-restoring area is generated by the existing advertisement comprised in described virtual indirect advertising and described video image content being compared;
Only reparation is performed to described restoring area; And
After the described restoring area of reparation, described virtual indirect advertising is inserted in described video image content.
14. 1 kinds of devices for insertion of advertising, this device comprises:
Marker recognition unit, it is constructed to identify the mark comprised in the frame of video;
Area determination unit, it is constructed to determine to mark corresponding marked region with described, will be inserted into the insertion region of virtual indirect advertising and use the mark in described marked region and described insertion region to eliminate region;
Mark eliminates unit, and it is constructed to only to eliminate region to described mark and performs reparation; And
Advertisement plug-in unit, it is constructed to, after the described mark of reparation eliminates region, be inserted into by described virtual indirect advertising in described insertion region;
Wherein, described area determination unit performs following operation:
Described marked region is divided into restoring area and non-restoring area, and described restoring area is defined as described mark elimination region; And
Determine whether each marked pixels in described marked region is present in described insertion region, the marked pixels be not present in described insertion region is defined as described restoring area, and the marked pixels be present in described insertion region is defined as described non-restoring area.
The method of 15. 1 kinds of insertion of advertisings, the method comprises the following steps:
Identify the mark comprised in the frame of video;
Determine to mark corresponding marked region with described, will the insertion region of virtual indirect advertising is inserted into and use the mark in described marked region and described insertion region to eliminate region;
Only region is eliminated to described mark and perform reparation; And
After the described mark of reparation eliminates region, described virtual indirect advertising is inserted in described insertion region.
16. 1 kinds for calculating the device of the boundary value for inserting virtual indirect advertising, this device comprises:
Area determination unit, it is constructed to the borderline region of the boundary pixel determining to comprise the virtual indirect advertising be inserted in frame of video;
Reference pixel selected cell, it is constructed to for each borderline region pixel in described borderline region, by considering that prospect reference pixel and reference background pixel are selected in the position of described borderline region pixel from the remaining area except described borderline region; And
Boundary value computing unit, it is constructed to use the weight of the weight of the pixel value of the pixel value of described prospect reference pixel and described reference background pixel and described prospect reference pixel and described reference background pixel to calculate the value of described borderline region pixel;
Wherein, the remaining area except described borderline region is divided into the foreground area corresponding with described virtual indirect advertising and background area by described area determination unit;
Wherein, described reference pixel selected cell comprises:
Prospect reference pixel selected cell, its be constructed to select in the pixel of described foreground area near of described borderline region pixel as described prospect reference pixel;
Reference background pixel selection unit, its be constructed to select in the pixel of described background area near of described borderline region pixel as described reference background pixel;
Former frame reference pixel selected cell, it is constructed to for each borderline region pixel in described borderline region, by considering that former frame reference pixel is selected in the position of described borderline region pixel from the former frame of described frame of video; And
A rear frame reference pixel selected cell, it is constructed to from a rear frame of described frame of video, select a rear frame reference pixel; And
Wherein, described boundary value computing unit uses the weight of the pixel value of described former frame reference pixel and the described rear pixel value of a frame reference pixel and the weight of described former frame reference pixel and a described rear frame reference pixel to calculate the value of described borderline region pixel.
17. devices according to claim 16, wherein, described boundary value computing unit arranges the described weight of described prospect reference pixel based on the distance between described borderline region pixel and described prospect reference pixel, and arranges the described weight of described reference background pixel based on the distance between described borderline region pixel and described reference background pixel.
18. 1 kinds of calculating are for inserting the method for the boundary value of virtual indirect advertising, and the method comprises the following steps:
Determine to comprise the borderline region of the boundary pixel of the virtual indirect advertising be inserted in frame of video;
For each borderline region pixel in described borderline region, by considering that prospect reference pixel and reference background pixel are selected in the position of described borderline region pixel from the remaining area except described borderline region; And
Use the weight of the weight of the pixel value of the pixel value of described prospect reference pixel and described reference background pixel and described prospect reference pixel and described reference background pixel to calculate the value of described borderline region pixel.
19. 1 kinds of media had program stored therein, this program is used for the described method set forth in any one in enforcement of rights requirement 9, claim 11, claim 13, claim 15 and claim 18.
CN201480033739.0A 2014-01-24 2014-12-10 For clustering the device and method to be inserted into advertisement by using frame Active CN105284122B (en)

Applications Claiming Priority (11)

Application Number Priority Date Filing Date Title
KR20140009195A KR101503029B1 (en) 2014-01-24 2014-01-24 Marker removal apparatus for inserting digital product placement and method for the same
KR10-2014-0009196 2014-01-24
KR10-2014-0009193 2014-01-24
KR10-2014-0009195 2014-01-24
KR10-2014-0009194 2014-01-24
KR1020140009196A KR101561083B1 (en) 2014-01-24 2014-01-24 Boundary value calculation apparatus for inserting digital product placement and method for the same
KR1020140009193A KR101573482B1 (en) 2014-01-24 2014-01-24 Apparatus for inserting advertisement using frame clustering and method thereof
KR1020140009194A KR101564313B1 (en) 2014-01-24 2014-01-24 Insertion area guide information apparatus for inserting digital product placement and method for the same
KR1020140013842A KR102135671B1 (en) 2014-02-06 2014-02-06 Method of servicing virtual indirect advertisement and apparatus for the same
KR10-2014-0013842 2014-02-06
PCT/KR2014/012159 WO2015111840A1 (en) 2014-01-24 2014-12-10 Device and method for inserting advertisement by using frame clustering

Publications (2)

Publication Number Publication Date
CN105284122A true CN105284122A (en) 2016-01-27
CN105284122B CN105284122B (en) 2018-12-04

Family

ID=53681611

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201480033739.0A Active CN105284122B (en) 2014-01-24 2014-12-10 For clustering the device and method to be inserted into advertisement by using frame

Country Status (3)

Country Link
US (1) US10904638B2 (en)
CN (1) CN105284122B (en)
WO (1) WO2015111840A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110996121A (en) * 2019-12-11 2020-04-10 北京市商汤科技开发有限公司 Information processing method and device, electronic equipment and storage medium
CN111818364A (en) * 2020-07-30 2020-10-23 广州云从博衍智能科技有限公司 Video fusion method, system, device and medium

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9870638B2 (en) * 2016-02-24 2018-01-16 Ondrej Jamri{hacek over (s)}ka Appearance transfer techniques
US9852523B2 (en) 2016-02-24 2017-12-26 Ondrej Jamri{hacek over (s)}ka Appearance transfer techniques maintaining temporal coherence
WO2018033137A1 (en) * 2016-08-19 2018-02-22 北京市商汤科技开发有限公司 Method, apparatus, and electronic device for displaying service object in video image
EP3533229A4 (en) * 2016-10-28 2020-06-24 Axon Enterprise, Inc. Systems and methods for supplementing captured data
CN108875692B (en) * 2018-07-03 2020-10-16 中影数字巨幕（北京）有限公司 Thumbnail film generation method, medium and computing device based on key frame processing technology
US11032607B2 (en) * 2018-12-07 2021-06-08 At&T Intellectual Property I, L.P. Methods, devices, and systems for embedding visual advertisements in video content
US11234027B2 (en) * 2019-01-10 2022-01-25 Disney Enterprises, Inc. Automated content compilation
US11074457B2 (en) * 2019-04-17 2021-07-27 International Business Machines Corporation Identifying advertisements embedded in videos
US11042969B2 (en) * 2019-05-23 2021-06-22 Adobe Inc. Automatic synthesis of a content-aware sampling region for a content-aware fill
US10950022B2 (en) * 2019-06-06 2021-03-16 Sony Interactive Entertainment Inc. Using machine learning and image recognition for automatic relocation of camera display area and sizing of camera image
CN110278446B (en) * 2019-06-20 2022-01-28 北京字节跳动网络技术有限公司 Method and device for determining virtual gift display information and electronic equipment
CN110290426B (en) * 2019-06-24 2022-04-19 腾讯科技（深圳）有限公司 Method, device and equipment for displaying resources and storage medium
CN110213629B (en) * 2019-06-27 2022-02-11 腾讯科技（深圳）有限公司 Information implantation method, device, server and storage medium
CN110381369B (en) * 2019-07-19 2022-02-22 腾讯科技（深圳）有限公司 Method, device and equipment for determining recommended information implantation position and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030002919A (en) * 2001-07-02 2003-01-09 에이알비전 (주) realtime image implanting system for a live broadcast
CN1728781A (en) * 2004-07-30 2006-02-01 新加坡科技研究局 Method and apparatus for insertion of additional content into video
KR100886149B1 (en) * 2007-10-18 2009-02-27 (주)이엠티 Method for forming moving image by inserting image into original image and recording media
CN101621636A (en) * 2008-06-30 2010-01-06 北京大学 Method and system for inserting and transforming advertisement sign based on visual attention module
KR20100088282A (en) * 2009-01-30 2010-08-09 서강대학교산학협력단 Method and apparatus of inpainting for video data
KR20120131425A (en) * 2011-05-25 2012-12-05 연세대학교 산학협력단 Interactive advertisement authoring device and interactive advertisement authoring method
CN103299610A (en) * 2011-01-12 2013-09-11 华为技术有限公司 Method and apparatus for video insertion

Family Cites Families (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB8822062D0 (en) * 1988-09-20 1988-10-19 Quantel Ltd Video processing
US5263933A (en) * 1988-12-14 1993-11-23 Patco Ventures Ltd. Safety syringe needle device with interchangeable and retractable needle platform
US5903317A (en) * 1993-02-14 1999-05-11 Orad Hi-Tech Systems Ltd. Apparatus and method for detecting, identifying and incorporating advertisements in a video
IL108957A (en) * 1994-03-14 1998-09-24 Scidel Technologies Ltd System for implanting an image into a video stream
US5436672A (en) * 1994-05-27 1995-07-25 Symah Vision Video processing system for modifying a zone in successive images
US5751838A (en) * 1996-01-26 1998-05-12 Nec Research Institute, Inc. Correction of camera motion between two image frames
FR2775813B1 (en) * 1998-03-06 2000-06-02 Symah Vision METHOD AND DEVICE FOR REPLACING TARGET PANELS IN A VIDEO SEQUENCE
AUPQ363299A0 (en) 1999-10-25 1999-11-18 Silverbrook Research Pty Ltd Paper based information inter face
US7079157B2 (en) * 2000-03-17 2006-07-18 Sun Microsystems, Inc. Matching the edges of multiple overlapping screen images
US7952583B2 (en) * 2000-06-19 2011-05-31 Mental Images Gmbh Quasi-monte carlo light transport simulation by efficient ray tracing
US7199793B2 (en) * 2002-05-21 2007-04-03 Mok3, Inc. Image-based modeling and photo editing
US6987520B2 (en) 2003-02-24 2006-01-17 Microsoft Corporation Image region filling by exemplar-based inpainting
US7375745B2 (en) * 2004-09-03 2008-05-20 Seiko Epson Corporation Method for digital image stitching and apparatus for performing the same
JP4487191B2 (en) 2004-12-24 2010-06-23 カシオ計算機株式会社 Image processing apparatus and image processing program
KR100733985B1 (en) 2005-04-01 2007-06-29 한국전자통신연구원 Interactive Digital Broadcasting System and Method for playing/storing particular contents, and Interactive Digital Broadcasting Receiving System and Method according to it
WO2006105660A1 (en) 2005-04-05 2006-10-12 Google Inc. Method and system supporting audited reporting of advertising impressions from video games
JP2007122232A (en) * 2005-10-26 2007-05-17 Casio Comput Co Ltd Image processor and program
US8031775B2 (en) * 2006-02-03 2011-10-04 Eastman Kodak Company Analyzing camera captured video for key frames
DE102006016452A1 (en) * 2006-04-07 2007-10-11 Bayer Materialscience Ag Nitrocellulose based binders for aqueous nail polishes
KR20070115348A (en) 2006-06-02 2007-12-06 김일 A method and system of exposure time analysis of internet advertisement
EP2126767A4 (en) 2007-03-12 2011-05-18 Vortex Technology Services Ltd Intentionality matching
US9911126B2 (en) 2007-04-10 2018-03-06 Google Llc Refreshing advertisements in offline or virally distributed content
US8717412B2 (en) * 2007-07-18 2014-05-06 Samsung Electronics Co., Ltd. Panoramic image production
US8108257B2 (en) 2007-09-07 2012-01-31 Yahoo! Inc. Delayed advertisement insertion in videos
US8351725B2 (en) * 2008-09-23 2013-01-08 Sharp Laboratories Of America, Inc. Image sharpening technique
US8488863B2 (en) * 2008-11-06 2013-07-16 Los Alamos National Security, Llc Combinational pixel-by-pixel and object-level classifying, segmenting, and agglomerating in performing quantitative image analysis that distinguishes between healthy non-cancerous and cancerous cell nuclei and delineates nuclear, cytoplasm, and stromal material objects from stained biological tissue materials
US8374457B1 (en) * 2008-12-08 2013-02-12 Adobe Systems Incorporated System and method for interactive image-noise separation
KR100987412B1 (en) 2009-01-15 2010-10-12 포항공과대학교 산학협력단 Multi-Frame Combined Video Object Matting System and Method Thereof
KR20100101397A (en) 2009-03-09 2010-09-17 김태규 Real-time online ad insert system using digital contents and method thereof
US8605783B2 (en) * 2009-05-22 2013-12-10 Microsoft Corporation Composite video generation
JP5269727B2 (en) 2009-09-04 2013-08-21 ヤフー株式会社 Content insertion management apparatus, method and program
WO2012011180A1 (en) 2010-07-22 2012-01-26 パイオニア株式会社 Augmented reality device and method of controlling the same
KR101164747B1 (en) 2010-07-28 2012-07-12 광주과학기술원 Method for sysnthesizing image in mobile terminal
KR20120071226A (en) 2010-12-22 2012-07-02 한국전자통신연구원 Apparatus and method for extracting object
KR20130047846A (en) 2011-11-01 2013-05-09 정인수 Internet advertising system
KR20130056532A (en) 2011-11-22 2013-05-30 주식회사 씬멀티미디어 Selective ppl inserting in motion video and advertisement service method
KR101273663B1 (en) 2011-11-22 2013-06-10 건국대학교 산학협력단 Inpainting system and method for h.264 error concealment image
KR20130073459A (en) * 2011-12-23 2013-07-03 삼성전자주식회사 Method and apparatus for generating multi-view
KR101340065B1 (en) 2011-12-29 2013-12-11 주식회사 씨에이취피커뮤니케이션 Method for detecting blank area in window, system and method for internet advertisement using the same
KR20130104215A (en) 2012-03-13 2013-09-25 계원예술대학교 산학협력단 Method for adaptive and partial replacement of moving picture, and method of generating program moving picture including embedded advertisement image employing the same
KR101353038B1 (en) 2012-08-06 2014-01-17 광주과학기술원 Apparatus and method for processing boundary part of depth image
US20160267349A1 (en) * 2015-03-11 2016-09-15 Microsoft Technology Licensing, Llc Methods and systems for generating enhanced images using multi-frame processing
WO2016200043A1 (en) * 2015-06-10 2016-12-15 엘지전자 주식회사 Method and apparatus for inter prediction on basis of virtual reference picture in video coding system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030002919A (en) * 2001-07-02 2003-01-09 에이알비전 (주) realtime image implanting system for a live broadcast
CN1728781A (en) * 2004-07-30 2006-02-01 新加坡科技研究局 Method and apparatus for insertion of additional content into video
KR100886149B1 (en) * 2007-10-18 2009-02-27 (주)이엠티 Method for forming moving image by inserting image into original image and recording media
CN101621636A (en) * 2008-06-30 2010-01-06 北京大学 Method and system for inserting and transforming advertisement sign based on visual attention module
KR20100088282A (en) * 2009-01-30 2010-08-09 서강대학교산학협력단 Method and apparatus of inpainting for video data
CN103299610A (en) * 2011-01-12 2013-09-11 华为技术有限公司 Method and apparatus for video insertion
KR20120131425A (en) * 2011-05-25 2012-12-05 연세대학교 산학협력단 Interactive advertisement authoring device and interactive advertisement authoring method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
殷伟良: "电视直播虚拟广告系统关键技术的研究", 《中国优秀硕士学位论文全文数据库-信息科技辑》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110996121A (en) * 2019-12-11 2020-04-10 北京市商汤科技开发有限公司 Information processing method and device, electronic equipment and storage medium
CN111818364A (en) * 2020-07-30 2020-10-23 广州云从博衍智能科技有限公司 Video fusion method, system, device and medium
CN111818364B (en) * 2020-07-30 2021-08-06 广州云从博衍智能科技有限公司 Video fusion method, system, device and medium

Also Published As

Publication number Publication date
US10904638B2 (en) 2021-01-26
WO2015111840A1 (en) 2015-07-30
US20160142792A1 (en) 2016-05-19
CN105284122B (en) 2018-12-04

Similar Documents

Publication Publication Date Title
CN105284122A (en) Device and method for inserting advertisement by using frame clustering
CN109618222B (en) A kind of splicing video generation method, device, terminal device and storage medium
US10762608B2 (en) Sky editing based on image composition
CN107493488B (en) Method for intelligently implanting video content based on Faster R-CNN model
Dong et al. Fast multi-operator image resizing and evaluation
CN1930585B (en) Creating a depth map
US8144919B2 (en) Annealing algorithm for non-rectangular shaped stained glass collages
US10672104B2 (en) Method and apparatus for generating an extrapolated image based on object detection
CN111815645B (en) Method and system for cutting advertisement video picture
CN105308636A (en) Apparatus and method for providing virtual advertisement
Inamoto et al. Intermediate view generation of soccer scene from multiple videos
Upadhyay et al. Augmented reality and machine learning based product identification in retail using Vuforia and MobileNets
Zhang et al. Hybrid image retargeting using optimized seam carving and scaling
EP3396964B1 (en) Dynamic content placement in a still image or a video
CN113709544A (en) Video playing method, device, equipment and computer readable storage medium
CN113673567B (en) Panorama emotion recognition method and system based on multi-angle sub-region self-adaption
US10674184B2 (en) Dynamic content rendering in media
Mancas et al. Applications of saliency models
CN111768264A (en) Commodity display picture generation method and device, storage medium and terminal
Efimova et al. Advertisement replacement in video
CN112070656B (en) Frame data modification method and device
WO2022201237A1 (en) Server, text field arrangement position method, and program
Chu Application of animation products via multimodal information and semantic analogy
Bai et al. Anything in Any Scene: Photorealistic Video Object Insertion
CN118014659A (en) Electronic propaganda product generation and playing method, system and storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20190403

Address after: Gyeonggi Do, South Korea

Co-patentee after: 11th Street Co., Ltd.

Patentee after: SK Telecom Co., Ltd.

Address before: Gyeonggi Do, South Korea

Patentee before: SK Telecom Co., Ltd.

TR01 Transfer of patent right