CN101945263A

CN101945263A - Method for using information sets in video resources

Info

Publication number: CN101945263A
Application number: CN2010102200381A
Authority: CN
Inventors: 孟智平
Original assignee: Individual
Current assignee: Individual
Priority date: 2007-05-08
Filing date: 2007-05-08
Publication date: 2011-01-12

Abstract

The invention discloses a method for using information sets in video resources. Extension of the video transmission contents is realized by introducing the information sets at the client, the server and the extension server to provide good platform for the video services based on various applications. The information sets include a position set, an operation set and a function set. Positions of new services or new application can be generated through more precise division of the position set and various positions are associated with specific objects, thus setting attribute information for various position objects. Various attribute information introduces richer video application. The invention introduces the intra-frame service and out-of-frame service mechanisms to better manage the existing position set, operation set and function set. The method changes the defect that the existing video technology only emphasizes compression and quality, puts emphasis on application and control of the videos and provides the good technical platform and application mode reference scheme for the possible video application technology in the future.

Description

A kind of method of in video resource, using information set

Technical field

The present invention relates to the video information process technical field, relate in particular to a kind of method of in video resource, using information set.

Background technology

In the prior art, piece image is formed by some, and every comprises a series of MB (MacroBlock, macro block).The arrangement of MB can be by raster scan order, also can be not according to scanning sequency, and raster scan is mapped to one-dimensional grating with the two-dimensional rectangle grating, the inlet of one-dimensional grating is from first row of two-dimensional grating, then scan second row, the third line then, and the like, the row in the grating from left to right scans.Wherein, flexible macro block ordering FMO (Flexible macroblock ordering is also referred to as slice-group slice groups technology) pattern is a big characteristic H.264, is applicable to H.264 basic class and the application of expanding class.

Image intra-prediction mechanism, for example infra-frame prediction or motion-vector prediction only allow to use space neighboring macro-blocks or band with in a slice group, each sheet is independently decoded, the macro block of different sheets can not be used for self sheet and make prediction reference, and therefore, the setting of sheet can not cause the error code diffusion.The FMO pattern is by the macroblock allocation map technology, each macroblock allocation in sheet not according to scanning sequency, the pattern of FMO mode division image comprises various, wherein, checker board pattern, rectangular pattern etc. are more important, and the FMO pattern also can make the macro block order in the frame cut apart certainly, and the size of the sheet after feasible cutting apart is less than MTU (the Maxim Transport Unit of wireless network, MTU) size, the view data after process FMO pattern is cut apart is separately transmitted.Though FMO can be used as single transmission or error correction unit, in this scope (Slice Group), still experience user's operation without any mechanism.

In the prior art, video or huge image information all are a unified integral body, for video, always following the function that is played to last frame from first frame, player can pass through the fast forwarding and fast rewinding that RTSP (Real-time Streaming Protocol, real-time streaming protocol) realizes video frequency program neatly.For image, normally adopt the fixed coordinates of the some positions of search, accurately navigate to the method for this particular location then.No matter be for video or image because aspect, position information is very limited, for example be difficult to navigate in certain frame certain concrete in certain zone macro block, therefore a lot of application all can not launch smoothly.Especially in video, this location resource determine to remain a blank.

Yet, because other relevant information beyond the video coding (as, information on services) scarcity, and video itself does not provide redirect or fetches the ways and means of data, therefore video and some services combine and with the timely interaction of the user comparison difficulty that can become, and then, existing IPTV (InternetProtocol Television) system is lacked with the user produce interactive effective ways, therefore can not collect user's data.

In the existing video resource processing method, owing to be simple video image to be pushed to the user, and can't effectively finish interaction with the user.More because existing video coding originally is to be purpose with the video compression, utilize the audiovisual information of existing network high quality, purpose of design own also can't realize the interaction with the user.H.264/MPEG comparative maturity has a video coding such as 4/MPEG 2/AVS in existing popular coding, and these codings all are to conciliate the boil down to purpose with compressed encoding.But along with the raising of network technology, network bandwidth problem solves gradually, and the user can propose more requirement to video, the not only qualitative requirement of video itself, and need more applications and interaction.

Summary of the invention

The problem that the embodiment of the invention will solve provides a kind of method of using information set in video resource, to solve video resource relevant information scarcity in the prior art, reaches user and the interactive inflexible defective of service.

To achieve these goals, embodiments of the invention provide a kind of method of using information set in video resource, may further comprise the steps:

Service end in video resource by increasing information set in adding mode in outer adding mode of frame of video or the frame of video; Described service end comprises that Video service end and/or information set add service end; The outer adding mode of described frame of video comprises the mode of information set description document mode, service frame mode or message communicating; Described video resource comprises: frame of video, video image, video file and video flowing; Described information set comprises: position collection and/or operation set and/or function collection;

Described service end sends to client with information set or at client configuration information collection;

The position collection information that described client is concentrated according to described information is determined active position, and described position collection and/or operation set institute corresponding function collection are operated, activated to the operation set that utilizes this position set pair to answer, the execution corresponding function;

Operation set that described position set pair is answered and function collection are provided with in client and/or send to client by described service end, position collection and/or operation set and/or function collection can not be included in the information that service end sends to client and concentrate, and are provided with in client or expansion service end.

Described position collection further comprises: the coordinate of particular location, perhaps macro block, the band positional information in the frame in the frame of video or in the image; Or frame of video is interior or the interior appointed area of image or appointed area profile or slice-group positional information; Or the station location marker of frame of video in the entire frame sequence; Or program frame sequence group id; Or traffic identifier;

Described function collection further comprises: fetch concrete assigned address content object information, jump to concrete assigned address, to the appointed object position send information, open or insert assigned address object, close the object that shows assigned address and the object of mobile assigned address; Described assigned address comprises: particular location in particular location, the player plays window in particular location, the browser in some memory locations, the display screen in some device addresses, the memory device in concrete URL, the hardware device in the network;

Described operation set further comprises: pre-set program operation and messaging program driving operation are also pressed in search information collection position when mouse action, keyboard operation, broadcast;

The proportionate relationship correspondence of described position collection, operation set and function collection comprises:

Position element of set element: a plurality of operation set elements: a plurality of function element of set elements;

A plurality of positions element of set element: a plurality of operation set elements: a plurality of function element of set elements;

Position element of set element: an operation set element: a plurality of function element of set elements:

A plurality of positions element of set element: a plurality of operation set elements: a function element of set element;

Position element of set element: a plurality of operation set elements: a function element of set element;

A plurality of positions element of set element: an operation set element: a plurality of function element of set elements;

Position element of set element: an operation set element: a function element of set element;

A plurality of positions element of set element: an operation set element: a function element of set element;

In the element of set element of position, do not comprise attribute or comprise an attribute or a plurality of attribute.

The all corresponding object in each position is concentrated in described position:

The coordinate of particular location, the perhaps macro block in the frame, a positional information-point of the correspondence object of band in the frame of video or in the image;

Or a block object in appointed area or appointed area profile, the slice-group position-corresponding video resource in the frame of video or in the image, described is the set of point or macro block or band;

Or video resource station location marker-frame object of correspondence in the entire frame sequence;

Or program frame sequence group id-program object of correspondence;

Or traffic identifier-flow object of correspondence;

Described location object all comprises the attribute information of one or more objects, and described attribute information comprises: the joining day of the source of precedence information, transparence information, enciphered message, copyright information, customer information, the operation set of being supported, information and/or target information, position collection and/or effective time, introduce the attribute of new object from the position collection;

Described object properties medium priority information is used for the union operation of diverse location collection: when the stream of different priorities is play simultaneously at same player, play the highest stream of priority; When the program frame sequence group of different priorities is play simultaneously, play the highest program frame sequence group of priority in same player; When the frame of different priorities is play simultaneously, play the highest frame of priority in same client; Or the zone of different priorities is when showing in same frame, the zone that display priority is the highest; The a plurality of information that are different priorities are in the concentrated same position in position, and the broadcast simultaneously in same player of described information, only play the highest information of priority;

Transparence information is used for the position set pair is answered the transparency definition of object in the described object properties;

Enciphered message is used for the position set pair is answered the encryption of object in the described object properties, comprises cipher mode, key information;

Copyright information is used for the position set pair is answered the copyright notice and the protection of object in the described object properties, comprises the attaching information of copyright, the authentication information of copyright, the use information of copyright;

Customer information is used for the position set pair is answered the client rights explanation of object and used client segmentation information in the described object properties, described client rights explanation comprises: download authority, play authority, described use client segmentation information comprises: to the classification control of content itself;

The attribute of introducing new object from the position collection in the described object properties is used to identify concentrates the attribute of introducing new object and the explanation of function and motion conditions from the position, described new object comprises: video, animation, picture, image, sound, literal; The described attribute of introducing new object from the position collection comprises: the creation-time of new object, the location parameter of concentrating in the position, motion state, continue or finish this object time and and position collection or surroundings relation.

Described position concentrates frame inner region acquisition methods to comprise:

FMO pattern in adopting H.264 comes at random the assignment macro block to different sheet groups, the position of sheet group zone as the adding information set by the macro-block order mapping table is set; Or

Adopt the method for the VOL among the MPEG4, object data stream in frame the corresponding display position as the position that adds information set; Or

Adopt algorithm that image recognition algorithm, object track algorithm, foreground object extract or by indicate the subject area method by interpolation more respectively at several frames of being separated by from background, mark off different zones in frame of video, above-mentioned zone is as the position that adds information set.

Client and/or service end and/or expansion service end are provided with the information set complete or collected works, comprise all position collection, operation set and function collection, and the position set pair answers the attribute of object, and client that obtain with the subclass of the information set video resource correspondence as described information set complete or collected works.

The position collection information that described client is concentrated according to described information is determined active position, and the operation set that utilizes this position set pair to answer operates, activates described position and concentrate the corresponding function collection, carries out corresponding function and specifically comprises:

Described client judges that at first position collection information that information concentrates is whether in the complete or collected works of position collection, if do not exist, then do not have the operation or operate invalid, if, then obtain current operation set, judge in this position to concentrate whether there is corresponding operation in operation set then, described operation set should be in the operation set complete or collected works; If have, then the program command of the function collection of executing location collection and operation set correspondence if do not have, is not then carried out the program command of function collection.

Described function specifically comprises turn function concentrated comprising: jump to another frame after a frame carries out respective operations; The viewing area jumps to appointed area in another frame in the frame; The viewing area jumps to another frame in the frame; Jump to appointed area in another frame from a frame.

In frame of video, divide described domain mode and comprise following two kinds of situations, with object zoning or free zoning.

The present invention also provides a kind of system that uses information set in video resource, comprises client and service end,

Described service end comprises that Video service end and/or information set add service end, be used for video resource by outer adding mode or frame of video with frame of video in the adding mode increase information set, described video resource comprises: frame of video, video image, video file and video flowing; Described information set comprises: position collection and/or operation set and/or function collection; And information set sent to described client; The outer adding mode of described frame of video comprises the mode of information set description document mode, service frame mode or message communicating;

Described client, determine active position according to the position collection information that described information is concentrated, and corresponding function collection in described position collection and/or the operation set is operated, activated to the operation set that utilizes this position set pair to answer, carry out corresponding function, described operation set and/or function collection are in the client setting and/or in the service end setting.

Described service end specifically comprises: medium import module, are used for Media Stream is imported service end;

Information adds module, is used to generate the information set file and/or information set is added media file;

The media store module is used to store described information set and/or media file;

Mixed-media network modules mixed-media is used for service end and sends information set and/or Media Stream to client;

Described client specifically comprises: mixed-media network modules mixed-media is used for obtaining information set and/or Media Stream from described service end;

The information Recognition module is used to obtain and the identifying information set content, comprises position collection, operation set and function collection;

The operation induction module is used to obtain the operation set institute operation that described position set pair is answered with carrying out;

Function realizes module, is used to trigger the pairing function collection of described position collection and/or operation set, carries out corresponding function;

The media play module is used to play corresponding media information;

Described service end cooperates realization information set corresponding function or described client to cooperate realization information set corresponding function with one or more service ends with one or more clients.

Also comprise the expansion service end, described client cooperates the function of finishing appointment with the expansion service end;

Described expansion service end comprises:

Function realizes module, is used for realizing that with client functionality module cooperates, and finishes described information and concentrates corresponding function;

Mixed-media network modules mixed-media is used for described client and communicates by letter with described expansion service client information;

Described expansion service end cooperates with one or more clients realizes that information set corresponding function or described client cooperate realization information set corresponding function with one or more expansion service ends;

On system level, service end, client and expansion service end merge in twos, and be separate on the function, is placed on to realize or be placed on the software platform in the hardware realizing;

Position collection, operation set and function collection occur with specific functional form, and operation set is defined in client or service end or expansion service end; The function collection is also realized in client or expansion service end with specific program.

The present invention also provides a kind of method that adds service frame in video resource, may further comprise the steps:

Service end is newly-built service frame in video resource;

In described service frame, add the information set content;

Described service end is used described service frame beared information collection, sends to client; Wherein, the corresponding continuous or discrete one or more frame of video of each service frame.

Described service frame has basic frame structure, packaging information collection in the described frame structure;

Described service frame loaded information collection comprises: the operation set that position collection and described position set pair are answered, and position collection and/or the pairing function collection of operation set;

Described position concentrates each position all to an object, described location object all comprises one or more object properties, and described position set pair resembles in the attribute and also comprises: the joining day of the source of precedence information, transparence information, enciphered message, copyright information, customer information, the operation set of being supported, information and/or target information, position collection and/or effective time, introduce the attribute of new object from the position collection.

Described service frame is created when creating the frame of video file or is created service frame again behind the generation frame of video file earlier;

Described service frame is transmitted or is transmitted in different transmission channels respectively a transmission channel the inside with frame of video;

Described service frame is resolved with same syntactic structure with frame of video or is resolved with different syntactic structures;

Described service frame and frame of video are kept in the identical file or are kept at respectively in the different files;

The method transmission that described service frame adopts the method for compression or do not compress.

The present invention also provides a kind of method that adds 2 frame sequence groups in video resource, may further comprise the steps:

Select a plurality of adjacent or non-conterminous frame in service end with logical relation, and these frames as an orderly set, i.e. frame sequence group;

The element that concentrate as the position position that the frame sequence group is begun and/or finishes;

And the attribute of this location object of frame sequence group joined in the attribute of concentrating corresponding position.

Described frame sequence group is corresponding with continuous in logic video segment, and the attribute of frame sequence group location object comprises:

The source of precedence information, enciphered message, copyright information, customer information, the operation set of being supported, information and/or target information, position collection joining day and/or effective time;

Customer information is used for the position set pair is answered the client rights explanation of object and used client segmentation information in the described object properties, described client rights explanation comprises: download authority, play authority, described use client segmentation information comprises: to the classification control of content.

The present invention also provides a kind of method that adds section object and section object attribute thereof in video resource, may further comprise the steps:

Service end is the zoning in video resource, and described area dividing mode comprises: with object zoning or free zoning;

Service end as object, and for each object is provided with corresponding attribute information, and is provided with the corresponding informance collection according to described zone.

Described object zoning comprises: by manually indicating subject area, and automatic again tracking object positions, and identify contours of objects information; Or, again by the method for interpolation, simulate the object motion track, and identify contours of objects information by manually indicating subject area respectively at several frames of being separated by.

The present invention also provides a kind of method that adds priority in video resource, may further comprise the steps:

Service end adds precedence information in the attribute information of information concentrated position collection;

Described client is carried out the union operation of diverse location according to described priority: when the frame of different priorities is play in same client simultaneously, the zone of only playing the highest frame of priority or different priorities when the demonstration of same frame, the zone that display priority is the highest.

The present invention also provides a kind of and has collected the method for user profile by position set pair in the frame of video being resembled operation, may further comprise the steps:

Client obtains the information set of Streaming Media and described Streaming Media correspondence;

The information of client executing and institute's receiving media correspondence is concentrated operation set, and information set content and customer information are sent to the expansion service end;

The expansion service end is collected from the customer information of client and medium relevant content information; Described customer information comprises: client's the network address, Customer ID, client properties.

The present invention also provides a kind of method of using information set in frame of video, may further comprise the steps:

Service end obtains to need to add the frame of video of information set;

Chosen position adds information set in frame; Described chosen position is included in the head of frame of video or at the afterbody of frame of video.

The present invention also provides a kind of method that adds the regional location profile in video resource, may further comprise the steps:

Described regional location is divided into the square of identical size, and described square according to pixels calculates and comprises: 1 * 1,2 * 2,4 * 4,8 * 8,16 * 16,32 * 32; And each straight line passed foursquare situation with a number mark;

Described foursquare when being passed by the regional location profile, mark penetrates and passes foursquare 2 points, connects described 2 parts that are used as the regional location profile with straight line then;

When described regional location profile all when passing foursquare straightway sign, pass foursquare situation according to straight line and find out of the most approaching existing number mark, pass foursquare situation number and come mark according to predefined again.

The present invention also provide a kind of on the existing video structure of frame of video the method for setting area or region contour, may further comprise the steps:

During video coding, on existing 3 D video data, add new plane, and in this plane setting area or region contour;

Service end is encoded new plane and original video data together and is sent to client;

Described in the plane method of setting area be: with the method for zone number or adopt the method for geometric shape parameters;

The number on described new plane can be one or more.

The present invention also provides the method for positional information in a kind of definite service layer and controlling object, may further comprise the steps:

Receive video information, and in the ordinary video broadcast layer displaying video information;

The service layer that superposes on the ordinary video broadcast layer determines the positional information in the service layer, and the Position Control new media object of determining in described service layer;

The position of described new media object information position collection definition that information is concentrated or in client by mouse or the selected fixed position of keyboard;

The method of described operation new media object comprises local control and far-end control, and local control refers to control the new media object by keyboard or mouse, and far-end control is that service end is controlled the new media object by the mode of information set;

The method of described control new media object comprises: create object, mobile object, nullify object, object conversion;

Described new media object comprises: video, animation, picture, sound or literal.

Compared with prior art, the embodiment of the invention has the following advantages:

In the embodiment of the invention, introduced the position set pair and resembled the notion that resembles attribute with the position set pair, can do more accurate control video.Change an existing video technique weight and contract, despise the present situation of application, for the application of video technique provides a good implementation platform.The present invention is using and video itself combines closely, and compounding practice collection and function collection are finished the interactive function with the video reception client then.The present invention is for the better function of performance location object, to the position object definition various attributes, the application to location object can be better brought into play in the introducing of these attributes.

In the embodiment of the invention, introduce the notion of position collection, operation set and function collection, and new communications method realizes the interaction function with the user; Well finished the interaction function with the user, can also finish accurately, therefore can realize the personalization of serving each user being pushed its needed content the collection and the analysis of user profile.For example, user often clicks in which type of perhaps commodity, pushes which type of advertisement just for this user, can realize that like this advertisement chases after the people, realizes the change of advertisement technology.

Description of drawings

Fig. 1 is a kind of method flow diagram that uses information set in video resource of the present invention;

Fig. 2 is position collection among the present invention, operation set and function collection correlation schematic diagram;

Fig. 3 utilizes position collection, operation set and function collection to carry out operational flowchart among the present invention;

Fig. 4 is that the position collection comprises object division schematic diagram among the present invention;

Fig. 5 is the program frame sequence group structure chart that has initial code and end code among the present invention;

Fig. 6 is the schematic diagram that jumps to another appointed area among the present invention in piece image from the appointed area;

Fig. 7 is the schematic diagram of corresponding position collection, operation set and the function collection in three zones in the piece image among the present invention;

Fig. 8 realizes fetching operation chart in the successive frame among the present invention;

Fig. 9 is the schematic diagram that a frame carries out jumping to after the respective operations another frame among the present invention;

Figure 10 is that the interior viewing area of a frame jumps to appointed area schematic diagram in another frame among the present invention;

Figure 11 is that the interior viewing area of a frame jumps to another frame schematic diagram among the present invention;

Figure 12 is the appointed area schematic diagram that a frame jumps to another frame among the present invention;

Figure 13 is a schematic diagram of representing an image inner region among the present invention with different set of digits;

Figure 14 adopts 16 dividing methods to represent an image outline schematic diagram among the present invention;

Figure 15 is that 8 * 8 macro blocks are handled schematic diagram among the present invention;

Figure 16 is the schematic diagram after Figure 13 process center processing among the present invention;

Figure 17 uses ellipse or profile schematic diagram of rectangle mark among the present invention;

Figure 18 is the method flow diagram that uses information set among the present invention in video resource;

Figure 19 is unique definite its position view in image in the position of each macro block among the present invention;

Figure 20 is a kind of area dividing schematic diagram among the present invention;

Figure 21 is that schematic diagram is divided in a kind of typical priority zone among the present invention;

Figure 22 is a kind of system construction drawing that adds information set in video resource among the present invention;

Figure 23 a and Figure 23 b are another kind of the present invention adds information set in video resource system construction drawings;

Figure 24 is a newly-increased service frame schematic diagram among the present invention;

Figure 25 a and Figure 25 b are service area schematic diagrames in the frame of video of the present invention;

Figure 26 be the present invention under the pattern of message-driven, service end, client and expansion service end are the schematic diagrames of cooperating;

Figure 27 is the present invention under the pattern that generates the information set file, and service end, client and expansion service end cooperate the schematic diagram of finishing function;

Figure 28 is that the present invention increases one or more dimensions and comes the distinguishable region schematic diagram on existing YUV 3 d video encoding basis;

Figure 29 is the structural representation of service layer of the present invention;

Figure 30 is service layer of the present invention and ordinary playing ATM layer relationsATM figure.

Embodiment

Among the present invention, in video resource, use information set, can adopt desired location collection in video resource for some TVs, film or advertising message, then the position collection is associated with relevant operation set, then position collection, operation set and a certain concrete function association are got up to realize certain function.

The position collection comprises: the coordinate of particular location, perhaps macro block, the band positional information in the frame in the frame of video or in the image; Or frame of video is interior or the interior appointed area of image or appointed area profile or slice-group positional information; Or the station location marker of frame of video in the entire frame sequence; Or program frame sequence group id; Or traffic identifier;

As shown in Figure 3, it is as follows the method for position collection to be set:

Frame of video the coordinate interior or particular location that image is interior is (x, y), and the macro block position in the frame can number be identified or be identified by the coordinate of macro block by intra-frame macro block, and band can be identified by the bar reel number, and band is easy to be identified as an independent transmission structure.Frame internal coordinate structure is a some object, though band or macro block also are zones, also is basic display unit, therefore in embodiments of the present invention also as a some object handles.In transmission, can be placed in the frame and transmit in the service area, also can transmit with the mode of service frame.

Slice-group, appointed area or appointed area profile are in embodiments of the present invention as a section object in the frame of video.The method that slice-group is represented is ripe now, has the label of slice-group to represent.The appointed area object can be used the method for slice-group and represents, is expressed as area code at last.In difference zones of different or profile, can adopt the area code of the embodiment of the invention, shown in Figure 13 to 17.If adopt the method representation zone of similar slice-group then need coding separately,, then do not need independent coding if adopt the mode of regional number.Can on existing YUV 3 d video encoding basis, increase one or more dimensions and come distinguishable region, as shown in figure 28, also can adopt the method for service frame, in service frame, distinguish different regional locations.When adopting the method for the existing dimension of above-mentioned increase video, can be placed on the information that increases that service area comes coding transmission in the frame of video, also can be placed on coding transmission in the service frame.Can certainly come transmission region information with the mode of control documents or message.

The station location marker of frame of video in the entire frame sequence is the sequence number of frame, and each frame all has a numbering or initial code/end code to represent this frame or the position of image in the entire frame sequence.Can be placed on this positional information in the service frame and to transmit, make things convenient for the adding of control and operation set and function like this.

The position of program frame sequence group can be identical with the position of frame of video, adopts the sequence number of a frame to identify, and perhaps adopts independent structure, as shown in Figure 5.Purpose is in order to distinguish program one by one in continuous video transmission process, and the differentiation of program often needs human intervention.Where artificial setting is the beginning of program, where is the end of program.Can adopt in the frame equally or the outer service control model of frame.

The method of video flowing sign can be provided with the number of video flowing, as 1,2,3....Perhaps adopt IP address (comprise raw address or destination address, comprise broadcast address and non-broadcast address) to distinguish not homogeneous turbulence from different places; Perhaps adopt the independent identifier number of each channel to identify.The method of transmission still is can adopt in the frame or two kinds of control models of the outer service of frame.

It should be noted that, because the position collection has certain attaching relation, for example, a coordinate or a macro block necessarily are included in the zone, this zone further is included in the frame again, a frame may be included in one section program frame sequence group, and this program frame sequence group necessarily belongs to some concrete stream, so just make if the more accurate position of sign, in Fig. 4, be expressed as the more position of lower floor, often need to comprise the more position attribution on upper strata of this position, for example, determine the position in a zone, tend in following a kind of mode:

* stream＞* * program frame sequence group＞* * frame or layer＞* * zone, wherein "＞" represents the hierarchical relationship in zone, this hierarchical relationship also has represented in Fig. 4.

Its middle level includes the service layer that defines among ordinary video broadcast layer and the present invention, and the size of service layer is identical with video playback layer size usually, but service layer is positioned on the video playback layer.Concentrate certain zone, region contour or the concrete coordinate position that can accurately navigate to equally in the service layer in the position.

Information set of the present invention, operation set and function collection all are abstract collective concepts, do not represent really to have such function title or unit in the application of reality.As long as belong to method logic of the present invention, all belong to the content of the present invention's protection.

The invention provides a kind of method of in video resource, using information set, as shown in Figure 1, may further comprise the steps:

Step s101, by managing in adding mode in outer adding mode of frame of video or the frame of video and transmitting information set as the carrier of information set, the outer adding mode of frame of video comprises the mode of information set description document mode, service frame mode or message communicating to service end in video resource.Wherein, information is concentrated and is comprised position collection, operation set and function collection.The position collection further comprises: the coordinate of particular location in the frame of video or in the image, and as the horizontal ordinate value of certain point or pixel in the frame of video or the longitude and latitude coordinate figure of sphere, the perhaps macro block in the frame of video, or band positional information; Or appointed area or appointed area profile, slice-group positional information in the frame of video or in the image, profile usually and in the video resource some position or object corresponding, adopt Methods for Coding to distinguish in the frame of video or in the image in concrete contours of objects or position coordinates, the frame of video or the zones of different position or the profile of division in the image; The station location marker of video resource in the entire frame sequence, as the initial code of video resource, end code etc., i.e. the position of certain concrete program segment corresponding beginning or end frame in this net cast program request or sequence numbering etc.; Or program frame sequence group id, in order to identify the set of the frame that one section content is associated, as a collection of drama of TV play, one section video recording etc.; Or traffic identifier.

In addition, the position collection also comprises the attribute information of position, comprise priority in the attribute information, priority is used for the union operation of diverse location: when the frame of different priorities is play simultaneously in same client, the zone of playing the highest frame of priority or different priorities when the demonstration of same frame, the zone that display priority is the highest.

The all corresponding object in each position is concentrated in the position: the coordinate of particular location, the perhaps macro block in the frame, a positional information-point of the correspondence object of band in the frame of video or in the image; Or a block object in appointed area or appointed area profile, the slice-group position-corresponding frame of video in the frame of video or in the image, this piece is the set of point or macro block or band; Or frame of video station location marker-frame object of correspondence in the entire frame sequence; Or program frame sequence group id-program object of correspondence; Or traffic identifier-flow object of correspondence; Location object all comprises the attribute information of one or more objects, and attribute information comprises: the source of precedence information, transparence information, enciphered message, copyright information, customer information, the operation set of being supported, information and/or target information, position collection joining day and/or effective time etc.

Object properties medium priority information is used for the union operation of diverse location collection: when the stream of different priorities is play simultaneously at same player, play the highest stream of priority; When the program frame sequence group of different priorities is play simultaneously, play the highest program frame sequence group of priority in same player; When the frame of different priorities is play simultaneously, play the highest frame of priority in same client; Or the zone of different priorities is when showing in same frame, the zone that display priority is the highest; The a plurality of information that are different priorities are in the concentrated same position in position, and its broadcast simultaneously in same player, only play the highest information of priority.Transparence information is used for the position set pair is answered the transparency definition of object in the object properties; Enciphered message is used for the position set pair is answered the encryption of object in the object properties, comprises cipher mode, key information; Copyright information is used for the position set pair is answered the copyright notice and the protection of object in the object properties, comprises the attaching information of copyright, the authentication information of copyright, the use information of copyright; Customer information is used for the position set pair is answered the client rights explanation of object and used client segmentation information in the object properties, the client rights explanation comprises (also can be placed among the DRM of copyright information): download authority, play authority, use client segmentation information to comprise: to the classification control of content itself.

The function collection further comprises: fetch concrete assigned address content object information, jump to concrete assigned address, to the appointed object position send information, open or insert assigned address object, close the object of real assigned address and the object of mobile assigned address.Wherein, assigned address comprises: particular location in particular location, the player plays window in particular location, the browser in some memory locations, the display screen in some device addresses, the memory device in concrete URL, the hardware device in the network.For the priority feature that realizes that the position is concentrated, need concentrate in function precedence information is set, in different zones different priority is set for area dividing, stack shows to multiple image in same image then, determines final image each several part priority.Typical case for area dividing such as Figure 21 uses, and different priority can be set in different zones, and priority is represented with P, supposes 0 grade for highest, and 1 grade time high, and priority reduces successively.Can in different images priority be set, stack shows in same image then.Such as, image 1 and image 2 are shown as image 3 after superposeing by priority.The priority of a-quadrant is up to 0 in the image 1, is greater than the E zone in the image 2, so the result after same position demonstrates stack in image 3 is a-quadrant value in the image 1.In like manner, the B zone priority in the image 1 will be higher than the F zone in the image 2, and therefore the result after the stack is a B regional value in the image 1 in image 3.Can find that in like manner the G in the image 2 and the priority in H zone are greater than position C and D identical in the image 1, therefore the situation of finally having synthesized image 3.

Operation set claims the active information collection again, further comprises: pre-set program operation and messaging program driving operation etc. are also pressed in search information collection position when mouse action, keyboard operation, broadcast.

Described position collection, operation set and function collection can adopt any proportionate relationship correspondence, comprising: position element of set element: a plurality of operation set elements: a plurality of function element of set elements; A plurality of positions element of set element: a plurality of operation set elements: a plurality of function element of set elements; Position element of set element: an operation set element: a plurality of function element of set elements: a plurality of positions element of set element: a plurality of operation set elements: a function element of set element; Position element of set element: a plurality of operation set elements: a function element of set element; A plurality of positions element of set element: an operation set element: a plurality of function element of set elements; Position element of set element: an operation set element: a function element of set element; A plurality of positions element of set element: an operation set element: a function element of set element.

In frame of video or the some zones of image the method for concentrating the frame inner region to obtain in the position be set have three kinds:

A kind of is FMO pattern in adopting H.264, by any assignment macro block of macro-block order mapping table (MBAmap) being set to different sheet groups, the position of sheet group zone as the adding information set.The FMO pattern has been upset former macro block order, has reduced code efficiency, has increased time delay, but has strengthened error-resilient performance.The pattern of FMO mode division image is various, and important have checker board pattern, a rectangular pattern etc.Certainly the FMO pattern also can make the macro block order in the frame cut apart, and the size of the sheet after feasible cutting apart is less than the MTU size of wireless network.Therefore can be sheet group position as the position that adds information set, promptly corresponding with a certain concrete information the sign of sheet group.

A kind of is the method that adopts the VOL among the MPEG4, i.e. independent foreground object stream, object data stream in frame the corresponding display position as the position that adds information set.

A kind of is to adopt algorithm that image recognition algorithm, object track algorithm, foreground object extract or by manually indicating the subject area method by interpolation more respectively at several frames of being separated by from background, mark off different zones in frame, above-mentioned zone is as the position that adds information set.

The information that adds will work, and at first must can be positioned in video resource, and promptly the position exists and can locate, and then can extract operation set and function collection.Usually the method for handling position collection information has two kinds of situations, a kind of is in existing video resource, can unique position of determining a certain frame as the frame informations such as sequence numbering of frame, and for example the position coordinates of image (pixel is represented) so only needs defining operation collection and function collection to get final product.Another kind is not have in the existing video resource, as the concrete contours of objects information in the video resource, for another example the information of an area information of dividing in the video resource and a complete programs of sign.These information all need to define in the present invention, and should get up these positional informations and operation set and function set pair.

Service area can be placed in the existing frame of video in the frame of video, existing frame of video is divided into frame header, with video requency frame data two parts, and the frame of video service area can be placed on existing frame of video afterbody, it is data division back in the frame of video, perhaps be clipped between existing frame of video head and video data two parts, shown in Figure 25 a and Figure 25 b.

Step s102, service end sends to client with information set.The position collection is defined in the video resource usually, and operation set and function collection have following two kinds of methods to realize usually.First kind is that the subset information of operation set and/or function collection is also transmitted to client by server end, and define the complete or collected works of operation set and/or function collection in client, client receives the operation set of service end or the subclass of function collection by preset program, and a certain function is carried out in the concrete operations that have more the user then.In transmission, can see that date information or control information transmit to operation and the subclass of function, often voice or video are separated with control information with rtcp protocol as Real-time Transport Protocol in the existing host-host protocol, also or in the TS structure divide the mode of opening packing to transmit Video, Audio and data, can also transmit the content of operation subset and/or function subset by an independent file.Second kind of service end open position put collection, and operation set and function collection only are defined in client or server end.Call by method complete operation collection such as program far call (callback) or message and function collection, finish predetermined function.Shown in Figure 23 a and Figure 23 b, both can look audio frequency and service data with the different port transmission respectively, also can be encapsulated in the structure, looking audio frequency and serving uniform data by same port transmission.After if client receives video content and information set again, equally again video content is edited, added new information set, and when video content issued service end or expansion service end, in this new reciprocal process, client is being played the part of the role of service end in fact.Therefore the in fact still pattern of C/S (client/service end), not change in essence.

As long as in fact client can the acquired information collection, just can finish the function of the embodiment of the invention.As for obtaining therefrom is not unique, can obtain from the information set service end, and as Figure 22, at this moment information set service end and media services end are referred to as service end, can finish appointed function at the artificial configuration information set content of client yet.Information set is normally put together with the media services end, but also information set can be placed on the server different with the media services end.

Step s103, the position collection information that client is concentrated according to information is determined active position, and the operation set that utilizes this position set pair to answer is operated, active position collection and/or operation set institute corresponding function collection, carry out corresponding function, wherein operation set and/or function collection can define in the client definition and/or in service end.Wherein, operation set that the position set pair is answered and function collection set in advance in client, perhaps send to client by service end, and this position collection must send to client by service end.Operation set and function collection can not be included in the information that service end sends to client and concentrate, but define in client or expansion service end in advance.

Client can define the complete or collected works of information set, comprises all position collection, operation set and function collection, thereby can judge whether the information that sends to client from service end is included in the complete or collected works of information set; Service end can define the complete or collected works of information, comprises all position collection, operation set and function collection, thereby can handle the adding information set to original video.

Be elaborated below in conjunction with specific embodiment, as shown in Figure 2, position collection, operation set and the function collection Trinity, collaborative work.The position collection guarantees that some positions can be determined by unique in video resource, and can guarantee that this position can be by one or more fixing operations or automatic actuator-activated one or multinomial new service function.Position collection information can be obtained by joining in the coding or in the mode of an independent file, perhaps and watch that the user sets up special interface channel and obtains by the mode of message, wherein the position collection is included in the video resource, as in the code stream, frame of video is medium.The position collection might not be corresponding with some positions in the video image of seeing, but abstract remembering with gratitude.The position collection is corresponding with operation set, and a kind of operation of some positions is corresponding with one or more function collection.And each function tends to a position is operated or function is realized that the result turns back to certain position, more than two kinds of positions not in the position centralized definition, reason is because the variation of function is varied, that be difficult to determine defines some positions as function operations or the position returned, the position that nearly all position all can be used as function operations or returns.Position collection, operation set and function can be provided with a complete or collected works' notion, but because the described envelop of function of function collection is too open, also complete or collected works can be set.These operation sets be obtained or be stipulated out to operation set information can by the mode that the user receives in the user side program.Corresponding again one or multinomial function collection of each operation in the operation set, function collection information can and be stipulated out these function collection by user's reception in the user side program, and also will stipulate out all function collection and realize these functions in the service end that the function set pair is answered.Sometimes client also realizes partial function as server end simultaneously, for example, the realization of turn function, the user can jump among some concrete URL by a certain particular location in the click video resource and go, and this turn function can be finished in service end automatically as the subclass of function collection.

The information set information that is provided with in some video datas or the image, the information type that corresponding one or more information are concentrated, the operation in corresponding a certain or several operation sets just can be finished function and concentrate a certain or several specific functions.As shown in Figure 3, client judges that at first position collection information that information concentrates is whether in the complete or collected works of position collection, if do not exist, then do not have the operation or operate invalid, if, then obtain current operation set, judge in this position to concentrate whether there is corresponding operation in operation set then, described operation set should be in the operation set complete or collected works; If have, then the program command of the function collection of executing location collection and operation set correspondence if do not have, is not then carried out the program command of function collection.

Increase the notion of service frame in Fig. 3, the effect of service frame is the carrying information on services, and the existing frame structure of the least possible change.For the convenience of transmitting, the most videos on the existing network all are the video informations after overcompression.In order to add the convenience of special services, corresponding existing frame of video as I frame, B frame, P frame, adds the notion of service frame, one or more frames that each service frame is corresponding continuous or discrete; As shown in figure 24, service frame X corresponding A BCD four frames.

Service frame comprises following content: the pairing frame of video of service frame (the frame of video here refers to the frame by the transmission of video coding of compression); The message set of corresponding frame of video comprises: position collection, function collection and operation set.Service frame can be placed in the video flowing shown in Figure 23 b to be transmitted, and perhaps is placed in the service flow and transmits, shown in Figure 23 a.And service frame corresponding discrete or continuous one or more frame of video.If during service frame of a service frame correspondence, all information on services of frame of video of the service that provides will be provided service frame, these information are included in the message set.

An emphasis of the present invention is the data structure that this off-gauge data structure of existing video flowing is made into standard, its target is, can locate any one position in this video flowing easily, as shown in Figure 4, for existing flow label goes out, the accurate positional informations such as position of the interior concrete coordinate of numbering, program frame sequence group position and numbering, frame position and numbering, subject area or the region contour position of stream and numbering and band/macro block/frame, and complete position collection of these information formations.

For the position of frame, existing MPEG-2 system specifications has defined three kinds of packets (PES, PS and TS) and two kinds of data flow (PS and TS).The multiplexing single data flow that forms of Packet Elementary Stream (PES-Packetized Elementary Stream) that will have common time reference is called program stream (PS-Program Stream).Video elementary code stream (ES-Elementary Stream) is meant the data flow that only comprises 1 source encoder.Each ES is made up of plurality of video (comprising I, P or B frame) or Audio storage unit (AU-Access Unit).Each AU comprises head and two parts of coded data.ES is grouped into after the PES, and each PES bag is made up of packet header, the peculiar information of ES and 3 parts of bag data.PES packet header ceases 3 parts by start code prefix, data flow identification and PES bag long letter and constitutes.The bag start code prefix constitutes with 23 continuous " 0 " and 1 " 1 "; The data flow identification of expression useful information kind is the integer of 1 8bit.By the bag initial code of the two synthetic 1 special use, can be used for the character and the sequence number of data flow under the recognition data bag (video, audio frequency or other).The peculiar both information of packet header and ES can be synthesized 1 data head, comprises predetermined demonstration time PTS of temporal information and decoding time D TS.The bag of PES stream can random length, even can be the length of whole sequence.PES further can break into PS bag or TS bag, forms program stream or transport stream.These characteristics have determined can change mutually between program stream PS and the transport stream TS.The PS bag wraps 3 parts by packet header, system's head, PES and constitutes.Wherein PS packet header is made up of multiplexing code check 4 parts of essential part, SCR expansion and PS that PS wraps initial code, system clock reference (SCR-System ClockReference).Therefore can in the calculator structure of TS, find the sequence number of each frame.Perhaps find the position of GOP (image sets), find the position of concrete frame then by the sequence number of frame in image sets.

Can define the sequence number of special frame of video in whole video sequence voluntarily equally, and this numbering is placed on is sent to client in the video flowing and discerned.The sequence of frames of video numbering should be calculated if press per second 30 frames more than or equal to 3 bytes, and 3 bytes of intraday video frequency program totalframes just can complete representation.This number of frames is placed on the head of transmission unit usually.Above method equally also can be placed in the RTP structure for the built-in sign of frame is placed among the existing TS.Can also be placed in the service frame of the present invention's definition.

Numbering for stream can be placed in the transmission structures such as existing TS or RTP, as the inside, TS packet header or extension bits or the like, also can be placed in the service frame of the present invention's definition.

Can be placed on equally in existing TS or the RTP transmission structure for program frame group row group # and location definition,, also can be placed in the service frame of the present invention's definition as the inside, TS packet header or extension bits etc.But it should be noted that program frame sequence group is different with GOP of the prior art (image sets), the image sets notion does not have the program notion, do not comprise between the image related logic implication in the image sets, just image sequence simple be divided into different image sets unit.And the program frame sequence group among the present invention is one group of related in logic frame of video, often an independent program or related in logic video segment.

Numbering or sequence number for the zone in frame of video or the image or slice-group or region contour can be placed in TS or the RTP transmission structure, and as the position, packet header, but the interior perhaps attribute in zone then advises being placed in the service frame of the present invention's definition.Can certainly all be placed on all frame of video or image inner region information in the service frame.For video internal coordinate, band (slice) and macro block also is to adopt above-mentioned processing method.But it should be noted that band, slice-group, the position of macro block has had clearly regulation in the prior art, and it is peculiar that other position then belongs to creativity and innovation of the present invention.

The above, every mode of utilizing the carrying of space in packet header or the frame in RTP or TS all belongs to service manner in the frame that the present invention mentions, and utilizes the mode of service frame or file all to belong to the outer service mode of frame.

Program frame sequence group in the video flowing, program frame sequence component is concrete frame, comprise slice-group, band, macro block and concrete point coordinates in the concrete frame, the scope of position set identifier is actually an object notion, as program frame sequence group corresponding to a video frequency program or a video segment object that logical communication link is arranged, this object is included between the initial code and end code and end code of program frame sequence group, the numbering that comprises this program frame sequence group simultaneously, and property location, this property location again to should the section program some attributes.Equally, the corresponding image object of frame of video is equivalent to a plane, and each frame of video also has the initial code and the end code of frame, and the attribute of himself is also arranged; Slice-group in the frame, zone and region contour are the equal of a section object in the image, the numbering that self is also arranged is or/and information such as attributes, protecting vital cell function is just in this zone or in the slice-group scope, band (slice), macro block, and the object of the corresponding point of the coordinate in the frame of collective, protecting vital cell function is in band, in the macro block or in certain concrete coordinate; As shown in Figure 4.Wherein, the reposition that video flowing numbering, program frame sequence group, zone and region contour are introduced for the present invention, its structure as shown in Figure 5, series of frames is divided into a frame group, the frame group has inner relevance usually, as a certain collection of drama in the TV series, and defines a program initial code and end code marks this section program, Fig. 5 is a kind of abstract implementation method, has promptly indicated initial code and end code, program code, programme attribute etc.This part can carry with existing TS or RTP mode, promptly is placed in the existing structure packet header, promptly is mode in the frame of mentioning among the present invention.

As shown in Figure 4, if adopt the mode of service frame, the position that can control comprises video flowing position, program frame sequence group position, frame of video position, subject area, region contour, band, empty piece, coordinate position.Service area then can be controlled other position collection information except that video flowing in the frame.What need stress is, the notion of service frame is an abstract concept among Fig. 4, its objective is in order to control continuous or discrete a frame or multiframe to be provided with, and why being service frame is to differentiate for the frame of video with other.Adopt the bearing protocol of which type of frame structure, frame length, employing all scope not to be discussed as for such service frame in the present invention.The present invention only stipulates the content of the information set that comprises in this frame.The size of service frame is also fixing, can be identical also can be inequality.Service area concept is and existing transmission packing manner in the frame, and a service concept of frame format correspondence, by in frame of video packing transmission courses such as TS stream or RTP or the mode that in existing frame format, adds all belong to service area pattern in the frame.Service document mode in Fig. 4 is meant with the mode of file indicates these positional informations, may also can comprise the out of Memory set content in the file certainly.The service document mode mainly is to generate such file, then the information set storage in this document.And massage pattern mainly is to be used in the mode that service end and client need be carried out message in real time, information set, comprises that position collection, operation set, function collection convert a rule message to and transmit between service end and client in this mode.

Can realize generally comprising control and management to Media Stream in outer management of frame and the frame and manage by in video resource, adding information set among the present invention.Wherein, the outer management of frame comprises service document pattern and direct transmission mode, service document pattern use location level, operation set and function collection; Direct transmission mode is used control data (for example service frame, control flows, control data).Management promptly increases the position set content in the frame in existing frame structure, and operation set and/or function collection also can comprise wherein.For example all be reserved with video expansion initial code or reserve code etc. in existing coding structure, these reserve initial code or end code that sign indicating number can be used as information set, the content of the information set that increases.

For example, in the AVS coding, initial code is one group of specific Bit String.In the bit stream that meets GB/T 20090.2, these Bit Strings should not appear in any case except that initial code.Initial code is made of start code prefix and initial code value.Start code prefix is Bit String ' 0,000 0,000 0,000 0,000 00000001 ', and all initial codes are all answered byte-aligned, and initial code value is one 8 bit integer, is used for representing the type of initial code, sees Table 1.

The initial code value of table 1

The initial code type	Initial code value (hexadecimal)
		Band initial code (slice_start_code)	00～AF
Video sequence initial code (video_sequence_start_code)	B0
		Video sequence end code (video_sequence_end_code)	B1
User data initial code (user_data_start_code)	B2
		I image initial code (i_picture_start_code)	B3
Keep	B4
		Video expansion initial code (extension_start_code)	B5
PB image initial code (pb_picture_start_code)	B6
		Video editing sign indicating number (video_edit_code)	B7
Keep	B8
		System's initial code	B9～FF

Can obtain the Bit String identical when the part grammar element is got particular value, be called pseudo-initial code with start code prefix.Reserve code B8 here and video expansion initial code, so the initial code B9～FF of system all can be used as the initial code or the end code of information set.In a word, when a kind of video coding of definition, the sign indicating number position that can stay of this sort initial code or nothing use temporarily in coding all can define original position or the end position of an information set in frame of video.Above information set initial code has been arranged, the content that just can between initial code and end code (if existence), add information set, and can indicate by different initial codes and distinguish the different information contents, the information content also can clearly define the information content more specifically, with different levels definition after the initial code of above definition.As, initial code B8 indicates that information set begins, and then C9 indicates it is the position collection, and then having D9 to indicate again is the regional location that concentrate the position, and E9 indicates that the attribute of regional position is a priority attribute.Like this can be to the definition of accurate realization position and attribute thereof.

When realizing program frame sequence group such as needs, also can adopt the method for above frame inner control to add information set, indicate information set as B10, then C10 indicates it is the initial code of a program sequence group, begin to define the attribute of this program after the D10, classification, enciphered message etc., so just can when decoding, clearly know the attribute of some contents of program, thereby can better control the broadcast of program, as, this program is not suitable for children and watches the classification that can indicate this program in attribute, so just can be when playing, according to playing whether broadcast program content of Object Selection.And for example, can in attribute, add and encrypt or authentication information, judge whether this program is legal program.Also can add DRM checking content therein.Below all belong to method by the mode beared information collection of service area in the frame.

Subject area also is a kind of in the present invention peculiar zone, this zone is corresponding with the some concrete object in the image, as shown in figure 17, mark a subject area with ellipse or rectangle, subject area generally is a closed area, if object motion is to the video border, then also can constitute closed areas with four image boundaries up and down.Usually adopting identical data set to identify in this closed area, such as being 1 in the zone, is not 0 etc. in the zone.The also available coordinate of subject area is represented, in image, can identify with horizontal ordinate, or a concrete macro block, or the some pixels in the macro block.

The schematic diagram that jumps to another appointed area in piece image from the appointed area specifically is illustrated in the situation that is jumped to the y zone in the image A by the x zone as shown in Figure 6, and wherein, display position is A:x, and respective operations is redirect, and jump location is A:y.

As shown in Figure 7, three zones in x, y, the z presentation graphs, x respective operations collection is a mouse action, and corresponding function collection is the information of fetching some positions, and this is fetched information bit and is changed to " http: // wait the network address "; The operation set of y correspondence is keyboard operation, and corresponding function collection is the information of fetching some positions, and this is fetched information bit and is changed to " hardware address (as address in the hard disk) "; The operation set of z correspondence is other button operation, and corresponding function collection is the information of fetching some positions, and this is fetched information bit and is changed to " memory address ".

As shown in Figure 8, in some continuous frames, frame initial code or end code drive certain operations, as after reading C frame position initial code, fetch some information to a certain position of internal memory automatically; And for example in the A frame, the mouse action of being scheduled to can be fetched the information that http protocol points in the network; In the A frame during by keyboard operation, the information of fetching local hardware is such as content in the hard disk etc. for another example.

As shown in Figure 9, the A frame has jumped to the situation of B frame after carrying out corresponding skip operation.

As shown in figure 10, the x zone in the A frame has jumped to the y zone in the B frame through corresponding skip operation.

As shown in figure 11, the x zone in the A frame has jumped to the position of B frame through corresponding skip operation.

As shown in figure 12, the B frame carries out corresponding skip operation, has jumped to the x zone of A frame.

As Figure 13, represent to represent the method for an image inner region with different set of digits, represent with 2 that at the heart type edge of image macro block macro block of heart type image inside is represented with 1.

As shown in figure 14, adopt image outline of the more accurate expression of 16 dividing methods.As shown in figure 15,, judge that then m is more close A or B if straight line L passes one 8 * 8 macro block and macro block AC limit meets at m and the CE limit meets at n, if hypothesis upwards for just, A, B are greater than 0, promptly Or

If set up then m point is moved to position with the coincidence of A point, the m point is moved to the position of B if inequality is false.In like manner the n point is implemented identical processing, so just can obtain the situation of the right that width of cloth figure among Figure 15.Remove to compare the coding among Figure 14 so again, this situation of can determining that the coding of Figure 15 has become " 2 ".Adopt this method, go to handle heart pattern among Figure 13 again, become Figure 16 situation, profile information is come out by good mark like this.

Figure 17 is to use ellipse or profile schematic diagram of rectangle mark, only needs three parameters, elliptical center coordinate, major axis and minor axis value during with oval mark.Equally also only need three parameters, rectangular centre coordinate, long limit and minor face value for rectangle marked.When ellipse long and short shaft equates, just become circle; When equating, rectangle length limit just become square.

The embodiment of the invention can comprise client, server 1, server 2 and server 3 according to realizing the function difference.Server 1 provides the media data service, and server 1 need be told client the function after the operation of positional information and correspondence and the operation; Server 2 is function servers, and the function collection is normally finished by server 2, and perhaps client oneself is finished, and perhaps the cooperation of client and function server is finished.Server 2 is finished or client and server 2 cooperates and finishes a function that then needs corresponding and tell server 2 by server 1 if desired, and server 2 just can help client to finish the concentrated concrete function realization of function like this.Server 3 is statistic analysis servers, be responsible for analyzing and adding up client user's behavior, such as what click all is which type of information content etc., so just can be the service of specific client end subscriber customized personal by analyzing, and tell server 1 users ' individualized requirement by server 3, guaranteeing to be pushed to user's data can be with having more attraction and better service efficiency.

Wherein, the specific implementation process comprises as shown in figure 18:

1, server 1 and client are synchronous, the service operations that exists in the invoking server 2;

2, server 1 sends data to client;

3, user end to server 2 sends the executable operations request;

4, server 2 is to the functional parameter of client return;

5, server 2 is collected client operation information to server 3;

6, server 3 is at the different data of different client push;

7, server 1 at different data to server 2 synchronous different services;

8, server 1 sends data to client.

In the present invention, since macro block can by macro block number or the position of macro block determine that if the type of macro block determines that then the length and width of macro block have been determined, so its position in image can uniquely be determined in the position of each macro block.As shown in figure 19, because the horizontal size and the vertical dimension of image define in sequence head, so can accurate localization go out the position of some pixels, with brightness is example, if macroblock size is 8 * 8, macro block position is (x, y), the position that o is ordered in macro block is that (a, b), using the same method to define each concrete locations of pixels in the video.Certainly, owing to know the horizontal size and the vertical dimension of image, also can identify a concrete location of pixels with horizontal coordinate m and vertical coordinate n.The value of m and n can provide, and also can suppose that x, y, a, b, m, n all are since 1 counting then by calculating:

m＝8×x+a

n＝8×y+b

The method of zoning comprises following two kinds of situations in frame, the one, and with the object zoning, the 2nd, free zoning.Wherein, the object zoning is divided into following two kinds of methods again: a kind of by manually indicating subject area, and automatic again tracking object positions, and identify contours of objects information; Another kind of by manually indicating subject area respectively at several frames of being separated by, by interpolation method, simulate the object motion track, and identify contours of objects information again.The method that precise marking can be arranged in the mark profile arrives Figure 16 as Figure 13; Also available pictorial symbolization realizes the rough profile of object, as shown in figure 17.The division of free space is divided into a lot of pieces to screen often according to actual needs, each piece and piece is not overlapping on every side, as shown in figure 20.

The present invention also provides a kind of system that adds information set in video resource, as shown in figure 22, comprises client and service end.By increasing information set in adding mode in outer adding mode of frame of video or the frame of video, and the code stream that will carry information set sends to client to service end in video resource; Wherein, the outer adding mode of frame of video comprises the mode of information set description document mode, service frame mode or message communicating; Client is determined active position according to the position collection information that information is concentrated, and the operation set that utilizes this position set pair to answer is operated, active position is concentrated the corresponding function collection, carries out corresponding function.

Wherein, service end specifically comprises: medium import module, are used for Media Stream is imported service end; Information adds module, is used to generate the information set file and/or information set is added media file; The media store module is used for stored information collection and/or media file; Mixed-media network modules mixed-media is used for service end and sends information set and/or media file to client.

Client specifically comprises: mixed-media network modules mixed-media is used for obtaining information set and/or media file from service end; The information Recognition module is used to obtain and the identifying information set content, comprises position collection, operation set and function collection; The operation induction module is used to obtain the operation set institute operation that the position set pair is answered with carrying out; Function realizes module, is used for the pairing function collection of trigger position collection and operation set, carries out corresponding function; The media play module is used to play corresponding media file.Wherein, service end can cooperate realization information set corresponding function or client can cooperate realization information set corresponding function with one or more service ends with one or more clients.

Certainly, upgrading or expansion needs for system can increase the expansion service end, and client cooperates the function of finishing appointment with the expansion service end; The expansion service end comprises: function realizes module, is used for realizing that with client functionality module cooperates, and the information of finishing is concentrated corresponding function; Mixed-media network modules mixed-media is used for client and communicates by letter with the expansion service end; The expansion service end can cooperate with one or more clients realizes that information set corresponding function or client can cooperate realization information set corresponding function with one or more expansion service ends.On system level, service end, client and expansion service end can merge in twos, and be promptly separate on the function, also can be placed on to realize or be placed on the software platform in the hardware realizing.In the utilization of reality, position collection, operation and function collection may go out item with specific functional form, as operation set being defined in client or service end or expansion service end; The function collection is also realized in client or expansion service end with specific program.

It should be noted that client and service end are notional the separation, client and service end may reside in hardware and/or the software environment.For example, when the user when client adds new object voluntarily, also carry out the function of service end in this time client simultaneously, need information set equally, information set comprises equally: position collection, operation set and function collection.Just these parts can be integrated in the client-side program language or be partially integrated in the client-side program language or in the file of independent client.Information set transmits or reads all has the cooperation of client software and hardware to finish.Such way main purpose is for the client is free to edit existing video frequency program or video file, and such file can be done and upload or download, that is, the user can pass through existing position collection editing video or video file.

Among Figure 22, Media Stream imports module importing medium service end by medium, adds module by information then and adds information set (position collection, operation set, function collection).Wherein collection information in position is to add, and operation set or function collection information can be selected to add according to concrete application demand.The medium that are added into information set by information adding module are sent to client by network, the information set that client adds by information Recognition module identification media services end, all information are concentrated in information extraction, and wait for that the user operates.Operation set and (or) obtaining of function collection can set or obtain from the media services end by Network Transmission in client in advance by program.If user's executable operations is concentrated predefined operation, then activate the functional module in the clients corresponding, and cooperate the expansion service end to realize predefined function.At the expansion service end, there is optional function to realize that module and client functionality module cooperate, may be the pattern of C/S or the service mode of equity.Also may not need the functional module of expansion service end and independently finish some function by the client functionality module.The expansion service end at client some specific service set, also be optional equipment in whole system.

In client, can be provided with the complete or collected works of information set, like this when the video resource of client acquired information collection and information correspondence, can make a decision according to the complete or collected works of information set, and in fact the information set with the video resource correspondence that client obtains can be used as information set complete or collected works' a subclass, and whether the content that can judge described information set subclass like this is rationally or within the range of definition.Same described information set complete or collected works also can be in service end or the definition of expansion service end.

In Figure 22, service end has comprised Video service end and two functions of information set service end.The Video service end be with video resource by offering client, client can be play by the media play module then.And the information set service end is that information set is offered client, and client can realize the function that some are specific according to the information set of receiving then.In the application of reality, the Video service end can be opened in different equipment or the system with information set service end branch and provide service for client.Which type of information set bearing mode will in Figure 22, client at first know it is, is mode or the outer mode of frame in the frame? then formerly under the situation of acquired information collection, the analytical information collection, the extracting position collection is as the active position of oneself.Realize specific function according to the operation set and the function collection of correspondence then.

As shown in figure 26, under the pattern of message-driven, service end, client and expansion service end are the schematic diagrames of cooperating, also are service end, client and the system construction drawing of expansion service end under the message-driven pattern.Service end is carried out real-time communicating by letter with client by message engine, comprises information set in the message engine, and concentrated position collection, operation set and the function collection of comprising of information.Under this pattern, Streaming Media and message can be placed in same transmission channel or the different transmission channel, send to client by service end.Since the real-time of transmission of messages, the adding information set content that service end can be real-time, and the information set of sensing adding that client also can be real-time.As service end can be in real time in the medium that send certain appointed positions concentrate and add advertising message, client is when the playing media content, also can detect possible operation set in real time, if at this moment sense the advertising message of adding, and correspondence is the operation of playing advertisement automatically in operation set, then can realize playing automatically the function that service end is inserted advertisement in client.In some cases, in the time of can not finishing some complicated functions separately as client, needs and expansion service end cooperating are together finished some functions.The mode that client is communicated by letter with the expansion service end has modes such as message, direct exchanges data (comprise and transmitting and receive data) and remote program call.Under the message-driven pattern, the message engine of service end and client must be the complete or collected works that comprise message set, the definition of promptly all position collection, operation set and function collection.

As shown in figure 27, under the pattern that generates the information set file, service end, client and expansion service end cooperate the schematic diagram of finishing function, also are service end, client and the system construction drawing of expansion service end under the message-driven pattern.At first obtain video information, then as required, adopt special edit tool or editor module to generate the information set file with service end.Then video information and information set file are sent to client, the mode of transmission can adopt elder generation to send the information set file and send video information again, perhaps sends video information earlier and sends file again, and perhaps both send simultaneously.When client has been obtained the information set file, use information set identification module or identification facility and come the identifying information set content, client induction user is in the operation of position collection then, if operation is included in the information of obtaining and concentrates, be valid function, then the pairing function collection of executable operations collection and position collection.If executable operations not in obtaining the operation set of information, then is an invalid operation.When realizing client functionality, often need the cooperation of expansion service end just can finish in the information set or be kept at function in client or the expansion service end.

The mutual method of expansion servers and client has modes such as message mode, data interactive mode and remote procedure call.Can adopt XML mode or text or binary data etc. when sending data.

As shown in figure 29, client includes playback equipment, and broadcast window in the playback equipment is supported ordinary playing layer and service layer when the displaying video medium in the broadcast window.Play the video content of being received by service end at the ordinary playing layer, service layer is used for inserting new object, and these objects comprise: video, animation, picture, sound or literal etc.Control to service layer is finished by information set.Service layer's end is exactly to send video media information and information set to client.Here service end and client comprise all modules shown in Figure 22.Service layer is a transparent layer normally, is positioned on the existing video playback layer, but can inserts media information arbitrarily.

As shown in figure 30, indicated the relation between ordinary playing layer and the service layer, service layer is the independent one deck that is produced by client on the ordinary playing layer.The characteristics of this one deck are, can insert the new media object at this one deck, and described new media object comprises: video, animation, picture, sound or literal etc.The time that this one deck produces can be just to create when having the new media object or exist, or exists in client always.This one deck is transparent except the object that inserts, and can make the user see through the content that this one deck is directly seen the ordinary playing layer like this, and visually can unite two into one two-layer.As Figure 30, new object " five-pointed star " all is women's head-ornaments around the five-pointed star in service layer, when the user sees this frame, can see the five-pointed star pattern on existing broadcast layer and remove the extra-regional broadcast layer image of five-pointed star like this.In broadcast layer, have a coordinate A, this coordinate is represented the position of five-pointed star, this position can be the center of five-pointed star or upper left, upper right, lower-left, position, bottom right in definition, can also be to comprise certain geometric certain summit or the center of inserting object, as, when a circle just can wrap this five-pointed star, the position of definition five-pointed star was this circular center.The position of inserting object just can uniquely be determined like this, and corresponding coordinate necessarily can be found in this position in the ordinary playing layer, yet, the position collection that information is concentrated, define at all places in the video flowing and corresponding objects, obviously service layer exists in client, not in this video flowing structure, determines unique position but the ordinary playing layer really can find in this flow structure.Therefore can find identical location map to the object coordinate in the service layer or the band of position in the ordinary playing layer, as Figure 30, the location map of position coordinates a in the ordinary playing layer of five-pointed star correspondence is A just in the service layer.Like this, just can be in the ordinary playing layer in a certain position and the service layer some object associations get up, related as A with five-pointed star, concentrate with regard to the position of information set correspondence like this and be associated with new object, be associated with five-pointed star as A, and A coordinate itself also is equivalent to some object in an image or the frame in the present invention.So the position collection can be indicated simultaneously the object of a position correspondence own in video,, can also indicate the new object in serving layer by layer of this position correspondence as the set of zone, frame, frame in point, frame or the image, stream etc.Therefore just can with in the frame among the present invention or the frame method of carrying information set outward come this new object is controlled or associative operation.As insert a position of new object five-pointed star in the service layer in the A position, there are one-to-one relationship in A and a, know that one just can be known another one, normally in the different layers of same position, are ordinary playing layer and service layer here.Above method be by the position of ordinary playing layer control or the operate services layer in the method for object, can also by concentrate in the position method of adding the service layer position control or the operate services layer in object.

Control method for the object in the service layer has two kinds, and a kind of is by client software, controls object in the service layer by mouse or keyboard or remote controller.As, control motion of objects in the service layer by defining up and down key in the keyboard, or indicate coordinate that object will arrive etc. with mouse; Another kind is to control object in the service layer by message set, and this method at first needs client to obtain message set, and then controls motion of objects in the service layer according to the position collection in the message set, operation set and function collection.As, the position collection is certain coordinate in the service layer, and the object in the corresponding service layer of this coordinate is operating as automatic operation, and function is for to be moved to the left 10 pixels to this object; Here can also be placed on mouse or keyboard operation in the operation set, promptly the position collection is the position of object in service layer, and operation set is a key up and down in left mouse button or the keyboard, and function is the position that moves to coordinate left button click location or keyboard motion.In Object Creation or deletion, equally can be with above two kinds of methods, during as new object of establishment in certain concrete service layer, the position collection is position or the concentrated position collection of information that mouse is chosen, be operating as automatic operation, function is play in service layer then for removing to extract some files at that URL or certain concrete document location.Object can also operation or the control of information centralized function collection by mouse or keyboard carry out some map functions, as becomes big, diminishes or other distortion etc.

Expansion service end and client cooperate the function of finishing to generally include following four aspects simultaneously:

The expansion service end sends data file to client:

Typical application has:

The expansion service end sends data file to client, and these information comprise video, image, flash, sound, literal, then in client terminal playing; The position of playing can be: the player of client, other supports the playout software of described media file the browser of client or client.When playing, can adopt and stop existing video image, intercut the media information of fetching from the expansion service end, perhaps under the situation that existing video does not stop, play the media information of fetching from the expansion service end.

Client sends data file to the expansion service end:

Typical application has:

Client is looked some media files such as audio frequency and is uploaded to the expansion service end.Function as the information set correspondence received in client is to open equipment such as local camera or recorder, these equipment in fact also are described to an address and a device id, will locally create the audio-video document that camera or recorder are recorded this time.Then these files are uploaded to the expansion service end.Last teletype command can be included in the function of information set correspondence, promptly sends information, also can manually upload.

Client sends message to the expansion service end

Typical application has:

The operating position of client need be added up or analyze to the expansion service end, need collect from the information of client.Function as the information set correspondence is in the client terminal playing advertisement, in order to add up the clicking rate of advertisement, click each time and all can send to the expansion service end to the information of client, the analysis of carrying out advertisement putting that so exactly can be real-time or non real-time so that future more accurate advertisement delivery.

The expansion service end is to client push message

Typical application has:

(1), the expansion service end is to client push message, and these information are preserved.Perhaps these information translation are become corresponding media object on the player of client or browser or software terminal, to play, method as online game, by finish control from the interacting message of expansion servers and client to client object, and the operation information of client sent to expansion servers, receive as client and control data about client object A A x position from video to be moved to y position in the video.Information is concentrated the position x that comprises A usually in this process, be included in the position and concentrate, and the control ID of A belongs to the attribute of x location object, and function then is that object A is moved to the y position from the x position.The content that function comprises is many, for example comprises mode, y positional information, run duration of motion etc.And for example, need in certain frame, create by certain coordinate position.

Though above some also need client and expansion service end just can finish alternately, mainly still lay particular emphasis on some aspects.Below some typical case utilizations all need client and expansion service end fully to cooperate just and can finish, comprising:

(1), add copyright authentication function and encryption function:

The existing popular DRM of copyright authentication system comprises: 1. authority explanation.Usually data that coexist with content, those set forth can by how/when/and wherein/by whom use/duplicate/store/distribute; Access with duplicate control.Be commonly referred to technology protection measure (TPM), utilize technological means to implement rights management, prevent that content from being obtained and duplicating by unauthorized user; 3. confirm and tracking.Technological means (digital watermarking or fingerprint recognition) is determined the source of content; 4. charge and payment subsystem.

DRM can protect content, if there is not suitable authority just can not use content.Authority provides by content permission, and it not only comprises the information of release protected content of being used for, and going back given content can be how/when/by whose use.The content permission that client needs can be brought in by expansion service and be issued.And DRM information can be placed in the interior service area of frame of the present invention, service frame or the service document, and perhaps the mode by message issues DRM information from service end.Wherein the basis of DRM and content protective system all is that cryptographic algorithm and agreement comprise: 1. symmetrical block encryption (AES, 3DES); 2. asymmetric public keys is encrypted (RSA, elliptic curve); 3. safe hash algorithm (SHA-1 ,-256); 4.Secret key exchange (DiffieHellman); 5. authentication and digital certificate (X.509).

Content-encrypt content, encryption method and key can be placed in the interior service area of frame of the present invention, service frame or the service document equally, perhaps transmit enciphered message by the mode of message.

(2), add new object and to the control of new object at the position collection: the new object that enters comprises: object video, animation, voice, picture and literal etc.On existing video playback layer, set up a new object layer, and give service and the outer service mode of frame in the frame of the present invention the control of this layer.With the picture is example, and the user adds a GIF picture in client in certain position of frame of video, and this position defines at the position collection that information is concentrated.If the GIF image is moved to the B position from the A position, the original position, attribute, motion mode, destination etc. that then concentrate to add GIF in information, and this control is two-way, can service end send to client, also can send to service end from client.Certainly in the present invention, when above-mentioned client sent to service end to information set, in fact client had played the effect of service end, and service end is equivalent to the position of client, therefore can change mutually conceptive.Technology at new video layer can realize with the technology based on DirectX of existing DirectShow, and perhaps the technology with two display chips of Intel realizes.In the service layer of service end on control client video layer, the location object that information transmitted is concentrated is above-mentioned GIF object, carries information such as above-mentioned original position, attribute, motion mode, destination in the attribute.It should be noted that, expansion realization technology on service layer and the video coding figure place is different, service layer is on traditional video playback layer, need the software and hardware support of client, service layer is an abstract concept, makes service end or client can insert new object video easily in video.The mode that new object adds has two kinds: a kind of is to add object video in service end, and transmission can be adopted identical traffic channel or different channels with video; Second kind is to leave function concentratedly in information to concentrate definite GIF in the position of client, inserts the GIF object in client in the service layer by being implemented in of function of information centralized function collection then; The third is to add the GIF object in service layer voluntarily the client user, and client at this moment and service end are same equipment or hardware environment.

(3), fetch the URL of a website, and play the service of this URL from expansion servers:

As in information set, adding the URL of a website, in the time of the client terminal playing video, extract concentrated position collection, operation set and the function collection of information of the inside.The position collection can be the position of some concrete frames in this example, and the respective operations collection is automatic extraction, and corresponding function collection is to open URL specified web information.Fetch the content of this URL address then from the website,, play then as webpage or picture of a WWW.

Some simple functions client under the situation that does not need independent expansion service end also can be finished:

Typical application has:

Turn function, carry out redirect by the position collection that in information set, is provided with, when the position collection does not all then need to get the meeting data in expansion servers in video, if jump location in expansion servers or in certain media file of expansion servers, then will arrive and reach back data in the expansion servers.As, in video some regional locations and forward turn function be associated, when this position is clicked, will jumps to appointed positions automatically and play the content of institute's jump location, so just can realize the time-shifting function of appointment, as jump to the video frequency program before 5 minutes.

Writing function, this function can be included in the copyright information, manages with DRM, and the position set pair in information set is answered the frame sequence group, and user property is for downloading in the attribute, and the function collection is for downloading, and operation set is for clicking.If at this moment the client user has clicked the assigned address that concentrate the position, then can be in the displaying video programs of one side foradownloaded video on one side.So just finished the writing function of video.

Priority feature, as the position collection in the first frame of video corresponding informance collection is the zone of an appointment, priority is for the highest, if have the position collection of the information set of the second frame of video correspondence this moment is same appointed area, and this two frame is play in same window simultaneously, and this regional priority of the second frame of video correspondence is lower, will only play this described zone in the first the highest frame of priority this time.Come other zone in the processed frame with same principle, so just can realize that the merging of multi-path video stream is play.

Transparent functional can be handled the problem that multi-channel video merges equally.In the time of need playing in same window if any two frames, can judge which frame last according to priority earlier, that frame is down, and then determines transparency according to the transparency attribute, transparency normally from 0 to 100.

The present invention also provides a kind of method that adds service frame in video flowing, may further comprise the steps:

Service end is newly-built service frame in video resource; Service frame is created when creating the frame of video file or is created service frame again behind the generation frame of video file earlier; Service frame is transmitted or is transmitted in different transmission channels respectively a transmission channel the inside with frame of video; Service frame is resolved with same syntactic structure with frame of video or is resolved with different syntactic structures; Service frame and frame of video are kept in the identical file or are kept at respectively in the different files; The method transmission that service frame can adopt the method for compression or not compress.Service frame has basic frame structure, packaging information collection in the frame structure; This service frame loaded information collection comprises: the pairing function collection of operation set that position collection and position set pair are answered and position collection and operation set; This position set pair resembles in the attribute and also comprises: each regional priority, the positional information of frame inner region and the movable information of frame inner region in the priority of each corresponding frame of video, the frame.

In this service frame, add the information set content.

Service end is used service frame beared information collection, sends to client; Wherein, the corresponding continuous or discrete one or more frame of video of each service frame.

The present invention also provides a kind of method that adds the frame sequence group in video resource, may further comprise the steps:

The a plurality of adjacent or non-conterminous frame that has logical relation in service end artificial selection, and these frames as an orderly set, i.e. frame sequence group.

The element that concentrate as the position position that the frame sequence group is begun and/or finishes.

Wherein, the frame sequence group is corresponding with continuous in logic video segment, and the attribute of frame sequence group location object comprises: the source of precedence information, enciphered message, copyright information, customer information, the operation set of being supported, information and/or target information, position collection joining day and/or effective time; Enciphered message is used for the position set pair is answered the encryption of object in the object properties, comprises cipher mode, key information; Copyright information is used for the position set pair is answered the copyright notice and the protection of object in the object properties, comprises the attaching information of copyright, the authentication information of copyright, the use information of copyright; Customer information is used for the position set pair is answered the client rights explanation of object and used client segmentation information in the object properties, the client rights explanation comprises (this part DRM that can be included in copyright information manages): download authority, play authority, use client segmentation information to comprise: to the classification control of content.

Concentrating in the position among the present invention to run into problem how to distinguish the zones of different object, as shown in figure 28, is an effective solution.Existing frame of video generally all is a three-dimensional structure, comprises brightness and colourity in this three-dimensional, as YUV.Same RGB also is a three-dimensional structure.The present invention is used to distinguish different zones at the increase one dimension on existing three-dimensional structure, the method for expressing of this one dimension is described in detail in Figure 13 to 17.The increase of this one dimension can well be expressed the profile in the position and the zone in zone.Parameters such as priority or transparency can also be set in this one dimension.The bearing mode of this one dimension can use the bearing mode of coverage in the frame among the present invention.Coded system can be identical with existing compression method with compression method, also can be different.

Can also introduce new object video in this one dimension, for example, the bianry image of a black and white if the bianry image of every frame is connected, just can be formed on the bianry image animation on the video playback layer.Same method can develop out the animation of the colour on existing YUV video.If on the YUV three-dimensional, continue stack three-dimensional or multidimensional, the stack of video on video in can realizing transmitting.And video location can be realized by priority approach up and down, and priority is higher is placed on the upper strata, blocks the low video location of priority.And can control descending the visible level of layer video with the transparency of upper video.Above method can occur in a sign indicating number frame in coding equally, adopts existing compression method or encoding scheme.When coding, newly-increased dimension data can adopt the method identical with existing encoding scheme, it is the method (coding/decoding method is then opposite: anti-entropy coding, inverse quantization, IDCT, motion compensation) of motion prediction, DCT, quantification, entropy coding, also can adopt method for distinguishing, perhaps not adopt compress technique.

The present invention also provides a kind of method that adds section object and object properties thereof in video resource, may further comprise the steps:

Service end is the zoning in video resource, and the area dividing mode comprises: with object zoning or free zoning; The object zoning comprises: by manually indicating subject area, and automatic again tracking object positions, and identify contours of objects information; Or, again by the method for interpolation, simulate the object motion track, and identify contours of objects information by manually indicating subject area respectively at several frames of being separated by.

Service end as object, and for each object is provided with corresponding attribute information, and is provided with the corresponding informance collection according to the zone.

Client is carried out the union operation of diverse location according to priority: when the frame of different priorities is play in same client simultaneously, the zone of only playing the highest frame of priority or different priorities when the demonstration of same frame, the zone that display priority is the highest.

Client obtains the information set of Streaming Media and Streaming Media correspondence;

The expansion service end is collected from the customer information of client and medium relevant content information;

Customer information comprises: client's the network address, Customer ID, client properties.

Service end obtains to need to add the frame of video of information set;

Chosen position adds information set in frame;

Chosen position is included in the head of frame of video or at the afterbody of frame of video.

Regional location is divided into the square of identical size, and square according to pixels calculates and comprises: 1 * 1,2 * 2,4 * 4,8 * 8,16 * 16,32 * 32; And each straight line passed foursquare situation with a number mark;

Foursquare when being passed by the regional location profile, mark penetrates and passes foursquare 2 points, connects 2 parts that are used as the regional location profile with straight line then;

When the regional location profile all when passing foursquare straightway sign, pass foursquare situation according to straight line and find out of the most approaching existing number mark, pass foursquare situation number and come mark according to predefined again.

The described technology of the embodiment of the invention can use hardware, software or combination to carry out.If carry out with software, then this technology can directly refer to comprise the computer-readable medium of program code, and this program code is carried out in the equipment that video sequence is encoded.Under this kind situation, computer-readable medium can comprise RAM (Random Access Memory, random asccess memory), SDRAM (SynchronousDynamic RAM, synchronous DRAM), ROM (Read Only Memory, read-only memory), NVRAM (non-volatile RAM non-volatile RAM), EEPROM (Electrically-Erasable Programmable Read-Only Memory, electricallyerasable ROM (EEROM)), FLASH (flash memory) etc.

Program coding can be stored in the memory with the form of computer-readable instruction.In this case, one or more processors can be carried out the instruction that is stored in the memory, thereby carry out one or more residual coding technology.In some cases, processor can use DSP (Digital Signal Processing, Digital Signal Processing) equipment to carry out, and DSP uses various hardware elements to come speech coding to handle; In other cases, encoding device can be used as one or more microprocessors, one or more or a plurality of ASIC (application-specific integrated circuit, application-specific integrated circuit (ASIC)), integrated or discrete logic circuitry of other equivalence of FPGA (FieldProgrammable Gate Array, field programmable gate array) or some or combination hardware-software are carried out.

More than disclosed only be several specific embodiment of the present invention, still, the present invention is not limited thereto, any those skilled in the art can think variation all should fall into protection scope of the present invention.

Claims

1. a method that adds service frame in video resource is characterized in that, may further comprise the steps:

Service end is newly-built service frame in video resource;

In described service frame, add the information set content;

2. in video resource, add the method for service frame according to claim 1, it is characterized in that described service frame has basic frame structure, packaging information collection in the described frame structure;

3. add the method for service frame according to claim 1 in video flowing, it is characterized in that, described service frame is created when creating the frame of video file or is created service frame again behind the generation frame of video file earlier;

4. a method that adds the frame sequence group in video resource is characterized in that, may further comprise the steps:

5. as in video resource, adding the method for frame sequence group as described in the claim 4, it is characterized in that described frame sequence group is corresponding with continuous in logic video segment, and the attribute of frame sequence group location object comprises: