RU2212709C1 - Method for processing object-oriented interactive video data - Google Patents

Abstract

FIELD: computer engineering; active video computer systems. SUBSTANCE: method includes stages of script file formation and video image reproduction. In order of shape card for all active objects of video frame intraframe coding is effected by separating series of active-object locations. Intraframe coding is made in next video frame when geometry or position of active object changes. Calculations are made to reduce data level. Active object properties are shaped and stored by establishing links with aid of at least one hyperlink for each active object. During this reproduction stage script file cards are decoded. EFFECT: enhanced productivity in processing great number of visual objects. 10 cl, 26 dwg, 1 tbl

Description

 The invention relates to computer technology and can be used in computer systems for active video (AV) to create various scenes and view digital video files, as well as in areas such as interactive television, video on demand, broadcasting TV over the Internet, interactive training systems, video conductors, personal, mobile communications, computer and television games.

 In traditional systems for viewing digital video (VCRs, CDs, DVD players, computer applications for playing videos; televisions), it is possible to control according to the level of the means (device) for playing, for example, “play”, “stop”, “pause”, “ fast forward, rewind. Watching a video does not provide support for many scenarios, i.e., the user does not have the ability to change one viewing scenario to another while playing a video. In active video (AV) systems, the user can immediately access the objects that he sees on the screen during video playback and control video playback using clicks of the manipulator (mouse) directly from video images of active objects.

 There is a method and apparatus for combining hyperlinks with video, in which at the stage of creating the script file, a video frame is displayed on the monitor screen with an image of at least one visual object, which is subsequently used to activate it during playback, the active object (AO), approximated using marking the active object with a simple geometric figure, for example, a circle, an oval, a rectangle, using the author’s toolkit, mark up individual key frames, form and remember their The active objects are produced by establishing links using one hyperlink for each active object, as a result of which a script file is obtained, and the markup areas (circle, oval, rectangle) are linearly interpolated to the remaining intermediate frames (PCT International Application W098 / 44435, G 06 F 17/30, publ. 1998).

 In this technical solution, communication is established using a hyperlink and compact marking data is stored, but markup accuracy is sacrificed. The author of the script file cannot access the markup process on non-key frames. In the process of reproducing the resulting file, the markup regions on key frames are linearly interpolated to intermediate frames. Thus, during the playback process, due to linear interpolation of the markup regions on non-key intermediate frames and due to the approximation of the active object by a simple geometric figure, there may be cases when the markup of the object does not fall when it is clicked by the manipulator, since the user focuses only on the script file playback stage visual information, and the marking lines themselves are transparent and not visible to the viewer.

 The well-known "Parallel computer system of Active Video" that implements the method of object-oriented interactive processing of video information, which is closest to the claimed one (Patent of the Russian Federation for the invention 2173883, G 06 T 1/20, publ. 2001).

 The method includes the step of creating a video file script file and the step of reproducing moving video images.

 In this method, at the stage of creating the script file, a video frame is displayed on the monitor screen with an image of at least one visual object, which is subsequently used to activate it at the playback stage, the active object (AO). The video frame is divided into XxY cells, where X and Y are 1, 2, 3 ..., having the form of elementary squares or rectangles. Cells are selected whose location coincides and does not coincide with the location of the AO on the video frame, assigning the numbers corresponding to the cells of the cells of the AO, and assigning zeros to the cells whose location does not coincide with the location of the AO cells. Intraframe coding is performed, forming an array of map data for all active objects of a given video frame. Form and remember the properties of AO, form and remember the transition corresponding to this AO. If the position of active objects in the following video frames of the video file is constant, then for such video frames the map of the previous video frame belongs to them, and if the position of the AO in the next video frame changes, a new map is formed for these AO. A script file is formed and stored in a sequence of cards in which each card corresponds to at least one video frame of the video file.

 At the stage of video playback, a video file, a script file and current position information of the manipulator cursor on the monitor screen are used. When the manipulator cursor moves on the monitor screen and the cursor location coincides with the AO - with those cells whose location coincides on the video frame with the location of the active object, the cursor will be transformed. If the cursor coincides with the location of the AO and when you press one of the keys on the manipulator, a transition corresponding to this AO is stored and stored at the stage of creating the script file. When playing a video frame of a video file with another AO on the same video frame, when the cursor coincides with the location of another AO and pressing the same key on the manipulator, a transition corresponding to another AO is performed.

 In this technical solution, at the stage of creating the script file, when generating and storing the description for each AO, the number of the video file and the script file, the number of the video frame in the video file and the script file, identifier, when generating and storing the playback control parameters for each active object, remember the activity indicator, number video file, the numbers of the first and last frames in the reproduced sequence of video frames from this video file, the number of n looping plays When creating and storing the address of the transition from one AO to another AO, the characteristics of the activity of another active object, the number of the video file of another object, the number of the video frame in this video file, and the identifier of another object are stored in the video frames from the first frame to the last.

 At the playback stage in this technical solution, when the left button of the manipulator is pressed for a video frame of a video file with this active object, the system switches to the first video frame of the video file corresponding to this AO, i.e., it starts viewing the video file of this AO starting from its initial frame.

The limitations of this technical solution are:
The need to use cells with zero that do not coincide with the locations of active objects during intraframe coding, which increases the data array for creating maps.

 The lack of inter-frame coding, which leads to redundancy of the stored data.

 When identifying and selecting cells whose location coincides with the location of the active object, it is necessary to encode all AO cells in X, Y coordinates to form a map of the active object, which also dramatically increases the data array.

 The need to save a large amount of data at the stage of creating the script file, in particular, it is necessary to generate and remember a description for each active object — the number of the video file and the script file, the number of the video frame in the video file and the script file, in addition, the number n of cyclic reproductions of the sequence of video frames from the first frame to the last. The formation and storage of a large amount of this data practically does not affect the expansion of the viewing capabilities at the playback stage. So, for example, at the playback stage, clicking on the AO of the video file being played for it causes only its viewing from the very beginning. In this well-known technical solution, when playing back a video frame of a video file directly related to this AO as the main one, it is impossible to switch to playback of various other video files or to communicate with other digital data sources. Thus, the ability to create completely different multifunctional scenarios is limited.

 The problem solved by the invention is to expand the functionality and increase the processing productivity of a large number of visual objects.

 The technical result that can be obtained by implementing the claimed method of interactive processing is to reduce the amount of memory required for storing and processing AO markup; ensuring the formation of small-sized script files separate from video files that can be decoded without delay and synchronously with the video display; providing the possibility of making completely different transitions from one active object to other completely different sources of digital data, in particular, for viewing completely different scenes based on different video files.

To solve the problem with achieving a technical result in the known method of object-oriented interactive processing of video information, which includes the step of creating a script file and the step of playing back video images, at the stage of creating the script file, a video frame with an image of at least one visual object intended subsequently, to activate it at the playback stage, the active object, the video frame is divided into X • Y cells, where X and Y are 1, 2, 3.. ., having the form of elementary squares or rectangles, select cells whose location coincides and does not coincide on the video frame with the location of the active object, assigning to the cells matching the location of the cells of the active objects corresponding numbers to them, and assigning zeros to cells whose location does not coincide with the location cells of active objects, perform intraframe coding, form an array of map data for all active objects of a given video frame, form and store properties of active new objects, form and remember the transition corresponding to this active object, and if the position of the active objects in the next video frames of the video file is constant, then for such video frames the map of the previous video frame belongs to them, and if the position of the active objects in the next video frame changes, they form a new map for these active objects, form and store a script file formed by a sequence of cards in which each card corresponds to at least one video frame of the video file, while at the stage of video playback, a video file, a script file and current information about the position of the manipulator cursor on the monitor screen are used, when the manipulator cursor moves on the monitor screen and when the cursor location coincides with the active object, those cells whose location coincides on the video frame with the location of the active object, view the cursor is converted, when the cursor matches the location of the active object and when you press one of the keys on the manipulator, a transition corresponding to active object and memorized at the stage of creating the script file, and when playing a video frame of a video file with another active object on the same video frame, when the cursor coincides with the location of another active object and when you press the same manipulator key, the transition corresponding to another active object according to the invention for the formation of the map data array of all active objects of the video frame, intraframe coding is performed by selecting the extreme row along one of the X or Y coordinates, of cells of the active object, determining the coordinates of the extreme cells of this row, determining the coordinates of the two extreme cells in subsequent rows, while if the extreme and subsequent rows contain cells whose location does not coincide with the location of the active object in these rows, then to the map data array they introduce a modifier for breaking a series of cells and remember the coordinates of the extreme cells of the active object in the place of breaking a series, analyze each subsequent series for the absence of active objects in the series of cells, and if the series is free from cells of active objects, then a row skip modifier is introduced, each subsequent row is analyzed for the identity of the previous and subsequent rows adjacent to it, if they are identical, then a row repeat modifier is entered, after intraframe coding, the video frame map is stored, if the geometric shape and / or the position of the active object in the next video frame produces its inter-frame coding, in which, if the position of the cells of the active object in the next video frame changes, and its geometric shape is saved If it is deleted, then the parameters of the cell displacement vector of the active object are introduced into the data array of the map of the next video frame, if the position of the cells of the active object and its geometric shape are changed, then the above-mentioned encoding of this video frame is performed, obtaining the data array A, then the parameters of the cell displacement vector are entered into the data array the active object and calculate the difference between the two matrices of data arrays for the active object on the previous and next video frame, obtaining a difference matrix obtained by the difference the matrix is encoded by the mentioned intra-frame coding, obtaining a data array B, the size in bytes of the data array A is compared with the data array B according to the formula
W = {[P (A) -P (B)] / P (A)} • 100%,
where W is the calculated gain,%,
P (A) - the size of the array A,
P (B) - the size of the array B,
and if the value W> K, where K is the value of the given threshold, then save array B as the data array of the next map, and when determining the location of the active object of this video frame, use the data array A of the previous map, and if W <K, then save the data array And as an array of the next map, the formation and storing of the properties of active objects is carried out by establishing links using at least one hyperlink for each active object, and a connection is established with various sources of digital data, When forming and remembering the transition corresponding to this active object, a sequence of showing other specified video frames for this active object is formed, at the playback stage, the script file cards are decoded, when the position of the manipulator cursor on the monitor screen coincides with the active object, and when one of the manipulator keys is pressed, following the first available hyperlink generated and remembered at the stage of creating the script file, if the cursor position matches Yator on the screen with the active object and pressing the other keys of the manipulator selects from a menu of several hyperlinks and then pressing the other key arm on one of the hyperlinks of the menu make the transition in accordance with the connection of the hyperlinks to the appropriate source of digital data.

There are additional options for implementing the method, in which it is advisable that:
- at the stage of creating the script file, the cells of the active object were selected using the manual marking tools;
- at the stage of creating the script file, the cells of the active object were automatically selected after specifying the characteristic points on the contour of the active object and identifying the cells inside this circuit, and the next video frame automatically tracked the change in the location of the characteristic points and the position of the cells of the active object;
- at the playback stage, a video file was used as a source of digital data;
- at the playback stage, a playback program was used as a source of digital data, providing a specified order of playback of frames of video files;
- at the playback stage, text documents, HTML documents, graphics, a computer program were used as a source of digital data;
- at the playback stage, clicks on hyperlinks of active objects were remembered to return to the plots from which the transitions were made;
- at the stage of reproduction, active objects were marked;
- at the playback stage, when the location of the cursor of the manipulator coincides with the location of the active object, a tooltip is generated;
- decoding was performed by the reverse operation of encoding.

 Due to the original implementation of the encoding of coordinates of active objects, the introduction of the inter-frame encoding operation, in which data arrays of the previous frame can be used, as well as due to the implementation of various transitions using multiple hyperlinks belonging to one AO, it was possible to solve the problem.

These advantages, as well as features of the present invention are illustrated by the best options for its implementation with reference to the accompanying drawings:
FIG. 1 depicts an array of data with two active objects after identifying cells for coding;
figure 2 - rows of cells with a line break, with a skip line, with repeating lines and line segments, described without the help of modifiers;
FIG. 3 - a) previous and b) subsequent frames with AO offset while maintaining its geometric shape;
figure 4 - an array of data from AO after identification of cells for filing for encoding, where a) - K-th frame with the active object; b) - K + 1 frame, in which AO changed the geometric shape; c) - the difference matrix obtained by subtracting the data arrays K + 1 frame and the K-th frame;
5 is a block diagram of an intraframe coding algorithm for the claimed method;
6 is a block diagram of an algorithm taking into account inter-frame coding;
7 is a block diagram of an editing step;
Fig. 8 is a flowchart of a method for automated marking of AO;
Fig.9 is a block diagram of a playback step;
figure 10 is a block diagram of a decoding card AO;
11 is a block diagram of a decoding key card;
12 is an embodiment of an interface for editing;
Fig - means of navigation through the frames at the editing stage;
Fig - means of identification of cells at the editing stage;
Fig - a) controls AO, b), c) controls hyperlinks to active objects, d) controls the availability of links to specific video frames;
Fig.16 - management tools for playback programs;
Fig - many transitions made from AO through a variety of hyperlinks of various types at the stage of playback;
FIG. 18 - a lot of hyperlinks AO, accessible from the menu when you right-click on the playback stage;
FIG. 19 is a tooltip when the cursor is over the AO and AO marker.

 In the claimed method of object-oriented interactive processing of video information, a video frame is displayed on the monitor screen with an image of at least one visual object, which is subsequently used to activate it during playback, the active object (AO) is divided into X • Y cells (Fig. 1), thereby discretizing the visual object. When discretizing the markup of visual objects on video frames, the author marks the areas covering the image of the object on the video frame that will relate to the AO predefined by him (for example, regions of the character of the film, car, plane, previously defined in the AV Editor). AO marking is defined in grid coordinates. The minimum markup unit is one grid cell. The smaller the cell, the more accurate marking of the AO can be made.

 Identification of cells whose location coincides and does not coincide on the video frame with the location of AO can be carried out by various means of manual marking or automatically, which will be later shown on examples of a specific implementation of the method. Assign AO cells for different AOs the numbers corresponding to them (for example, from 1 to 65000), which do not coincide for different AOs, thereby identifying the AOs themselves. Assign zeros to cells whose location does not coincide with the location of AO (figure 1).

 With intraframe coding, AO regions are encoded on this video frame. Intra-frame coding makes it possible to form an array of AO map data — the corresponding matrix in the X, Y coordinate system for cells whose location coincides on the video frame with the location of the AO. Map - encoded information about AO markup.

 The intraframe coding operation in the claimed technical solution differs from the intraframe coding used in similar technical solutions that use an array of video image data for the entire screen area (the data array matrix in the same way takes into account zero cells and coordinates of each of the AO cells). In the claimed method, the operation of intraframe coding eliminates the redundancy of data arrays that describe the marking areas of AO in the frame.

 In the claimed method for forming a map for all AOs of a given video frame, intraframe coding is performed by selecting rows (Fig. 2) consisting of AO cells located in only one direction X horizontally or only Y vertically. Intra-frame coding is performed sequentially by adding step 1 to one of the coordinates of the extreme row of the active object, for example, to the Y coordinate. Other coordinates, for example X, are determined for the extreme cells for each row of AOs. Modifiers are introduced into the map data array - break indicators in a row of cells, a row skip for cells, a row repeat for cells. If any of the rows contains a gap, then when storing an array of data, this series is described using the modifier of the gap of a number of cells, while the X coordinates of the cells of the active object in the place of the row break are additionally stored. If the next row in order is free from the cells of the active object, then when storing the data array, a skip modifier is introduced for this row. If any of the following adjacent rows has the X coordinates of the extreme cells in the row, the same as the previous row, then enter for this row a pointer to repeat the row of cells. After encoding, the video frame card is stored - an array of the received data.

 The intraframe coding process is illustrated using the following example.

 Let the intraframe coding process be conducted along horizontal rows of cells in which cells with the same Y coordinates belong to the same group and are located along horizontal lines. Groups of cells (figure 2) are located on line segments. When coding, it is enough to remember the data on the left and right X coordinates of each horizontal row. To reduce the amount of map data needed to save and fully describe AO, three types of modifiers are introduced: a row break modifier for a cell - line segment, a row skip modifier for cells - a line skip for cells, a row repeat modifier for cells - a line replay for cells.

 Let the Y coordinate of the upper left point of AO be initially remembered if coding is done horizontally. Then, the initial and final coordinates of X segments are determined. The segments follow one after the other, that is, the Y coordinate of the next saved segment is equal to the Y coordinate of the previous segment plus 1, therefore only one Y coordinate of the extreme point AO and the starting and ending X coordinates of the segments are saved.

In the case of line breaks, skipping lines (figure 2), the modifiers necessary to determine only the Y coordinates of the segments are introduced. The line segments are the same and identical for the rows of cells, i.e. if they have equal X coordinates of the starting and ending points, respectively. The break modifier of a number of cells indicates that the line segments with groups of cells before and after it are located on the same line and have a gap, i.e. they have the same Y coordinate and there is no group of cells in a row. The coordinates of the outermost cells at the row break point are remembered. The row skip modifier indicates that one of the horizontal rows of cells is absent and the Y coordinate of the next existing row of AOs, i.e., not an empty line, is remembered. The row repeat modifier indicates that the line with the cells is identical to the adjacent previous one (contains the same number of cells as the previous one, and the X coordinates for the extreme cells of the line are the same) and is repeated a number of times. If there is no modifier in the data matrix for a horizontal row of cells, then the line segments of the rows do not have repeats, gaps, or gaps. Such cells belong to different, but sequentially going from top to bottom lines: Y _{i + 1} = Y _i +1, where i is the number of the horizontal row of cells (i = 1 ... k).

 In FIG. 2 bold lines show AO, the matrix of which is encoded by the following sequence shown in table. 1a.

The scheme of intraframe coding of cards in a script file:
<object map i> :: = <begin point><countstretches><stretches>
<begin point>
The starting point of the markup area of the object, Y coordinate of the upper left point of the line segment of the upper horizontal row of cells.

<count stretches>
List of markup elements. Consists of the coordinates of line segments and modifiers.

<stretch i> :: = <start X><endX>
<start X>
X coordinate of the starting point of the line segment.

<end X>
X coordinate of the end point of the line segment.

<modifier> :: = (<line continue> | <pass lines> | <repeat stretch>)
<line continue>: = <line continue mod><stretchi>
The line break modifier describes the break of the current marking line or, in other words, the duration of marking in the current line. If the current line consists of more than one line, the modifier indicates that the next line belongs to the same line.

<line continue mod>
Line break modifier, byte modifier identifier.

<stretch i>
The next segment is in the same line.

<pass lines> :: = <pass lines mod><Ynew>
The line skip modifier describes lines that do not contain object markings. Defines the Y coordinate of the next line in which the markup occurs.

<pass lines>
Line skip modifier, byte modifier identifier.

<Y new>
The y coordinate of the line where the markup appears.

<repeat stretch>: = <stretches on line><repeat stretch mod><countline>
The line repeat modifier is used to encode repeating line segments or repeating sequences of line segments.

<stretches on line>
A list of segments representing repeating sequences
<repeat stretch mod>
Retry modifier, byte modifier identifier.

<count line>
The number of repeating lines.

 When changing the geometric shape or position of the active object in the next video frame, interframe coding is performed to reduce the array of map data. In this case, inter-frame coding is performed for each subsequent video frame, with the exception of those video frames in which the location of the AO and its shape does not change. If the position and shape of the active objects in the following video frames of the video file is constant, then for such video frames the map of the previous video frame belongs to them, therefore, it is not necessary to memorize the data array of maps of such video frames. The operation of inter-frame coding eliminates the redundancy of data arrays that describe the regions of the markup of objects on adjacent frames. The process of coding AO areas is based on data from the current and previous video frames.

 At the first stage, the array of AO coordinates related to the previous video frame is compared with the array of coordinates of the active objects of the current frame. If the matrices - data arrays are the same, then the AO markup map related to the previous video frame also applies to the current video frame, i.e. for such video frames, the map of the previous video frame belongs to them.

 If the matrices do not match, then analyze the data array for each AO (if there are several).

The following options are possible:
1. The coordinates of the AO are the same.

 2. The shape of the AO coincides, but the AO is biased in two-dimensional space (Fig. 3).

 3. The shape of the AO is changed in relation to the shape of the AO of the previous video frame, but the object itself is not biased (Fig. 4).

 4. The shape of the AO is changed with respect to the shape of the AO of the previous frame and the object is displaced (the joint action of FIGS. 3, 4).

 If the coordinates of the active objects on the previous and next frame are equal, then data on the displacement vector equal to (0, 0) are entered into the AO map. Such a map can be decoded using information about the previous map of the active object. For the convenience of further explaining the operation of inter-frame coding, the data array of the card, which is used to encode the next card and allows you to reduce the amount of data for the card of the next frame, is the key, and such a previous card is the key. The map of the next frame, with which information cannot be read without using the key, is relative.

 If the shape of the active objects coincides, but the active objects are displaced in two-dimensional space, then this displacement is taken into account using the <displacement vector> (Fig. 3). Only the coordinates of the displacement vector are entered into the data array of the AO map - two numbers, the displacement along the X axis and the displacement along the Y axis. The map with the displacement data is relative, and the map with the initial position of the AO is the key. So, in Fig. 3, the displacement vector for the AO (K + 1) -th frame relative to the K-th frame is characterized by the AO displacement by 5 cells in X and by 1 cell in Y. Therefore, in the map data array (K + 1) -th Only two numbers (5, 1) are introduced into the frame, which characterize the coordinates of the displacement vector. The card of the K-th frame is the key one, and the card of the (K + 1) -th frame is relative, and its full data array can be obtained by subsequent decoding only using the key card.

If the AO shape of the subsequent frame (Fig. 4B) is changed with respect to the AO shape of the previous frame (Fig. 4A), but the object itself is not offset, then interframe coding is performed by two methods. The intraframe coding described above encode the data array of the (K + 1) th frame. The result is a resultant data array A. Then, the second method determines the difference between the coordinate data of the AO matrix of the previous Kth frame and the coordinate data of the AO matrix of the subsequent (K + 1) th frame with the changed shape of the object (Fig. 4B). As can be seen from figure 4, for AO (K + 1) -th frame differs from the K-th frame by the location of only four cells, and the difference matrix will be characterized only by the X and Y coordinates of these cells. Next, the resulting difference matrix is encoded using the intraframe coding method described above. The result is a resulting data array B. Compare the size in bytes of the data array A with the data array B by the formula
W = {[P (A) -P (B)] / P (A)} • 100%,
where W is the calculated gain,%,
P (A) - the size of the array A,
P (B) - the size of the array B,
and if the value W> K, where K is the value of the given threshold, then the array B and the displacement vector equal to (0,0) as the description of the next map are stored, and if W <K, then the array A is saved as the description of the next map of the active object. If array B is stored, then such a map is relative, and if array A is key.

 The value of the specified threshold K can be selected in the range of 30-99%. The value of K is determined by the author of the script file. The threshold of 30% gives a large number of relative cards of AO, and at 99% almost all cards are key. This affects the amount of information stored. When using only key cards in practice, the amount of data stored is greater, but the access speed is faster and, accordingly, decoding is faster. When using a large number of relative cards in a script file, performance decreases, but the amount of required memory is reduced - the percentage of data storage. Therefore, it is advisable to set the specific threshold value depending on the technical characteristics of the digital equipment used. It has been experimentally established that for most applications, a threshold value of 40-50% can be set.

 If the AO is shifted and the shape of the AO is changed in relation to its shape on the previous video frame (i.e., there was a joint situation of changing the AO map, as was shown separately in Figs. 3 and 4), then the next frame is encoded intraframe obtaining the data array A. Then, the displacement vector is determined (as previously described for FIG. 3). In this case, the vector is taken as the direction and magnitude of the displacement vector, which, when displaced by the value of this vector, AO from the previous frame gives the maximum coincidence of the cells with this AO of the next frame. That is, all matching cells of the previous and next frame are determined and the parameters X and Y of the displacement vector are found for these cells, as described previously. Next, the difference between the coordinate data of the cells of the matrix AO of the previous frame and the coordinate data of the cells of the matrix AO of the next frame with the changed shape of the object is determined (as was previously described for figure 4). That is, those cells that do not match on the previous and next video frames are detected. Next, the resulting difference matrix is encoded using the intraframe coding method described above. As a result, the resulting data array B is obtained. If W> K, where K is the value of the given threshold, then the encoded difference array B and the parameters of the displacement vector are stored, i.e. get a relative map. If W <K, then store array A, i.e. key card as described above. The value of the specified threshold K can be selected in the range of 30-99%.

 The process is applied to each active object in this video frame.

 When encoding a subsequent frame, the previous map of the active object is checked. Determines whether it is relative or key. If the previous map is relative, i.e. for it W> K, then the map of the active object on the subsequent frame can also be only relative to the previously obtained key map. That is, its data is read using a key card, bypassing the data array of previous relative cards. But if W <K for her, then she will be the key for the subsequent cards of the active object. Thus, a sequence is obtained - a “chain” of AO cards, where there can be several relative AO cards following the key one.

 For example, a sequence might look like this: K R R R R K R R, where K is the Key Card AO and R is the Relative Card AO. The process of forming maps is carried out for all frames of video that have markup. As will be shown below, such a description in matrix form of cards requires minimal memory and computational resources for their decoding.

 In the claimed technical solution, the encoding process includes the described intraframe and interframe coding (FIGS. 5, 6).

 At the input of the encoder in block 1 (Figs. 5, 6), an array of AO coordinates of the current frame is received, which is a matrix with a size equal to the number of cells in the markup. In the cells covering AO, numbers of active objects are stored, in all other cells zeros. For simplicity, horizontal and left-to-right coding are considered.

 If the array of coordinate data AO is the first (comparison block 2, Fig.6), then we encode it in block 3 by intraframe coding. The content of the functional elements of block 3 (Fig.6) is shown in Fig.5. At the first stage, determine the upper left ordinate Y0 AO - block 4 (figure 5). After determining it, the ordinate is written in the initial value of the counter in rows - block 5. Next, in block 6, the abscissas of the beginning and end of the first segment of the horizontal line for the cells are determined and stored in the resulting array in block 8. Then, in the comparison block 9, it is checked whether a line segment with the same ordinate: if there is, then we use the modifier to break a number of cells along this horizontal line in block 10 and determine the coordinates of the beginning and end of the next line segment with cells horizontally in block 6. Parse operation the horizontal row of cells along the horizontal line occurs until the last segment of the studied horizontal line is found, this condition is checked in comparison block 7. If there is no such segment, then in block 11 the transition to the next adjacent row of cells - to the next horizontal line ( increment is 1). If there are no cells - AO segments on the next horizontal line, this check is carried out in comparison block 12, a row skip modifier is indicated in block 13. If a line with a number of cells is found in comparison block 14, consisting of a segment or segments with abscissas and modifiers, corresponding to the previous line, in block 15, the row repeat modifier is indicated. The process continues, starting from block 6, until all segments with a series of cells belonging to AO are described, this condition is checked in block 16. The process, starting from block 6, is repeated for each AO matrix of this frame, this condition is checked in block 17. The resulting array is stored in block 18 (Fig. 5). The resulting array from block 3 (Fig. 6) is stored in the buffer 19. If there are no more active objects, then the matrix of the next frame arrives at the input of the encoder block 1 (Fig. 6). This matrix is compared with the matrix of the previous frame in block 20 (Fig.6). If the matrices are identical, then in block 21 we assign the map of the previous frame to the current one. If the matrices are different, then in block 22 the coordinates of one AO are selected from the matrix and two calculations are performed. According to the first calculation in block 23, the coordinates of the first AO are encoded using the intraframe coding method. According to the second calculation, the displacement vector is determined in block 24; for this calculation, the AO coordinate matrix of the previous frame is extracted from block 25. In block 26, the difference between the AO matrices is determined. Block 27 encodes the difference array B obtained after calculations in block 26 by intraframe coding (Fig. 5). In block 28 (Fig.6) compares the gain W with a given threshold K.

W = {[P (A) -P (B)] / P (A)} • 100%,
where W is the calculated gain,%,
P (A) - the size of the array A,
P (B) - the size of the array B.

 After comparison, the calculated array A or B is recorded in block 13. If W> K, where K is the value of the specified threshold, then array B and the parameters of the displacement vector are stored in block 29 as a description of the next map, and if W <K, then at block 29, an array A is stored as another description of the next map of the active object. The process is repeated for each active object present in the current frame, which is checked by the comparison unit 30. The resulting array is stored in block 19.

 Next, the properties of active objects are formed and stored (Fig. 7). Establish a connection between the AO and the digital data source using at least one hyperlink for each active object. When forming and storing a transition corresponding to a given active object, a sequence of displaying the specified video frames of another separate plot for this active object is formed.

 These operations are performed in the editor at the stage of creating the script file (Fig. 7). In block 40, a previously created script is opened or a new script file is created. When a script file is opened, the script file structures are read into the computer’s RAM, as described below. When creating a script file in RAM, a place is allocated for the structure of the script file and their initial values are determined. Block 41 forming a list of video files fills the structure of the script file in the computer's RAM, describing the video files that will be used in this scenario. Data entry is carried out using the keyboard and the manipulator (mouse). The structure of the script file is described in detail below. In block 42 of the formation of objects is the filling of the structure of the script file in the random access memory of the computer that describes the objects. The structure includes information about the name of the object and a list of its hyperlinks. The description of the hyperlink includes information about the name of the link (for visual presentation during playback), directly hyperlinks to the source of digital data, for example, for an AO soccer ball, the link can point to its manufacturer’s WEB site: http: // www. adidas.com. Hyperlinks can be of different types. For example, a hyperlink may point to WEB sites, or may point to the Video File Playback Program, which is generated in block 42. This information is also included in the structure describing AO. The structure describing the AO includes information about the scope of the link: the video file and the range of frames (start and end frame), on which the link will be available when playing the script file.

 In block 43, the structure of the script file is filled in the random access memory of the computer that describes the playback program. This structure includes information about the name of the playback program, a text description of the playback program, the start video file, the start frame of this video file, from which it starts to play. The structure also includes information about a set of video file playback commands. Commands control the playback of a video file, for example, go to a certain video frame, stop on a certain video frame, etc. Block 44 navigates the video files — positioning on the desired video frame. Positioning may include, for example, playing a video file in normal mode, stopping the frame during playback, moving back and forth in increments of 1 frame, or 25 frames, switching to the first and last frame of the video file. In block 45, the structure of the script file is filled in the random access memory of the computer, which describes the markup of active objects. In the same block, the coordinates of the AO markers can be determined. The marking and selection of the cells of the active object can be performed manually - block 46, or automatically - block 47.

Manual methods include marking using well-known tools:
"Pencil" - identification of individual cells by highlighting cells on the video frame displayed on the screen using the manipulator (mouse);
"Rectangle" - identification of cells in a rectangular area on the frame, by stretching the image of the rectangle;
"Erase" - undo the identification of the cells of the object;
“Fill” - identification of all cells within the area bounded by previously identified cells;
“Magic Wand” - identification of cells adjacent to a given one and similar in color (similar to the Magic Wand tool in graphic editors such as Adobe Photoshop).

 Manual methods also include copying object cells from one frame to a specified frame range and frame-by-frame copying of object cells during frame-by-frame viewing of a video file.

 Automated markup methods can be performed in various ways during their development. One of the options for an automated way to perform markup will be described later.

 The markup coding unit 48 encodes the markup cells of active objects, as previously described. In block 49, the script file is written from the computer's random access memory to disk.

 The result is a script file having the following structure.

 The script file header includes information about the version, size, creation date, author, description of the script file.

 Description of video files describes the video files used in the script. Includes information about the hyperlink to the video file, video file ID, number of frames marked, video file duration in frames, video file length, for example, in 100 nanoseconds, number of frames per second, video size in X and Y, grid size in X and Y.

 Description of the playback programs determines the order and rules for playing video files, as well as the execution of additional events during playback, such as pauses, transitions, etc.

Description :: = [... <playlist> ...]
<playlist i> :: = <playlist id><namesize><name><descriptionsize><description><mediafileid><framenumber><commands_size> [.. <command i> ..]
<command i> :: = <command ID> [.. <command value i> ..]
<playlist id> - identifier of the playback program
<name size> - length of the field name of the playback program
<name> - name of the playback program
<description size> - size of the description field of the playback program
<description> - textual description of the playback program
<mediafile id> - identifier of the first video file to be played
<frame number> - frame number in the first video file to be played, from which playback starts
<commands size> - the number of commands in the playback program
[. . <command i> ..] - a list of commands in the playback program, the commands control the playback of the video file, for example, go to a certain video frame, stop on a certain video frame, etc.

 Description of Active Objects describes all AO scenarios. This section contains information about Active Objects and their properties.

Description :: = <object count> [... <object i> ...]
<object count> - the number of active objects in the script file
<object i>:: = <object id><namesize><name><type><options><reserve><linkscount> [... <links description i> ...]
<links description i> :: = <link name size><linkname><linksize><link><link conditions size> [.. <link condition i> ..] <link options>
<link condition i> :: = <condition id> [.. <condition parameter> ..]
<object id> - unique for this script file object identifier
<name size> - length of the field object name
<name> - object name
<links count> - the number of hyperlinks related to the active object
[... <links description i> ...] - list of hyperlinks related to the active object
<links description i> :: = <link name size><linkname><linksize><link><link parameters size><linkparameters><linkoptions>
<link name size> - field length hyperlink name
<link name> - the name of the hyperlink that appears in the context menu of the active object
<link size> - field length hyperlink
<link> - hyperlink
<link conditions size> - number of conditions link validity
[. . <link condition i> ..] - a list of conditions for the applicability of the link, sets the conditions for the applicability of the link, for example, the link acts on such a video file from such to such a frame
<link condition i> :: = <condition id> [.. <condition parameter> ..]
<condition id> - type of link availability on specific video frames.

 Description of Maps describes maps of all video files of the script.

Description :: = [... <map i> ...]
<map i>:: = <frame number><countframe><countmap><countmarkers> [.. <marker i> ..] [... <object map i> ...]
<frame number> - the number of the video frame from which this card is applied
<count frame> - the number of frames to which this card is applied.
<count map> - the number of active objects, the markup of which is described by this map
<count markers> - number of markers
[... <marker i> ...] - list of markers
<marker i> :: = <marker><pos><marker Y pos>
<marker X pos> - X coordinate in grid coordinates
<marker Y pos> - Y coordinate in grid coordinates
[... <object map i> ...] - list of markings of active objects
<object map i> - markup of one active object
<object map i>:: = <type + object ID>[<key map number><motionvector><typedifference>][<beginpoint><circumscribingframe><countstretches><stretches>]
<type + object ID> - bits 2-15 - a unique identifier of the object, bit 1 - determines the object map relative or key (0-key, 1 - relative)
[<key map number>] - key card number, the field is present if the card is relative
[<motion vector>] - coordinates of the displacement vector. The field is present if the map is relative
[<type difference>] - defines the type type of the relative map
0 - the object is displaced, the shape has not changed, the difference is not recorded
1 - the shape of the object is changed and the difference markup must be added to the moved object
2 - the shape of the object is changed and the difference markup must be subtracted from the moved object
3 - the shape of the object is changed and there is a difference markup that needs to be added to the moved object, then there is a difference markup that needs to be subtracted from the moved object.

 The following fields are present if bit 1 of <type + object ID> is equal to 0 or <type difference> is not zero.

[<begin point>] is the starting point of the markup of the object, Y is the coordinate of the upper left marking point of the object, if coding is performed horizontally, X is the coordinate of the lower right marking if coding is carried out vertically
[<count stretches>] - number of markup elements
[<stretches>] :: = [... (<stretch i> | <modifier>) ...] - list of markup elements
<stretch i> :: = <start X / Y><end X / Y>
<start X / Y> - X / Y coordinate of the starting point of the segment
<end X / Y> - X / Y coordinate of the end point of the segment
X - horizontal coding
Y - vertical coding
<modifier>:: = (<line continue> | <pass lines> | <repeat stretch>) - modifier
<line continue> :: = <line continue mod><stretchi> - Describes the break of the current marking line or, in other words, the length of the marking in the current line. If the current line consists of more than one line, the modifier indicates that the next line belongs to the same line
<line continue mod> - Line Break modifier, modifier identifier
<stretch i> - the next segment in this line
<pass lines>:: = <pass lines mod><Y / X new> - Modifier for skipping lines. Describes lines that do not contain the markup of an object. Defines Y (horizontal coding) / X (vertical coding) the coordinate of the next line where the markup occurs
<pass lines mod> - Line Skip modifier, modifier identifier
<Y / X new> - Y (horizontal coding) / X (vertical coding) the coordinate of the line where the markup appears
<repeat stretch> :: = <repeat stretch mod><countline><stretches on line> - Repeat modifier, used to encode repeating segments or repeating sequences of segments
<stretches on line> - [<stretches>] - a list of segments representing repeating sequences
<repeat stretch mod> - Repeat modifier, modifier identifier
<count line> - the number of repetitions of the line.

 The map index contains the card numbers and their offset from the beginning of the script file, for quick access to the cards.

<addr of index>. Maps index :: = [... <index i> ...]
<index i> :: = <map number><countframe><addr of map>
<map number> - map number
<count frame> - the number of video frames to which this card applies
<addr of map> - offset from the beginning of the script file to this map
At the stage of creating the script file, the selection and identification of AO cells is carried out using manual marking tools or automatically.

 For example, at the stage of creating the script file, the cells are automatically identified after specifying the characteristic points on the AO circuit and the cells inside this circuit are identified, and the next video frame automatically monitors the location of the characteristic points and the cell position (Fig. 8).

In block 60 (Fig. 8), the coordinates of points on the AO contour are determined. The coordinates are determined by the author, for example, the author marks points on the video using the manipulator. These coordinates are stored in the data array. Next, in block 61, the luminance component of the video image of this frame is determined by the formula:
Y = α ^* R + β ^* G + γ ^* B,
where α = 0.299, β = 0.587, γ = 0.114, R, G, B are the color components of the video image in the RGB color space adopted for displaying images on computers. The result is an array C. Then, cells are determined in block 62 inside the AO circuit defined in block 60. First, points inside the AO contour are determined using one of the well-known methods of "filling" the figure along a given contour, for example, using the standard Windows FillRgn API function. Then, the cells inside the AO contour are determined by approximating previously found points on the grid. Next, in block 63, the coordinates obtained in block 62 about AO marking on this frame are stored. Then, in block 64, the transition to the next video frame is performed. In block 65, the luminance component of the video image of this frame is determined by the formula described above. The result is an array D. In block 66, a search is made for points defined on the AO contour in video frame C in block 60 on video frame D. The Pyramidal Implementation of the Lucas Kanade Feature Tracker method described in the Pyramidal Implementation of the Lukas Kanade article is used Feature Tracker. Description of the algorithm "Jean-Yves Bouguet. Intel Corporation Microprocessor Research Labs (http: // www. Intel.com). Arrays C, D and an array of coordinates of points on the AO contour are used as data. Then, in the comparison block 67, it is checked whether points are found on the video frame D. If the points are found, then the cells inside the AO contour on the video frame D defined by these points are determined in block 62. Further, information about AO marking is stored on video frame D in block 63. In block 68, it is checked whether the process should be continued - the author did not interrupt the process? If the process needs to be continued in block 69, array C is assigned the value of array D, the array of coordinates of points defined for video frame D is taken as the array of coordinates of the points, and then the process is repeated starting from block 64. The process continues until the contour points are in block 66 , the condition is checked in block 67, and until it is interrupted by the author, the condition is checked in block 68.

 At the playback stage, the script file maps are decoded, when the position of the manipulator cursor on the monitor screen coincides with the active object and when one of the manipulator keys is pressed, for example, the left mouse button, the hyperlink is accessed through the first available, generated and stored at the editing stage. If the position of the manipulator cursor on the monitor screen coincides with the active object and when the right button of the manipulator is pressed, several hyperlinks formed and stored during editing are selected from the menu. When you then press the left button of the manipulator to one of the menu hyperlinks, a transition is made in accordance with the connection of this hyperlink with the corresponding source of digital data.

 The playback stage is as follows (Fig.9). At block 80, the selected script file is opened. At this stage, the following script file structures are read into the computer’s RAM: header, description of video files, description of active objects, description of playback programs, description of indices. After that, in block 81, the script is read from the script file and the active objects markup map is decoded, related to the initial video frame described in the initial playback program. Decoding a map with the marking of active objects can be carried out by determining whether the cursor is between the extreme X coordinates for each row of mouse cursor location cells (whether the cursor falls into the AO region), however, the limitation of this method is the case in which the AO map refers to several cards , and the cursor quickly changes its location. In this case, processing a very large data array is required. Therefore, the description of another decoding operation by an operation inverse to encoding, which reduces the amount of processed data, is given in the description below.

 In block 82 (Fig. 9), the image of object markers is formed and superimposed on the contents of the video frame in accordance with the coordinates contained in the frame layout. Block 83 displays a video frame on a monitor screen. In the comparison block 84, a determination is made whether the mouse pointer is in the AO markup area; for this, the information obtained at the decoding stage in block 81 is used. If these coordinates are in the cell of any of the AOs, then in block 85, on the screen below the cursor image displays a tooltip - the name of the object. Next, in the comparison block 86, it is checked whether the left key of the manipulator (mouse) is pressed, if pressed, then in block 87 a transition is performed according to the first object hyperlink located in the visibility zone. Unit 87 also remembers the number of the video file and the number of the video frame from which the transition was made, in order to ensure, at the playback stage, the possibility of returning back to the video frame from which the transition was made. This is done similarly to Internet browsers, but in a new application to watching videos. Next, the process is repeated, starting with block 81 for a new video frame after the transition. If the right mouse button is pressed, which is checked in the comparison block 88, then the menu — a list of hyperlink names — is displayed on the monitor screen with block 89. If the right mouse button is not pressed, then block 90 moves to the next video frame in accordance with the playback program. Next, the process is repeated starting from block 81 for a new video frame after the transition. After the menu is displayed by block 89 when you select one of the hyperlinks, move the cursor to it and press the left mouse button, which is checked by block 91, the action specified by this link by block 92 will be performed. If none of the menu items in block 91 are selected, then block 90 performs the transition to the next video frame in accordance with the playback program. Next, the process is repeated starting from block 81 for the new video frame.

 The decoding of cards from the script file is as follows (figure 10, 11).

 At the input of the decoder (figure 10), block 100 receives an array of encoded coordinates of AO cards. Further in the block of comparison 101 it is checked: this card is key or not. A card is a key card if all AO cards included in it are key. If it is key, then in block 102, all AO cards included in this card are decoded as follows. A card of one AO is selected in block 120 (Fig. 11). Next, in block 121, the resulting matrix is filled with zeros. Then block 122 selects the ordinate Y0 of the initial cell of AO, which was determined at the coding stage of block 4 (Fig. 5). Then, in block 123 (Fig. 11), the ordinate of the current decoded line for the cells is assigned the value Y0. Then, in block 124, the first segment of the AO marking line is extracted from the AO map and the corresponding cells in the matrix (previously filled with zeros in block 121) are filled in with the number of this AO. Next, in the comparison block 125, the next record in the card is checked - is it a break modifier? If yes, then the next segment with cells in block 124 is extracted. If this is not a gap modifier, then in the comparison block 126 this record is checked - skip modifier? If it is a skip modifier, then in block 127 the ordinate of the current decoded line with a number of cells is assigned the value determined by the skip modifier parameter. If this is not a skip modifier, then in the comparison block 128 the current record in the AO card is checked - repeat modifier? If it is a line-with-cell repeat modifier, then in block 129 the coordinates of the current row (row) of the resulting matrix are copied to the following adjacent rows, the number of which is indicated in the modifier parameter. Then, in block 130, the ordinate of the current decoded row of a series of cells is assigned the ordinate of the previous decoded row plus the value of the modifier parameter. If the current record in the AO card is not a repeat modifier, then in block 131 the ordinate value of the previous decoded row of cells (row) is increased by one. The process continues until at least one line segment of a row of cells with the current ordinate Y is detected. This condition is checked in block 132. The result is a matrix (for example, as shown in Fig. 1), in whose cells, respectively, the AO coordinates are stored AO numbers , and in all other cells - zeros. The decoding result of block 102 (Fig. 10) is stored in block 104 and in the key card buffer AO - block 103. Block 103 serves to accelerate the decoding of relative AO cards. If the result of the comparison of block 101 is "NO", then this card is relative. Then, in block 105, a card of one AO is selected from the key card buffer of block 103, and if the key card is not in block 103, it is selected from block 100 and decoded by block 106 (Fig. 10), as previously shown in blocks 120-132 ( Fig. 11).

 Next, in block 107 (Fig. 10), the AO markup defined on the key map is shifted by the parameters of the displacement vector described in the relative map of the AO. Then, in block 108, the difference between the AO marking on the key card and the relative one is decoded, using the method described above in blocks 120-132 (Fig. 11). Next, in block 109 (Fig. 10), the difference defined in block 108 is added to the offset in block 107 by the parameters of the offset vector of the AO markup. The process continues for all AOs described in this map, starting at block 105. This condition is checked at block 110.

 An example of a specific implementation of the method is explained below (Fig-19).

The Active Video Editor (AV Editor) includes a complete set of tools for creating AV Script Files. The author can create a new AV script file or open an existing project (AV script file). To create and edit an AV Script, perform the following actions:
- Setting the properties of the Script.

 - Manage media files.

 - Creating and editing playback programs.

 - Creating and editing AO.

 - Creating AO markup.

 The AV editor is implemented in an interface that allows you to create script files of several media files in one project (Fig. 12).

 Setting script properties is implemented by calling "Project-> Options" from the menu of the AV Editor. An important operation is to set the dimension of the grid for discretizing the regions of AO marking, which is selected by the author based on his preferences for the marking accuracy. The marking of all AOs is created in grid coordinates, so setting a small size of the grid cell allows the author to more accurately mark the regions of the AO, which, at the same time, will increase the size of the Script File and increase the processor load during AV playback. Conversely, setting a large grid cell size will reduce the script file and reduce processor load. The author also establishes his name and description of the script in the operations of the AB project. The first Playback Program is also installed, which should be executed first when playing the Script File. The first playback program is selected from the list of currently created ones.

 Media files are managed in the AV Editor through the use of the Video File Inspector. Provides the ability to add, delete media files, change the names of media files, access paths, select media files to create and edit markup. After selecting a media file, its contents are displayed in one of the windows of the AV Editor and the author can start creating or editing descriptions of the media file, Playback Programs, AO properties, drawing AO markup, etc.

 Creation and editing of the Playback Programs is carried out through the interface of the Playlists Inspector (Figs. 12, 16). The author can create, delete, edit Playback Programs. The author enters the name, description of the Playback Program, selects the media file, start frame, and commands of the Playback Program. Teams are an important part of the Replay Program during its creation. At least one media file must be added to the project before creating the first Playback Program. The concept of Playback Programs allows you to create complex playback sequences, including loops, pauses, stops, changing the playback speed, turning sound on or off, calling one Playback Program from another, thus creating many Playback Programs. This flexibility is achieved using the commands listed below in table 1.

 Each command has 3 required parameters and a variable number of additional parameters. 2 required parameters - “launch video file” and “launch frame” determine the event - the point of the Playback program where this command will be executed. Accordingly, if the playback program is designed in such a way that this point (frame or file) is not reached, this command will never be executed.

 In this AV editor, the Playback program commands are visible to the author at the bottom of the Edit Playlist window. The corresponding buttons allow the author to control the commands of the Playback program - add, delete, edit. The author can select from the list any commands available for use.

 The Object Inspector (Fig. 12, 15) serves the author as a tool for creating, deleting and editing AO properties. The author defines and enters the names of AOs and their hyperlinks. The name of the AO will be shown in the form of a hint during further playback when the user hovers the cursor of the pointing device over the markup region corresponding to the given AO. All links of the highlighted AO are listed at the bottom of the Object Inspector interface. AO links can be added, deleted, edited by the author, using the facilities of the Object Inspector (Fig. 15 a). The author has access to the property fields of hyperlinks (Fig. 15 b): the name that will be displayed in the context menu when playing the script, the path of the hyperlink, the type of hyperlink selected by the author from the list of possible (Fig. 15 c) —total 5 types.

 The communication type "Playback program" (Fig. 15 c) is the default value that appears in the drop-down menu of the Object Inspector. The type of connection “Playback program as an external process” is used to play sound simultaneously with the current video playback. The type of communication of JSC "execute file and play video" is used to communicate the JSC with any other files. For example, if a user of AV Player (version for Windows OS) clicks on such a connection, the corresponding Windows application will be launched and the file associated with the application will be loaded into this application. In the case of the “execute file and pause video” connection type, clicking on such a connection launches the application with the required file, but at the same time pauses the viewing of the current video. Based on the last two types of links, the author indicates the paths to files or the URL of a site on the Internet or Intranet. The type of connection "streaming video" allows you to establish a connection with streaming video.

 To control the display of connections during the playback of Script AB, the parameter "communication availability on multiple frames" (Fig. 15 g) is used. This means that, in the case of a call to the context menu by the user of the AV Player, communication will be available in the menu if the frame on which the menu was called falls into the range of communication availability frames. If the frame does not fall within the communication availability range, this communication will be unavailable. Accessibility of links is controlled from the Link Conditions Editing window, which contains 3 fields: link to the media file, start and end frames to determine the range of frames for link availability. If no conditions are specified, the connection is considered as accessible on any video frame of any media file included in the current project (of course, if AO markup is present on the frame).

 Marking areas AO can be done by many means of the AV Editor. To create markup for a specific object, the author must select the appropriate AO from the drop-down menu of the Active Objects menu. After that, the author finds a frame on which it is necessary to apply markup using the navigation tools available on the AV Editor toolbar (Fig. 12, 13). The instrumental ruler includes the playback control buttons: Play, Pause, Stop, Go to the first frame, Go to the last frame, Go to frame back, Go to frame forward, Go 25 frames forward, Go 25 frames back.

 The buttons that provide access to the Scene Definition, Single-Frame Action functions are used to facilitate the work of the author of marking AO script. The "Scene Definition" function is similar to the "Play" function, except that playback stops when a scene changes. The sensitivity of this function is controlled by the "Scene Definition" parameters, accessible by "File-> Reference". The frame-by-frame viewing function processes the video file without loss or frame skipping. Used in combination with a drop down menu. With the menu option "no action" is used only for navigation. One pair of buttons on the interface of the AV Editor works while the author holds them pressed with the mouse buttons - the AV Editor plays the video frame by frame and performs the selected actions. When the author releases the buttons, the program stops playing frame by frame. Another pair of buttons implements the same function, but, unlike the previous one, the author does not need to hold the buttons - the author presses the control button to start frame-by-frame editing and the AV Editor performs the specified action until the author repeatedly presses the same button to end the frame-by-frame editing.

 As soon as the author finds the desired frame, he can begin the process of marking, using manual and automatic methods (Fig.12, 14).

 Manual methods include using the following tools: Object Selector, Pencil, Auto Fill, Fill, Erase, Move. The author selects a tool by clicking the appropriate buttons on the Layout toolbar.

 The menu for selecting the current object (Fig. 12) allows you to obtain information about the affiliation of marking cells to a specific AO by double-clicking on the markup. Other tools are similar in use with image editor tools, except that the AV Editor draws and defines markup using grid cells, not pixels.

 The automated marking method allows you to track changes in the shape, displacements of objects and apply the correct markup to the next frame with minimal use of manual tools. The author selects the AO, manually determines the outline of the image of the object that will correspond to the AO, and starts the automatic tracking process. The AV Editor moves to the next frame and determines the contour in the next frame based on the contour data of the previous frame. Next, the tracking process is repeated from frame to frame until the author stops it. At each step, information from the previous frame is used.

 After the description and markup processes are completed, the author saves the script to the AV Script file (Fig. 12) using the file management tools of the AV Editor’s main menu "File-> Save project" or "File-> Save project as ...".

 The Active Video Player (AV Player) (Fig. 17) is intended for reading, playing interactive AV Scripts and media files. AV Scenarios are an interactive video, which can be controlled directly through objects displayed on the screen. Characters, objects, goods or other visual objects that the viewer sees on the screen of the AV Player can respond to the requests of the viewer. The viewer can access such Active Objects through, for example, clicks of the manipulator. In response to the clicks of the viewer produced directly in the area of the Active Object, the AV Player returns information related to this AO. For example, another Active Video Playback Program is launched, regular video, streaming video, or control is transferred to other applications responsible for processing specific files, for example, documents, hypertext pages, graphics, etc. (Fig.17). The same AO may have different hyperlinks in the context menu, depending on the video frame on which the manipulator clicked.

 For successful orientation of the viewer, AOs are indicated on the screen of the AV Player using Markers (Fig. 19). In order to avoid inconvenience during viewing, AV Markers are positioned in the upper left marking point of the AO region. While the manipulator cursor is on AO, the cursor changes its shape from arrow to pointer and at the same time a tooltip with the name of AO is displayed. When the manipulator clicks the buttons back or forward, a previously memorized transition through the links occurs.

 If the viewer clicks on the AO, for example, using the left button of a computer mouse, the user clicks on the hyperlink set by default. In the AO context menu, by default the link takes the first position.

 When right-clicking the manipulator, a context menu is displayed (Fig. 18). The context menu consists of several parts: AO name, a list of hyperlink names. AO can have any number of hyperlinks, the first of which is the default link. The same AO can have different hyperlinks depending on the video frame on which the context menu is called.

 Right-clicking on a screen area where there is no AO will bring up the general control menu of the AV Player, which is not dependent on the context of the current display.

 A version of the AV Player, which implements the above functionality, was developed and tested in the environment of Windows 98 / Me / 2000 / XP.

 The most successfully claimed method of object-oriented interactive processing of video information is industrially applicable in computer technology, mainly in computer active video (AV) systems for creating scripts and viewing digital video files, as well as in areas such as interactive television, video on demand, TV broadcasting via the Internet, interactive training systems, video guides, personal, mobile communications, computer and television games.

Claims (10)

1. A method of object-oriented interactive processing of video information, including the step of creating a script file and the step of playing back video images, while at the stage of creating the script file, a video frame is displayed on the monitor screen with an image of at least one visual object intended to be subsequently activated at the playback stage , - of the active object, the video frame is divided into X • Y cells, where X and Y-1, 2, 3.. . in the form of elementary squares or rectangles, cells are selected whose location coincides and does not coincide on the video frame with the location of the active object, assigning to the cells matching the location of the cells of the active objects corresponding numbers to them and assigning zeros to the cells whose location does not coincide with the location of the cells of the active objects, perform intraframe coding, form an array of map data for all active objects of a given video frame, form and store properties of an asset objects, form and remember the transition corresponding to this active object, and if the position of the active objects in the next video frames of the video file is constant, then for such video frames the map of the previous video frame belongs to them, and if the position of the active objects in the next video frame changes, they form a new map for these active objects, form and store a script file formed by a sequence of cards in which each card corresponds to at least one video frame of the video file, while the video playback stage uses a video file, a script file and current information on the position of the manipulator cursor on the monitor screen, when the manipulator cursor moves on the monitor screen and when the cursor location coincides with the active object - with those cells whose location coincides with the location of the active object on the video frame, cursor type transform, when the cursor coincides with the location of the active object and when you press one of the keys on the manipulator, a transition corresponding to yes the active object and memorized at the stage of creating the script file, and when playing a video frame of a video file with another active object on the same video frame, when the cursor coincides with the location of another active object and when you press the same manipulator key, the transition corresponding to another active object is different, that in order to form an array of map data of all active objects of a video frame, intraframe coding is performed by selecting the extreme row along one of the X or Y coordinates from the cells of the active object, determining the coordinates of the extreme cells of this row, determining the coordinates of the two extreme cells in subsequent rows, and if the extreme and subsequent rows contain cells whose location does not coincide with the location of the active object in these rows, then enter the map data array modifier of breaking a number of cells and remember the coordinates of the extreme cells of the active object at the place of breaking the series, analyze each subsequent row for the absence of active objects in the row of cells, and if the row is free from i of active objects, then the row skip modifier is introduced, each subsequent row is analyzed for the identity of the previous and subsequent rows adjacent to it, if they are identical, then the row repeat modifier is entered, after the intraframe coding, the video frame map is stored, if the geometric shape and / or the position of the active object in the next video frame produces its inter-frame coding, in which if the position of the cells of the active object in the next video frame changes, and its geometric shape is saved Thus, the parameters of the cell displacement vector of the active object are introduced into the data array of the map of the next video frame, if the position of the cells of the active object and its geometric shape are changed, then the mentioned intraframe coding of this video frame is performed, obtaining the data array A, then the parameters of the cell displacement vector are entered into the data array the active object and calculate the difference between the two matrices of data arrays for the active object on the previous and next video frame, obtaining a difference matrix, the resulting difference matrix the code is encoded with the above-mentioned intraframe coding, obtaining a data array B, the size in bytes of the data array A is compared with the data array B according to the formula
W = {[P (A) -P (B)] / P (A)} • 100%,
where W is the calculated gain in%;
P (A) - the size of the array A;
P (B) - the size of the array B,
and if the value W> K, where K is the value of the given threshold, then save array B as the data array of the next map, and when determining the location of the active object of this video frame, use the data array A of the previous map, and if W <K, then save the data array A as an array of the next map, the formation and storing of the properties of active objects is carried out by establishing links using at least one hyperlink for each active object, and a connection is established with various sources of digital data, etc. forming and storing the transition corresponding to this active object, the sequence of showing other specified video frames for this active object is formed, at the playback stage, the script file cards are decoded, when the position of the manipulator cursor on the monitor screen coincides with the active object, and when one of the manipulator keys is pressed, the transition by the first available hyperlink generated and stored at the stage of creating the script file, with the coincidence of the cursor position a torus on the monitor screen with the active object and when you press another of the manipulator keys, several hyperlinks are selected from the menu and when you then press another manipulator key to one of the hyperlinks in this menu, the transition is made in accordance with the link of this hyperlink with the corresponding digital data source.  2. The method according to p. 1, characterized in that at the stage of creating the script file, the selection of the cells of the active object is carried out using manual markup.  3. The method according to p. 1, characterized in that at the stage of creating the script file, the cells of the active object are selected automatically after specifying the characteristic points on the contour of the active object and identifying the cells inside this circuit, and the next video frame automatically tracks the location of the characteristic points and changes in the position of the cells of the active object.  4. The method according to p. 1, characterized in that at the stage of playback as a source of digital data using a video file.  5. The method according to p. 1, characterized in that at the stage of playback as a source of digital data using a playback program that provides a specified order of playback of frames of video files.  6. The method according to p. 1, characterized in that at the playback stage, text documents, HTML documents, graphics, a computer program are used as a source of digital data.  7. The method according to p. 1, characterized in that at the playback stage, the transitions are stored by hyperlinks of active objects, to return to the plots from which and to which the transitions were made.  8. The method according to p. 1, characterized in that at the playback stage, active objects are marked.  9. The method according to p. 1, characterized in that at the playback stage, when the cursor position of the manipulator coincides with the location of the active object, a tooltip is generated.  10. The method according to p. 1, characterized in that the decoding is carried out by the operation, the reverse encoding.

Images