RU2550544C1 - Method for motion compensation in digital dynamic stereoscopic video - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: method for motion compensation in digital dynamic stereoscopic video, wherein during motion compensation by a stereo video codec in stereo pairs of a video stream, a combination of search algorithm for motion compensation of frame fragments is used to obtain plane frame projections of the dynamically tracked and periodically approximated geometrically reconstructed real imprinted sculpted surface, as well as approximation of the movement of frame fragment projections with approximation physical laws of motion of real imprinted mobile objects corresponding to said projections.

EFFECT: higher average values of transmission frame features of stereo video codecs operating in real time.

2 cl, 21 dwg, 3 tbl

Description

The technical solution relates to the field of digital television, in particular, to methods of motion compensation in digital dynamic stereo video images and can be applied in digital television broadcasting systems using stereo video codecs and operating in real time.

A similar method is known for motion compensation in digital dynamic stereo video images based on the extension of the MPEG-4 standard for stereo and multi-angle coding using a basic profile, described in Vetro, A .; Wiegand, T .; Sullivan G.J. “Overview of the Stereo and Multiview Video Coding Extensions of the H.264 / MPEG-4 AVC Standard”, TR 2011-022 January 2011, http://www.merl.com, as well as in the book of Ian Richardson “Video coding N. 264 and MPEG-4 - the standards of a new generation ”Moscow: Technosphere, 2005. - 368 p. and consisting of the following: before processing the video stream, all its frame stereopairs are logically divided into reference and predicted, moreover, the reference stereopairs are arranged with a periodicity of one or more predicted stereopairs, and the repetition frequency of the reference stereopairs is pre-set, then rectangular Cartesian frames are selected in the planes of the frame images of stereopairs coordinate systems whose axes are directed parallel to or along the sides of the frame rectangles of stereo pairs, and as units of measurement along the axes in pixels are taken, during the processing of the stereo stream for receiving each next stereo pair, the encoder polls the image sensors, then the resulting images are transferred to the format and color system required for processing, then they are stored in the encoder memory as the current frame stereo pair, then the left and right frames of the current stereo pair are logically they are divided into rectangular fragments of the same size, called macroblocks, and the sizes of the macroblocks are pre-set, and immediately after the division of the next stereopair adres on macroblocks for each macroblock fix the coordinates of its position in the corresponding coordinate system used, during the processing of the stereo stream, the stereo pair indexes are tracked, during the processing of each current stereo pair, first it is determined by the stereopair index that it belongs to the reference or predicted stereo pairs, if the current stereo pair belongs to reference stereo pairs, motion compensation in the image is used only for the right frame of the stereo pair, and after processing, save this stereo a pair both in the encoder’s memory and in the decoder’s memory, if the current stereo pair belongs to the predicted stereo pairs in both its frames, motion compensation is used in the image, when conducting motion compensation in the images, the encoder generates a motion compensation forecast, and the decoder reconstructs the motion compensation, for performing motion compensation of each left predicted stereo pair frame as a sequential reference frame, both the encoder and the decoder select the previous frame in the video sequence sequence, and to carry out the motion compensation of each right predicted frame of the stereo pair as a parallel reference frame, both the encoder and the decoder select the parallel left frame of the video sequence, in the process of generating the motion compensation prediction encoder, a binary motion mask is first constructed for each macro block of the predicted stereo pair , then, according to the movement masks, for all macroblocks, signs of movement are fixed, according to which all macroblocks, in relation to movement, are classified as external, boundary and internal, while the sign of movement of the external macroblock is assigned to those macroblocks that contain only fixed pixels, the sign of movement of the boundary macroblock is assigned to those macroblocks that contain both moving and fixed pixels, and the sign of movement of the internal macroblock is assigned to those macroblocks that contain only moving pixels, then for internal and boundary macroblocks stereopairs search for their integer displacement vectors to the best macroblock forecasts in searches areas of the reference extrapolated frames, and the processing order of the stereo pair frames when searching for macroblock forecasts is arbitrarily set, before searching for macroblock forecasts, the macroblocks of the reference frames are first adjusted by interpolating the values of the stationary pixel signals of these frames from the values of their moving pixel signals, and then the extrapolation of the reference frames by their bordering pixels to sizes sufficient to carry out motion compensation using zag numerical displacement vectors, in this case, the motion signs of all macroblocks and the motion mask of the boundary macroblocks of the reference frames are used for fitting, and the dimensions of the extrapolated frames are predefined, the search algorithm for the predicted displacement vectors and the quality criterion of the search macroblock forecasts are preselected, when setting the positions of the search areas, the centers of these areas are combined with the centers of the corresponding macroblocks, while the sizes of the search areas are pre-set during the search for the best m of acroblock matches, only those possible macroblock forecasts are predicted that fully fit into the format of the corresponding reference extrapolated frame of the stereo pair, after the end of the search for macroblock forecasts for internal and boundary macroblocks of the current stereo pair, their macroblock residues are determined, while for finding matrices of the macroblock remainder of any of the indicated macroblocks from the original image matrices of this macroblock, the corresponding matrices of its found macroblock forecast are subtracted, yielded e the encoder’s current encoded stereo pair is stored in the encoder’s memory, as well as the motion signs of all its macroblocks and the motion mask of its boundary macroblocks, and then the motion signs of all macroblocks, difference macroblock residues, integer predicted displacement vectors and motion mask of the boundary macroblocks of the current stereo pair are prepared for further coding in the process of reconstruction reconstruction of the motion compensation by the decoder, frames of the current stereo pair are restored, while the left is restored first, and then right st frame of the stereopair, when restoring the left frame from the decoded forecast vectors of displacement of the macroblocks of this frame, the positions of macroblock forecasts of internal and boundary macroblocks are determined in the previous reference extrapolated frame of the left stream, and before determining the position of macroblock forecasts to the specified reference frame according to the same rules and using those the same parameters as in the encoder, first fit its macroblocks, and then extrapolate the frame, after determining the positions of the macroblocks initial forecasts of the left frame, the current left frame of the stereo pair is restored, in the process of recovering this frame of the stereo pair, the corresponding image matrices of its specific macroblock forecast and the image matrix of its decoded macroblock remainder are added to find the source image matrices of each internal macroblock, and the corresponding image matrices of each boundary macroblock are added image matrices of its specific macroblock forecast and image matrix pressing its decoded macroblock remainder, after which the fixed pixel values of the boundary macroblock are loaded with the values of the corresponding pixels of the corresponding reference extrapolated frame without offset, to find the source image matrices of each external macroblock, their values are loaded with the values of the corresponding matrices of the corresponding reference extrapolated frame without offset, then restore the right frame of a stereo pair, and immediately before its restoration By attaching to a parallel reference frame for it and already decoded the left frame according to the same rules and using the same parameters as in the encoder, first fit its macroblocks and then extrapolate this frame, then the current restored stereo pair is stored in the decoder memory, and signs of movement of all its macroblocks and a mask of movement of its boundary macroblocks, after which the current stereo pair is transferred to the format and color system required for display and then the image display device is displayed Eden. The disadvantage of this analogue is the low average value of the throughput of personnel in processing the stereo video stream in real time, due to the use of the search algorithm for motion compensation for the predicted frames of the stereo video stream.

As a prototype, a method of motion compensation in digital dynamic stereo-video images based on the extension of the MPEG-4 standard for stereo and multi-angle encoding using a basic profile, described in Vetro, A .; Wiegand, T .; Sullivan G.J. “Overview of the Stereo and Multiview Video Coding Extensions of the H.264 / MPEG-4 AVC Standard”, TR2011-022 January 2011, http://www.merl.com, as well as in Ian Richardson’s book “H.264 Video Encoding and MPEG-4 - standards of a new generation ”Moscow: Technosphere, 2005. - 368 p. and consisting of the following: before processing the video stream, all its frame stereopairs are logically divided into reference and predicted, moreover, the reference stereopairs are arranged with a periodicity of one or more predicted stereopairs, and the repetition frequency of the reference stereopairs is pre-set, then rectangular Cartesian frames are selected in the planes of the frame images of stereopairs coordinate systems whose axes are directed parallel to or along the sides of the frame rectangles of stereo pairs, and as units of measurement along the axes in pixels are taken, during the processing of the stereo stream for receiving each next stereo pair, the encoder polls the image sensors, then the resulting images are transferred to the format and color system required for processing, then they are stored in the encoder memory as the current frame stereo pair, then the left and right frames of the current stereo pair are logically they are divided into rectangular fragments of the same size, called macroblocks, and the sizes of the macroblocks are pre-set, and immediately after the division of the next stereopair adres on macroblocks for each macroblock fix the coordinates of its position in the corresponding coordinate system used, during the processing of the stereo stream, the stereo pair indexes are tracked, during the processing of each current stereo pair, first it is determined by the stereopair index that it belongs to the reference or predicted stereo pairs, if the current stereo pair belongs to reference stereo pairs, motion compensation in the image is used only for the right frame of the stereo pair, and after processing, save this stereo a pair both in the encoder’s memory and in the decoder’s memory, if the current stereo pair belongs to the predicted stereo pairs in both its frames, motion compensation is used in the image, when conducting motion compensation in the images, the encoder generates a motion compensation forecast, and the decoder reconstructs the motion compensation, for performing motion compensation of each left predicted stereo pair frame as a sequential reference frame, both the encoder and the decoder select the previous frame in the video sequence sequence, and to carry out the motion compensation of each right predicted frame of the stereo pair as a parallel reference frame, both the encoder and the decoder select the parallel left frame of the video sequence, in the process of generating the motion compensation prediction encoder, a binary motion mask is first constructed for each macro block of the predicted stereo pair , then, according to the movement masks, for all macroblocks, signs of movement are fixed, according to which all macroblocks, in relation to movement, are classified as external, boundary and internal, while the sign of movement of the external macroblock is assigned to those macroblocks that contain only fixed pixels, the sign of movement of the boundary macroblock is assigned to those macroblocks that contain both moving and fixed pixels, and the sign of movement of the internal macroblock is assigned to those macroblocks that contain only moving pixels, then for internal and boundary macroblocks stereopairs search for their integer displacement vectors to the best macroblock forecasts in searches areas of the reference extrapolated frames, and the processing order of the stereo pair frames when searching for macroblock forecasts is arbitrarily set, before searching for macroblock forecasts, the macroblocks of the reference frames are first adjusted by interpolating the values of the stationary pixel signals of these frames from the values of their moving pixel signals, and then the extrapolation of the reference frames by their bordering pixels to sizes sufficient to carry out motion compensation using zag numerical displacement vectors, in this case, the motion signs of all macroblocks and the motion mask of the boundary macroblocks of the reference frames are used for fitting, and the dimensions of the extrapolated frames are predefined, the search algorithm for the predicted displacement vectors and the quality criterion of the search macroblock forecasts are preselected, when setting the positions of the search areas, the centers of these areas are combined with the centers of the corresponding macroblocks, while the sizes of the search areas are pre-set during the search for the best m of acroblock matches, only those possible macroblock forecasts are predicted that fully fit into the format of the corresponding reference extrapolated frame of the stereo pair, after the end of the search for macroblock forecasts for internal and boundary macroblocks of the current stereo pair, their macroblock residues are determined, while for finding matrices of the macroblock remainder of any of the indicated macroblocks from the original image matrices of this macroblock, the corresponding matrices of its found macroblock forecast are subtracted, yielded e the encoder’s current encoded stereo pair is stored in the encoder’s memory, as well as the motion signs of all its macroblocks and the motion mask of its boundary macroblocks, and then the motion signs of all macroblocks, difference macroblock residues, integer predicted displacement vectors and motion mask of the boundary macroblocks of the current stereo pair are prepared for further coding in the process of reconstruction reconstruction of the motion compensation by the decoder, frames of the current stereo pair are restored, while the left is restored first, and then right st frame of the stereopair, when restoring the left frame from the decoded forecast vectors of displacement of the macroblocks of this frame, the positions of macroblock forecasts of internal and boundary macroblocks are determined in the previous reference extrapolated frame of the left stream, and before determining the position of macroblock forecasts to the specified reference frame according to the same rules and using those the same parameters as in the encoder, first fit its macroblocks, and then extrapolate the frame, after determining the positions of the macroblocks initial forecasts of the left frame, the current left frame of the stereo pair is restored, in the process of recovering this frame of the stereo pair, the corresponding image matrices of its specific macroblock forecast and the image matrix of its decoded macroblock remainder are added to find the source image matrices of each internal macroblock, and the corresponding image matrices of each boundary macroblock are added image matrices of its specific macroblock forecast and image matrix pressing its decoded macroblock remainder, after which the fixed pixel values of the boundary macroblock are loaded with the values of the corresponding pixels of the corresponding reference extrapolated frame without offset, to find the source image matrices of each external macroblock, their values are loaded with the values of the corresponding matrices of the corresponding reference extrapolated frame without offset, then restore the right frame of a stereo pair, and immediately before its restoration By attaching to a parallel reference frame for it and already decoded the left frame according to the same rules and using the same parameters as in the encoder, first fit its macroblocks and then extrapolate this frame, then the current restored stereo pair is stored in the decoder memory, and signs of movement of all its macroblocks and a mask of movement of its boundary macroblocks, after which the current stereo pair is transferred to the format and color system required for display and then the image display device is displayed Eden. The disadvantage of the prototype is the low average value of the throughput of personnel in the processing of the personnel stereo video stream in real time, due to the use of the search algorithm for motion compensation for the predicted frames of the stereo video stream.

The objective of the technical solution is to significantly increase the average values of the throughput capabilities of stereo video codecs in the processing of stereo stereo video streams in real time, and with a slight decrease in the average values of code volumes and quality of decoded images.

The problem is solved due to the fact that in the method of motion compensation in digital dynamic video images containing the following sequence of actions: before processing the video stream, all its frame stereopairs are logically divided into reference and predicted, moreover, the reference stereopairs are arranged at a frequency of one or more predicted stereopairs, and reference stereopairs are pre-set, then rectangular decars are selected in the planes of frame images of stereopairs coordinates of the coordinate system, the axes of which are directed parallel to or along the sides of the frame rectangles of the stereo pairs, and pixels are selected as the units of measurement, during the processing of the stereo stream, image sensors are polled by the encoder to obtain each next stereo pair, then the resulting images are converted to the format and color required for processing system, after which they are saved in the encoder memory as the current frame stereo pair, then the left and right frames of the current stereo pair are logically divided into identical frames rectangular fragments, called macroblocks, and the sizes of the macroblocks are predefined, and immediately after dividing the next frame of the stereo pair into macroblocks, each macroblock is assigned the coordinates of its position in the corresponding coordinate system used, during processing of the stereo stream, the stereo indexes are monitored, in the process of processing each current stereo pair first, the stereopair index determines its belonging to the reference or predicted stereopairs, in the case of the current stereo pairs to reference stereo pairs, motion compensation in the image is used only for the right frame of the stereo pair, and after processing, this stereo pair is saved both in the encoder memory and in the memory of the decoder, if the current stereo pair belongs to the predicted stereo pairs in both its frames, motion compensation in the image is applied , when conducting motion compensation in the images, the encoder forms a forecast for motion compensation, and the decoder reconstructs the motion compensation to perform motion compensation each about the left predicted frame of the stereo pair as a sequential reference frame, in the encoder and in the decoder, the previous frame of the video sequence is selected, and for the motion compensation of each right predicted frame of the stereo pair as a parallel reference frame, both the encoder and the decoder choose parallel the left frame of the video sequence, in the process of forming the motion compensation prediction encoder, first, for each macroblock of the predicted stereo frame, a binary motion mask is built, then, according to the movement masks, all the macroblocks are assigned motion signs, according to which all macroblocks, relative to the motion, are classified into external, boundary and internal, while the macroblock’s motion sign is assigned to those macroblocks that contain only fixed pixels, the boundary macroblock’s motion sign is assigned those macroblocks that contain both moving and fixed pixels, and the attribute of movement of the internal macroblock is assigned to those macroblocks that contain only moving pixels Then, for internal and boundary macroblocks, stereopairs search for their integer displacement vectors to the best macroblock forecasts in the search areas of the reference extrapolated frames, and the order of processing the stereo pairs when searching for macroblock forecasts is arbitrary, and before the search for macroblock forecasts, the reference frames are extrapolated by their bordering pixels to sizes sufficient to carry out motion compensation using foreign displacement vectors moreover, the dimensions of the extrapolated frames are pre-set, the search algorithm for the predicted displacement vectors and the quality criterion of the search macroblock forecasts are pre-selected, when setting the positions of the search areas, the centers of these areas are combined with the centers of the corresponding macroblocks, while the sizes of the search areas are pre-set during the search for the best macroblock matches are checked only by those possible macroblock forecasts that completely fall into the format of the corresponding reference extrapolar After completing the search for macroblock forecasts for the inner and boundary macroblocks of the current stereopair, their macroblock residues are determined, and the matrices of this macroblock forecast are subtracted from the source image matrices of this macroblock to find the matrixes of macroblock residues from the source image matrices of this macroblock; the encoder’s memory stores the current encoded stereo pair, and then signs of movement of all macroblocks are prepared for further encoding, different the remaining macroblock residues, integer vectors of predicted offsets and motion masks of the boundary macroblocks of the current stereopair, during the motion compensation reconstruction decoder, restore the frames of the current stereopair, first restore the left and then right frame of the stereopair when restoring the left frame from the decoded predicted macroblock displacement vectors This frame determines the position of macroblock forecasts of internal and boundary macroblocks in the previous reference extrapoli frame of the left stream, and before determining the position of macroblock forecasts to the specified reference frame according to the same rules and using the same parameters as in the encoder, the frame is extrapolated, after determining the positions of macroblock forecasts of the left frame, the current left frame of the stereo pair is restored, in the process to restore a given stereo pair frame to find the source image matrices of each internal macroblock add up the corresponding image matrices of its specific macroblock of the prediction and image matrix of its decoded macroblock remainder, to find the source image matrices of each boundary macroblock, add the corresponding image matrices of its specific macroblock forecast and image matrix of its decoded macroblock remainder, then the fixed pixel values of the boundary macroblock are loaded with the corresponding pixel values of the corresponding reference extrapolated frame without offset , to find the original image matrices of each of the external macroblock, their values are loaded with the values of the corresponding matrices of the corresponding reference extrapolated frame without offset, then the right frame of the stereo pair is restored in the same way, and immediately before its restoration to the parallel reference frame for it and the already decoded left frame according to the same rules and using the same parameters as and in the encoder, an extrapolation of this frame is used, then the current restored stereo pair is stored in the decoder memory, after which the current stereo the dough is transferred to the format and color system required for display and then displayed on the screen of the image reproducing device; The following differences are provided: during the video codec motion compensation in stereo pairs of the video stream, a combination of the search algorithm for compensating the movement of frame fragments using flat frame projections of a dynamically tracked and periodically approximately geometrically reconstructed real imprinted spatial surface, as well as approximating the motion of frame fragment projections by approximate physical laws, are used movements of real captured moving objects For the corresponding projections, for this, before operating the video codec, they check for the presence of a timer in the encoder, and in the event of its initial absence, an additional timer is added to the encoder, in the process of processing a stereo stereo stream, immediately after each successive stereo pair is received, the encoder is polled a timer, after which, according to a timer survey, a real-time mark is assigned to the stereo pair, while the time units are pre-set, and after receiving the time stamp of the current stereo pairs its value is compared with the time stamp value of the previous stereopair, and in case of equality, according to the results of comparing the values of these time stamps, the correction value of the time stamps of stereopairs is added to the value of the time stamp of the current stereopair, the value of which is pre-set with a positive number less than or equal to the selected unit of measurement time, in the process of processing a stereo stream for each reference stereo pair, the encoder predicts motion compensation according to the same rule and using the same parameters as in the processing of the predicted stereo pair, but the result data of this forecast is not used for encoding and transmission to the decoder, in addition to the data of the right frame prediction result, for the inner and boundary macroblocks of the left frame of each stereo pair in the process of motion compensation, the encoder is additionally searched parallel displacement vectors, while the parallel right frame of the video sequence is selected as a parallel reference frame, and the search algorithm for the best offsets sets the anal In accordance with the search algorithm for vectors of parallel macroblock forecasts of the right frame, in addition to the left and right frame streams, stereo pairs additionally introduce an artificial central frame stream, each frame of an artificial central stream is logically divided into fragments, the number and relative position of which corresponds to the number and relative position of macroblocks of the corresponding left frame , for the fragments of the central frame, the same characteristics are fixed as for the macroblocks of the left frame, then in the plane At the ends of the frame images for all frame flows, including the artificial central one, three planar rectangular Cartesian coordinate systems are overridden, the beginnings of which for all flows, including the central one, are set by the centers of the corresponding frame rectangles, the abscissa axis is directed to the right, the ordinate axis is up, and as the unit of measurement pixels are taken along the axes, then, in addition to planar coordinate systems, three spatial rectangular Cartesian coordinate systems are introduced, the beginnings of which are for the left and right of the flows are combined with the central points of the rectangles of the working surfaces of the respective stereo pair sensors, and the origin of the artificial central frame stream is set by the middle of the segment connecting the beginnings of the left and right spatial coordinate systems, the abscissa axis is directed to the right, the ordinate axis is up, the applicate axis is directed towards the shooting scene, and as the units of measurement along the axes take meters, immediately after the introduction of spatial coordinate systems for all fragments arbitrarily set the coordination the positions of their spatial copies in the central spatial coordinate system, then, for all fragments and macroblocks, the coordinates of their positions in the corresponding planar coordinate systems are updated, and the plane coordinates of each fragment and each macroblock are set by the coordinates of one of its points, and the coordinate points of the fragments and macroblocks are assigned as so that when moving any two fragments or macroblocks until their coordinate points are aligned, a complete coincidence of the positions of the of the corresponding corresponding points of fragments or macroblocks, in the process of processing the stereo stream of the matrix of signals of fragments of the central frame, the signs of movement of fragments, as well as the movement masks of the boundary fragments are reloaded with the corresponding data of the left frame immediately after they are received, for each frame of the central video stream, sequential motion compensation is performed similar to sequential compensation movement of the predicted frames of the left stream, while as a sequential reference predictive the frames use the previous frame of the central stream, each current frame of the artificial central stream, immediately after its fragmentation, is additionally logically divided into system blocks, while for each system block there is a sign of processing a system block with two possible states corresponding to the absence of processing and the presence of processing, and the shape and size of the system units is chosen such that an integer number of fragments is contained inside the system unit and that at least three inside the system unit are present and a fragment together with their nearest previous neighbors along the coordinate axes, then three calculation fragments are selected inside each system block with the condition that the current system block contains the nearest previous neighbors of the selected calculation fragments along the coordinate axes, then for each fragment of the central frame, and also for each with a macroblock of each frame, stereo pairs fix a sign of approximation of motion with four possible states corresponding to the absence of approximation of motion, the presence of a system approximation imitation of movement, the presence of an individual approximation of the movement and the presence of a search approximation of the movement, for all stereopairs of the video stream before applying integer macroblock displacement vectors to the best predictions in the encoder, but after extrapolating the reference frames, the current captured spatial surface is reconstructed based on the dynamic recalculation of the spatial coordinates of the copy fragments image, and then, starting from the third and for all subsequent stereo pairs, motion compensation in stereopairs using analysis of the movement of fragments in the plane of the frame image of the central stream, followed by projecting the images of the fragments onto the frames of stereopairs by taking into account the current calculated spatial coordinates of the copies of the fragments, when conducting an approximate geometric reconstruction of the current imprinted spatial surface, first they perform a spatial systemic and then spatial individual approximation movement of copies of fragments in the central spatial coordinate system, and immediately before the spatial motion approximations are made, the processing signs of all system units are reset to the non-processing state, the motion approximation signs of all fragments are reset to the non-approximated motion state, the current speeds of spatial copies of external fragments are assigned zero values, and the current coordinates of spatial copies external fragments loaded with previous values, when conducting spatial systems approximating the motion within the framework of the geometric reconstruction of the current imprinted spatial surface, first identify the system blocks that do not contain external fragments, after which the processing signs of the data of the system blocks are set to the processing state and, further, for copies of the embedded fragments of each such system block for each of the three spatial coordinate axes carry out the approximation of the movement of spatial copies of fragments, while for spatial spatial approximation imatsii movement using the equation of spatial system approximation type of movement

u _s is the speed of spatial copies of fragments of the current system unit along the current coordinate axis;

t is the time;

x is the abscissa of fragments in a given planar central coordinate system;

y is the ordinate of fragments in a given planar central coordinate system;

a _{s, 1} , a _{s, 2} , a _{s, 3} - coefficients of the spatial system approximation of the movement of copies of fragments of the current system unit along the current coordinate axis;

in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for the calculations, first using the data of the two previous frames of the central stream and the data of the two previous stereo pairs, the coefficients of the spatial system approximation of motion are calculated, for which each coordinate axis compose and solve in a discrete form a system of three such equations in accordance with the three selected design fragments, then the signs of app the motion approximations of the embedded fragments of the processed system units are transferred to the state of the system approximation of motion, and then, according to the data of the current and previous frames of the central stream and the data of the current and previous stereopairs for the copies of fragments with signs of the system approximation of motion, the current spatial velocities of the movement of the copies of the fragments are calculated using the specified spatial equation system approximation of motion, and then approximation values of current spatial coordinates m the position of the copy fragments, then check the condition for a slight change in the current spatial coordinates of the copy fragments compared with the previous coordinates, and the mathematical model of this condition is pre-set, if this condition is not met, the current spatial coordinates of the copy of the fragment are reloaded with the previous values, and the sign of approximation of the movement of the corresponding fragment reset to a state of lack of approximation of motion when conducting a spatial individual For the approximate movement approximation within the framework of the geometric reconstruction of the current imprinted spatial surface, first, for internal and boundary fragments with signs of the absence of motion approximation, these signs are transferred to the state of individual approximation of motion, then for each copy of the fragment with a sign of individual approximation of motion along each of the three spatial coordinate axes spatial approximation of motion using the spatial equation th approximation of individual types of traffic

u _s is the speed of the spatial copy of the current fragment along the current coordinate axis;

t is the time;

k _s is the coefficient of the spatial individual approximation of the motion of the spatial copy of the current fragment along the current coordinate axis;

in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for calculations, first, approximation coefficients are calculated from the data of two previous frames of the central stream and the data of two previous stereo pairs, and then, according to the data of the current and previous frames and data of the current and previous stereopairs current spatial velocities and coordinates of copies of fragments, then for a spatial copy of each fragment with an individual app The motion approximations perform processing that is absolutely similar to the processing of a copy of a fragment with a sign of system approximation during spatial system approximation of motion, starting from checking for insignificance of changes in the current spatial coordinates of the copies of fragments, when performing motion compensation in a stereo pair, they first perform a planar systemic, then planar individual, and then planar search approximation of the movement of fragments in the central planar coordinate system with after by projecting images of fragments onto stereopair frames by taking into account the spatial coordinates of copies of fragments, moreover, immediately before the plane approximations of motion are processed, the processing signs of all system units are reset to the non-processed state, the signs of approximated motion of all fragments and macroblocks are reset to the non-approximated state of motion, and integer sequential and parallel displacement vectors of external fragments of the current central frame and the macrobe locks of the current stereo pair are forcibly filled with zero values, when conducting a planar system approximation of motion, first identify the system blocks that do not contain external fragments, after which the processing signs of the data of the system blocks are set to the processing state and, further, for fragments of each such system block for each of two planar coordinate axes carry out a planar approximation of motion, while for carrying out a planar system approximation of motion, use the equation e plane approximation type movement system

u _p is the speed of fragments of the current system unit along the current coordinate axis;

t is the time;

x is the abscissa of fragments in a given planar central coordinate system;

y is the ordinate of fragments in a given planar central coordinate system;

a _{p, 1} , a _{p, 2} , a _{p, 3} are the coefficients of a planar system approximation of the movement of fragments of the current system block along the current coordinate axis;

in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for the calculations, first using the data of the previous two frames of the central stream and the data of the two previous stereo pairs, the coefficients of approximation of motion are calculated, for which, for each coordinate axis, and solve in a discrete form a system of three such equations in accordance with three selected fragments of calculation, then on each coordinate axis check the junction condition for the absolute value of the coefficient at the time derivative to prevail over the absolute values of the coefficients for spatial derivatives, the degree of prevalence and the mathematical formula for checking the coefficients are preliminarily set, then the signs of approximating the movement of nested fragments of system blocks that pass the specified test positively are transferred to the state of the system approximation of motion, and then according to the current and previous frames of the central stream and according to the current and of the previous stereopairs for fragments with signs of a systemic approximation of motion, calculate the current velocities of the fragments using the shortened equation of the planar systemic approximation of motion without terms with spatial derivatives, and then calculate the approximation values of the integer vectors of the displacements of the fragments to the best forecasts within the reference extrapolated frame, then check the hit condition found approximation forecasts in the format of the link extrapolated about the frame, as well as the condition for satisfying the forecast to the approximate value of the forecast quality criterion, the value of which is set previously, while in the case of fulfilling these conditions, a shortened search for fragmentary forecasts is organized within the framework of the extrapolated reference frame, while the centers of the search regions corresponding to the fragments are shifted along the coordinate axes by the values the corresponding approximation displacement vectors found, and the geometric dimensions of the search areas are reduced to values that pre-set, otherwise, the sign of the approximation of the movement of the fragment is reset to the state of absence of approximation, then the images of the fragments with the sign of the systemic approximation of motion are projected onto the personnel planes, while the values of the integer vectors of successive displacements of the inner and boundary macroblocks of the left frame are loaded to the best forecasts in the previous reference frame the corresponding found values of the displacement vectors of the fragments, and to calculate the values of the vectors parallel to the For the macroblocks of the left and right frames, pairs of projection macroblocks in the left and right extrapolated frames are determined, and the algorithm for projecting spatial images on the frame planes introduces the use of the directing vector of the line passing through the beginning of the corresponding lateral spatial coordinate system and the spatial position point of the corresponding copy of the fragment, and the connection of lateral planar and corresponding lateral spatial coordinate systems is set using coordinates of guiding vectors, then check the condition for the projection macroblocks of projection pairs to fall into the formats of their respective extrapolated frames, in which case the signs of the motion approximation of the corresponding macroblocks of the projection pairs are set to the state of the system approximation of motion, then, using the coordinates of the projection pairs, the vectors of parallel displacements of the macroblocks with signs system approximation of motion for the left and right frames, after which they determine the conditions for the negative predicted offsets in the abscissa of the projection macroblocks of the left frame and the conditions for the positiveness of the predicted offsets in the abscissa of projection macroblocks of the right frame, if at least one of them fails, the sign of approximation of motion of the corresponding macroblock is reset to the state of absence of approximation of motion, when conducting a planar individual approximation of motion, for internal and boundary fragments with signs of lack of approximation of motion they transfer these attributes into the state of an individual approximation of motion, then for each fragment with a sign of an individual approximation of motion, a planar approximation of motion is carried out along each of the two plane coordinate axes using the equation of a plane individual approximation of motion of the form

u _p is the speed of the current fragment along the current coordinate axis;

t is the time;

k _p is the coefficient of individual plane approximation of the movement of the current fragment along the current coordinate axis;

in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for calculations, first, approximation coefficients are calculated from the data of two previous frames of the central stream and the data of two previous stereo pairs, and then, according to the data of the current and previous frames and current and previous stereopairs data, current speeds and integer displacement vectors of fragments to approximate forecasts, then for each fragment with an individual motion approximations carry out processing that is absolutely similar to fragment processing with a system approximation sign when conducting a systematic planar motion approximation, starting from finding spatial copies of fragments on stereopair frames, when conducting a planar search approximation of motion, first, for internal and boundary fragments with signs of lack of motion approximation, they translate these signs into a state of search approximation of motion, then integer vector the predicted displacements of the fragments are found in the standard way through the organization of the search for forecasts in the previous reference extrapolated frame of the central stream, then for each fragment with the sign of the search approximation of motion, processing is carried out that is absolutely similar to the processing of the fragment with the sign of individual approximation during the individual planar approximation of motion, starting from finding the projections of spatial copies of fragments to frames of a stereo pair when performing approximations of motion exclude the occurrence of division by zero errors, when searching for macroblock forecasts, use the original values of the signals of the fixed pixels of the reference extrapolated frames, when organizing the algorithm for searching integer displacement vectors to the best forecasts, only internal and boundary macroblocks with signs of the lack of approximation of motion are processed, with the exception of internal and boundary macroblocks of the left frames for which signs of approximation of the movement of the fragments corresponding to them in the indices are not in There is no approximation of motion, and the values of successive displacement vectors of such macroblocks are loaded with the corresponding values from the vectors of these fragments, and the values of parallel displacement vectors are found through the organization of the forecast search in the reference frame, the sizes and location of the search areas of parallel macroblock forecasts are set so that the ordinates of the forecasts are are equal to the ordinates of the corresponding macroblocks themselves, and the values of the predicted abscissa shifts could only take negative values for macroblocks of the left frame and only positive values for macroblocks of the right frame, immediately after calculating the macroblock residual of the boundary macroblock, the pixel signal values of the macroblock remainder of its fixed pixels are filled with zero values, after searching for integer displacement vectors, form fractional sequential displacement vectors for all fragments of the central frame and fractional parallel displacement vectors for all macroblocks of the left and right frames, with The fractional displacement vectors of the external fragments and macroblocks are forced to be filled with zero values, and the method for calculating the values of the fractional displacement vectors for the inner and boundary fragments and macroblocks is preliminarily selected with the condition that the values of the negative values of the offset values for abscissas for the macroblocks of the left frame and the condition for the positive values of the values of the offset values for abscissas for macroblocks right frame, after the formation of fractional displacement vectors of all fragments and macroblocks for each fragment, recalculated the current values of the fragment motion velocities along the coordinate axes of the central planar coordinate system are obtained, and the current reconstructed surface is corrected by the found fractional displacement vectors of the macroblocks by recalculating the spatial coordinates and copy speed of the fragments, while the spatial coordinates of the copies of the external fragments are reloaded with the previous values, the spatial copy speeds external fragments are loaded with zero values, and to calculate the coordinates of the copy to One of the internal or boundary fragments uses the direction vectors of two intersecting straight lines passing through the beginning of the corresponding lateral spatial coordinate systems, and the coordinates of the direction vectors are pre-set through the connection of the lateral plane and corresponding lateral spatial coordinate systems, the character of which is pre-set, then using values recalculated spatial coordinates of copies of internal and boundary fragments recalculated space their speed, then the time stamps of the current and previous stereo pairs are additionally stored in the encoder memory, as well as the values of the spatial speeds and position coordinates of all copies of the fragments and the values of the plane speeds of all fragments for the current and previous stereopairs, by the decoder when restoring the original image matrices macroblocks of the current stereo pair use the original values of the signals of the fixed pixels of the reference frames.

An explanation of the need to introduce a set of distinctive features of the claimed method from the prototype method, considered in the context of methods of motion compensation in digital dynamic stereo video images and designed to build tele-systems on the basis of stereo-video codecs, one of the possible versions of which with structural schemes of tele-systems, architectures of stereo-video codecs and principles of organization of motion compensation in images are presented in the figures (FIG. 1.), (FIG. 2.), (FIG. 3.) for the prototype th method as well as the figures (fig) (Figure 5.) (Figure 6). For the proposed method is as follows. The fundamental idea of combining the basis for achieving the required technical result, in particular for a significant increase in the average value of the frame rate with a slight decrease in the average values of the code volume and the quality of the decoded image of stereo video codecs using motion compensation in digital dynamic video images when processing a frame video stream in real time, search algorithm for compensating the movement of frame fragments to obtain a planar frame O projections tracked dynamically and periodically approximately geometrically reconstructed imaged real spatial surface, and also with approximation of human motion fragmentary projections approximate physical laws of motion of the real movable objects photographed corresponding to these projections. An approximate geometric reconstruction of a real imprinted spatial surface allows a fairly high degree of accuracy to obtain projections of spatial copies of fragments on frames of stereo pairs, and, as a result, to obtain approximate values of parallel vectors of predicted displacements bypassing the complete algorithm for enumerating possible macroblock forecasts. Fragmented approximation of motion in the image plane allows one to approximately determine the positions of successive previous forecasts of a part of moving fragments of frames, and also to obtain approximation values of vectors of successive predicted displacements of a part of moving macroblocks bypassing the full enumeration algorithm. To clarify the positions of the found preliminary approximate macroblock forecasts, you can use the search algorithm for motion compensation, but with reduced sizes of the forecast search areas and the shift of the centers of these areas to the found approximation bias vectors. As a result of this, on the one hand, the average processing time of a stereo pair is reduced due to the approximation of the movement of part of the macroblocks, and on the other hand, the code size and quality of the decoded image remain at approximately the same level, since the position of the approximation forecasts is further refined by the search algorithm. This approach is especially effective when shooting and processing video scenes with real objects that have significant linear speeds and moving parallel to the plane of image sensors. For the convenience of processing a stereo video stream, an artificial central personnel stream, as well as three planar and three spatial coordinate systems, are additionally introduced. Geometric restoration of the captured spatial surface is performed for all stereopairs at the level of movement of spatial copies of fragments in the central spatial coordinate system using spatial system and spatial individual approximations of the movement of copies of fragments. Fragmental approximation of movement is carried out for all stereopairs, starting from the third, at the level of movement of fragments of the central personnel stream in the plane of the central plane coordinate system using planar systemic, individual and plane search approximation of motion. Approximations of motion are characterized by a discrete representation of formulas. Data on the nature of the movement of fragmented copies in space, as well as the fragments themselves in the plane of the frame image, make it possible, including through projections, to switch to the predicted displacement vectors of macroblock frames of a stereo pair. The transition from displacement vectors to the speeds of movement of macroblocks and vice versa is carried out using the time stamps of stereo pairs received from the timer device in real time. To increase the reliability of the results of approximation of motion, fractional values of the parameters of fragments and macroblocks are used. The principles of geometric reconstruction of the spatial surface and the principles of motion compensation in planar digital dynamic video images used in the present method are presented in the figures (FIG.7.), (FIG.8.), (FIG.9.), (FIG.10.) . Thus, the processing of each stereo pair is carried out at the level of copies of the fragments and the fragments of the central frame with the subsequent transition to the motion parameters of the macroblocks of the left and right frames. The main task of the system approximation of motion is to identify the nature of the movement of copies of fragments and the nature of the movement of the fragments themselves within the framework of the so-called system units, into which each artificial central frame is additionally logically divided, and which contain a sufficient number of moving fragments inside, the coordinates of each of which in turn, they are set by the coordinates of one of the points of the fragments, and inside each system unit three selected fragments are selected with the condition m having their nearest neighbors on the coordinate axes. In this case, the system approximation of the motion is operated on by the spatio-temporal motion parameters of the copies of the fragments and the fragments themselves and cover the dynamic data of the set of embedded fragments within the system blocks, which significantly affects the reliability of the system forecast. Individual motion approximations, in turn, operate only on the temporal parameters of the movement of copies of fragments and the fragments themselves and imply predicting the positions of fragments in the current frame solely from their own previous values of the dynamic motion data. The process of carrying out approximations of motion is governed by signs of processing system units, as well as signs of approximation of the movement of fragments of the central frame and macroblocks of the left and right frames of the stereo pair. The geometry of the current imprinted spatial surface is restored at the level of spatial copies of fragments, the coordinates of each of which are calculated based on the intersection of two straight lines passing through the beginning of the lateral spatial coordinate systems with the involvement of the direction vectors of these lines, and the search areas of parallel displacement vectors for the left and right frames are specified so that these lines necessarily have one intersection point in the front half-space with respect to to the lens. All the theoretical formulas presented below imply fixing one of the coordinate axes, as well as one of the system blocks or one of the fragments, without specifying fixed objects in the index formulas.

Spatial systemic and individual motion approximations contribute to the rapid determination of the values of the vectors of parallel predicted displacements for the macroblocks of the left and right frames of the current stereo pair, by projecting parts of the current captured spatial surface onto the frames of this stereo pair, the dynamic reconstruction and obtaining of the current form of which is carried out at the level of analysis of the movement of copies of image fragments in space. The theoretical equation of the spatial system approximation of motion is as follows:

u _s is the speed of spatial copies of fragments of the current system unit along the current coordinate axis;

t is the time;

x is the abscissa of fragments in a given planar central coordinate system;

y is the ordinate of fragments in a given planar central coordinate system;

a _{s, 1} , a _{s, 2} , a _{s, 3} - coefficients of the spatial system approximation of the movement of copies of fragments of the current system unit along the current coordinate axis;

it can be obtained from the equation of mathematical physics, known as the equation of oscillation of the membrane, if you first agree that the input action is constant for a certain short period of time and on the entire working surface, then both parts of this equation are divided into a constant input effect, and, then, introduce new notation, designating fractions in front of derivatives as new coefficients, and replace second-order derivatives themselves with first-order derivatives, using instead Nij speed. It is assumed that during the system approximation of the movement in the current stereopair for copies of fragments of each system unit along the current spatial coordinate axis, the input actions on copies of fragments of the current system unit are considered constant for all copies of nested fragments and over a time period spanning three consecutive stereopairs, including the current and two previous ones. When conducting a spatial system approximation of motion in a stereo pair for copies of fragments of each processed system unit for each of the coordinate axes, first, based on the data of two previous frames of the central stream and the data of two previous stereo pairs, the coefficients of approximation of movement are calculated, and then according to the current and previous frames of the central stream and according to the current and previous stereo pairs, the current positions and speeds of copies of fragments of processed system units are calculated at. The calculation is carried out using the static and dynamic parameters of the copies of the three calculated fragments, using the static and dynamic parameters of the three calculated fragments themselves of the current system unit, as well as taking into account the static and dynamic parameters of their neighbors along the coordinate axes of the plane central coordinate system. In this case, the values of the orders of derivatives of the system can be controlled by the choice of a unit of time. Units of time can be selected empirically and chosen so that all derivatives of the final theoretical equations would have approximately the same order. The algorithm for solving systems of equations can be chosen in such a way as to ensure the speediest solution of the systems together with high precision solutions. The theoretical equation of the spatial individual approximation of motion is as follows:

u _s is the speed of the spatial copy of the current fragment along the current coordinate axis;

t is the time;

k _s is the coefficient of the spatial individual approximation of the motion of the spatial copy of the current fragment along the current coordinate axis;

it can be obtained from the equation of Newton’s second law if it is previously agreed that the input action is constant for a certain short period of time, then both parts of this equation are divided into a constant input effect, and then new designations are introduced, designating the fraction in front of the derivative as new coefficient, and the second-order derivative itself, replace the first-order derivative, using speed instead of position. It is assumed that when conducting a spatial individual approximation of the motion, the input actions for a copy of each fragment along the current coordinate axis are considered constant over a time period spanning three consecutive stereo pairs, including the current and two previous ones. Next, the motion approximation coefficients are calculated first, and then the current coordinates and motion speeds of the copy fragments, similar to fragments of the system approximation.

Plane system, individual, and search approximations of the movement facilitate the rapid determination of the values of the successive forecast displacement vectors for the macroblocks of the left frame of the current stereopair by approximating the movement of fragments in the plane of the frame image of the central frame, followed by finding the values of the successive displacement vectors of part of the moving fragments for the frame of the artificial central stream and at the same time for macroblocks of the left frame of the current stereo pair. The theoretical equation of the planar system approximation of motion is as follows:

u _p is the speed of fragments of the current system unit along the current coordinate axis;

t is the time;

x is the abscissa of fragments in a given planar central coordinate system;

y is the ordinate of fragments in a given planar central coordinate system;

a _{p, 1} , a _{p, 2} , a _{p, 3} are the coefficients of a planar system approximation of the movement of fragments of the current system block along the current coordinate axis;

the mechanism of its obtaining and theoretical calculation assumptions are similar to the mechanism of obtaining and the assumptions of the equations of the spatial system approximation of motion. We will conventionally call a _{p, 1 the} corpuscular coefficient, and a _{p, 2} and a _{p, 3 the} wave coefficients of the theoretical equation of the planar system approximation of motion. Then the task of a planar systematic approximation of motion is to identify those fragments of projections of captured objects or their parts that have much more corpuscular properties than wave ones, that is, when moving, they behave as monolithic objects that do not change their geometric shape. When carrying out a planar system approximation of motion for each processed system unit along each of the coordinate axes, first, according to the data of two previous frames of the central stream and data of two previous stereo pairs, the coefficients of approximation of motion are calculated, then the conditions for the values of the found coefficients are checked, and then according to the data of the current and previous frames the central stream and the current and previous stereo pairs, the current speeds of the fragments of the processed system are calculated blocks. To calculate the coefficients of a planar systemic approximation of motion, a system of three equations of the indicated type is initially compiled and solved in a discrete form, the mechanism for solving systems is similar to the mechanism for solving systems of spatial systemic approximation of motion. From a mathematical point of view, the preference in favor of corpuscular coefficients means the prevalence of their absolute values over the absolute values of wave coefficients, so in practice it is convenient to use the conjunctive condition for the prevalence of the absolute value of the coefficient in the time derivative over the absolute values of the coefficients in the case of spatial derivatives, with the degree of prevalence and the mathematical formula for checking the coefficients pre-set. Units of time can be selected empirically and chosen so that all derivatives of the final theoretical equations would have approximately the same order. The algorithm for solving systems of equations can be chosen in such a way as to ensure the speediest solution of the systems together with high precision solutions. The smallness of the absolute values of the wave coefficients in comparison with the value of the corpuscular coefficient of the final theoretical equation of the systemic planar approximation of motion, with approximately the same order of all derivatives, means that the two terms in this equation can be neglected. For this reason, the current speeds of movement of fragments along the coordinate axes are calculated using the presented systemic plane approximation of motion in discrete form, in which the current speed of the macroblock is taken as an unknown quantity, while the terms with spatial derivatives in the second and third are previously excluded from this equation ratios. The theoretical equation for an individual approximation of motion is as follows:

u _p is the speed of the current fragment along the current coordinate axis;

t is the time;

k _p is the coefficient of individual plane approximation of the movement of the current fragment along the current coordinate axis; the mechanism of its obtaining and theoretical calculation assumptions are similar to the mechanism of obtaining and assumptions of the equation of spatial individual approximation of motion. When conducting a plane individual approximation of motion for each moving fragment along each of the coordinate axes of a plane central coordinate system, first, according to the data of two previous frames of the central stream and the data of two previous stereo pairs, the coefficient of individual approximation of movement is calculated, and then according to the data of the current and previous frames of the central stream and data current and previous stereo pairs, the current fragment speed is calculated. Moreover, to calculate the coefficient of the planar individual approximation of motion along any of the coordinate axes, one specified equation is initially compiled and solved in discrete form, in which the coefficient of individual approximation of motion is taken as an unknown value. The current speeds of the fragment along the coordinate axes are calculated using the discrete individual plane approximation of the motion in which the current speed of the macroblock is taken as unknown. A planar search approximation of the movement helps to obtain both successive predicted displacement vectors and parallel displacement vectors of the macroblocks of the left and right frames, but it has the least efficiency, since the data are obtained by applying the standard search algorithm to moving fragments of the central frame, with the subsequent transition to the calculation of parameters macroblocks of the current stereo pair. Summing up all of the above, we can briefly highlight the following main points for the proposed method: restoration of the geometry of the current imprinted spatial surface facilitates the quick indirect obtaining of values of parallel displacement vectors for macroblocks of the left and right frames of the stereo pair through the analysis of the movement of copies of fragments of the artificial central frame in space, and the approximation of the movement of parts of the image laws similar to the real physical laws of nature, promotes rapid obtaining the values of successive displacement vectors through the analysis of the motion of fragments of the artificial central frame in the plane of the frame image. Thus, in the inventive method of motion compensation, the same set of input and output encoding-decoding parameters appears as in the prototype method, but with a different mechanism for their production, and the specified technical result is achieved mainly by shifting and reducing the search areas for forecasts for macroblocks with signs of an approximation of motion.

A device for implementing the proposed method of motion compensation in digital dynamic stereo video images consists of a SAMSUNG R530 laptop, a pair of hama AC-150 digital web cameras, a stand for web cameras, a pendulum, a connecting thread, a tripod, a light source, a splitter, a source electricity, film set. A pair of digital webcams 2 is connected to laptop 1, located on a stand 3, designed for shooting a pendulum 4 attached by a connecting thread 5 to a tripod 6. The pendulum is illuminated by a light source 7. A laptop and a light source are connected to splitter 8, and it is connected to electricity source 9. All of the above items are located on the set 10. The laptop and the light source are on, and the laptop is loaded with software for comparative analysis of proto typical and proposed methods of motion compensation in digital dynamic stereo video. All parameters and technical characteristics of the above structural elements, the parameters of the stereo video codec model based on the proposed method, as well as the parameters of the compared models of stereo video codecs that implement the prototype and claimed methods of motion compensation in digital dynamic stereo video images, all other things being equal, are presented on (FIG. 11 .) and (FIG. 12.), as well as in the tables (Table 1) and (Table 2). The laptop screen resolution allows you to display graphic results of experiments on the screen, and its clock frequency allows you to compare stereo video codec models taking into account real-time operation, as well as save and subsequently display the results of a comparative analysis of video codec models in the process of processing a stereo video stream. The laptop is equipped with software that allows you to implement the inventive method separately, as well as carry out an experiment to conduct a comparative analysis of the models of stereo video codecs based on the prototype and the inventive methods. In an experiment, a video sequence of one hundred stereo pairs is processed by comparing video codec models. In this case, the video codec model based on the prototype method processes the experimental video sequence without involving time stamps of stereo pairs, and the video codec model based on the proposed method processes the experimental video sequence using time stamps of stereo pairs. The experiment is organized as follows: first, a stereo video sequence is recorded and processed on-line by a model of a stereo video codec based on a prototype method, while in the laptop’s memory

Table 1 Parameters of the equipment of the experimental setup. Item No. Name Main characteristics one Notebook "SAMSUNG R530" Resolution 1024 × 768 pixels, clock frequency 2.0 GHz, the availability of software for comparative analysis of models of stereo video codecs based on the prototype and the proposed methods 2 A pair of hama AC-150 digital webcams Resolution 640 × 480 pixels, the distance between the centers of the rectangles of the image sensors is 0.06 m, the lenses are parallel to the O ₂ , O ₃ plane at a distance of approximately b = 0.6 m from the center of gravity of the pendulum when the pendulum is in a free position and at a height from the top stand surface d = 0.05 m. 3 Stand for a pair of web cameras Height e = 0.1 m four Pendulum Weight 0.350 kg, diameter 0.2 m 5 Connecting thread Length 1.5 m, diameter 0.001 m, the initial angle of deviation from the axis O ₂ a = 15 ° 6 Tripod Limits all linear movements and angular rotations of the pendulum, except for rotation around the axis O ₁ 7 Light source Power 100 W, stable lighting, located on the set so that the light flux is directed to the pendulum 8 Splitter 3 connectors 9 Source of electricity Voltage 220 V, frequency 50 Hz 10 Set The size and shape are sufficient to install the equipment, the color properties of the background differ from the color properties of the pendulum

table 2 parameters of the compared models of stereo video codecs. Block diagram of a digital television system based on a video codec using a prototype method (see FIG. 1) The architecture of the video codec using the prototype method (see FIG. 2) The principle of motion compensation in frames of stereo pairs based on the prototype method (see FIG. 3) The structural diagram of a digital television system based on a video codec using the proposed method (see FIG. 4) The architecture of the video codec using the proposed method (see FIG. 5) The principle of motion compensation in frames of stereo pairs based on the proposed method (see FIG.6) The number of compared video codec models 2 models Number of experiments 1 experiment The number of experimental stereo pairs 100 stereo pairs Reference Stereopairs Following Period 10 stereo pairs Distance between centers of image sensors 0.06 m Input Frame Format 640 × 480 pixels Input frame color system R, G, B Working frame format 80 × 64 pixels Extrapolated Frame Format 88 × 72 pixels Work Frame Color System Y, Cr, Cb Sampling Format 4: 4: 4 Output frame format 400 × 300 pixels Input frame color system R, G, B Processing time of any stereo pair 0.3 s (no more) Temporary mismatch between receiving stereo pair frames 0.03 s (no more) Time unit 0.1 s Timestamp Correction Value 0.01 s Dimension of system units 4 × 4 fragments Dimension of fragments and macroblocks 4 × 4 pixels Integer macroblock prediction search algorithm Full search Algorithm for calculating fractional fragmentary and macroblock forecasts In neighboring positions regarding the best forecast The size of the standard search area for sequential macroblock prediction 16 × 16 pixels The size of the standard parallel macroblock prediction search area 16 × 4 pixels The size of the approximation area of the search for sequential macroblock forecast 6 × 6 pixels The size of the approximation region of the search for parallel macroblock forecast 8 × 4 pixels Search criteria for the best block forecasts Minimum MAE (avg abs abs e) (relative unit brightness) ([0..255]) Time measuring tool Monitor counter (temporary resolution << 1 ms) The method of geometric reconstruction of the captured spatial surface (see FIG. 7) Coordinate systems (see FIG. 8) Fragment Coordinate Points (see FIG. 9) The position of the fragments in the system units (see FIG. 10) The condition for the relations of the absolute values of the coefficients of a planar system approximation of motion (| a _{p, 2} | / | a _{p, 1} | <arg) & (| a _{p, 3} | / | a _{p, 1} | <arg) a _{p, 1} , a _{p, 2} , a _{p, 3} - coefficients, arg - coefficient ratios regulator The threshold for changing the spatial coordinate of a copy of a fragment during the reconstruction of the current spatial surface 10% of the initial value The threshold level of the ratio of the coefficients of approximation of motion 0.1 (no more) Threshold level of the criterion for the quality of a planar approximation forecast 4 units MAY, (no more) The quantization-dequantization step of the coefficients four The principle of allocation of numerical series of variable length “Series, meaning, end” The principle of entropy coding - decoding of numerical series and displacement vectors Tabular The principle of arithmetic coding motion mask Context sensitive

the processed video sequence and time stamps of the stereopairs are stored, after which the saved video sequence is processed off-line by a model of a stereo video codec based on the proposed method. The algorithm for solving a system of linear algebraic equations is based on the search for a basic minor, drawing up, if necessary, a new system of equations of a lower order based on the found minor, and solving the final system using Cramer's formulas. Coefficients, the factors at which do not participate in the formation of the basic minor, are assigned zero values. The formation of the fractional components of the displacement vectors of fragments and macroblocks is carried out for each coordinate axis according to the existing positions with different values of the prediction quality criterion existing within the framework of the previous extrapolated frame and neighboring, relative to the found best integer prediction, while the sign of the fractional part is determined by the need to shift towards the neighboring position with lower value of the forecast quality, and the absolute value of the fractional part is defined as the difference between the unit and the ratio of a lower value of the quality criterion of one of the neighboring forecasts to a larger value of the quality criterion of another neighboring forecast. For the input stereo video stream, one of the standard resolutions of web-cameras is used. The mutual arrangement of web-cameras, web-cameras and the pendulum, taking into account the height of the stand for web-cameras and the distance from them to the pendulum, is chosen so that during the experiments the image of the pendulum occupies significant parts of the frames of stereo pairs. The length of the connecting thread is chosen so as to provide a period of oscillation of the pendulum, in which the positions of the images of the pendulum on two frames adjacent in time differ slightly, that is, by several pixels, and the movement of the pendulum in short sections of the trajectory can be considered as a linear motion with accelerations, which the queue allows the approximation of the movement of fragments of the image of the pendulum using a mathematical model of the proposed method. The power of the light source is sufficient to detect differences in the color properties of the pendulum and the background of the video scene. The method of motion compensation described above in digital dynamic stereo-video images is as follows: the pendulum is deflected by an angle a = 15 ° from the free axis of rotation, after which it is released. After 5-10 seconds, the laptop starts the software version for the implementation of the proposed method individually or an experiment to conduct a comparative analysis of the models of stereo video codecs based on the prototype and the claimed methods by pressing the corresponding button. After that, they expect the end of processing the stereo video stream and displaying the results of the implementation of the proposed method separately, or the results of an experiment comparing models of stereo video codecs using the prototype and the claimed methods. After this, the pendulum is stopped, that is, it is brought back to the initial initial free position.

The presence of a causal relationship between the totality of the essential features of the claimed object and the achieved technical result is shown in table 3. Tabular data are based on the results of comparative experiments

Table 3 A causal relationship between the totality of the essential features of the claimed object and the achieved technical result. Types of technical result and their dimension Method indicators, actual or estimated prototype claimed Bandwidth of the model of a stereo video codec based on the method, f, stereo pair / s 5.9 6.6 The total processing time of the video stream spent by the stereo video codec model based on the method, T, s 16.8 15,2 The total amount of video stream code obtained by the stereo video codec model based on the method, V, bits 5105920 5182508 Image quality error, defined as the average absolute deviation of the signals of the encoded and decoded video streams from the numerical range [0..255], accumulated by the stereo video codec model based on the method, E, rel. units brightness 4,53 4,56 The quality of the motion approximation, determined by the total number of moving macroblocks with signs of the lack of motion approximation when processing a video stream by a model of a stereo video codec based on the method, A, pcs. 32435 22654

analysis of models of stereo video codecs based on the prototype and the proposed methods. In more detail, the results of a comparative analysis of two models of stereo video codecs based on the prototype and the proposed methods are presented in the figures (FIG.13.), (FIG.14.), (FIG.15.), (FIG.16.), And also in the figures ( FIG. 17.), (FIG. 18.), (FIG. 19.), (FIG. 20.). Also, experiments were conducted with the operation of both models of video codecs in off-line mode, the stereo video sequence for which was formed using the FUJIFILM finepix REAL 3D W3 digital stereo camera shown in the figure (FIG. 21) and used in the video mode with a constant known time step between stereo pairs with ensuring minimization of the temporal mismatch of receiving frames of a stereo pair in comparison with the use of separate web-cameras, while the stereo-video images received by him were stored in the device memory Twa in its original form, that is, without the use of any mechanisms for precoding stereo video images. Further, the obtained stereo-video images were copied to the laptop memory and processed off-line by the models of the compared stereo-video codecs when processing file data and using known values of the time intervals between stereo pairs. In this case, there was a slight improvement in the quality of the approximation of motion in the images and the throughput of the stereo video codec. In addition, experiments were carried out with the change of the off-line and on-line modes for video codec models, as well as experiments using the algorithms of shortened search for macroblock forecasts in models of video codecs, such as the “three-step search algorithm” and the “nearest neighbor search algorithm” . The results of these experiments are generally identical to the results of experiments using the full enumeration algorithm, but with a less pronounced effect of improving the personnel throughput. According to experimental data, the use of the exhaustive search algorithm improves the throughput of the stereo video codec by 5-10%, and the use of shortened search algorithms gives the improvement of the throughput of the stereo video codec by 4-8%. Given the results of experimental data comparing models of stereo video codecs based on the prototype and the claimed methods of motion compensation in digital dynamic stereo video images, we can draw the following conclusion regarding the proposed method: the proposed technical solution provides a significant increase in the average throughput of the stereo video codec when processing the stereo video stream in real time with little lower mean code size and quality and the decoded image.

The technical nature of the claimed technical solution is illustrated by the following additional materials.

FIG. 1. The structural diagram of a digital television system based on a stereo video codec that implements the prototype method.

FIG. 2. The principles of organizing the structure of the stereo video codec and the processing of the stereo stream using the prototype method.

FIG. 3. The principle of motion compensation in frames of stereo pairs based on the prototype method.

FIG. 4. The structural diagram of a digital television system based on a stereo video codec that implements the inventive method.

FIG. 5. The principles of organizing the structure of the stereo video codec and the processing of the stereo stream using the proposed method.

FIG.6. The principle of motion compensation in frames of stereo pairs based on the proposed method.

FIG. 7. General principles of geometric reconstruction of a captured spatial surface and obtaining its projections onto stereo pair frames.

FIG. 8. The logical partition of the artificial central frame into system blocks and fragments in the present method.

FIG. 9. The principle of setting the coordinates of fragments and macroblocks in the present method.

FIG. 10. One of the options for the selection of fragments within the system unit in the inventive method.

FIG. 11. A snapshot of the experimental setup for comparing the performance of stereo video on the bases of the prototype and the proposed methods.

FIG. 12. The experimental setup for comparing the performance of stereo video on the basis of the prototype and the proposed methods.

FIG.13. A picture of an encoded stereo pair using the proposed method.

FIG. 14. A snapshot of the decoded stereo pair using the proposed method.

FIG. 15. A snapshot of a binary mask of a stereo pair using the proposed method.

FIG.16 A snapshot of the color gradation of the macroblocks of a stereo pair using the proposed method according to the signs of approximation of motion with the notation: gray — lack of approximation of motion, green — presence of a systematic approximation of motion, red — presence of an individual approximation of motion, blue — presence of a search approximation of motion.

FIG. 17. A graph of the relationship of the processing time of a stereo pair using the proposed method to the processing time of a stereo pair using the prototype method from the number of the stereo pair.

FIG. 18. The dependence of the ratio of the volume of the stereo code using the proposed method to the code volume of the stereo using the prototype method from the stereo number.

FIG. 19. The graph of the dependence of the image quality error, calculated as the ratio of the average absolute deviation of the signals of the encoded and decoded stereo pairs when processing the experimental video sequence using the proposed method to the average absolute deviation of the signals of the encoded and decoded stereo pairs when processing the experimental video sequence using the prototype method, from the number of the stereo pair.

FIG. 20. A graph of the dependence of the quality of the approximation of motion, calculated as the ratio of the number of moving macroblocks with signs of lack of approximation of motion when processing an experimental video sequence using the proposed method to the number of moving macroblocks with signs of lack of approximation of motion when processing an experimental video sequence, using the prototype method, from the number of a stereo pair.

FIG. 21. A snapshot of the FUJIFILM finepix REAL 3D W3 stereo camera used in the video mode for comparative analysis of stereo video codec models on the basis of the prototype and the proposed methods in off-line mode with a high degree of synchronization in time of receiving frames of stereo pairs, as well as withstanding predetermined time intervals between receiving stereo pairs and storing stereo video in its original form, that is, without the use of any precoding mechanisms for the resulting video information and.

The feasibility study of the proposed method of motion compensation in digital dynamic stereo video images, intended for use in stereo video codecs of digital television systems operating in real time, is as follows. Firstly, to obtain economic benefits, you can use the initial parameters of the achieved technical result in their pure form. In other words, the increased throughput of the system while maintaining the volume of the transmitted code and the quality of the decoded image at the same level smooths the sampling of the stereo stream and, as a result, improves the visual effect of viewing video images, which in turn makes such a television system preferable from the point of view of the consumer. Secondly, to obtain economic benefits, you can use the conversion of the parameters of the achieved technical result. For example, the gain in bandwidth can be converted into a reduction in the cost of the digital computing devices that make up the television system. Such a television system will work with the same throughput capacity as its existing counterparts, but the cost of such a system will be cheaper, which in turn makes such a television system more preferable from the point of view of the manufacturer. Finally, to obtain economic benefits, you can use the inventive method not only in the field of digital television broadcasting, but also in other technical areas related to the processing and transmission of digital dynamic stereo video. An example is real-time online communication of users using stereo web-cameras. To date, to obtain a higher bandwidth of such systems, stereo web-cameras are used with integrated microprocessor devices working in parallel and in coordination with the central processing units of computers. The cost of such stereo web-cameras is ten times higher than the cost of conventional stereo web-cameras. From this point of view, the use of a conventional stereo web camera complete with software for implementing the inventive method looks more preferable than the use of a stereo web camera with an integrated microprocessor device.

Claims (2)

1. The method of motion compensation in digital dynamic stereo-video images, which consists in the following, before processing the video stream, all its frame stereopairs are logically divided into reference and predicted, moreover, the reference stereopairs are arranged at a frequency of one or more predicted stereopairs, and the reference stereopairs themselves must be set in advance, then, in the planes of frame images of stereo pairs, rectangular Cartesian coordinate systems are selected whose axes are directed parallel or along a hundred it is the frame rectangles of stereo pairs, and pixels are selected as the units of measurement along the axes, during the processing of the stereo stream, image sensors are interviewed by the encoder to receive each next stereo pair, then the resulting images are converted to the format and color system required for processing, and then stored in the encoder memory as the current frame stereo pair, then the left and right frames of the current stereo pair logically divide into rectangular fragments of the same size, called macroblocks, and macroblock zooms are pre-set, and immediately after dividing the next frame of the stereo pair into macroblocks, each macroblock is assigned the coordinates of its position in the corresponding coordinate system used, during processing of the stereo stream, the indices of stereo pairs are tracked, during the processing of each current stereo pair, it is first determined by reference to the stereo pair that it belongs to the reference or predicted stereo pairs, if the current stereo pair belongs to reference stereo pairs, I use motion compensation in the image t only for the right frame of the stereo pair and after processing, this stereo pair is saved both in the encoder memory and in the decoder memory, if the current stereo pair belongs to the predicted stereo pairs in both of its frames, motion compensation in the image is applied, when conducting motion compensation in the images, the encoder generates a forecast motion compensation, and the decoder reconstructs the motion compensation, to carry out the motion compensation of each left predicted frame of the stereo pair as a series of of the reference frame in both the encoder and the decoder, the previous frame of the video sequence is selected, and for performing motion compensation of each right predicted frame of the stereo pair as a parallel reference frame, both the encoder and the decoder select the parallel left frame of the video sequence, in the process of generating the motion compensation prediction encoder for each macroblock of the predicted frame, stereopairs construct a binary motion mask, then, according to the motion masks, for all macroblocks, signs of movement are fixed I, according to which all macroblocks, with respect to movement, are classified into external, boundary and internal, while the sign of movement of the external macroblock is assigned to those macroblocks that contain only fixed pixels, the sign of movement of the boundary macroblock is assigned to those macroblocks that contain both moving and fixed pixels, and the sign of movement of the internal macroblock is assigned to those macroblocks that contain only moving pixels, then for the internal and boundary macroblocks stereopairs are searched x integer displacement vectors to the best macroblock forecasts in the search areas of the link extrapolated frames, and the order of processing the stereopair frames when searching for macroblock forecasts is arbitrarily set, and before searching for macroblock forecasts, the reference frames are extrapolated from their bordering pixels to sizes sufficient to compensate for the motion using foreign displacement vectors, while the dimensions of the extrapolated frames are pre-set, algorithm m search of predicted displacement vectors and the quality criterion of search macroblock forecasts are pre-selected, when setting the positions of the search areas, the centers of these areas are combined with the centers of the corresponding macroblocks, while the dimensions of the search areas are pre-set, while searching for the best macroblock matches, only those possible macroblock forecasts are checked that completely fall into the format of the corresponding reference extrapolated frame of the stereo pair, after the search for macroblock forecasts for I macroblock residues determine the internal and boundary macroblocks of the current stereopair, and to find the matrices of the macroblock remainder of any of the indicated macroblocks, the corresponding matrices of its found macroblock prediction are subtracted from the source image matrices of the macroblock, then the current encoded stereo pair is stored in the encoder memory, after which prepare for further coding the motion signs of all macroblocks, differential macroblock residues, integer vectors of predicted displacements and motion patterns of the boundary macroblocks of the current stereo pair, during the motion compensation reconstruction decoder, restore the frames of the current stereo pair, first restore the left and then the right frame of the stereo pair, when restoring the left frame using the decoded predicted displacement vectors of the macroblocks of this frame, determine the positions of macroblock forecasts of internal and boundary macroblocks in the previous reference extrapolated frame of the left stream, and before determining the position of macroblock predictions to the specified reference frame according to the same rules and using the same parameters as in the encoder, apply frame extrapolation, after determining the positions of macroblock forecasts of the left frame, the current left frame of the stereo pair is restored, in the process of restoring this frame of the stereo pair to find the original image matrix of each the internal macroblock add the corresponding image matrix of its specific macroblock forecast and image matrix of its decoded macroblock remainder, To find the source image matrices of each boundary macroblock, add the corresponding image matrices of its specific macroblock forecast and the image matrix of its decoded macroblock remainder, after which the fixed pixel values of the boundary macroblock are loaded with the corresponding pixel values of the corresponding reference extrapolated frame without offset to find the source image matrices of each external macroblock their values are loaded with the values of the corresponding matrices c of the corresponding reference extrapolated frame without offset, then in the same way the right frame of the stereo pair is restored, and immediately before its restoration to the parallel reference frame for it and the already decoded left frame according to the same rules and using the same parameters as in the encoder, an extrapolation of this frame, then the current restored stereo pair is stored in the decoder memory, after which the current stereo pair is transferred to the format and color system required for display and then displayed the image reproducing device is displayed on the screen, characterized in that in the process of the video codec performing motion compensation in stereo pairs of the video stream, a combination of the search algorithm for compensating the movement of frame fragments is used to obtain planar frame projections of a dynamically monitored and periodically approximately geometrically reconstructed real imprinted spatial surface, as well as with an approximation of motion frame fragmentary projections approximate physical laws move In order to do this, before operating the video codec, they check for the presence of a timer in the encoder, and in the case of its initial absence, an additional timer is added to the encoder, in the process of processing the frame stereo stream, immediately after each successive stereopairs, the encoder polls the timer, after which, according to the polling of the timer, a real-time mark is assigned to the stereopair, while the units of time are pre-set, and after the time stamp of the current stereopair, its value is compared with the value of the time stamp of the previous stereopair, and in case of equality, according to the results of comparing the values of these time stamps, the correction value of the time stamps of the stereopairs is added to the value of the time stamp of the current stereopair, the value of which is preliminarily set by a positive number less or equal to the selected unit of time, in the process of processing the stereo stream for each reference stereo pair, the encoder performs prog Compensating for motion according to the same rules and using the same parameters as when processing the predicted stereo pair, but the data of the result of this forecast are not used for encoding and transmission to the decoder, except for the data of the result of prediction of the right frame for the inner and boundary macroblocks of the left frame of each stereo pair in in the process of motion compensation, the encoder additionally searches for parallel displacement vectors, while the parallel right frame of the video sequence is selected as a parallel reference frame, and al The search algorithm for the best offsets is set similar to the search algorithm for vectors of parallel macroblock forecasts of the right frame, in addition to the left and right frame streams of the stereo pair, an artificial central frame stream is additionally introduced, each frame of the artificial central stream is logically divided into fragments, the number and relative position of which correspond to the number and relative the location of the macroblocks of the corresponding left frame, the same characteristics are assigned to the fragments of the central frame as behind the macroblocks of the left frame, then in the planes of frame images for all frame flows, including the artificial central one, three plane rectangular Cartesian coordinate systems are restarted, the beginnings of which for all flows, including the central one, are set by the centers of the corresponding frame rectangles, the abscissa axes are directed to the right, the ordinate axes up, and as the units of measurement along the axes take pixels, then in addition to the planar coordinate systems, three spatial rectangular Cartesian systems are introduced coordinates, the beginnings of which for the left and right flows are combined with the central points of the rectangles of the working surfaces of the respective stereo pair sensors, and the origin of the coordinates of the artificial central frame stream is set by the middle of the segment connecting the beginnings of the left and right spatial coordinate systems, the abscissa axis is directed to the right, the ordinate axis are up, the axis the applicate is directed towards the shooting scene, and as the units of measurement along the axes take meters, immediately after the introduction of spatial coordinate systems for these fragments are arbitrarily set the coordinates of the position of their spatial copies in the central spatial coordinate system, then for all fragments and macroblocks the coordinates of their position in the corresponding planar coordinate systems are updated, the plane coordinates of each fragment and each macroblock are set by the coordinates of one of its points, and the coordinate points of the fragments and macroblocks are assigned so that when moving any two fragments or macroblocks to align their coordinate points on full coincidence of the positions of the remaining corresponding points of fragments or macroblocks would be observed, during the processing of the stereo stream of the signal matrix of the fragments of the central frame, the signs of movement of the fragments, as well as the movement masks of the boundary fragments are reloaded with the corresponding data of the left frame immediately after they are received, sequential compensation is carried out for each frame of the central video stream motion similar to sequential compensation of the motion of the predicted frames of the left stream, while as successive reference forecast frame use the previous frame of the central stream, each current frame of the artificial central stream, immediately after its fragmentation, is additionally logically divided into system blocks, while for each system block there is a sign of processing a system block with two possible states corresponding to the absence of processing and the presence of processing, and the shape and size of the system units is chosen such that inside the system unit contains an integer number of fragments and so that in the morning of the system unit, there were at least three fragments together with their nearest previous neighbors along the coordinate axes, then three design fragments are selected inside each system unit with the condition that the current system unit contains the nearest previous neighbors of the selected design fragments along the coordinate axes, then behind each with a fragment of the central frame, as well as with each macroblock of each frame, stereo pairs attach a sign of approximation of motion with four possible states corresponding to the absence of motion simulations, the presence of a systematic motion approximation, the presence of an individual motion approximation, and the presence of a search motion approximation, for all stereopairs of the video stream before applying integer macroblock displacement vectors to the best predictions in the encoder, but after extrapolating the reference frames, the current captured spatial surface is reconstructed based on the dynamic recalculation of spatial coordinates of copies of image fragments, and then starting from the third and for all x subsequent stereopairs, motion compensation in stereopairs using analysis of the movement of fragments in the plane of the frame image of the central stream with subsequent projection of fragments images onto stereopairs frames by taking into account the current calculated spatial coordinates of the copies of the fragments, when performing an approximate geometric reconstruction of the current imprinted spatial surface, the spatial system is first performed, and then the spatial individual approximation of the motion to opium fragments in the central spatial coordinate system, and immediately before the spatial approximation of the movement, the processing signs of all system units are reset to the non-processing state, the signs of approximation of the movement of all fragments are reset to the absence of the approximation of motion, the current speeds of the spatial copies of the external fragments are assigned zero values, and the current coordinates of spatial copies of external fragments are loaded with the previous values ii, when conducting a spatial system approximation of motion within the framework of a geometric reconstruction of the current imprinted spatial surface, first identify system units that do not contain external fragments, after which the signs of processing data of system units are set to the state of processing and, further, for copies of embedded fragments of each such system unit for each of the three spatial coordinate axes, an approximation of the movement of spatial copies of fragments is carried out, while for The behavior of a spatial system approximation using the equation of motion of the spatial system approximation type of movement

u _s - the speed of spatial copies of fragments of the current system unit along the current coordinate axis;
t is the time;
x is the abscissa of fragments in a given planar central coordinate system;
y is the ordinate of fragments in a given planar central coordinate system;
a _{s, 1} , a _{s, 2} , a _{s, 3} - coefficients of the spatial system approximation of the movement of copies of fragments of the current system unit along the current coordinate axis;
in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for calculations, first, using the data of two previous frames of the central stream and the data of two previous stereo pairs, the coefficients of the spatial system approximation of motion are calculated, for which each coordinate axis compose and solve in a discrete form a system of three such equations in accordance with the three selected design fragments, then the signs of appro simulations of the movement of nested fragments of processed system units are transferred to the state of a system approximation of motion, and then, according to the current and previous frames of the central stream and the data of the current and previous stereo pairs for fragments with signs of a system approximation of motion, the current spatial velocities of movement of copies of fragments are calculated using the specified spatial equation system approximation of motion, and then approximation values of current spatial coordinates the position of the copy of the fragments, after which the condition for a slight change in the current spatial coordinates of the copy of the fragments compared to the previous coordinates is checked, and the mathematical model of this condition is pre-set, if this condition is not fulfilled, the current spatial coordinates of the copy of the fragment are reloaded with the previous values, and the sign of approximation of the movement of the corresponding fragment is reset in a state of lack of approximation of motion, when conducting a spatial individual To approximate the motion within the framework of the geometric reconstruction of the current imprinted spatial surface, first, for internal and boundary fragments with signs of the absence of a motion approximation, these signs are transferred to the state of an individual approximation of motion, then for each copy of a fragment with a sign of an individual approximation of motion along each of the three spatial coordinate axes spatial approximation of motion using the spatial equation individual approximation of the movement of the species

u _s - the speed of the spatial copy of the current fragment along the current coordinate axis;
t is the time;
k _s - coefficient of individual spatial approximation of the movement of the spatial copy of the current fragment along the current coordinate axis;
in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for calculations, with first approximation coefficients calculated from the data of two previous frames of the central stream and two previous stereo pairs, and then from the data of the current and previous frames and data current and previous stereopairs current spatial speeds and coordinates of copies of fragments, then for a spatial copy of each fragment with an individual motion simulations carry out processing that is absolutely similar to processing a copy of a fragment with a sign of system approximation during spatial system approximation of movement, starting from checking for insignificance of changes in the current spatial coordinates of copy fragments, when performing motion compensation in a stereo pair, first carry out a planar systemic, then planar individual, and then planar search approximation of the movement of fragments in the central planar coordinate system followed by projecting images of fragments onto stereopair frames by taking into account the spatial coordinates of copies of fragments, moreover, immediately before making planar approximations of motion, processing signs of all system units are reset to the non-processed state, signs of approximating motion of all fragments and macroblocks are reset to the absence of approximating motion, and integer sequential and parallel displacement vectors of external fragments of the current central frame and macroblock The current stereopair is forcibly filled with zero values, when conducting a planar system approximation of the movement, first identify system units that do not contain external fragments, after which the processing signs of the system units are set to the processing state and then for fragments of each such system unit for each of the two planar coordinates the axes carry out a planar approximation of motion, while the equation is used to conduct a planar system approximation of motion loskostnoy system approximation type of movement

u _p - the speed of the fragments of the current system unit along the current coordinate axis;
t is the time;
x is the abscissa of fragments in a given planar central coordinate system;
y is the ordinate of fragments in a given planar central coordinate system;
a _{p, 1} , a _{p, 2} , a _{p, 3} - coefficients of a planar system approximation of the movement of fragments of the current system unit along the current coordinate axis;
in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for calculations, while first, using the data of the two previous frames of the central stream and the data of the two previous stereo pairs, approximation coefficients of motion are calculated, for which, for each coordinate axis, solve in a discrete form a system of three such equations in accordance with the three selected fragments, then on each coordinate axis check conjun An active condition is that the absolute value of the coefficient at the time derivative prevails over the absolute values of the coefficients at spatial derivatives, and the degree of prevalence and the mathematical formula for checking the coefficients are predefined, then the signs of approximating the movement of nested fragments of system units that pass the specified test positively are transferred to the state of the system approximation of motion, and then according to the current and previous frames of the central stream and according to the current and other of the previous stereopairs for fragments with signs of a systemic approximation of motion, calculate the current velocities of the fragments using the shortened equation of the planar systemic approximation of motion without terms with spatial derivatives, and then calculate the approximation values of the integer vectors of the displacements of the fragments to the best forecasts within the reference extrapolated frame, then check the condition of the hit found approximation forecasts in the format of extrapolated reference frame, as well as the condition for satisfying the forecast to the approximate value of the forecast quality criterion, the value of which is set previously, while in the case of fulfilling these conditions, a shortened search for fragmentary forecasts is organized within the reference extrapolated frame, while the centers of the fragments of the search areas are shifted along the coordinate axes by the values of the corresponding of approximation displacement vectors found, and the geometric dimensions of the search areas are reduced to values that are beyond they are preliminarily set up; otherwise, the sign of approximation of the fragment motion is reset to the absence of approximation state, then the images of fragments with the sign of systemic approximation of motion are projected onto the personnel planes, while the values of the integer vectors of successive displacements of the inner and boundary macroblocks of the left frame are loaded to the best forecasts in the previous reference frame the corresponding found values of the displacement vectors of the fragments, and to calculate the values of the vectors of parallel displacement for macroblocks of the left and right frames, pairs of projection macroblocks in the left and right extrapolated frames are determined, and the algorithm for projecting spatial images on the frame planes introduces the use of the directing vector of the line passing through the beginning of the corresponding lateral spatial coordinate system and the spatial position point of the corresponding copy of the fragment, and the connection of lateral planar and corresponding lateral spatial coordinate systems is set using coordinates, for example vectors, then check the condition for the projection macroblocks of projection pairs to fall into the formats of their respective extrapolated frames, in which case the signs of approximation of the motion of the corresponding macroblocks of projection pairs are set to the state of the system approximation of motion, then, using the coordinates of the projection pairs, the vectors of parallel displacements of the macroblocks with signs system approximation of motion for the left and right frames, after which they check negative prediction displacement conditions for abscissa of projection macroblocks of the left frame and positive prediction displacement conditions for abscissa of projection macroblocks of the right frame, if at least one of them fails, the sign of approximation of motion of the corresponding macroblock is reset to the state of absence of approximation of motion, when conducting a plane individual approximation of motion, first for internal and boundary fragments with signs of a lack of approximation of motion translation of these signs into the state of an individual approximation of motion, then for each fragment with a sign of an individual approximation of motion, a planar approximation of motion is carried out on each of the two plane coordinate axes using the equation of a plane individual approximation of motion of the form

u _p - the speed of the current fragment along the current coordinate axis;
t is the time;
k _p - coefficient of plane individual approximation of the movement of the current fragment along the current coordinate axis;
in which each parameter is either an integer or fractional value, and a discrete representation of this equation is used for calculations, with first approximation coefficients calculated from the data of two previous frames of the central stream and two previous stereo pairs, and then from the data of the current and previous frames and data current and previous stereopairs current speeds and integer displacement vectors of fragments to approximation forecasts, then for each fragment with an individual motion approximations carry out processing that is absolutely similar to fragment processing with a system approximation sign when conducting a systematic plane approximation of motion, starting from finding projections of spatial copies of fragments on stereopair frames, when conducting planar search approximation of motion, first, for internal and boundary fragments with signs of lack of motion approximation, they translate these signs to the state of search approximation of motion, then integer vectors The hypnosis displacements of the fragments are found in the standard way through the organization of the search for forecasts in the previous reference extrapolated frame of the central stream, then for each fragment with the sign of the search approximation of the movement, processing is carried out that is absolutely similar to the processing of the fragment with the sign of the individual approximation during the individual planar approximation of the movement, starting from finding the projections of spatial copies of fragments to stereopair frames when performing approximations of motion and exclude the occurrence of division by zero errors, when searching for macroblock forecasts, use the initial values of the signals of the fixed pixels of the reference extrapolated frames, when organizing the algorithm for searching integer displacement vectors to the best forecasts, only internal and boundary macroblocks with signs of the lack of approximation of motion are processed, with the exception of internal and boundary macroblocks of the left frames for which signs of approximation of the movement of the fragments corresponding to them in the indices are not in a state There are no approximation of motion, moreover, the values of successive displacement vectors of such macroblocks are loaded with the corresponding values from the vectors of these fragments, and the values of parallel displacement vectors are found through the organization of the forecast search in the reference frame, the sizes and location of the search areas of parallel macroblock forecasts are set so that the ordinates of the forecasts are equal to the ordinates of the corresponding macroblocks themselves, and the values of the predicted offsets for abscissas could only take negative values for macroblocks of the left frame and only positive values for macroblocks of the right frame, immediately after calculating the macroblock residue of the boundary macroblock, the pixel signal values of the macroblock remainder of its fixed pixels are filled with zero values, after searching for integer displacement vectors, form fractional sequential displacement vectors for all fragments of the central frame and fractional parallel displacement vectors for all macroblocks of the left and right frames, with fractional displacement vectors of external fragments and macroblocks are forcibly filled with zero values, and the method of calculating the values of fractional displacement vectors for internal and boundary fragments and macroblocks is preliminarily selected with the condition that the values of the negative values of the offset values for abscissas for the macroblocks of the left frame and the condition for the positive values of the values of the offset values for abscissas for macroblocks right frame, after the formation of fractional displacement vectors of all fragments and macroblocks for each fragment, I recalculate m the current values of the speeds of the fragment along the coordinate axes of the central planar coordinate system, and according to the found parallel fractional vectors of displacements of the macroblocks, the current reconstructed surface is corrected by recalculating the values of the spatial coordinates and copy speeds of the fragments, while the spatial coordinates of the copies of the external fragments are reloaded with the previous values, the spatial copy speeds external fragments are loaded with zero values, and to calculate the coordinates of the copy as one of the internal or boundary fragments uses the direction vectors of two intersecting straight lines passing through the origin of the corresponding lateral spatial coordinate systems, and the coordinates of the direction vectors are pre-set through the relationship of the lateral plane and corresponding lateral spatial coordinate systems, the character of which is pre-set, then using values recalculated spatial coordinates of copies of internal and boundary fragments recalculated space the speed of their movement, after that the time stamps of the current and previous stereopairs, as well as the values of the spatial velocities and position coordinates of all copies of the fragments and the values of the planar speeds of all fragments for the current and previous stereopairs, are saved in the encoder memory by the decoder when restoring the original image matrices macroblocks of the current stereo pair use the original values of the signals of the fixed pixels of the reference frames. 2. The method of motion compensation in digital dynamic stereo video images according to claim 1, characterized in that in the encoder the frames of each stereo pair are further scaled immediately after they are received, and after receiving the output data, backscaling is applied to the frames of the stereo pair, as well as to macroblock residuals, forecast vectors displacements and masked movements of macroblocks of a stereo pair.

Images