CN117596394B - Depth map inter-frame compression method with self-adaptive precision - Google Patents
Depth map inter-frame compression method with self-adaptive precision Download PDFInfo
- Publication number
- CN117596394B CN117596394B CN202410071618.0A CN202410071618A CN117596394B CN 117596394 B CN117596394 B CN 117596394B CN 202410071618 A CN202410071618 A CN 202410071618A CN 117596394 B CN117596394 B CN 117596394B
- Authority
- CN
- China
- Prior art keywords
- depth
- frame
- value
- differential
- image frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007906 compression Methods 0.000 title claims abstract description 77
- 230000006835 compression Effects 0.000 title claims abstract description 77
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims abstract description 16
- 230000003044 adaptive effect Effects 0.000 claims description 17
- 230000000875 corresponding effect Effects 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 230000002596 correlated effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/93—Run-length coding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a depth map inter-frame compression method with self-adaptive precision, which comprises the following steps: judging whether the current depth image frame is a key frame or not; if yes, compressing the current depth image frame data by using a differential run-length coding algorithm, wherein the differential run-length coding algorithm firstly calculates differential result data of the current depth image frame, and then carries out lossy run-length coding processing on the differential result data; if not, calculating the interframe difference value of the current depth image frame and the previous depth image frame, and compressing the interframe difference value by using a difference run-length coding algorithm; and calculating the inter-frame difference value according to the depth values of the current depth image frame and the depth image frame of the previous frame. According to the invention, the compression precision is adaptively and dynamically adjusted according to the depth value, and the compression rate is improved on the basis of keeping the key information of the original data.
Description
Technical Field
The invention relates to the field of image data compression, in particular to a depth map inter-frame compression method with self-adaptive precision.
Background
As a novel machine vision system device, the depth camera can capture the surface shape and depth information of an object, and the machine vision system can effectively realize automatic identification and detection functions. The depth camera works on the principle that a special image is generated, and the image can capture not only the shape of the object surface, but also the depth information of the object surface. It captures depth information of the object surface by means of a laser and some pattern matching algorithm. The pattern matching algorithm maps the laser emitted pattern to the object surface and calculates the distance data of the object surface.
The depth camera generates a depth map sequence, and the scene depth information output by the capturing calculation is written into the depth map sequence. In application scenarios such as unmanned, digital security, etc., a computing architecture of a terminal edge cloud is generally used. The end-side depth camera sensor firstly transmits the captured depth map data to the side or cloud side through a network, and then performs machine learning calculation such as target recognition and the like, so that compression and network transmission of the depth map become key bottlenecks of the whole system. The traditional depth map compression method uses lossless compression or uses 16-bit JPG/PNG/TIFF standard image formats for compression, so that the problems of low compression rate, long compression time, lost scene object edge information and the like can occur.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a depth image inter-frame compression method with self-adaptive precision, which comprises the steps of firstly determining whether a current depth image frame is a key frame, and directly compressing current depth image frame data if the current depth image frame is the key frame; otherwise, calculating the difference value between the frame and the previous frame, and compressing the difference value between the frames.
The specific technical scheme is as follows:
an adaptive-precision depth map inter-frame compression method comprises the following steps:
s1: judging whether the current depth image frame is a key frame or not; if yes, executing S2; if not, the current depth image frame is considered as an interval frame, and S3 is executed;
S2: compressing the current depth image frame data by using a differential run-length coding algorithm; the differential run-length coding algorithm firstly calculates differential result data of the current depth image frame, and then carries out lossy run-length coding processing on the differential result data;
S3: calculating the inter-frame difference value of the current depth image frame and the previous depth image frame, and compressing the inter-frame difference value by using the difference run-length coding algorithm; the calculation process of the interframe difference value specifically comprises the following steps:
Each element of the inter-frame difference value D d includes a depth value and an illegal depth value counter, and the depth value and illegal depth value counter are initialized to 0; traversing all depth values D c [ i, j ] of the current depth image frame, and obtaining a depth value D p [ i, j ] of a depth image frame of a previous frame corresponding to the pixel coordinate [ i, j ];
If D c [ i, j ] is a legal value and D p [ i, j ] is an illegal value, D d[i,j].value=Dc[i,j]–Dp [ i, j ];
If D c[i,j]、Dp i, j are legal values, determining whether D d i, j, counter is greater than a preset threshold T c, and if D d[i,j].counter＞Tc, letting D d[i,j].counter=0,Dd i, j, value=0; if D d[i,j].counter≤Tc, setting D d [ i, j ] counter=0, and setting an adaptive accuracy error threshold T e according to D c [ i, j ] and D p [ i, j ]; let D d [ i, j ] value=0 if abs (D c[i,j]-Dp[i,j])<Te; otherwise D d[i,j].value=Dc[i,j]-Dp [ i, j ];
If D c [ i, j ] is an illegal value, let D d[i,j].counter=Dd [ i, j ]. Counter+1.
Further, in the step S1, the first frame depth image frame is necessarily a key frame, and the step of judging whether the rest of the current depth image frames are key frames includes the following sub-steps:
S1.1: each frame of depth image frame is allocated with a frame sequence number, the frame sequence number of the first frame of depth image frame is 0, and the frame sequence numbers of the rest depth image frames are sequentially increased;
S1.2: randomly generating a two-dimensional bounding box in a two-dimensional image space of the depth map;
S1.3: the two-dimensional bounding box is used for respectively collecting the depth value of the current depth image frame and the depth value of the corresponding pixel coordinate of the previous frame of depth image frame, and calculating the average mean square difference value of the depth values of the current depth image frame and the depth value of the previous frame of depth image frame which are legal;
S1.4: if the average mean square error is greater than 1000, or if the frame sequence number is divisible by 30, the current depth image frame is a key frame, otherwise it is an interval frame.
Further, the differential run length encoding algorithm specifically includes the following steps:
(1) Calculating a differential result of the compressed data by adopting a retrace scanning method to obtain a differential depth sequence;
(2) Compressing the repeated part in the differential depth sequence by using a run-length coding algorithm to obtain a compressed differential depth sequence, and presetting a differential depth threshold T d during compression; the compressed differential depth sequence is formed by splicing a continuous coding unit and a discontinuous coding unit;
(3) And processing the compressed differential depth sequence by using a space bit coding algorithm to obtain a compression result of differential run length coding.
Further, the consecutive encoding unit comprises an integer P and a first differential depth value VP, corresponding to a sub-sequence of the differential depth sequence, which sub-sequence fulfils the following condition: the subsequence has P elements, the absolute value of the difference value between the value of each element and VP is smaller than a differential depth threshold T d, and VP is any one of the P elements;
The discontinuous coding unit comprises an integer Q and Q second differential depth values VQ, and corresponds to a subsequence of the differential depth sequence, wherein the subsequence meets the following conditions: the sub-sequence has Q elements, wherein each element has a value corresponding to VQ one-to-one, and the absolute value of the difference between the second differential depth value VQ and any element is not less than the differential depth threshold T d.
Further, in the step (1), the retrace scanning method starts from the first pixel of the original depth map, scans from left to right, scans from right to left after scanning to the rightmost pixel, scans from right to left after scanning to the leftmost pixel, scans from left to right again, and repeats cyclically until the whole original depth map is scanned.
Further, in S3, the illegal depth value is a numerical abnormal depth value smaller than a preset minimum depth threshold T min and larger than a preset maximum depth threshold T max.
Further, in S3, the value of the adaptive accuracy error threshold T e is positively correlated with the average value of the depth value D c [ i, j ] of the current depth image frame and the depth value D p [ i, j ] of the previous depth image frame.
Further, adding bit zone information at the tail end of the compression result data of the differential run-length coding as final depth map compression result data; if the current depth image frame is a key frame, setting the flag bit as 1, otherwise setting the flag bit as 0.
An adaptive precision depth map inter-frame compression device comprises one or more processors for realizing the adaptive precision depth map inter-frame compression method.
A computer readable storage medium having stored thereon a program which, when executed by a processor, implements the adaptive precision depth map inter-frame compression method.
The beneficial effects of the invention are as follows:
The invention discloses a depth image inter-frame compression method with self-adaptive precision, which is characterized in that the compression precision is self-adaptively and dynamically adjusted according to the size of a depth value, and the compression rate is improved as much as possible on the basis of keeping the key information of the original data of the current depth image frame; the time continuity is considered, the inter-frame differential value is compressed by adopting an inter-frame differential run coding algorithm, and the compression rate of the depth map is obviously improved.
Drawings
Fig. 1 is a flowchart of a depth map inter-frame compression method with adaptive accuracy according to an embodiment of the present invention.
FIG. 2 is a flowchart of determining whether a current depth image frame is a key frame according to an embodiment of the present invention.
Fig. 3 is a flowchart of calculation of an interframe difference value in the embodiment of the invention.
FIG. 4 is a schematic diagram of differential run length encoding calculation according to an embodiment of the present invention.
Fig. 5 is a map of an average value of the depth values of the current depth image frame and the depth values of the previous depth image frame and an error value of adaptive accuracy in an embodiment of the present invention.
Fig. 6 is an application effect diagram of a depth map inter-frame compression method using adaptive precision according to an embodiment of the present invention, where (a) is an original depth map, (b) is a compressed depth map, and (c) is a difference value before and after compression.
Fig. 7 is a schematic diagram of an adaptive-precision depth map inter-frame compression apparatus according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the following description, reference is made to "one particular embodiment" which describes a subset of all possible embodiments, but it is to be understood that "one particular embodiment" describes the same subset or a different subset of all possible embodiments and can be combined with each other without conflict.
Unless defined otherwise, all technical and scientific techniques used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.
As shown in fig. 1, a depth map inter-frame compression method with adaptive precision includes the following steps:
s1: judging whether the current depth image frame is a key frame or not, and if the current depth image frame is the key frame, executing S2; if not, the current depth image frame is considered to be an interval frame, and S3 is executed.
In a computer vision system, three-dimensional scene information provides more possibility for various computer vision applications such as image segmentation, target detection, object tracking and the like, and a Depth image (Depth Map) is widely applied as a general three-dimensional scene information expression mode. The gray value of each pixel point of the depth image can be used for representing the distance between a certain point in the scene and the camera, and reflects the geometric shape of the surface of an object in the scene. Depth maps are typically stored using two-dimensional images with bit depths of 32 bits or 16 bits, and may represent depth values within the precision range of floating point numbers and semi-floating point numbers.
In this embodiment, the depth map is stored with a bit depth of 16 bits, the depth value is represented by unsigned short types, the numerical sampling range is [0,65535], the depth unit of the depth value is 0.1cm, that is, the actual depth distance is increased by 0.1cm every time the depth value is increased by 1; the depth map has a size of 720p, namely an image resolution of 1280×720, and an acquisition frame rate of 30hz; depth map data is acquired using intel's REALSENSE D455,455 sense depth camera connected to Jetson Orin NX embedded device.
In this embodiment, the type of the current depth image frame is first determined, and then the subsequent compression encoding operation is performed according to the type. If the current depth image frame is the first frame, the current depth image frame is necessarily the key frame, otherwise, the current depth image frame can only be the key frame or the interval frame, as shown in fig. 2, the method for judging whether the current depth image frame except the first frame is the key frame comprises the following sub steps:
S1.1: each frame of depth image frame is allocated a frame sequence number, the frame sequence number of the first frame of depth image frame is 0, and the frame sequence numbers of the rest depth image frames are sequentially increased by one increment.
S1.2: 6 two-dimensional bounding boxes are randomly generated in a two-dimensional image space of the depth map, and the sizes of the two-dimensional bounding boxes are 10px multiplied by 10px.
S1.3: and respectively acquiring the depth value of the current depth image frame and the depth value of the corresponding pixel coordinate of the previous depth image frame by using a two-dimensional bounding box, counting only that the two depth values are legal depth values, and calculating the average mean square difference value.
S1.4: if the average mean square error is greater than 1000, or if the frame sequence number is divisible by 30, the current depth image frame is a key frame, otherwise it is an interval frame.
S2: the current depth image frame data is compressed using a differential run-length encoding algorithm.
Run length encoding (Run Length Encoding, RLE), also known as run length encoding, etc., is a type of statistical encoding. The technical principle of run-length coding is to detect repeated bit or character sequences and replace them with their number of occurrences. The encoding is more suitable for binary images, but is not suitable for compression of continuous tone images. The run-length coding technology is quite visual and economical, and the operation is quite simple, so that the decompression speed is quite high. Run-length encoding techniques are particularly useful for computer-generated graphics images, which have the effect of reducing memory capacity. The conventional run-length encoding technique is a lossless compression algorithm, which compresses only adjacent and identical data in an original data sequence, and if the original data is not continuous enough or noise exists, the compression rate is drastically reduced.
In this embodiment, a differential run-length encoding algorithm is used, and the algorithm firstly calculates differential result data of a current depth image frame, and then performs lossy run-length encoding processing on the differential result data, and specifically includes the following sub-steps:
S2.1: and calculating a differential result of the compressed data by adopting a retrace scanning method to obtain a differential depth sequence.
In this embodiment, the retrace scanning method starts from the first pixel of the original depth map, scans from left to right, scans from right to left to the rightmost pixel, then scans from right to left to the leftmost pixel, scans from left to right to the leftmost pixel, and repeats cyclically until the whole original depth map is scanned. Conventional image scan line-changing starts from scratch, and the line-changing can occur jumping of distance values due to locality of depth data, increasing dynamic range. The depth map can be straightened into a one-dimensional data sequence in a retrace scanning mode based on differential calculation of a retrace scanning method, so that jumping of depth values is effectively reduced, and subsequent compression processing is facilitated.
S2.2: compressing the repeated part in the differential depth sequence by using a run-length coding algorithm to obtain a compressed differential depth sequence, presetting a differential depth threshold T d during compression, wherein the differential depth threshold T d is a parameter which can be set in an adjustable way, and if the differential depth threshold is set to be 0, degrading the differential run-length coding into lossless compression; otherwise, lossy compression. Obviously, the larger the differential depth threshold T d, the higher the compression rate, but the lower the compression accuracy; whereas the lower the compression ratio, the higher the compression accuracy. In this embodiment, the differential depth threshold T d is set to 30, i.e., depth values within 30 (3 cm) of adjacent errors are considered to be the same depth value. The compressed differential depth sequence is formed by splicing two coding units, namely a continuous coding unit and a discontinuous coding unit, and the two coding units are characterized in that:
The continuous coding unit comprises an integer P and a first differential depth value VP, and corresponds to a subsequence of the differential depth sequence, wherein the subsequence meets the following conditions: the sub-sequence has P elements, and the absolute value of the difference between the value of each element and VP is less than the differential depth threshold T d, where VP is any one of the P elements.
The discontinuous coding unit comprises an integer Q and Q second differential depth values VQ, and corresponds to a subsequence of the differential depth sequence, and the subsequence meets the following conditions: the sub-sequence has Q elements, wherein each element has a value corresponding to the second differential depth value VQ one by one, and the absolute value of the difference between the second differential depth value VQ and any element is not less than the differential depth threshold T d.
As shown in fig. 3, in this embodiment, the differential depth sequence may be divided into a class 0 sub-sequence (0,1,0,0,2) and other non-class 0 sub-sequences (12,14,12,9), (137,137,135), (67,6), (43, 42), where the class 0 sequence represents a low frequency region in the depth image, and corresponds to an internal region of the object; the non-class 0 subsequence represents high frequency regions in the depth image, corresponding to edges of the object. Compressing the differential depth sequence by using a run-length coding algorithm to obtain a compressed differential depth sequence, wherein (4, 12), (5, 0), (3, 137), (2,43) are continuous coding units, and (2,67,6) is a discontinuous coding unit; the method can more effectively and pointedly compress the low-frequency internal area, and keep the critical object edge information.
S2.3: and processing the compressed differential depth sequence by using a space bit coding algorithm to obtain a compression result of differential run length coding.
In this embodiment, since differential data is encoded, there is a large amount of data whose value is close to 0, and these data close to 0 have many gaps at the binary level. And (3) cutting the compressed differential depth sequence into a plurality of binary strings with the length of 16 by using a vacancy coding algorithm, uniformly cutting the binary strings into binary substrings with the length of 4, traversing all binary substrings from low order to high order, and compressing high order vacancies of the binary substrings.
S3: the method specifically comprises the following steps of:
S3.1: setting an inter-frame difference value D d, wherein each element of the inter-frame difference value D d comprises a depth value and an illegal depth value counter, and initializing the depth value and the illegal depth value counter to 0.
S3.2: traversing all depth values D c [ i, j ] of the current depth image frame, and obtaining a depth value D p [ i, j ] of a depth image frame of a previous frame corresponding to the pixel coordinate [ i, j ]; according to the values of the depth value D c [ i, j ] of the current depth image frame and the depth value D p [ i, j ] of the previous depth image frame, the corresponding inter-frame difference value D d [ i, j ] is calculated, and the specific calculation process is as follows:
S3.2.1: if the depth value D c [ i, j ] of the current depth image frame is a legal value and the depth value D p [ i, j ] of the previous depth image frame is an illegal value, D d[i,j].value=Dc[i,j]–Dp [ i, j ] is used for finishing the calculation of the inter-frame differential value; otherwise, the jump is carried out to the next step.
S3.2.2: if the depth value D c [ i, j ] of the current depth image frame is an illegal value, D d[i,j].counter=Dd[i,j].counter+1,Dd [ i, j ] value is kept unchanged. If the depth value D c [ i, j ] of the current depth image frame and the depth value D p [ i, j ] of the previous depth image frame are legal values, judging whether the D d [ i, j ] counter is larger than a preset threshold value T c, setting the threshold value T c by human, and if D d[i,j].counter＞Tc, making D d[i,j].counter=0,Dd [ i, j ] value=0, and finishing the calculation of the inter-frame differential value; otherwise, the jump is carried out to the next step.
S3.2.3: setting D d [ i, j ] counter=0, and setting an adaptive accuracy error threshold T e according to D c [ i, j ] and D p [ i, j ]; in the present embodiment, it is determined whether the absolute value of the difference between D c [ i, j ] and D p [ i, j ] is smaller than the adaptive accuracy error threshold T e, if abs (D c[i,j]-Dp[i,j])<Te, let D d [ i, j ] value=0, otherwise D d[i,j].value=Dc[i,j]-Dp [ i, j ].
And regarding the inter-frame difference value obtained through final calculation, if D d [ i, j ]. Value=0, considering that the depth value of the current frame depth map and the depth value of the previous frame depth map at the pixel coordinates [ i, j ] are the same, compressing at the pixel coordinates [ i, j ], otherwise, not compressing.
In this embodiment, the illegal depth values in the depth map refer to numerical abnormal depth values smaller than a preset minimum depth threshold value T min =10 and larger than a preset maximum depth threshold value T max =65000. Most depth maps output by depth cameras contain illegal depth values, which are typically caused by hardware defects of the camera or failure of the camera's depth evaluation algorithm to solve a partial region of the object. The illegal depth value is an outlier that needs to be discarded in the application algorithm such as the subsequent perception algorithm, and therefore is also a pixel area that needs to be emphasized by the depth map compression algorithm.
In an actual depth map, dynamic objects of the foreground are often more important than static objects such as wall surfaces and ground surfaces of the background; in addition, due to the device acquisition characteristics of the depth camera itself, the depth value error of distant objects is larger than that of near objects. Therefore, for depth map compression, depth values at the near and far positions need to be treated differently, and different precision errors are set for compression respectively. In this embodiment, the adaptive accuracy error threshold T e is not a fixed value, but is an adaptive value that is positively correlated to the average value of the depth values D c [ i, j ] of the current depth image frame and the depth values D p [ i, j ] of the previous depth image frame, and the larger the average depth value is, the larger the error threshold is. The specific mapping relation between the average depth value and the error value of the self-adaptive precision is shown in fig. 5, and when the average depth value is smaller than the minimum depth value 5000, the self-adaptive precision error threshold T e =20 is taken; when the average depth value is greater than the maximum depth value 50000, taking an adaptive accuracy error threshold value T e =200; when the average depth value is between the minimum depth value and the maximum depth value, the average depth value is linearly related to the error value of the adaptive accuracy, and the adaptive accuracy error threshold T e interpolates between 20 and 200. By dynamically adjusting the self-adaptive precision error threshold, the self-adaptive dynamic setting of compression precision errors is realized, so that near errors are smaller, far errors are larger, the device acquisition characteristics of a depth camera are more met, meanwhile, the precision of far background can be sacrificed on the basis of near detail precision is reserved, and the overall depth map compression rate is improved.
In this embodiment, the differential run-length encoding algorithm used in S2 and S3 is the same algorithm, and the compression result of the differential run-length encoding includes flag bit information of whether the current depth image frame is a key frame, where the flag bit information is stored at the extreme end of the compression algorithm, and the specific encoding and decoding modes are as follows:
(1) Encoding part: if the current depth image frame is a key frame, setting a flag bit as 1; otherwise, set to 0. And adding bit zone information at the tail end of the compression result data of the differential run-length coding to serve as final depth map compression result data.
(2) The decoding end receives the depth map compression result data, firstly reads the flag bit of the last bit, and if the flag bit is 1, a frame of key frame is illustrated; otherwise, a frame interval frame is received. And removing the last bit of the received depth map compression result data, and sending the data to a corresponding decoding module for decoding according to the frame type.
The final compression result between depth maps with self-adaptive precision is shown in fig. 6, in which the original depth map, the compressed depth map and the difference values before and after compression are displayed, wherein the depth values are mapped from [0,65535] in fig. 6 (a) to [0,255] in fig. 6 (b), and are visualized in the form of single-channel gray map, and the closer the pixel area is, the darker the further the pixel area is. According to the embodiment, the depth map acquired in the 673-frame indoor scene is compressed and processed, the scene comprises a plurality of office tables, chairs and dynamically walking characters, the average compression rate is 7%, and compared with the prior art, the compression rate is remarkably improved. In addition, as can be seen from fig. 6 (c), the difference values before and after compression are mainly distributed in the illegal value pixel areas around the characters and office equipment, and the difference values inside the scene object and at the edges are reduced, so that the invention is embodied to improve the compression rate as much as possible on the basis of keeping the key information of the original data.
Corresponding to the foregoing embodiments, the present invention further provides an embodiment of an adaptive-precision depth map inter-frame compression apparatus, as shown in fig. 7, where the apparatus includes one or more processors configured to implement the adaptive-precision depth map inter-frame compression method.
The embodiment of the adaptive-precision depth map inter-frame compression device can be applied to any device with data processing capability, and the device with data processing capability can be a device or a device such as a computer. The apparatus embodiments may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking software implementation as an example, the device in a logic sense is formed by reading corresponding computer program instructions in a nonvolatile memory into a memory by a processor of any device with data processing capability. In terms of hardware, in addition to the processor, the memory, the network interface, and the nonvolatile memory, any device with data processing capability in the embodiments of the present invention generally may further include other hardware according to the actual function of the any device with data processing capability, which will not be described herein.
The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The embodiment of the present invention also provides a computer-readable storage medium having a program stored thereon, which when executed by a processor, implements the adaptive-precision depth map inter-frame compression method in the above embodiment.
The computer readable storage medium may be an internal storage unit, such as a hard disk or a memory, of any of the data processing enabled devices described in any of the previous embodiments. The computer readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart memory card (SMARTMEDIACARD, SMC), an SD card, a flash memory card (FLASHCARD), etc. provided on the device. Further, the computer readable storage medium may include both internal storage units and external storage devices of any data processing device. The computer readable storage medium is used to store the computer program, as well as other programs and data required by any of the data processing capable devices, and may also be used to temporarily store data that has been or is to be output.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely illustrative of the preferred embodiments of the present invention and it will be appreciated by those skilled in the art that the invention is not limited thereto, and that various modifications and changes in the embodiments will be apparent to those skilled in the art, and that the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. The depth map inter-frame compression method with self-adaptive precision is characterized by comprising the following steps of:
s1: judging whether the current depth image frame is a key frame or not; if yes, executing S2; if not, the current depth image frame is considered as an interval frame, and S3 is executed;
S2: compressing the current depth image frame data by using a differential run-length coding algorithm; the differential run-length coding algorithm firstly calculates differential result data of the current depth image frame, and then carries out lossy run-length coding processing on the differential result data;
The differential run-length coding algorithm specifically comprises the following steps:
(1) Calculating a differential result of the compressed data by adopting a retrace scanning method to obtain a differential depth sequence;
(2) Compressing the repeated part in the differential depth sequence by using a run-length coding algorithm to obtain a compressed differential depth sequence, and presetting a differential depth threshold T d during compression; the compressed differential depth sequence is formed by splicing a continuous coding unit and a discontinuous coding unit;
The continuous coding unit comprises an integer P and a first differential depth value VP, and corresponds to a subsequence of the differential depth sequence, which subsequence meets the following conditions: the subsequence has P elements, the absolute value of the difference value between the value of each element and VP is smaller than a differential depth threshold T d, and VP is any one of the P elements;
The discontinuous coding unit comprises an integer Q and Q second differential depth values VQ, and corresponds to a subsequence of the differential depth sequence, wherein the subsequence meets the following conditions: the subsequence has Q elements, wherein the value of each element corresponds to the VQ one by one, and the absolute value of the difference value between the second differential depth value VQ and any element is not smaller than the differential depth threshold T d;
(3) Processing the compressed differential depth sequence by using a space bit coding algorithm to obtain a compression result of differential run length coding;
S3: calculating the inter-frame difference value of the current depth image frame and the previous depth image frame, and compressing the inter-frame difference value by using the difference run-length coding algorithm; the calculation process of the interframe difference value specifically comprises the following steps:
Each element of the inter-frame difference value D d includes a depth value and an illegal depth value counter, and the depth value and illegal depth value counter are initialized to 0; traversing all depth values D c [ i, j ] of the current depth image frame, and obtaining a depth value D p [ i, j ] of a depth image frame of a previous frame corresponding to the pixel coordinate [ i, j ];
If D c [ i, j ] is a legal value and D p [ i, j ] is an illegal value, D d[i,j].value=Dc[i,j]–Dp [ i, j ];
If D c[i,j]、Dp i, j are legal values, determining whether D d i, j, counter is greater than a preset threshold T c, and if D d[i,j].counter＞Tc, letting D d[i,j].counter=0,Dd i, j, value=0; if D d[i,j].counter≤Tc, setting D d [ i, j ] counter=0, and setting an adaptive accuracy error threshold T e according to D c [ i, j ] and D p [ i, j ]; let D d [ i, j ] value=0 if abs (D c[i,j]-Dp[i,j])<Te; otherwise D d[i,j].value=Dc[i,j]-Dp [ i, j ];
If D c [ i, j ] is an illegal value, let D d[i,j].counter=Dd [ i, j ]. Counter+1.
2. The adaptive-precision depth map inter-frame compression method according to claim 1, wherein in S1, the first frame depth image frame is necessarily a key frame, and determining whether the remaining current depth image frames are key frames comprises the following sub-steps:
S1.1: each frame of depth image frame is allocated with a frame sequence number, the frame sequence number of the first frame of depth image frame is 0, and the frame sequence numbers of the rest depth image frames are sequentially increased;
S1.2: randomly generating a two-dimensional bounding box in a two-dimensional image space of the depth map;
S1.3: the two-dimensional bounding box is used for respectively collecting the depth value of the current depth image frame and the depth value of the corresponding pixel coordinate of the previous frame of depth image frame, and calculating the average mean square difference value of the depth values of the current depth image frame and the depth value of the previous frame of depth image frame which are legal;
S1.4: if the average mean square error is greater than 1000, or if the frame sequence number is divisible by 30, the current depth image frame is a key frame, otherwise it is an interval frame.
3. The adaptive-precision depth-map inter-frame compression method according to claim 1, wherein in the step (1), the retrace scanning method starts from the first pixel of the original depth map, scans from left to right, places the cursor at the pixel below after scanning to the rightmost pixel, scans from right to left, scans to the leftmost pixel, places the cursor at the pixel below after scanning to the leftmost pixel, scans from left to right, and repeats cyclically until the whole original depth map is scanned.
4. The adaptive-precision depth map inter-frame compression method according to claim 1, wherein in S3, the illegal depth value is a numerical abnormal depth value smaller than a preset minimum depth threshold T min and larger than a preset maximum depth threshold T max.
5. The adaptive-precision depth map inter-frame compression method according to claim 1, wherein in S3, the value of the adaptive-precision error threshold T e is positively correlated with the average value of the depth values D c [ i, j ] of the current depth image frame and the depth values D p [ i, j ] of the previous depth image frame.
6. The adaptive-precision depth map inter-frame compression method according to claim 1, wherein flag bit information is added at the tail end of compression result data of differential run-length coding as final depth map compression result data; if the current depth image frame is a key frame, setting the flag bit as 1, otherwise setting the flag bit as 0.
7. An adaptive-precision depth-map inter-frame compression apparatus, comprising one or more processors configured to implement the adaptive-precision depth-map inter-frame compression method of any one of claims 1-6.
8. A computer-readable storage medium, having stored thereon a program which, when executed by a processor, implements the adaptive-precision depth map inter-frame compression method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410071618.0A CN117596394B (en) | 2024-01-18 | 2024-01-18 | Depth map inter-frame compression method with self-adaptive precision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410071618.0A CN117596394B (en) | 2024-01-18 | 2024-01-18 | Depth map inter-frame compression method with self-adaptive precision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117596394A CN117596394A (en) | 2024-02-23 |
| CN117596394B true CN117596394B (en) | 2024-05-24 |
Family
ID=89916982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410071618.0A Active CN117596394B (en) | 2024-01-18 | 2024-01-18 | Depth map inter-frame compression method with self-adaptive precision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117596394B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103392189A (en) * | 2012-02-23 | 2013-11-13 | 史克威尔·艾尼克斯控股公司 | Moving image distribution server, moving image playback device, control method, program, and recording medium |
| CN104104964A (en) * | 2013-04-09 | 2014-10-15 | 乐金电子(中国)研究开发中心有限公司 | Depth image interframe encoding and decoding method, encoder and decoder |
| CN104904199A (en) * | 2013-01-11 | 2015-09-09 | 联发科技（新加坡）私人有限公司 | Method and apparatus for efficient coding of depth lookup table |
| WO2018127629A1 (en) * | 2017-01-09 | 2018-07-12 | Nokia Technologies Oy | Method and apparatus for video depth map coding and decoding |
| CN109379594A (en) * | 2018-10-31 | 2019-02-22 | 北京佳讯飞鸿电气股份有限公司 | Video coding compression method, device, equipment and medium |
| CN112750157A (en) * | 2020-08-11 | 2021-05-04 | 腾讯科技（深圳）有限公司 | Depth image generation method and device |
| CN114529585A (en) * | 2022-02-23 | 2022-05-24 | 北京航空航天大学 | Mobile equipment autonomous positioning method based on depth vision and inertial measurement |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2002952873A0 (en) * | 2002-11-25 | 2002-12-12 | Dynamic Digital Depth Research Pty Ltd | Image encoding system |
-
2024
- 2024-01-18 CN CN202410071618.0A patent/CN117596394B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103392189A (en) * | 2012-02-23 | 2013-11-13 | 史克威尔·艾尼克斯控股公司 | Moving image distribution server, moving image playback device, control method, program, and recording medium |
| CN104904199A (en) * | 2013-01-11 | 2015-09-09 | 联发科技（新加坡）私人有限公司 | Method and apparatus for efficient coding of depth lookup table |
| CN104104964A (en) * | 2013-04-09 | 2014-10-15 | 乐金电子(中国)研究开发中心有限公司 | Depth image interframe encoding and decoding method, encoder and decoder |
| WO2018127629A1 (en) * | 2017-01-09 | 2018-07-12 | Nokia Technologies Oy | Method and apparatus for video depth map coding and decoding |
| CN109379594A (en) * | 2018-10-31 | 2019-02-22 | 北京佳讯飞鸿电气股份有限公司 | Video coding compression method, device, equipment and medium |
| CN112750157A (en) * | 2020-08-11 | 2021-05-04 | 腾讯科技（深圳）有限公司 | Depth image generation method and device |
| CN114529585A (en) * | 2022-02-23 | 2022-05-24 | 北京航空航天大学 | Mobile equipment autonomous positioning method based on depth vision and inertial measurement |
Non-Patent Citations (3)
| Title |
|---|
| 3D-HEVC深度图视频快速算法研究;粘春湄;《中国优秀硕士学位论文全文数据库》;20200115;全文 * |
| Intra-frame fast coding algorithm for 3D-HEVC depth map based on two-dimensional entropy and variance;Wangze Yao;《2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC)》;20210405;全文 * |
| 面向虚拟视点图像绘制的深度图编码算法;朱波;蒋刚毅;张云;郁梅;;光电子.激光;20100515(05);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117596394A (en) | 2024-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110267041B (en) | Image encoding method, image encoding device, electronic device, and computer-readable storage medium | |
| JP5875637B2 (en) | Image processing apparatus and image processing method | |
| KR101877372B1 (en) | Method for embedding and extraction of watermarking data | |
| TWI426774B (en) | A method for classifying an uncompressed image respective to jpeg compression history, an apparatus for classifying an image respective to whether the image has undergone jpeg compression and an image classification method | |
| EP2963926A1 (en) | Method and apparatus for downscaling depth data for view plus depth data compression | |
| CN109344676B (en) | Automatic induction triggering method and system based on Hash algorithm | |
| US9300840B2 (en) | Image processing device and computer-readable storage medium storing computer-readable instructions | |
| WO2018002436A1 (en) | Method and apparatus for removing turbid objects in an image | |
| US11190803B2 (en) | Point cloud coding using homography transform | |
| KR20180127261A (en) | Method for embedding and extraction of watermarking data | |
| CN113810654A (en) | Image video uploading method and device, storage medium and electronic equipment | |
| JPH0937246A (en) | Image processing unit and its method | |
| JP2016054564A (en) | Image processing system and image processing method | |
| CN117596394B (en) | Depth map inter-frame compression method with self-adaptive precision | |
| US7570821B2 (en) | Apparatus and method for image coding | |
| KR20060007901A (en) | Apparatus and method for automatic extraction of salient object from an image | |
| CN113810692A (en) | Method for framing changes and movements, image processing apparatus and program product | |
| JP4083670B2 (en) | Image coding apparatus and image coding method | |
| CN117440154B (en) | Depth map sequence compression method considering floating point digital splitting | |
| CN111654701B (en) | Video coding preprocessing method, device, equipment and computer readable storage medium | |
| JP5799681B2 (en) | Image processing apparatus, integral image creation method and program | |
| JP3456617B2 (en) | Area dividing device | |
| KR20160072831A (en) | Image processing system and providing method thereof | |
| JP2005122571A (en) | Image processor, image processing method, program and storage medium | |
| Wu | A Radar Image Compression Algorithm Based on Machine Vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |