WO2016161675A1 - 一种基于目标的视频编码方法及系统 - Google Patents

一种基于目标的视频编码方法及系统 Download PDF

Info

Publication number
WO2016161675A1
WO2016161675A1 PCT/CN2015/077730 CN2015077730W WO2016161675A1 WO 2016161675 A1 WO2016161675 A1 WO 2016161675A1 CN 2015077730 W CN2015077730 W CN 2015077730W WO 2016161675 A1 WO2016161675 A1 WO 2016161675A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
background
frame
encoding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/077730
Other languages
English (en)
French (fr)
Inventor
武晓阳
浦世亮
沈林杰
俞海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Publication of WO2016161675A1 publication Critical patent/WO2016161675A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding

Definitions

  • the invention belongs to the technical field of video image compression, and in particular relates to a target-based video coding method and system.
  • the digital video compression standard began in the 1980s. After more than 30 years of development, the existing standards include H.261 of the ITU-T for ITU Telecommunication Standardization Sector (ITU-T). H.263, MPEG-1, MPEG-4 of the International Organization for Standardization (ISO), and MPEG-2/H.262, H.264/AVC, HEVC jointly developed by the two organizations.
  • HEVC High Efficiency Video Coding
  • there are other organizations' standards such as domestic AVS, Microsoft's VC-1, and Google's VP8.
  • these standards use a block-based hybrid coding method that combines predictive coding, transform coding, and entropy coding.
  • the user In the application of video surveillance, the user usually pays different attention to different areas in the image, and has high attention to the moving object and the specific target, that is, the target area; while the other background parts are not concerned, that is, the background area.
  • the target area For example, in road monitoring, users only pay attention to targets such as moving vehicles on the road, and do not care about the background such as road surface and green belt.
  • each collection point is continuously recorded for 24 hours, generating a large amount of data, and the user cannot view each video data in real time. Instead, when needed, especially in the event of an accident, look up the relevant video from the video and retrieve specific information from the video, which is usually related to moving objects.
  • the above-mentioned existing coding standard divides the image into coded blocks and sequentially encodes them by raster scanning, and does not distinguish whether the coded blocks belong to the background area or the target area.
  • each coding block is associated with a current frame neighboring block and also with a reference frame block, and these related blocks may not belong to the same object.
  • the entire frame image needs to be decoded to obtain an image of the user's area of interest, and decoding the entire frame image requires a lot of computing resources and memory bandwidth, which takes a lot of time and slow retrieval speed.
  • An object of the present invention is to provide a target-based video encoding method and system.
  • the present invention extracts a target image and a background image from an image to be encoded, and independently encodes the target image and the background image.
  • the code generated by each object is generated.
  • the stream is combined in the current image code stream in a certain order, and the code streams of different objects are separated by an identifier, and the code stream of the image includes header information to record the position information and reference relationship information of the target, thereby improving video retrieval. Efficiency, only need to find the target code stream in the code stream for decoding.
  • the target's motion trajectory can be determined by the predicted relationship of the target.
  • an aspect of the present invention provides a target-based video encoding method, including: acquiring an image to be encoded; extracting a target image and a background image based on the image to be encoded; performing interframe prediction encoding based on the target image; The image is subjected to inter-frame predictive coding; the target code stream generated by encoding the target image and the background code stream generated by encoding the background image are combined into a single code stream.
  • the step of performing inter prediction encoding based on the target image further includes: dividing the coding block based on the target image; and when encoding the target image of the current coding frame, the previous frame corresponding to the target area in the current frame image
  • the target area in the image is inter-predictive encoded as a reference frame.
  • the step of performing inter prediction encoding based on the background image further includes: when encoding the background image of the current encoded frame, using a background region in the previous frame image corresponding to the background region in the current frame image as a reference The frame is subjected to interframe predictive coding.
  • the individual code stream includes header information, target code stream information, and background code stream information; the header information includes location information of the target area, reference relationship information; and the target code stream information and the background code stream information includes Start code identification.
  • the step of performing inter-prediction encoding further includes: when encoding the target image of the current encoded frame, if a new target image appears in the current encoded frame, the new target image is subjected to intra-frame encoding processing.
  • a target-based video coding system including an acquisition module, an extraction module, a target encoding module, a background encoding module, and a code stream composite module; an obtaining module for acquiring an image to be encoded; a module, configured to extract a target image and a background image based on the image to be encoded; a target encoding module, configured to perform inter prediction encoding based on the target image; a background encoding module, configured to perform inter prediction encoding based on the background image; and a code stream composite module, The background code stream generated by encoding the target code stream generated by encoding the target image and the background image is combined into a separate code stream.
  • the target encoding module further includes: a coding block dividing unit configured to divide the coding block based on the target image; and a target reference unit configured to: when encoding the target image of the current encoded frame, The target area in the previous frame image corresponding to the target area is inter-predictive encoded as a reference frame.
  • the background encoding module further includes: a background reference unit, configured to: when encoding the background image of the current encoded frame, use a background region in the previous frame image corresponding to the background region in the current frame image as the reference frame Inter-prediction coding is performed.
  • a background reference unit configured to: when encoding the background image of the current encoded frame, use a background region in the previous frame image corresponding to the background region in the current frame image as the reference frame Inter-prediction coding is performed.
  • the individual code stream includes header information, target code stream information, and background code stream information; the header information includes location information of the target area, reference relationship information; and the target code stream information and the background code stream information includes Start code identification.
  • the target reference unit further includes: a handover coding processing module, configured to perform intra-frame coding processing on the new target image if a new target image appears in the current coding frame.
  • the present invention can effectively improve the retrieval efficiency by adopting a method of separately encoding the target and the background into different objects.
  • the separately coded code streams are combined in a certain order to form a new single code stream, so that when the video is retrieved later, the first need to find the target code stream in the code stream for decoding.
  • the target's motion trajectory can be determined by the predicted relationship of the target.
  • FIG. 1 is a schematic diagram of a block-based hybrid coding process in the prior art
  • FIG. 2 is a schematic diagram of a decoding process of block-based hybrid coding in the prior art
  • FIG. 3 is a schematic diagram of a process of a block coding sequence in the prior art
  • FIG. 4 is a schematic diagram of a process of performing predictive coding in the prior art
  • FIG. 5 is a schematic diagram of a P frame multiframe reference when performing inter prediction in the prior art
  • FIG. 6 is a schematic diagram of a B frame multiframe reference when performing inter prediction in the prior art
  • Figure 7 is a video image of a region of interest
  • FIG. 8 is a schematic diagram showing the principle of a target-based video encoding method of the present invention.
  • FIG. 9 is a schematic flow chart of a target-based video encoding method of the present invention.
  • FIG. 10 is a schematic diagram showing the principle of combining the target image and the background image of the present invention to form a separate code stream after being separately encoded;
  • FIG. 11 is a schematic diagram showing the principle of inter-frame coding of a target image and a background image in the object-based video encoding method of the present invention
  • Figure 12 is a flow chart of step S3 of the present invention.
  • Figure 13 is a flow chart of step S4 of the present invention.
  • FIG. 14 is a schematic structural diagram of a target-based video coding system of the present invention.
  • FIG. 15 is a schematic structural diagram of a target encoding module of the present invention.
  • Figure 16 is a block diagram showing the structure of the background coding module of the present invention.
  • Intra prediction refers to prediction using only the data of this frame in predictive coding.
  • Inter-frame prediction means that when predictive coding, data of other frames can be used for prediction.
  • Reconstruction of images refers to images that are decoded after encoding and are mainly used for reference.
  • DCT Discrete Cosine Transform, DCT for short
  • ICT integer cosine transform, which is a transformation method developed on the basis of DCT.
  • a group of pictures (GOP), an I-frame and an image set consisting of a refresh P frame and a normal P frame with or without a reference relationship directly or indirectly, and a GOP is a set of consecutive pictures.
  • FIG. 1 is a schematic diagram of a block-based hybrid encoding process in the prior art.
  • the block-based hybrid coding method is: first, the image to be encoded is subjected to block processing, and is divided into 16 ⁇ 16 blocks, which are called macroblocks (the block size of HEVC can be changed, from 8x8 to 64x64, called the maximum coding unit LCU). For each macroblock, first predictive coding is performed, and the reconstructed image of the previous frame or the already encoded portion around the macroblock is used as a reference to obtain the residual data after prediction; then the residual data is spatially transformed and encoded, and DCT or ICT is adopted.
  • macroblocks the block size of HEVC can be changed, from 8x8 to 64x64, called the maximum coding unit LCU.
  • LCU maximum coding unit
  • the residual data is transformed according to different size blocks to obtain transform coefficients in the frequency domain; after the transform coefficients are quantized, they are sent to the entropy coding to obtain the final code stream.
  • the current quantized data needs to be inversely processed, that is, inverse quantized and inverse transformed, and then added to the predicted data to obtain a decoded image, that is, reconstructed image, reconstructed image placed In the reference buffer, the reference image is encoded as the next frame image.
  • FIG. 2 is a schematic diagram of a decoding process of block-based hybrid coding in the prior art.
  • the block-based hybrid coding method is: after the entropy decoding, inverse quantization, and inverse transform, the encoded code stream is added to the predicted image to obtain a decoded image (ie, a video). signal). Finally, the decoded image is stored for use as a reference image for decoding of the next frame.
  • FIG. 3 is a schematic diagram of a process of a block encoding sequence in the prior art.
  • macroblocks are encoded in raster scan order from left to right and top to bottom.
  • FIG. 4 is a schematic diagram of a process of performing predictive coding in the prior art.
  • the coded image may be divided into an I frame (intra prediction frame, Intra), a P frame (inter prediction frame, a Prediction), B frame (bidirectional prediction frame, Bi-Prediction).
  • I frame intra prediction frame, Intra
  • P frame inter prediction frame, a Prediction
  • B frame bidirectional prediction frame, Bi-Prediction
  • I frame intra prediction frame
  • P frame inter prediction frame
  • B frame bidirectional prediction frame
  • FIG. 4 when the I frame is subjected to predictive coding, only the data of the current frame is used for prediction, and when the I frame is decoded, it can be independently decoded without depending on other frames.
  • the P frame is subjected to predictive coding, the reconstructed image of the encoded image of the previous frame is used as a reference.
  • the P frame is decoded, the image of the reference frame must be decoded before decoding.
  • the previous frame and the following frame can be used as reference at the same time to become a bidirectional reference frame.
  • the B frame must be decoded after the previous reference frame and the subsequent reference frame are successfully decoded.
  • the P frame and the B frame can use the frame data as a reference in the same manner as the I frame in coding, and the optimal reference mode can be used for encoding. Since I frames can be decoded independently, they are usually used for random insertion. For example, digital TV requires 1 to 1.5 seconds to insert an I frame, which ensures that the user can see the image as soon as possible when switching channels.
  • I frame compression efficiency is low, and the code rate is relatively large, usually 4 to 10 times or even several times of the P frame.
  • I frame ⁇ P frame ⁇ B frame is usually used.
  • I frame ⁇ P frame ⁇ B frame is usually used.
  • FIG. 5 is a schematic diagram of a P frame multiframe reference when performing inter prediction in the prior art.
  • the previous multiple reconstructed images may be used as the reference frame.
  • the first two frames are used as the reference frame of the P frame.
  • FIG. 6 is a schematic diagram of a B frame multiframe reference when performing inter prediction in the prior art.
  • the forward reference frame of the B frame has two frames, and the backward reference frame is one frame.
  • the first two frames of the image and the next frame are used as the reference frames of the B frame.
  • Multi-frame reference can improve compression efficiency and increase the complexity of the operation.
  • Figure 7 is a video image of a region of interest.
  • the area 1, the area 2, and the area 3 are moving objects or specific target areas (the areas can be referred to as target areas), and other parts are background parts, and the user only pays attention to the three target areas, and the other Part is not very concerned.
  • a block-based hybrid coding video coding method divides an image into coding blocks when encoding, and sequentially encodes them by raster scanning, and does not distinguish which object the coding block belongs to.
  • each coding block is associated with a current frame neighboring block and also with a reference frame block, and these related blocks may not belong to the same object.
  • the entire frame image needs to be decoded to obtain the image of the user's area of interest, and decoding the entire frame image requires a lot of computing resources and memory bandwidth, which takes a lot of time and the retrieval speed is slow.
  • the present invention proposes a target-based video coding method.
  • FIG. 8 is a schematic diagram showing the principle of a target-based video encoding method of the present invention.
  • the present invention adopts a target-based video coding method, which first divides an image to be encoded into a target and a background, and may have multiple targets.
  • each target and background are separate encoding objects, and are independently encoded to generate a corresponding code stream.
  • the code streams generated by each coding object are combined into a separate code stream in order.
  • the target coding and background coding still use the existing coding method, that is, the block-based hybrid coding method.
  • FIG. 9 is a flow chart showing the object-based video encoding method of the present invention.
  • the object-based video coding method of the present invention includes:
  • step S1 an image to be encoded is acquired.
  • the image to be encoded is acquired in real time.
  • step S2 the target image and the background image are extracted based on the image to be encoded.
  • the target image and the background image are extracted from the image to be encoded acquired in real time.
  • step S3 inter prediction encoding is performed based on the target image.
  • the target image and the background image are extracted from the image to be encoded, and the extracted target image and the background image are independently encoded, and the extracted target image may have multiple, as shown in FIG. 8 , the target 1 and Goal 2 is the extracted multiple targets.
  • the existing encoding method that is, the block-based hybrid encoding method, is still adopted.
  • the step S3 further includes the following steps S31-S32.
  • step S4 inter prediction encoding is performed based on the background image.
  • the existing encoding method that is, the block-based hybrid encoding method, is also employed, see FIG.
  • the step S4 further includes the following steps S41-S42.
  • step S3 and step S4 is not unique, and may be performed sequentially or simultaneously.
  • Step S5 combining the target code stream generated by encoding the target image and the background code stream generated by encoding the background image into a single code stream.
  • the separately encoded target image and the background image are combined to form a separate code stream, see FIG.
  • FIG. 10 is a schematic diagram showing the principle of combining the target image and the background image of the present invention to form a separate code stream after being separately encoded.
  • the code streams generated after being separately encoded are combined together in the manner of FIG. 10.
  • the top of the current image corresponding code stream is header information, and the header information includes, but is not limited to, location information of the target area, and reference. Relationship information, etc.; followed by code stream information for each object (ie, the target image and the background image).
  • code stream information for each object ie, the target image and the background image.
  • the code stream information of each target is first arranged, and the code stream information of the background is further arranged. It may also be the code stream information of the background first, and then the code stream information of the target.
  • the code stream information of each object has a start code identifier, and the purpose is to facilitate distinguishing.
  • FIG. 11 is a schematic diagram showing the principle of inter-frame coding of a target image and a background image in the object-based video encoding method of the present invention.
  • the target when encoding a target, the target is equally divided into coding blocks, and each block is encoded.
  • the coding is also divided into intra coding and inter coding, and intra coding uses an existing coding method, that is, a block-based hybrid coding method.
  • the existing coding mode is different from the selection of the reference frame.
  • the corresponding frame of the previous frame is used as the reference frame. 11.
  • Target 1 in image 2 refers to target 1 in image 1
  • target 2 in image 2 refers to target 2 in image 1.
  • the target can be intra-coded.
  • the target hole portion in the background image is first filled (filled with an arbitrary value) to form an entire frame image.
  • a block of the coded block is encountered, skipped directly, and only the coded block other than the padding portion is encoded.
  • Intra coding is similar to existing standards; in interframe coding, the reference picture of the coding block can only come from the background portion of the reference frame, not from the target area.
  • Figure 12 is a flow chart of step S3 of the present invention.
  • step S3 of the present invention further includes the following steps:
  • Step S31 dividing the coded block based on the target image.
  • the target image when encoding the target image, the target image is equally divided into encoding blocks, and each block is encoded.
  • Step S32 When encoding the target image of the current encoded frame, the target region in the previous frame image corresponding to the target region in the current frame image is used as the reference frame for inter prediction encoding.
  • intra-frame coding adopts an existing coding method.
  • the present invention differs from the existing coding method in the selection of the reference frame.
  • the existing coding method uses the entire previous frame as a reference frame, and the present invention corresponds to the previous frame.
  • the target area is used as a reference frame.
  • the target 1 in the image 2 refers to the target 1 in the image 1
  • the target 2 in the image 2 refers to the target 2 in the image 1.
  • the new target image that is, the target 3 is subjected to intra-frame coding processing.
  • Figure 13 is a flow chart of step S4 of the present invention.
  • Step S41 the target area is filled based on the background image to obtain an entire frame image.
  • the target cavity portion in the background image may be filled (filled with any value), as shown in FIG. 8, the target 1 and the target 2 are filled to form Full frame image.
  • the step of filling the target area based on the background image of the step S41 to obtain the entire frame image is not a necessary step, and is preferably used as an auxiliary means to supplement the step S42 below. That is to say, if the background image is not filled in step S41, the subsequent step S42 may be triggered, that is, inter prediction encoding is performed based on the background image.
  • Step S42 When encoding the background image of the current encoded frame, the background region in the previous frame image corresponding to the background region in the current frame image is used as the reference frame for inter prediction encoding.
  • the padding partial coding block is encountered, that is, the target 1 and the target 2 in FIG. 8 are directly skipped, and only the coding block other than the padding part is encoded, that is, only the figure is The background portion of 8 is encoded.
  • the existing coding method is also adopted; when inter-frame coding is performed, the reference picture of the coding block can only come from the background part of the reference frame, but not from the target area, that is, the target and the background do not mutually reference.
  • Figure 14 is a block diagram showing the structure of a target-based video coding system of the present invention.
  • a target-based video coding system includes an acquisition module 1, an extraction module 2, a target encoding module 3, a background encoding module 4, and a code stream composite module 5.
  • the module 1 is obtained for acquiring an image to be encoded. Specifically, the image to be encoded is acquired in real time through the module.
  • the extraction module 2 is connected to the acquisition module 1 for extracting a target image and a background image based on the image to be encoded. Specifically, the target image and the background image are extracted from the image to be encoded acquired in real time, and the extracted target image is sent to the target encoding module 3 for encoding separately, and the extracted background image is sent to the background encoding module 4 for separate coding.
  • the target encoding module 3 is connected to the extraction module 2 for inter-prediction encoding based on the target image. Specifically, the target image transmitted by the extraction module 2 is separately encoded to form a target code stream.
  • intraframe coding an existing coding method is employed; when interframe coding is performed, as shown in FIG. 11, inter prediction coding is performed based on the target image.
  • a background encoding module 4 is coupled to the extraction module 2 for inter-prediction encoding based on the background image. Specifically, the background image sent by the extraction module 2 is separately encoded to form a background code stream. Frame in progress In the case of intra coding, an existing coding method is employed; when inter coding is performed, as shown in FIG. 11, inter prediction coding is performed based on the background image.
  • the code stream combining module 5 is connected to the target encoding module 3 and the background encoding module 4, and is configured to combine the target code stream generated by encoding the target image and the background code stream generated by encoding the background image into a single code stream. Specifically, the target encoding module 3 separately encodes the target image to generate a target code stream, and the background encoding module 4 separately encodes the background image to generate an image code stream, and the code stream combining module 5 performs the generated target code stream and the background code stream. Composite, forming a separate stream.
  • the code stream information is detailed in FIG. 10, and details are not described herein again.
  • Figure 15 is a block diagram showing the structure of the target encoding module 3 of the present invention.
  • the target encoding module 3 further includes an encoding block dividing unit 31 and a target reference unit 32.
  • the coding block dividing unit 31 is configured to divide the coding block based on the target image. Specifically, when the target image is separately encoded, the target image is first divided into coding blocks, and each coding block is encoded.
  • the target reference unit 32 is connected to the encoding block dividing unit 31, and is configured to use a target area in the previous frame image corresponding to the target area in the current frame image as a reference frame when encoding the target image of the current encoded frame. Inter-prediction coding is performed. Specifically, as shown in FIG. 11, the image 2 is the current frame image, and the image 1 is the previous frame image. At this time, when the target 1 in the image 2 is encoded, when the target 1 performs intraframe coding, the existing image is used. The encoding method of the target 1 is the target 1 of the image 1 instead of the entire image 1. Similarly, the reference frame of the target 2 is the target 2 in the image 1, not the entire image 1. Finally, the target 1 is encoded to form the target code stream, and the target 2 is encoded to form the target code stream.
  • the target reference unit 32 further includes a handover coding processing module, configured to perform intra-frame coding processing on the new target image if a new target image appears in the current coding frame.
  • a handover coding processing module configured to perform intra-frame coding processing on the new target image if a new target image appears in the current coding frame.
  • Figure 16 is a block diagram showing the structure of the background coding module 4 of the present invention.
  • the background encoding module 4 further includes a filling unit 41 and a background reference unit 42.
  • the filling unit 41 is configured to fill the target area based on the background image to obtain an entire frame image.
  • the target hole portion in the background image may be filled (arbitrary value) to obtain an entire frame image, and then the entire frame image is encoded.
  • the coded block of the padding part is encountered, skipped directly, and only the coded block other than the padding part is encoded, as shown in FIG.
  • the target hole portion in the background image may not be filled.
  • the target hole portion is directly skipped, and only the coded block other than the target hole portion is encoded.
  • the existing coding method is adopted; when inter-frame coding is performed, the reference image of the background image can only be from the background part of the reference frame, but not from the target area, that is, the target and the background do not mutually reference.
  • the target reference unit 32 is connected to the encoding block dividing unit 31, and is configured to use a target area in the previous frame image corresponding to the target area in the current frame image as a reference frame when encoding the target image of the current encoded frame. Inter-prediction coding is performed. Specifically, as shown in FIG. 8, when the background image is intra-coded, the existing encoding method is used; and when inter-frame encoding is performed, the background image refers only to the background portion of the image of the previous frame, as shown in FIG. Shown, that is, the background portion in image 1, not the entire image 1.
  • the target is 3 is used for intra-frame coding processing, that is, coding is performed using an existing coding method.
  • the target code stream formed by the target 1, the target code stream formed by the target 2, the target code stream formed by the target 3, and the background code stream formed by the background image are combined to form a final code stream.
  • the object-based video coding system of the present invention separately separates the target and the background into different objects, and performs the target code stream and the background code stream formed separately. Composite to form a new code stream.
  • the target code stream information is included in the newly formed code stream, so that when the user searches for the video, only the target code stream needs to be found in the code stream for decoding, thereby improving the video retrieval efficiency.
  • the separate code stream formed by the present invention includes target code stream information, where the target code stream information includes location information of the target area and reference relationship information, so that the user can determine the motion track of the target through the prediction relationship of the target, and expand The applicable range of the encoded code stream.
  • the present invention provides a target-based video encoding method and system.
  • the present invention improves video retrieval efficiency by separately encoding a target and a background into different objects.
  • the target code stream information in the separate code stream of the present invention includes location information and reference relationship information of the target area, so that the user can determine the motion track of the target through the prediction relationship of the target, and expand the application of the coded code stream. range.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

一种基于目标的视频编码方法及系统，所述方法及系统通过从待编码图像中提取目标图像和背景图像，对目标图像和背景图像进行独立编码；在进行帧间编码时，每个目标只参考图像对应目标区域，同时背景只参考图像的背景区域；最后，将目标图像产生的目标码流和背景图像产生的码流按照一定顺序复合在当前图像码流中，通过标识符将不同对象的码流分隔开，同时包含头信息，以记录目标的位置信息和参考关系信息等。能够提高视频检索效率，只需要在码流中找到目标码流进行解码即可。同时可以通过目标的预测关系，确定目标的运动轨迹。

Description

一种基于目标的视频编码方法及系统 技术领域
本发明属于视频图像压缩技术领域，特别涉及一种基于目标的视频编码方法及系统。
背景技术
数字视频压缩标准开始于20世纪80年，经过30多年的发展，现有的标准有国际电信联盟远程通信标准化组织(简称ITU-T，ITU-T for ITU Telecommunication Standardization Sector)系列的H.261、H.263，国际标准化组织(简称ISO、International Organization for Standardization)的MPEG-1、MPEG-4，以及两个组织共同制定的MPEG-2/H.262、H.264/AVC、HEVC。其中HEVC(High Efficiency Video Coding)是一种新的视频压缩标准，于2013年最新发布，可以替代H.264/AVC编码标准。除了上述国际化的编码标准外，还有其他组织的标准，比如国内的AVS、微软的VC-1、谷歌的VP8等。相同的是，这些标准都采用基于块的混合编码方法，融合预测编码、变换编码、以及熵编码3大编码技术。
在视频监控的应用中，通常情况下用户对图像中不同区域的关注程度不一样，对运动的物体、特定目标的关注度高，即目标区域；而对其他背景部分不太关心，即背景区域。例如道路监控中，用户只关注道路上运动车辆等目标，对路面、绿化带等背景不关心。同时，在监控系统中，由于采集点比较多，每个采集点都24小时不间断录像，产生大量的数据，用户不可能实时查看每路视频数据。而是在需要的时候，特别是事故发生时，从录像中查找相关视频，从视频中检索特定的信息，这些信息通常与运动物体相关。
但是，上述提到的现有编码标准将图像等分为编码块后，采用光栅扫描方式依次编码，并不区分编码块属于背景区域或是目标区域。在进行预测编码时，每个编码块与当前帧相邻块相关联，也与参考帧的块相关，而这些相关的块有可能并不属于同一物体。这样，在进行视频检索的时候，需要将整帧图像解码，才能获得用户关注区域的图像，解码整帧图像需要消耗很多计算资源以及内存带宽，花费时间多，检索速度慢。
发明内容
本发明的目的是提供一种基于目标的视频编码方法及系统，本发明从待编码图像中提取目标图像和背景图像，对目标图像和背景图像进行独立编码；最后，将每个对象产生的码流按照一定顺序复合在当前图像码流中，通过标识符将不同对象的码流分隔开，同时图像的码流包含头信息，以记录目标的位置信息和参考关系信息等，能够提高视频检索效率，只需要在码流找到目标码流进行解码即可。同时，可以通过目标的预测关系，确定目标的运动轨迹。
为实现上述目的，本发明的一个方面提供了一种基于目标的视频编码方法，包括，获取待编码图像；基于待编码图像提取目标图像和背景图像；基于目标图像进行帧间预测编码；基于背景图像进行帧间预测编码；将目标图像编码后产生的目标码流和背景图像编码后产生的背景码流，复合成单独码流。
其中，所述基于目标图像进行帧间预测编码的步骤进一步包括：基于目标图像划分编码块；当对当前编码帧的目标图像进行编码时，将与当前帧图像中的目标区域对应的前一帧图像中的目标区域作为参考帧进行帧间预测编码。
其中，所述基于背景图像进行帧间预测编码的步骤进一步包括：当对当前编码帧的背景图像进行编码时，将与当前帧图像中的背景区域对应的前一帧图像中的背景区域作为参考帧进行帧间预测编码。
其中，所述单独码流包括头信息、目标码流信息和背景码流信息；所述头信息包括目标区域的位置信息、参考关系信息；所述目标码流信息和所述背景码流信息包括起始码标识。
其中，所述进行帧间预测编码的步骤还包括：当对当前编码帧的目标图像进行编码时，若当前编码帧中有新的目标图像出现时，将新的目标图像作帧内编码处理。
根据本发明的另一方面，提供了一种基于目标的视频编码系统，包括获取模块、提取模块、目标编码模块、背景编码模块和码流复合模块；获取模块，用于获取待编码图像；提取模块，用于基于待编码图像提取目标图像和背景图像；目标编码模块，用于基于目标图像进行帧间预测编码；背景编码模块，用于基于背景图像进行帧间预测编码；码流复合模块，用于将目标图像编码后产生的目标码流和背景图像编码后产生的背景码流，复合成单独码流。
其中，所述目标编码模块进一步包括：编码块划分单元，用于基于目标图像划分编码块；和目标参考单元，用于当对当前编码帧的目标图像进行编码时，将与当前帧图像中的目标区域对应的前一帧图像中的目标区域作为参考帧进行帧间预测编码。
其中，所述背景编码模块进一步包括：背景参考单元，用于当对当前编码帧的背景图像进行编码时，将与当前帧图像中的背景区域对应的前一帧图像中的背景区域作为参考帧进行帧间预测编码。
其中，所述单独码流包括头信息、目标码流信息和背景码流信息；所述头信息包括目标区域的位置信息、参考关系信息；所述目标码流信息和所述背景码流信息包括起始码标识。
其中，所述目标参考单元还包括：切换编码处理模块，用于若当前编码帧中有新的目标图像出现时，将新的目标图像作帧内编码处理。
如上所述，本发明通过采用将目标和背景分成不同对象进行单独编码的方式，能够有效提高检索效率。将单独编码后的码流按照一定的顺序进行复合形成新的单独码流，使得后期在对视频进行检索时，首只需要在码流找到目标码流进行解码即可。同时，可以通过目标的预测关系，确定目标的运动轨迹。
附图说明
图1是现有技术中基于块的混合编码过程示意图；
图2是现有技术中基于块的混合编码的解码过程示意图；
图3是现有技术中块编码顺序的过程示意图；
图4是现有技术中进行预测编码的过程示意图；
图5是现有技术中进行帧间预测时的P帧多帧参考示意图；
图6是现有技术中进行帧间预测时的B帧多帧参考示意图；
图7是存在感兴趣区域的视频图像；
图8是本发明的基于目标的视频编码方法原理示意图；
图9是本发明的基于目标的视频编码方法的流程示意图；
图10是本发明的目标图像和背景图像进行单独编码后复合形成单独码流的原理示意图；
图11是本发明的基于目标的视频编码方法中目标图像和背景图像进行帧间编码的原理示意图；
图12是本发明的步骤S3的流程图；
图13是本发明的步骤S4的流程图；
图14是本发明的基于目标的视频编码系统的结构示意图；
图15是本发明的目标编码模块的结构示意图；
图16是本发明的背景编码模块的结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚明了，下面结合具体实施方式并参照附图，对本发明进一步详细说明。应该理解，这些描述只是示例性的，而并非要限制本发明的范围。此外，在以下说明中，省略了对公知结构和技术的描述，以避免不必要地混淆本发明的概念。
为了更清楚的描述本发明，首先对涉及到的专业术语进行解释：
帧内预测，是指在预测编码时，只采用本帧的数据做预测。
帧间预测，是指在预测编码时，可以采用其它帧的数据做预测。
重建图像，是指编码后又解码出来的图像，主要用于参考。
DCT/ICT：DCT(Discrete Cosine Transform，简称DCT变换)是离散余弦变换，一种变换编码方式；ICT是整数余弦变换，是在DCT的基础上发展出来的一种变换方式。
图像集合(Group of Pictures，简称GOP)，I帧及其后与其直接或间接存在参考关系的刷新P帧、普通P帧组成的一个图像集合，一个GOP就是一组连续的画面。
图1是现有技术中基于块的混合编码过程示意图。
如图1所示，现有技术中，基于块的混合编码方法是：将要编码的图像先进行分块处理，分成16x16的块，称作宏块(Macroblock)(HEVC的块大小可以变化，从8x8到64x64，叫做最大编码单元LCU)。针对每个宏块，首先进行预测编码，利用前面一帧重建图像或者宏块周围已经编码部分做参考，获得预测后的残差数据；然后对残差数据进行空间的变换编码，采用DCT或者ICT按不同大小块对残差数据进行变换，得到频域中的变换系数；变换系数经过量化后，送到熵编码中，获得最终的码流。为了有效地对下一帧图像进行编码，当前量化后的数据需要经过反向处理，也就是反量化、反变换，再与预测数据相加获得解码出来的图像，也就是重建图像，重建图像放在参考缓存中，作为下一帧图像编码的参考图像。
图2是现有技术中基于块的混合编码的解码过程示意图。
如图2所示，现有技术中，基于块的混合编码方法是：编码码流经过熵解码、反量化、反变换后，再与预测的图像进行相加就得到了解码的图像(即视频信号)。最后将解码出的图像进行存储，以作下一帧解码的参考图像使用。
图3是现有技术中块编码顺序的过程示意图。
如图3所示，在现有技术中进行编码时，宏块按照从左至右、从上至下的光栅扫描顺序进行编码。
图4是现有技术中进行预测编码的过程示意图。
如图4所示，现有技术中，在进行预测编码时，根据预测数据来源不同，可以把编码图像分成I帧(帧内预测帧、Intra)、P帧(帧间预测帧、Prediction)、B帧(双向预测帧、Bi-Prediction)。如图4所示，I帧在进行预测编码时，只采用本帧的数据进行预测，I帧在解码时，可以独立解码，不依赖其他帧。P帧在进行预测编码时，采用前面一帧已编码图像的重建图像作为参考，P帧在解码时，必须等参考帧的图像解码完成后才能解码。B帧在进行预测编码时，可以同时采用前面帧和后面帧作参考，成为双向参考帧，B帧在解码时，必须等前面参考帧和后面参考帧都解码成功后才能解码。此外，P帧、B帧在编码时除了用其它帧作参考，也可以像I帧一样用本帧数据作参考，在编码时，选择最优的参考方式进行编码即可。由于I帧可以独立进行解码，通常将其用作随机插入使用，比如数字电视要求1～1.5秒插入I帧，这样能够保证用户切换频道时，能够尽快看到图像。但这样做同时也带来一个问题，就是I帧压缩效率低，码率比较大，通常是P帧的4～10倍，甚至几十倍。就压缩效率来说，通常情况下I帧<P帧<B帧，就运算复杂度来说，通常情况下I帧<P帧<B帧。
图5是现有技术中进行帧间预测时的P帧多帧参考示意图。
P帧在进行帧间预测时，可以采用前面多个重建图像做参考帧，如图5所示，在编码第2个P帧时，采用前面两帧图像做P帧的参考帧。
图6是现有技术中进行帧间预测时的B帧多帧参考示意图。
如图6所示，B帧在进行帧间预测时，B帧的前向参考帧有两帧，后向参考帧为一帧。如图6所示，在编码第B帧时，采用前面两帧图像和后面一帧作B帧的参考帧。多帧参考可以提高压缩效率，同时也会增加运算的复杂度。
图7是存在感兴趣区域的视频图像。
如图7所示，图像中区域1、区域2、区域3为运动物体或特定目标区域(可以称这些区域为目标区域)，其他部分都是背景部分，用户只关注这3个目标区域，其他部分不是很关心。
现有技术中，基于块的混合编码的视频编码方法，在进行编码时，将图像等分为编码块，采用光栅扫描方式依次编码，并不区分编码块属于哪个物体中。在进行预测编码时，每个编码块与当前帧相邻块相关联，也与参考帧的块相关，而这些相关的块有可能并不属于同一物体。这样，在进行监控视频的检索时，需要将整帧图像解码，才能获得用户关注区域的图像，解码整帧图像需要消耗很多计算资源以及内存带宽，花费时间多，检索速度慢。基于上述缺陷，本发明提出了基于目标的视频编码方法。
图8是本发明的基于目标的视频编码方法原理示意图。
如图8所示，本发明采用基于目标的视频编码方法，首先将待编码的图像分成目标和背景，目标可以有多个。在编码时，每个目标以及背景都是单独的编码对象，进行独立编码产生对应的码流。最后将每个编码对象产生的码流按照顺序复合成单独码流。目标编码和背景编码仍然采用现有的编码方式，即基于块的混合编码方法。
图9是本发明的基于目标的视频编码方法的流程示意图。
如图9所示，本发明的基于目标的视频编码方法包括：
步骤S1，获取待编码图像。
在本步骤中，实时获取需要编码的图像。
步骤S2，基于待编码图像提取目标图像和背景图像。
在本步骤中，从实时获取的待编码图像中，提取出目标图像和背景图像。
步骤S3，基于目标图像进行帧间预测编码。
在本步骤中，从待编码图像中提取出目标图像和背景图像，对提取出的目标图像和背景图像进行独立编码，提取出的目标图像可以有多个，如图8所示，目标1和目标2是提取出的多个目标。其中，在对目标图像进行编码时，仍然采用现有的编码方法，即基于块的混合编码方法。
所述步骤S3进一步包括下面的步骤S31-S32。
步骤S4，基于背景图像进行帧间预测编码。
在本步骤中，在对提取出的背景图像进行编码时，同样采用现有的编码方法，即基于块的混合编码方法，参见图8。
所述步骤S4进一步包括下面的步骤S41-S42。
这里，步骤S3和步骤S4的执行顺序不唯一，可以先后执行或同时执行。
步骤S5，将目标图像编码后产生的目标码流和背景图像编码后产生的背景码流，复合成单独码流。
在本步骤中，将单独进行编码的目标图像和背景图像进行复合，形成单独码流，参见图10。
图10是本发明的目标图像和背景图像进行单独编码后复合形成单独码流的原理示意图。
如图10所示，将单独编码后产生的码流按照图10的方式复合在一起，当前图像对应码流的最前面是头信息，所述头信息包括但不限于目标区域的位置信息、参考关系信息等；随后是每个对象(即目标图像和背景图像)的码流信息。可选的，可以是先排每个目标的码流信息，再排背景的码流信息。也可以是先排背景的码流信息，再排目标的码流信息。其中，每个对象的码流信息都有起始码标识，目的是便于区分。
图11是本发明的基于目标的视频编码方法中目标图像和背景图像进行帧间编码的原理示意图。
如图11所示，在对目标进行编码时，首先将目标等分成编码块，对每个块进行编码。编码同样分成帧内编码和帧间编码，帧内编码采用现有的编码方式，即基于块的混合编码方法。而在帧间编码时，与现有的编码方式在参考帧的选择上有所不同，并不是将前一帧图像整个作为参考帧，而是将前一帧对应目标区域作为参考帧，参见图11，图像2中的目标1参考图像1中的目标1，图像2中的目标2参考图像1中的目标2。当然，如果当前编码帧有新的目标出现，在前一帧并没有对应的目标，比如图11中的目标3，这时可以将该目标做帧内编码处理。
参见图11，在对背景进行编码时，首先将背景图像中的目标空洞部分进行填充(填充任意值)以形成整帧图像。在编码的过程中，遇到填充部分编码块，直接跳过，只对填充部分之外的编码块进行编码。帧内编码与现有标准类似；而在帧间编码时，编码块的参考图像只能来自参考帧的背景部分，而不能来自目标区域。
图12是本发明步骤S3的流程图。
本发明的前述步骤S3进一步包括下述步骤：
步骤S31，基于目标图像划分编码块。
具体的，在对目标图像进行编码时，将目标图像等分成编码块，对每个块进行编码。
步骤S32，当对当前编码帧的目标图像进行编码时，将与当前帧图像中的目标区域对应的前一帧图像中的目标区域作为参考帧进行帧间预测编码。
具体的，在编码时，同样分成帧内编码和帧间编码，帧内编码采用现有的编码方法。而本发明在帧间编码时，与现有的编码方法在参考帧的选择上有所不同，现有的编码方法是将前一帧图像整个作为参考帧，而本发明是将前一帧对应目标区域作为参考帧，如图11所示，图像2中的目标1参考图像1中的目标1，图像2中的目标2参考图像1中的目标2。当然，如果当前编码帧有新的目标出现，在前一帧图像中并没有对应的目标，如图11中的目标3，此时，将新的目标图像即目标3，作帧内编码处理。
图13是本发明步骤S4的流程图。
如图13所示，本发明的前述步骤进一步S4包括下述步骤：
步骤S41，基于背景图像将目标区域填充以得到整帧图像。
在本步骤中，在对背景图像进行编码时，优选的，可以将背景图像中的目标空洞部分进行填充(填充任意值)，如图8所示，将目标1和目标2进行填充，以形成整帧图像。
本实施例中，步骤S41的基于背景图像将目标区域填充以得到整帧图像不是必需执行步骤，优选的作为一种辅助手段来作为下文的步骤S42的补充。也就是说，如果步骤S41中未对所述背景图进行填充，则可能触发后面的步骤S42，即基于背景图像进行帧间预测编码。
步骤S42，当对当前编码帧的背景图像进行编码时，将与当前帧图像中的背景区域对应的前一帧图像中的背景区域作为参考帧进行帧间预测编码。
具体的，在对背景图像的进行编码时，遇到填充部分编码块，即图8中的目标1和目标2，直接跳过，只对填充部分之外的编码块进行编码，即只对图8中的背景部分进行编码。在进行帧内编码时，同样采用现有的编码方法；而在进行帧间编码时，编码块的参考图像只能来自参考帧的背景部分，而不能来自目标区域，即目标和背景间不互相参考。
图14是本发明的基于目标的视频编码系统的结构示意图。
如图14所示，本发明另一实施方式的基于目标的视频编码系统包括获取模块1、提取模块2、目标编码模块3、背景编码模块4和码流复合模块5。
获取模块1，用于获取待编码图像。具体来说，通过该模块实时获取需要编码的图像。
提取模块2与所述获取模块1连接，用于基于待编码图像提取目标图像和背景图像。具体来说，从实时获取的待编码图像中提取出目标图像和背景图像，将提取出的目标图像发送给目标编码模块3进行单独编码，将提取出的背景图像发送给背景编码模块4进行单独编码。
目标编码模块3与所述提取模块2连接，用于基于目标图像进行帧间预测编码。具体来说，将提取模块2发送来的目标图像进行单独编码，形成目标码流。在进行帧内编码时，采用现有的编码方法；在进行帧间编码时，如图11所示，基于目标图像进行帧间预测编码。
背景编码模块4与所述提取模块2连接，用于基于背景图像进行帧间预测编码。具体来说，将提取模块2发送来的背景图像进行单独编码，形成背景码流。在进行帧 内编码时，采用现有的编码方法；在进行帧间编码时，如图11所示，基于背景图像进行帧间预测编码。
码流复合模块5与所述目标编码模块3和所述背景编码模块4连接，用于将目标图像编码后产生的目标码流和背景图像编码后产生的背景码流，复合成单独码流。具体来说，目标编码模块3对目标图像进行单独编码产生目标码流，背景编码模块4对背景图像进行单独编码产生图像码流，码流复合模块5将产生的目标码流和背景码流进行复合，形成单独码流。所述码流信息详见图10，此处不再赘述。
图15是本发明的目标编码模块3的结构示意图。
如图15所示，在本发明的一可选实施方式中，目标编码模块3进一步包括编码块划分单元31和目标参考单元32。
编码块划分单元31，用于基于目标图像划分编码块。具体来说，在对目标图像进行单独编码时，首先将目标图像划分成编码块，再对每个编码块进行编码。
目标参考单元32与所述编码块划分单元31连接，用于当对当前编码帧的目标图像进行编码时，将与当前帧图像中的目标区域对应的前一帧图像中的目标区域作为参考帧进行帧间预测编码。具体来说，如图11所示，图像2是当前帧图像，图像1是前一帧图像，此时在对图像2中的目标1进行编码时，目标1进行帧内编码时，采用现有的编码方法；而在进行帧间编码时，目标1的参考帧为图像1的目标1，而不是整个图像1。同理，目标2的参考帧是图像1中的目标2，而不是整个图像1。最后，目标1经过编码形成目标码流，目标2经过编码也形成目标码流。
另外，所述目标参考单元32还包括切换编码处理模块，用于若当前编码帧中有新的目标图像出现时，将新的目标图像作帧内编码处理。如图11中的目标3，在对图像2进行编码时，出现新的目标3，此时将目标3作帧内编码处理。
图16是本发明的背景编码模块4的结构示意图。
如图16所示，在本发明的一可选实施方式中，背景编码模块4进一步包括填充单元41和背景参考单元42。
填充单元41，用于基于背景图像将目标区域填充以得到整帧图像。具体来说，在对提取模块2提取出的目标图像进行单独编码时，可以将背景图像中的目标空洞部分进行填充(任意值)以得到整帧图像，然后对该整帧图像进行编码，在编码的过程中，遇到填充部分的编码块，直接跳过，只对填充部分之外的编码块进行编码，参见图8。 此外，也可以不对背景图像中的目标空洞部分进行填充，在后续基于背景图像进行帧间预测编码时，遇到该目标空洞部分直接跳过，只对目标空洞部分之外的编码块进行编码。在进行帧内编码时，采用现有的编码方法；而在进行帧间编码时，背景图像的参考图像只能是来自参考帧的背景部分，而不能来自目标区域，即目标和背景间不互相参考。
目标参考单元32与所述编码块划分单元31连接，用于当对当前编码帧的目标图像进行编码时，将与当前帧图像中的目标区域对应的前一帧图像中的目标区域作为参考帧进行帧间预测编码。具体来说，如图8所示，背景图像在进行帧内编码时，采用现有的编码方法；而在进行帧间编码时，背景图像只参考前一帧图像的背景部分，如图11所示，即图像1中的背景部分，而不是整个图像1。
另外，如图11所示，如果在对当前帧图像进行编码时，出现了新的目标图像，在前一帧图像并没有对应的目标，如图11中的目标3，此时，将该目标3作帧内编码处理，即采用现有的编码方法进行编码。
如图8所示，最后，将目标1形成的目标码流、目标2形成的目标码流、目标3形成的目标码流以及背景图像形成的背景码流，进行复合形成最终的码流。
如上所述，详细介绍了本发明的基于目标的视频编码系统，所述基于目标的视频编码系统将目标和背景分成不同对象进行单独编码，将单独编码后形成的目标码流和背景码流进行复合形成新的码流。所述新形成的码流中包目标码流信息，使得用户在对视频进行检索时，只需要在码流中找到目标码流进行解码即可，提高了视频检索效率。同时，本发明形成的单独码流中包含目标码流信息，所述目标码流信息中包括目标区域的位置信息和参考关系信息，使得用户可以通过目标的预测关系，确定目标的运动轨迹，扩大了所述编码码流的适用范围。
如上所述，本发明提供了一种基于目标的视频编码方法和系统，本发明通过将目标和背景分成不同对象进行单独编码，提高了视频检索效率。同时，本发明的单独码流中的目标码流信息中包括目标区域的位置信息和参考关系信息，使得用户可以通过目标的预测关系，确定目标的运动轨迹，扩大了所述编码码流的适用范围。
应当理解的是，本发明的上述具体实施方式仅仅用于示例性说明或解释本发明的原理，而不构成对本发明的限制。因此，在不偏离本发明的精神和范围的情况下所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。此外，本发明所附权利要求旨在涵盖落入所附权利要求范围和边界、或者这种范围和边界的等同形式内的全部变化和修改例。

Claims (10)

  1. 一种基于目标的视频编码方法，所述方法包括：
    获取待编码图像；
    基于待编码图像提取目标图像和背景图像；
    基于目标图像进行帧间预测编码；
    基于背景图像进行帧间预测编码；
    将目标图像编码后产生的目标码流和背景图像编码后产生的背景码流，复合成单独码流。
  2. 根据权利要求1所述的方法，其中，所述基于目标图像进行帧间预测编码的步骤进一步包括：
    基于目标图像划分编码块；
    当对当前编码帧的目标图像进行编码时，将与当前帧图像中的目标区域对应的前一帧图像中的目标区域作为参考帧进行帧间预测编码。
  3. 根据权利要求1所述的方法，其中，所述基于背景图像进行帧间预测编码的步骤进一步包括：
    当对当前编码帧的背景图像进行编码时，将与当前帧图像中的背景区域对应的前一帧图像中的背景区域作为参考帧进行帧间预测编码。
  4. 根据权利要求1所述的方法，其中，所述单独码流包括头信息、目标码流信息和背景码流信息；
    所述头信息包括目标区域的位置信息、参考关系信息；
    所述目标码流信息和所述背景码流信息包括起始码标识。
  5. 根据权利要求2所述的方法，其中，所述进行帧间预测编码的步骤还包括：
    当对当前编码帧的目标图像进行编码时，若当前编码帧中有新的目标图像出现时，将新的目标图像作帧内编码处理。
  6. 一种基于目标的视频编码系统，其中，包括:
    获取模块(1)，用于获取待编码图像；
    提取模块(2)，用于基于待编码图像提取目标图像和背景图像；
    目标编码模块(3)，用于基于目标图像进行帧间预测编码；
    背景编码模块(4)，用于基于背景图像进行帧间预测编码；
    码流复合模块(5)，用于将目标图像编码后产生的目标码流和背景图像编码后产生的背景码流，复合成单独码流。
  7. 根据权利要求6所述的系统，其中，所述目标编码模块(3)进一步包括：
    编码块划分单元(31)，用于基于目标图像划分编码块；和
    目标参考单元(32)，用于当对当前编码帧的目标图像进行编码时，将与当前帧图像中的目标区域对应的前一帧图像中的目标区域作为参考帧进行帧间预测编码。
  8. 根据权利要求6所述的系统，其中，所述背景编码模块(4)包括：
    背景参考单元(42)，用于当对当前编码帧的背景图像进行编码时，将与当前帧图像中的背景区域对应的前一帧图像中的背景区域作为参考帧进行帧间预测编码。
  9. 根据权利要求6所述的系统，其中，所述单独码流包括头信息、目标码流信息和背景码流信息；
    所述头信息包括目标区域的位置信息、参考关系信息；
    所述目标码流信息和所述背景码流信息包括起始码标识。
  10. 根据权利要求7所述的系统，其中，所述目标参考单元(32)还包括：
    切换编码处理模块，用于若当前编码帧中有新的目标图像出现时，将新的目标图像作帧内编码处理。
PCT/CN2015/077730 2015-04-08 2015-04-28 一种基于目标的视频编码方法及系统 Ceased WO2016161675A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510164535.7A CN106162191A (zh) 2015-04-08 2015-04-08 一种基于目标的视频编码方法及系统
CN201510164535.7 2015-04-08

Publications (1)

Publication Number Publication Date
WO2016161675A1 true WO2016161675A1 (zh) 2016-10-13

Family

ID=57073044

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/077730 Ceased WO2016161675A1 (zh) 2015-04-08 2015-04-28 一种基于目标的视频编码方法及系统

Country Status (2)

Country Link
CN (1) CN106162191A (zh)
WO (1) WO2016161675A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108073690A (zh) * 2017-11-30 2018-05-25 江西洪都航空工业集团有限责任公司 一种视频目标检索方法
CN112004114A (zh) * 2020-08-31 2020-11-27 广州市百果园信息技术有限公司 视频处理方法、系统、装置、可读存储介质及电子设备
CN112073722A (zh) * 2020-07-23 2020-12-11 西安万像电子科技有限公司 图像处理方法、装置、设备及存储介质
CN112434186A (zh) * 2020-11-23 2021-03-02 杭州海康威视数字技术股份有限公司 数据处理方法、装置、电子设备及存储介质
CN113691818A (zh) * 2021-08-25 2021-11-23 深圳龙岗智能视听研究院 视频目标检测方法、系统、存储介质、计算机视觉终端
CN113965749A (zh) * 2020-12-14 2022-01-21 深圳市云数链科技有限公司 静态摄像机视频传输方法及系统

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108243338A (zh) * 2016-12-23 2018-07-03 杭州海康威视数字技术股份有限公司 视频解码、编码方法、装置及系统
CN110881140B (zh) * 2018-09-05 2022-01-21 北京高德云图科技有限公司 从视频中截取图片的方法、装置及嵌入式设备、存储介质
CN110267041B (zh) 2019-06-28 2021-11-09 Oppo广东移动通信有限公司 图像编码方法、装置、电子设备和计算机可读存储介质
CN112449198B (zh) * 2019-09-02 2023-09-29 珠海格力电器股份有限公司 视频编码方法、装置以及计算设备、存储介质
CN113556551B (zh) * 2020-04-23 2023-06-23 上海高德威智能交通系统有限公司 一种编码、解码方法、装置及设备
CN112954343A (zh) * 2021-01-20 2021-06-11 西安万像电子科技有限公司 编解码方法、设备及系统
CN112954346A (zh) * 2021-01-20 2021-06-11 西安万像电子科技有限公司 编解码方法、设备及系统
CN112954345A (zh) * 2021-01-20 2021-06-11 西安万像电子科技有限公司 编解码方法、设备及系统
CN112995713A (zh) * 2021-03-02 2021-06-18 广州酷狗计算机科技有限公司 视频处理方法、装置、计算机设备及存储介质
WO2022261838A1 (zh) * 2021-06-15 2022-12-22 Oppo广东移动通信有限公司 残差编码和视频编码方法、装置、设备和系统
CN116112675B (zh) * 2023-04-11 2023-06-09 深圳市海威恒泰智能科技有限公司 一种视频编码方法和视频编码系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1313706A (zh) * 2000-03-15 2001-09-19 汤姆森多媒体公司 用于编码视频图象的方法和装置
US6625316B1 (en) * 1998-06-01 2003-09-23 Canon Kabushiki Kaisha Image processing apparatus and method, and image processing system
CN101505413A (zh) * 2009-02-16 2009-08-12 中国科学院长春光学精密机械与物理研究所 与电视跟踪器相结合的智能图像压缩装置
CN103391438A (zh) * 2013-07-19 2013-11-13 哈尔滨工程大学 一种高光谱图像压缩编码方法及装置
CN104427337A (zh) * 2013-08-21 2015-03-18 杭州海康威视数字技术股份有限公司 基于目标检测的感兴趣区域视频编码方法及其装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101610411B (zh) * 2009-07-16 2010-12-08 中国科学技术大学 一种视频序列混合编解码的方法和系统
US9626769B2 (en) * 2009-09-04 2017-04-18 Stmicroelectronics International N.V. Digital video encoder system, method, and non-transitory computer-readable medium for tracking object regions
CN103402087A (zh) * 2013-07-23 2013-11-20 北京大学 一种基于可分级位流的视频编解码方法
CN103475882B (zh) * 2013-09-13 2017-02-15 北京大学 监控视频的编码、识别方法和监控视频的编码、识别系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6625316B1 (en) * 1998-06-01 2003-09-23 Canon Kabushiki Kaisha Image processing apparatus and method, and image processing system
CN1313706A (zh) * 2000-03-15 2001-09-19 汤姆森多媒体公司 用于编码视频图象的方法和装置
CN101505413A (zh) * 2009-02-16 2009-08-12 中国科学院长春光学精密机械与物理研究所 与电视跟踪器相结合的智能图像压缩装置
CN103391438A (zh) * 2013-07-19 2013-11-13 哈尔滨工程大学 一种高光谱图像压缩编码方法及装置
CN104427337A (zh) * 2013-08-21 2015-03-18 杭州海康威视数字技术股份有限公司 基于目标检测的感兴趣区域视频编码方法及其装置

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108073690A (zh) * 2017-11-30 2018-05-25 江西洪都航空工业集团有限责任公司 一种视频目标检索方法
CN112073722A (zh) * 2020-07-23 2020-12-11 西安万像电子科技有限公司 图像处理方法、装置、设备及存储介质
CN112073722B (zh) * 2020-07-23 2024-05-17 西安万像电子科技有限公司 图像处理方法、装置、设备及存储介质
CN112004114A (zh) * 2020-08-31 2020-11-27 广州市百果园信息技术有限公司 视频处理方法、系统、装置、可读存储介质及电子设备
CN112434186A (zh) * 2020-11-23 2021-03-02 杭州海康威视数字技术股份有限公司 数据处理方法、装置、电子设备及存储介质
CN113965749A (zh) * 2020-12-14 2022-01-21 深圳市云数链科技有限公司 静态摄像机视频传输方法及系统
CN113691818A (zh) * 2021-08-25 2021-11-23 深圳龙岗智能视听研究院 视频目标检测方法、系统、存储介质、计算机视觉终端
CN113691818B (zh) * 2021-08-25 2023-06-30 深圳龙岗智能视听研究院 视频目标检测方法、系统、存储介质、计算机视觉终端

Also Published As

Publication number Publication date
CN106162191A (zh) 2016-11-23

Similar Documents

Publication Publication Date Title
WO2016161675A1 (zh) 一种基于目标的视频编码方法及系统
TWI558178B (zh) 用於寫碼視訊資料之方法及器件及電腦可讀儲存媒體
CN102055982B (zh) 三维视频编解码方法及装置
CN103650505B (zh) 视频译码中的运动向量预测
JP5475464B2 (ja) ビデオ信号処理方法及び装置
CN105637862B (zh) 用于视频信息的可缩放译码的装置和方法
CN110495169A (zh) 视频图像处理方法与装置
KR102252319B1 (ko) 시간적 스케일러빌러티를 갖는 비디오 부호화 방법 및 장치, 시간적 스케일러빌러티를 갖는 비디오 복호화 방법 및 장치
HUE029350T2 (en) Low latency video buffering in video encoding
KR20120080122A (ko) 경쟁 기반의 다시점 비디오 부호화/복호화 장치 및 방법
TW201342882A (zh) 在三維視訊寫碼中信號傳輸視圖合成預測支援
EP2536143A1 (en) Method and a digital video encoder system for encoding digital video data
EP2090113A1 (en) Method of and apparatus for video encoding and decoding based on motion estimation
KR20150105434A (ko) 3d 비디오에서의 뷰 합성
WO2016161674A1 (zh) 一种视频图像压缩和读取的方法、装置及系统
US8630345B2 (en) Method and apparatus for video interprediction encoding /decoding
KR101676788B1 (ko) 멀티코어 시스템 기반 비디오 병렬 복호화 방법 및 그 장치
KR20120083200A (ko) 깊이 영상 부호화/복호화 장치 및 방법
US8098946B2 (en) Apparatus and method for image encoding and decoding using prediction
KR102942193B1 (ko) Gdr 또는 irap 픽처에 대한 가용 슬라이스 타입 정보에 기반하는 영상 부호화/복호화 방법, 장치 및 비트스트림을 저장한 기록 매체
US20130156114A1 (en) Data Movement Reduction In Video Compression Systems
KR20220160043A (ko) 혼성 nal 유닛 타입에 기반하는 영상 부호화/복호화 방법, 장치 및 비트스트림을 저장하는 기록 매체
RU2820148C2 (ru) Оборудование кодирования и декодирования изображений и оборудование для передачи данных для изображения
RU2806784C1 (ru) Способ и устройство кодирования/декодирования изображений на основе смешанного типа nal-единицы и способ для передачи потока битов
KR102960117B1 (ko) 레이어간 참조 여부에 기반하여 서브 레이어를 결정하는 영상 부호화/복호화 방법, 장치 및 비트스트림을 전송하는 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15888230

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205N DATED 29/03/2018)

122 Ep: pct application non-entry in european phase

Ref document number: 15888230

Country of ref document: EP

Kind code of ref document: A1