KR20200113477A - Method and Apparatus for Processing Image in Compressed Domain - Google Patents

Abstract

The present embodiment relates to a method and a device for processing an image in a compressed area capable of reducing the usage amount of a storage space while improving the quality of a video summary by storing meaningful pictures at intervals suitable for human perception by analyzing a video stream transmitted from a CCTV camera in the compressed area to detect the movements of objects and based on the same, generate a thumbnail image for the video summary. The device for processing an image comprises: an input buffer part; an extraction part; an analysis part; and a movement detection part.

Description

Method and Apparatus for Processing Image in Compressed Domain}

The present embodiment relates to an image processing method and apparatus in a compressed region. More specifically, it relates to an image processing method and apparatus for more efficiently generating image summary information for a very long image such as a CCTV image.

The content described in this section merely provides background information on the present embodiment and does not constitute the prior art.

1 is an exemplary diagram illustrating a configuration of a general video stream. Referring to FIG. 1, a video stream has an IDR picture of a predetermined period, and a unit of a picture from an IDR picture to a next IDR picture is called a GOP (Group of Picture). Although the IDR picture is larger in size than the b/p picture, there is an advantage in that it is easy to store because one complete picture can be composed of one picture. On the other hand, since the size is large, there is a disadvantage that a large amount of bandwidth is used during network transmission and a large amount of space is occupied during storage.

FIG. 2 is a diagram illustrating a procedure of generating a thumbnail for summarizing an image from a conventional video stream. In the conventional case, when a thumbnail for summarizing an image is generated from a video stream, it has been used in a method of storing an IDR picture periodically included in a video stream. However, this method is not suitable for storing very long video streams such as CCTV images. In particular, storing a large number of sessions based on the cloud appears to be a very large waste of space. That is, since a video stream of a CCTV image has a wide interval between IDR pictures, thumbnails are generated at long intervals for human perception in the case of a conventional method. In addition, since a large number of pictures that are periodically generated are also meaningless pictures, there is a limitation in that a large amount of storage space is wasted when storing a very long time such as a CCTV.

Therefore, when storing the video stream for the video stream transmitted from the CCTV camera and simultaneously storing the thumbnail for video summary, meaningful thumbnail images are frequently stored to improve the quality of video summary and reduce the use of storage space. It requires new skills to enable.

In this embodiment, a video stream transmitted from a CCTV camera is analyzed in a compression area to detect the motion of an object, and based on this, a thumbnail image for image summary is generated, so that a meaningful picture is stored at an appropriate interval for human perception. Accordingly, an object of the present invention is to provide a method and apparatus for processing an image in a compressed region capable of reducing the amount of storage space while improving the quality of an image summary.

The present embodiment includes: an input buffer unit receiving an encoded video stream of an image; An extractor configured to analyze the encoded video stream in a compressed domain to extract motion vector information and DCT coefficients in the encoded video stream; An analysis unit that provides an analysis result obtained by analyzing the motion vector information and the DCT coefficient; And a motion detector configured to detect a motion of the object in the image based on an analysis result of at least one of the motion vector information and the DCT coefficient.

In addition, according to another aspect of the present embodiment, there is provided a process of receiving an encoded video stream of an image; Analyzing the encoded video stream in a compression region and extracting motion vector information and DCT coefficients in the encoded video stream; Providing an analysis result obtained by analyzing the motion vector information and the DCT coefficient; And detecting a motion of the object in the image based on an analysis result of at least one of the motion vector information and the DCT coefficient.

Further, according to another aspect of the present embodiment, there is provided a computer program stored in a computer-readable recording medium in order to execute each step of the image processing method according to claim 12.

According to the present embodiment, a video stream transmitted from a CCTV camera is analyzed in a compression region to detect the motion of an object, and based on this, a thumbnail image for image summary is generated, so that an appropriate interval for a person to recognize a meaningful picture. By storing it as an image, it is possible to reduce the amount of storage space while improving the quality of the video summary.

1 is an exemplary diagram illustrating a configuration of a general video stream.
2 is a diagram for describing a procedure of generating a thumbnail for summarizing an image from a conventional video stream.
3 is a block diagram schematically illustrating an image processing apparatus according to the present embodiment.
4 is a block diagram schematically illustrating an execution unit of the image processing apparatus according to the present embodiment.
5 is a flowchart illustrating an image processing method according to the present embodiment.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings.

Recently, the number of applications that summarize and display images in the form of thumbnails are increasing. The length of the image to be summarized varies, and very long images such as CCTV images are greatly affected by usability and storage space depending on the storage method. The video stream of CCTV video has different characteristics from the general video stream. That is, in order to reduce network bandwidth and storage space, the GOP unit is widely used, and the frame ray is decreased, thereby forming a large interval between IDR pictures. For example, in the case of 60 GOP and 15 fps, IDR pictures are included once every 4 seconds. On the other hand, if the fps is low and the GOP is high, like a video stream of a CCTV video, IDR is included once every few seconds, so there is a high possibility that a scene where a thumbnail is important will be missed.

In addition, when storing a video stream and a thumbnail image for video summary at the same time, when periodically extracting an image from the video stream, all pictures from the video must be decoded, which consumes a lot of CPU resources. As images continue to be stored, storage space is consumed very much.

On the other hand, since the motion vector information of the video stream of the CCTV video has the background still and the motion information of the moving object, if the motion vector is detected, this is likely to be meaningful information. Due to this, in the case of this embodiment, it is meaningful by analyzing the video stream transmitted from the CCTV camera in the compression region to detect the motion of the object through the motion vector, and creating a thumbnail image for the image summary based on this. A method is proposed that allows pictures to be stored at appropriate intervals for human perception to improve the quality of video summary while reducing the amount of storage space used.

3 is a block diagram schematically illustrating an image processing apparatus according to the present embodiment.

As shown in FIG. 3, the image processing apparatus 300 according to the present embodiment includes an input buffer unit 310, an execution unit 320, and an image processing unit 330. Here, components included in the image processing apparatus 300 according to the present embodiment are not necessarily limited thereto. That is, the case of FIG. 3 exemplarily shows only essential components for detecting motion of an object through motion vector extraction from a compressed region and generating a thumbnail image for image summary based on the motion vector. It should be appreciated that 300) may have more or less components than those shown or configurations of different components for the implementation of other functions.

The input buffer unit 310 performs a function of buffering a video stream of an image generated from an image capturing device. On the other hand, in this embodiment, the video capture device may preferably be an intelligent monitoring device such as a CCTV device. Accordingly, the video stream buffered by the input buffer unit 310 according to the present embodiment has a wide GOP unit. It is used, has a low frame rate, and thus the interval of IDR pictures can be formed large.

The input buffer unit 310 according to the present embodiment may receive a video stream encoded according to a preset compression standard from an image capturing apparatus. For example, the input buffer unit 310 may receive a video stream encoded according to a compression standard such as H.264/H.265 based on motion compensation in units of blocks.

Meanwhile, in this embodiment, a b/p picture that is not an IDR picture in an encoded video stream must be decoded from the leading IDR picture to obtain a raw image, and then can be compressed into a jpeg/png image. . Accordingly, the input buffer unit 310 may receive the coded video stream buffered in GOP (Group of Pictures) units defined by an Instantaneous Decoder Refresh (IDR) picture from the image capturing apparatus.

The execution unit 320 performs a function of detecting a meaningful scene from the encoded video stream. On the other hand, in the present embodiment, in the case of an image generated from an image capturing apparatus, the background is stopped as a specific area is designated and monitored, and thus, the variation of the image is small. Therefore, when an object in the image moves, it can be classified as a meaningful scene.

Due to this, the execution unit 320 according to the present embodiment detects the motion of an object by analyzing the encoded video stream, and provides a result thereof. Later, in the image processing process of the image processing unit 330, Allow video summaries to be made on important events.

The execution unit 320 may detect motion vector information in the encoded video stream and detect motion of an object based on this. That is, in the present embodiment, since the background of motion vector information in the encoded video stream is stationary and has motion information on the moving object, if a motion vector is detected, this is likely to be meaningful information.

The execution unit 320 according to the present embodiment analyzes the encoded video stream in a compressed domain. That is, the execution unit 320 does not decompress the encoded video stream, but analyzes the region of the motion vector in the encoded state to detect whether motion has occurred. In general, in the case of a media stream, a lot of CPU consumption occurs in the decompression process, but in the case of the present embodiment, it is possible to reduce this by analyzing in a compressed state.

In addition, in the present embodiment, the execution unit 320 may additionally detect and utilize Discrete Cosine Transform (DCT) coefficients in the encoded video stream in order to improve the accuracy of motion detection of an object through motion vector information. Meanwhile, DCT coefficient is an abbreviation of Discrete Cosine Transform and refers to a method of reducing the amount of data by expressing n data as the sum of n cosine functions. This method basically starts from Fourier's hypothesis, that is, the Fourier transform that n values in the time domain can be represented by n trigonometric functions in the frequency domain. In MPEG, which is an international standard image compression method, an image is encoded in units of 8x8, that is, 64 blocks, and the expression of 64 data in 64 cosine functions is called DCT, and the coefficients of the transformed cosine functions are DCT coefficients. It is called. Using these DCT coefficients, it is possible to extract the difference between the frames before and after the video.

In this embodiment, the execution unit 320 utilizes the DCT coefficient as a verification parameter in verifying the validity of the detected motion vector information.

Hereinafter, a detailed operation of the execution unit 320 according to the present embodiment will be described with reference to FIG. 4. Meanwhile, FIG. 4A is a diagram illustrating a procedure for detecting motion of an object in a pixel area. FIG. 4B is a block diagram schematically showing the execution unit 320 according to the present embodiment, and is a diagram illustrating a motion detection procedure of an object through motion vector information in a compressed region.

First, referring to FIG. 4A, when motion detection of an object in a pixel area is performed, a task that consumes a lot of resources such as reorder, IDCT, and motion compensation must be performed. In addition, in the case of motion detection of an object in a pixel area, a lot of resources are consumed because calculations are required for all pixels.

On the other hand, referring to (b) of FIG. 4, in the case of this embodiment, as the motion of the object is detected in the compressed region, the movement while skipping the part that requires a large amount of computation (the dotted line mark in FIG. 4(a)) Since it can be detected, there is an effect of reducing the waste of resources.

As shown in (b) of FIG. 4, the execution unit 320 according to the present embodiment includes an extraction unit 400, an analysis unit 410, and a motion detection unit 420. Here, the components included in the execution unit 320 are not necessarily limited thereto.

The extraction unit 400 performs a function of analyzing the encoded video stream in the compression region and extracting motion vector information and DCT coefficients in the encoded video stream. The operation of the extraction unit 400 corresponds to a process of extracting only necessary information for motion detection by decoding a partial region of the encoded video stream.

To this end, the extraction unit 400 according to the present embodiment may be implemented as an entropy decoder configured to entropy decode an encoded video stream. That is, the extraction unit 400 classifies a case where the encoded video stream is encoded based on context-adaptive variable-length coding (CAVLC) and a case that is encoded based on context-adaptive binary arithmetic coding (CABAC), and Motion vector information and DCT coefficients may be extracted by performing different decoding methods, respectively.

The analysis unit 410 receives and analyzes motion vector information and DCT coefficients extracted through the extraction unit 400 and performs a function of providing an analysis result for each. That is, in this embodiment, the analysis unit 410 may be implemented by including a first analysis unit (motion vector analysis) 412 and a second analysis unit (DCT coefficient analysis).

The first analysis unit 412 analyzes motion vector information and provides an analysis result thereof. In this embodiment, the first analysis unit 412 may correct motion vector information based on an analysis result of the motion vector information. Meanwhile, the motion vector information has vector information that finds the most similar macroblock by comparing it with the previous image in units of macroblocks. This could be 16×16, 8×8, 4×4, etc. When motion vector information is generated in this way, motion may have been detected, but it must be corrected for various reasons. That is, motion vector information is found through a motion estimation process. In this case, since a block advantageous for compression is found, the motion vector information may not coincide with the actual motion. In addition, in the case of a block that processes by performing intra prediction in its own block rather than generating motion vector information, motion vector information is not generated. In addition, there may be a case of erroneously finding motion vector information in an area that is usually pattern uniform or has few features.

Due to this, the first analysis unit 412 adaptively applies a motion vector to a specific block that performs processing based on intra prediction among neighboring blocks of the block matching the motion vector information through analysis of the motion vector information. Information can be given. This corresponds to a correction process to improve the accuracy of motion detection within an object. For example, the first analysis unit 412 may provide motion vector information for a specific block based on motion vector information of a neighboring block. .

The first analysis unit 412 transmits the corrected motion vector information and the analysis result to the motion detection unit 420, through which the motion detection unit 420 performs a motion detection procedure.

The second analysis unit 414 analyzes the DCT coefficient and provides the analysis result. The DCT coefficient is a coefficient generated after discrete cosine transformation of a pixel value for a difference value from a previous screen for a macroblock, and can be used for two purposes.

First, the DCT coefficient may be used as a variable value for detecting motion of an object separately from motion vector information. That is, if motion is detected, there will be a change in the DCT coefficient, and if this change amount exceeds a specific threshold, it can be said that the motion is detected. To this end, the second analysis unit 414 may provide the amount of change in the DCT coefficient as the analysis result.

In addition, the DTC coefficient may be used to verify the validity of motion vector information analyzed and corrected by the first analysis unit 212. That is, since the approximate actual pattern of the image can be known through the DCT coefficient, it is possible to determine whether motion vector information is valid through this. The second analysis unit 214 transmits the analysis result of the DCT coefficient to the motion detection unit 420, and through this, the motion detection unit 420 determines the effectiveness of motion vector information through the analysis result of the DCT coefficient. To determine.

The motion detection unit 420 performs a function of detecting a motion of an object in an image based on an analysis result of at least one of motion vector information and DCT coefficients provided through the analysis unit 210.

In this embodiment, the motion detection unit 420 may determine whether the motion of the object is detected by integrating the motion vector information and the analysis result of the DCT coefficient. That is, the motion detection unit 420 performs correction by reflecting the analysis result of the DCT coefficient in the analysis result of the motion vector information.

In this case, the motion detection unit 420 may determine the validity of the motion vector information by using the analysis result of the DCT coefficient, and detect the motion of the object according to the determination result. For example, the motion detection unit 420 may perform motion when a weight reflection value reflecting a predefined weight for an analysis result of motion vector information and an analysis result of a DCT coefficient corresponding to the motion vector information exceeds a preset threshold. It is determined that the vector information is valid, and that the motion of the object is detected. Conversely, when the weight reflection value is less than a preset threshold, the motion detection unit 420 determines that the motion vector information is not valid, and determines that the motion of the object is not detected.

In another embodiment, the motion detection unit 420 may calculate a change amount of the DCT coefficient based on an analysis result of the DCT coefficient and detect the motion of the object based on the calculated change amount. For example, the motion detection unit 420 determines that the motion of the object is detected when the calculated change amount exceeds a preset threshold.

In another embodiment, the motion detection unit 420 may receive a result of recognizing a face or a person from the image capturing device, and detect the motion of the object based on the received result.

The image processing unit 330 generates image summary information based on the object motion detection result of the motion detection unit 420. In the present embodiment, the image summary information is preferably a thumbnail image, but is not limited thereto. The image processing unit 330 may include a video decoder, an image resize, an image decoder, and first to second processing units (Black Hole). Here, since the operation of each component included in the image processing unit 330 is the same as in the related art, a detailed description will be omitted.

The image processing unit 330 stores, in a table, a timestamp for a scene in which motion of an object in an image is detected based on the object motion detection result.

The image processing unit 330 generates image summary information by storing an image corresponding to a specific time point based on the stored time stamp. For example, the image processing unit 330 decodes the coded video stream, re-encodes a time including a time corresponding to the time stamp in the retrofitted video stream, and then stores it as an image.

On the other hand, the image processing unit 330 further combines information on a reference image (ex: IDR picture) in the decoded video stream when the scene in which the motion of the object in the image is detected is a b/p picture other than an IDR picture. Re-encoding is performed. This has the effect of enabling decoding of the video summary information without decoding the reference video when the video summary information is subsequently searched.

The image processing unit 330 may change the generation period of the image summary information as necessary. For example, if the image processing unit 330 generates an image too often, it consumes a lot of CPU and storage space, so that it is stored at the most appropriate interval. For example, if the video stream is 15fps, the interval of pictures is 67ms. When saving as an image, storing about 1 picture per second can save storage space without being qualitatively awkward for humans to perceive.

5 is a flowchart illustrating an image processing method according to the present embodiment.

The image processing apparatus 300 receives an encoded video stream of an image generated from an image capturing apparatus (S502). In step S502, the image processing apparatus 300 may receive the encoded video stream buffered in GOP units defined by the IDR picture from the image capturing apparatus.

The image processing apparatus 300 analyzes the encoded video stream in step S502 and extracts motion vector information and DCT coefficients in the encoded video stream (S504). In step S504, the image processing apparatus 300 extracts a region of a motion vector and a DCT coefficient in the encoded state without decompressing the encoded video stream. That is, the image processing apparatus 300 may perform entropy decoding on the encoded video stream, and through this, decode a partial region of the encoded video stream, and extract motion vector information and DCT coefficients.

The image processing apparatus 300 analyzes the motion vector information and DCT coefficients extracted in step S504 to generate an analysis result (S506). In step S506, the image processing apparatus 300 may correct motion vector information based on an analysis result of the motion vector information. For example, the image processing apparatus 300 may adaptively provide motion vector information to a specific block that performs processing based on intra prediction among neighboring blocks of a block matching the motion vector information through analysis of motion vector information. have.

The image processing apparatus 300 detects a motion of an object in the image based on at least one analysis result of the analysis result of the motion vector information generated in step S506 and the analysis result of the DCT coefficient (S508). In step S508, the image processing apparatus 300 may determine the validity of the motion vector information by using the analysis result of the DCT coefficient, and detect the motion of the object according to the determination result.

In addition, the image processing apparatus 300 may calculate a change amount of the DCT coefficient based on an analysis result of the DCT coefficient, and detect the motion of the object based on the calculated change amount.

The image processing apparatus 300 generates image summary information based on the motion detection result in step S508 (S510). The image processing apparatus 300 stores a timestamp for a scene in which motion of an object in an image is detected in a table based on the motion detection result of step S508, and stores an image corresponding to a specific point in time including the time of the stored timestamp. Save to generate video summary information.

Here, since steps S502 to S510 correspond to the operation of each component of the image processing apparatus 300 described above, further detailed descriptions are omitted.

In FIG. 5, it is described that each process is sequentially executed, but is not limited thereto. In other words, since the process described in FIG. 5 may be changed and executed or one or more processes may be executed in parallel, FIG. 5 is not limited to a time series order.

As described above, the image processing method illustrated in FIG. 5 is a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) that is implemented as a program and can be read using software of a computer. Can be written on.

The above description is merely illustrative of the technical idea of the present embodiment, and those of ordinary skill in the technical field to which the present embodiment belongs will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present exemplary embodiments are not intended to limit the technical idea of the present exemplary embodiment, but are illustrative, and the scope of the technical idea of the present exemplary embodiment is not limited by these exemplary embodiments. The scope of protection of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

300: image processing device 310: input buffer unit
320: execution unit 330: image processing unit
400: extraction unit 410: analysis unit
420: motion detection unit

Claims (13)

An input buffer unit receiving an encoded video stream of an image;
An extractor configured to analyze the encoded video stream in a compressed domain to extract motion vector information and DCT coefficients in the encoded video stream;
An analysis unit that provides an analysis result obtained by analyzing the motion vector information and the DCT coefficient; And
A motion detection unit that detects motion of an object in the image based on the analysis result of at least one of the motion vector information and the DCT coefficient
Image processing apparatus comprising a. The method of claim 1,
The input buffer unit,
An image processing apparatus, characterized in that receiving an encoded video stream based on a compression standard based on motion compensation in units of blocks. The method of claim 1,
The input buffer unit,
An image processing apparatus comprising: receiving the encoded video stream buffered in GOP (Group of Pictures) units defined by an Instantaneous Decoder Refresh (IDR) picture. The method of claim 1,
The analysis unit includes a first analysis unit for analyzing the motion vector information and a second analysis unit for analyzing the DCT coefficient,
The first analysis unit adaptively assigns motion vector information to a specific block that performs processing based on intra prediction among neighboring blocks of the block matching the motion vector information through analysis of the motion vector information. An image processing device made into. The method of claim 1,
The motion detection unit,
And determining the validity of the motion vector information by using the analysis result of the DCT coefficient, and detecting the motion of the object according to the determination result. The method of claim 5,
The motion detection unit,
When the motion vector information and the weight reflection value reflecting the predefined weight for the analysis result of the DCT coefficient corresponding to the motion vector information exceeds a preset threshold, it is determined that the motion vector information is valid. Image processing device. The method of claim 1,
The motion detection unit,
And calculating a change amount of the DCT coefficient based on an analysis result of the DCT coefficient, and detecting a motion of the object based on the change amount. The method of claim 1,
And an image processing unit that generates image summary information based on the detection result of the motion detection unit. The method of claim 8,
The image processing unit,
And generating the image summary information by storing a timestamp for a scene in which the motion of the object in the image is detected, and storing an image corresponding to a specific point in time based on the timestamp. The method of claim 9,
The image processing unit,
And storing the image as the image after decoding the encoded video stream, re-encoding a time including a time corresponding to the time stamp in the decoded video stream. The method of claim 10,
The image processing unit,
And performing the re-encoding by additionally combining information on a reference image in the decoded video stream. Receiving an encoded video stream of an image;
Analyzing the encoded video stream in a compression region and extracting motion vector information and DCT coefficients in the encoded video stream;
Providing an analysis result obtained by analyzing the motion vector information and the DCT coefficient; And
The process of detecting the motion of the object in the image based on the analysis result of at least one of the motion vector information and the DCT coefficient
Image processing method comprising a. A computer program stored in a computer-readable recording medium to execute each step of the image processing method according to claim 12.

Images