KR102061915B1 - syntax-based method of providing object classification for compressed video - Google Patents

Abstract

The present invention relates to a technology which effectively performs object classification from a compressed image such as H.264 AVC and H.265 HEVC. In particular, unlike known technology, in which the existence of an object is recognized through a complex image processing to classify the type of the object with respect to a compressed image generated by a CCTV camera, according to the present invention, syntax information (a motion vector, a coding type and the like), which is acquired after parsing compressed image data, is utilized in order to extract an area in the image in which a significant movement exists, in other words, a moving object area, before allowing a deep neural network to learn a motion vector pattern of the moving object area to utilize the same for object classification. Therefore, according to the present invention, object classification can be effectively performed from a CCTV compressed image without going through a complex processing such as decoding, downscale resizing, differential image acquisition, and image analysis. In addition, because object classification can be performed by 1/10 of the calculation amount of the known technology, the number of accommodating channels of an image analysis server can be increased by approximately 10 times.

Description

Syntax-based method of providing object classification for compressed video

The present invention generally relates to a technique for effectively performing object classification from compressed images such as H.264 AVC and H.265 HEVC.

More specifically, the present invention, for example, the syntax information obtained by parsing the compressed image data rather than recognizing the existence of the object and classifying the object type through the complex image processing, for example, for the compressed image generated by the CCTV camera Motion vector, coding type) to extract a region of significant motion in the image, that is, a moving object region, and to learn the motion vector pattern of the moving object region in a deep neural network, and to apply it to object classification. .

Recently, it is common to build a video control system using CCTV for crime prevention and security. With multiple CCTV cameras installed by region, images generated by these CCTV cameras are displayed on a monitor and stored in a storage device. When a control agent finds a scene where a crime or accident occurs, he or she immediately responds appropriately and, if necessary, retrieves the image stored in the storage to secure post evidence.

By the way. The reality is that the number of control personnel is very low compared to the installation status of CCTV cameras. In order to effectively perform video surveillance with such limited number of people, simply displaying CCTV images on the monitor screen is not enough. It is preferable to process the object to be effectively detected by detecting the motion of an object present in each CCTV image and displaying something in the corresponding area in real time. In this case, the monitoring personnel do not monitor the entire CCTV image with uniform interest, but monitor the CCTV image centering on the part where the object moves.

In particular, if the video control system can not only automatically detect the movement of the object in the CCTV image, but also classify the object (eg, a person, a car, an animal, or a flag), the video control efficiency can be further improved. Not only can you selectively monitor certain types of objects, but you can also quickly detect ex post evidence from large amounts of CCTV video data stored in storage.

Recently installed CCTV cameras adopt high resolution (e.g. Full HD) and high frame rate (e.g. 24 frames per second), so that H.264 AVC and H.265 HEVC take into account the burden of network bandwidth and storage space. High compression ratio complex image compression technology such as is being adopted. The CCTV camera device provides a compressed image generated by encoding the captured image according to image compression technology, and the side utilizing the CCTV image decodes the compressed image in reverse according to the corresponding technical standard. Therefore, in order to recognize an object in a CCTV image to which image compression technology is applied and to classify it as a kind of object, for example, a person, a car, an animal, a flag, a leaf, conventionally, a compressed image is decoded to reproduce a compressed image, In other words, it was necessary to process the image for content analysis after obtaining the original uncompressed image.

Hereinafter, a process of performing object classification from a CCTV compressed image in the prior art will be described with reference to FIGS. 1 and 2.

1 is a block diagram illustrating a general configuration of a video decoding apparatus according to the H.264 AVC Technical Standard. Referring to FIG. 1, a video decoding apparatus according to H.264 AVC may include a parser 11, an entropy decoder 12, an inverse converter 13, a motion vector operator 14, a predictor 15, and deblocking. And a filter 16. These hardware modules sequentially process the data of the compressed image to decompress the compressed image and restore the original image data. At this time, the parser 11 parses the motion vector and the coding type for the coding unit of the compressed image. Such a coding unit is generally an image block such as a macroblock or a subblock.

2 is a flowchart illustrating a process of performing object classification from CCTV images in the existing image analysis solution. Referring to FIG. 2, in the prior art, the compressed video is decoded according to H.264 AVC and H.265 HEVC, etc. (S10), and the frame images of the playback video are downscaled to a small image, for example, 320x240 (S20). ). At this time, the reason for downsizing resizing is to reduce the processing burden in the subsequent image analysis process. Then, after obtaining differential images of the resized frame images, the moving object is extracted through image analysis (S30). Then, the object classification is achieved by recognizing the moving object through image content analysis of the series of frame images (S40).

In the prior art, to extract and classify moving objects, compressed image decoding, downscale resizing, and image analysis are performed. These are very complicated processes, and therefore, in a conventional video control system, the capacity of a single video analysis server can be processed at a time is quite limited. Currently, the maximum CCTV channels that a high performance video analytics server can cover are typically up to 16 channels. Since a large number of CCTV cameras were installed, a number of video analysis servers were required for the video control system, which caused problems of increased cost and difficulty in securing physical space.

An object of the present invention is generally to provide a technique for effectively performing object classification from compressed images such as H.264 AVC and H.265 HEVC.

In particular, an object of the present invention is syntax information (eg, motion) obtained by parsing compressed image data instead of recognizing object existence and classifying object types through complex image processing, for example, for compressed images generated by CCTV cameras. By using vector and coding type), it is to provide a technique for extracting an area where there is significant motion in an image, that is, a moving object area, and learning the motion vector pattern of the moving object area in a deep neural network, and then using it for object classification. .

In order to achieve the above object, a syntax-based object classification method for a compressed image may include: a first step of parsing a bitstream of the compressed image to obtain a motion vector and a coding type for a coding unit; A second step of obtaining a motion vector cumulative value for a predetermined time for each of the plurality of image blocks constituting the compressed image; A third step of comparing a motion vector cumulative value with a first threshold value for a plurality of image blocks; A fourth step of marking an image block having a motion vector accumulation value exceeding a first threshold as a moving object region; A fifth step of collecting a motion vector pattern for each moving object region; A sixth step of performing deep neural network learning by setting a motion vector pattern of a moving object region as a training data set; And a seventh step of obtaining a result of object classification by inputting a motion vector pattern for a specific moving object region set as an object classification object to a deep neural network.

In this case, the motion vector pattern of the moving object region may include a two-dimensional array of direction components and size components of the motion vector obtained in the first step with respect to the plurality of image blocks belonging to the moving object region.

In this case, the sixth step may include receiving an object classification reference for the moving object region; And performing deep neural network learning by inputting a combination of a motion vector pattern and an object classification reference to the deep neural network for the moving object region.

In addition, the syntax-based object classification method for a compressed image according to the present invention, a step of identifying a plurality of adjacent image blocks (hereinafter referred to as 'neighbor block') around the moving object area; B) comparing the motion vector values obtained in the first step with respect to the plurality of neighboring blocks with a second preset threshold; C) additionally marking a neighboring block having a motion vector value exceeding a second threshold value as a result of the comparison of step b among the plurality of neighboring blocks as a moving object region; D) additionally marking a neighboring block having a coding type of an intra picture among the plurality of neighboring blocks as a moving object region; The method may further include an e-step of performing interpolation on the plurality of moving object regions to additionally mark the moving object region with a predetermined number of non-marked image blocks surrounded by the moving object region.

In addition, the syntax-based object classification method for a compressed image according to the present invention, a step of identifying a plurality of adjacent image blocks (hereinafter referred to as 'neighbor block') around the moving object area; B) comparing the motion vector cumulative value with respect to the plurality of neighboring blocks with a second threshold preset to a value less than the first threshold; C) additionally marking, as a moving object region, a neighboring block having a motion vector accumulation value exceeding a second threshold value as a result of the comparison of step b of the plurality of neighboring blocks; D) additionally marking a neighboring block having a coding type of an intra picture among the plurality of neighboring blocks as a moving object region; The method may further include an e-step of performing interpolation on the plurality of moving object regions to additionally mark the moving object region with a predetermined number of non-marked image blocks surrounded by the moving object region.

On the other hand, the computer program according to the present invention is stored in a medium in combination with hardware to execute the syntax-based object classification method for the compressed image as described above.

According to the present invention, there is an advantage in that object classification can be effectively performed from CCTV compressed images without complex processing such as decoding, downscale resizing, difference image acquisition, image analysis, and the like. In particular, it is possible to perform object classification with a calculation amount of about 1/10 compared with the prior art, which has an advantage of increasing the number of receiving channels of the image analysis server by approximately 10 times or more.

1 is a block diagram showing a general configuration of a video decoding apparatus.
2 is a flowchart showing a process of performing object classification from a CCTV compressed image in the prior art.
FIG. 3 is a flowchart illustrating the entire process of performing object classification on a syntax-based basis from a compressed image according to the present invention. FIG.
4 is a diagram illustrating a concept of classifying an object from a motion vector pattern of a moving object region through a deep neural network in the present invention.
FIG. 5 is a flowchart illustrating an example of a process of detecting an effective motion region from a compressed image in the present invention. FIG.
6 is a diagram illustrating an example of a result of applying an effective motion region detection process to a CCTV compressed image.
7 is a flowchart illustrating an example of a process of detecting a boundary region for a moving object region in the present invention.
FIG. 8 is a diagram illustrating an example of a result of applying a boundary region detection process to the CCTV image of FIG. 6. FIG.
FIG. 9 is a diagram illustrating an example of a result of arranging a moving object region through interpolation for the CCTV image of FIG. 8. FIG.
10 is a diagram showing a motion vector pattern superimposed on the area of the moving object detected from the CCTV image.
FIG. 11 is a diagram illustrating only a motion vector pattern for a moving object region in FIG. 10. FIG.
12 to 14 are enlarged views of a motion vector pattern for three moving object regions in FIG. 11.

Hereinafter, with reference to the drawings will be described in detail the present invention.

3 is a flowchart illustrating an entire process of classifying an object based on a syntax from a compressed image according to the present invention, and FIG. 4 is a classifying an object from a motion vector pattern of a moving object region through a deep neural network in the present invention. It is a figure which shows the concept to make.

The object classification process according to the present invention may be performed by an image analysis server in a system for handling a plurality of compressed images, for example, a CCTV image control system or a CCTV image analysis system. In addition, the object classification process of the present invention uses a deep neural network, which may be utilized by implementing a deep neural network inside an image analysis server, or using an open API that is implemented in an external cloud server. It can also be used through the Program Interface.

In order to perform object classification, it is necessary to first identify a part of the compressed image as a moving object. In the present invention, a moving object region is quickly extracted through syntax information of each image block by parsing a bitstream of the compressed image without having to decode the compressed image. As an image block, any one or a combination of macro blocks and sub blocks may be adopted, and as motion information, a motion vector and a coding type are preferable. Do. The moving object region thus obtained does not accurately reflect the boundary of the moving object existing in the image as confirmed in the various images attached to the present specification, but has a high processing speed and high reliability.

However, the present invention is conceptually different from the prior art in that the present invention is a method of extracting a block of an image block that is estimated to include a moving object, rather than extracting a moving object. Since image analysis was not performed, object classification should be performed without information on image contents.

Accordingly, in the present invention, the deep neural network is trained using the motion vector pattern formed in the moving object region as a training data set. After training a deep neural network using multiple training datasets, we apply it to object classification. That is, in the step of performing object classification, for example, the motion vector pattern is input to the deep neural network for a specific mobile object area designated by the controller or a plurality of mobile object areas detected in the CCTV image, and the operation result is examined by looking at the operation result. It is to classify what kind of object is the object area.

Meanwhile, according to the present invention, the moving object region can be extracted and the object classification can be performed without decoding the compressed image. However, the apparatus or software to which the present invention is applied should not perform an operation of decoding a compressed image, but the scope of the present invention is not limited.

Hereinafter, a concept of a process of performing object classification from a compressed image according to the present invention will be described with reference to FIG. 3.

Step S100: First, an effective motion that can substantially recognize meaning is detected from the compressed image based on the motion vector of the compressed image, and the image region in which the effective motion is detected is set as the moving object region.

To this end, the motion vector and coding type of a coding unit of a compressed image are parsed according to a video compression standard such as H.264 AVC and H.265 HEVC. In this case, the size of the coding unit is generally about 64x64 pixels to 4x4 pixels and may be variously set according to a designer's selection.

The motion vectors are accumulated for a predetermined time period (for example, 500 msec) for each image block, and it is checked whether the motion vector accumulation value exceeds the first threshold value (for example, 20). If such an image block is found, it is considered that an effective motion is found in the image block, and then marked as a moving object area. Accordingly, even if the motion vector is generated, if the cumulative value for a predetermined time does not exceed the first threshold, the image change is estimated to be insignificant and ignored.

Step S200: The moving object areas detected in the previous step S100 are examined based on the motion vector and the coding type to extend the boundaries of the moving object areas. Through this process, the mobile object region detected in the form of the fragmented image block in step S100 is connected to each other, thereby obtaining a significant lump shape.

In the previous step (S100), by selecting the image blocks according to the strict judgment criteria, the image block that seems to correspond to the moving object in the compressed image is detected and marked as the moving object area. In this step (S200), other image blocks located around the image block marked as the moving object area in S100 are examined. In the present specification, these are referred to as 'neighborhood blocks'. For these neighboring blocks, it is determined whether or not it corresponds to the moving object region according to a criterion that is relatively relaxed compared to the criterion applied in S100.

Macroblocks and subblocks are very small in compressed video. Therefore, if the image of people, cars, bicycles, animals, etc., such as CCTV images, the moving object is difficult to appear in only one image block, it is expected to appear over several image blocks. That is, it is expected that the image block existing near the image block on which the moving object is taken is relatively more likely to have the moving object captured than the image block. Reflecting such a technical idea (S200), it is determined whether or not it corresponds to the moving object area according to a relatively relaxed judgment criterion for the neighboring block existing around the moving object area.

Preferably, each neighboring block is inspected, and if the motion vector value detected in the current frame is greater than or equal to a preset second threshold (eg, 0) or the coding type is an intra picture, the corresponding video block is also a moving object. Mark the area. In another exemplary embodiment, when the motion vector cumulative value calculated in operation S100 for the neighboring block is equal to or greater than the second threshold value (eg, 5) or the coding type is an intra picture, the corresponding image block may also be marked as a moving object region. Can be. At this time, it is logically reasonable that the second threshold is set to a smaller value than the first threshold.

Conceptually, if an effective block is found and there is some movement in the vicinity of the moving object area, it is marked as a moving object area because it is likely to be a mass with the previous moving object area. In addition, in the case of an intra picture, since a motion vector does not exist, it is impossible to determine whether a motion object region is based on the motion vector. Therefore, if the intra picture is located adjacent to the image block already detected as the moving object region, it is assumed that the intra picture forms a mass together with the extracted moving object region. This is because the loss when one image block other than the moving object area is included in the moving object area is not very large, whereas the loss when the moving object area is fragmented is large.

Step S300: The interpolation is applied to the moving object areas detected at S100 and S200 to clean up the fragmentation of the moving object area. In the above process, since it is determined whether the moving object area is the image block unit, even though it is actually a moving object (for example, a person), there is an image block that is not marked as the moving object area in the middle. The phenomenon of dividing into may occur. Accordingly, if there is one or a few unmarked image blocks surrounded by a plurality of image blocks marked with the moving object region, they additionally mark the moving object region. Through this, it is possible to make the mobile object region divided into several groups into one. The influence of such interpolation is clearly seen when comparing [FIG. 8] and [FIG. 9].

At least one region of moving object is obtained from the compressed image through the above processes (S100) to (S300). The obtained moving object region is shown in blue in FIG. 9 and is a lump in which a plurality of image blocks marked as belonging to the moving object region in a series of processes are connected to each other. In each of the steps (S100) to (S300) it is determined whether or not to belong to the moving object region in the image block unit, but finally, the chunks of the image blocks formed by agglomeration are treated as the moving object region. Such a moving object region is a portion estimated to include one or more moving objects based on the syntax information of the compressed image. It is preferable to assign and manage a unique ID to each mobile object area for software processing. Referring to FIG. 10, three moving object regions were detected from the CCTV compressed image, and unique IDs of 001, 002, and 003 were allocated to them.

Step S400: A pattern of motion vectors is collected for each moving object region obtained through the above process as shown in FIG. 10. The moving object region is composed of a plurality of image blocks, and is implemented in a two-dimensional array in which the direction component and the size component of the motion vector obtained in step S200 are arranged for each image block constituting the moving object region. Can be. FIG. 10 is a diagram illustrating a motion vector pattern superimposed on a moving object region detected in a CCTV image image, and FIG. 11 is a diagram showing only a motion vector pattern for a moving object region in FIG. 10. 12] to [14] are enlarged views of a motion vector pattern for three moving object areas (Unique ID = 001, 002, and 003) in FIG.

In these figures, in order to intuitively represent the concept of a motion vector pattern, a direction component and a size component of a motion vector acquired in a corresponding video frame are visually displayed for each image block. 12 and 14, the motion vector patterns of the three moving object regions are compared and shown to show a significant difference in the size of the motion vectors, uniformity of the direction of the motion vectors, and arrangement of the motion vectors. have. Such a difference is related to the morphological characteristics and the movement characteristics of the moving object included in the moving object area.

Steps S500 and S600: Examine a case in which the deep neural network is trained by determining an operation mode of the deep neural network. In this case, the deep vector network training is performed by setting the motion vector patterns for the plurality of moving object regions acquired in step S400 as a training dataset. Supervised learning and unsupervised learning are the learning methods of the deep neural network. In the case of CCTV video control, the supervised learning method is preferable because the result of object classification through the deep neural network is generally set clearly. Do.

In the case of the supervised learning method, step S600 is performed for each moving object area after receiving an object classification reference which is guide information on what kind of object (eg, person, car, animal, flag). The combination of the motion vector pattern and the object classification reference for the moving object region may be input to the deep neural network to perform deep neural network learning.

Steps (S500, S700): Next, the operation mode of the deep neural network to determine the case of applying the deep neural network will be described. In this case, the motion vector pattern for the specific mobile object region set as the object classification object is input to the deep neural network, and an object classification result for the mobile object region is obtained from the calculation result of the deep neural network. One or more moving object areas designated by the controller may be set as the object classification object, or all moving object areas sequentially identified in the CCTV compressed image may be sequentially set as the object classification object.

The motion vector pattern obtained in step S400 is input to the deep neural network with respect to the moving object region that is an object classification object, and the operation result value outputted by the deep neural network corresponds to the object classification result of the corresponding mobile object region. For example, when the deep neural network outputs '001' as an operation result value, this means that it is determined that 'car' is included in the corresponding mobile object area. As described above, the present invention can classify an object without decoding the compressed image and analyzing the image.

Step S800: The combination of the moving object region and the object classification result thereof is provided to the image control apparatus, which may use the object classification result in various forms. For example, different colors may be designated and displayed for each object classification on the CCTV monitor screen. In addition, for example, when conducting an image search for the purpose of securing a crime resolution clue, the scope of the search target population can be reduced by limiting the search target to a specific classification from the large-scale CCTV photographed images stored in the storage. You can speed it up.

FIG. 5 is a flowchart illustrating an example of a process of detecting an effective movement region from a compressed image in the present invention, and FIG. 6 is a result of applying an effective movement region detection process to a CCTV compressed image. It is a figure which shows an example. The process of FIG. 5 corresponds to step S100 in FIG.

Step S110: First, a coding unit of a compressed image is parsed to obtain a motion vector and a coding type. Referring to FIG. 1, a video decoding apparatus performs parsing (header parsing) and motion vector calculation on a stream of a compressed video according to a video compression standard such as H.264 AVC and H.265 HEVC. Through this process, the motion vector and coding type are parsed for the coding unit of the compressed image.

Step S120: Acquire a motion vector cumulative value for a preset time (for example, 500 ms) for each of the plurality of image blocks constituting the compressed image.

This step is presented with the intention to detect if there are effective movements that are practically meaningful from the compressed image, such as driving cars, running people, and fighting crowds. Shaky leaves, ghosts that appear momentarily, and shadows that change slightly due to light reflections are not detected because they are moving but practically meaningless objects.

To this end, a motion vector cumulative value is obtained by accumulating the motion vectors in units of one or more image blocks for a predetermined time period (for example, 500 msec). In this case, the image block is used as a concept including a macroblock and a subblock.

Steps S130 and S140: Comparing a motion vector cumulative value with respect to a plurality of image blocks with a preset first threshold value (eg, 20) and moving the image block having a motion vector cumulative value exceeding the first threshold value. Mark with

If an image block having a predetermined motion vector accumulation value is found as described above, it is regarded that something significant motion, that is, effective motion, is found in the image block, and then marked as a moving object region. For example, in a video surveillance system, a human run is to detect and detect a movement that is worth the attention of the control personnel. On the contrary, even if the motion vector is generated, if the cumulative value for a predetermined time is small enough not to exceed the first threshold, the change in the image is assumed to be small and insignificant and is neglected in the detection step.

FIG. 6 is an example of visually showing a result of detecting an effective motion region from a CCTV compressed image through the process of FIG. 5 in the present invention. In FIG. 6, an image block having a motion vector accumulation value greater than or equal to a first threshold is marked with a moving object area and displayed in red. Referring to FIG. 6, the sidewalk block, the road, and the part with the shadow are not displayed as the moving object area, while the walking people or the driving car are displayed as the moving object area.

FIG. 7 is a flowchart illustrating an example of a process of detecting a boundary area of a moving object area in the present invention, and FIG. 8 is a view of the CCTV image of FIG. 2 is a diagram illustrating an example of a result of additionally applying a boundary region detection process according to the present invention. The process of FIG. 7 corresponds to step S200 in FIG.

Looking at the above [Fig. 6] it can be found that the image block corresponding to the moving object is not properly marked and only marking is done for some. That is, when looking at a walking person or a driving car, it can be found that only some of the image blocks are marked, not all of the objects are marked. In addition, it is also found that a plurality of moving object regions are formed for one moving object. Looking at a car, a plurality of moving object regions are formed. This means that the criterion of the moving object region adopted in S100 is very useful for filtering out the general region but is quite strict. Accordingly, the process of detecting boundary of the moving object area is performed by reviewing the image blocks around the moving object area marked in S100 and marking the moving object area additionally if a certain criterion is satisfied. need.

Step S210: First, a plurality of adjacent image blocks are identified based on the image blocks marked as moving object areas by the previous S100. These are referred to herein as 'neighboring blocks'. These neighboring blocks are parts that are not marked as moving object areas by (S100). In the process of FIG. 7, the neighboring blocks are examined in detail to determine whether any of these neighboring blocks may be included in the boundary of the moving object area. .

Steps S220 and S230: Compare the motion vector values with respect to the plurality of neighboring blocks with a second preset threshold value, and mark the neighboring block having the motion vector value exceeding the second threshold as the moving object region. If it is located adjacent to the moving object area where effective motion that is practically meaningful is found, and a certain amount of movement is found for itself, then the image block is characterized by the previous moving object area due to the characteristics of the captured image (eg CCTV image). It's likely a chunk. Therefore, such neighboring blocks are also marked as moving object regions.

As a first embodiment to implement this, each neighboring block is inspected, and when the motion vector value detected in the current frame is equal to or greater than a second preset threshold (eg, 0), the corresponding image block is also marked as a moving object region.

Meanwhile, as a second embodiment, when the motion vector cumulative value calculated in step S100 with respect to the neighboring block is greater than or equal to a preset second threshold value (eg, 5), the corresponding image block may also be marked as the moving object region. At this time, it is reasonable that the second threshold is set to a smaller value than the first threshold.

Step S240: Also, the coding type is an intra picture among the plurality of neighboring blocks, as a moving object region. In the case of the intra picture, since there is no motion vector, it is impossible to determine whether the neighboring block corresponds to the moving object region based on the motion vector. If the intra picture is located adjacent to an image block already detected as the moving object region, it is preferable to estimate that the moving object region forms a mass together with the extracted moving object region. This is because the loss when one image block other than the moving object area is included in the moving object area is not very large, whereas the loss when the moving object area is fragmented is large.

FIG. 8 is a diagram visually showing a result of applying a boundary region detection process to a CCTV compressed image in the present invention. A plurality of image blocks marked as moving object regions are displayed in blue color through the above process. Referring to FIG. 8, the blue moving object area was further extended to the vicinity of the moving object area shown in red in FIG. 6 and covered the entire moving object when compared with the actual image photographed by CCTV. You can find that it is enough.

FIG. 9 is a diagram illustrating an example of a result of arranging a moving object region through interpolation for the CCTV image of FIG. 8.

Step S300 is a process of arranging the division of the moving object area by applying interpolation to the moving object areas detected in the previous steps S100 and S200. Referring to FIG. 8, an unmarked image block is found between the moving object regions shown in blue. If there is an unmarked image block in the middle, it can be regarded as if they are a plurality of individual moving objects. When the moving object region is fragmented as described above, the result of step S400 may be inaccurate, and the number of moving object regions may increase, thereby complicating the process of step S400.

Accordingly, in the present invention, if there is one or a few unmarked image blocks surrounded by a plurality of image blocks marked as the moving object region, this is marked as the moving object region, which is called interpolation. Referring to FIG. 9, in contrast to FIG. 8, all of the non-marked image blocks existing between the moving object regions are marked as moving object regions. By doing this, all the moving areas are bundled together and treated as a moving object.

6, 8, and 9, it can be found that the moving object region properly reflects the actual image situation through the boundary region detection process and the interpolation process. In Fig. 6, if it is determined as a block marked with red color, it will be treated as if a lot of very small objects move in the video screen, which is not consistent with reality. On the other hand, if it is determined as a block marked in blue in FIG. 9, several moving objects having a certain volume will be treated as being present, similarly reflecting the actual scene.

Meanwhile, the present invention may be embodied in the form of computer readable codes on a computer readable nonvolatile recording medium. Such nonvolatile recording media include various types of storage devices, such as hard disks, SSDs, CD-ROMs, NAS, magnetic tapes, web disks, cloud disks, etc., and code is distributed in a plurality of networked storage devices. Forms that are implemented and executed may also be implemented. In addition, the present invention may be implemented in the form of a computer program stored in a medium in combination with hardware to execute a specific procedure.

Claims (8)

Parsing the bitstream of the compressed image to obtain a motion vector and a coding type for the coding unit;
A second step of obtaining a motion vector cumulative value for a predetermined time for each of the plurality of image blocks constituting the compressed image;
A third step of comparing the motion vector cumulative value with a first threshold value for the plurality of image blocks;
A fourth step of marking an image block having a motion vector accumulation value exceeding the first threshold as a moving object region;
A fifth step of setting a cluster of image blocks marked as the moving object region in the compressed image by being connected to each other as a moving object region extracted from the compressed image;
Acquiring a two-dimensional array in which a direction component and a size component of the motion vector obtained in the first step are disposed corresponding to the position of each image block for a plurality of image blocks belonging to each moving object region, A fifth step of setting a motion vector pattern for a moving object region;
A sixth step of collecting motion vector patterns for the plurality of moving object regions;
A sixth step of receiving an object classification reference for the corresponding moving object region for each motion vector pattern;
Performing a deep neural network learning by setting a combination of the motion vector pattern and the object classification reference as a training data set for the plurality of moving object regions and inputting the deep neural network prepared in advance;
A seventh step of acquiring a motion vector pattern (hereinafter, referred to as a 'classification motion vector pattern') for a specific moving object region to be classified;
Inputting the classification target motion vector pattern into the deep neural network and obtaining an object classification result for a specific moving object region that is the object classification object from the calculation result of the deep neural network;
A syntax based object classification method for a compressed image including a.
delete delete The method according to claim 1,
Performed between the fourth step and the fifth step,
A step of identifying a plurality of adjacent image blocks (hereinafter, referred to as 'neighbor block') around the moving object area;
B) comparing a motion vector value obtained in the first step with respect to the plurality of neighboring blocks with a second preset threshold value;
C) additionally marking, as a moving object region, a neighboring block having a motion vector value exceeding the second threshold value as a result of the comparison in the b of the plurality of neighboring blocks;
A syntax based object classification method for a compressed image, characterized in that the configuration further comprises.
The method according to claim 1,
Performed between the fourth step and the fifth step,
A step of identifying a plurality of adjacent image blocks (hereinafter, referred to as 'neighbor block') around the moving object area;
B) comparing the motion vector cumulative value with respect to the plurality of neighboring blocks with a second threshold preset to a value less than the first threshold;
C) additionally marking a neighboring block having a motion vector cumulative value exceeding the second threshold as a moving object region among the plurality of neighboring blocks as a result of the comparison in the b step;
A syntax based object classification method for a compressed image, characterized in that the configuration further comprises.
The method according to claim 4 or 5,
Carried out after the step c,
D) additionally marking a neighboring block having a coding type of an intra picture among the plurality of neighboring blocks as a moving object region;
A syntax based object classification method for a compressed image, characterized in that the configuration further comprises.
The method according to claim 6,
Carried out after the d step,
Performing an interpolation operation on the plurality of moving object areas to additionally mark up to a predetermined number of unmarked image blocks surrounded by the moving object area as a moving object area;
Syntax-based object classification method for a compressed image, characterized in that comprises a.
A computer program stored in a medium in combination with hardware to execute a syntax-based object classification method for a compressed image according to any one of claims 1, 4 and 5.

Images