WO2020027513A1

WO2020027513A1 - Syntax-based image analysis system for compressed image, and interworking processing method

Info

Publication number: WO2020027513A1
Application number: PCT/KR2019/009374
Authority: WO
Inventors: 이현우; 정승훈; 이성진
Original assignee: 이노뎁 주식회사
Priority date: 2018-07-30
Filing date: 2019-07-29
Publication date: 2020-02-06
Also published as: KR20200013340A; KR102090785B1

Abstract

The present invention relates to a technique for improving processing performance for a plurality of compressed images in a CCTV control system, and the like. More specifically, the present invention relates to a technique for improving processing performance for a compressed image by: extracting, by an image analysis system, an area where significant motion exists in an image, that is, a moving object area, on the basis of syntax (e.g.: a motion vector and a coding type) of the compressed image without the need for object identification and behavior recognition via a complicated image processing for the entire compressed image as in the conventional technique; and then analyzing a result of the extraction by interworking with the image analysis system. According to the present invention, it is advantageous that processing performance for a compressed image can be significantly improved by quickly identifying a moving object area on the basis of syntax of the compressed image without the need for performing a complicated image processing for the entire compressed image as in the conventional technique, and then selectively processing only an identified part by interworking with an image analysis system.

Description

Syntax-based Image Analysis System for Compressed Video

The present invention relates to a technique for improving the processing performance of a plurality of compressed images in a CCTV control system.

More specifically, the present invention provides a syntax (eg, a motion vector, coding type) of a compressed image without the need for object identification and behavior recognition through complex image processing of the entire compressed image as in the conventional art in an image analysis system. The present invention relates to a technique for improving the processing performance of a compressed image by extracting a region in which something meaningful movement exists in the image, that is, a moving object region, and analyzing the extracted result in conjunction with an image analysis system.

Recently, it is common to establish a video control system using CCTV for crime prevention, illegal surveillance, and securing after evidence. With multiple CCTV cameras installed by region, images generated by these CCTV cameras are displayed on a monitor and stored in a storage device. When a control agent finds a scene where a crime or accident occurs, he or she immediately responds appropriately, and if necessary, searches for images stored in the storage to secure post evidence.

By the way. The reality is that the number of control personnel is very low compared to the installation status of CCTV cameras. In order to effectively perform video surveillance with such limited number of people, simply displaying CCTV images on the monitor screen is not enough. It is preferable to process the object to be effectively detected by detecting the movement of the object present in each CCTV image and displaying something in the corresponding area in real time. In this case, the monitoring personnel do not monitor the entire CCTV image with uniform interest, but monitor the CCTV image centering on the part where the object moves.

On the other hand, the video detection system adopts compressed video for the efficiency of the storage space. In particular, as the number of CCTV cameras is rapidly increasing and high quality cameras are mainly installed, high-compression complex video compression technologies such as H.264 AVC and H.265 HEVC have been adopted. CCTV cameras generate and provide compressed images according to these technical specifications, and the video control system decodes the compressed images in reverse. In order to determine moving objects in a CCTV image to which image compression technology is applied, conventionally, a process of image processing is required after decoding a compressed image to obtain a reproduced image, that is, a decompressed original image.

1 is a block diagram illustrating a general configuration of a video decoding apparatus according to the H.264 AVC Technical Specification. Referring to FIG. 1, a video decoding apparatus according to H.264 AVC includes a parser 11, an entropy decoder 12, an inverse converter 13, a motion vector operator 14, a predictor 15, and a deblocking filter ( 16) is configured to include. These hardware modules process compressed video sequentially to decompress and restore the original video data. At this time, the parser 11 parses the motion vector and the coding type for the coding unit of the compressed image. Such a coding unit is generally an image block such as a macroblock or a subblock.

2 is a flowchart illustrating a process of extracting a moving object by analyzing a compressed image and performing object classification and event identification through the conventional art. Referring to FIG. 2, in the prior art, the compressed image is decoded according to H.264 AVC and H.265 HEVC, etc. (S10), and the frame images of the reproduced image are downscaled to a small image, for example, 320 × 240 (S20). In this case, the reason for downscaling resizing is to reduce processing burden in subsequent steps. Then, after obtaining differential images of the resized frame images, the moving objects are extracted through a complicated image analysis process, and the coordinates of the moving objects are calculated (S30). Finally, object classification and event identification are performed using the extracted object thumbnail images and coordinates of the moving objects (S40).

As described above, in order to extract a moving object, compressed image decoding, downscale resizing, and image analysis are performed. These are very complicated processes, and therefore, in a conventional video control system, the capacity that a single video analysis server can process simultaneously is quite limited. Currently, the largest CCTV channels that a high performance video analytics server can cover are 16 channels. Since a large number of CCTV cameras were installed, a video control system required a plurality of video analysis servers, which caused problems such as increased cost and difficulty in securing physical space.

An object of the present invention is to provide a technique for improving the processing performance of a plurality of compressed images in a CCTV control system.

In particular, an object of the present invention is to perform an image based on the syntax (eg, motion vector, coding type) of the compressed image without the need of object identification and behavior recognition through complex image processing of the entire compressed image in the image analysis system as in the prior art. It is to provide a technique for improving the processing performance of compressed images by extracting a region in which something meaningful movement exists, that is, a moving object region, and analyzing the extracted result in conjunction with an image analysis system.

In order to achieve the above object, a syntax-based image analysis system and an interworking processing method for a compressed image according to the present invention include a first step of parsing a bitstream of a compressed image to obtain a motion vector and a coding type for a coding unit. ; A second step of obtaining a motion vector cumulative value for a first preset time for each of the plurality of image blocks constituting the compressed image; A third step of comparing a motion vector cumulative value with a first threshold value for a plurality of image blocks; A fourth step of marking an image block having a motion vector accumulation value exceeding a first threshold as a moving object region; A fifth step of generating an object property for the moving object area from a series of frame images constituting the compressed image by assigning and managing a unique ID to the marked one or more moving object areas; A sixth step of extracting a thumbnail image and coordinate information of the moving object area and providing the same to the image analysis system; And a seventh step of receiving the object classification result and the event identification result for the moving object area from the image analysis system and managing the connection with the Unique ID of the moving object area.

In this case, the present invention is performed between the sixth step and the seventh step, the image analysis system sorts the thumbnail image and coordinate information on the basis of the unique ID and image analysis processing by the unit of unique ID to classify objects and events for the moving object area Performing identification; may be configured to further include.

According to the present invention, the fifth step may include: determining whether a moving object region of the same object exists in a previous frame for each moving object region based on the calculation of the overlapping degree of the image blocks between the moving object regions; A 52nd step of determining whether a unique ID is pre-assigned for each moving object region according to the determination result of the 51st step; A fifty-third step of maintaining a pre-allocated Unique ID for the mobile object area in the Unique ID allocation state according to the determination result of the fifty-second step; A new step of allocating a unique ID to a mobile object area in the unique ID unassigned state according to the determination result of the step 52; If a unique ID has been assigned in the previous frame but disappeared from the current frame image is identified, step 55 for revoking the unique ID assigned to the disappeared mobile object region may be performed.

In addition, the present invention may further include a first step of identifying a plurality of adjacent image blocks (hereinafter, referred to as 'neighbor block') around a moving object area, which are performed between the fourth step and the fifth step; B) comparing a motion vector value with a second preset threshold value for a plurality of neighboring blocks; C) additionally marking a neighboring block having a motion vector value exceeding a second threshold as a moving object region; D) additionally marking a neighboring block having a coding type of an intra picture among the plurality of neighboring blocks as a moving object region; The method may further include an e-step of performing interpolation on the plurality of moving object regions to additionally mark a predetermined number or less of unmarked image blocks surrounded by the moving object region as the moving object region.

On the other hand, the computer program according to the present invention is stored in the medium in combination with hardware to execute the syntax-based image analysis system and the interworking processing method for the compressed image as described above.

According to the present invention, by quickly identifying the moving object region from the syntax of the compressed image without performing complicated image processing on the entire compressed image as in the prior art, only the identified portion is selectively processed in association with the image analysis system. There is an advantage that can greatly improve the processing performance of compressed video in CCTV control system. In particular, according to the present invention, it is possible to process a compressed image even with a calculation amount of about 1/10 of the prior art, thereby increasing the number of available channels per server by approximately 10 times without large-scale investment.

1 is a block diagram showing a general configuration of a video decoding apparatus.

2 is a flowchart illustrating a process of performing object classification and event identification by analyzing a compressed image in the prior art.

3 is a view illustrating a concept in which an image analysis system and an object region identification device interoperate in the present invention.

4 is a flowchart illustrating a process of interworking with an image analysis system based on compressed image syntax according to the present invention.

5 is a flowchart illustrating an example of a process of detecting effective motion from a compressed image in the present invention.

6 is a diagram illustrating an example of a result of applying an effective motion region detection process according to the present invention to a CCTV compressed image.

FIG. 7 is a flowchart illustrating an example of a process of detecting a boundary region for a moving object region in the present invention. FIG.

8 is a diagram illustrating an example of a result of applying a boundary region detection process to the CCTV image of FIG. 6.

9 is a diagram illustrating an example of a result of arranging a moving object region through interpolation with respect to the CCTV image of FIG. 8.

FIG. 10 is a diagram for one example in which a unique ID is assigned to a moving object area in the present invention; FIG.

FIG. 11 is a diagram for one example of deriving a thumbnail image for a moving object region in the present invention; FIG.

12 is a diagram illustrating an example in which location information and size information are identified for a moving object region in the present invention.

Hereinafter, with reference to the drawings will be described in detail the present invention.

3 is a diagram illustrating a concept in which an image analysis system and an object region identification apparatus interoperate in the present invention.

Referring to FIG. 3, the present invention is a technique for effectively processing a compressed image transmitted from a CCTV camera 100. For example, CCTV surveillance systems collect high-quality captured images from hundreds to tens of thousands of CCTV cameras 100 compressed with complex image compression algorithms (eg H.264 AVC, H.265 HEVC). . Conventionally, since these compressed images are processed through the process of FIG. 2, the processing burden of the image analysis server is very high, and the maximum CCTV channel that one server can accommodate is usually only 16 channels.

The present invention increases the overall processing performance by linking the object region identification apparatus 200 to the general image analysis system 300. The image analysis system 300 analyzes the contents of the image in the manner used in the related art, recognizes an object, and identifies an event (eg, an offender wandering, a wall story, a crime, a fight, etc.) according to a result of analyzing the behavior of the object. . On the other hand, the image analysis system 300 in the present invention is not limited to operating according to the conventional image analysis technology, but may be so.

Conventionally, since the image analysis system 300 performs image analysis on the entire compressed image, processing performance is limited. On the other hand, in the present invention, the object region identification apparatus 200 extracts a region in which something meaningful movement exists in the image, that is, a moving object region, based on the syntax of the compressed image (eg, motion vector, coding type), and then the thumbnail image or the like. A structure that analyzes and processes location information in conjunction with the image analysis system 300 is adopted. Through this, the image analysis system 300 recognizes an object and analyzes the behavior of the object to significantly reduce the amount of data required to identify the event to improve the overall performance.

The object region identification apparatus 200 parses a bitstream of a compressed image without having to decode the compressed image, and obtains syntax information, for example, a motion vector, for each image block, that is, a macro block and a sub block. The moving object region is quickly extracted through the motion vector and coding type information. The moving object region thus obtained does not accurately reflect the boundary of the moving object, but its processing speed is about 20 times faster than image analysis, but it shows a certain level of reliability for the existence of significant movement. As described above, the object area identification apparatus 200 quickly filters most of the compressed images based on the syntax, thereby reducing the burden on the image analysis system 300 and increasing the overall processing performance.

However, since the moving object region extracted by the object region identification apparatus 200 is merely a lump of an image block estimated to include the moving object, there is a limit in determining something therefrom. This is because the contents of the images are not judged, but the characteristics of the movements in the images are distinguished. Accordingly, the moving object region identification information and the thumbnail image or position information of the moving object region acquired by the object region identification apparatus 200 are transmitted to the image analysis system 300. The image analysis system 300 performs image analysis on thumbnail images derived from a series of frame images constituting the compressed image, for example, classifies objects by determining image contents in a moving object region and performs an event. To identify.

When the compressed image is transmitted as described above, the object region identification apparatus 200 quickly picks out image chunks that seem to be important because there is something moving, and the image analysis system 300 properly analyzes the contents of the selected image chunks to classify the objects. Eg people, cars, animals, etc.) and events. The object area identification apparatus 200 receives an image analysis result, that is, an object classification result and an event identification result from the image analysis system 300, and provides the control personnel to utilize the image analysis result.

Meanwhile, according to the present invention, the object region identification apparatus 200 does not need to decode the compressed image in the process of extracting the moving object region. However, the apparatus or software to which the present invention is applied should not perform the operation of decoding the compressed image, but the scope of the present invention is not limited. For example, in order to obtain a thumbnail image of the moving object region, an operation of decoding the compressed image partially or entirely may be performed. Similarly, the image analysis system 300 may be a system for performing a conventional image analysis process, but the scope of the present invention is not limited thereto.

First, a process of extracting a moving object region from each frame image by the object region identification apparatus 200 based on the syntax (motion vector, coding type) of the compressed image through steps S100 to S300 will be described. .

Step S100: An effective motion that can substantially recognize meaning is detected from the compressed image based on the motion vector of the compressed image, and the image region in which the effective motion is detected is set as the moving object region.

To this end, data of a compressed image is parsed according to video compression standards such as H.264 AVC and H.265 HEVC to obtain a motion vector and a coding type for a coding unit. In this case, the size of the coding unit is generally about 64x64 to 4x4 pixels and may be set to be flexible.

The motion vectors are accumulated for a predetermined time period (for example, 500 msec) for each image block, and it is checked whether the motion vector accumulation value exceeds a first predetermined threshold (for example, 20 pixels). If such an image block is found, it is considered that effective motion has been found in the image block and marked as a moving object area. On the other hand, even if a motion vector is generated, if the cumulative value for a predetermined time does not exceed the first threshold, the image change is assumed to be insignificant and ignored.

Step S200: Detects how far the boundary region is to the moving object region detected in S100 based on the motion vector and the coding type. If a motion vector occurs above a second threshold (for example, 0) or a coding type is an intra picture by inspecting a plurality of adjacent image blocks centered on the image block marked as a moving object area, the corresponding image block is also moved. Mark as an object area. Through this process, the corresponding image block is substantially in the form of forming a lump with the moving object region detected in S100.

If an effective motion is found and there is a certain amount of motion in the vicinity of the moving object area, it is marked as a moving object area because it is likely to be a mass with the previous moving object area. In addition, in the case of an intra picture, since a motion vector does not exist, determination based on a motion vector is impossible. Accordingly, the intra picture located adjacent to the image block already detected as the moving object region is estimated as a mass together with the previously extracted moving object region.

Step S300: The interpolation is applied to the moving object areas detected at S100 and S200 to clean up the fragmentation of the moving object area. In the above process, since it is determined whether the moving object area is the image block unit, even though it is actually a moving object (for example, a person), there is an image block that is not marked as the moving object area in the middle. The phenomenon of dividing into may occur. Accordingly, if there are one or a few unmarked image blocks surrounded by a plurality of image blocks marked with the moving object region, they additionally mark the moving object region. By doing so, it is possible to make the mobile object region divided into several into one. The influence of such interpolation is clearly seen when comparing FIG. 8 and FIG.

As described above, the moving object region was quickly extracted from each frame image based on the syntax (motion vector, coding type) of the compressed image through steps S100 to S300. However, the moving object region derived through this process is merely a concept of a mass of images that seem to have something moving in each frame image, and a concept of an object that is uniformly recognized as a frame progresses in a compressed image. There is no.

Next, a process of containing the concept of an object in the moving object area by managing an identification code (Unique ID) in the moving object area previously extracted by the object area identification apparatus 200 through the step (S400).

Step S400: Unique ID is managed for the moving object region extracted based on the syntax from the compressed image. The moving object region is derived from each image frame constituting the compressed image. The image content is not determined by analyzing the image content, but merely extracting a chunk of an image that seems to be moving in the image frame. Accordingly, by assigning and managing a unique ID for the mobile object area, attributes as an object are created in the mobile object area. Through this, the area of the moving object can be treated like an object, not just a region, and the movement of a specific object can be interpreted while moving over a series of frame images in the compressed image.

Unique ID management of the mobile object area is handled in the following three cases. If a unique ID is assigned in a previous frame and a moving object area is identified in the current frame image (S410), the moving object area identified as an unassigned ID in the current frame image because it has not been identified in the previous frame. In the case of newly assigning a unique ID to the SID, a mobile object region in which a unique ID is assigned in the previous frame but disappeared from the current frame image is identified and revokes the allocated unique ID (S430).

However, in a series of frame images constituting the compressed image, it should be possible to determine whether the chunk of the image block marked as the moving object region in the previous frame is related to the same object between front and rear frames. This is because it is possible to determine whether the Unique ID has been previously assigned to the moving object area handled in the current frame.

In the present invention, since only the image block is a moving object region without checking the contents of the original image, it is not possible to confirm whether the chunks of the moving object region are actually the same in the image frames before and after. That is, since the contents of the moving object area are not known, such a change cannot be identified, for example, when the cat is replaced by a dog between the front and rear frames at the same point. However, considering that the time interval between frames is very short and that the observation object of the CCTV camera moves at a normal speed, the possibility of this happening can be excluded.

Accordingly, the present invention estimates that the ratio or number of image blocks overlapping between the chunks of the moving object region in the front and back frames is equal to or greater than a predetermined threshold. According to this approach, even if the contents of the image are not known, it is possible to determine whether the previously identified moving object region is moved or whether a new moving object region is newly discovered or the existing moving object region is lost. This judgment is lower in accuracy than the prior art, but can greatly increase the data processing speed, which is advantageous in practical applications.

In operation S410, when the moving object region to which the Unique ID has been assigned in the previous frame is identified in the current frame image, the previously allocated Unique ID is allocated to the corresponding moving object region. According to the implementation example, the identification may be marked in the management database of the Unique ID.

In step S420, when a new object is unidentified in the current frame image because it has not been identified in the previous frame, a new ID is newly assigned to the corresponding mobile object area. This means that a new moving object is found in the image. 10 illustrates an example in which unique IDs are allocated to three moving object regions derived from CCTV images.

In operation S430, when the moving object region in which the unique ID is assigned in the previous frame of the compressed image disappears from the current frame image, the movement object region is allocated in step S420 with respect to the previous frame for the moving object region. Revoke the Unique ID maintained in S410). In other words, the moving object that has been discovered and managed before has disappeared from the image.

Next, a process of performing object classification and event identification by the object region identification apparatus 200 and the image analysis system 300 interworking the moving object region through steps S500 to S900 will be described.

Step S500: Next, a thumbnail image and coordinate information (eg, location information and size information) of the moving object area are derived. FIG. 11 is a diagram illustrating an example in which thumbnail images are derived for three moving object areas of a CCTV photographing image in the present invention, and FIG. 12 shows position information and size information as coordinate information for these moving object areas. It is a figure which shows the identified example.

In order to obtain the thumbnail image of FIG. 11, the object region identification apparatus 200 may be configured to have a function of decoding a compressed image or selectively decoding a portion of the compressed image. Meanwhile, when the object region identification apparatus 200 transmits the location information to the image analysis system 300, the image analysis system 300 may be configured to obtain a thumbnail image of the moving object region therefrom. However, for this purpose, since the internal process of the image analysis system 300 has to be changed a lot compared with the prior art, it is determined that it is not a very preferable approach. Rather, a method of generating a thumbnail image of the object region identification apparatus 200 is more preferable.

Meanwhile, the position information of the moving object region means a position where the moving object region is disposed in the image of the corresponding video block. As shown in FIG. 12, the upper left coordinate of the rectangle optimally surrounding the moving object region may be used, or the rectangular The center grid can also be used as location information. In addition, the size information may use a rectangular size that optimally surrounds the moving object region as shown in FIG. 12.

Step (S600, S700): Next, the object region identification apparatus 200 provides the coordinate information and thumbnail image of the moving object region derived from the compressed image to the image analysis system 300, the image analysis system 300 Classifies and performs event classification by analyzing the moving object region based on the unique ID.

In operation S400, the object region identification apparatus 200 performs unique ID management on the moving object region, and through this, the moving object region in the compressed image is not simply a region but a concept of an object. To have it. Therefore, the image analysis system 300 may treat the series of moving object region identification information provided by the object region identification apparatus 200 in the concept of an object.

The image analysis system 300 sorts a plurality of moving object region identification information (thumbnail image, coordinate information) transmitted from the object region identification apparatus 200 based on a unique ID and performs image analysis in units of a unique ID. Through such image analysis, it is possible to recognize the contents of the moving object area by unique ID, and accordingly, the object classification result (eg, person, car, animal, etc.) regarding what the object is, and the object in the image Obtain event identification results (e.g. offender roaming, wall talks, crimes, fights, etc.) as to whether or not you are acting. In this case, the image analysis system 300 does not perform image analysis on the entire compressed image, but performs image analysis only on a series of moving object regions derived by the object region identification apparatus 200 from the compressed image. Significantly lower than the prior art.

Step (S800, S900): Next, the object region identification apparatus 200 receives the object classification and event identification results for the moving object region from the image analysis system 300, and the control personnel can use the area of the moving object region. It manages connection of Unique ID, object classification and event identification.

Hereinafter, referring to FIGS. 5 to 9, even without decoding a compressed image and analyzing the contents of the image, the object region identification apparatus 200 uses something in the image using syntax information of the compressed image, for example, a motion vector and a coding type. The process of extracting a region where significant motion exists, that is, a moving object region, will be described. As will be described later, this process is conceptually characterized in that it does not recognize the moving object by interpreting the contents of the image, but extracts a block of the image block that is assumed to contain the moving object without knowing the contents. have.

5 is a flowchart illustrating an example of a process of detecting effective motion from a compressed image in the present invention, and FIG. 6 is a diagram illustrating an example of a result of applying the effective motion region detection process according to the present invention to a CCTV compressed image. . The process of FIG. 5 corresponds to step S100 in FIG. 4.

Step S110: First, a coding unit of a compressed image is parsed to obtain a motion vector and a coding type. Referring to FIG. 1, a video decoding apparatus performs parsing (header parsing) and motion vector operations on a stream of compressed video according to a video compression standard such as H.264 AVC and H.265 HEVC. Through this process, the motion vector and coding type are parsed for the coding unit of the compressed image.

Step S120: Acquire a motion vector cumulative value for a preset time (for example, 500 ms) for each of the plurality of image blocks constituting the compressed image.

This step is presented with the intention to detect if there are effective movements that are practically recognizable from the compressed image, such as driving cars, running people, and fighting crowds. Shaky leaves, ghosts that appear momentarily, and shadows that change slightly due to light reflections, though they are moving, are virtually meaningless objects and should not be detected.

To this end, a motion vector cumulative value is obtained by accumulating a motion vector in units of one or more image blocks for a predetermined time period (for example, 500 msec). In this case, the image block is used as a concept including a macroblock and a subblock.

Steps S130 and S140: Comparing a motion vector cumulative value with respect to a plurality of image blocks with a preset first threshold value (for example, 20 pixels), and moving the video block having a motion vector cumulative value exceeding the first threshold value. Mark the area.

If an image block having a predetermined motion vector accumulation value is found as described above, it is considered that something significant movement, that is, effective movement, is found in the image block and is marked as a moving object region. For example, we want to detect and detect object movements that are worth the attention of the controller, to the extent that a person runs. On the contrary, even if a motion vector is generated, if the cumulative value for a predetermined time is small enough not to exceed the first threshold, the change in the image is assumed to be small and insignificant and is neglected in the detection step.

FIG. 6 is an example illustrating a result of detecting an effective motion region from a CCTV compressed image through the process of FIG. 5. In FIG. 6, an image block having a motion vector accumulation value equal to or greater than a first threshold is marked as a moving object area and displayed as a bold line area. Referring to FIG. 6, the sidewalk block, the road, and the part with the shadow are not displayed as the moving object area, while the walking people or the driving car are displayed as the moving object area.

FIG. 7 is a flowchart illustrating an example of a process of detecting a boundary region for a moving object region in the present invention, and FIG. 8 is a boundary region according to FIG. 7 with respect to the CCTV image of FIG. Figure 1 shows an example of the results of further applying the detection process. The process of FIG. 7 corresponds to step S200 in FIG. 4.

Referring to FIG. 6, it can be found that marking is not properly performed on an object (moving object) actually moving in the image, and only some of them are marked. In other words, if you look at a person walking or driving a car, you will find that not all of the objects are marked, but only some blocks. In addition, although it is actually one moving object, many are marked as if they are a plurality of moving object areas. This means that the criterion of the moving object region adopted in (S100) above was useful for filtering out the general region but was a very strict condition. Therefore, it is necessary to detect the boundary of the moving object by looking around the moving object area.

Step S210: First, a plurality of adjacent image blocks are identified based on the image blocks marked as moving object areas by the previous S100. In the present specification, these are referred to as 'neighborhood blocks'. These neighboring blocks are portions that are not marked as the moving object region by S100, and the process of FIG. 7 further examines them to determine whether any of these neighboring blocks may be included in the boundary of the moving object region.

Steps S220 and S230: compare a motion vector value with respect to a plurality of neighboring blocks with a second preset threshold (eg, 0), and mark the neighboring block having a motion vector value exceeding the second threshold as a moving object region. do. If the movement is located adjacent to the area of the moving object where effective motion that is practically meaningful is found and a certain amount of movement is found for itself, the image block is likely to be a block with the area of the adjacent moving object due to the characteristics of the photographed image. . Therefore, such neighboring blocks are also marked as moving object regions.

Step S240: Also, the coding type is an intra picture among the plurality of neighboring blocks as a moving object region. In the case of an intra picture, since a motion vector does not exist, it is fundamentally impossible to determine whether a motion exists in a corresponding neighboring block based on the motion vector. In this case, it is safer for the intra picture located adjacent to the image block already detected as the moving object region to maintain the settings of the previously extracted moving object region.

FIG. 8 is a diagram visually showing a result of applying a boundary region detection process to a CCTV compressed image. A plurality of image blocks marked as a moving object region through the above process are indicated by a bold line. Referring to FIG. 8, the area of the moving object was further extended to the vicinity of the moving object area indicated by the bold line in FIG. 6, and thus, it was found that the moving object area was enough to cover the moving object when compared to the image captured by CCTV. Can be.

FIG. 9 is a diagram illustrating an example of a result of arranging a moving object region through interpolation according to the present invention for a CCTV image image to which the boundary region detection process illustrated in FIG. 8 is applied.

Step S300 is a process of arranging the division of the moving object area by applying interpolation to the moving object areas detected in the previous steps S100 and S200. Referring to FIG. 8, an unmarked image block is found between the moving object regions indicated by the bold lines. If there is an unmarked image block in the middle, it can be regarded as if they are a plurality of individual moving objects. When the moving object region is fragmented in this way, the result of step S500 may be inaccurate, and the number of moving object regions may increase, thereby complicating the process of steps S500 to S700.

Accordingly, in the present invention, if there is one or a few unmarked image blocks surrounded by a plurality of image blocks marked as the moving object region, this is marked as the moving object region, which is called interpolation. Referring to FIG. 9, in contrast to FIG. 8, all of the non-marked image blocks existing between the moving object regions are marked as moving object regions. Through this, it is possible to derive the result of detecting the moving object more intuitively and accurately for the control personnel.

Comparing FIG. 6 and FIG. 9, it can be found that the moving object region properly reflects the situation of the actual image through the boundary region detection process and the interpolation process. In FIG. 6, if the block is marked as a bold line area, a large number of very small objects move in the image screen, which does not correspond to reality. On the other hand, if it is determined as a block marked with the bold line area in Fig. 9 will be treated as a few moving objects having a certain volume to reflect the actual scene similarly.

Meanwhile, the present invention may be embodied in the form of computer readable codes on a computer readable nonvolatile recording medium. Such nonvolatile recording media include various types of storage devices, such as hard disks, SSDs, CD-ROMs, NAS, magnetic tapes, web disks, and cloud disks. Forms that are implemented and executed may also be implemented. In addition, the present invention may be implemented in the form of a computer program stored in a medium in combination with hardware to execute a specific procedure.

Claims

Parsing the bitstream of the compressed image to obtain a motion vector and a coding type for the coding unit;

A second step of obtaining a motion vector cumulative value for a first preset time for each of the plurality of image blocks constituting the compressed image;

A third step of comparing the motion vector cumulative value with a first threshold value for the plurality of image blocks;

A fourth step of marking an image block having a motion vector accumulation value exceeding the first threshold as a moving object region;

A fifth step of generating an object property for a moving object area in a series of frame images constituting a compressed image by allocating and managing a unique ID for the at least one marked moving object area;

A sixth step of extracting a thumbnail image and coordinate information of the moving object area and providing the same to an image analysis system;

A seventh step of receiving an object classification result and an event identification result of the moving object area from the image analysis system and managing the connection with the Unique ID of the moving object area;

Syntax-based image analysis system and interworking processing method for a compressed image comprising a.
The method according to claim 1,

Performed between the sixth and seventh steps,

The image analysis system aligning thumbnail images and coordinate information based on a unique ID and performing image analysis on a unique ID basis to perform object classification and event identification on the moving object region;

Syntax-based image analysis system and interworking processing method for a compressed image, characterized in that further comprises.
The method according to claim 1,

The fifth step,

A step 51 for determining whether a moving object region for the same object exists in a previous frame for each moving object region based on the calculation of the overlapping degree of the image blocks between the moving object regions;

A 52nd step of determining whether a unique ID is pre-assigned for each moving object region according to the determination result of the 51st step;

A fifty-third step of maintaining a pre-allocated Unique ID for the mobile object area in the Unique ID allocation state according to the determination result of the fifty-second step;

A 54th step of newly assigning a unique ID to the mobile object area in the unique ID unassigned state according to the determination result of the 52nd step;

A 55th step of revoking a unique ID allocated to the disappeared moving object region when a moving object region allocated to a unique ID in the previous frame but disappeared from the current frame image is identified;

Syntax-based image analysis system and interworking processing method for a compressed image, characterized in that comprises a.
The method according to claim 3,

Performed between the fourth and fifth steps,

A step of identifying a plurality of adjacent image blocks (hereinafter, referred to as 'neighbor block') around the moving object area;

B) comparing a motion vector value obtained in the first step with respect to the plurality of neighboring blocks with a second preset threshold value;

C) additionally marking, as a moving object region, a neighboring block having a motion vector value exceeding the second threshold value as a result of the comparison in the b of the plurality of neighboring blocks;

Syntax-based image analysis system and interworking processing method for a compressed image, characterized in that further comprises.
The method according to claim 4,

Carried out after the step c,

D) additionally marking a neighboring block having a coding type of an intra picture among the plurality of neighboring blocks as a moving object region;

Syntax-based image analysis system and interworking processing method for a compressed image, characterized in that further comprises.
The method according to claim 5,

Carried out after the d step,

Performing an interpolation operation on the plurality of moving object regions to additionally mark up to a predetermined number of non-marked image blocks surrounded by the moving object region as a moving object region;

Syntax-based image analysis system and interworking processing method for a compressed image, characterized in that further comprises.
A computer program stored in a medium in combination with hardware to execute a syntax-based image analysis system and an interworking processing method for a compressed image according to any one of claims 1 to 6.