KR102592089B1 - Moving Frame Detecting Method, Device and Computer Program Thereof - Google Patents

Abstract

The present invention relates to a method for detecting a frame in which motion exists, and more specifically, to preliminary detect whether the frame in question is a frame in which motion exists based on parameter values derived from the video decoding process and analyze the image. By referring to , it relates to an object area detection method, device, and computer program for the same that can further speed up analysis such as object recognition and tracking.

Description

Method for detecting frames with movement {Moving Frame Detecting Method, Device and Computer Program Thereof}

The present invention relates to a method for detecting a frame in which motion exists, and more specifically, to preliminary detect whether the frame in question is a frame in which motion exists based on parameter values derived from the video decoding process and analyze the image. By referring to , it relates to an object area detection method, device, and computer program for the same that can further speed up analysis such as object recognition and tracking.

Recently, video data collected from smartphones, CCTV, black boxes, high-definition cameras, etc. has been rapidly increasing. Accordingly, the requirements for recognizing people or objects based on unstructured image data, extracting meaningful information, and visually analyzing and utilizing the content are increasing.

Video data analysis technology refers to various technologies for searching for desired images or recognizing situations such as event occurrence by learning and analyzing various images.

However, since the technology to recognize, analyze, and track image data is an algorithm that requires a significant amount of calculation, that is, the complexity is high, so as the size of the image data increases, a significant load is placed on the computing device. Accordingly, it takes increasingly longer time to analyze image data that has become larger in size. Therefore, there is a constant need for a method to reduce image information analysis time.

Meanwhile, in recent years, as awareness of security has strengthened due to terrorism, etc., the video security market has continued to grow, and accordingly, the demand for intelligent video processing is also increasing.

Recently, technology that can efficiently compress, transmit, and view high-resolution video based on block-based video codecs following protocols such as H.265 (HEVC) has spread. Such high-resolution images are also applied to monitoring images such as CCTV, but in the analysis and tracking of such high-resolution images, as the resolution of the image increases, the conventional object detection method requires a higher amount of calculation, and thus real-time video There was a problem that the analysis was not carried out smoothly.

Meanwhile, Prior Document 1 (Korean Patent No. 10-1409826, registered on June 13, 2014) calculates the motion vector of the reference frame based on the histogram of the motion vectors of blocks within the reference frame, and calculates the motion vector of the reference frame based on the global motion vector. A technology for determining the area type of a block is being disclosed.

However, in the case of Prior Document 1, since motion vectors must be calculated for the entire region and histogram data must be calculated for the entire region, it is difficult to achieve a speed that enables real-time processing of current high-resolution images, and furthermore, Since motion vectors must be considered for all blocks, there is a problem that operations must first be performed on unnecessary blocks as well.

Additionally, since the entire frame must be processed in relation to object detection, there is a problem that a large amount of basic computation is required.

Meanwhile, recent developments in various electronic devices have made it possible to acquire 4K UHD (Ultra High Definition) video, and people are now discussing 8K UHD video. Mobile devices can now handle 2048×1530 video, which has a higher resolution than HD (High Definition). HEVC (High Efficiency Video Coding) is a next-generation video compression standard developed with the goal of improving coding performance in high-definition applications such as HD mobile, home cinema, and UHDTV (Ultra High Definition TV).

Prior Document 1: Korean Patent No. 10-1409826, ‘Motion prediction method using adaptive search range’, registered on June 13, 2014)

The purpose of the present invention is to preliminary detect whether the frame in question is a frame in which movement exists based on the parameter values derived from the video decoding process and refer to this for video analysis, thereby enabling further analysis such as object recognition and tracking. The aim is to provide a high-speed object area detection method, device, and computer program for the same.

In order to solve the above problem, in one embodiment of the present invention, a method of detecting a frame in which movement is performed in a computing system including one or more processors and a main memory storing instructions executable by the processor, comprising: A frame extraction step of extracting a P frame or B frame from data; For the P frame or B frame, an intra mode confirmation step of checking information about whether each of a plurality of coding units included in each frame is an intra mode; A ratio information calculation step of calculating ratio information of a coding unit corresponding to an intra mode among a plurality of coding units for each of the P frame or B frame; And a motion frame determination step of determining that the P frame or B frame is a frame in which motion exists or a frame in which motion does not exist, based on the ratio information of the P frame or B frame. Provides a frame detection method.

In some embodiments of the present invention, the video data may include video data encoded according to the HEVC standard.

In some embodiments of the present invention, the motion frame determination step is performed when the ratio information of each P frame or B frame is higher than a preset first threshold value, and the corresponding P frame or B frame is selected as a frame in which movement exists. It can be judged as follows.

In some embodiments of the present invention, the first threshold calculates ratio information of coding units corresponding to the intra mode for each frame of training video data including P frames or B frames labeled as having no movement. learning rate derivation step; A first statistical value derivation step of deriving first statistical information from the ratio information of each of the plurality of P frames or B frames of the learning video data; and a first critical value determination step of determining the first critical value based on the first statistical information.

In some embodiments of the present invention, the first statistical value may be determined based on the average value and standard deviation value of each ratio information of a plurality of P frames or B frames of the training image data.

In some embodiments of the present invention, the motion frame determination step is to select a related P frame or B frame when the standard deviation value of the ratio information of a plurality of P frames or B frame groups is higher than a preset second threshold value. It can be judged by the frame in which movement exists.

In some embodiments of the present invention, the motion frame determination step is performed when determining whether the Nth P frame or B frame is a frame in which movement exists, and the N+Mth P frame or B frame is selected from the Nth P frame or B frame. If the standard deviation value of the ratio information of a plurality of P frames or B frame groups up to the B frame is higher than the preset second threshold value, the Nth P frame or B frame can be determined as a frame in which movement exists. .

In some embodiments of the present invention, the second threshold calculates ratio information of coding units corresponding to the intra mode for each frame of training video data including P frames or B frames labeled as having no movement. learning rate derivation step;

A second statistical value derivation step of deriving second statistical information from the ratio information of each of the plurality of P frames or B frames of the learning video data; and a second critical value determination step of determining the second critical value based on the second statistical information.

In some embodiments of the present invention, the second statistical value may be determined based on the standard deviation value of the ratio information of each of the plurality of P frames or B frames of the training image data.

In some embodiments of the present invention, the motion frame determination step is performed when the ratio information of each P frame or B frame is higher than a preset first threshold value, and the corresponding P frame or B frame is selected as a frame in which movement exists. , and if the standard deviation value of the ratio information of a plurality of P frames or B frame groups is higher than the preset second threshold value, the related P frame or B frame can be determined as a frame in which movement exists.

In order to solve the above problem, in one embodiment of the present invention, there is a detection device for frames in which movement is present, including one or more processors and a main memory that stores instructions executable by the processors, and detects P frames from image data. Or a frame extraction unit that extracts B frames; For the P frame or B frame, an intra mode confirmation unit that checks information on whether each of a plurality of coding units included in each frame is an intra mode; A ratio information calculation unit that calculates ratio information of a coding unit corresponding to an intra mode among a plurality of coding units for each of the P frame or B frame; and a motion frame determination unit that determines that the P frame or B frame is a frame in which motion exists or a frame in which motion does not exist, based on the ratio information of the P frame or B frame. A frame detection device is provided.

In order to solve the above problem, in one embodiment of the present invention, a computer program stored in a computer-readable medium including a plurality of instructions executed by one or more processors, the computer program A frame extraction step of extracting a P frame or B frame; For the P frame or B frame, an intra mode confirmation step of checking information about whether each of a plurality of coding units included in each frame is an intra mode; A ratio information calculation step of calculating ratio information of a coding unit corresponding to an intra mode among a plurality of coding units for each of the P frame or B frame; and a motion frame determination step of determining that the P frame or B frame is a frame in which motion exists or a frame in which motion does not exist, based on the ratio information of the P frame or B frame. .

According to one embodiment of the present invention, based on the parameter values derived from the video decoding process, it is preliminary detected whether the frame is a frame in which movement exists, and this is referred to for video analysis, thereby improving object recognition, tracking, etc. It can have the effect of further speeding up the analysis.

According to one embodiment of the present invention, by determining whether there is movement or no movement in the frame based on the decoding parameters of the CU in the pre-video decoding stage, the effect of high-speed filtering of the frame that is the target of object area detection is achieved. It can be performed.

According to an embodiment of the present invention, despite a low computational amount, it is possible to achieve the effect of high-speed discrimination of frames containing object movement with high accuracy.

According to an embodiment of the present invention, it is possible to determine whether there is motion in a frame only through an algebraic operation based on information about the mode included in the syntax information of the CU without decoding the image.

According to one embodiment of the present invention, the amount of computation can be reduced to a significant level by performing object detection, object analysis, etc. only on frames in which movement is determined to exist.

According to one embodiment of the present invention, although a significant computational load is required for recent CCTV high-definition images, it is possible to achieve the effect of reducing the analysis target area to a significant level with only frames in which movement exists.

Figure 1 schematically shows the overall steps of a method for detecting a frame in which motion exists according to an embodiment of the present invention.
Figures 2 to 7 schematically show the CTU and CU of HEVC according to an embodiment of the present invention.
Figure 8 exemplarily shows video data including an I frame, a B frame, and a P frame.
Figure 9 shows the SYNTAX of a coding unit in the HEVC standard.
Figure 10 schematically shows the process of extracting learning video data according to an embodiment of the present invention.
Figure 11 schematically shows the process of extracting critical values according to an embodiment of the present invention.
Figure 12 schematically shows the process of the motion frame determination step according to an embodiment of the present invention.
Figure 13 exemplarily shows ratio information in an example video frame.
Figure 14 exemplarily shows a frame with movement and a frame without movement.
Figure 15 exemplarily shows the internal configuration of a computing device according to an embodiment of the present invention.

Various embodiments are now described with reference to the drawings, in which like reference numbers are used to indicate like elements throughout the drawings. In this specification, various descriptions are presented to provide an understanding of the invention. However, it is clear that these embodiments may be practiced without these specific descriptions. In other instances, well-known structures and devices are presented in block diagram form to facilitate describing the embodiments.

As used herein, the terms "component", "module", "system", "part", etc. refer to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or an implementation of software. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, a thread of execution, a program, and/or a computer. For example, both an application running on a computing device and the computing device can be a component. One or more components may reside within a processor and/or thread of execution, and a component may reside within a computer

It can be localized within the computer, or distributed between two or more computers. Additionally, these components can execute from various computer-readable media having various data structures stored thereon. Components can transmit, for example, signals with one or more data packets (e.g., data and/or signals from one component interacting with other components in a local system, standard deviation system, to other systems and over a network such as the Internet). Data) may be communicated through local and/or remote processing.

Additionally, the terms "comprise" and/or "comprising" mean that the feature and/or element is present, but exclude the presence or addition of one or more other features, elements and/or groups thereof. It should be understood as not doing so.

Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those skilled in the art to which the present invention pertains. It has the same meaning as Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless clearly defined in the embodiments of the present invention, have an ideal or excessively formal meaning. It is not interpreted as

Figure 1 schematically shows the overall steps of a method for detecting a frame in which motion exists according to an embodiment of the present invention.

The method of detecting a frame in which motion exists according to an embodiment of the present invention is performed in a computing system that includes one or more processors and a main memory that stores instructions executable by the processors.

A method of detecting a frame in which such movement exists includes a frame extraction step (S100) of extracting a P frame or B frame from video data; For the P frame or B frame, an intra mode confirmation step (S200) of checking information about whether each of a plurality of coding units included in each frame is an intra mode; A ratio information calculation step (S300) of calculating ratio information of a coding unit corresponding to an intra mode among a plurality of coding units for each of the P frame or B frame; and a motion frame determination step (S400) of determining that the P frame or B frame is a frame in which motion exists or a frame in which motion does not exist, based on the ratio information of the P frame or B frame.

In one embodiment of the present invention, S100, S200, S300, and S400 include a frame extraction unit 100, an intra mode confirmation unit 200, a ratio information calculation unit 300, and a motion frame determination unit 400, respectively. It is performed by.

Preferably, the video data may include video data encoded according to the HEVC standard.

Figures 2 to 7 schematically show the CTU and CU of HEVC according to an embodiment of the present invention.

First, in the hierarchical coding structure, which is the biggest feature of HEVC, HEVC's CTU (Coding Tree Unit), CU (Coding Unit), CTB (Coding Tree Block), CB (Coding Block), PB (Prediction Block), and TB ( Transform Block is a concept that corresponds to macro block and block in existing standards. In order to efficiently encode large-resolution images, a larger coding unit is required.

The problem here is that small details in the image cannot be ignored, so small-sized blocks such as 4×4 are also needed.

To solve these problems, HEVC uses a flexible and effective hierarchical coding structure. In the first step of encoding, one video, or frame, is divided into multiple CTUs as shown in FIG. 2. The horizontal and vertical information of the CTU is transmitted to the decoder through SPS (Sequence Parameter Set), and all CTUs have the same size for the entire image. The sizes of CTUs available in HEVC are 64×64, 32×32, or 16×16.

Before explaining CTB, let's look at the naming regulations in the HEVC standard. Among the various terms used in the description of the hierarchical coding structure of the HEVC standard, there are those referred to as Unit and those referred to as Block.

Units, such as CTU or CU, are coding logical units and are information that is encoded into an actual bitstream. On the other hand, those referred to as Block, such as CTB, CB, PB, or TB, refer to video frame buffers.

Meanwhile, CTU, one of the logical units, is divided into a total of three CTBs and related syntax elements, as shown in FIG. 3. The luminance (Luma) CTB has the same size as the CTU of 16×16 to 64×64, and the chrominance (Cb and Cr) CTB has different sizes depending on the chrominance format.

The CTB is too large to determine whether to perform intra- or inter-screen prediction. Therefore, each CTB can be divided into multiple CBs, and the size of the available CBs varies from the size of the CTB to 8×8. Figure 3 shows how a 64×64 CTB is divided into multiple CBs.

CB becomes the root of the prediction tree and transformation tree, and prediction type information that indicates whether to perform intra-screen prediction (intra mode) or inter-screen prediction (inter mode) is encoded at the CU level.

The CU consists of three CBs and related syntax elements, as shown in Figure 4. CB is sufficient to determine the prediction type, but is too large to determine the motion vector in inter-prediction or intra-prediction mode.

For example, if a very small object, such as an eye, is moving in the middle part of an 8×8 CB, rather than using one motion vector (MV) for the 8×8 CB, divide the CB into smaller pieces for each part. It would be better to use different motion vectors for The result used is PB. Any CB can be divided into PBs according to temporal/temporal predictability.

Figure 5 shows the available PB partitions. The basic unit used in conversion coding after prediction is TB. As explained previously, it is not appropriate to perform the conversion in CB units because CB can contain both complex high-frequency parts and flat low-frequency parts.

Therefore, any CB is divided into smaller transform blocks, TBs. In HEVC, for any CB, TB and PB are determined independently, so a TB larger than PB can be used.

Figure 6 shows an example where one CB is divided into TB.

Meanwhile, intra prediction is a method of performing prediction using the correlation between adjacent pixels within one screen. In general, the prediction efficiency is not as high as inter-screen prediction due to the small amount of data that can be referenced, but it has the advantage that encoding is performed independently and the encoding speed is fast. Unlike H.264/MPEG-4AVC, which uses up to 9 prediction modes, HEVC intra-picture coding uses DC and Planar modes and 33 directional prediction modes, as shown in FIG. 7.

Figure 8 exemplarily shows video data including an I frame, a B frame, and a P frame.

The video portion of a typical video consists of I frames (frames shown as “I”), P frames (frames shown as “P”), and B frames (frames shown as “B”).

An I frame is a key frame that contains the entire image, can function as an access point in a video file, corresponds to an independently encoded frame, and has a low compression rate.

Meanwhile, in the case of a P frame, it is a frame created by forward prediction with reference to a previous I frame or P frame, and does not correspond to an independently encoded frame. Such P frames have a higher compression rate than I frames. Here, the “previous” frame refers to not only the immediately preceding frame but also one of a plurality of frames that exist before the frame, and the “post” frame refers to not only the next frame but also one of the plurality of frames that exist after the frame. It means one.

Meanwhile, in the case of the B frame, it is a frame created by forward and backward prediction with reference to the previous frame and the subsequent frame, and does not correspond to an independently encoded frame. Such B frames have a higher compression rate than I and P frames. Accordingly, the independently encoded frame may correspond to an I frame, and the non-independently encoded frame may correspond to the remaining B frame or P frame.

The B and P frames correspond to reference frames, and preferably, the object area detection unit performs object area detection on these reference frames.

Figure 9 shows the SYNTAX of a coding unit in the HEVC standard.

According to the HEVC standard document “ITU-T H.265 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (11/2019) SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS Infrastructure of audiovisual services - Coding of moving video High efficiency video coding”, the syntax of CU mode is It's at the beginning of the CU. In this way, it is possible to check whether the CU mode is intra mode or inter mode without decoding the bitstream.

In the conventional technology, a decoding process for the image is required to judge or compare a static background with no movement and a video frame with movement, and there is a problem in that the computational load increases accordingly.

In contrast, in the method for detecting a frame in which motion exists according to embodiments of the present invention, it is possible to determine whether the frame in question is a frame in which motion exists using information in the part of the CU data indicating the mode of the CU without decoding. , it can have the effect of significantly reducing the computational load.

The intra mode confirmation multisystem checks information on whether each of the plurality of coding units included in each frame is an intra mode with respect to the P frame or B frame. In this process, as described above, it is determined whether each CU is intra mode or inter mode in a state where the bitstream has not been decoded.

Thereafter, in the ratio information calculation step, ratio information of a coding unit corresponding to the intra mode among a plurality of coding units is calculated for each of the P frame or B frame.

Preferably, for each frame, the ratio of CUs corresponding to the intra mode among all CUs may correspond to the ratio information. For example, if there are 100 CUs for a specific frame, and 30 of them are in intra mode, the ratio information may be 30% or 0.3.

Meanwhile, Figure 9(B) shows a state in which multiple CUs exist for a single frame. The ratio of CUs with intra mode among all CUs may correspond to the ratio information described above.

Figure 10 schematically shows the process of extracting learning video data according to an embodiment of the present invention.

In embodiments of the present invention, in the motion frame determination step, determination is performed based on a preset threshold value.

This preset threshold value is derived from training image data, which may include image data with movement and image data without movement.

More preferably, when trying to detect a frame with movement among video data captured by a CCTV installed in a specific area, the training video data includes past video data of the CCTV installed in the specific area. This method takes into account the specificity of each captured image and can detect frames in which movement exists.

The entire frame of the training image data in the initial state is shown in the upper part of Figure 10 (A). The hatched portion of the entire frame corresponds to a group of frames labeled as motionless frames.

Here, the training video data used to finally determine the threshold value is derived by grouping frames excluding the I frame among the frames without movement in the training video data in the initial state.

In another embodiment of the present invention, as shown in Figure 10 (B), when all frames of the training video data in the initial state correspond to frames without movement, all frames except the I frame among the training video data in the initial state are Frames are derived from training video data.

Based on this training image data, a threshold value used in the motion frame judgment step is determined, and this threshold value is compared with the derived value of the captured judgment target image data to determine whether or not it is a frame in which movement exists. do.

Figure 11 schematically shows the process of extracting critical values according to an embodiment of the present invention.

In the motion frame determination step, when the ratio information of each P frame or B frame is higher than a preset first threshold value, the corresponding P frame or B frame is determined as a frame in which movement exists.

In other words, the ratio information corresponding to the intra mode is extracted from the CUs included in each P frame or B frame of the video data corresponding to the analysis target, and if the extracted ratio information is higher than the first threshold value, the corresponding frame is extracted. It is judged by the frame in which movement exists. Here, the first threshold value can be derived from the above-described training image data.

Specifically, the first threshold value includes a learning ratio derivation step of calculating ratio information of coding units corresponding to an intra mode for each frame of training video data including a P frame or B frame labeled as having no movement; A first statistical value derivation step of deriving first statistical information from the ratio information of each of the plurality of P frames or B frames of the learning video data; and a first critical value determination step of determining the first critical value based on the first statistical information.

The learning ratio derivation step is the ratio of coding units corresponding to the intra mode for each frame of the training video data including the P frame or B frame labeled as having no movement from the training video data extracted as shown in FIG. 10. Calculate information.

The configuration shown in the upper part of FIG. 11 corresponds to each frame (B frame or P frame) of training video data. Each of these frames includes a plurality of CUs as shown in FIG. 11, and the ratio of CUs corresponding to the intra mode to all CUs is derived.

Thereafter, the first statistical value derivation step derives first statistical information from the ratio information of each of the plurality of P frames or B frames of the learning video data. The first statistical information may correspond to the average value and standard deviation value of the ratio information of the intra mode in all P frames and all B frames of the learning video data.

The first critical value determination step determines the first critical value based on the first statistical information.

In one embodiment of the present invention, the first threshold value may correspond to the average value + M*standard deviation value. Here M corresponds to a natural number. As an example, the first critical value may be determined by the following equation.

First critical value = average value + 2 * standard deviation value

That is, the first statistical value can detect a frame in which movement is determined based on the average value and standard deviation value of the ratio information of each of the plurality of P frames or B frames of the learning video data.

Meanwhile, in another embodiment of the present invention, the motion frame determination step is performed when the standard deviation value of the ratio information of a plurality of P frames or B frame groups is higher than the preset second threshold value, the related P frame or B frame is determined. The frame is judged as a frame in which movement exists.

Here, the second threshold value is also derived from the above-described training image data.

Preferably, the second threshold value is a learning rate derivation method that calculates ratio information of coding units corresponding to the intra mode for each frame of training video data including P frames or B frames labeled as having no motion. step; A second statistical value derivation step of deriving second statistical information from the ratio information of each of the plurality of P frames or B frames of the learning video data; and a second critical value determination step of determining the second critical value based on the second statistical information.

The second statistical value is determined based on the standard deviation value of the ratio information of each of the plurality of P frames or B frames of the learning video data.

In one embodiment of the present invention, in the second statistical value derivation step, the standard deviation value of the ratio information of all P frames and B frames of the training video data is derived, and this is derived as second statistical information, and the second threshold value is derived. is derived based on the standard deviation value.

The second critical value can be determined by the following equations.

(1) Second critical value = standard deviation value

(2) Second critical value = standard deviation value + constant 1

(3) Second critical value = constant 2 * standard deviation value + constant 3

Preferably, in one embodiment of the present invention, in the second statistical value derivation step, a plurality of standard deviation values of ratio information of P frames and B frames for each specific section of the training video data are derived, and the derived plurality of standards are derived. The average of the deviation values is derived as second statistical information, and the second threshold value is derived based on the average of the standard deviation values.

Specifically, if the training video data consists of 100 P frames or B frames, the standard deviation value of the ratio information from the 1st frame to the 10th frame is called standard deviation value #1, and in the 2nd frame, If the standard deviation value of the ratio information up to the 11th frame is called standard deviation value #2, 91 standard deviation values are derived from the total standard deviation value #1 to standard deviation value #91, and these 91 standard deviation values are derived. The average of the values may correspond to the average of the standard deviation values described above.

The second critical value can be determined by the following equations.

(1) Second critical value = average of standard deviation values

(2) Second critical value = average of standard deviation values + constant 1

(3) Second critical value = constant 2 * average of standard deviation values + constant 3

In one embodiment of the present invention, when the CU mode ratio information of grouped B frames and P frames of video data for which a frame with motion is to be detected is higher than the second threshold value, the frame is selected as a frame with motion. decide.

Specifically, in the motion frame determination step, when determining whether the Nth P frame or B frame is a frame in which movement exists, a plurality of frames from the Nth P frame or B frame to the N+Mth P frame or B frame are used. If the standard deviation value of the ratio information of the P frame or B frame group is higher than the preset second threshold value, the Nth P frame or B frame is determined as a frame in which movement exists.

Figure 12 schematically shows the process of the motion frame determination step according to an embodiment of the present invention.

In the present invention, in order to detect frames in which movement exists without omission with higher accuracy, as described above, it is checked whether the ratio of the intra mode of the CU of the individual frame of the image data to be detected exceeds the first threshold value and the individual frame and By simultaneously considering whether the standard deviation of the intra mode of the CU of a plurality of related frames exceeds the second threshold value, a frame in which movement exists can be detected by OR sum.

Specifically, in the motion frame determination step, when the ratio information of each P frame or B frame is higher than a preset first threshold value, the corresponding P frame or B frame is judged as a frame in which movement exists, respectively. Even if the ratio information of the P frame or B frame is not higher than the preset first threshold value, if the standard deviation value of the ratio information of a plurality of P frames or B frame groups is higher than the preset second threshold value, The related P frame or B frame can be judged as a frame in which movement exists.

That is, the upper part of FIG. 12 corresponds to the B frame or P frame of the video data to be judged. Intra-mode ratio information is derived from each of these frames, and in the first judgment step, the ratio information of the derived frames is compared with a first threshold value, and if the first threshold value is exceeded, a frame with movement is selected. It is judged by

Afterwards, in the second judgment step, the standard deviation of a plurality of related frames (for example, in the case of the 1st frame, from the 1st frame to the 1+Nth frame) is compared with the second threshold value, and the second threshold value is determined. If it exceeds it, it is judged to be a frame with movement.

Figure 13 exemplarily shows ratio information in an example video frame.

In FIG. 13, the horizontal axis indicates each frame, and the vertical axis indicates the ratio of intra-mode CUs among CUs of each frame.

As shown in Figure 13, in the case of a frame with movement, it can be seen that the ratio of intra-mode CUs increases rapidly from front to back. Therefore, as in the embodiment described with reference to FIG. 12, not only does the individual intra-mode CU ratio exceed the first threshold, but also the standard deviation of the CU ratio of the frames from the corresponding frame to after the preset range is determined. 2 By determining whether the threshold value is exceeded, a frame with movement can be derived more accurately.

That is, according to embodiments of the present invention, it is possible to determine whether motion exists in a frame only through algebraic operations based on information about the mode included in the syntax information of the CU without decoding the image. Afterwards, by performing object detection and object analysis only on frames in which movement is determined to exist, the amount of computation can be reduced to a significant level. Recently, a significant computational load is required for CCTV high-definition images, but according to the present invention, it is possible to reduce the analysis target area to a significant level with only frames in which movement exists.

Figure 14 exemplarily shows a frame with movement and a frame without movement.

Figure 15 exemplarily shows the internal configuration of a computing device according to an embodiment of the present invention.

Figure 13 is a diagram schematically showing examples of frames of video data.

Figure 15 exemplarily shows the internal configuration of a computing device according to an embodiment of the present invention. All or part of the components shown in FIG. 1 may include components of a computing device described later.

As shown in FIG. 15, the computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, and an input/output subsystem ( It may include at least an I/O subsystem (11400), a power circuit (11500), and a communication circuit (11600).

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. there is. The memory 11200 may include software modules, instruction sets, or various other data necessary for the operation of the computing device 11000.

At this time, access to the memory 11200 from other components such as the processor 11100 or the peripheral device interface 11300 may be controlled by the processor 11100. The processor 11100 may be composed of a single processor or a plurality of processors, and may include GPU and TPU type processors to improve calculation processing speed.

The peripheral interface 11300 may couple input and/or output peripherals of the computing device 11000 to the processor 11100 and the memory 11200. The processor 11100 may execute a software module or set of instructions stored in the memory 11200 to perform various functions for the computing device 11000 and process data.

The input/output subsystem 11400 can couple various input/output peripheral devices to the peripheral interface 11300. For example, the input/output subsystem 11400 may include a controller for coupling peripheral devices such as a monitor, keyboard, mouse, printer, or, if necessary, a touch screen or sensor to the peripheral device interface 11300. According to another aspect, input/output peripheral devices may be coupled to the peripheral interface 11300 without going through the input/output subsystem 11400.

Power circuit 11500 may supply power to all or some of the terminal's components. For example, power circuit 11500 may include a power management system, one or more power sources such as batteries or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or a power source. It may contain arbitrary other components for creation, management, and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, if necessary, the communication circuit 11600 may include an RF circuit to transmit and receive RF signals, also known as electromagnetic signals, to enable communication with other computing devices.

This embodiment of FIG. 15 is only an example of the computing device 11000, and the computing device 11000 omits some components shown in FIG. 15, further includes additional components not shown in FIG. 15, or 2. It may have a configuration or arrangement that combines more than one component. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or sensor in addition to the components shown in FIG. 15, and the communication circuit 1160 may be equipped with various communication methods (WiFi, 3G, LTE). , Bluetooth, NFC, Zigbee, etc.) may also include a circuit for RF communication. Components that can be included in the computing device 11000 may be implemented as hardware, software, or a combination of both hardware and software, including an integrated circuit specialized for one or more signal processing or applications.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to this embodiment may be composed of a PC-based program or a mobile terminal-specific application. The application to which the present invention is applied can be installed on a user terminal through a file provided by a file distribution system. As an example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request from the user terminal.

According to one embodiment of the present invention, when performing analysis such as recognition, tracking, identification, etc. of an object from encoded image data, the object area is detected in advance and image analysis is performed only on the detected object area, so faster It can be effective in processing video at high speed.

According to one embodiment of the present invention, by selectively removing unnecessary macroblocks when detecting an object area before image analysis, the amount of computation for object area detection is reduced, which has the effect of increasing the computation speed of the system as a whole. It can be performed.

According to one embodiment of the present invention, despite the minimum amount of calculation in object area detection, it is possible to detect an object area in which an object may exist with a fairly high level of accuracy.

According to an embodiment of the present invention, without changing the configuration of the decoder unit for decoding the video data, the codec is changed if it is a codec method using variable size blocks by using parameters generated during the decoding process in the decoder unit. Even so, it can have an effect that can be easily applied.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used by any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be stored or executed in a standardized manner on a networked computing device. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (12)

A method of detecting a frame in which movement is performed in a computing system including one or more processors and a main memory storing instructions executable by the processors, comprising:
A frame extraction step of extracting a P frame or B frame from video data;
For the P frame or B frame, an intra mode confirmation step of checking information about whether each of a plurality of coding units included in each frame is an intra mode;
A ratio information calculation step of calculating ratio information of a coding unit corresponding to an intra mode among a plurality of coding units for each of the P frame or B frame; and
Based on the ratio information of the P frame or B frame, a motion frame determination step of determining that the P frame or B frame is a frame in which motion exists or a frame in which motion does not exist; a frame in which motion exists, including; Detection method.
In claim 1,
A method of detecting a frame in which motion exists, wherein the video data includes video data encoded according to the HEVC protocol.
In claim 1,
The motion frame judgment step is,
When the ratio information of each P frame or B frame is higher than a preset first threshold, a method of detecting a frame in which motion is present, determines that the P frame or B frame in question is a frame in which motion is present.
delete delete In claim 1,
The motion frame judgment step is,
When the standard deviation value of the ratio information of a plurality of P-frames or B-frame groups is higher than the preset second threshold, the relevant P-frame or B-frame is judged as a frame with motion, and the frame with motion is determined. Detection method.
In claim 6,
The motion frame judgment step is,
When determining whether the Nth P frame or B frame is a frame in which movement exists, the above ratio of a plurality of P frames or B frame groups from the Nth P frame or B frame to the N+Mth P frame or B frame A method of detecting a frame with motion, where the Nth P frame or B frame is determined as a frame with motion when the standard deviation value of information is higher than a preset second threshold value.
delete delete In claim 1,
The motion frame judgment step is,
If the ratio information of each P frame or B frame is higher than the preset first threshold value, the corresponding P frame or B frame is judged as a frame in which movement exists,
When the standard deviation value of the ratio information of a plurality of P-frames or B-frame groups is higher than the preset second threshold, the corresponding P-frame or B-frame is judged as a frame with motion, and the frame with motion is determined. Detection method.
A detection device for frames in which movement is present, comprising one or more processors and a main memory storing instructions executable by the processors,
A frame extraction unit that extracts P frames or B frames from video data;
For the P frame or B frame, an intra mode confirmation unit that checks information on whether each of a plurality of coding units included in each frame is an intra mode;
A ratio information calculation unit that calculates ratio information of a coding unit corresponding to an intra mode among a plurality of coding units for each of the P frame or B frame; and
A frame in which movement exists, including a motion frame determination unit that determines that the P frame or B frame is a frame in which movement exists or a frame in which movement does not exist, based on the ratio information of the P frame or B frame. detection device
1. A computer program stored on a computer-readable medium comprising a plurality of instructions executed by one or more processors, comprising:
The computer program is,
A frame extraction step of extracting a P frame or B frame from video data;
For the P frame or B frame, an intra mode confirmation step of checking information about whether each of a plurality of coding units included in each frame is an intra mode;
A ratio information calculation step of calculating ratio information of a coding unit corresponding to an intra mode among a plurality of coding units for each of the P frame or B frame; and
A computer program comprising; a motion frame determination step of determining that the P frame or B frame is a frame in which motion exists or a frame in which motion does not exist, based on the ratio information of the P frame or B frame.