KR102243049B1 - Apparatus and method for providing clip video from cctv image - Google Patents

Abstract

Disclosed is an apparatus and method for providing a clip image from a CCTV image, and a method of providing a clip image from a CCTV image according to an embodiment of the present application includes receiving a bit stream for a CCTV image, a predetermined situation to be searched Receiving a user input associated with, determining a Group of Picture (GOP) corresponding to the predetermined situation through a machine learning model previously learned based on the bit stream and the user input, the determined GOP, Determining a GOP set including a preceding GOP and a following GOP for the determined GOP, and decoding a bit stream corresponding to the GOP set to provide a clip image corresponding to the GOP set.

Description

Device and method for providing clip video from CCTV video {APPARATUS AND METHOD FOR PROVIDING CLIP VIDEO FROM CCTV IMAGE}

The present application relates to an apparatus and method for providing a clip image from a CCTV image.

In recent years, the number of closed circuit televisions (CCTVs) installed for various purposes is gradually increasing. In addition, with the development of image technology, CCTV images are also gradually becoming clearer and developing to high definition. Accordingly, in order to store high-definition CCTV images, a larger storage space must be secured, and it is required to compress CCTV images with high efficiency.

In particular, in the CCTV system, as the occurrence of various crimes and theft accidents increases, users' expectations for their functions are also increasing. Existing CCTV systems are mainly used to detect criminals after a crime or theft occurs. However, it is very difficult to specify when a crime or theft occurs from CCTV images continuously photographed for a long time or when a specific person appears. To this end, conventionally, it was only possible for an observer to observe the entire time interval of a corresponding CCTV image, or to determine a certain time interval based on statements or evidence from witnesses.

In this regard, a specific part of the image including motion or object corresponding to a certain situation that the observer wants to search for the captured CCTV image is automatically detected, and a clip image of a certain length centered on the part is provided. The device or method to be performed is not known or commonly used.

The technology behind the present application is disclosed in Korean Patent Publication No. 10-1176743.

The present application is to solve the problems of the prior art described above, based on a predetermined situation that the user wants to search for from the CCTV image, detects a specific part in the CCTV image corresponding to the situation, and provides a clip image to the user based on this. It is an object of the present invention to provide an apparatus and method.

The present application is to solve the problems of the prior art described above, since it is not necessary to decode all sections of a CCTV video having a long playback time, a large amount of data can be analyzed using a small amount of computational resources in the process of searching for a specific situation. It is an object of the present invention to provide an apparatus and method for providing a clip image from a CCTV image to be used.

However, the technical problem to be achieved by the embodiments of the present application is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the method of providing a clip image from a CCTV image according to an embodiment of the present application includes the steps of receiving a bit stream for the CCTV image, in connection with a predetermined situation to be searched. Receiving a user input, determining a Group of Picture (GOP) corresponding to the predetermined situation through a pre-learned machine learning model based on the bit stream and the user input, the determined GOP, and the determined GOP Determining a GOP set including a preceding GOP and a following GOP for, and decoding a bit stream corresponding to the GOP set to provide a clip image corresponding to the GOP set.

In addition, the determining of a GOP (Group of Picture) corresponding to the predetermined situation may include dividing the bit stream in units of GOPs including a plurality of frames, and header data from the bit stream. Acquiring, obtaining motion data of each of the GOPs divided based on the header data, generating a partially decoded image using the obtained motion data, and the pre-learned based on the partially decoded image It may include determining a GOP corresponding to a partially decoded image including a motion associated with a predetermined situation to be searched through the machine learning model as a GOP corresponding to the predetermined situation.

In addition, generating the partial decoded image may include generating the partial decoded image based on a motion composition (MC) based on a prediction unit (PU) of the GOP.

In addition, the preceding GOP may include a predetermined number of GOPs temporally preceding the determined GOP, and the following GOP may include a predetermined number of GOPs temporally following the determined GOP. have.

In addition, the preceding GOP may include two or three GOPs temporally preceding the determined GOP, and the following GOP may include two or three GOPs temporally following the determined GOP. have.

In addition, in the determining of the GOP set, the GOP set may be determined so that the total playback time of the clip image is 60 seconds or less.

In addition, the predetermined situation may include at least one of a situation associated with a body movement of a subject or a situation in which a predetermined object is detected in the CCTV image.

In addition, the previously learned machine learning model may be learned by a deep learning technique.

Meanwhile, an apparatus for providing a clip image from a CCTV image according to an embodiment of the present invention includes an input receiver configured to receive a bit stream for a CCTV image and receive a user input associated with a predetermined situation to be searched, the bit stream. And a GOP determination unit for determining a GOP (Group of Picture) corresponding to the predetermined situation through a machine learning model previously learned based on the user input, the determined GOP, and a preceding GOP and a subsequent GOP for the determined GOP. It may include a clip image providing unit determining a GOP set to include, decoding a bit stream corresponding to the GOP set, and providing a clip image corresponding to the GOP set.

In addition, the GOP determination unit includes a GOP dividing unit that divides the bit stream in units of GOPs including a plurality of frames, obtains header data from the bit stream, and divides the GOP based on the header data. A data acquisition unit that acquires each motion data, a partial decoding unit that generates a partially decoded image using the obtained motion data, and a predetermined to be searched through the pre-learned machine learning model based on the partially decoded image. It may include a machine learning unit that determines a GOP corresponding to a partial decoded image including a motion associated with a situation of as a GOP corresponding to the predetermined situation.

In addition, the partial decoder may generate the partial decoded image based on a motion composition (MC) based on a prediction unit (PU) of the GOP.

In addition, the clip image providing unit may determine the GOP set such that the total playback time of the clip image is 60 seconds or less.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present application. In addition to the above-described exemplary embodiments, additional embodiments may exist in the drawings and detailed description of the invention.

According to the above-described problem solving means of the present application, a clip image can be provided to the user based on the detection of a specific part in the CCTV image corresponding to the situation based on a predetermined situation that the user wants to search for from the CCTV image.

According to the above-described problem solving means of the present application, it is not necessary to decode all sections of a CCTV image having a long playback time, so that a large amount of data can be analyzed using a small amount of computational resources in the process of searching for a specific situation.

However, the effect obtainable in the present application is not limited to the above-described effects, and other effects may exist.

1 is a schematic configuration diagram of a system for providing a clip image based on a CCTV image including a device for providing a clip image from a CCTV image according to an embodiment of the present disclosure.
FIG. 2 is a diagram illustrating a process of determining a GOP corresponding to a predetermined situation to be searched by an apparatus for providing a clip image from a CCTV image according to an embodiment of the present disclosure and providing a clip image.
3 is a schematic configuration diagram of an apparatus for providing a clip image from a CCTV image according to an embodiment of the present application.
4 is a schematic configuration diagram of a GOP determination unit according to an embodiment of the present application.
5 is a flowchart illustrating a method of providing a clip image from a CCTV image according to an embodiment of the present application.
6 is a flowchart illustrating a method of determining a GOP corresponding to a predetermined situation according to an embodiment of the present disclosure.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present application. However, the present application may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted in order to clearly describe the present application, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the present specification, when a part is said to be "connected" with another part, it is not only the case that it is "directly connected", but also "electrically connected" or "indirectly connected" with another element interposed therebetween. "Including the case.

Throughout the present specification, when a member is positioned "on", "upper", "upper", "under", "lower", and "lower" of another member, this means that a member is located on another member. This includes not only the case where they are in contact, but also the case where another member exists between the two members.

In the entire specification of the present application, when a certain part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

1 is a schematic configuration diagram of a system for providing a clip image based on a CCTV image including a device for providing a clip image from a CCTV image according to an embodiment of the present application.

Referring to FIG. 1, a system for providing a clip image based on a CCTV image according to an embodiment of the present application 10 is an apparatus 100 for providing a clip image from a CCTV image according to an embodiment of the present application (hereinafter, referred to as'providing a clip image). The device 100'), a CCTV photographing device 200, a user terminal 20, and a network 30 may be included.

For example, the user terminal 20 may include a smartphone, a smart pad, a tablet PC, a computer, a notebook, etc., a personal communication system (PCS), a global system for mobile communication (GSM), a personal digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro ( Wireless Broadband Internet) may include all types of wired/wireless communication devices such as terminals.

The clip image providing device 100, the user terminal 20, and the CCTV photographing device 200 may be connected to each other through a network 30, and the network 30 provides information between each node such as terminals and servers. It refers to a connection structure that can be exchanged, and examples of such a network include a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a 5G network, a World Interoperability for Microwave Access (WIMAX) network, and the Internet. ), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), wifi network, Bluetooth network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) networks, etc. are included, but are not limited thereto.

The clip image providing apparatus 100 may receive a bit stream for a CCTV image from the CCTV photographing apparatus 200. According to an embodiment of the present application, the CCTV photographing apparatus 200 may include a memory for storing a captured CCTV image or a bit stream compressed (encoded) through an encoder, or may be linked to the memory, The clip image providing apparatus 100 may receive a bit stream for a CCTV image from the memory.

In addition, the CCTV photographing apparatus 200 in the present application may be provided in plural so that each photographs a different space or each photographs a different direction. In this case, the clip image providing apparatus 100 may receive a bit stream for a CCTV image from each of the plurality of CCTV photographing apparatuses 200.

The clip image providing apparatus 100 may receive a user input associated with a predetermined situation to be searched from the user terminal 20. Here, the predetermined situation to be searched may include at least one of a situation associated with the body movement of the person to be photographed in the CCTV image or a situation in which a predetermined molar body is detected in the CCTV image, but is not limited thereto, and the user terminal It can be determined in consideration of various situations that the user of (20) wants to search from the CCTV image. In addition, the predetermined situation to be searched may be related to the purpose of the user (observer) to check the CCTV image.

Exemplarily, the predetermined situation to be searched is a crime situation related to injury, assault, theft, escape, arson, etc., a physical movement of the subject that may appear in a disaster situation related to a traffic accident, fire, explosion, etc., a certain object itself, It may include a situation in which a motion of a predetermined object or the like is detected.

More specifically, in relation to the criminal situation, the situation related to the movement of the body is a situation in which the filmed person crosses a wall, flees, harms someone else, damages a door or window, etc. I can. In addition, a situation in which an object related to a criminal situation is detected may be a situation in which a weapon, a vehicle, or a specific tool is detected.

However, it should be understood that the above description of a predetermined situation to be searched is exemplary for aiding understanding.

In addition, according to an embodiment of the present application, the user input associated with a predetermined situation to be searched is a voice input type in which the content uttered by the user with respect to the user terminal 20 equipped with the voice recognition function is input as an input, a keyboard. , It may have various formats including text or keyword input type in which content typed through a separate input device such as a touch screen is input.

In addition, according to an embodiment of the present application, the clip image providing device 100 provides a list of situations in which the clip image providing device 100 can navigate to the user terminal 20 in a list form, a combo box form, a check box form, etc. User input may be received in a manner that a user (an observer) provided in the form of and provided with a list selects a specific searchable situation from the list.

In addition, the user input according to the exemplary embodiment of the present application may include information on a time zone of a CCTV image that the user (observer) wants to search, information on a region to be photographed, information on a photographing direction, and the like. For example, the user may input a specific date (or section) for checking CCTV images, a specific location, or a specific photographing direction.

In addition, a user input according to an embodiment of the present application may include information on a person to be photographed in a CCTV image that the user (observer) wants to search. For example, when a user (observer) wants to search for a predetermined situation for a male photographee, this intention may be included in the user input. In this regard, the previously learned machine learning model, which will be described later, may determine a GOP suitable for a situation to be searched in consideration of the intention of the user (observer). That is, even if there is a GOP including the body movement of the person to be photographed that the user (observer) wants to search for, when the corresponding person is a female, it may be determined that the corresponding GOP does not correspond to a predetermined situation.

In addition, referring to FIG. 1, a user input associated with a predetermined situation to be searched is shown as being input through the user terminal 20 and transmitted to the clip image providing apparatus 100, but is not limited thereto, According to an embodiment, it may be implemented so that a user (observer) can directly input a user input to the clip image providing apparatus 100.

The clip image providing apparatus 100 may determine a Group of Picture (GOP) corresponding to a predetermined situation through a pre-learned machine learning model based on a bit stream and a user input.

Here, a group of picture (GOP) may be understood as a unit obtained by dividing a corresponding CCTV image (or an entire bit stream of a corresponding CCTV image), and one GOP may include a plurality of frames. In addition, the above-described GOP may be differently referred to as a GOF (Group of Frame) according to an embodiment. Exemplarily, the clip image providing apparatus 100 may divide a CCTV image (or the entire bit stream of a CCTV image) into N GOPs (GOP 1, GOP 2, ..., GOP N), and each GOP May be divided to include M frames (Frame 1, Frame 2, ..., Frame M). In addition, the description of the GOP number throughout the specification of the present application may mean a GOP that follows (reproduced later) based on a reproduction order from GOP 1 to GOP N. That is, the first GOP in the reproduction order may be GOP 1, and the last GOP in the reproduction order may be GOP N.

In addition, the clip image providing apparatus 100 may determine a GOP set including the determined GOP, a preceding GOP for the determined GOP, and a subsequent GOP.

Hereinafter, after the clip image providing apparatus 100 receives a bit stream and a user input, based on this, it determines a GOP corresponding to the situation that the user wants to search for, and further provides a clip image based on the determined GOP to the user. The process of this will be described in detail with reference to FIG. 2.

Referring to FIG. 2, first, the apparatus 100 for providing a clip image may divide a bit stream for a CCTV image in units of a GOP including a plurality of frames. At this time, the divided GOP 40 may include N GOPs as described above (GOP 1, GOP 2, ..., GOP N).

Also, the clip image providing apparatus 100 may obtain header data from a bit stream. According to an embodiment of the present application, the header data may be obtained from a header file including various image information of a CCTV image that is a decoding target image.

In addition, for example, header data is various image information of CCTV images, information associated with the color of each frame (or each GOP) included in the CCTV image, information associated with luminance, information associated with the background area , Motion-related information, and the like. In addition, the fact that the header data may include the above-described various types of information means that it may include a position or address value of a bit representing a specific type of information among the entire bit stream encoded (compressed) for the CCTV image. Can be understood as.

In addition, the clip image providing apparatus 100 may obtain motion data of each of the divided GOPs 40 based on the header data.

According to an embodiment of the present disclosure, motion data may mean information related to motion included in a bit stream identified based on header data. For example, the motion data may be specific bits, parameters, flags, motion vectors, etc. including information related to the motion of an object in the GOP.

According to an embodiment of the present application, the clip image providing apparatus 100 analyzes motion data obtained based on header data to determine whether a movement (motion) of a person or an object is captured in a partial image corresponding to each GOP. It could be grasping.

For example, in the case of header data linked to a GOP for a portion in which only a still image is captured, motion data may not be included or a motion data value may be designated as a default value. In this case, the clip image providing apparatus 100 may determine that the GOP does not capture motion. As such, a GOP in which motion is not captured may be analyzed as being irrelevant to a predetermined situation to be searched, and thus may not be input into a previously learned machine learning model to be described later.

In summary, the clip image providing apparatus 100 according to an embodiment of the present application acquires and analyzes motion data of each of the GOPs 40 divided based on header data, so that a partial image corresponding to the GOP moves (motion ), a candidate GOP that is likely to be linked to a predetermined situation to be searched may be deduced.

Also, the clip image providing apparatus 100 may generate a partially decoded image by using the acquired motion data. Here, whether the generation of the partially decoded image using motion data is related to a predetermined situation to be searched by performing partial decoding based on motion data for each of the aforementioned candidate GOPs or the divided GOPs 40? It can be understood as generating an image reflecting information related to motion at a level that can be determined.

For example, the partially decoded image may include information related to motion, but may not include information related to a background area, information related to color, and the like.

That is, the apparatus 100 for providing a clip image according to an embodiment of the present application relates to the motion of the object in the analysis process for grasping information related to the motion of the object included in the candidate GOP or the divided GOP 40. Since partial decoding is performed with only motion data, excluding background area information and color information that are not present, CCTV images can be analyzed at high speed with fewer computational resources.

According to an embodiment of the present application, the apparatus 100 for providing a clip image is a partial decoded image based on a motion composition (MC) based on a prediction unit (PU) of the divided GOP 40. Can be created.

The High Efficiency Video Coding (HEVC) standard adopts an encoding/decoding scheme through block-based spatial prediction and temporal prediction. HEVC divides an encoding target image into rectangular blocks called a coding tree unit (CTU). In addition, each CTU may be divided into smaller rectangular blocks referred to as coding units (CUs). In addition, each CU may include one or more blocks referred to as prediction units (PUs). The PU may be used as a unit for performing the above-described spatial prediction or temporal prediction. For example, when a CU is encoded in an intra mode, each PU of the CU may have its own spatial prediction direction. If the CU is coded in the inter mode, each PU of the CU may have its own motion vector and an associated reference picture.

In this regard, the apparatus 100 for providing a clip image according to an embodiment of the present disclosure uses a prediction unit (PU) encoded in an inter mode of each of the divided GOPs 40 for temporal prediction as a unit. A partial decoded image may be generated based on the composition.

In addition, the clip image providing apparatus 100 may store a GOP corresponding to a partial decoded image including a motion associated with a predetermined situation to be searched through a machine learning model previously learned based on the partially decoded image. It can be determined by the matching GOP (41). In the present specification, the GOP 41 corresponding to a predetermined situation may be referred to differently as the GOP 41 determined (by a pre-learned machine learning model).

Here, the partially decoded image may be a learning target feature of the machine learning model in the present application. That is, the machine learning model in the present application may be a model that predicts motion (motion) or an object identified in the partially decoded image based on the partially decoded image and a motion or object associated with a certain situation. Furthermore, the machine learning model herein may determine whether a motion or a situation related to an object identified in the predicted partial decoded image corresponds to a predetermined situation that the user (observer) wants to search, and output the determination result. .

Exemplarily, the machine learning model in the present application analyzes the partially decoded image of each of the divided GOPs 40 when a predetermined situation that the user (observer) wants to search for is a'character falls situation', and the partial decoded image is performed. When the motion analyzed as being included in corresponds to the'a situation in which a person falls', a GOP corresponding to the corresponding partial decoded image may be determined as the GOP 41 corresponding to a predetermined situation.

In this case, referring to FIG. 2, the machine learning model in the present application may determine one GOP 41 corresponding to a predetermined situation, but the clip image providing apparatus 100 according to an exemplary embodiment of the present application It may be implemented to determine a plurality of GOPs 41 corresponding to the situation.

Exemplarily, the machine learning model in the present application probabilistically analyzes whether it meets a predetermined situation, and converts the GOP with the highest probability of matching the predetermined situation into a single GOP 41 that meets the predetermined situation. Can act to determine. As another example, the machine learning model of the present application may operate to determine the entire GOP as the GOP 41 corresponding to the predetermined situation when the probability of matching a predetermined situation is greater than or equal to a preset threshold probability.

According to an embodiment of the present application, the previously learned machine learning model may be learned by a deep learning technique, but this is only an exemplary description to aid understanding, and other machine learning embodiments are applied to the present idea. It should not be construed as limiting or limiting what is being done, for example, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), and a logistic. It may be related to machine learning techniques such as Logistic Regression and Random Forest.

In addition, according to an embodiment of the present application, the previously-learned machine learning model receives a large amount of images (especially, images in which information related to motion is partially decoded and reflected), and information associated with the motion of each image. It may be a model in which a correlation between and a predetermined situation implied by the corresponding movement is repeatedly learned. Therefore, when the partially decoded image for the GOP 40 divided into the previously learned machine learning model in the present application is newly input, the motion (motion) or object included in the partial decoded image is based on the previously learned correlation. It is possible to predict which motion or object is associated with any situation. Furthermore, the apparatus 100 for providing a clip image according to an embodiment of the present application may determine whether the situation shown in the partially decoded image predicted by the machine learning model previously learned corresponds to a predetermined situation that the user (observer) wants to search. It is possible to determine the GOP 41 corresponding to a predetermined situation that is the basis of the clip image 50. In addition, in the previously-learned machine learning model according to an embodiment of the present application, as the CCTV image is repeatedly provided to the clip image providing apparatus 100 of the present application, learning may be accumulated to improve prediction performance or analysis performance.

In addition, the clip image providing apparatus 100 may determine a GOP set 44 including the determined GOP 41, a preceding GOP 42 and a trailing GOP 43 for the determined GOP 41.

According to an exemplary embodiment of the present disclosure, the preceding GOP 42 may include a predetermined number of GOPs preceding the determined GOP 41 in time. For example, the preceding GOP 42 may include two or three GOPs that temporally precede the determined GOP, but is not limited thereto.

For example, when the clip image providing apparatus 100 determines a GOP corresponding to a predetermined situation to be searched by the user as GOP 7, when the preceding GOP 42 includes two GOPs, GOP 5 and GOP 6 It may include, and when three GOPs are included, GOPs 4 to 6 may be included.

According to the exemplary embodiment of the present application, the following GOP 43 may include a predetermined number of GOPs that are temporally followed by the determined GOP 41. For example, the following GOP 43 may include two or three GOPs that temporally precede the determined GOP, but is not limited thereto.

For example, when the clip image providing apparatus 100 determines a GOP corresponding to a predetermined situation to be searched by the user as GOP 7, when the following GOP 43 includes two GOPs, GOP 8 and GOP 9 It may include, and when three GOPs are included, GOPs 8 to 10 may be included.

In addition, when the clip image providing apparatus 100 determines a GOP corresponding to a predetermined situation that the user wants to search as GOP 7 and the preceding GOP 42 and the following GOP 43 each include three GOPs, GOP The set 44 may be determined to include GOP 4 to GOP 10.

In addition, the clip image providing apparatus 100 may provide a clip image 50 corresponding to the GOP set 44 by decoding a bit stream corresponding to the GOP set 44.

According to an embodiment of the present application, when there are a plurality of GOPs 41 determined through a previously learned machine learning model, a plurality of GOP sets 44 for each determined GOP 41 may be determined. Accordingly, a plurality of clip images 50 corresponding to each GOP set 44 may also be provided. Here, the clip image providing apparatus 100 may be implemented to provide a probability corresponding to a predetermined situation to be searched analyzed for each of the plurality of clip images 50 together with the clip image 50.

In addition, according to an embodiment, when the clip image providing apparatus 100 provides a plurality of clip images 50, the plurality of clip images 50 will be provided in an order with a high probability of matching a predetermined situation to be searched. I can.

In addition, according to an embodiment of the present application, when the GOP 41 corresponding to a plurality of predetermined situations is determined, the clip image providing apparatus 100 is the determined GOP 41, the preceding GOP 42, or the following GOP ( When a part of 43) overlaps, the GOP set 44 for the overlapping GOP 41 may be integrated to provide a single clip image 50.

For example, when the GOP 41 corresponding to a predetermined situation is determined as GOP 5 and GOP 8, and each of the preceding GOP 42 and the following GOP 43 includes three GOPs, the first predetermined situation is met. The GOP set 44 for GOP 5, which is the GOP 41, is GOP 2 to GOP 8, and the GOP set 44 for GOP 8, which is the GOP 41 corresponding to the second predetermined situation, is GOP 5 to GOP 11 Can be At this time, since the following GOP 43 for GOP 5 and the preceding GOP 42 for GOP 8 are overlapped, the clip image providing apparatus 100 provides a single GOP set 44 including GOPs 2 to 11 A clip image 50 may be provided as a basis.

In addition, according to the exemplary embodiment of the present disclosure, the apparatus 100 for providing a clip image may determine the GOP set 44 so that the total playback time of the clip image 50 is 60 seconds or less. Here, determining the GOP set 44 may be understood as determining at least one of the number of GOPs to be included in the preceding GOP 42 or the following GOP 43.

However, the total playback time of the above-described clip image 50 is not limited to 60 seconds, and the total playback time may be determined differently according to the type of a predetermined situation that the user wants to search for (that is, the preceding GOP 42). Alternatively, the number of GOPs to be included in the following GOPs 43 may be differently determined), according to an embodiment, the playback time of the clip image desired by the user in the user input transmitted to the clip image providing apparatus 100 by the user of the user terminal 20 It can be implemented to include information. For example, when a user who wants to check a clip image for 5 minutes (300 seconds) sets the playback time of the clip image to 5 minutes (300 seconds), the clip image providing apparatus 100 may include a GOP set 44 The number of GOPs to be included in the preceding GOP 42 or the following GOP 43 may be determined so that the clip image corresponding to) is close to a preset 5 minutes (300 seconds) by a predetermined or more.

In addition, after the GOP set 44 is determined, the clip image providing apparatus 100 decodes the bit stream corresponding to the GOP set 44 to provide the clip image 50 corresponding to the GOP set 44. I can.

According to an embodiment of the present application, in order to generate the clip image 50 corresponding to the GOP set 44, the clip image providing apparatus 100 performs full decoding of the bit stream corresponding to the GOP set 44. I can. The clip image 50 is an image that is generated by the clip image providing apparatus 100 and is finally provided to a user (observer), and corresponds to a part of the CCTV image provided so that the user can visually check it. Therefore, the full decoding for generating the clip image 50 refers to a process of decoding in consideration of the entire information included in the bit stream, unlike the partial decoding process for generating the partial decoded image input of the machine learning model described above. can do.

In other words, the partial decoded image input to the machine learning model in the present application is decoded to a level that can grasp information related to motion, and does not include information related to the background area, information related to color, etc. The decoded clip image 50 may be generated by reflecting all information associated with a background area and information associated with a color.

In addition, according to an exemplary embodiment of the present disclosure, the clip image providing apparatus 100 may provide the generated clip image 50 to the user terminal 20 or a separate image reproducing device (not shown).

In summary, the clip image providing apparatus 100 according to an embodiment of the present application receives a user input for a situation that the user wants to search for from a user, receives a compressed bit stream for a CCTV image, and corresponds to it based on this. Among the entire sections of the CCTV image, a partial section corresponding to a situation to be searched by the user may be specified, and a clip image in which a predetermined playback time is added before and after the specified partial section may be provided. Accordingly, the user can obtain the desired image by transmitting the desired situation to the clip image providing device 100 without the need to check or reproduce the entire time section of the CCTV image captured by the CCTV image photographing device 200. have.

According to the conventional CCTV system, it was very difficult for the observer to check the video from the CCTV image continuously photographed for a long period of time and the video close to the time when a certain situation occurred, and the entire bitstream of the CCTV video was decoded. If there is no additional information such as statements or evidence from witnesses, there is a limit to observe the entire time section of the CCTV video. On the other hand, the clip image providing apparatus 100 in the present application can search for a part related to the situation that the user (observer) wants to find, regardless of the entire time period of the previously captured CCTV image, and It is possible to provide a short clip image 50 in which the bit strip is decoded.

In addition, in terms of the amount of computation or computational processing speed required for decoding to provide an image to the user, the conventional CCTV system has no choice but to perform the entire decoding on a given bit stream, and the bit stream that satisfies a predetermined condition. Only a specific part could not be decoded and provided to the observer. On the other hand, the clip image providing apparatus 100 in the present application can generate a clip image 50 by partially decoding a partial bit stream as much as a specific part associated with a predetermined situation that an observer wants to search. Compared to the CCTV system, it has fewer computational resources and can provide the image desired by the user (observer) at a high speed. In other words, since it is sufficient for the clip image providing apparatus 100 of the present application to decode only the bit stream corresponding to the finally searched portion, there is an effect of remarkably reducing the amount of computation required for decoding.

In addition, as described above, it is possible to search for a part related to the situation that the user (observer) wants to find by considering only motion-related information (i.e., based on motion data), not all information included in the bit stream. , It is possible to quickly determine the part related to the situation the user (observer) is looking for by analyzing a large part of the CCTV image at a high speed with less computational resources.

3 is a schematic configuration diagram of an apparatus for providing a clip image from a CCTV image according to an embodiment of the present application.

Referring to FIG. 3, an apparatus 100 for providing a clip image from a CCTV image according to an embodiment of the present application (hereinafter referred to as a'clip image providing apparatus 100') includes an input receiving unit 110, A GOP determination unit 120 and a clip image providing unit 130 may be included.

The input receiver 110 may receive a bit stream for a CCTV image, and may receive a user input associated with a predetermined situation to be searched. According to an embodiment of the present application, the input receiving unit 110 may receive a bit stream for a CCTV image from the CCTV photographing apparatus 200 and receive a user input from the user terminal 20.

The GOP determination unit 120 is a group of pictures (GOP) that meets a predetermined situation that a user wants to search through a machine learning model previously learned based on a bit stream received through the input receiving unit 110 and a user input. Can be determined. The detailed configuration, operation, etc. of the GOP determination unit 120 will be described later with reference to FIG. 4.

The clip image providing unit 130 may determine a GOP set 44 including the determined GOP 41, a preceding GOP 42 and a trailing GOP 43 for the determined GOP 41.

According to an exemplary embodiment of the present disclosure, the clip image providing unit 130 may determine the GOP set 44 so that the total playback time of the clip image is 60 seconds or less.

In addition, the clip image providing unit 130 may provide a clip image 50 corresponding to the GOP set 44 by decoding a bit stream corresponding to the GOP set 44.

4 is a schematic configuration diagram of a GOP determination unit according to an embodiment of the present application.

4, the GOP determination unit 120 according to an embodiment of the present application includes a GOP division unit 121, a data acquisition unit 122, a partial decoding unit 123, and a machine learning unit 124 can do.

The GOP dividing unit 121 can divide the bit stream in units of GOPs including a plurality of frames.

The data acquisition unit 122 may acquire header data from a bit stream and acquire motion data of each of the divided GOPs based on the acquired header data.

The partial decoder 123 may generate a partial decoded image by using the acquired motion data. According to an embodiment of the present application, the partial decoder 123 uses a prediction unit (PU) of a GOP (for example, each GOP of the divided GOP 40) as a unit of motion composition Composition, MC) may be used to generate a partially decoded image.

The machine learning unit 124 generates a GOP corresponding to a partial decoded image including a motion associated with a predetermined situation to be searched through a machine learning model previously learned based on the partially decoded image. Can be determined by

Hereinafter, based on the details described above, the operation flow of the present application will be briefly described.

5 is a flowchart illustrating a method of providing a clip image from a CCTV image according to an embodiment of the present application.

The method of providing a clip image from a CCTV image according to an embodiment of the present application illustrated in FIG. 5 may be performed by the apparatus 100 for providing a clip image from a CCTV image according to an exemplary embodiment of the present application described above. . Therefore, even if omitted below, the description of the apparatus 100 for providing a clip image from a CCTV image may be equally applied to a description of a method of providing a clip image from a CCTV image.

Referring to FIG. 5, in step S510, the input receiver 110 may receive a bit stream for a CCTV image.

Next, in step S520, the input receiving unit 110 may receive a user input associated with a predetermined situation to be searched.

Next, in step S530, the GOP determiner 120 may determine a Group of Picture (GOP) that corresponds to a predetermined situation to be searched through a machine learning model that has been previously learned based on a bit stream and a user input.

Next, in step S540, the clip image providing unit 130 may determine a GOP set 44 including the determined GOP 41, the preceding GOP 42 and the following GOP 43 for the determined GOP.

Next, in step S550, the clip image providing unit 130 may decode a bit stream corresponding to the GOP set 44 to provide a clip image 50 corresponding to the GOP set 44.

In the above description, steps S510 to S550 may be further divided into additional steps or may be combined into fewer steps, according to an embodiment of the present disclosure. In addition, some steps may be omitted as necessary, or the order between steps may be changed.

6 is a flowchart illustrating a method of determining a GOP corresponding to a predetermined situation according to an embodiment of the present disclosure.

The method of determining a GOP corresponding to a predetermined situation according to an embodiment of the present application illustrated in FIG. 6 is performed by the apparatus 100 for providing a clip image from a CCTV image according to an exemplary embodiment of the present application described above. I can. Accordingly, even if omitted below, the description of the apparatus 100 for providing a clip image from a CCTV image may be equally applied to a description of a method of determining a GOP suitable for a predetermined situation.

Referring to FIG. 6, in step S610, the GOP dividing unit 121 may divide a bit stream in units of a GOP including a plurality of frames.

Next, in step S620, the data acquisition unit 122 may acquire header data from the bit stream.

Next, in step S630, the data acquisition unit 122 may acquire motion data of each of the divided GOPs 40 based on the header data.

Next, in step S640, the partial decoder 123 may generate a partial decoded image by using the acquired motion data.

Next, in step S650, the machine learning unit 124 determines a GOP corresponding to a partial decoded image including a motion associated with a predetermined situation to be searched through a machine learning model previously learned based on the partial decoded image. It can be determined by the GOP 41 that matches the situation of.

In the above description, steps S610 to S650 may be further divided into additional steps or may be combined into fewer steps, according to an embodiment of the present disclosure. In addition, some steps may be omitted as necessary, or the order between steps may be changed.

The method of providing a clip image from a CCTV image according to an exemplary embodiment of the present disclosure may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

In addition, the method of providing a clip image from the CCTV image described above may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The foregoing description of the present application is for illustrative purposes only, and those of ordinary skill in the art to which the present application pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and are not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present application is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present application.

10: Clip video providing system based on CCTV video
100: a device that provides a clip image from a CCTV image
110: input receiver
120: GOP decision section
121: GOP division
122: data acquisition unit
123: partial decoding unit
124: Machine Learning Department
130: clip image providing unit
200: CCTV imaging device
20: user terminal
30: network
40: segmented GOP
41: determined GOP
42: Leading GOP
43: trailing GOP
44: GOP set
50: clip video

Claims (12)

In a method for providing a clip image from a CCTV image,
Receiving a bit stream obtained by compressing a CCTV image;
Receiving a user input associated with a predetermined situation to be searched;
Determining a GOP (Group of Picture) corresponding to the predetermined situation through a machine learning model previously learned based on the bit stream and the user input;
Determining a GOP set including the determined GOP, a preceding GOP and a subsequent GOP for the determined GOP; And
Decoding a bit stream corresponding to the GOP set to provide a clip image corresponding to the GOP set,
Including,
The step of determining a GOP (Group of Picture) corresponding to the predetermined situation,
Dividing the compressed bit stream in units of a GOP including a plurality of frames;
Obtaining header data from the compressed bit stream;
Obtaining motion data of each of the divided GOPs based on the header data;
Generating a partially decoded image using the acquired motion data; And
Determining a GOP corresponding to a partially decoded image including a motion associated with a predetermined situation to be searched through the machine learning model based on the partially decoded image as a GOP corresponding to the predetermined situation,
Including
Generating the partially decoded image includes:
The method for providing a clip image, wherein data including at least one of color information, luminance information, and background area information included in the header data is not decoded, but only the motion data is decoded. delete The method of claim 1,
Generating the partially decoded image includes:
The method for providing a clip image, wherein the partial decoded image is generated based on a motion composition (MC) based on a prediction unit (PU) of the GOP. The method of claim 1,
The preceding GOP includes a predetermined number of GOPs that temporally precede the determined GOP,
The following GOP includes a predetermined number of GOPs that follow in time with respect to the determined GOP,
The preceding GOP is,
Includes two or three GOPs preceding in time with respect to the determined GOP,
The trailing GOP,
The method of providing a clip image comprising two or three GOPs that are temporally followed by the determined GOP. The method of claim 4,
The step of determining the GOP set,
The method of providing a clip image, wherein the GOP set is determined so that the total playback time of the clip image is 60 seconds or less. The method of claim 1,
The predetermined situation is,
The method for providing a clip image comprising at least one of a situation associated with a body movement of a subject or a situation in which a predetermined object is detected in the CCTV image. The method of claim 1,
The pre-learned machine learning model,
A method of providing a clip image, which is learned by a deep learning technique. In the apparatus for providing a clip image from a CCTV image,
An input receiving unit for receiving a bit stream obtained by compressing a CCTV image and receiving a user input associated with a predetermined situation to be searched;
A GOP determination unit for determining a Group of Picture (GOP) corresponding to the predetermined situation through a machine learning model previously learned based on the bit stream and the user input; And
A clip image providing unit that determines a GOP set including the determined GOP, a preceding GOP and a subsequent GOP for the determined GOP, and provides a clip image corresponding to the GOP set by decoding a bit stream corresponding to the GOP set,
Including,
The GOP determination unit,
A GOP dividing unit for dividing the compressed bit stream in units of a GOP including a plurality of frames;
A data acquisition unit acquiring header data from the compressed bit stream and acquiring motion data of each of the divided GOPs based on the header data;
A partial decoder that generates a partial decoded image using the acquired motion data; And
Machine learning for determining a GOP corresponding to a partial decoded image including a motion associated with a predetermined situation to be searched through the machine learning model based on the partially decoded image as a GOP corresponding to the predetermined situation part,
Including,
The partial decoding unit,
The apparatus for providing a clip image, wherein data including at least one of color information, luminance information and background area information included in the header data is not decoded, but only the motion data is decoded. delete The method of claim 8,
The partial decoding unit,
The apparatus for providing a clip image, wherein the partial decoded image is generated based on a motion composition (MC) based on a prediction unit (PU) of the GOP. The method of claim 8,
The clip image providing unit,
To determine the GOP set so that the total reproduction time of the clip image is 60 seconds or less. A computer-readable recording medium in which a program for executing the method of any one of claims 1, 3 to 7 on a computer is recorded.

Images