KR101979375B1 - Method of predicting object behavior of surveillance video - Google Patents

Abstract

The present invention relates to a method for predicting an object behavior of a monitoring image, performed by a computer, which comprises the steps of: (a) recognizing an object in a monitoring image; (b) analyzing the object by determining an object state through a predetermined query of the object and determining an object behavior through a motion pattern of the object; (c) generating object metadata on the basis of the object state and the object behavior; and (d) classifying a behavior type of the object and predicting a future behavior of the object using a learning result of existing object metadata. According to an embodiment of the present invention, the step (d) may include a step of generating virtual learning data as the existing object metadata on the basis of the recognized object to perform neural network learning on the basis of deep learning. Also, the learning efficiency can be improved by predicting the future behavior of the object by using the learning result of the object metadata.

Description

Method of predicting object behavior of surveillance video {METHOD OF PREDICTING OBJECT BEHAVIOR OF SURVEILLANCE VIDEO}

The present invention relates to an object behavior prediction technique, and more particularly, to a method for predicting object behavior of a surveillance image that can improve learning efficiency by predicting future behavior of an object by using learning results of object metadata. will be.

The object analysis technology of an image may recognize an object from image data photographed using a surveillance camera, analyze the object, and use the image analysis process. The conventional object analysis technique of image has a disadvantage in that it takes a long time and its efficiency decreases because it analyzes the object based on the actual object in the image, analyzes behavior patterns, and performs the learning based on the analysis result.

Korean Patent Registration No. 10-1795798 (Nov. 02, 2017) relates to an intelligent video surveillance system, which receives a plurality of cameras each photographing a set area, and receiving images of the plurality of cameras, and analyzes the received images. An intelligent image processing apparatus for performing any one or more detection operations of object detection and region detection, and a monitoring apparatus connected to the intelligent image processing apparatus and receiving detection data and the captured image analyzed by the intelligent image processing apparatus. Include.

Korean Patent Laid-Open No. 10-2015-0112712 (2015.10.07) relates to an object behavior pattern CCTV image analysis server, and includes an object behavior pattern CCTV image composed of a microprocessor, an image analysis API, an object recognition module, and a transceiver. In the analysis server, the image analysis API receives the original image data captured by the digital surveillance camera and stores the original image data in the long-term storage memory of the main fisherman and the object recognition module in the hippocampal neural network memory; The transmission and reception unit measures the transmission traffic with the integrated crime center control server connected through a network and transmits to the main control unit; The main control unit transmits the low-capacity video compression file obtained by encrypting and compressing the original video data received by the object recognition module to the low-quality low-capacity data below the traffic measured by the transmission / reception unit to the integrated security center center control server through a network; The object recognition module stores the divided image by dividing the original image data received in the hippocampal neural network memory into one or more on the X-axis and Y-axis so as to match the X-Y coordinate value and the time data to digitally zoom in and store the divided image; The object recognition module transmits an object classification image, which classifies the object recognized after object recognition by the object recognition program, to the main control unit for each of the digital zoom-expanded divided image and the divided image; The main control part encrypts and compresses a risk behavior priority feature value by comparing the object pattern image and the behavior pattern image classifying the behavior pattern of the segmented image with the risk object and behavior pattern stored in the long-term storage memory. The event compression file is transmitted to the integrated crime center control server.

Korean Patent Publication No. 10-2015-0112712 (2015.10.07) Korean Patent Publication No. 10-2015-0112712 (2015.10.07)

An embodiment of the present invention is to provide a method for predicting object behavior of a computer-implemented surveillance video that can improve learning efficiency by predicting future behavior of an object by using learning results of object metadata.

An embodiment of the present invention provides a computer-implemented surveillance image that can classify object behavior types more efficiently and predict future behavior through object metadata generated based on query-based object state and movement pattern-based object behavior. To provide an object behavior prediction method of.

Among the embodiments, the method of predicting the object behavior of a computer-implemented surveillance image includes (a) recognizing an object in the surveillance image, (b) determining an object state through a preset query of the object, and determining a motion pattern of the object. Determining the object behavior through analyzing the object; (c) generating object metadata based on the object state and the object behavior; and (d) using the learning result of the existing object metadata to determine the object's behavior. Classifying the type of behavior and predicting future behavior of the object.

The step (b) may include determining an analysis section of the object based on whether the main appearance of the object in the surveillance image.

The step (b) may further include determining whether the main appearance of the object is based on the appearance region range of the object and the appearance time period of the object.

The step (c) may include determining the object state as the first sentence type according to the answer to the query and determining the object behavior as the second sentence type according to the movement pattern.

The step (c) may further include generating the object metadata through the first and second sentence types and generating an image search tag from the object metadata.

The step (c) may further include associating the image search tag with a start time of an image corresponding to the analysis section of the object.

Step (d) may include performing deep learning based neural network learning by generating virtual learning data as the existing object metadata based on the recognized object.

The step (d) may include providing a behavior type and future behavior of the object to the user terminal when the future behavior of the object is the same as the behavior defined in a predetermined specific scenario.

The disclosed technique can have the following effects. However, since a specific embodiment does not mean to include all of the following effects or only the following effects, it should not be understood that the scope of the disclosed technology is limited by this.

The object behavior prediction method of a computer-implemented surveillance image according to an embodiment of the present invention may improve learning efficiency by predicting future behavior of an object by using a learning result of object metadata.

According to an embodiment of the present invention, an object behavior prediction method of a computer-implemented surveillance image classifies an object's behavior type more efficiently through object metadata generated based on a query-based object state and a movement pattern-based object behavior. And predict future behavior.

1 is a diagram illustrating an object behavior prediction system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration of an object behavior prediction apparatus of FIG. 1.
3 is a block diagram illustrating functional elements of a processor in FIG. 2.
4 is a flowchart illustrating a process of executing an object behavior prediction procedure by the processor of FIG. 3.

Description of the present invention is only an embodiment for structural or functional description, the scope of the present invention should not be construed as limited by the embodiments described in the text. That is, since the embodiments may be variously modified and may have various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical idea. In addition, the objects or effects presented in the present invention does not mean that a specific embodiment should include all or only such effects, the scope of the present invention should not be understood as being limited thereby.

On the other hand, the meaning of the terms described in the present application should be understood as follows.

Terms such as "first" and "second" are intended to distinguish one component from another component, and the scope of rights should not be limited by these terms. For example, the first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being "connected" to another component, it should be understood that there may be other components in between, although it may be directly connected to the other component. On the other hand, when a component is referred to as being "directly connected" to another component, it should be understood that there is no other component in between. On the other hand, other expressions describing the relationship between the components, such as "between" and "immediately between" or "neighboring to" and "directly neighboring", should be interpreted as well.

Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprise" or "have" refer to a feature, number, step, operation, component, part or implementation thereof. It is to be understood that the combination is intended to be present and does not exclude in advance the possibility of the presence or addition of one or more other features or numbers, steps, operations, components, parts or combinations thereof.

In each step, an identification code (e.g., a, b, c, etc.) is used for convenience of description, and the identification code does not describe the order of the steps, and each step clearly indicates a specific order in context. Unless stated otherwise, they may occur out of the order noted. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer readable code on a computer readable recording medium, and the computer readable recording medium includes all kinds of recording devices in which data can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. Generally, the terms defined in the dictionary used are to be interpreted to coincide with the meanings in the context of the related art, and should not be interpreted as having ideal or excessively formal meanings unless clearly defined in the present application.

1 is a diagram illustrating an object behavior prediction system according to an embodiment of the present invention.

Referring to FIG. 1, the object behavior prediction system 100 may include an object behavior prediction apparatus 110, a video surveillance apparatus 120, and a user terminal 130.

The object behavior prediction apparatus 110 corresponds to a computing device that may be connected to the video surveillance apparatus 120 and the user terminal 130 through a network. In one embodiment, the object behavior prediction apparatus 110 may perform prediction of future behavior of the object based on the surveillance image received from the video surveillance apparatus 120.

The video surveillance apparatus 120 corresponds to a computing device that is connected to the object behavior prediction apparatus 110 through a network to provide a surveillance image to the object behavior prediction apparatus 110. In an embodiment, the video surveillance apparatus 120 may receive an image from the surveillance camera 125 through the surveillance camera 125 and the wired / wireless communication network or may be electrically connected, and perform video surveillance based on the received image. At least a portion of the image may be stored, and the received image or the stored image may be transmitted to the object behavior prediction apparatus 110 as a surveillance image. In one embodiment, the video surveillance apparatus 120 may analyze the received image to detect the movement of the object, and selectively store the image from which the movement of the object is detected, and predict the object behavior 110 as the surveillance image. Can be sent to.

In one embodiment, the video surveillance apparatus 120 may be implemented as a DVR (Digital Video Recorder), and may be used in a stand-alone set-top box or a PC capable of storing the image on a disk or playing the stored image. May include software. In one embodiment, the surveillance camera 125 may be implemented through at least one of an analog closed circuit television (CCTV) camera and an ultra high definition (UHD) CCTV, but is not limited thereto.

The user terminal 130 may correspond to a computing device that may be connected to the object behavior predicting apparatus 110 through a network, and may be implemented as, for example, a desktop, a laptop, a tablet PC, or a smartphone. In one embodiment, the user terminal 130 may be connected to the object behavior prediction apparatus 110 through the object behavior prediction agent, where the object behavior prediction agent is installed in the user terminal 130. With the approval of the user terminal 130 and the object behavior prediction apparatus 110 corresponds to the agent program which is software that can interoperate with each other, for example, it may be implemented as a mobile application.

In one embodiment, the object behavior prediction device 110 and the video surveillance device 120 may be implemented as mutually independent computing devices, but in another embodiment, may be implemented as a physically or functionally integrated computing device. have. Hereinafter, for convenience, it will be described with reference to the former embodiment.

FIG. 2 is a diagram illustrating a configuration of an object behavior prediction apparatus of FIG. 1.

Referring to FIG. 2, the object behavior prediction apparatus 110 may include a processor 210, a memory 220, a user input / output unit 230, a network input / output unit 240, and an object database 250.

The processor 210 may execute the object behavior prediction procedure in FIG. 4, manage the memory 220 that is read or written in this process, and synchronize between volatile memory and nonvolatile memory in the memory 220. You can schedule a time. The processor 210 may control the overall operation of the object behavior prediction apparatus 110, and may be electrically connected to the memory 220, the user input / output unit 230, the network input / output unit 240, and the object database 250. The data flow between them may be controlled and may be implemented as a central processing unit (CPU) of the object behavior prediction apparatus 110.

The memory 220 may include a secondary memory device which is implemented as a nonvolatile memory such as a solid state disk (SSD) or a hard disk drive (HDD), and is used to store overall data required for the object behavior predicting device 110. The main memory may include a main memory implemented with volatile memory such as random access memory (RAM). As such, the memory 220 may be implemented as volatile and nonvolatile memory, and if implemented as nonvolatile memory, the memory 220 may be connected through a hyperlink.

The user input / output unit 230 includes an environment for receiving user input and an environment for outputting specific information to the user, and includes, for example, a mouse, trackball, touch pad, graphic tablet, scanner, touch screen, keyboard, or pointing. It may include an input device including an adapter such as a device and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input / output unit 230 may correspond to a computing device connected through a remote connection, in which case, the object behavior prediction device 110 may be performed as a server.

The network input / output unit 240 may include an environment for connecting to the user terminal 130 through a network, and may include, for example, an adapter for local area network (LAN) communication.

The object database 250 may include learning results of existing object metadata as a database, and in one embodiment, may include a query and an object behavior prediction model preset for each object type. In one embodiment, the processor 210 may execute an object behavior prediction procedure based on the object database 250, perform deep learning based neural network learning, and perform big data analysis periodically or aperiodically to perform an object Database 250 may be updated.

3 is a block diagram illustrating functional elements of a processor in FIG. 2.

Referring to FIG. 3, the processor 210 may include an object recognizer 310, an object analyzer 320, an object metadata generator 330, an object future behavior predictor 340, and a controller 350. Can be.

The object recognition unit 310 recognizes the object in the surveillance image. In one embodiment, when the surveillance image is provided from the video surveillance apparatus 120, the object recognition unit 310 may detect the presence or absence of at least one object in the surveillance image by analyzing the surveillance image, and if so, the detected at least Shape information of each object may be analyzed to determine each object type (eg, a person or a vehicle).

In one embodiment, the object recognition unit 310 has a region size of at least a reference value each of the surveillance image based on the movement of the previous image or based on whether the color corresponds to a color similar to the surrounding area in the surveillance image. N (N is a natural number) may be divided into divided regions, and each of the divided regions may be analyzed to recognize at least one object. According to an embodiment, the object recognizer 310 may set N to a specific value based on Pan-Tilt-Zoom (PTZ) information of the surveillance image.

The object analyzer 320 determines an object state through a predetermined query of the object and analyzes the object by determining the object behavior through the movement pattern of the object. In one embodiment, the object analyzer 320 may determine the object state and object behavior by analyzing the object recognized in the surveillance image based on the previously learned object behavior prediction model.

The object analyzer 320 searches the object database 250 to obtain a query set for the object type of the object recognized in the surveillance image, and performs a query processing on the object based on the query to determine the object state of the object. For example, if the object type of the recognized object is a person, the object may be determined as the object state of the object by analyzing the object based on a plurality of query data set to match the person. In one embodiment, the query may be set differently for each object type by the designer or the user and stored and managed in the object database 250, and the first query for the overall shape (eg, the overall shape of the recognized object). A query as to whether it matches one of the previously stored at least one overall shape), a second query as to the detail shape (e.g., at least a detail shape of a specific portion (e.g., a face) of the recognized object is stored in advance). At least one of a third query on object properties (e.g., a query on whether a recognized object matches an object property associated with the object type) and a third query on object properties. Can be.

The object analyzer 320 analyzes the objects recognized in the surveillance image based on an object behavior prediction model previously stored in the object database 250 and includes a behavior analysis algorithm to determine an object behavior matching the object type of the corresponding object. For example, if the object type of the recognized object is a human, the object is analyzed based on the continuous movement pattern of the object in a series of surveillance images to determine [the person moving to the right] as the object behavior of the object. Can be.

The object analyzer 320 may determine the analysis period of the object based on whether the main appearance of the object in the surveillance image. In one embodiment, the object analyzing unit 320 may determine whether the main appearance of the object based on the appearance region range of the object and the appearance time period of the object. More specifically, the object analyzer 320 may include an appearance time interval (eg, 30 seconds) at which the object is recognized in a series of surveillance images most recently received during a specific time interval (eg, 1 minute) and Arithmetic operations (eg average or maximum-minimum) of the range of appearance areas occupying the entire screen range can be detected to determine whether they are major appearances. It may be set to an associated first analysis interval (eg, 1 minute) or a second analysis interval (eg, 30 seconds) associated with a non-major appearance subject. Here, the specific time interval and the first to second analysis intervals may be set by the designer or the user to the same or different values, respectively.

In one embodiment, the object analysis unit 320 may determine whether the main appearance of the object through the comparison between the detected appearance area range and the reference area range and the detected appearance time interval and the reference time interval. For example, the main appearance index r may be calculated based on Equation 1 below to determine whether the main appearance of the corresponding object is greater than or equal to a preset reference appearance index r ₀ . Here, each of the reference region range, the reference time interval, and the reference occurrence index may be set by the designer or the user, respectively.

[Equation 1]

(Where r denotes a major occurrence index, t denotes an appearance time interval, t ₀ denotes a reference time interval, a denotes an appearance region range, a ₀ denotes a reference region range, m is a natural number between 1 and 3, the larger the value, the larger the movement of the object (for example, m = 1: no movement, m = 2: static movement, m = 3: dynamic movement)

The object metadata generator 330 generates object metadata based on object state and object behavior. In one embodiment, the object metadata generator 330 may determine the object state as the first sentence type according to the answer to the query, and determine the object behavior as the second sentence type according to the movement pattern, and the first and second sentence types. Through this, object metadata can be created, and the object type of the object can be included as a separate sentence in the object metadata. For example, the object metadata generator 330 converts the object type [person] of the corresponding object determined by the object analyzer 320 into a 0th sentence and the keyword [wear bag] of the object state into a first sentence. And determine a keyword [move to right] of the object behavior of the object as the second sentence type, and generate object metadata including the determined 0 to second sentence types.

In one embodiment, the object metadata generation unit 330 determines the third sentence type based on whether the main appearance of the object analyzed by the object analysis unit 320 and the time interval determined to be the main appearance, the corresponding object meta Can be included in the data. For example, the object metadata generator 330 may analyze the [main appearance, time intervals A to B] when the object is analyzed as being the main appearance in [time intervals A to B] based on the entire time interval of the surveillance image. The third sentence can be determined.

The object metadata generator 330 may generate an image search tag from the object metadata. In one embodiment, the object metadata generator 330 may generate an image search tag composed of at least one tag based on at least one of an object type and first to third sentence types in the object metadata. For example, the object type [person] of the object, the keyword [bag-wear] of the first sentence type, the keyword [right-movement] of the second sentence type, and the keyword [main appearance, time interval A ~ B] of the third sentence type, respectively. By extracting the corresponding text as a tag, an image search tag composed of the tags can be generated.

The object metadata generator 330 may associate the image search tag with the start time of the image corresponding to the analysis section of the object. According to an embodiment, the object metadata generator 330 may index the image search tag generated through the object metadata at the start point of the surveillance video, and the specific tag in the image search tag by the user with respect to the surveillance video. When is searched for, a specific time point of the image combined with the searched specific tag may be detected so that the image is played from the specific time point.

In one embodiment, the object metadata generator 330 associates the generated image search tag with a start point of an image, and when the playback of the corresponding image starts, the corresponding image search is synchronized with a subsequent time interval from the start point. You can set a tag.

The object future behavior predicting unit 340 classifies the behavior type of the object by using the learning result of the existing object metadata and predicts the future behavior of the object. In an embodiment, the object future behavior prediction unit 340 may store the object behavior prediction model previously learned based on the existing object metadata in the object database 250 according to a previously stored learning algorithm, and store the stored object behavior prediction model. By applying the object metadata created in the above, the behavior type of the object can be classified and the future behavior can be predicted. For example, the object future behavior prediction unit 340 is defined as object metadata of ([person], [wear bag], [move to right]) in the analysis section of the surveillance image based on the object behavior prediction model. By analyzing the behavior type of the first object, the behavior type of the object can be classified as [parking move], and based on the classified behavior type and the object metadata, the future behavior of the object [move to the car] It can be predicted.

The object future behavior predicting unit 340 may provide the user terminal 130 with the behavior type and the future behavior of the object when the predicted future behavior of the object is the same as the behavior defined in the specific scenario. In one embodiment, the object future behavior predictor 340 may detect at least one specific scenario in which the same behavior as the future behavior of the predicted object is defined among the plurality of scenarios stored in the object behavior prediction model, and detected. For each of the at least one specific scenario, if the degree of matching with the object metadata and the classified behavior type of the object is calculated and determined to be equal to or greater than the reference matching degree, the behavior type of the object and the user terminal 130 associated with the surveillance image are determined. You can decide to provide future actions.

In one embodiment, the object future behavior predictor 340 corresponds to any one of a series of risk levels defined in the scenario in which the behavior on the scenario that matches the predicted future behavior with respect to each of the at least one detected specific scenario is determined. The risk level of the anticipated future behavior can be determined by detecting whether it is. In one embodiment, the object future behavior predicting unit 340 outputs the warning content through the sound module (not shown) or the video surveillance apparatus 120 when the determined risk level of future behavior is greater than or equal to the reference risk level. Alert content can be provided that encourages behavioral patterns, future actions, and risk alerts.

In one embodiment, the object future behavior predicting unit 340 corresponds to the surveillance image (not shown) on the control server (not shown) if the calculated matching degree is higher than the reference matching degree and the risk level of the predicted future behavior is higher than the reference risk level. The type of behavior of the object and the future behavior and the risk stage of the predicted future behavior may be transmitted to request a rescue and transmit an emergency message to the user terminal 130.

In an embodiment, the object future behavior predicting unit 340 may calculate the object future behavior predictor 340 through the object behavior prediction agent installed in the user terminal 130 when the calculated matching degree is greater than or equal to the reference matching degree and the risk level of the predicted future behavior is less than the reference risk level. The object behavior prediction message including the behavior type and future behavior of the object may be transmitted.

In an embodiment, the object future behavior predicting unit 340 calculates the object metadata of the object for a specific time interval from the point in time when the calculated matching degree is less than the reference matching degree and the risk level of the predicted future behavior is less than the reference risk level. And behavior type can be tracked to generate a comparison with the scenario.

The object future behavior predicting unit 340 may perform deep learning based neural network learning by generating virtual learning data as existing object metadata based on the recognized object. In one embodiment, the object future behavior predicting unit 340, when the prediction of the future behavior of the object is completed by using the object metadata generated in the prediction process as the virtual learning data to perform deep learning-based neural network learning to pre-stored objects Update behavior prediction models. That is, the object future behavior prediction unit 340 learns the stored object behavior prediction model based on neural network learning through the object metadata, classifies the object behavior type based on the object metadata, and predicts the future behavior. Can improve the efficiency.

According to an embodiment, the object future behavior predicting unit 340 may use the object metadata corresponding to at least the reference level as the risk level of the future behavior predicted in the neural network learning process as the virtual learning data. The reflection weight can be adjusted upward. For example, the object future behavior predicting unit 340 may perform neural network learning using the object metadata as virtual learning data based on a learning algorithm implemented as a multilayer neural network, and in this process, each node in the multilayer neural network is selected. Among the weights to be linked, the weight associated with the node that is highest associated with the object metadata can be adjusted.

The controller 350 may control the overall operation of the processor 210, and may be configured between the object recognizer 310, the object analyzer 320, the object metadata generator 330, and the object future behavior predictor 340. You can control the data flow.

4 is a flowchart illustrating a process of executing an object behavior prediction procedure by the processor of FIG. 3.

In FIG. 4, the object recognition unit 310 recognizes an object in the surveillance image (step S410). The object analyzer 320 determines an object state through a predetermined query of the object, and analyzes the corresponding object by determining the object behavior through the movement pattern of the object (step S420). The object metadata generator 330 generates object metadata based on the object state and the object behavior (step S430). The object future behavior predicting unit 340 classifies the behavior type of the object by using the learning result of the existing object metadata and predicts the future behavior of the corresponding object (step S440). In an embodiment, the object future behavior predictor 340 may perform deep learning based neural network learning by generating virtual learning data as existing object metadata based on the recognized object.

According to an embodiment of the present invention, the object behavior prediction apparatus 110 may improve the learning efficiency by predicting future behavior of the object by using the learning result of the object metadata.

Although described above with reference to a preferred embodiment of the present application, those skilled in the art various modifications of the present application without departing from the spirit and scope of the invention described in the claims below And can be changed.

100: object behavior prediction system
110: object behavior prediction device 120: video surveillance device
130: user terminal
210: processor 220: memory
230: user input and output unit 240: network input and output unit
250: object database
310: object recognition unit 320: object analysis unit
330: object metadata generation unit 340: object future behavior prediction unit
350: control unit

Claims (8)

(a) recognizing an object in the surveillance image;
(b) analyzing the object by determining an object state through a predetermined query of the object and determining an object behavior through a movement pattern of the object;
(c) generating object metadata based on the object state and the object behavior; And
(d) classifying the behavior type of the object by using the learning result of the existing object metadata and predicting future behavior of the object,
The step (b) determines whether the main appearance of the object is based on the appearance region range of the object and the appearance time interval of the object in the surveillance image, and determines the analysis period of the object based on the determined main appearance. And calculating a main appearance index based on Equation 1 below, which is related to the comparison between the appearance area range and the reference area range and the comparison between the appearance time period and the reference time period. Determining whether the main appearance of the object further includes:
In the step (c), the object state is determined as the first sentence type according to the answer to the query, and the object behavior is determined as the second sentence type according to the movement pattern, and whether or not the main appearance and the main appearance of the corresponding object are determined. Determining a third sentence type based on a time interval and generating the object metadata through the first to third sentence types, and generating an image search tag from the object metadata to determine the image search tag by analyzing the object. And associating with a start time point of an image corresponding to
In the step (d), the virtual learning data is generated as the existing object metadata based on the recognized object to perform deep learning based neural network learning, and the future behavior of the object is defined in a specific scenario in which the predefined action is specified. If the same as the step of providing the type of action and future behavior of the object to the user terminal and detecting and detecting which of the risk stages of the plurality of risk stages defined for the predetermined specific scenario If the risk level is higher than the reference level, if the object metadata is used as the virtual learning data, increasing the reflection weight of the object metadata further comprises predicting the object behavior of the computer-executable surveillance image Way.
[Equation 1]

Here, r denotes a major occurrence index, r0 denotes a reference occurrence index preset by the designer, t denotes an appearance time interval, t0 denotes a reference time interval, a denotes an appearance region range, a0 means a reference area range, and m is a natural number of 1 or more and 3 or less, and the larger the value, the greater the movement of the object.
delete delete delete delete delete delete delete

Images