KR101765722B1 - System and method of generating narrative report based on cognitive computing for recognizing, tracking, searching and predicting vehicles and person attribute objects and events - Google Patents

Abstract

The present invention relates to a cognitive-type vehicle and an impression object capable of providing an application service based on recognition information of moving information of objects, based on CCTV image information, and a narrative for recognition, tracking, 1. A reporting system, comprising: an object information collection unit for dividing a specific object among a plurality of single objects; It analyzes and analyzes the movement and changes of single objects and event events in different time and space, extracts and stores intelligent information for complex objects composed of them, distinguishes interrelationships of intelligent information, A terrain processing unit which is an object relation for distinguishing mutual relations; A narrative report output unit for outputting alarms and reports on crimes that have been tracked through analysis of a single object and a composite object, or a report including at least a narration summarizing the entire contents; And an application service unit for providing a user interface used as an application service for solving a crime using intelligent information, wherein the narrative report output unit includes: a neural tally construction unit for managing a neural tile for generating a narration; A narration expression unit for generating a type scenario and a Korean narration expression based on a Naray Tollage; And a scenario generating unit for summarizing and specifying the scenarios according to the objects of information transmission and expressing them in a designated language.

Description

FIELD OF THE INVENTION The present invention relates to a system and method for creating a narrative report for recognizing, tracking, searching, and predicting objects and events in a cognitive-type vehicle and an impression of a vehicle. AND EVENTS}

The present invention relates to a system and method for creating a narrative report for recognizing, tracking, searching and predicting a vehicle and a suspect using CCTV image information. More particularly, the present invention relates to a system and method for generating awareness, , And a system and method for generating a narrative report for prediction.

The technical field of the present invention is to recognize the events related to specific objects and events among a large number of objects existing in a moving picture and to analyze the correlation of the recognized information in various time and space, Information comes out. On the basis of such sub information, a normal moving image may be long, and it may be summarized and output in the form of a short narrative report of a story telling method or a summarized moving image. Based on this information, the upper application service layer includes a system and a method for performing services such as object and event search, tracking, prediction, and report output.

In the past, local government and police agencies actively installed CCTV and increased the number of installations every year because crime prevention closure (CCTV) is effective in suppressing crime. However, there are many opinions that the effect of crime prevention is not clear in recent years. In the academic world, there is also a constant controversy that CCTV does not play a big role in crime prevention though it is effective in crime investigation after the incident.

CCTVs installed in existing local governments and police departments use CCTV images at the control center with a separate monitor and monitoring staff. The monitoring staff sees the changes in real time, Or after the occurrence of the problem, the existing image data was provided at the request of the police. At this time, the number of image data that one monitoring agent checks at a time is too much, so it is difficult to detect the occurrence of crime in advance or to confirm the occurrence of crime in real time.

Using CCTV installed in existing local governments and police agencies, it is possible to identify the disaster situation of natural disasters such as earthquakes, floods and forest fires in real time, Has recently moved to integrate CCTV image information, which is operated by local governments and police agencies.

In order to solve the problems of the related art and to cope with the trend of the related art, the present invention extracts crime-related data by analyzing the photographed moving picture information and creates a report for predicting and tracking the crime suspect through the extracted data A system and method for creating a report for recognizing, tracking and predicting vehicles and suspects using CCTV image information that can improve crime prevention and suspect tracking performance, in particular, recognition and tracking of objects and events in cognitive vehicles and impressions And to provide a system and method for generating a narrative report for prediction.

In order to achieve the above object, a first aspect of the present invention is a system for creating a narrative report for recognizing, tracking, searching, and predicting objects and events of a cognitive-type vehicle and an impression through a video management system (VMS) The method comprising the steps of: recognizing an object related to an object in an image including an object related to impression recognition of at least a vehicle and a suspect, storing tagging for the recognized object recognition information, A system for recognizing movement information when moving in a space and providing application services based on the recognition information, the system comprising: an application for outputting search, tracking, prediction, automatic alarm and report, To handle the abbreviated requirements for the requested person, object, or event among the contents of the long video, Or generating a narrative report for video summarization. The system recognizes at least one single object necessary for providing the application service by inputting a CCTV image, and recognizes a specific object An object information collecting unit for classifying the object information; The intelligent information is extracted and stored for the composite objects constructed by analyzing the movement and the change of the single object and the event events occurring in the different time and space, and the intelligent information is distinguished from the interrelationship, A terrain processing unit which is an object relation for distinguishing mutual relations among the plurality of objects; An alarm and report for the crime tracked through the analysis of the single object and the composite object or a narrative report output unit for outputting at least a narrative which can summarize the entire contents and narration that can accompany a natural language expression of a short- Wow; And an application service unit for providing a user interface used as the application service for resolving the crime using the stored intelligent information, wherein the narrative report output unit includes: a narrate report output unit for outputting a narrate report, A building part; A narration expression unit for generating a type scenario and a Korean narration expression based on the Naritology; And a scenario generating unit for summarizing and specifying the scenarios according to the objects of information transmission and expressing them in a designated language.

Preferably, the object information collecting unit is configured to directly load the CCTV image in the road and the building using the VMS system, or indirectly load the offline video data to input streaming data and recognize general objects included in the image A video object recognition engine unit including the video object recognition unit; Among the objects for various objects and people included in the general object, the system recognizes only specific objects (a specific object includes at least one of specific objects including people, people, and impressions), separates and tags them And an object recognition unit for storing the tagging meta data and distinguishing a used object from the stored object.

Preferably, the image object recognition engine unit is an interface for receiving image streaming data required by a narrative report generating system for cognitive-based crime recognition and search, and is capable of receiving one streaming or at least several thousand streaming data at the same time A video input unit that can receive input in parallel or in a distributed processing manner; An algorithm including at least one of a hidden markov model (HMM), a gaussian mixture model (GMM), a deep neural network, and a long short-term memory (LSTM) is used to recognize objects included in a moving image. And an object recognition engine unit for recognizing and recognizing an object by touching the object.

Preferably, the usage object recognizing unit includes: an object recognizing unit that recognizes an object object including at least a road or an object appearing in the building from the general object; A human recognition unit for distinguishing a human object including at least one of up / down color, approximate key, wearing glasses / bag / hat as impression of a person from the general object; A vehicle recognition unit for distinguishing a vehicle object including at least one of a vehicle type, a maker, a model name, a color, and a license plate recognition from the general object; (Including at least one of a human object, a object object, and a vehicle object) that can be observed with respect to a crime from a general object extracted through the image object recognition engine, A symptom analysis unit for performing analysis of post-use behavior; And an event object analyzing unit for identifying a single object related to the usage action or an object for a single object unit.

Preferably, the symptom analysis unit includes: an emergency event analysis unit for distinguishing event events such as bombardment observed on the road, specific events such as bombardment, violent incidents, and theft incidents; A general event analyzing unit for analyzing a fault event including at least one of a traffic accident, a crossing elderly person, and a handicapped person and analyzing an incident event requiring a police call; And an event management unit for calculating a priority and an accuracy of the event event and managing an event event history.

Preferably, the terrain processing unit, which is the object relation, extracts complex object information and intelligent information for a complex time and space situation by connecting a plurality of single object information to single temporal / spatial information, An event tracing unit for tracking the progress of an event event including at least an escape route tracing based on the event event of the dragon object identified as < RTI ID = 0.0 > And a correlation relation part for analyzing the correlation between the event events.

delete

Preferably, the application service unit includes an object tracking unit for tracking and monitoring a suspect in a time and space based on the information, when the user designates an object for use in event events on the intelligent information suspected of being crime; An object retrieval unit for inputting feature information of a necessary object for a user to retrieve an object for the satisfaction of the input value on the intelligent information and informing the location of the temporal / spatial information of the CCTV image; An alarm and report output unit for reporting a crime alarm and a history of the forgery object on the basis of a narration scenario generation technique; And a crime occurrence prediction service unit for a specific area or time zone based on the accumulated intelligence information.

Preferably, the application service unit provides a service including at least search, trace, alarm, report output, and prediction, and may include an advanced service that may include a query of a user (including at least a police officer) And a simple keyword-based general service can be manually or automatically selected. Application services that summarize the text alerts and report services, or the entire contents, as a summary report or a summary video, must be configured And a neural report writing system for event recognition, tracking, retrieval, and prediction of objects and events of impersonal vehicles and impressions .

A second aspect of the present invention for achieving the above object is a method for generating a narrative report for recognizing, tracking, searching and predicting objects and events of a cognitive vehicle and impression using CCTV image information, Recognizing the single object from the image data and recognizing the object; The intelligent information is extracted and stored for the single object and the complex object using the tagging meta data of the single object stored in the object recognition unit for analyzing the interrelationships of the intelligent information, Displaying and storing the sex distinctively; And outputting a report including at least a narration capable of expressing a natural language of a short or long sentence based on the stored correlation, and the step of outputting the report comprises: Generating a type scenario and a Korean narration using the object information for the use of the object; Converting the generated narration into a language and a scenario suitable for the information receiver, generating a sentence or a moving image, and transmitting the result to a receiving terminal device of the information receiver.

Preferably, the step of receiving the image data from the VMS and distinguishing the single object from the image data and distinguishing the object of the dragging may include directly loading the CCTV image in the road and the building using the VMS in real time, And recognizing a general object included in the image using an algorithm including at least one of an HMM, a GMM, a neural network, and an LSTM; Separating and tagging a single object (object object, human object, vehicle object) through filtering in the general object, storing the tagging meta data, and distinguishing the object for use.

Preferably, the intelligent information is extracted and stored for the single object and the composite object using the tagging meta data of the single object stored in the object recognition unit, and the correlation of the intelligent information is analyzed, The step of separately displaying and storing the interrelationships of the crimes comprises the steps of extracting the composite object information for a complex time and space situation by connecting a plurality of single object information to the single object information or the single time and space information, Processing the event event of the forgery object, tracking the escape route of the forgery object, and storing the generated intelligent information; And analyzing the interrelationships between event events based on the information of the single object and the complex object and the event events and storing the correlation information generated at the time.

According to a third aspect of the present invention, there is provided a method of generating a narrative report for recognizing, tracking, searching, and predicting objects and events of a cognitive vehicle and impression using CCTV image information, Searching for, tracking, crime occurrence prediction and alarm of the vehicle for the user; selecting a report service and receiving a query based on Korean natural language sentence or keyword in a query language window provided by the GUI; If the requested service is search, trace, and forecast, and at the same time keyword-centered output, the stored intelligent information is loaded and designated as a tracking object for obtaining information, and the retrieval, tracking, prediction and alarm information of the tracking object is extracted and the keyword- ; And outputting the result in the form of a report if the requested service is a report output, and outputting the result in the form of a report if the requested service is a report output, Outputting a report including at least a step of outputting the report, the method comprising: generating a type scenario and a Korean narration for object information for use by a neural network managed by a neural tile construction unit; Converting the generated narration into a language and a scenario suitable for the information receiver, generating a sentence or a moving image, and transmitting the result to a receiving terminal device of the information receiver.

Preferably, if the requested service is a search, a trace, a prediction, and at the same time a keyword-centered output, the stored intelligent information is loaded and designated as a tracking object for obtaining information, and the searching, tracking, The step of outputting a result may include the steps of: specifying an object suitable for the query condition if the input query is an object tracking for the inputted query; tracking the movement path of the designated object using the event in the intelligent information; If the input query is an object search for the object, searching the object for the query condition on the intelligent information and informing the object list and the object information specified for the object; If the input query is a crime occurrence prediction, informing the result of the crime occurrence prediction for a specific region or time zone based on the intelligent information; And if the input query is an alarm, reporting a crime alarm and a history of the forgery object based on a narration scenario generation technique when an event event corresponding to a query condition is generated.

delete

According to the above configuration, the present invention can obtain the following effects.

First, the monitoring staff can reduce the burden of surveillance, detect the occurrence of crime in advance or check the occurrence of crime in real time, so that the agents can provide effective and quick solutions to prevent crime and minimize adverse effects of crime .

Second, in order to minimize disaster damage by promptly understanding the disaster situations of natural disasters such as earthquakes, floods, and forest fires in real time, It is possible to provide an effective solution to the movement to integrate information integration.

More specifically, with respect to the configuration, the following three software layers can be used to create a narrative report provided by the present invention.

In the lower layer, CCTV video is received and the deep learning method using deep neural net method and the open image data including a large number of photographs posted on the portal site are used to recognize the objects in the video and to detect the event, There is an intelligent information collecting layer which recognizes objects and events of an object which is created by the model and receives the images in real time. This learning process was implemented with deep learning consisting of more than ten layers of neural networks.

In the middle part, based on this sub information, it traces the development and movement of the objects in the time and space and the generated events, recognizes the interrelationships between the completion of each event and the events, and the necessary neuralized ontology And narration scenarios based on the narratorology, and to create a report using Korean narration expression technology. In this layer, taking into account who is using the report and which information medium is used to convey it, considering the same information as well as a short sentence or a long sentence form, as well as a spoken expression or an octopus expression, To apply the technology to generate. In particular, the narrative report can be very useful when it is necessary to summarize, summarize, or summarize the contents of a long video using storytelling techniques, focusing on specific events, characters, and objects. For example, suppose the police need to arrest a suspect through a CCTV video. In a system that supports such narrative reporting, a police investigator does not have to watch the videos sequentially for a long time and visually identify the suspects, but rather, they give specific suspects, time and space information, , The system of the present invention generates a report on the basis of the input information, thereby making it possible to use the police investigation personnel efficiently. In addition, the video summary system can summarize long videos and shorten the core content to create re-edited video summaries based on core content.

It is composed of layers that provide application services such as cognitive type crime information extraction, recognition, search, and prediction as the upper application layer using the information in the middle part. The present invention features not only elemental description of each layer but also an integrated system and method for the relationship between these layers of technology that need to be connected vertically.

In consideration of the fact that most of the vector operations are performed in processing hardware required for performing the deep learning using the CPU of the Intel series or deep neural networks, the hardware layer includes GPGPU (General Purpose Graphic Processing Unit) such as NVIDIA CUDA (Compute Unified Device Architecture, a kind of library interface that provides an application programming interface (API) as a parallel computing platform to facilitate programming using the GPU). For example, how many computing resources will be required to simultaneously process object recognition and tagging for data streaming from 1,000 CCTVs in parallel? 1 Gflops x 24 layers x 30 fps x 1,000 CCTVs = 720 Tflops 72,000 CPU cores Need supercomputer-level computing. Since it corresponds to 16,000 CPU cores = 3 GPUs, 72,000 cores can be processed with 14 GPUs.

In addition, storage storage devices including hardware layers and software layers are also relevant in the present invention. In particular, the objects recognized in the image and the tagging information describing the objects are stored in the form of metadata in a database or a file system. This metadata is used to search the original image more efficiently when performing a search such as a keyword in the upper application service layer. According to the related art, the storage metadata is stored in the management server of the storage of file metadata such as a UNIX inode of the block information of the lower file system, which is separated from the data server storing the data, .

These and other objects and features of the present invention will become apparent from the detailed description of the preferred embodiments with reference to the accompanying drawings, in which:

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating a narrative report generating system for recognizing, tracking, and predicting a vehicle and a suspect using a CCTV image information according to an embodiment of the present invention; FIG.
BACKGROUND OF THE INVENTION 1. Field of the Invention [0001]
FIG. 3 is an internal configuration view of a video object recognition engine according to an exemplary embodiment of the present invention; FIG.
4 is an internal configuration diagram of a dragon object recognition unit according to an embodiment of the present invention;
5 is an internal configuration diagram of a symptom analyzing unit according to an embodiment of the present invention;
FIG. 6 is an internal configuration diagram of an object-relational-aware type processing unit according to an embodiment of the present invention; FIG.
FIG. 7 is an internal configuration view of a report output unit according to an exemplary embodiment of the present invention; FIG.
8 is an internal configuration view of an application service unit according to an embodiment of the present invention;
9 is a flowchart illustrating a method for generating intelligent information on a single object and a composite object in a report generating method for recognizing, tracking, and predicting a vehicle and a suspect using CCTV image information according to an exemplary embodiment of the present invention FIG.
FIG. 10 is a flowchart for distinguishing objects by inputting image data according to an embodiment of the present invention; FIG.
11 is a flowchart for tracking or predicting a movement path of a dragon object from an object distinguished from image data according to an embodiment of the present invention;
12 is a flowchart for generating and delivering a scenario based on intelligent information according to an embodiment of the present invention;
FIG. 13 is a flow chart illustrating application of intelligent information according to an embodiment of the present invention. FIG.
14 is a flowchart illustrating a method of outputting a final report by applying intelligent information according to an embodiment of the present invention.
15 is a block diagram illustrating an embodiment of a report generating system for recognizing, tracking, and predicting a vehicle and a suspect using the CCTV image information according to an embodiment of the present invention.

First, the construction and operation of a report generating system for recognizing, tracking and predicting a vehicle and a suspect vehicle using CCTV image information will be described in detail with reference to FIG.

 The present invention is characterized in that the CCTV managed by the existing VMS 1000 or the image data stored in the data storage unit 2000 is received through the VMS 1000 and the input image data is transmitted to the object information collection unit 3100 through the network 4000. [ Lt; / RTI >

 As shown in FIG. 2, the image object recognition engine 3110 distinguishes the background and single objects from the input image data through color and shape changes and movement changes. The object recognizing unit 3120 recognizes specific objects (specific objects, people and impressions, vehicles) among the objects of various objects and people included in the separated single object, separates them, Stores the tagging metadata in the data storage unit 2000, and distinguishes the objects.

 As shown in FIG. 3, the video input unit 3111 receives video streaming data of one or more video streaming data through the VMS 1000, and processes a video stream input through a distributed processing method . The objects recognized by the image data inputted to the image input unit 3111 may be recognized by an object recognition engine unit 3112 using an HMM (Markov Model), a Gaussian Mixture Model (GMM), a deep neural network, a long short-term memory (LSTM) (Edge) to distinguish objects on the GUI (Graphic User Interface). At this time, the object identified through the object recognition engine 3112 is referred to as a general object.

As shown in FIG. 4, the general object distinguished through the object recognition engine 3112 is input,

The object recognition unit 3121 recognizes object objects using information such as the shape, color, and position of objects appearing on the road or building. In this case, object objects represent objects such as glasses, bags, hats, clothes, and the like. The person recognizing unit 3122 recognizes the person as an impression of a person by inputting the general object, and determines whether or not the color, the key, and the object object (eyeglasses / bag / shepherd / Recognize human objects. Among the object objects, the vehicle is separately managed through the vehicle recognizing unit 3123 by inputting the general object. The car object is distinguished by distinguishing the model, the manufacturer, the model name, the color, and the license plate number. The symptom analysis unit 3124 receiving the information of the objects (object object, human object, vehicle object) distinguished distinguishes the movement of the objects and the state change in order to analyze the behavior of the crime dictionary / posture. The event object analyzing unit 3125 analyzes the monitored actions and distinguishes the objects from the objects.

5, the emergency event analysis unit 3124a distinguishes event events such as bomb terrorism, violent events, and theft events through monitoring and transmits them to the event management unit 3124c. The general event analysis unit 3124b, And event events such as a traffic accident, a situation where a person is stopped at the crossing, and the like to the event management unit 3124c. The event management unit 3124c receiving the event event calculates the priority and the accuracy of the event events, manages the object information and the event event history related to the event event, and stores the event information and the event event history in the data storage unit 2000 when the long-term storage is needed. The emergency event analyzer 3124a recognizes a specific situation such as a bomb terror through recognizing a human object moving away from a bag or the like, recognizes a situation where a human object climbs a wall or a wall, generates an event event do. In addition, the general event analyzer 3124b recognizes a collision between vehicle objects, or recognizes a situation in which a human object is stopped in the middle of a crossing while a vehicle object is moving at a crossing, and generates an event event.

The object recognition unit 3120 generates an event object using the event information generated by the event management unit 3124c and the object information that generated the event event, . The object relationship cognitive type processing unit 3200 stores intelligent information generated based on the single object, the composite object, the event event, and the object information. The complex object classifies several single objects that generate event events in a single temporal / spatial information, analyzes complex multiple temporal / spatial information, deduces the association between single objects that generate event events, It is a set of single objects that can designate objects as a single object.

6, the event tracing unit 3210 uses a single object or a composite object to point to a dragon object and transmits an event event progress of the dragged object from the event management unit 3124c It tracks the object of acceptance and plays the role of predicting escape route. The interrelation recognition unit 3220 analyzes the correlation between the event events generated by the forgery object and the event events generated by the single object or the composite object, thereby enhancing the accuracy of escape route tracking or tracking route prediction. At this time, the current movement status and the expected path of the dragged object to be tracked are displayed on the displayed monitor screen, and the dragged object position can be displayed on the map in conjunction with the map. The object relation awareness type processing unit 3200 stores the calculated data in the data storage unit 2000 when it is necessary to store the data for a long period of time.

7, the narrative report output unit 3300 receives intelligence information, object information for use, and correlation information between the objects for tracking, which are tracked from the object relationship awareness type processing unit 3200, The narration expression unit 3320 generates a type scenario based on the Naray-Tolerance managed by the logic building unit 3310 and delivers the narration of the language suited to the receiver to the scenario generation unit 3330. [ The language of the narration can be changed according to the person who receives the information. The scenario generating unit 3330 selects an information receiving subject recorded in the list of subjects to which information is to be transmitted and summarizes the information into a scenario that the receiving subject can understand and delivers the information to the subject. At this time, a reception terminal such as a mobile phone character, a facsimile, an instant messenger text, a printer, or the like can be used as the delivery method.

 8, the application service unit 3400 includes an object information collecting unit 3100 and objects (a human object, a object object, a vehicle object, a dragon object) generated through the object relationship awareness processing unit 3200, And event events through the use of the generated intelligent information. Object tracking unit 3410 selects an object suspected of being a crime among the objects in the intelligent information, designates it as a dragon object, and tracks the object for the designated dragon. The object searching unit 3420 searches the intelligent information for the object having the selected property value by selecting the object characteristics to be searched on the intelligent information and extracts the space time information of the object list specified by the used object, Number, priority, and other information. The alarm and report output unit 3430 reports the crime alarm and history of the dragon object based on the narration scenario generation technique. The crime occurrence prediction service unit 3440 plays a role of predicting the occurrence of a crime in a specific region or time zone on the map based on the intelligent information accumulated in the data storage unit 2000 and informing the user.

The intelligent information includes information of objects (human object, object object, vehicle object) generated through real-time or non-real-time image data analysis from roads and building CCTV images or event events Or a combination of object information extracted from the object information and analysis of the event events. In this case, the object of dragon means various objects that can be observed related to the crime (eg, vehicle, person, criminal behavior, etc.). Furthermore, the intelligent information can be upgraded in connection with the existing criminal database held by the police.

Hereinafter, a method for generating intelligent information on a single object and a composite object in a report generating method for recognizing, tracking and predicting a vehicle and a suspect using the CCTV image information according to the present invention will be described with reference to FIGS. 9 to 11 Will be described in detail.

9, the object information collecting unit 3100 receives image data from the VMS 1000, distinguishes and recognizes a single object from the image data, and distinguishes the objects for use (S100).

  Referring to FIG. 10, a method of distinguishing a dragon object from image data will be described. The image object recognition engine 3110 directly loads a CCTV image in the road and a building using the VMS 1000 in real time Indirectly loads video data stored offline and recognizes general objects included in the image using algorithms such as HMM, GMM, a neural network, and LSTM (S110)

The object recognition unit 3120 separates a single object (object object, human object, and vehicle object) through filtering in the general object, stores the tagging metadata, stores the tagging metadata, and distinguishes the object. S120)

As described above, the terrain processing unit 3200, which is an object relation, extracts and stores intelligent information for the single object and the complex object using the tagging metadata of a single object stored in the object recognition unit 3120, The interrelationships of the information are analyzed and the interrelationships between the complex crimes are distinguished, displayed and stored (S200).

11, a method of storing intelligent information and analyzing interrelationships will be described. The event tracking unit 3210 may combine a plurality of single object information for a single object information or a single time / space information, Space object, extracts complex object information from the object, processes event events of the object, tracks the escape route of the object, and stores the generated intelligent information (S210).

The interrelation recognition unit 3220 receives the information of the single object and the complex object and event events, analyzes the interrelation between event events in a complex time space, and stores the interrelation information generated at this time. )

Hereinafter, with reference to FIG. 12 to FIG. 14, a method of generating a report for recognition, tracking and prediction of a vehicle and a suspect using CCTV image information according to the present invention, And how to provide application services.

12, a user selects a suspect or a dragon's vehicle search, trace, crime occurrence prediction, alarm, and report service through the GUI of the application service section 3400 and displays a Korean natural language sentence Or a keyword-based query is input (S300)

 If the requested service is retrieval, tracking, and forecasting, and at the same time the keyword-centric output, the stored intelligent information is loaded as a tracking object for obtaining information, and the retrieval, tracking, prediction and alarm information of the tracking object is extracted and the keyword- (S400)

Referring to FIG. 12, a method for retrieving, tracking, predicting, and alarming information of a dragon object and outputting a result will be described. If the input query is an object trace for the input query, And the object tracking unit 3410 uses the event event on the intelligent information to track the movement path of the designated object. (S410)

 If the input query is an object search for the object, the object search unit 3420 searches the intelligent information for the query object with the input value, and notifies the object list and the object information of the object designated by the object . (S420)

If the input query is a crime occurrence prediction, the crime occurrence prediction service unit 3440 informs the result of the crime occurrence prediction for the specific region or time zone that is queried based on the intelligent information (S430)

If the input inquiry is an alarm, the alarm and report output unit informs the crime alarm and history of the forgery object based on the narration scenario generation technique when an event event corresponding to the query condition occurs (S440)

If the requested service is a report output as described above, the report output unit S3300 is called to output the result in a report form, and the result is output (S500)

Referring to FIG. 14, a narration expression unit 3320 outputs a report using a narrative structure managed by the narlitology construction unit 3310, The narration is generated (S510)

 The scenario generating unit 3330 converts the generated narration into a language and a scenario suitable for the information receiving subject, and then transmits the converted narration to the receiving terminal device of the information receiving subject (S520)

15 is a block diagram illustrating an embodiment of a report generating system for recognizing, tracking, and predicting a vehicle and a suspect using the CCTV image information according to an embodiment of the present invention. However, FIG. 15 is an embodiment implementing the system of the present invention, and various various implementations are possible.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited thereto, and various changes and modifications may be made by those skilled in the art without departing from the scope of the following claims.

1000: VMS 2000: Data storage unit
3000: Inventive system 3100: Object information collection unit
3110: image object recognition engine unit 3111: image input unit
3112: object recognition engine unit 3120: object recognition unit
3121: object recognition unit 3122: human recognition unit
3123: vehicle recognition unit 3124: symptom analysis unit
3124a: emergency event analysis unit 3124b: general event analysis unit
3124c: an event management unit 3125:
3200: Object relation awareness type processing unit 3210: Event tracking unit
3220: Correlation relation recognition part 3300: Narrative report output part
3310: Naray Tallocation building part 3320: Narration presentation part
3330: scenario generation unit 3400: application service unit
3410: object search unit 3420: object search unit
3430: alarm, report output unit 3440: crime prediction service unit
4000: central processing unit (CPU) 5000: graphics processing section
6000: Network

Claims (15)

The CCTV image information is received in real time via a VMS (Video Management System) or received as a stored image, at least an object related to an object in an image including an object related to a vehicle and a suspect, A system for storing tagging information on the recognition information, further recognizing movement information when the objects move in the time space, and providing an application service based on the recognition information, Generate narrative reports or narrative reports that provide video summaries by processing tracking requirements, predicting, automatic alarms and reports, or automatically handling the abbreviated requirements for the person, The system comprising:
An object information collecting unit for inputting a CCTV image to separately recognize a minimum number of single objects necessary for providing the application service and distinguishing objects for specific use among the single objects;
The intelligent information is extracted and stored for the composite objects constructed by analyzing the movement and the change of the single object and the event events occurring in the different time and space, and the intelligent information is distinguished from the interrelationship, A terrain processing unit which is an object relation for distinguishing mutual relations among the plurality of objects;
An alarm and report for the crime tracked through the analysis of the single object and the composite object or a narrative report output unit for outputting at least a narrative which can summarize the entire contents and narration that can accompany a natural language expression of a short- Wow;
An application service unit for providing a user interface used as the application service for resolving a crime using the stored intelligent information;
Lt; / RTI >
The narrative report output unit includes:
A neural network construction unit for managing a neural network for generating a narration;
A narration expression unit for generating a type scenario and a Korean narration expression based on the Naritology;
A scenario generating unit for summarizing and specifying the scenarios according to the information delivery object and expressing them in a designated language;
And a navigation report generating system for recognizing, tracking, searching, and predicting events and objects of the cognitive type vehicle and the impression landing. The method according to claim 1,
The object information collecting unit,
A video object recognition engine unit for inputting streaming data by indirectly loading video data stored in offline and real-time loading of CCTV images in roads and buildings using a VMS system, and recognizing general objects included in the video; ;
Among the objects for various objects and people included in the general object, the system recognizes only specific objects (a specific object includes at least one of specific objects including people, people, and impressions), separates and tags them An object recognition unit for storing the tagging meta data and distinguishing a used object from the stored object;
And a navigation report generating system for recognizing, tracking, searching, and predicting events and objects of the cognitive type vehicle and the impression landing . 3. The method of claim 2,
Wherein the image object recognition engine unit comprises:
It is an interface for receiving input of video streaming data required by cognitive crime awareness tracking and narrative reporting system for retrieval. It is an interface that can input one streaming or several thousand streaming data at the same time by parallel or distributed processing An input unit;
An algorithm including at least one of a hidden markov model (HMM), a gaussian mixture model (GMM), a deep neural network, and a long short-term memory (LSTM) is used to recognize objects included in a moving image. An object recognizing engine unit for recognizing and recognizing an object by touching an edge;
And a navigation report generating system for recognizing, tracking, searching, and predicting events and objects of the cognitive type vehicle and the impression landing . 3. The method of claim 2,
The object recognizing unit recognizes,
An object recognition unit for recognizing an object object including at least a road or an object appearing in the building from the general object;
A human recognition unit for distinguishing a human object including at least one of up / down color, approximate key, wearing glasses / bag / hat as impression of a person from the general object;
A vehicle recognition unit for distinguishing a vehicle object including at least one of a vehicle type, a maker, a model name, a color, and a license plate recognition from the general object;
(Including at least one of a human object, a object object, and a vehicle object) that can be observed with respect to a crime from a general object extracted through the image object recognition engine, A symptom analysis unit for performing analysis of post-use behavior;
An event object analyzing unit for identifying a single object or an object for use in a unit of a single object related to the action;
And a navigation report generating system for recognizing, tracking, searching, and predicting events and objects of the cognitive type vehicle and the impression landing . 5. The method of claim 4,
The symptom analyzing unit,
An emergency event analysis unit for distinguishing event events such as a specific situation such as a bombing terrorism observed on the road, violent incidents, and theft incidents;
A general event analyzing unit for analyzing a fault event including at least one of a traffic accident, a crossing elderly person, and a handicapped person and analyzing an incident event requiring a police call;
An event management unit for calculating a priority and an accuracy of the event event and managing an event event history;
And a navigation report generating system for recognizing, tracking, searching, and predicting events and objects of the cognitive type vehicle and the impression landing . The method according to claim 1,
The terrain processing unit, which is the object relation,
Multiple object information is connected to a single time / space information to extract complex object information and intelligent information for complex time and space situations, and event events of a dragon object identified as a vehicle of a suspect and a dragon are processed An event tracing unit for tracking the progress of the incident event including at least the escape route tracing based on the trajectory tracing;
A correlation relation part for analyzing a correlation between the event events;
And a navigation report generating system for recognizing, tracking, searching, and predicting events and objects of the cognitive type vehicle and the impression landing . delete The method according to claim 1,
The application service unit,
An object tracking unit for tracking and monitoring a suspect in a time and space based on the information when a user designates an object for use in an event event of intelligent information suspected of crime;
An object retrieval unit for inputting feature information of a necessary object for a user to retrieve an object for the satisfaction of the input value on the intelligent information and informing the location of the temporal / spatial information of the CCTV image;
An alarm and report output unit for reporting a crime alarm and a history of the forgery object on the basis of a narration scenario generation technique;
A crime occurrence prediction service unit for a specific region or time zone based on the accumulated intelligence information;
And a navigation report generating system for recognizing, tracking, searching, and predicting events and objects of the cognitive type vehicle and the impression landing . The method according to claim 1,
The application service unit,
Provides services that include at least search, trace, alarm, report output, and forecasting,
You can manually or automatically select advanced services that can include queries or search queries for users in Korean (including at least police) and relatively simple, keyword-centric, general services in a natural language format, summarize text alerts and report services Wherein the application service outputting the summary report or the summary video comprises a preparation and output of a narrative report, and a narrative report generating system for recognizing, tracking, searching, and predicting the events and events of the cognitive vehicle and the impression . A method of generating a narrative report for recognizing, tracking, searching, and predicting objects and events of a cognitive vehicle and impression using CCTV image information,
Directly or indirectly receiving image data from a CCTV or a VMS, distinguishing and recognizing a single object from image data, and distinguishing objects of the image;
The intelligent information is extracted and stored for the single object and the complex object using the tagging meta data of the single object stored in the object recognition unit for analyzing the interrelationships of the intelligent information, Displaying and storing the sex distinctively;
Outputting a report including at least a narration that can accompany a natural language expression of a short or long sentence based on the stored correlation;
Lt; / RTI >
The step of outputting the report includes:
Generating a type scenario and a Korean narration for use object information using a neural tile managed by a neural tile construction unit;
Converting the generated narration into a language and a scenario suitable for the information receiver, generating a sentence or a moving picture, transmitting the result, and transmitting the result to a receiving terminal device of the information receiver;
And generating a narrative report for recognizing, tracking, searching, and predicting an object and an event of the cognitive vehicle and the impression. 11. The method of claim 10,
The step of recognizing the single object from the image data by receiving the image data from the VMS and distinguishing the object of the image
It is possible to directly load CCTV images in roads and buildings using VMS in real time, or indirectly load offline video data, and use an algorithm including at least one of HMM, GMM, ;
Separating and tagging a single object (object object, human object, vehicle object) through filtering in the general object, storing the tagging meta data, and distinguishing the object for use;
And generating a narrative report for recognizing, tracking, searching, and predicting an object and an event of the cognitive vehicle and the impression. 11. The method of claim 10,
The intelligent information is extracted and stored for the single object and the complex object using the tagging meta data of the single object stored in the object recognition unit for the purpose of analyzing the mutual relation of the intelligent information, The step of separately displaying and storing the relationships comprises:
Space information, extracts composite object information for a composite time / space situation by connecting a plurality of single object information to the single object information or single time / space information, processes the event event of the forgery object using the extracted composite object information, Tracking the escape route of the object and storing the generated intelligent information;
Analyzing the interrelationships between event events based on the information of the single object and the composite object and event events, and analyzing the events in the time space and storing the correlation information;
And generating a narrative report for recognizing, tracking, searching, and predicting an object and an event of the cognitive vehicle and the impression. A method of generating a narrative report for recognizing, tracking, searching, and predicting objects and events of a cognitive vehicle and impression using CCTV image information,
Searching, tracking, crime occurrence prediction and alarming of a suspect or a dragon vehicle through a GUI of an application service part; receiving a query based on a Korean natural language sentence or a keyword in a query language window provided by the GUI;
If the requested service is search, trace, and forecast, and at the same time keyword-centered output, the stored intelligent information is loaded and designated as a tracking object for obtaining information, and the search, tracking, prediction, and alarm information of the tracking object are extracted, ;
Outputting a result in the form of a report if the requested service is a report output;
Lt; / RTI >
Wherein the outputting of the result in the form of a report is a step of outputting a report including at least a narration that can accompany a natural language expression of a short or long sentence if the requested service is a report output,
Generating a type scenario and a Korean narration for use object information using a neural tile managed by a neural tile construction unit;
Converting the generated narration into a language and a scenario suitable for the information receiver, generating a sentence or a moving picture, transmitting the result, and transmitting the result to a receiving terminal device of the information receiver;
And generating a narrative report for recognizing, tracking, searching, and predicting an object and an event of the cognitive vehicle and the impression. 14. The method of claim 13,
If the requested service is search, trace, and forecast, and at the same time keyword-centered output, the stored intelligent information is loaded and designated as a tracking object for obtaining information, and the retrieval, tracking, prediction and alarm information of the tracking object is extracted and the keyword- In the step,
If an input query is an object tracking for a query, specifying an object suitable for the query condition and using the event event on the intelligent information to track the movement path of the designated object;
If the input query is an object search for the object, searching the object for the query condition on the intelligent information and informing the object list and the object information specified for the object;
If the input query is a crime occurrence prediction, informing the result of the crime occurrence prediction for a specific region or time zone based on the intelligent information;
If the input query is an alarm, notifying a crime alarm and a history of the forgery object based on a narration scenario generation technique when an event event corresponding to a query condition is generated;
And generating a narrative report for recognizing, tracking, searching, and predicting an object and an event of the cognitive vehicle and the impression. delete

Images