KR102052184B1 - Apparatus and method for storing and managing primitive visual knowledge information - Google Patents

Abstract

The present invention relates to a system and method for storing and managing visual knowledge information for storing visual knowledge information generated from various input images according to a predefined schema or data structure, and utilizing part or all visual knowledge information in an analysis or application service. In this regard, the storage and management system generates a visual knowledge that generates visual knowledge, which is standardized metadata, by taking an image section containing at least one or more important information from an input image and then recognizing and understanding the section. part; A visual knowledge storage unit for storing the generated visual knowledge information in various types of storage devices so as to correspond to a predefined standardized schema or data structure in association with the visual knowledge generator; A schema provider which stores the formalized schema or data structure and provides the visual knowledge utilization unit; And a visual knowledge utilization unit that provides a part or all of the visual knowledge collected in association with the visual knowledge storage unit to an application service using visual knowledge analysis or visual knowledge.

Description

Visual knowledge information storage and management system and its method {Apparatus and method for storing and managing primitive visual knowledge information}

The present invention relates to a system and method for storing and managing visual knowledge information, and more particularly, to store visual knowledge information generated from various input images according to a predefined schema or data structure, and to analyze or analyze some or all visual knowledge information. A visual knowledge information storage and management system for use in an application service, and a method thereof.

In recent years, as the production of video data is rapidly increasing worldwide, the contents of video are understood and used for monitoring or detecting abnormal conditions in real time, or searching for a specific event or situation from the result of understanding the video. For example, in the case of broadcast content, a method in which an expert writes a tag or metadata directly for each situation in a video and tries to find a desired scene using keyword search is being considered.

Also, recently, as interest in visual intelligence technology soars, technologies for automatically recognizing objects in images and understanding situations and events have been developed. For example, in the field of intelligent CCTV, roaming detection technology that detects objects that stay long in a specific place in indoor and outdoor public places, organic detection technology that detects situations where objects are left for a certain time, and multi-object tracking that tracks objects in CCTV Many advances have been made in automated visual intelligence technologies, such as technology, abnormal event detection technology that detects sudden changes in the behavior of crowds, and efforts are being made to apply them to real images.

In this way, by recognizing and understanding image data, only important information is extracted from the image to generate “visual knowledge”, which is standardized metadata, and various applications such as development of application services such as analysis of visual knowledge data or specific event search. Consideration can be given to how it is used. However, there are some inefficiencies to use visual knowledge information for analysis or application service. For example, when searching for a video section where a specific event occurs such as “ride on” or “ride on a bicycle,” it is necessary to use only information such as event information, related object information, and video section among visual knowledge information. May be easy.

As another example, when performing path data analysis of a moving object in an image such as path clustering, it may be easy for an analysis system to utilize only information such as position information of an object for each frame and appearance frames of an object among visual knowledge information. .
[Prior literature]
Published Patent Publication No. 10-2016-0055591 (20160518)

Accordingly, an object of the present invention is to extract and store only important information from an input image, to format and store visual knowledge generated by recognizing and understanding content, and to provide a system and a method for equipping visual knowledge with analysis or various application services. It is to provide a visual knowledge information storage and management system to provide a description of the components.

According to an aspect of the present invention, a system for storing and managing visual knowledge information according to an aspect of the present invention includes taking an image section including at least one important information from an input image, and then recognizing and understanding the section. A visual knowledge generating unit for generating visual knowledge which is standardized metadata; A visual knowledge storage unit for storing the generated visual knowledge information in various types of storage devices so as to correspond to a predefined standardized schema or data structure in association with the visual knowledge generator; A schema provider which stores the formalized schema or data structure and provides the visual knowledge storage unit; And a visual knowledge utilization unit that provides a part or all of the visual knowledge collected in association with the visual knowledge storage unit to an application service using visual knowledge analysis or visual knowledge.
The visual knowledge storage unit may define a database schema using a data definition language provided by a database management system and store visual knowledge information using a data manipulation language. In addition, it describes the structure for storing using XML schema and stores visual knowledge information in the form of XML, which is suitable for XML schema structure, in the form of file.
The visual knowledge generation unit may include an image information input unit configured to receive an image that is a target of visual knowledge generation; A scene generation unit generating a scene, which is an image section including a specific event occurring in an image input through the image information input unit; An object information generator for generating information about objects appearing in at least one or more scenes generated by the scene generator; An inter-object relationship generation unit analyzing the relationship between objects using the object information generated by the object information generation unit and generating the information; An event information generation unit for generating event information by detecting an event occurring in a corresponding scene using the object information generated by the object information generation unit and the relationship generation information between objects generated by the relationship generation unit; And a knowledge generator for generating visual knowledge information according to the scene information generated by the scene generator, the object information generated by the object information generator, and the event information generated by the event information generator.
The visual knowledge storage unit may include: a knowledge input unit configured to analyze the visual knowledge structure by using the visual knowledge information generated by the visual knowledge generator and to provide the analyzed visual knowledge; A scene information storage unit for storing scene information generated by the scene generation unit of the knowledge information generation unit; An object information storage unit for storing object information generated by the object information generation unit of the knowledge information generation unit; An event information storage unit for storing event information generated by the event information generation unit of the knowledge information generation unit; And an index structure generation unit for generating various types of index structures for quick querying and efficient management after storing the scene information, object information, and event information.
The visual knowledge utilization unit may include: a schema constructing unit configured to analyze a method of storing or constructing data necessary for utilizing visual knowledge, and to configure a schema according to an application method; A data receiver configured to receive, from the visual knowledge storage, visual knowledge information necessary based on a schema configured in the schema constructing unit; A data storage unit for storing visual knowledge information received through the data receiving unit based on a schema configured in the schema constructing unit; A data control unit which receives data necessary to provide an analysis or application service using visual knowledge from the data storage unit and performs operations such as data processing, query generation, and data visualization; A user input unit for receiving a separate requirement from a user to provide an analysis or application service using visual knowledge in association with a user; And a result output unit for outputting a result of the analysis or application service of the data controller to the user using a separate interface according to a user search request through the user input unit.
On the other hand, the method for storing and managing visual knowledge information according to another aspect of the present invention is to take a video section containing at least one or more important information from the input image, and then to recognize and understand the corresponding section, Generating visual knowledge; Storing the generated visual knowledge information in various types of storage devices so as to correspond to a predefined standardized schema or data structure; Storing the formalized schema or data structure and providing a stored formalized schema or data structure; And providing part or all of the previously collected visual knowledge to an application service using visual knowledge analysis or visual knowledge.
The storing may include defining a database schema using a data definition language provided by a database management system and storing visual knowledge information using a data manipulation language. In addition, it describes the structure for storing using XML schema and stores visual knowledge information in the form of XML, which is suitable for XML schema structure, in the form of file.
The generating of the visual knowledge may include receiving an image that is a target of visual knowledge generation; Generating a scene, which is an image section including a specific event occurring in the input image; Generating information about objects appearing in each of the at least one generated scene; Analyzing relationship between objects using the generated object information to generate relationship information between objects; Generating event information by detecting an event occurring in a corresponding scene by using the generated object information and relationship information between objects; And generating visual knowledge information according to the generated scene information, object information, and event information.
The storing may include analyzing the visual knowledge structure using the generated visual knowledge information and then providing the analyzed visual knowledge; Storing the generated scene information, object information, and event information; After storing the stored scene information, object information and event information, generating an index structure of various types for quick query and efficient management.
The providing to the application service may include analyzing a method of storing or configuring data necessary for utilizing visual knowledge, and constructing a schema according to the application method; Receiving necessary visual knowledge information from a storage device based on the configured schema; Storing the received visual knowledge information based on the configured schema; Receiving data necessary to provide an analysis or application service using visual knowledge and performing operations such as data processing, query generation, and data visualization; Receiving a separate requirement from a user to provide an analysis or application service using visual knowledge in association with a user; And outputting a result of the analysis or application service to a user using a separate interface according to the user search request.

According to the present invention, a standardized visual knowledge of an image section containing important information from an input image can be generated and stored and managed according to a standardized schema or data structure. In addition, the present invention can easily utilize some or all of the information in the visual knowledge analysis or application service development, including a method and components for providing some or all of the previously stored visual knowledge.

1 is a block diagram illustrating a system for storing and managing visual knowledge information according to an embodiment of the present invention.
2 is a view for explaining the operation principle of the visual knowledge storage information and management system according to an embodiment of the present invention.
3 is a block diagram illustrating a detailed block configuration of the visual knowledge generating unit illustrated in FIG. 1.
4 is a block diagram illustrating a detailed block structure of the visual knowledge storage unit illustrated in FIG. 1;
FIG. 5 is a diagram showing the data structure of the visual knowledge storage unit shown in FIG. 1; FIG.
FIG. 6 is a diagram illustrating a detailed block configuration of the visual knowledge utilization unit illustrated in FIG. 1. FIG.
FIG. 7 is a diagram illustrating an example of a schema structure of the visual knowledge utilization unit illustrated in FIG. 6.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It will be described in detail focusing on the parts necessary to understand the operation and action according to the present invention. In describing the embodiments of the present invention, descriptions of technical contents that are well known in the technical field to which the present invention belongs and are not directly related to the present invention will be omitted. This is to more clearly communicate without obscure the subject matter of the present invention by omitting unnecessary description.

In addition, in describing the components of the present invention, different reference numerals may be given to components having the same name according to the drawings, and the same reference numerals may be given to different drawings. However, even in such a case, it does not mean that the corresponding components have different functions according to embodiments, or does not mean that they have the same functions in different embodiments. Judgment should be made based on the description of each component in.

1 is a block diagram illustrating a system for storing and managing visual knowledge information according to an embodiment of the present invention.

As shown in FIG. 1, the visual knowledge information storage and management system according to the present invention includes a visual knowledge generation unit 10, a visual knowledge storage unit 20, a schema providing unit 30, and a visual knowledge utilization unit 40. ) May be included.

The visual knowledge generator 10 may be configured with at least one, and after taking an image section containing at least one or more important information from the input image, and visualize the metadata as a process of recognizing and understanding the section The knowledge may be generated and provided to the visual knowledge storage unit 20. Here, the input image may include a stream image which is continuously stored, such as a pre-stored image file, CCTV, and the like.

Meanwhile, the image section containing the important information may determine a part of the entire image according to an application field, a subjective standard of a person, or an automated algorithm. For example, an image section containing important information may be set to the moment when a screen change occurs and the next screen change occurs, or until a moving object appears and disappears, or the magnitude of a numerical value such as an optical flow increases rapidly. It may include a method for determining the interval.

Visual knowledge is a representation of information contained in an image section including important information in a standardized metadata format. However, the visual knowledge here may include not only simple object information and event information but also detailed properties thereof. For example, in case of object information, representative color, color by frame, estimated size, motion or average speed, feature point vector, body part when object is human, causal relationship between event information, inclusion relation, upper and lower Relationships, space-time feature point vectors, and the like.

The visual knowledge storage unit 20 may store visual knowledge information generated in association with at least one visual knowledge generator 10 in various types of storage devices.

The visual knowledge storage unit 20 may define a formal schema or data structure in advance, and store visual knowledge information accordingly. For example, the visual knowledge storage unit 20 defines a database schema using a data definition language provided by a database management system and uses a data manipulation language. Visual knowledge information can be stored. As another example, the visual knowledge storage unit 20 may describe a structure for storing using an XML schema, and may store the visual knowledge information in a XML form corresponding to the XML schema structure and in a file format.

The visual knowledge storage unit 20 may include all of the visual knowledge information provided by the at least one visual knowledge generator 10.

The visual knowledge utilization unit 40 may use part or all of the visual knowledge collected in association with the visual knowledge storage unit 20 to provide an application service using visual knowledge analysis or visual knowledge. For example, by using the location information of each frame of the objects described in the visual knowledge can be used to analyze the path data of the moving object in the image, such as path clustering, semantic region analysis in the image.

As another example, the visual knowledge utilization unit 40 may provide an application service for searching for an image section in which a specific event occurs using information such as event information, related object information, and image section among the visual knowledge information.

The visual knowledge utilization unit 40 may have a separate schema or data structure capable of storing only part of visual knowledge information according to an analysis or application service type.

In addition, the visual knowledge utilization unit 40 may have a schema or data structure different from the schema or data structure of the visual knowledge storage unit 20 in part or all of the visual knowledge.

The visual knowledge utilization unit 40 is limited to a developer, a user, or the like authorized to provide OAuth or the like publicly or separately, file transfer using a network protocol, file transfer using a file download function such as a browser, data transfer using an API, and the like. Through the method of the visual knowledge storage unit 20 may be provided with some or all of the visual knowledge.

The schema provider 30 may store information about a schema or data structure predefined by the visual knowledge storage 20. Such information may be provided through a document, an API disclosure, a portal, or the like.

The visual knowledge utilization unit 40 may be provided with information on a schema or data structure provided by the schema provider through an authentication method such as OAuth, which is publicly provided or separately provided by the schema provider 30.

Hereinafter, the operation principle of the visual knowledge storage and management system according to the present invention configured as described above will be described with reference to FIG.

2 is a system block diagram for explaining the operation principle of the visual knowledge information storage and management system according to the present invention.

As shown in FIG. 2, the visual knowledge generator 10 may generate visual knowledge from an input image. The visual knowledge storage unit 20 may receive visual knowledge from at least one visual knowledge generator 10 and store the visual knowledge information according to a predefined schema or data structure.

In addition, the visual knowledge storage unit 20 may provide the schema provider 30 with information about a schema or data structure that is defined in cooperation with the schema provider 30. The at least one visual knowledge utilization unit 40 may be used to provide an application service using visual knowledge analysis or visual knowledge by receiving some or all of the pre-collected visual knowledge in association with the visual knowledge storage unit 20.

The visual knowledge utilization unit 40 may receive information on the schema or data structure used by the visual knowledge storage unit 20 through the schema provider 30, and the visual knowledge storage unit 20 may provide the information. The visual knowledge may be converted into a schema or data structure used by the visual knowledge utilization unit 40.

In addition, the visual knowledge utilization unit 40 may provide an analysis or application service utilizing visual knowledge by interworking with at least one user through separate input and output units.

The schema provider 30 may store information on a schema or data structure used by the visual knowledge storage 20.

In addition, the schema providing unit 30 may provide a pre-stored schema or data structure in association with the at least one visual knowledge utilization unit 40.

Here, the visual knowledge generation unit 10 in the visual knowledge information storage and management system according to the present invention will be described in detail.

3 is a block diagram illustrating the detailed block diagram of the visual knowledge generator 10 in the visual knowledge information storage and management system according to an exemplary embodiment of the present invention.

As shown in FIG. 3, the visual knowledge generation unit 10 includes an image information input unit 11, a scene generation unit 12, an object information generation unit 13, an inter-object relationship generation unit 14, and event information generation. The unit 15 and the knowledge generator 16 may be included.

The image information input unit 11 receives an image, which is the object of visual knowledge generation, as an input and provides it to the scene generator 12. In this case, the input image may include a pre-stored image file, a stream image that is continuously stored, such as CCTV.

The scene generator 12 may generate a scene, which is an image section including a specific event occurring in the image provided from the image information input unit 11. For example, in the case of a long CCTV video, only similar background screens are constantly photographed, but sometimes an object such as a person or a car may appear to generate an event. Considering this video section as a scene, and generating and storing visual knowledge only for the scene enables an effective and efficient system configuration.

The object information generator 13 may generate information about objects appearing in at least one or more scenes generated by the scene generator 12. Here, the object information may be generated by a human being directly tagged and tagged or automatically generated by an automated object detection and recognition algorithm.

The object-to-object relationship generator 14 may analyze the relationship between objects using object information generated by the object information generator 13 and generate the information. For example, by analyzing the spatial relationship of the objects, it is possible to extract information such that the two objects are closer to each other, or to maintain a constant distance, and to generate the information between the objects.

The event information generating unit 15 detects an event occurring in a corresponding scene by using the object information generated by the object information generating unit 13 and the relationship generating information between the objects generated by the relationship generating unit 14 and the information. Can be generated. For example, in a scene, when a person and a car appear as an object, and the person and the car are near by the relationship information between objects, and soon after the person disappears, the event may be recognized as “a person gets into the car”.

The knowledge generator 16 generates visual knowledge information according to the scene information generated by the scene generator 12, the object information generated by the object information generator 13, and the event information generated by the event information generator 15. To generate.

Meanwhile, a detailed configuration and operation of the visual knowledge storage unit 20 in the visual knowledge information storage and management system shown in FIG. 1 will be described with reference to FIG. 4.

4 is a block diagram illustrating a detailed block configuration of the visual knowledge storage unit 20 shown in FIG. 1.

As shown in FIG. 4, the visual knowledge storage unit 20 generates a knowledge input unit 21, a scene information storage unit 22, an object information storage unit 23, an event information storage unit 24, and an index structure. It may include a portion (25).

The knowledge input unit 21 analyzes the visual knowledge structure using the visual knowledge information generated by the visual knowledge generator 10, and then extracts necessary parts from the scene information storage unit 22, the object information storage unit 23, and event information. It may be provided to the storage unit 24. Here, the visual knowledge information generated by the visual knowledge generator 10 may include scene information generated by the scene generator 12 illustrated in FIG. 3, object information and event information generated by the object information generator 13. It may include event information generated by the generation unit 15.

The scene information storage unit 22 may store scene information generated by the scene generator 12 of the knowledge information generator 10 of visual knowledge.

The object information storage unit 23 may store object information generated by the object information generation unit 13 of the knowledge information generation unit 10 of visual knowledge. In this case, as shown in FIG. 3, since the object information is detected in the scene, the object information may be stored in a form dependent on the scene information.

The event information storage unit 24 may store event information generated by the event information generator 14 of the knowledge information generator 10 of visual knowledge. In this case, as shown in FIG. 3, since event information is generated using object information and relationship information between objects, the event information may be stored in a form dependent on the object information.

After storing the scene information, object information, and event information, the index structure generator 25 may generate various types of index structures for quick querying and efficient management.

Hereinafter, the data structure of the visual knowledge storage unit 20 will be described with reference to FIG. 5.

5 is a diagram showing the data structure of the visual knowledge storage unit 20 according to an embodiment of the present invention.

First, the data structure of the visual knowledge storage unit 20 may be configured to correspond to constituent items (Syntactics) and semantics (Semantics).

At least one scene information (Scenes) may be stored in the configuration item. The scene information may be stored in association with at least one frame information (Frames).

In addition, object information detected in the scene may be stored in the constituent item. Here, the object information may store the detected location information (ObjectObservations) every frame. For example, the object position information may be stored in the form of a reference to the detected frame, an X axis coordinate, a Y axis coordinate, a horizontal length, and a vertical length. The semantic item may store event information (Events) detected in the scene. In this case, the event information may be stored in relationship with at least one object information.

The event information may store the detected location information (EventObservations) every frame. For example, the event location information may be stored in the form of a reference to the detected frame, an X axis coordinate, a Y axis coordinate, a horizontal length, and a vertical length in a similar manner to the object location information.

Meanwhile, a detailed configuration and operation of the visual knowledge utilization unit 40 illustrated in FIG. 1 will be described with reference to FIG. 6.

6 is a block diagram showing a detailed configuration of the visual knowledge utilization unit 40 according to an embodiment of the present invention.

As shown in FIG. 6, the visual knowledge utilization unit 40 includes a schema constructing unit 41, a data receiving unit 41, a data storage unit 43, a data control unit 44, a user input unit 45, and a result output. It may include a portion 46.

The schema configuration unit 41 may analyze a method of storing or configuring data necessary for utilizing visual knowledge, and may perform a task of creating a schema according to an application method. In this case, the visual knowledge schema or storage structure used by the visual knowledge storage unit 20 may be provided from the schema provider 30 to be referred to when constructing the schema.

The data receiver 42 receives the visual knowledge information necessary from the visual knowledge storage unit 20 based on the schema configured in the schema configuration unit 41.

The data storage unit 43 may store visual knowledge information received through the data receiving unit 42 based on the schema configured in the schema forming unit 41.

The data controller 44 may receive data necessary to provide an analysis or application service using visual knowledge from the data storage unit 43 and perform operations such as data processing, query generation, and data visualization.

If necessary, the data controller 44 may receive a separate requirement from the user input unit 45 and reflect the analysis or application service. For example, in the case of a search service using visual knowledge, a query may be generated to search from the data storage unit 43 by receiving a search target as a requirement.

The user input unit 45 may receive a separate requirement from the user in order to provide an analysis or application service using visual knowledge in association with the user. For example, in the case of a search service using visual knowledge, an object type such as "People", "Car", "Bag", etc., such as "People", "Ride", "Ride", " Ride a bicycle ”, etc. You can enter the type of event.

The result output unit 46 may output the results of the analysis or application service of the data controller 44 to the user through a separate interface according to the user search request through the user input unit 45.

7 is a diagram illustrating a configuration of a schema structure used by the data storage unit 43 of the visual knowledge utilization unit 40 in FIG. 6 according to an embodiment of the present invention. Here, as an example of the visual knowledge search application, when the event type that the user wants to search, for example, “ride on”, “ride on a bicycle”, is input through the user input unit 45 of FIG. 6, the data controller of FIG. In the data storage unit 43 of FIG. 6, the data storage unit 43 generates a query for obtaining scene information including the corresponding event, related objects, and a time point at which the corresponding event starts. After receiving the visual knowledge information, a case in which visual knowledge information is provided to the result output unit 46 of FIG. 6 and outputted to the user will be described.

As shown in FIG. 7, the visual knowledge utilization unit 40 may include a scene table and an events table.

The scene table stores necessary information about the scene and may include a video file location (VideoLocation), a video resolution (VideoResolution), a video frame rate (VideoFps), a scene start point (FrameStart), and a scene end point (FrameEnd). .

The event table can make the event dependent on the scene through a reference to the scene table (SceneRef).

The event table may include object information (Subject, Object1, Object2) for the efficiency of the search. Here, the event may further include an event name (Activity), an event start point (FrameStart), and an event end point (FrameEnd).

Compared with the data structure of the visual knowledge storage unit 20 shown in FIG. 5, the schema structure of the visual knowledge utilization unit 40 shown in FIG. 7 does not use the form of the original data structure as it is. The item may not be used. For example, the schema structure of the visual knowledge utilization unit 40 of FIG. 7 does not include ObjectObservtions and EventObservations items used in the data structure of the visual knowledge storage unit 20 of FIG. 5.

The embodiments described above are just one example, and various modifications and changes may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

10: visual knowledge generation unit 11: image information input unit
12: Scene generation unit 13: Object information generation unit
14: relationship between the object generation unit 15: event information generation unit
16: knowledge generation unit 20: visual knowledge storage unit
21: Knowledge input unit 22: Scene information storage unit
23: object information storage unit 24: event information storage unit
30: schema providing unit 40: visual knowledge utilization unit
41: schema configuration unit 42: data receiving unit
43: data storage 44: data control
45: user input unit 46: result output unit

Claims (10)

In the visual knowledge information storage and management system,
Recognizing an image section including objects from an input image by using an image processing algorithm, object information on the objects, relationship information between objects indicating a relationship between the objects, relationship between the object information and the object A visual knowledge generating unit generating visual knowledge information expressing event information generated based on the information in a standardized metadata format;
A visual knowledge storage unit which stores the generated visual knowledge information according to a predefined standardized schema structure in association with the visual knowledge generating unit;
A schema provider for storing the formalized schema structure provided from the visual knowledge storage unit and providing the stored formalized schema structure; And
By referring to the formal schema structure provided from the schema provider, construct a schema structure necessary to use the visual knowledge, and store the visual knowledge information provided from the visual knowledge storage unit according to the schema structure necessary to use the visual knowledge And a visual knowledge utilization unit providing the application service for utilizing the visual knowledge by using the stored visual knowledge information according to the schema structure necessary for utilizing the visual knowledge.
The method of claim 1,
The visual knowledge storage unit,
Define a database schema using a data definition language provided by a database management system, store visual knowledge information using a data manipulation language, or use an XML schema. A visual knowledge information storage and management system for describing a structure for storing and storing the visual knowledge information in an XML form that conforms to the XML schema structure, in a file format.
The method of claim 1,
The visual knowledge generation unit,
An image information input unit which receives an image to be visually generated;
A scene generation unit generating a scene, which is an image section including a specific event occurring in an image input through the image information input unit;
An object information generator for generating information about objects appearing in at least one or more scenes generated by the scene generator;
An inter-object relationship generation unit analyzing the relationship between objects using the object information generated by the object information generation unit and generating the information;
An event information generation unit for generating event information by detecting an event occurring in a corresponding scene using the object information generated by the object information generation unit and the relationship generation information between objects generated by the relationship generation unit; And
And storing the visual knowledge information based on the scene information generated by the scene generator, the object information generated by the object information generator, and the event information generated by the event information generator. Management system.
The method of claim 3,
The visual knowledge storage unit,
A knowledge input unit for analyzing the visual knowledge structure using the visual knowledge information generated by the visual knowledge generator and providing the analyzed visual knowledge;
A scene information storage unit for storing scene information generated by the scene generation unit of the visual knowledge generation unit;
An object information storage unit for storing object information generated by the object information generation unit of the visual knowledge generation unit;
An event information storage unit for storing event information generated by the event information generation unit of the visual knowledge generation unit; And
And storing the scene information, object information, and event information, and generating an index structure for quick querying and efficient management.
The method of claim 4, wherein
The visual knowledge utilization unit,
A schema constructing unit constituting a schema structure necessary to utilize visual knowledge;
A data receiving unit for receiving the necessary visual knowledge information based on the schema structure configured in the schema constructing unit from the visual knowledge storing unit;
A data storage unit for storing visual knowledge information received through the data receiving unit based on a schema configured in the schema constructing unit;
A data controller configured to perform operations including data processing, query generation, and data visualization on the visual knowledge information provided from the data storage unit;
A user input unit for receiving a separate requirement from a user to provide an analysis or application service using visual knowledge in association with a user; And
And a result output unit configured to output a result of an analysis or application service of a data controller to a user using a separate interface according to a user search request through the user input unit.
The visual knowledge generating unit recognizes an image section including objects from an input image by using an image processing algorithm, and includes object information on the objects, inter-object relationship information indicating a relationship between the objects, and the object information. Generating visual knowledge information expressing event information generated on the basis of the relation information between objects in a standardized metadata format;
A visual knowledge storage unit, storing the generated visual knowledge information in a storage device according to a predefined standardized schema structure;
A schema provider for storing the formalized schema or data structure and providing a stored formalized schema structure; And
The visual knowledge utilization unit constructs a schema structure necessary for utilizing visual knowledge by referring to the formalized schema structure provided from the schema provider, and stores the generated visual knowledge information according to the schema structure necessary for utilizing the visual knowledge. And providing the application service for utilizing the visual knowledge by using the stored visual knowledge information according to the schema structure necessary for utilizing the visual knowledge.
The method of claim 6,
The storing step,
Define a database schema using a data definition language provided by a database management system, store visual knowledge information using a data manipulation language, or use an XML schema. A method of storing and managing visual knowledge information by describing a structure for storing and storing the visual knowledge information in an XML form conforming to the XML schema structure and in a file format.
The method of claim 6,
Generating the visual knowledge,
Receiving an image that is a target of visual knowledge generation;
Generating a scene, which is an image section including a specific event occurring in the input image;
Generating information about objects appearing in each of the at least one generated scene;
Analyzing relationship between objects using the generated object information to generate relationship information between objects;
Generating event information by detecting an event occurring in a corresponding scene by using the generated object information and relationship information between objects; And
And generating visual knowledge information according to the generated scene information, object information, and event information.
The method of claim 8,
The storing step,
Analyzing the visual knowledge structure using the generated visual knowledge information and providing the analyzed visual knowledge;
Storing the generated scene information, object information, and event information;
And storing the stored scene information, object information, and event information, and then creating an index structure for quick querying and efficient management.
The method of claim 9,
Providing to the application service,
Constructing a schema structure necessary for utilizing visual knowledge;
Receiving necessary visual knowledge information from a storage device based on the configured schema structure;
Storing the received visual knowledge information based on the configured schema structure;
Performing operations including data processing, query generation, and data visualization on the stored visual knowledge information;
Receiving a separate requirement from a user to provide an analysis or application service using visual knowledge in association with a user; And
And outputting the result of the analysis or application service to a user using a separate interface according to the user search request.

Images