KR102014267B1 - Method and Apparatus of building Inverse Index DB for high speed searching of moving picture object - Google Patents

Abstract

According to the present invention, provided is a method for constructing an inverse index DB for high speed image searching. The method is performed by an apparatus for constructing an inverse index DB, and comprises: a rowkey value definition step for defining a rowkey value of a detection object NoSQL DB in which representative images of the detected object are inserted for each camera type; and an object searching step for searching for a specific object detected from a specific camera based on the rowkey value of the NoSQL DB. When a user wants to search for an object by camera or only to search for a specific object instead of a photographed camera, a different type of DB can be used, thereby enabling high-speed image searching.

Description

Method and Apparatus of building Inverse Index DB for high speed searching of moving picture object}

The present invention relates to a method and apparatus for constructing an inverse index DB for fast image retrieval. More specifically, the present invention relates to a method and apparatus for constructing an inverse index DB for fast image retrieval, and an object search method and apparatus using the same.

Database (DB) means a set of data that is integrated and managed for the purpose of being shared and used by a plurality of information and communication devices or users. Data is stored and stored with regularity by eliminating duplication of data items. Means a collection of data to improve efficiency when searching, changing, and adding new material.

With the development of big data and ICT technology, data such as photographs, audio, and video are being produced by various information communication devices such as mobile phones, tablet PCs, webcams, and CCTVs. The information and data produced here are enormous and are shared in real time without being constrained by space online. As the amount of information and data increases rapidly, the necessity of efficient data management emerges. Is being used.

On the other hand, in the case of a large-scale video system with more than thousands of video data inputs, the size of the video data increases enormously with time, and thus, when the video data is retrieved through the database, the speed becomes very slow. . In particular, in order to detect a desired object in such a large image database, there is a problem in that it is almost impossible because the objects in all stored images must be sequentially searched and found.

 Accordingly, it is an object of the present invention to provide a method and apparatus for constructing an inverse index DB for fast image retrieval and an object search method and apparatus using the same.

In addition, an object of the present invention is to provide a high-speed image search system, a detection object NoSQL DB, Inverse Index DB to solve this problem.

In addition, an object of the present invention is to provide a structure that allows a user to quickly search for a desired object in a large amount of image data by receiving the results of the object detected in such a database and the object detection module.

In order to solve the above problems, there is provided a method of constructing an inverse index DB for fast image retrieval according to the present invention. The method may further include: defining a rowkey value of a detection object NoSQL DB in which a representative image of a detected object is inserted for each camera type, which is performed by an inverse index DB construction apparatus for high-speed image retrieval; And an object searching step of searching for a specific object detected by a specific camera based on the rowkey value of the NoSQL DB, and the user wants to search for an object by camera or only to search for a specific object other than the photographed camera. Different types of DBs can be used in some cases, so that high-speed video retrieval is possible.

In an embodiment, the rowkey value defining step may variably set the number of the representative images of the detection object NoSQL DB according to the importance of contributing to the search for the specific object.

According to an embodiment, after the rowkey value defining step, the structure may be rearranged by inserting the objects detected by the object detection module into the rowkeys of the inverse index DB for each type.

In an embodiment, in the object searching step, a camera-based search may be performed using a rowkey value of the NoSQL DB defined for each camera type. In this case, a representative image based search may be performed on the representative images of the detected objects using the inverse index DB.

According to an embodiment, after the object searching step, the method may further include additional information modification.

According to an embodiment, in the object searching step, the detection period of the object search may be detected on a variable or event basis according to the camera type, the position of the camera, and the property of the object photographed by the camera. On the other hand, after the object searching step, for the image after the detection period, the importance that contributes to the search of the specific object of the camera according to the property of the object is corrected, and based on the modified importance of the representative image The method may further include adjusting additional information.

According to an embodiment, after the additional information modification step, the object searching step of searching for a specific object detected by a specific camera is performed based on the modified importance, the number of the adjusted representative image, and the rowkey value of the NoSQL DB. You can do more.

According to at least one embodiment of the present invention, a fast image search system including an inverse index DB for fast image search is different depending on when a user wants to search for an object for each camera or only a specific object instead of a photographed camera. Provide convenience by using kind DB.

In addition, according to at least one embodiment of the present invention, the Inverse Index DB may provide a new concept NoSQL-based database for relocating the structure of the database for each type of detected object.

In addition, according to at least one embodiment of the present invention, there is an advantage that a user can quickly search for an object that a user wants to find in large-scale image data by introducing a NoSQL-based database.

1 shows a detailed structure of an object searching apparatus according to the present invention.
2 shows a detailed structure of an object searching apparatus according to the present invention.
3 is a conceptual diagram of a detection object NoSQL DB according to the present invention.
4 shows a conceptual diagram of an Inverse Index DB according to the present invention.
5 is another example of a conceptual diagram of an Inverse Index DB according to the present invention.
6 is a flowchart illustrating a method of constructing an inverse index DB for fast image search and an object search method according to the present invention.

The above-described features and effects of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, and thus, those skilled in the art to which the present invention pertains may easily implement the technical idea of the present invention. Could be.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

In describing each drawing, like reference numerals are used for like elements.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art.

Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and are not construed in ideal or excessively formal meanings unless expressly defined in this application. Should not.

The suffix modules, blocks, and parts for components used in the following description are given or mixed in consideration of ease of specification, and do not have distinct meanings or roles from each other.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described to be easily carried out by those of ordinary skill in the art. In the following description of the embodiments of the present invention, when it is determined that the detailed description of the related known functions or known configurations may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

Hereinafter, a method and apparatus for constructing an inverse index DB for fast image retrieval according to the present invention. In more detail, a method and apparatus for constructing an inverse index DB for high-speed image search and an object search method and apparatus using the same will be described.

In this regard, Figure 1 shows a detailed structure of the object searching apparatus according to the present invention. That is, FIG. 1 shows a conceptual diagram for searching for an object in an image. Image data taken by multiple cameras is stored in a designated image data file system and DB (database, image data storage space, DB type (RDB / NoSQL) irrespective of this patent). In order to find an object of interest (eg, a person's face, a car, an animal, etc.) in a stored image, the user must find a desired object by sequentially searching all stored image data. In other words, if you find all the objects while viewing the images sequentially, you can get the desired result by gathering the appropriate information such as camera number, location, time, and appearance.

However, the conventional object search method described above may take a long time because a person needs to find the desired object by sequentially searching all the image data. In addition, as the number of cameras increases, and as the number of stored images increases, the object search time increases proportionally. This makes it difficult to search for objects in hundreds or thousands of large-scale image data in a conventional manner.

2 shows a detailed structure of an object searching apparatus according to the present invention. FIG. 2 shows an Inverse Index DB proposed by the present invention and a fast image search system (dotted box) using the same to solve this problem.

Image data shot by multiple cameras is stored in the image data file system and DB and sent to the object detection module. This module includes the existing object recognition algorithm and AI neural network (Faster R-CNN, YOLO, Single Shot multibox Detector, etc.) algorithm, and receives the original image, detects the object and sends it to the high speed image retrieval system. Perform. Here, the image information (object type, detected still image, camera, shooting location, shooting time, etc.) of the detected object is stored in a structure suitable for the detection object NoSQL DB and Inverse Index DB, respectively. The user can use two DBs of the high-speed image search system according to the intention (1.Search by camera: when interested in a specific camera / 2. Search by object: when interested in photographed objects) Can be provided at a high speed. This will be explained in more detail later.

The detection capability and the type of detection object vary according to the type of algorithm for detecting the object in the object detection module. For example, suppose there is an algorithm called Model-1, which is designed to recognize human faces, and Model-2, which is designed to recognize types of cars. If the user wants to detect all the cars and the types of the cars in the stored image, 'model-2' is used in the object detection module, and thus the object detection module detects the car. Likewise, if you want to detect human face, use an algorithm called 'Model-1' in the module.

3 is a conceptual diagram of a detection object NoSQL DB according to the present invention. Referring to FIG. 3, information about an object detected by the object detection module is provided. 'Object NoSQL DB' is characterized by NoSQL-based database. Existing relational databases (RDBs), which are not in NoSQL form, are not flexible because the number of columns or rows has to be redesigned from scratch to accommodate the increased number of predefined databases. However, since this is a NoSQL-type database, it does not matter if the camera is increased (number of rows) or the amount of objects detected by the same camera is increased (number of columns).

As an example for the sake of understanding, assuming that 1000 cameras are installed in a zoo, different animals will be photographed on 1000 different cameras. In this DB, a row of cameras is included in a row called Rowkey, so objects such as monkeys and lions photographed by camera 1 contain several thumbnails (representative images), shooting time, shooting location and location. In this example, N thumbnails are extracted for one photographed object, and the number can be set by the user according to importance.

If the user wants to search for the object shot by camera, he uses 'NoSQL DB', which is a member of the 'high-speed image search system'.

On the other hand, Figure 4 shows a conceptual diagram of the Inverse Index DB according to the present invention. That is, FIG. 4 is an example of a conceptual diagram drawn to explain in detail the structure of an Inverse Index DB. Inverse Index DB is characterized by NoSQL-based database as shown in [Figure 3].

Assuming an example as shown in FIG. 3, different animals will be photographed by 1000 different cameras. If the user is not interested in the camera and just wants to search for the monkey object, he can use Inverse Index DB, which belongs to the high-speed video search system. Conventionally, the user has to search for the monkey by playing the original video recorded by 1000 cameras, but this wastes a lot of time and manpower. Even if only the object detection module exists, it is necessary to find all the objects called monkeys detected by the 1000 cameras, which is also time-consuming.

However, Inverse Index DB is a structure that rearranges rows based on the objects detected by the object detection module. Eventually, the detection object NoSQL DB of FIG. 3 is rearranged, and the type of the detected object is included in a row called Rowkey, and thus, an object of a kind such as lion, giraffe, and monkey is included in the example of FIG. In the rowkey where the lion is detected, the lion objects captured by the 1000 cameras are captured together with thumbnails (typical images), the photographed camera, and the shooting time. When a user wants to quickly find an object of a lion type, the user can obtain information such as when and what camera was taken by referring only to the detected rowkey, so that the object can be detected at a high speed in a large image.

5 does not actually create a database according to the present invention as shown in FIG. Accordingly, the DB of FIG. 5 may be an auxiliary DB indicating auxiliary information of an inverse DB configured as shown in FIG. 4.

6 is a flowchart illustrating a method of constructing an inverse index DB for fast image searching and an object searching method according to the present invention. Referring to FIG. 6, a rowkey value defining step S110, an object searching step S120, and an additional information modification step S130 are included. Meanwhile, after the additional information modification step S130, the object search step S120 may be performed on the image to be searched. Meanwhile, each step of the inverse index DB construction method and the object search method may be performed by an object detection module and a high speed image search system. In detail, the rowkey value defining step S110 may be performed by the object detection module and the detection object NoSQL DB. In addition, the object search step (S120) may be performed by the detection object NoSQL DB and Inverse Index DB. In addition, the additional information modification step (S130) may be performed by the detection object NoSQL DB.

In the rowkey value defining step (S110), the rowkey value of the detection object NoSQL DB into which the representative images of the detected objects are inserted is defined for each camera type. Meanwhile, in the object search step (S120), a specific object detected by a specific camera may be searched based on the rowkey value of the NoSQL DB.

Meanwhile, in the rowkey value defining step S110, the number of representative images of the detection object NoSQL DB may be variably set according to the importance that contributes to the search for the specific object. In this case, after the rowkey value defining step (S110), the structure may be rearranged by inserting the objects detected by the object detection module into the rowkeys of the inverse index DB for each type.

On the other hand, in the object search step (S120), the camera-based search can be performed using the rowkey value of the NoSQL DB defined for each camera type. In this case, a representative image based search may be performed on the representative images of the detected objects using the inverse index DB.

On the other hand, as described above, after the additional information modification step (S130), the object search step (S120) may be performed again for the image to be searched.

Meanwhile, in the object searching step (S120), the detection period of the object searching may be detected on a variable or event basis according to the camera type, the position of the camera, and the attributes of the object photographed by the camera.

In this regard, in the additional information modification step (S130) after the object search step (S120), for the image after the detection period, it is possible to modify the importance that contributes to the search of the specific object of the camera according to the property of the object. have. In addition, in the additional information modification step (S130), the number of representative images may be adjusted based on the modified importance.

On the other hand, after the additional information modification step (S130), based on the modified importance, the number of the adjusted representative image and the rowkey value of the NoSQL DB, an object search step (S120) for searching for a specific object detected by a specific camera May be further performed.

The above relates to a method and apparatus for constructing an inverse index DB for fast image retrieval according to the present invention. More specifically, the method and apparatus for building an inverse index DB for fast image retrieval and the object searching method and apparatus using the same have been described. Looking at the effects of the present invention as follows.

According to at least one embodiment of the present invention, a fast image search system including an inverse index DB for fast image search is different depending on when a user wants to search for an object for each camera or only a specific object instead of a photographed camera. Provide convenience by using kind DB.

In addition, according to at least one embodiment of the present invention, the Inverse Index DB may provide a new concept NoSQL-based database for relocating the structure of the database for each type of detected object.

In addition, according to at least one embodiment of the present invention, there is an advantage that a user can quickly search for an object that a user wants to find in large-scale image data by introducing a NoSQL-based database.

According to the software implementation, the design and parameter optimization for each component as well as the procedures and functions described herein may be implemented as separate software modules. Software code may be implemented in software applications written in a suitable programming language. The software code may be stored in a memory and executed by a controller or a processor.

Claims (7)

In the Inverse Index DB construction method for high speed image retrieval,
A rowkey value defining step of defining a rowkey value of the detection object NoSQL DB into which representative images of the detected objects are inserted for each camera type; And
And an object searching step of searching for a specific object detected by a specific camera based on the rowkey value of the NoSQL DB. According to claim 1,
The rowkey value defining step,
And setting the number of the representative images of the detection object NoSQL DB variably according to the importance of contributing to the search for the specific object. According to claim 1,
After the rowkey value definition step,
A method of constructing an inverse index DB for high-speed image retrieval, characterized in that the objects are detected by the object detection module, and the structures are rearranged by inserting the detection objects into rowkeys of the inverse index DB. The method of claim 3, wherein
In the object search step,
Perform a camera-based search using the rowkey value of the NoSQL DB defined for each camera type,
And performing representative image based searching on the representative images of the detected objects using the inverse index DB. delete The method of claim 3, wherein
In the object searching step, the detection period of the object search is detected on a variable or event basis according to the camera type, the position of the camera and the attributes of the object photographed by the camera,
After the object searching step,
For the image after the detection period, the additional information correction step of correcting the importance that contributes to the search of the specific object of the camera according to the property of the object, and adjusts the number of the representative image based on the modified importance Further comprising, Inverse Index DB construction method for high-speed image search. The method of claim 6,
After the additional information modification step,
And performing the object searching step of searching for a specific object detected by a specific camera based on the modified importance, the adjusted number of representative images, and the rowkey value of the NoSQL DB. How to build an Inverse Index DB.

Images