KR102435494B1 - Apparatus for searching object trajectories in video clips captured by static cameras - Google Patents

Abstract

The present invention relates to an apparatus for searching an object path inside a fixed camera image, and the apparatus for searching an object path inside a fixed camera image according to an embodiment of the present invention includes a path extracting unit for extracting path data of an object inside a fixed camera image. ; a path analysis unit for classifying the path data according to a preset criterion to generate a path index representing a path through which the object actually passes; and a route search unit configured to provide a route search application service by setting a search route, a search condition, and a search accuracy using the route data and the route index.

Description

Object path navigation device inside fixed camera image {APPARATUS FOR SEARCHING OBJECT TRAJECTORIES IN VIDEO CLIPS CAPTURED BY STATIC CAMERAS}

The present invention relates to an apparatus for searching an object path inside a fixed camera image, and more particularly, by extracting and analyzing the movement path of an object within a fixed camera image, a path search application service (path search for an object, abnormal path detection for an object) is provided to the user. , movement path prediction, etc.), relates to a device for searching an object path inside a fixed camera image.

In recent years, there is a growing demand for fixed camera systems to be installed at major bases that threaten personal and social safety, such as disasters, disasters, crimes, and accidents. However, conventional methods for effectively analyzing and utilizing fixed camera images are insufficient.

Conventionally, it is difficult to efficiently utilize one or more fixed camera images because the manager directly observes and monitors the images. Since this method observes and monitors the video depending on the manager's own memory and subjectivity, important parts of the video may be missed or incorrect results may be caused by mistake.

In particular, a technique for analyzing a path along which an object in a fixed camera image moves is not used in the prior art. In fact, the fixed camera image can be applied in various ways to search for a vehicle moving along a specific path, detect a person moving along an abnormal path, or predict a path that a lost child moves. As such, the fixed camera image is not used in a passive position to simply observe and monitor the external situation, but it is necessary to propose a method for predicting the future path by analyzing the movement of the object in the image more actively. .

The present invention was created by the above needs, and by extracting and analyzing the moving path of an object from within the fixed camera image, it provides a path search application service (path search for an object, abnormal path detection, movement path prediction, etc.) to the user. An object of the present invention is to provide a device for searching an object path inside a fixed camera image for providing.

In order to achieve the above object, an apparatus for searching an object path inside a fixed camera image of the present invention includes: a path extracting unit for extracting path data of an object inside a fixed camera image; a path analysis unit for classifying the path data according to a preset criterion to generate a path index representing a path through which the object actually passes; and a route search unit configured to provide a route search application service by setting a search route, a search condition, and a search accuracy using the route data and the route index.

According to the present invention, the apparatus for searching an object path inside a fixed camera image of the present invention extracts and analyzes a movement path of an object from within the fixed camera image to provide a user with a path search application service (path search for an object, abnormal path detection, It has the effect of providing movement path prediction, etc.).

In addition, the object path search apparatus of the present invention has the effect of providing a path-based application service by providing a filtering function such as a path search interface, a search condition setting, and a precision setting.

In addition, the object path search apparatus of the present invention has the effect of providing various path search services by generating a path index that can be expressed in a more subdivided hierarchical structure according to the path configuration.

1 is a diagram illustrating an apparatus for searching an object path inside a fixed camera image according to an embodiment of the present invention.
2 is a diagram of a path analysis unit according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided so that those who have it can easily understand the scope of the invention, and the present invention is defined by the description of the claims. Meanwhile, the terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” or “comprising” refers to the presence or absence of one or more other components, steps, operations and/or elements other than the stated elements, steps, acts and/or elements. addition is not excluded.

1 is a diagram illustrating an apparatus for searching an object path inside a fixed camera image according to an embodiment of the present invention.

An apparatus for searching an object path in a fixed camera image according to an embodiment of the present invention (hereinafter, referred to as an "object path search apparatus", 100) extracts and analyzes a movement path of an object in a fixed camera image to provide a path search to a user Provides application services (path search for an object, abnormal path detection, movement path prediction, etc.). Here, the object means an object moving in the image, that is, a car, a person, an unidentified object, and the like.

As shown in FIG. 1 , the object path search apparatus 100 includes a path extractor 110 , a path database 120 , a path analyzer 130 , and a path search unit 140 .

In this case, the path extractor 110 receives one or more fixed camera images from the fixed camera system 200 . Here, the fixed camera system 200 stores the images collected at one or more fixed camera installation points, that is, when transferring the fixed camera images to the path extraction unit 110, and stores the images collected for a certain period of time and delivers them at a time or at a predetermined time. , the video collected every moment can be delivered in real-time streaming. In this case, the fixed camera system 200 may transmit the collected image itself without modification or may transmit only the image corresponding to the preset ROI by distinguishing it.

In addition, the fixed camera system 200 may be mounted inside the object path search apparatus 100 to be operated as an integrated body, or may be installed separately from the object path search apparatus 100 and operated as a separate type.

Hereinafter, components of the object path search apparatus 100 will be described in detail.

First, the path extractor 110 is connected to the fixed camera system 200 to which one or more fixed camera images are transmitted. When the image collected from the fixed camera system 200 is transmitted, the path extraction unit 110 extracts a path to which point an object in the image moves by using a conventional object extraction technique or an object tracking technique. . At this time, the path extractor 110 checks the path data (ie, time and location data) on which the object moves in every frame or partly selected frame of the image transmitted from the fixed camera system 200, and thus the object in the corresponding screen. can extract the path (that is, the change in the position of the object) of .

For example, the path (s) of the object may be represented by a pair of time (t) and the position (p) of the object in the screen. That is, paths s ₁ , s ₂ ,… ,s _i ,… ,s _k can be expressed as an array of pairs of time t ⁱ _j and in-screen position p ⁱ _j , respectively. Then, the path extractor 110 may extract the movement path of the object in the corresponding screen as a pair arrangement of time and the position of the object in the corresponding screen.

The path database 120 receives and stores and manages the movement path data of each object, that is, the path data 121 from the path extractor 110 . The path data 121 may be transmitted to the path analyzer 130 as an input for generating the path index 122 . In addition, the route data 121 may be transmitted to the route search unit 140 as a target or result of the route search.

Meanwhile, the path database 120 receives and stores and manages the path index 122 of each object from the path analysis unit 130 . The path index 122 may be used for various purposes according to the application of the path search unit 140 . Also, the method of using the path index 122 may vary according to the purpose of the path search unit 140 and search conditions.

As such, the path database 120 stores and manages the path data 121 and the path index 122 .

The path analysis unit 130 classifies the path data 121 expressed in an arrangement of time and location according to a preset criterion to generate a path index 122 representing a path through which an object actually passes.

Hereinafter, the path index 122 will be described as an example.

Paths s ₁ , s ₂ ,… ,s _i ,… ,s _k can be expressed as an array of pairs of time t ⁱ _j and in-screen position p ⁱ _j , respectively. In this case, each path obtains a sub-path r ⁱ _j through which the object actually passes through the path classification unit 132 of the path analysis unit 130 . Here, the sub-path can be expressed as data indicating the hierarchical relevance of each item due to the nature of the data (that is, it can be expressed as a structure that can encompass the upper and lower structures), It means that it is a path of a substructure of each path. The sub-path r ⁱ _j is used to configure an index (path index) of each path through the index generator 133 .

As an example, the path s _i is a sub-path r ⁱ _{j ,} r ⁱ _{2 , …} as in [Equation 1] below. _, r ⁱ _mi passes through, and the index of the path ^{si is [r i} _{j , r} ⁱ _{2 , … ,} r ⁱ _mi ].

Meanwhile, as shown in Equation 2, the expression method of the path index 122 may vary according to the expression method of the sub-path.

For example, it is assumed that a sub-path is partitioned into another sub-path. If a sub-path c ⁱ exists as an index of the path s _i , and the sub-path c ⁱ is another sub-path r ⁱ _{j ,} r ⁱ _{2 , ... ,} ri _mi , the index of the path _si is not only generated by one of the partitions, c ⁱ , but also by the other one of _the partitions, ^{ri j} _, ^ri _{2 , ... ,} ^ri _mi can also be created.

At this time, the path index [r ⁱ _j, r ⁱ _{2, ... ,} ri _mi ] and the path index c ^{i may} have a hierarchical structure. In this case, the path index may be expressed in a more granular hierarchical structure according to the configuration of the lower path, or may be expressed as a plurality of individual indices without a hierarchical structure.

In [Equation 2], the path index is expressed in a two-level hierarchical structure such as 'path index 1' and 'path index 2'. 'Path index 1' of path s _i is one of the divisions of path s _i s _i ←[r ⁱ _{j ,} r ⁱ _{2 , … ,} r ⁱ _mi ], and 'path index 2' is s _i ← c ⁱ , which is another one of the divisions of path s _i .

As described above, the path analysis unit 130 analyzes and refines the path data 121 to generate the path index 122 . To this end, the path analyzing unit 130 includes a path learning unit 131 , a path classifying unit 132 , and an index generating unit 133 .

First, the path learning unit 131 provides the previously learned path model to the path classifying unit 132 . In this case, the path classification unit 132 may perform a path classification task for generating the path index 122 using the path model provided from the path learning unit 131 . Here, the path model may be built by performing learning in advance before proceeding with the path classification task, or may be constructed by performing learning while performing the path classification task. In the latter case, it can be constructed using an adaptive path learning model that continuously updates the path model through input values.

The path classification unit 132 performs a path classification operation on the path data 121 of the object by using the path model provided from the path learning unit 131 . Here, the path classification operation is an operation of classifying which sub-path each path has passed, which cluster each path is included in, and the like. The path classification unit 132 transmits the classification result of the path classification task to the index generator 133 .

The index generating unit 133 generates a path index 122 by using the classification result transmitted from the path classifying unit 132 . Here, the classification result of the path classification task is used to generate a key value of the path index 133 . In this case, when generating the path index 122 , the index generator 133 may express each path as a data structure such as one key value or an array of key values. The index generator 133 generates the path index 122 and transmits it to the path database 120 .

Meanwhile, the index generator 133 may generate one or more path indexes 122 . In this case, the index generating unit 133 may generate the path index 122 having a hierarchical structure according to the configuration of the sub-path (the degree of subdivided expression).

The route search unit 140 provides algorithms and interfaces for providing various route search application services (such as route search for an object, abnormal route detection, movement route prediction, etc.). To this end, the path search unit 140 may include a path search screen 141 , a search path input unit 142 , a search condition setting unit 143 , and a search precision adjustment unit 144 .

First, the path search screen 141 provides an interface for facilitating the user's input and output. That is, the route search screen 141 provides a user interface environment for providing various route search application services according to the user's needs by using the route data 121 or the route index 122 . In this case, the route search screen 141 may implement an algorithm for a route search application service in conjunction with the search route input unit 142 , the search condition setting unit 143 , and the search precision adjustment unit 144 .

Here, the user may input a keyword or a natural language, and for this purpose, one or more input methods such as voice input, screen touch input, and keyboard input may be used. In addition, the user may check through one or more output methods such as file output, screen display, and voice output.

The search path input unit 142 searches for a path input by the user. In this case, the user searches for a corresponding route by inputting a keyword, natural language, or a manually drawn route related to the route to be checked.

In addition, the search condition setting unit 143 may provide the user with a search filtering condition setting function for more specifically and detailed confirmation of a path to be searched among the results searched by the user. In this case, the user may set one or more search filtering conditions such as object center, time center, screen space center, feature value center, and the like. The user can set these search filtering conditions individually or simultaneously.

The search precision adjusting unit 144 may provide the user with a search accuracy setting function of a result searched by the user. For example, the search precision adjustment unit 144 may provide a function of improving the accuracy of a search result searched by a user by using an approximate algorithm for a quick search. As another example, when the path index 122 is configured in a hierarchical structure, the search precision adjustment unit 144 uses the path index 122 with a high degree of detail expression to provide a high-precision search result, and uses the low-precision search result. If you want to provide a path index 122 with a low degree of detail expression can be used.

2 is a diagram of the path analysis unit 130 according to an embodiment of the present invention.

As shown in FIG. 2 , the path analysis unit 130 includes a path learning unit 131 , a path classifying unit 132 , and an index generating unit 133 , a detailed description thereof will be described with reference to FIG. 1 . Therefore, we will omit it. In order not to duplicate the path data 121 and the path index 122, detailed descriptions will be omitted.

In FIG. 2 , the data format conversion unit 132a and the sub-path classification unit 132b, which are components of the path classification unit 132 of the path analysis unit 130, will be described.

As described above in FIG. 1 , the path classification unit 132 performs a path classification operation on the path data 121 of the object. To this end, the path classifying unit 132 includes a data format converting unit 132a and a lower path classifying unit 132b.

First, when the route data 121 is transmitted, the data format conversion unit 132a converts the route data into a data format for route classification. In this case, the data format conversion unit 132a uses a data format conversion method such as Discrete Fourier Transform (DFT), Principal Component Analysis (PCA), and Vector Quantization. For example, the data format conversion unit 132a may use the location information and velocity information in the screen through any one of the discrete Fourier transform coefficients and vector quantization during principal component analysis. As such, the data format conversion unit 132a may use the information processed through the above-described data format conversion method as one or more combinations. If necessary, the data format conversion unit 132a may use the path data 121 itself without a separate data format conversion.

Next, the sub-path classifying unit 132b may use the data converted by the data format conversion unit 132a to obtain a sub-path passed by the corresponding path or arrangement information of the sub-path. In this case, the sub-path classification unit 132b performs sub-path classification using one or more of time series analysis, clustering, and probability modeling.

In addition, the sub-path classifying unit 132b transmits the analysis result to the index generating unit 133, and the index generating unit 133 uses the analysis result transmitted from the lower path classifying unit 132b to perform the path index 122. create

Meanwhile, the sub-path classifying unit 132b may receive information or a path model required for classification in this process from the path learning unit 131 . In addition, the lower path classifying unit 132b may transmit data to the path learning unit 131 when self-learning of the path model is performed.

As described above, according to the present invention, the apparatus for searching an object path inside a fixed camera image of the present invention extracts and analyzes a movement path of an object from within the fixed camera image to provide a user with a path search application service (path search for an object, an abnormal path). detection, movement path prediction, etc.).

In addition, the object path search apparatus of the present invention has the effect of providing a path-based application service by providing a filtering function such as a path search interface, a search condition setting, and a precision setting.

In addition, the object path search apparatus of the present invention has the effect of providing various path search services by generating a path index that can be expressed in a more subdivided hierarchical structure according to the path configuration.

Although the configuration of the present invention has been described in detail with reference to the preferred embodiment and the accompanying drawings, this is merely an example and various modifications are possible without departing from the technical spirit of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments and should be defined by the claims described below as well as the claims and equivalents.

100: object path search device 110: path extraction unit
120: route database 121: route data
122: path index 130: path analysis unit
131: path learning unit 132: path classification unit
133: index generation unit 140: path search unit
141: route search screen 142: search route input unit
143: search condition setting unit 144: search precision adjustment unit
200: fixed camera system

Claims (10)

a path extractor for extracting path data of an object inside the fixed camera image;
a path analysis unit for classifying the path data according to a preset criterion to generate a path index representing a path through which the object actually passes;
a route database for storing and managing the route data and the route index; and
A route search unit for providing a route search application service by setting a search route, a search condition, and a search accuracy using the route data and the route index,
The path search unit,
a path search screen that provides an interface for facilitating the user's input and output;
a search path input unit for retrieving a path input by a user;
a search condition setting unit for providing a user with a search filtering condition setting function for more specifically and in detail checking a path to be searched among the results searched by the user; and
and a search precision adjustment unit that provides a user with a search accuracy setting function of the results searched by the user,
The search precision adjusting unit provides a function of improving the accuracy of a search result searched by a user using an approximate algorithm,
When the path index is structured in a hierarchical structure, the apparatus for searching an object path inside a fixed camera image, characterized in that the path index corresponding to the search result of the found precision is used.
The method of claim 1,
Fixed camera system,
An apparatus for searching an object path inside a fixed camera image to transmit the collected image itself without modification, or to distinguish only the image corresponding to a preset interest section and transmit it to the path extractor.
The method of claim 1,
The route database is
An apparatus for searching an object path inside a fixed camera image for storing and managing the path data and a path index generated by analyzing and refining the path data.
The method of claim 1,
The path analysis unit,
a path learning unit providing the previously learned path model to the path classifying unit;
a path classification unit that performs a path classification task for generating a path index by using the path model provided from the path learning unit; and
and an index generator for generating the path index by using the classification result transmitted from the path classifying unit and transmitting it to the path database.
5. The method of claim 4,
The route classification unit
a data format conversion unit for converting the route data into a data format for route classification when the route data is transmitted; and
and a sub-path classifier configured to obtain information about a sub-path passed by the corresponding path or arrangement information of the sub-path by using the data converted by the data format conversion unit.
5. The method of claim 4,
The route classification task is
An object path search device inside a stationary camera image that classifies each path into which sub-path passed or which cluster each path is included in.
5. The method of claim 4,
The index generator,
An object path search device inside a fixed camera image that generates a path index having a hierarchical structure according to the configuration of a sub-path.
delete delete delete

Images