KR20210055567A - Positioning system and the method thereof using similarity-analysis of image - Google Patents

Abstract

The present invention relates to a positioning system using similarity-analysis of an image and a method thereof. The positioning system includes: an image storage system (160) in which grouped fingerprints and position information of each frame of a video containing a plurality of frames are indexed and stored; a moving means (400) for taking the video containing the frames; a feature point extraction unit (110) for generating the grouped fingerprints for each frame of the video taken by the moving means; an image analysis unit (120) for comparing the video of the moving means, from which the feature point is extracted, with the video of the image storage system, wherein the image analysis unit compares the feature points of the video of the moving means with the feature points of the video of the storage system to determine whether the similarity is greater than or equal to a threshold, thereby finding out the video of the storage system that corresponds to the video of the moving means, and recognizing position information of the video of the found storage system as position information of the moving means. Thus, the position of the imaging device is quickly checked.

Description

Position measurement system and method using image similarity analysis {POSITIONING SYSTEM AND THE METHOD THEREOF USING SIMILARITY-ANALYSIS OF IMAGE}

The present invention relates to a system for comparing screens of digital image files and measuring a location through similarity analysis and a method thereof. In particular, it is not an image captured with the camera fixed, but an image captured by a mobile camera that changes according to the shooting location and viewpoint to determine the location of the system to be photographed by comparing the previous image.

In order to operate a moving means such as a car or a drone, the location of the moving means is determined, and the driving direction and speed of the moving means are controlled based on the identified position.

GPS is used to determine the location of such a means of transportation, but the GPS cannot be used inside a building, under a bridge, or in a tunnel.

In the present invention, an image captured by a photographing means (a vehicle's black box, a drone's camera) attached to the vehicle is compared with an existing image to determine the current position of the vehicle, and Thus, it is intended to control the moving means.

Publication Patent No. 10-2015-0033043 (2015.04.01) Registered Patent No. 10-1240924-0000 (2013.03.04)

The present invention is to determine the location of the moving means by means of a photographing device provided in the moving means in order to operate a moving means such as a vehicle or a drone.

In order to determine the location, a singular point of a captured video image is extracted, stored in a storage system, and a singular point of a video image captured by a photographing device is extracted, and the singular points are compared, and thereby the location of the photographing device is determined.

The singularity is to divide the frame image of the captured image into a plurality of blocks, calculate the average value of the image brightness data of each pixel for each divided block unit, and then generate a pinker print using the difference in the average value of the image brightness data between blocks. , By comparing the frame images using the fingerprint generated in this way, it is intended to speed up the comparison of the frame images.

The present invention includes an image storage system 160 in which finger prints and location information grouped for each frame of a moving picture including a plurality of frames are indexed and stored; A moving means 400 for photographing a moving picture including a plurality of frames; A feature point extracting unit 110 for generating a fingerprint grouped for each frame of the moving picture photographed by the moving means; And an image analysis unit 120 that compares the video of the moving means from which the feature points are extracted with the video of the image storage system, wherein the image analysis unit compares the feature points of the video of the moving means with the feature points of the video of the storage system. By checking whether the similarity is greater than or equal to a threshold, the video of the storage system corresponding to the video of the moving means is found, and the location information of the found video of the storage system is recognized as the location information of the moving means. It relates to a position measurement system using.

In addition, in the present invention, the feature point extracting unit decodes a video stream of a moving picture into an RGB format or a black-and-white video format, and then divides the decoded frame image into a plurality of block units, and the image brightness data of each pixel for each block unit After calculating the average value of, a fingerprint grouped for each frame may be generated by extracting a feature point based on a difference between the average value of the image brightness data between adjacent blocks.

In addition, in the present invention, the image analysis unit compares the feature points extracted as a fingerprint grouped for each frame image of the image of the moving means and the indexed feature points of the image pre-stored in the image storage system, A pre-stored image including position information having a similarity of a feature point corresponding to a frame is greater than or equal to a threshold value may be found, and position information of the pre-stored image may be determined as the position of the moving means.

In addition, in the present invention, the storage system 160 includes a plurality of pre-stored images 200 and 210 including a finger print for each frame and location information, and the image analysis unit 120 is The image 300 is transmitted to the image analysis unit 120 in real time, and the image analysis unit 120 is 301, 302, 303, 304, 305, 306, 307 , 308) for each frame (P ₃₀₁ , P ₃₀₂ , P ₃₀₃ , P ₃₀₄ , P ₃₀₅ , P ₃₀₆ , P ₃₀₇ , P ₃₀₈ ) and the image (200, 210) previously stored in the storage system 160 ( 211, 212, 213, 214, 215, 216, 217, 218) with finger prints (P ₂₁₁ , P ₂₁₂ , P ₂₁₃ , P ₂₁₄ , P ₂₁₅ , P ₂₁₆ , P ₂₁₇ , P ₂₁₈ ) The abnormal frame may be found and the frame position information of the pre-stored image may be determined as the position information of the vehicle.

In addition, the present invention includes the steps of indexing the finger print and location information grouped for each frame of a moving picture including a plurality of frames and storing them in the image storage system 160; The moving means 400 photographing a moving picture including a plurality of frames; Generating a fingerprint grouped for each frame of the moving picture captured by the moving means by transmitting the moving means 400 to the feature point extracting unit 110; And comparing, in the image analysis unit 120, the moving picture of the moving means from which the feature points are extracted by the feature point extracting unit 110 with the moving picture of the image storage system; It's about the method.

The present invention extracts a singular point of an image photographed by a photographing device provided in the transport means in order to operate a transport means such as a vehicle or a drone, and compares the singular point of the photographed image image stored in the storage system to determine the location of the photographing device. It is designed to be able to grasp, and by this, the location of the photographing device is to be quickly identified.

In addition, according to the present invention, there is an advantageous effect of comparing the captured image at the current point of time and the image at a normal time in real time to identify a specific matter.

1 is a block diagram of the present invention.
2 is an example of dividing the frame 100 of an image in order to extract a finger print in the present invention.
3 is an embodiment for tracking location information.
4A and 4B are examples of images stored in the image storage system 160.
5 is an example of a frame-by-frame fingerprint of a video 300 captured by a moving means.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. First of all, in adding reference numerals to elements of each drawing, it should be noted that the same elements are assigned the same numerals as possible, even if they are indicated on different drawings. In describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is a block diagram showing a location tracking system using similarity analysis of a mobile photographed image according to the present invention, wherein an image capture unit 100, a feature point extraction unit 110, an image analysis unit 120, and an image storage system 160 Includes.

The image capturing unit 100 may be a camera or the like capable of capturing various videos mounted on a moving means, and transmits the captured video to the feature point extracting unit 110.

The feature point extracting unit 110 extracts a video fingerprint, which is a video feature point, by analyzing video data included in the moving picture file.

Here, the video fingerprint refers to a kind of video unique ID element capable of uniquely identifying video data, and the feature point extracting unit 110 selectively separates only video data from an image file prior to extracting the video fingerprint. You can precede the process of doing it.

The feature point extracting unit 110 selectively separates a video stream from a video file to extract a video fingerprint, and decodes the separated video stream into an RGB format. Subsequently, sub-video feature points are extracted for each frame image constituting the decoded video data.

A process in which the feature point extracting unit 110 extracts sub-video feature points for each frame image is as follows. This refers to the paper by Job Oostveen, Ton Kalker, and Jaap Haitsma, "Feature Extraction and a Database Strategy for Video Fingerprinting" (Lecture Notes in Computer Science 2314 Springer 2002).

First, the feature point extraction unit 110 divides a frame image into blocks.

For example, the frame image is divided into n×m blocks by dividing the vertical axis by n and the horizontal axis by dividing by m. As another example, the frame image is divided into n blocks by dividing the vertical axis of the image into n equal parts, and the frame image is divided into m blocks by dividing the horizontal axis into m blocks and divided into n×m blocks.

In addition, the feature point extracting unit 110 calculates the average of the Y values of each image brightness data for each divided block unit, and then generates a sub-video fingerprint of a predetermined number of bits based on the difference between the average image brightness values between adjacent blocks.

[Equation 1] exemplifies a bit allocation function'B(r, c, p)' that can be used to generate a sub-video fingerprint for one frame image.

Here, B(r, c, p) is a function that allocates bits to blocks located at the rth and cth positions, respectively, based on the horizontal and vertical axes of the p-th frame image. In addition, F(r, c, p) denotes the average of the image brightness values of the r-th and c-th blocks, respectively, based on the horizontal and vertical axes of the p-th frame image.

In F(r, c, p) of [Equation 1], r has an integer value from 1 to n, and c has an integer value from 1 to m-1. Where a is a number less than 1 and is an appropriately selectable constant.

Meanwhile, the feature point extracting unit 110 may generate a sub-video fingerprint using a Cb or Cr value that is color difference data of each pixel in addition to the Y value that is the brightness data of the image.

After the feature point extracting unit 110 extracts the sub-video fingerprint for each frame image constituting the moving picture, the extracted sub-video fingerprint is grouped to complete the generation of the video fingerprint for the moving picture file.

In the present invention, any algorithm capable of extracting a video fingerprint other than the method described in the above paper by Narrow Ustbin can be applied.

In this way, a video finger print for a video file is generated and stored in the image storage system 160, and when the video file is stored, location information for each frame of the video file is also stored.

When a photographing means mounted on a vehicle or a moving means such as a drone newly photographs an image, the image of the photographed image is transmitted to the feature point extracting unit 110 to generate a finger print, and the generated finger print is the image analysis unit 120 And compare it with the fingerprint of the video file in the storage system.

As a result of determining whether the similarity between the finger prints is greater than or equal to the threshold value by comparing the video fingerprint generated in the captured image of the newly captured image with the fingerprint stored in the storage system, the location of the newly captured image is It is determined that it matches the photographed location of the image stored in the previous storage system 160.

In this way, by comparing the image captured by the photographing means of the transport means and the previous image of the storage system in which the location information is recorded, the position of the image photographing means can be grasped in real time to determine the position of the transport means. Vehicles or drones can be controlled.

2 is an example of segmenting a block during a process of extracting a feature point of an intelligent movement monitoring system using similarity analysis of a mobile photographed image according to the present invention. In Fig. 2, it is divided into 9 blocks.

The feature point extracting unit 110 may use a color histogram graph to extract a feature point based on a color. This can be done using the contents of Wei-Lun Chao's thesis "The Core of Video Fingerprinting Example of Feature Extraction".

In addition, the feature point extraction unit 110 may use an object recognition-based feature point extraction algorithm to extract a feature point based on object movement. It is desirable to use Huitao Luo's Algorithms for Video Object Detection and Segmentation with Application to Content-Based Multimedia Systems.

Meanwhile, the feature point extraction unit 110 transmits the unique values of each frame from which the sub-video feature point is extracted to the image analysis unit 120. The image analysis unit 120 may search from a specific point in time according to a transmitted parameter or a transmission factor.

It is preferable that the image storage system 160 indexes an elapsed photographing time point, frame, and GPS coordinate.

It is preferable that the image analysis unit 120 performs a function of searching the image storage system 160 by limiting to a preset period. It is possible to increase the search speed by limiting to a search section preset by an administrator without searching for all data previously stored in the image storage system 160.

As shown in FIG. 3, an image photographed on September 10 through a moving means 400 such as a drone, and an image having a fingerprint of a pre-stored image above a threshold value are searched, and through the location information of the pre-stored image. You can find out the location of the means of transportation taken on September 10th.

Comparing the target image (September 10 image) currently being photographed in real time, not the image itself, but the feature points (finger print) extracted for each frame of the image by the feature point extraction unit 110 through bit operation. Therefore, the calculation speed is fast. Therefore, real-time comparison is also possible, and the image analysis unit 120 performs this comparison.

In FIG. 3, as a result of mutual comparison of the feature points from point A to point D of the image, the similarity is greater than or equal to a threshold value, and the location of the photographing means taken on September 10 can be identified through the location information of the previously stored image.

4A and 4B, images 200 and 210 including a plurality of frames 201, 202,..; 211, 212. ....) are stored in the image storage system. _{Finger print (P 211} , P ₂₁₂ , P ₂₁₃ , P ₂₁₄ , P ₂₁₅ , P ₂₁₆ , P ₂₁₇ , P ₂₁₈ ) for each frame of the saved image (211, 212, 213, 214, 215, 216, 217, 218) Has been created.

As shown in FIG. 5, the moving means captures an image 300 in real time, and the captured image 300 is frame-by-frame (301, 302, 303, 304, 305, 306, 307, 308) A finger print (P ₃₀₁ , P ₃₀₂ , P ₃₀₃ , P ₃₀₄ , P ₃₀₅ , P ₃₀₆ , P ₃₀₇ , P ₃₀₈ ) is generated, and the image analysis unit 160 performs a finger print of the captured image 300. It searches for a matching frame among the previously stored images.

_{For example, the finger prints (P 213,} P ₂₁₄ , P ₂₁₅ , P ₂₁₆ , P ₂₁₇ ) of the frames (213, 214, 215, 216, 217) of the pre-stored image 210 are the frames of the image 300 ( 302, 303, 304, 305, 306) of the finger print (P ₃₀₂ , P ₃₀₃ , P ₃₀₄ , P ₃₀₅ , P _306, P ₃₀₇ ) and above the threshold value, The location is determined to be the same as the location of the frames 213, 214, 215, 216, 217 of the image 210.

_{In addition, when the similarity between the finger print values (P 307} , P ₂₁₈ ) of the frame 307 of the image 300 and the frame 218 of the image 210 is less than or equal to the threshold (eg, less than 90%), the image The analysis unit searches again for an image having a similarity with a fingerprint for each frame of the image 300 photographed by the moving means equal to or greater than a threshold value.

As described above, in the present invention, feature points are extracted by frame-by-frame finger prints, and after storing the feature points in the storage system including frame-by-frame location information, the feature points of the images taken by the moving means are compared to match the video of the storage system. It is possible to determine the location of the current vehicle through the location information of the frame.

The present invention described above is not limited by the above-described embodiments and the accompanying drawings, and various substitutions, additions, and changes are possible within the scope of the technical spirit of the present invention. It will be obvious to those of ordinary skill.

100: image capturing unit 110: feature point extraction unit
120: image analysis unit 130: manager censorship unit
160: video storage system

Claims (6)

An image storage system 160 in which finger prints and location information grouped for each frame of a moving picture including a plurality of frames are indexed and stored;
A moving means 400 for photographing a moving picture including a plurality of frames;
A feature point extracting unit 110 for generating a fingerprint grouped for each frame of the moving picture taken by the moving means; And
Including; an image analysis unit 120 comparing the video of the moving means from which the feature points are extracted with the video of the image storage system,
The image analysis unit compares the feature points of the moving picture of the moving means and the feature points of the moving picture of the storage system to check whether the similarity is greater than or equal to a threshold value, finds a moving picture of the storage system corresponding to the moving picture of the moving means, and A position measurement system using image similarity analysis, characterized in that the position information is recognized as position information of a moving means. The method of claim 1,
The feature point extracting unit decodes the video stream of the moving picture into an RGB format or a black-and-white video format, divides the decoded frame image into units of a plurality of blocks, and calculates an average value of image brightness data of each pixel for each of the divided blocks. After that, a fingerprint grouped for each frame is generated by extracting a feature point based on the difference in the average value of the image brightness data between adjacent blocks,
The image analysis unit compares a feature point extracted as a fingerprint grouped for each frame image of the image of the moving means and an indexed feature point of an image pre-stored in the image storage system to correspond to a plurality of frames in a predetermined section. A position measurement system using image similarity analysis, characterized in that: finding a pre-stored image including location information having a similarity of greater than or equal to a threshold value, and determining the location information of the pre-stored image as the location of the moving means. The method of claim 1,
The storage system 160 includes a plurality of pre-stored images 200 and 210 including a finger print and location information for each frame,
The image analysis unit 120 transmits the image 300 captured by the moving means to the image analysis unit 120 in real time,
_{The image analysis unit 120 uses a finger print (P 301} , P ₃₀₂ , P ₃₀₃ , P) for each frame (301, 302, 303, 304, 305, 306, 307, 308) of the image 300 captured by the moving means. ₃₀₄ , P ₃₀₅ , P ₃₀₆ , P ₃₀₇ , P ₃₀₈ ) and each frame (211, 212, 213, 214, 215, 216, 217, 218) of the image (200, 210) previously stored in the storage system 160 Finger prints (P ₂₁₁ , P ₂₁₂ , P ₂₁₃ , P ₂₁₄ , P ₂₁₅ , P ₂₁₆ , P ₂₁₇ , P ₂₁₈ ) are compared to find a frame with a similarity greater than or equal to a threshold, and the frame location information of the pre-stored image is transferred to the location information of the vehicle Position measurement system using the similarity analysis of the image, characterized in that the determination as. Indexing finger prints and location information grouped for each frame of a moving picture including a plurality of frames and storing them in the image storage system 160;
The moving means 400 photographing a moving picture including a plurality of frames;
Generating a fingerprint grouped for each frame of the moving picture captured by the moving means by transmitting the moving means 400 to the feature point extracting unit 110; And
Comprising, in the image analysis unit 120, comparing the moving picture of the moving means from which the feature points are extracted by the feature point extracting unit 110 with the moving picture of the image storage system,
The image analysis unit compares the feature points of the moving picture of the moving means and the feature points of the moving picture of the storage system to check whether the similarity is greater than or equal to a threshold value, finds a moving picture of the storage system corresponding to the moving picture of the moving means, and Position measurement method using image similarity analysis, characterized in that the position information is recognized as position information of the moving means. The method of claim 4,
The feature point extracting unit decodes the video stream of the moving picture into an RGB format or a black-and-white video format, divides the decoded frame image into units of a plurality of blocks, and calculates an average value of image brightness data of each pixel for each of the divided blocks. After that, a fingerprint grouped for each frame is generated by extracting a feature point based on the difference in the average value of the image brightness data between adjacent blocks,
The image analysis unit compares a feature point extracted as a fingerprint grouped for each frame image of the image of the moving means and an indexed feature point of an image pre-stored in the image storage system to correspond to a plurality of frames in a predetermined section. A method for measuring a location using similarity analysis of an image, characterized in that: finding a pre-stored image including location information having a similarity of greater than or equal to a threshold value, and determining the location information of the pre-stored image as the location of the moving means. The method of claim 5,
The storage system 160 includes a plurality of pre-stored images 200 and 210 including a finger print and location information for each frame,
The image analysis unit 120 transmits the image 300 captured by the moving means to the image analysis unit 120 in real time,
_{The image analysis unit 120 uses a finger print (P 301} , P ₃₀₂ , P ₃₀₃ , P) for each frame (301, 302, 303, 304, 305, 306, 307, 308) of the image 300 captured by the moving means. ₃₀₄ , P ₃₀₅ , P ₃₀₆ , P ₃₀₇ , P ₃₀₈ ) and each frame (211, 212, 213, 214, 215, 216, 217, 218) of the image (200, 210) previously stored in the storage system 160 Finger prints (P ₂₁₁ , P ₂₁₂ , P ₂₁₃ , P ₂₁₄ , P ₂₁₅ , P ₂₁₆ , P ₂₁₇ , P ₂₁₈ ) are compared to find a frame with a similarity greater than or equal to a threshold, and the frame location information of the pre-stored image is transferred to the location information of the vehicle A method of measuring a location using similarity analysis of an image, characterized in that it is determined as.

Images