KR101074850B1 - Serch system of images - Google Patents

Abstract

The present invention relates to an image retrieval system, and more particularly, to quickly and accurately extract a retrieval image including a retrieval object to be retrieved from an image stored in a storage means through an image retrieval engine, thereby shortening retrieval time. It is an image retrieval system that can reduce the search manpower. In particular, the image retrieval engine is an image retrieval system with excellent retrieval quality, reliability, and retrieval speed. It is about.
Image retrieval system according to the present invention for achieving the above object is a storage means for storing the image; An image standardization module for reconstructing the image frames of the reference image and the target image as the reference and target of the pixel by pixel unit, and unifying each pixel into a representative color to obtain a standard image, and numerical data of each pixel color of the standard image. And comparing the image data conversion module for acquiring the index image with the index data acquired by the image data conversion module and numerical data between matching pixels of the reference image and the target image, and determining the similarity between the reference image and the target image. An operation server including an image search engine including an image comparison module; And an input unit for receiving a user's manipulation signal and a monitor on which an image stored in the storage unit is displayed. The monitor reproduces a search target image of a video transmitted through the operation server on a playback screen, and the search images remain. In the image retrieval system comprising a; panoramic display on the screen, if any one of the search image displayed on the remaining screen is selected, the search means for playing the image after the selected search image on the playback screen; Among the images stored in the storage means, the similarity of the comparison result of the image comparison module deviates from the reference value, and extracts a target image determined to include a search object and a video including the target image as a search image. Characterized in that the transmission to the search means.

Description

Image search system {Serch system of images}

The present invention relates to an image retrieval system, and more particularly, to quickly and accurately extract a retrieval image including a retrieval object to be retrieved from an image stored in a storage means through an image retrieval engine, thereby shortening retrieval time. It is an image retrieval system that can reduce the search manpower. In particular, the image retrieval engine is an image retrieval system with excellent retrieval quality, reliability, and retrieval speed. It is about.

Almost 20,000 CCTV infrastructures have already been installed nationwide for the purpose of crime prevention, traffic checks, accident checks, suspects and arrests, and video footage taken by CCTVs is effective in detecting criminals and identifying responsible materials for accidents. Despite the controversy over privacy, the demand for more CCTV installations is strong.

Although the video taken by CCTV has a great effect in criminal arrest and the identification of responsibility for accident occurrence, it is not easy to find out the part where the criminal was shot or the part where the situation at the time of the accident was recorded.

The CCTV is fixedly installed in a certain area, and the area is continuously photographed, and the video recorded by the CCTV is recorded and stored as a file.

Therefore, if you do not know the exact time and place of a crime or accident, you should play back the entire recorded video file to see if the situation at the time of the crime or accident was shot.

It takes 24 hours to play and check the entire video file of 24 hours a day, and it takes 6 hours or more even if it is checked at 4 times speed playback.

And since the video is usually composed of about 30 frames per second (still image), even if you try to search for a minute image, the search speed is significantly reduced because you have to search while watching 1800 (30 * 60) video frames.

However, many CCTVs are installed in areas where there is little change in the background such as alleys or parking lots, and accordingly, there are not many parts that need to be searched because people or objects emerge from the entire video filmed by CCTV.

In other words, the object to be searched in the entire 24 hours of video files is captured at most, only 1-2 hours.

Therefore, searching the entire video file is an inefficient and unnecessary waste of manpower.

In order to solve such a problem, Korean Patent No. 0452097 "Image data storage method using change value extraction of image data" and Patent No. 0873445 "Image difference recognition method using image difference recognition system and image system", etc. The present invention relates to an image retrieval system that reduces monitoring and retrieval time by extracting and displaying only a portion of an image which is changed.

In the image retrieval system, an image retrieval engine for retrieving and retrieving only still images, which are the targets of retrieval, among the continuous still images constituting the moving image is most important.

How fast and how accurately a search engine extracts a still image that includes an object to be searched can be said to determine the overall quality of an image search system.

The image search engine searches and extracts target images that can be searched by comparing the reference images and the target images to be searched with each other to determine similarity between the two images.

In the prior art, the image search engines of the two patents divide image frames of the reference image and the target image by n × m pixel units, and brightness of two pixels at the same position in the image frame of the reference image and the image frame of the target image. That is, a method of searching for similarity by using the same number of pixels among all pixels of an image frame by comparing gray scales) or colors with each other is used.

The conventional search engine has a problem in that the comparison time is long because the brightness or color of two pixels in the same position are directly compared between the reference image and the target image.

In addition, the conventional search engine assumes that the background image of the reference image and the target image are the same. That is, the main purpose is to search for whether or not a new object has appeared in the target image.

In other words, in the prior art, the camera selects a reference image and a target image from among video frames of a video photographed in an elevator, a parking lot, and the like at a fixed position under a fixed angle and focus, so that the camera searches and compares each other. ,

In addition to the same focus and resolution of the reference image and the target image, the background image has the same position, color, size, and the like.

Therefore, in the related art, even if the same search object image exists in the reference image and the target image, the background image is different in size, in different colors, in different positions, or in different colors. Is different,

As a result of the comparison of search engines, the similarity between the reference image and the target image is low, and there is no change of the image because there is no object appearance in the image. As a result of the low similarity even though the object is included, it is determined that the object is not included in the target image, and thus a problem arises that the target image cannot be extracted.

In image search, an object to be searched (ie, searched for) is specified. When the search object is not clearly specified, a video including an image similar to the search object and a still image constituting the video are extracted. It is necessary to be able to clearly specify an object, and to be able to clearly specify an object, a function to search in various ways needs to be provided.

For example, depending on the type of search object (a person, an object, etc.), only a moving picture or a still image including similar objects is extracted, or only a change of a specific area is searched and extracted from the entire shooting area of the video, or a specific pattern is included. It is necessary to extract only the moving and still images.

The present invention has been made to solve the problem of the image retrieval system according to the prior art as described above, significantly reducing the search time and manpower for all the images stored in the storage means, in particular the image search engine target image The purpose of the present invention is to provide a video retrieval system that shortens the time for searching for similarity and improves the accuracy and reliability of image retrieval.

Image retrieval system according to the present invention for achieving the above object is

Storage means for storing an image;

An image standardization module for reconstructing the image frames of the reference image and the target image, which are the criteria and the target of the search, in units of pixels, and unifying each pixel into a representative color to obtain a standard image;

An image data conversion module for converting each pixel color of the standard image into numerical data to obtain an index image;

And an image search engine including an image comparison module for determining similarity between the reference image and the target image by comparing numerical data between matching pixels of the reference image and the target image as index images acquired by the image data conversion module. server;

Including an input unit for receiving a user's operation signal, and a monitor to display the image stored in the storage means,

The monitor reproduces the search target image of the video transmitted through the operation server on the playback screen, and the search images are displayed on the residual screen in a panoramic manner, and when any one of the search images displayed on the residual screen is selected, the playback screen is selected. In the image retrieval system comprising a; search means for reproducing the image after the selected retrieval image,

The operation server searches for a video including the target image and the target image determined to include a search object among the images stored in the storage means, when the similarity of the comparison result of the image comparison module is out of a reference value. Extracting the image and transmitting the image to the search means.

And the image search engine

In the standard image, one pixel corresponding to a specific object is selected as a reference pixel, a component pixel constituting the specific object is selected from among the neighboring pixels by comparing the color of the neighboring pixel adjacent to the selected reference pixel. The method may further include an object region extraction module that extracts a region of a specific object from a standard image by comparing colors of neighboring pixels with the constituent pixels.

The image data conversion module generates an index image by using a region for each of the specific objects (eg, a person and a thing) extracted by the object region extraction module as a new target image.

The image comparison module compares a reference image with a target image for a specific object extracted by the image data conversion module to determine similarity,

The operation server extracts a target image for the specific object and a video including the target image as a search image and transmits it to the search means.

The reference image and the target image, which are the reference and the target of the search, are set as an image of a specific region selected from the entire photographing regions of the images stored in the storage means.

The operation server extracts a target image having a change in an image and a video including the target image as a search image from the reference image set in a specific region, and transmits it to the search means. ,

The search means further includes a registration unit for registering the search reference image, and a transmission and reception unit for transmitting the search reference image registered in the registration unit to the operation server,

The image search engine of the operation server extracts a search target image including a search object of the search reference image received from the search target image of the storage means as a primary search image and transmits the search target image to the search means,

When the search means is transmitted to select a specific first search image among the displayed first search images and registers as a new search reference image in the register, the image search engine of the operation server is transmitted from the search target image of the storage means. And extracting a search target image including the search object of the received new search reference image as a secondary search image and transmitting the extracted search image to the search means.

The image search engine

Comprising a standardized image obtained by the image standardization module and a matching module of the reference standard image and the target standard image, and further comprising a transparent module for obtaining a transparent processed image from the target standard image by transparently processing the matching pixel,

The image data conversion module

A data converter which converts the color of each pixel of the standard image into numerical data to obtain an initial index image;

And a row elimination unit for removing the row and column of the region that does not correspond to the search object based on the numerical data of each pixel in the initial index image to obtain the adjusted index image.

Image retrieval system according to the present invention having the configuration as described above can be quickly and precisely search the image significantly shortened the search work time, thereby reducing the search manpower.

In addition, the image search engine has a high image retrieval speed, high accuracy and reliability, and upgrades the overall efficiency of the image retrieval system.

1 is an explanatory diagram schematically illustrating an image retrieval technique according to the prior art and the present invention;
2 is a block diagram of an image retrieval system according to the present invention;
3 is an example of display of a retrieval means monitor;
4 is a block diagram of an image search engine according to the present invention;
FIG. 5 is a diagram for explaining a method of generating and acquiring a standard image through standardization processing in an image search engine; FIG.
6A and 6B illustrate a method for generating and acquiring an initial index image and a method for generating and obtaining an adjusted index image, respectively, in an image search engine.
FIG. 7 is a diagram for describing a method of generating and obtaining an object standard image by extracting an area of a specific object from all standard images in an image search engine; FIG.
8 is a view for explaining a method of generating and obtaining a transparent processed image by transparently processing a target image with respect to a reference image in an image search engine, and generating and obtaining an object standard object from the transparent processed image.
FIG. 9 illustrates an example of standard images of a reference image and a target image to be compared in an image frame of the reference image and the target image in an image search engine; FIG.

Hereinafter, an image search system and an image search engine according to the present invention will be described in detail with reference to FIGS. 1 to 9.

For reference, a reference image is referred to as a reference image among two images that are compared with each other in an image search engine, and a target image is referred to as a target image.

In order to clearly identify the search object, the two images that are compared during monitoring are called 'monitoring reference image' and 'monitoring image', and the two images that are compared during the search are 'searching reference image' and 'searching image' It is called.

The target image extracted by the search operation among the target images extracted as a result of the comparison is called a 'search image'.

First, referring to FIG. 1, a method of searching for an image using a conventional technique and a method of searching for an image using a technique of the present invention are illustrated.

As can be seen in Figure 1, the search using the present invention can significantly reduce the search time, it is possible to quickly search with less manpower.

1 is a case in which the search object is extracted from the video after the search object is confirmed through the monitoring and the image of the portion where the search object is photographed through the search is not clearly determined. The situation is assumed to be a suspect.

In the above assumption, the prior art and the search method of the present invention are as follows.

The prior art collects video files captured by CCTVs installed in a crime-prone area, and plays a video one by one to find a suspect.

When the suspect is found, the rest of the videos are played back to confirm that the suspect actually committed the crime.

The present invention collects video files captured by CCTVs installed in a crime-prone area and stores them in a storage means, and then monitors the stored videos using an image search engine of an operation server.

Then, the images of the time when the preliminary suspect appears (ie, there is a change in the image) for each moving image are displayed on the monitor in a panoramic manner.

When the searching agent finds the suspect in the images displayed on the monitor, cuts out the image of the suspect (that is, the search object) from the displayed image and registers it as a search reference image in the register, the image search engine of the operation server again stores the means. The video containing the suspect is extracted from the videos stored in the video (ie, the search video).

The extracted images including the suspect are displayed on the monitor in a panoramic manner, and the search agent reproduces the displayed images to check whether the suspect is likely to commit a real crime.

FIG. 1 schematically illustrates a process of searching for a still image of a video and a video including a person similar to the person to be searched when the search object is a person (ie, a suspect) in the CCTV video.

Even when the search object is a specific object (car, motorcycle, etc.), the moving image and the still image including the object to be searched can be extracted in a similar manner as described above.

When the search object is a specific place instead of a person or an object, a specific region may be selected from the entire photographing region of the video, and whether the selected region is changed may be extracted to extract a moving image and a still image.

In some cases, the search object may be a specific pattern (eg, color, reference image information, etc.), and when searching for a specific pattern, a moving image and a still image including the specific pattern may be extracted in a similar manner to the above.

As shown in FIG. 2, the video surveillance system according to the present invention includes an operation server 100 including an image storage means 300, a search means 200, and a search engine 120.

For reference, the image stored in the storage means for retrieval may be a video mainly composed of still images captured by a CCTV camera, but also includes a still image captured by a general camera or a still image scanned from a photograph. . Therefore, what is described below as "video" rather than "video" will also mean most videos, but does not exclude still images.

Images are stored in the storage means 300. The image includes not only an image photographed by a CCTV camera CM, but also an image photographed by a general camera, an image scanned by a photograph, and the like.

The video taken by the CCTV camera or the general camera may be encoded and directly stored in real time, or the pre-photographed video may be stored later by a download method.

The storage means 300 may store not only a video but also a target image extracted for each video, and a reference image serving as a reference for extracting the target image.

Also, the video, the target video and the reference video for the video may be stored in a specific place by being distinguished from other videos in the storage means 300. In other words, a database may be used as the storage means 300.

The search means 200 is a device for a user, that is, a search agent to retrieve a video of the storage means 300 to perform a search operation,

And an input unit 230 for receiving a user's manipulation signal, and a monitor 250 for providing a screen for displaying a user's manipulation and a screen for displaying a video.

In addition, the CPU 210 which controls the search means as a whole, the memory 220 storing the extracted video, the extracted target video, the search video, the registration unit 260 to register the search reference image, and the search means. And the like.

The registration unit 260 registers a reference image directly input by the user.

Both the monitoring reference image and the search reference image can be registered in the register.

The search object may include a moving search object such as a suspect or a car, and a fixed search object at a specific location, such as a place where an item is stolen or a contact incident occurs.

In the case of a moving search object, the search object is determined in advance, and when an image including the search object is obtained, the acquired image is registered as the search reference image in the registration unit 260 so that a search operation can be performed immediately without monitoring. Do.

However, the accuracy of the search may be lowered because the search object of the previously acquired image and the search object of the target image stored in the storage means are different in color or different in the photographing direction.

In this case, the object has a shape similar to that of the search object of the search reference image by lowering the search extraction criterion of the image search engine (that is, the similarity of the two images being compared) or by limiting the type of the search object (eg, a person, an animal, a car, etc.). All of the search target images including or the search target images including the same object as the search object are extracted as the first search image, and the user directly extracts the first search images and the search reference image displayed on the monitor. After checking, one of the primary search images including the search object is selected and registered as a new search reference image in the registration unit 260. Then, the search engine 120 may search the comparison image with the search target image of the storage means 300 by using the newly registered search reference image to extract a second search image including the search object to increase the search extraction accuracy.

In the case of a moving search object, the search object is not determined in advance, or in the case of a fixed search object for a specific place, a search object is determined by monitoring a target image stored in the storage means 300, or a specific place is The search object is determined by cutting the reference image to include a specific place in the captured target image.

In the former case, the retrieval operation is performed after the search object is confirmed by line monitoring.In the latter case, the image change (object appearance) in the region outside the region cut out from the target image of the video is cut out within the cut out region. It can be said that the monitoring and retrieval work are performed at the same time by extracting the retrieval image with image change in.

In addition, when the occurrence time zone of the event for which the retrieval operation is performed is known, a time zone is set as a condition when the target image is extracted, so that a faster retrieval is possible.

The search means 200 communicates with the operation server 100 by wire or wirelessly, and transmits the operation signal input by the user and data about the monitoring reference image, search reference image, etc. set by the user to the operation server 100. In addition, the operation server 100 receives the extracted video and the extracted target video data.

3 shows an example of a display state of the monitor 250 of the searching means 200.

Referring to FIG. 3, a menu for selecting whether to perform a monitoring operation or a search operation is displayed on the upper and both sides of the monitor 250, and a menu for an operation such as setting or registering a monitoring or search reference image is displayed. In the upper part, there is a playback screen in which a video is played, and in the lower part, a target image extracted by a search engine (for example, a target image at a time when there is a change in the image extracted by a monitoring task, and a search object extracted by a search task are included. There is a residual screen in which a search image) is displayed as a target image in a panoramic manner.

When the user selects a specific target image from the remaining screen by using the input unit 230 such as a mouse, the selected target image and the subsequent video are played through the playback screen.

The operation server 100 connects the retrieval means 200 and the storage means 300 to read an image of the storage means 300 and transmits the image to the retrieval means 200.

In addition, the operation server 100 receives a video from a CCTV camera, a general camera, a video recording film and the like and stores in the storage means 300,

The image search engine 120 searches for a video stored in the storage means to extract the target image, and transmits the extracted target image to the search means 200.

The operation server 100 as shown in Figure 2

An input interface 130 for receiving a video from a camera CM, etc.,

A communication unit 140 including a transmitting unit 141 and a receiving unit 143 in data communication with the searching unit 200;

An image search engine 120 for searching an image of the storage means 300 and extracting a target image;

It includes a control unit 110 for controlling the operation server 100 as a whole.

An encoder for encoding and compressing an input video may be connected to the input interface 130, and a decoder for decoding a video and a target image transmitted to the search means 200 may be connected to the transmission unit 141.

The controller 110 stores the video input from the input interface 130 in the storage means, processes the operation signal input from the search means 200 through the receiver 143, and stores the storage means 300. Read the video stored in the transmission to the monitoring means 200 through the search engine 120 and the transmission unit 141, and the search means 200 through the transmission unit 141 the target image extracted by the search engine 120 The overall operation and control of the operation server 100, such as to deliver.

The image search engine 120 compares a video transmitted from the storage means 300 or a video input in real time through the control unit 110 with a preset reference video to the similarity degree of each target image constituting the video. Determine and extract the target images whose similarity deviates from the reference value and transmit the extracted target images to the search means 200 through the control unit 110 and the transmission unit 141.

For reference, the reference image may be directly set by the user through the search means 200, or the search engine 120 may be set by itself.

In the reference image, the first target image among the target images constituting the video is set as the reference image, or the target image compared with the reference image is set as the reference image for the next target image to be compared, or more than a predetermined time. The reference image may be set in various ways according to the monitoring and retrieval purposes, such as a target image having no change in the image being set as the reference image.

Hereinafter, the image search engine 120 will be described in more detail with reference to FIGS. 4 to 9.

For reference, the image includes a still image and a moving image consisting of still images. The image to be searched through the image search engine according to the present invention is a video taken by a digital camera as well as a video taken by a CCTV. Or a still image, a still image stored in a codec of various formats, or a still image or a video file of a still image, a picture or a movie.

That is, all of the images that can be filed and stored in the memory may be the reference image and the target image of the comparison search through the search engine of the present invention.

The reference image is referred to as the reference image among the two images that are compared with each other in the image search engine, and the target image is referred to as the target image.

In the following description, 'image' generally refers to any one of the continuous still images constituting a video, but may also mean a video that is not a still image, and an individual still image that is not one of the continuous still images. It may be.

As shown in FIG. 4, the image search engine 120 according to the present invention includes a memory 20, a microcomputer 10, an image standardization module 30, an image data conversion module 40, an image comparison module 70, and an object. It comprises a region extraction module 50, the transparency module 60.

In the memory 20, image frames to be searched are stored, and standard images, index images, object-specific images, transparent processed images, and the like acquired from each module 30, 40, 50, and 60 are stored.

The microcomputer 10 connects the memory 20 with the modules 30, 40, 50, 60, and 70 and controls the search engine as a whole.

The image standardization module 30 generates and acquires a standard image by standardizing image frames stored in the memory 20, and stores the image in the memory 20.

In this case, the standardization means to match (ie, standardize) various image elements such as size and resolution in order to compare and search the reference image and the target image.

In addition, standardization includes standardization of an entire still image and standardization of each object included in the still image.

The image standardization module 30

A pixel constitution unit 31 for reconstructing the image frame in units of pixels of size n × m,

And a color providing unit 33 for generating and obtaining a standard image by uniting each pixel of the reconstructed image frame into a representative color.

The image frame standardized by the image standardization module 30 may include an object standard image of a specific object extracted by the object region extraction module 50 as well as an initial image frame which is not processed at all.

The standardization of the object standard image extracted for a specific object in a still image is to match the size of the object standard images to be compared.

The total number of pixels of the image frame reconstructed by the pixel configuration unit 31 will be appropriately selected according to the size of the image frame. Larger numbers of pixels will allow more accurate comparisons, but will require more comparison time.

The color imparting unit 33 unifies each pixel into a representative color. In other words, when the color of the pixel is mixed, it is unified with the color occupying the largest area.

In some cases, the ratio of the different colors may be the same. In this case, the colors may be given priority or the similar colors to the surrounding pixels become representative colors.

In addition, the color providing unit 33 unitizes each pixel into a representative color of the color, and also integrates each pixel into one of two colors, a white series and a black series.

That is, the image of each object included in the image frame reconstructed by pixel unit is shaded to generate and obtain a shaded standard image.

Since the shaded standard image is displayed in a white series or a black series, each pixel is used to determine the similarity of the overall shape of each image except for the color when compared to the reference image and the target image.

5 illustrates an example of a standard image generated and acquired by the image standardization module 30.

The image data conversion module 40 converts the color data of each pixel of the standard image into numerical data, thereby reducing the comparison time when the reference image and the target image are compared.

In other words, when comparing the image data of the image frame as it is, or compared to other post-deformation processing requires a lot of constraints and processing time, the color data of each pixel is converted into the most efficient and fast numerical data for mutual comparison.

When the image data conversion module 40 converts the image data of the image into numerical data to generate and obtain an index image, it is preferable that the additional information on the converted index image is also separately indexed and acquired.

The matched and acquired information may include a play position in the entire video, a total play time of the video, a video type, a codec type, and the like.

The image data generated by the image data conversion module 40 converting the color of each pixel of the standard image into numerical data is an index image.

The image data conversion module 40 includes a data converter 41 and a matrix remover 43.

The data converter 41 generates and acquires an initial index image by converting the color of each pixel of the standard image into predetermined numerical data.

For example, if numbers are assigned by arranging colors in a similar order from white to black, red can be converted into 1, yellow is 2, blue is 3, white is 0, black is 4, green is 5, and FIG. 6A An example of converting a standard image into an initial index image based on these criteria is shown.

The above example of converting colors and numbers is an example for easy understanding, and different numbers only mean that colors are different.

The color of each pixel of the standard image is not limited to several colors, and there is a similar degree between the colors.

Therefore, when converting the color of each pixel to numerical data, it is preferable that the difference of the numerical data can represent the difference of the color and at the same time express the degree of the difference.

Thus, in the present invention, each color is expressed as a ratio of three primary colors of the color.

All colors can be implemented as a mixture of three primary colors: red (more precisely magenta), blue (more precisely cyan), and yellow (yellow).

If the primary colors have a specific gravity of 100 (0 to 99), and the order of each primary color in the order of red, blue, and yellow, the numbers 990000 converted from the numerical values are red, 009900 is blue, 000099 is yellow, 999999 means black and 000000 means white.

The matrix removing unit 43 is a region that does not correspond to the search object in the initial index image in order to prevent the similarity which is a result of the comparison because the position of the search object to be searched is different from the reference image and the target image to be compared. Remove and obtain the adjustment index image.

FIG. 6B shows the same adjustment index image by removing pixels of rows and columns of regions not corresponding to the search object from two initial index images in which the search object (pixels represented by numbers 4 and 5) shown in FIG. 6A differs. An example of generating and acquiring is shown.

Referring to the two initial index images on the left in FIG. 6B, the numerical data of the search object is the same, but the positions are different. Thus, when comparing two initial index images, the similarity is low because the values of the same preset are different. However, when the two initial index images are converted into the adjusted index images, they are the same as shown in the center, and the similarity is high (matched).

The row removing unit 43 naturally removes the rows and columns that do not have any values (4 and 5 in the drawing) corresponding to the search object in the initial index image. When the number of pixels present is significantly less than the total number of pixels in that row and column, the rows and columns can be removed.

The object region extraction module 50 separates the images included in the entire standard image for each object.

The standard image obtained by standardizing the image frame in the image standardization module 30 may be referred to as an entire standard image, and the standard image for each object extracted by the object region extraction module 50 may be defined as an object standard image.

The object region extraction module is configured to select one pixel corresponding to a specific object in the entire standard image as a reference pixel, and to configure the specific object among the neighboring pixels by comparing the selected reference pixel with a color of an adjacent peripheral pixel. By selecting a pixel and comparing the color of the selected neighboring pixel with the neighboring pixel again, an object standard image is generated and obtained by extracting an area for a specific object from the entire standard image.

7 schematically illustrates a method of extracting an object standard image from the entire standard image.

In general, it proceeds sequentially in the direction of the row or the column in the first row and the first column to select whether each pixel corresponds to a reference pixel or a constituent pixel.

As we proceed from row 1 to column 1, a pixel is selected as the reference pixel for a particular object. Whether it becomes a reference pixel is determined by the color of the pixel.

When the reference pixel is selected, it is determined whether the eight neighboring pixels of the reference pixel correspond to a specific object and the component pixel is selected, and the peripheral component pixel is selected again using the selected peripheral component pixel as the reference pixel. The selection of the constituent pixel is determined by comparing with the color of the reference pixel.

Generally, a plurality of objects will be extracted from the entire standard image, and only the object having a certain number of pixels or more is extracted as the object standard image. Objects below a certain number may be video errors and may be ignored because they have little influence on the similarity judgment.

After extracting a plurality of objects from the entire standard image, the object standard image for each object is extracted by removing rows and columns that do not correspond to each object in the same manner as the matrix removing unit.

And the object standard image connects the rows and columns to which the constituent pixels arranged externally belong to form a standard image of a rectangular shape, and the pixels that are placed inside the external pixels but are not constituent pixels also constitute the object standard image. Extract in pixels.

In addition, the object region extraction module 50 has been described to extract the object standard image from the entire standard image that is assigned a representative color to each pixel,

The object index image may be extracted from the entire index image to which each pixel is assigned numerical data.

The method is the same as the method using a standard image, except that the color data is replaced with numerical data.

When the background image of the reference image and the target image to be compared are the same, the transparency module 60 transparently processes the same background image in the target image to facilitate extraction and similarity determination of the object.

The transparency module 60 selects a reference standard image and a target standard image from among the standard images acquired by the image standardization module 30, compares the matched pixels of the selected two reference standard images with the target standard image, and matches each other. Transparent processing to obtain transparent processing image from the target standard image.

8 shows an example of a transparent processed image generated and acquired by transparently processing a target standard image and an object standard image extracted from the transparent processed image.

For reference, the transparency module 60 may also generate and acquire a transparent processed image using the index image as well as the standard image like the object region extraction module 50.

The image comparison module 70 selects one of the index images in which each pixel is converted into numerical data as a reference image, selects another as the target image, and then matches pixels of the two selected reference images with the target image. The similarity between the reference image and the target image is determined by comparing the numerical data therebetween.

For example, in the case where the index image is numerical data displayed by mixing the colors of each pixel as a mixture of three primary colors,

Extract numerical data from each pixel of the reference image and the target image,

Calculate the difference value of numerical data for each primary color (red, blue, yellow) at the same pixel (i.e. matched pixel) of the reference image and the target image, and calculate the average of the difference between the three primary color data of the corresponding pixel. Calculate the similarity of the pixels,

After the similarity is calculated for all the pixels, the similarity of the entire reference image and the target image is determined based on the average of the similarity.

FIG. 9 shows an example of a reference image and a target image for comparison in the center by normalizing, shading, and extracting regions of the image frame corresponding to the reference image on the left and the image frame corresponding to the target image on the right, respectively. Is shown in the form of a standard image.

The reference image and the target image to be compared include the entire top standard image normalized to the representative color, the second full standard image standardized to the shade, and the objects below it, as shown at left and right of the center. Object standard images extracted from the region are represented by representative colors and normalized by shading.

The object standard images of each object are readjusted in rows and columns so that pixels match one-to-one. In other words, after extracting the region for each object, the standardized process is used to reconstruct the extracted region in units of pixels and to unify the color of each pixel.

In the above, the configuration of the image retrieval system of the present invention has been described, but the following briefly summarizes the function of retrieving CCTV images in various ways by using the image retrieval system of the present invention.

First, a video including an object of an image similar to the search object image to be searched from the CCTV video to be searched and the corresponding still images are extracted.

At this time, the video search engine is responsible for the extraction of video and still images, and whether or not the object image of the reference image to be compared and the object image of the target image is similar is based on the similarity (accuracy) between the object images of the two images set by the search agent. Follow. For example, when the search agent sets the similarity to 80%, only the target images having more than 80% matching degree are extracted as a result of comparing the object image of the reference image with the object image of the target image.

If the object to be searched for in the searched CCTV videos is a person, only a video including a person similar to the person to be searched and the still images of the video is extracted.

If the object to be searched is a specific object (for example, a car or a motorcycle), the search extracts only a video including a similar object to the specific object and a still image of the video.

If the object to be searched is a specific place (for example, the place where theft accident occurred, the place where the car crash occurred, etc.), the area corresponding to the specific place in the videos is designated, and the change is detected by searching for the change of the image in the designated area. Extract the video and the still images of the video.

In the above, when the object to be found is a person or an object, the search system of the present invention does not target the entire still image including the corresponding object, but the object region extraction module of the image search engine is the corresponding object in the whole still image. The search speed is fast because it extracts the area for and creates it as an index image for comparison.

When the object to be searched is a specific place, that is, when the object is to be searched for a specific area from all still images, an index image is generated for the selected specific area without extracting an object such as a person or a thing from the specific area. The search speed is fast for comparison.

On the contrary, in the conventional search system, only a specific object such as a person or an object is extracted from the entire image and cannot be compared, and the search is slow because the coordinates are changed in the entire image and compared. You can't expect to search only for a specific area by specifying the area as a search object.

If it is difficult to specify the shape or location of the object to be searched and can be specified by a pattern, a video including a specific pattern (eg, color, reference image information, etc.) and still images of the corresponding video are extracted. do.

In the above description of the present invention has been described with respect to the image retrieval system having a specific configuration with reference to the accompanying drawings, the present invention can be variously modified and changed by those skilled in the art, such variations and modifications are within the scope of the present invention Should be interpreted as belonging.

10: microcomputer 20: memory
30: image standardization module 40: image data conversion module
50: object area extraction module 60: transparency module
70: image comparison module CM: Kemera
100: operation server 110: control unit
120: image search engine 150: alarm unit
200: monitoring means 210: CPU
230: input unit 250: monitor
260: Register 300: Video storage means

Claims (5)

Storage means for storing an image;

An image standardization module for reconstructing the image frames of the reference image and the target image, which are the criteria and the target of the search, in units of pixels, and unifying each pixel into a representative color to obtain a standard image;
An image data conversion module for converting each pixel color of the standard image into numerical data to obtain an index image;
And an image search engine including an image comparison module for determining similarity between the reference image and the target image by comparing numerical data between matching pixels of the reference image and the target image as index images acquired by the image data conversion module. server;

Including an input unit for receiving a user's operation signal, and a monitor to display the image stored in the storage means,
The monitor reproduces the search target image of the video transmitted through the operation server on the playback screen, and the search images are displayed on the residual screen in a panoramic manner, and when any one of the search images displayed on the residual screen is selected, the playback screen is selected. In the image retrieval system comprising a; search means for reproducing the image after the selected retrieval image,

The operation server searches for a video including the target image and the target image determined to include a search object among the images stored in the storage means, when the similarity of the comparison result of the image comparison module is out of a reference value. The image retrieval system, characterized in that the image is extracted and transmitted to the search means. The method of claim 1,
The image search engine
In the standard image, one pixel corresponding to a specific object is selected as a reference pixel, a component pixel constituting the specific object is selected from among the neighboring pixels by comparing the color of the neighboring pixel adjacent to the selected reference pixel. The method may further include an object region extraction module that extracts a region of a specific object from a standard image by comparing colors of neighboring pixels with the constituent pixels.

The image data conversion module generates an index image by using a region for each of the specific objects (eg, a person and a thing) extracted by the object region extraction module as a new target image.
The image comparison module compares a reference image with a target image for a specific object extracted by the image data conversion module to determine similarity,

The operation server extracts a target image of the specific object and a video including the target image for which the similarity is out of the reference value as a comparison result of the image comparison module and transmits the image to the search means. system. The method of claim 1,
The reference image and the target image, which are the reference and the target of the search, are set as an image of a specific region selected from the entire photographing regions of the images stored in the storage means.
The operation server extracts a target image having a change in an image and a video including the target image as a search image and transmits it to the search means in comparison with a reference image set as a specific region. Video retrieval system. The method according to any one of claims 1 to 3,
The search means further includes a registration unit for registering the search reference image, and a transmission and reception unit for transmitting the search reference image registered in the registration unit to the operation server,
The image search engine of the operation server extracts a search target image including a search object of the search reference image received from the search target image of the storage means as a primary search image and transmits the search target image to the search means,
When the search means is transmitted to select a specific first search image among the displayed first search images and registers as a new search reference image in the register, the image search engine of the operation server is transmitted from the search target image of the storage means. And a search target image including the search object of the received new search reference image as a second search image and transmitted to the search means. The method of claim 4, wherein
The image search engine
Comprising a standardized image obtained by the image standardization module and a matching module of the reference standard image and the target standard image, and further comprising a transparent module for obtaining a transparent processed image from the target standard image by transparently processing the matching pixel,

The image data conversion module
A data converter which converts the color of each pixel of the standard image into numerical data to obtain an initial index image;
And a row removal unit for removing a row and a column of a region that does not correspond to a search object based on the numerical data of each pixel in the initial index image to obtain an adjusted index image.

Images