JP3823129B2 - Image search system and image search method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to an image search system and an image search method for automatically searching image data captured by a video camera or the like.
[0002]
[Prior art]
  The introduction of security camera equipment for the purpose of preventing theft cases and various crimes and criminalizing the crime is increasing in product sales, factories and companies. Security camera equipment used in supermarkets, etc., introduces 2 to 4 units, with 8 color cameras (video cameras), 1 camera switcher and 1 long time video deck as 1 unit. That is, 16 to 32 color cameras, 2 to 4 camera switchers and long-time video decks are introduced.
[0003]
  The security camera equipment records one image per frame for about 1 second. In many cases, the contents of about 12 to 24 hours are intermittently recorded on one video tape.
[0004]
  In playback of recorded contents, it is difficult to fast forward or rewind playback of a video tape as in a general video deck. This is because, as described above, it is intermittent recording of about one sheet per second and there is little content continuity. The contents can be confirmed at about double speed. However, at higher speeds, the content of the image cannot be confirmed because switching of the content of the image is instantaneous.
[0005]
  When a videotape is recorded in the 12 hour mode, confirmation of all the reproduced images requires 6 hours for double speed reproduction. Further, eight color cameras are connected to one video deck. Accordingly, it takes 48 hours in total to confirm all the recorded contents. Under circumstances where the incident occurrence time cannot be specified, the above-mentioned time is spent searching for criminals.
[0006]
  Actually, not only the playback image is seen, but the video tape is paused, fast forwarded, and rewinded repeatedly. It causes considerable physical and mental pain to the operator. As a result, the person who has been in charge of this operation once will be reluctant to operate the video deck from the next time.
[0007]
  The above problems are also caused by the device configuration. Video decks record images in analog format on videotape. Due to the performance of the video deck, (1) long-time still image playback is not possible. (2) Skip (interlace) playback is not possible. (3) A search with a specified time cannot be performed. There is a problem as described above, which causes a long operation.
[0008]
  In addition to security camera equipment, there are equipment using video cameras. For example, in special nursing homes for the elderly, the introduction of surveillance camera equipment for the monitoring of dredging is increasing. To monitor wrinkles with surveillance camera equipment, the entrance is photographed with a video camera. Monitor the entrance and exit images on the monitor TV for 24 hours. It also supports the case of recording on videotape.
[0009]
  In addition to surveillance camera equipment, there are spear sensors such as necklace transmitters and wristband transmitters. Some facilities have been installed to issue an alarm when the signal from the sensor goes out of the monitoring area.
[0010]
  However, surveillance camera equipment has the following problems. Surveillance camera equipment requires a lot of labor to monitor a monitor TV for 24 hours. In addition, human error due to oversight occurs.徘徊 Sensors sometimes forget to carry transmitters.
[0011]
[Problems to be solved by the invention]
  An object of the present invention is to provide an image search system and an image search method that eliminates the continuous monitoring of images projected on a monitor television.
[0012]
[Means for Solving the Problems]
  The gist of the image search system of the present invention is as follows.An image retrieval system for retrieving image data that satisfies a condition set by a searcher from a plurality of image data photographed by a photographing unit and stored in a data storage unit, wherein a reference is selected from the plurality of image data A reference setting unit for setting reference image data to be, a setting unit having a condition setting unit for setting an area corresponding to the color and shape as the conditions, the reference image data in the plurality of image data, Comparison means for comparing other image data other than the reference image data for each corresponding pixel of the reference image and the other image data, and the reference image compared by the comparison means for each corresponding pixel For the other image data in which the difference between the other image data is a predetermined value or more, Image processing means having feature extraction means for searching for an area corresponding to the color and shape of the subject.. As described above, image data matching the condition is searched.
[0013]
  An image search system includes the image processing unit.Search results by feature extraction meansIncludes a result storage means for storing.
[0014]
  The image data is composed of a plurality of pixel data arranged vertically and horizontally.
[0015]
  The pixel data includes data representing numerical values of hue, saturation, lightness, red, green, and blue.
[0016]
  The color data set by the setting means is data representing the hue, saturation, brightness, red, green, and blue as numerical values.
[0017]
  The comparing means extracts a part or all of pixel data from a plurality of pixel data constituting the reference image data, and in the other image data, pixel data extracted from the reference image data; Means for extracting pixel data at the same position, and means for comparing the pixel data of the reference image data with the pixel data of other image data.
[0018]
  The feature extracting means includes means for dividing the other image data into a plurality of areas, means for extracting a connected area that includes the color set by the setting means in the plurality of areas, and Means for comparing the area of the connected region and the area corresponding to the shape set as a condition.
[0019]
  The image search system is set as the conditionData corresponding to the area corresponding to the color and shapeMeans for storingThe data includes a plurality of the pixel data.
[0020]
  The photographing means includes a video camera or a digital still camera.
[0021]
  The gist of the image search method of the present invention is as follows.An image search that searches for image data that meets the conditions set by the searcher from a plurality of image data that is continuously or intermittently captured by the image capturing means and is composed of a plurality of pixel data stored in the data storage means. A method of setting reference image data serving as a reference from among the plurality of image data, and an area corresponding to the color and shape serving as the conditions; and the reference image data in the plurality of image data; Comparing the other image data other than the reference image data for each pixel corresponding to the reference image and the other image data, and the reference image compared by the comparison unit for each corresponding pixel. Searching for an area corresponding to the color and shape as the condition for the other image data in which the difference between the other image data is a predetermined value or more..
[0022]
  After the step of searching for the area corresponding to the color and shapeAnd storing the search result.
[0023]
  In setting the color in the setting step, the hue, saturation, lightness, red, green, and blue of the color are set as numerical data.
[0024]
  The comparing step includes a step of extracting a plurality of pixel data from the reference image data, and a step of extracting pixel data in the same position as the pixel data extracted from the reference image data in the other image data. And comparing the pixel data of the reference image data with the pixel data of the other image data.
[0025]
  The step of comparing the pixel data includes a step of taking a difference of numerical values of hue, saturation, and brightness included in the pixel data, and calculating the difference between the reference image data and the other image data. Determining a difference.
[0026]
  The step of searching for an area corresponding to the color and shape includes extracting a plurality of connected regions including the step of dividing the other image data into a plurality of regions and the color set in the setting step. And a step of comparing the area corresponding to the shape set in the setting step and the area of the connected region.
[0027]
DETAILED DESCRIPTION OF THE INVENTION
  Next, an embodiment of an image search system and a search method of the present invention will be described using the drawings. The image retrieval system is used for security camera equipment and surveillance camera equipment.
[0028]
  As shown in FIG. 1, the image retrieval system 10 includes a video camera 12 that captures image data and a device (computer) 22 that retrieves the image data. The computer 22 includes a data storage unit 14 for storing a plurality of image data captured by the video camera 12, a setting unit 18 for setting a searcher's desired condition, an image processing unit 16 for searching for an image, and a search. Result storage means 20 for storing the results is included.
[0029]
  The video camera 12 captures image data intermittently or continuously in time. The image data is digital format data. The video camera 12 is a means that can capture image data in a digital format. When there are a plurality of locations to be monitored, a plurality of video cameras 12 are installed. When image data is intermittently captured, one image data is captured per second.
[0030]
  Further, the image data captured by the video camera 12 may be in an analog format. In that case, an analog / digital converter is provided between the video camera 12 and the data storage means 14.
[0031]
  When image data is photographed intermittently in time, photographing means other than the video camera 12 may be used. For example, a digital still camera can be used. By operating the digital still camera every predetermined time, image data can be taken intermittently in time.
[0032]
  As described above, the digital image data is stored in the data storage unit 14.
[0033]
  The image data is composed of a plurality of pixel data arranged vertically and horizontally. The pixel data includes color data. The color data is data obtained by quantifying the primary colors and the hues that are the elements of the colors. The primary colors are red, green and blue. Hue is hue, saturation, and lightness. The digitized data is obtained by dividing red and the like into a plurality of gradations and applying numerical values for each gradation. Therefore, there are a plurality of numerical values in one element such as red.
[0034]
  The data storage means 14 is a means capable of storing a plurality of image data in a digital format. For example, hard disk, MO (Magnet Optical) disk, DVD (Digital Versatile)
Disc). The data storage means 14 may be either internal or external to the computer 22. If the storage capacity of the data storage means 14 built in the computer 22 is lost, the external data storage means 14 can be added. By adding the data storage means 14, the system 10 can be used over a long period of time.
[0035]
  The setting means (program) 18 sets a reference setting unit for setting reference image data serving as a reference (background) from among a plurality of image data, and a color, shape, or both color and shape as search conditions. A condition setting unit. Further, the condition includes a region in the image data, and a search start time and an end time. The search start time and end time correspond to the time when the image data was taken. When an area or the like is set, image processing is performed only within that area. Accordingly, since the image processing is reduced, the search time is shortened.
[0036]
  The reference image data and conditions are set by inputting data for searching using a keyboard, mouse, touch panel, or the like in an input field displayed on the display. The input data is sent to the image processing means 16. As will be described later, when the condition to be set includes color and shape, the feature extraction unit of the image processing unit 16 operates, and when the condition includes the shape without including the color, the shape search unit of the image processing unit 16 operates. To do.
[0037]
  The reference image data is determined by the searcher selecting one image data from a plurality of image data. As the reference image data, it is preferable to select image data in which a moving object is not photographed. This is because the reference image data is compared with other image data by a comparison means described later, and a difference between the reference image data and the other image data is detected.
[0038]
  The color set by the condition setting unit is a hue that is a primary color and a color element. Data that digitizes these is set. The primary colors are red, green and blue. Hue is hue, saturation, and lightness. The digitized data is obtained by dividing red and the like into a plurality of gradations and applying numerical values for each gradation. These conditions are used for image processing to be described later. A numerical value can be automatically set by inputting a numerical value in the input field or selecting a color displayed on the display. Moreover, in order to include similar colors, an allowable range is provided for the numerical data of the conditions. This allowable range is configured so that the searcher can arbitrarily change it with a keyboard or a mouse.
[0039]
  Furthermore, in order to detect the area corresponding to the shape by the feature extraction means described later, the searcher can set the area. For example, as the detection threshold, an area where the selected color is included in the image data or a ratio of the area of the color included in the image data is set. Instead of setting this area, the shape may be set with a keyboard or a mouse. In this case, the computer 22 stores an area corresponding to the shape. By setting the shape, the area is automatically determined. Also, in these areas, an allowable range may be set as in the case of color.
[0040]
  Means for storing shape data corresponding to the shape is included in the computer 22. This means is means for storing digital data in the same manner as the data storage means 14. When the setting unit sets a shape without including a color, shape data corresponding to the shape is sent to a shape search unit of the image processing unit 16 described later. This shape data is composed of a plurality of pixel data. This pixel data has the same configuration as the pixel data constituting the image data. Since the shape varies depending on how the video camera 12 is reflected, an allowable range is provided for the shape data. Further, this allowable range may be changed by setting means.
[0041]
  The image processing means 16 is a comparison means for comparing the reference image data with other image data other than the reference image data in the plurality of image data, a feature extraction means for searching for an area corresponding to the color and shape, and a shape search. Includes shape retrieval means for performing
[0042]
  The comparison means is means for extracting a part or all of the pixel data from the plurality of pixel data constituting the reference image data, and is in the same position as the pixel data extracted from the reference image data in the other image data. Means for extracting pixel data, and means for comparing the pixel data of the reference image data with the pixel data of other image data.
[0043]
  As described above, the pixel data is handled as numerical data such as primary colors. Therefore, the comparison of the pixel data is performed by taking the difference of the pixel data. If the difference is 0 or a value close to it, there is little or no difference in pixel data. Further, the difference in pixel data increases as the distance from 0 increases. Therefore, the comparison means regards pixel data whose difference is greater than or equal to a predetermined value as pixel data that has changed. Of the pixel data obtained from the difference, the ratio of the changed pixel data is obtained. When this ratio is equal to or higher than a predetermined ratio (for example, 20%), the image data is sent to the feature extraction unit or the shape search unit. The image data is sent to the feature extraction means when the setting means includes color and shape in the condition, and is sent to the shape search means when it contains the shape without including the color.
[0044]
  As shown in FIG. 3, the feature extraction unit includes a unit that divides the other image data 30 into a plurality of regions 32, and a region that includes the colors set by the setting unit in the plurality of regions 32 and that is connected ( (Hereinafter, referred to as a connected region) and means for comparing the area of the connected region with the area corresponding to the shape set as a condition. For example, in FIG. 3, the shaded portion is a connection region.
[0045]
  When dividing into a plurality of regions 32, the area of one region 32 may be arbitrarily changed according to a searcher's setting. When the area is large, the image processing by the feature extraction unit is completed in a short time. When the area is small, the area of the connection region can be accurately obtained. When the area of the connection region becomes accurate, the comparison with the area set by the setting means 18 becomes accurate.
[0046]
  As shown in FIG. 4, the shape search means provides a means for providing an area 34 in other image data, moving the area 34, pixel data in the area 34, and shape data of a shape set as a condition. Means for comparing.
[0047]
  For example, the region 34 is moved pixel by pixel. If the pixel is moved pixel by pixel, the desired shape can be reliably searched. The pixel data in the region 34 and the shape data are compared by taking the difference. Pixel data and shape data are data representing the hue and the like as numerical values. Therefore, if the difference is 0 or a value close thereto, the shape to be searched exists in the area 34. Therefore, the shape search means determines that the image data 30 includes the shape of the condition when the difference is equal to or less than a predetermined value.
[0048]
  As described above, after the image processing is performed by the comparison unit, the image data in which the rate of change of the pixel data is a predetermined value or more is sent to the feature extraction unit and the shape search unit. This is for shortening the image processing time. More specifically, the comparison means (1) extracts pixel data from the reference image data. (2) Extract pixel data of other image data. (3) The difference between the pixel data of the reference image data and the other image data is taken, and the change of the image data is determined from the difference value. Although it is the above three steps, the step (1) is only once, and the second and subsequent steps are two steps. The feature extraction means (1) divides the image data into a plurality of regions. (2) Extracting a plurality of connected regions including the color set by the setting means. (3) The area corresponding to the shape set in the setting step is compared with the area of the connected region. As described above, the image processing of the feature extraction unit is three steps. Accordingly, only the image data for which the difference in pixel data is confirmed by the comparison means is subjected to image processing by the feature extraction means, thereby shortening the image processing time.
[0049]
  The advantage of using feature extraction means is that image processing can be performed even with unclear image data. In general, there is a method using edge detection as a method of extracting a predetermined shape or the like by image processing. In edge detection, a color boundary portion (edge) is detected based on a change in color. For example, when the color of the pixel data is digitized as described above, a portion having a large difference in the numeric value becomes an edge. By detecting the edge, a region having a desired color can be extracted.
[0050]
  However, the system 10 is used for security camera equipment. Therefore, the image data photographed by the video camera 12 often includes defocus and noise. That is, there are many unclear images in the image data. As a matter of course, the edge of the unclear image data is difficult to understand. Therefore, it is difficult to perform edge detection, and wrong search is performed. Since the feature extraction means of the present invention detects whether or not a desired color or shape exists in the region, the sharpness of the image does not matter. As described above, the system 10 can perform image processing even for unclear image data and can be used for security camera equipment and the like.
[0051]
  The system 10 includes a result storage unit 20 that stores the results of the feature extraction unit or the shape search unit. The result storage means 20 uses a hard disk or the like as in the data storage means 14. Further, one hard disk may be divided into two areas, and these areas may be used as the data storage means 14 and the result storage means 20. The result to be stored is image data that meets the conditions in image processing such as feature extraction means.
[0052]
  The computer 22 includes an MPU (Micro Processor Unit). The operations of the setting unit 18 and the image processing unit 16 are controlled by the MPU.
[0053]
  As shown in FIG. 2, the system 10 includes an instruction unit 24 for instructing reproduction of image data, a monitor television 28 for displaying the image data stored in the result storage unit 20, and a video deck for controlling the display. 26. The video deck 26 may be changed to a printer and output to a paper medium.
[0054]
  The computer 22 may be connected to a network such as the Internet or a LAN (Local Area Network). The image data stored in the result storage unit 20 may be displayed on another computer connected to the network.
[0055]
  An image search method using the system 10 will be described. The video camera 12 takes a picture of the monitoring target location. The image data to be photographed is digital format data. The configuration of the image data is that a plurality of pixel data are arranged vertically and horizontally. The pixel data includes color data. The color data is data obtained by digitizing primary colors or color elements. Such image data is taken intermittently or continuously in time.
[0056]
  The captured image data is stored in the data storage unit 14. A searcher searches for image data as necessary. For example, a search is performed when the occurrence of a crime is known.
[0057]
  The searcher inputs necessary conditions into the input field of the setting means 18 displayed on the display of the computer 22 using a keyboard and a mouse.
[0058]
  The conditions are reference image data serving as a reference and conditions to be searched. The reference image data is selected from a plurality of image data stored in the data storage unit 14. As the reference image data, it is preferable to select image data in which a moving object is not photographed. This compares the reference image data with other image data by the comparison means. As a result of the comparison, image data in which pixel data of a predetermined ratio or more has changed is sent to the feature extraction means or the like. Therefore, if there is a moving object, all the image data is sent to the feature extraction means and the image processing takes time.
[0059]
  Search conditions include color and / or shape. Further, the condition includes a region in the image data, and a search start time and an end time. The search start time and end time correspond to the time when the image data was taken.
[0060]
  The condition color is a hue that is an element of the primary color and the color. Data that digitizes these is set. The primary colors are red, green and blue. Hue is hue, saturation, and lightness. A numerical value is automatically set by entering a numerical value in the input field or selecting a color displayed on the display. Moreover, in order to include similar colors, an allowable range is provided for the numerical data of the conditions. The reason why similar colors are included is that, for example, when searching for clothes, the colors to be set differ depending on how the light hits the clothes and individual differences in the searcher's vision.
[0061]
  The shape is a shape to be actually searched for such as clothes and glasses. Set by entering the name of the shape or selecting the name displayed on the display. When setting the shape simultaneously with the color, the area of the shape may be set instead of selecting the shape.
[0062]
  When the reference image data and the search conditions are set by the setting means 18, they are sent to the image processing means 16. When the search condition includes a shape without including a color, the shape data is sent regarding the shape. When the color and shape (area of shape) are included, the area corresponding to the shape is sent for the shape.
[0063]
  The comparison means of the image processing means 16 compares the reference image data with other image data. The comparison method extracts part or all of the pixel data of the reference image data. In other image data, pixel data having the same coordinates as the pixel data extracted from the reference image data is extracted. The difference between the reference image data at the same coordinates and the pixel data of other image data is taken. If the difference value is greater than or equal to a predetermined value, the pixel data has changed. Differences are taken in all pixel data. If the ratio of the changed pixel data is equal to or greater than a predetermined ratio, the image data is sent to the feature extraction means or the shape search means. When the color and shape are set by the setting means, image data is sent to the feature extraction means. When the shape is set without including the color by the setting means, the image data is sent to the shape search. If the ratio is less than the predetermined ratio, the image processing is ended for the image data.
[0064]
  A case where image data is sent to the feature extraction unit will be described. The image data 30 is divided into a plurality of regions 32 as shown in FIG. A connected region (shaded portion in the figure) including a condition color is extracted from the plurality of regions 32. The area of the connection region is compared with the area corresponding to the shape of the condition. The comparison is performed by taking the difference between the two. If it is less than the predetermined value, the area set by the setting means is determined. The image data is stored in the result storage means on the assumption that the conditions are met. Moreover, when it is more than a predetermined value, it determines with it not being the set area. When it is determined that the area is not the set area, the image processing is ended for the image data. This predetermined value can be changed by the setting means 18 as an allowable range of the area.
[0065]
  The reason for determining whether or not the condition is met when the difference is taken is less than or greater than the predetermined value. The video camera 12 is generally fixed. Since it is fixed, the size in the image data changes depending on the position of the subject. This is because it is not possible to search for image data that matches the condition unless a range that is considered to be matched has a width.
[0066]
  A case where the image data 30 is sent to the shape search means will be described. As shown in FIG. 4, an area 34 is set in the image data 30. Pixel data in the region 34 is extracted. The difference between the shape data of the shape 36 set by the setting means and the pixel data is taken. If the difference is 0 or less than the predetermined value, it is determined that the condition shape 36 exists in the region 34. The image data is stored in the result storage means as matching the condition. The predetermined value is an allowable range of the shape data. For example, as shown in FIG. 4, when the region 34 moves to a position where the shape 38 stored in the image data is located, the condition shape 36 is detected. If the difference is other than the above, it is determined that there is no condition shape 36 in the region 34. In that case, the image processing is ended for the image data.
[0067]
  As described above, when the image data 30 matching the condition is detected in the feature extraction unit or the shape search unit, the image data 30 is stored in the result storage unit 20.
[0068]
  The image data 30 stored in the result storage means 20 is sent to the monitor television 28 by the instruction means 24 and displayed. When the computer 22 is connected to a network, the image data 30 stored in the result storage unit 20 may be browsed by another computer.
[0069]
  When a plurality of video cameras 12 are installed, a search can be performed from the image data 30 captured by all the video cameras 12. Therefore, the present invention can be used for security camera equipment and the like.
[0070]
  As described above, the search of the image data 30 can be automatically performed only by the searcher setting the search condition. Therefore, the searcher is not physically or mentally painful.
[0071]
  Next, a method for monitoring the storage of the image data 30 including desired conditions while storing the image data 30 will be described.
[0072]
  Before taking a picture with the video camera 12, the setting means 18 sets the conditions of what is desired to be monitored. The setting method is the same as the above method. The setting means 18 sends the input conditions to the image search means 16.
[0073]
  The video camera 12 starts to capture the image data 30. Shooting is performed continuously or intermittently in time. The captured image data 30 is sequentially stored in the data storage unit 14. The stored image data 30 is sequentially subjected to image processing by the image processing means 16. The image processing method is the same as that described above. By this image processing, it is monitored whether image data 30 matching the conditions is stored.
[0074]
  The image data 30 that matches the conditions is stored in the result storage unit 20. Further, the instruction means 24 is used to display on the monitor TV 28. Instead of displaying, it may be printed on a paper medium. Further, a warning buzzer may be provided in the system 10 so that the warning buzzer sounds when the image data 30 is stored in the result storage unit 20.
[0075]
  As described above, the system 10 does not require the monitor to constantly monitor the monitor television 28. If the alarm buzzer is used, the alarm buzzer can notify the supervisor even if the supervisor is doing other work.
[0076]
  When the system 10 is used for wrinkle monitoring or the like, the color of the wearer's clothes is a condition. Further, by installing the video camera 12 at the entrance or the like, it is possible to monitor the deaf person. Further, a plurality of positions can be monitored by positioning a plurality of video cameras 12.
[0077]
  As described above, it is not necessary for the monitor to constantly monitor the video displayed on the monitor television 28. Therefore, the labor of the supervisor can be reduced. Also, do not forget to monitor with an alarm buzzer.
[0078]
  As mentioned above, although embodiment of this invention was described, this invention is not limited to said embodiment. For example, the setting unit 18 and the like are configured by a program, but other configurations may be used. As an example, DSP (Digital Signal
Processor) or a logic circuit.
[0079]
  Since the feature extraction unit and the shape search unit are used separately, the system 10 may be configured to include only one of them. Alternatively, both means are used separately, but when both color and shape are set, both means may be used. That is, after image processing is performed by the feature extraction means, image processing is further performed by the shape search means.
[0080]
  In the feature extraction means, the area of the set color and shape is searched, but image data larger than the set area may be stored in the result storage means.
[0081]
  When the shape is set without including a color by the setting means 18, image processing is performed by the shape search means. However, there are many different colors with the same shape. For example, clothes are sold in a number of different colors even if they have the same shape. Therefore, even when the color and shape are set by the setting means 18, the shape search means can be used. Specifically, the setting means 18 can select a feature extraction means or a shape search means. The shape data is stored in the system 10 in correspondence with a plurality of colors in the system 10 even if the shape is the same.
[0082]
  When the image data captured by the video camera 12 is black and white data, the feature extraction means cannot be used. Therefore, only the shape retrieval means can be used.
[0083]
  A sensor may be provided at a shooting monitoring position for shooting with the video camera 12. The video camera 12 is operated by the sensor signal. When the sensor detects an object, a signal from the sensor is transmitted to the video camera 12. Then, the video camera 12 operates. When the sensor no longer detects an object, a signal is sent from the sensor to the video camera 12. The video camera 12 is stopped by this signal. Alternatively, the time from when the video camera 12 is activated to when it is stopped is determined. As described above, it is not always necessary for the video camera 12 to capture image data intermittently or continuously in time. Therefore, the amount of image data stored in the data storage unit 14 is reduced.
[0084]
  In addition, the present invention can be carried out in a mode to which various improvements, modifications and variations are added based on the knowledge of those skilled in the art without departing from the spirit of the present invention.
[0085]
【The invention's effect】
  According to the present invention, a plurality of image data can be automatically searched, and the search can be completed in a short time. There is no physical or mental distress for those who retrieve desired image data from the photographic data. By providing a plurality of image processing means, it is possible to select an image processing means according to a condition to be searched, and it is not possible to search according to the condition.
[0086]
  Furthermore, in the case of constantly monitoring the monitoring target location, it is not necessary to always monitor the image on the monitor television by configuring the alarm buzzer to sound when image data that meets the conditions is stored. It is possible to reduce the number of personnel in monitoring.
[0087]
  Further, before the image processing is performed by the feature extraction unit or the shape search unit, the comparison unit extracts image data that has changed in comparison with the reference image data. It is not necessary to perform image processing on all image data by feature extraction means or shape search means. Therefore, the time for image processing is shortened.
[0088]
  Image processing is performed by feature extraction means or shape search means. Unlike general image processing that extracts edges of image data (abrupt changes in color), image processing can be performed even on unclear image data that is noisy or out of focus. Therefore, it is possible to use a video camera of a security camera facility that has been used conventionally. The cost for installing the system can be kept low.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of the present invention.
FIG. 2 is a diagram showing a configuration for displaying search results.
FIG. 3 is a diagram showing a method of extracting a shape by a shape extracting unit.
FIG. 4 is a diagram showing a method of searching for a shape by a shape search means.
[Explanation of symbols]
  10: Image search system
  12: Video camera (photographing means)
  14: Data storage means
  16: Image processing means
  18: Setting means
  20: Result storage means
  22: Computer
  24: Instruction means
  26: VCR
  28: Monitor TV
  30: Image data
  32, 34: Area
  36, 38: Shape

Claims (15)

An image retrieval system for retrieving image data that satisfies a condition set by a searcher from a plurality of image data photographed by a photographing means and stored in a data storage means,
A setting unit having a reference setting unit that sets reference image data serving as a reference from among the plurality of image data, and a condition setting unit that sets an area corresponding to the color and shape as the conditions;
Comparing means for comparing the reference image data in the plurality of image data and other image data other than the reference image data for each corresponding pixel of the reference image and the other image data ;
For each of the corresponding pixels, an area corresponding to the color and shape as the condition is searched for the other image data in which the difference between the reference image and the other image data compared by the comparison unit is a predetermined value or more . Image processing means having feature extraction means;
Image search system including The image search system according to claim 1, further comprising a result storage unit that stores a search result obtained by the feature extraction unit of the image processing unit. The image search system according to claim 1 or 2, wherein the image data includes a plurality of pieces of pixel data arranged vertically and horizontally. The image search system according to claim 3, wherein the pixel data includes data representing numerical values of hue, saturation, brightness, red, green, and blue. 5. The image search system according to claim 1, wherein the color data set by the setting means includes data representing numerical values of hue, saturation, brightness, red, green, and blue. The comparing means is
Means for extracting part or all of the pixel data from the plurality of pixel data constituting the reference image data;
Means for extracting pixel data in the same position as the pixel data extracted from the reference image data in the other image data;
Means for comparing pixel data of the reference image data with pixel data of other image data;
The image search system according to claim 3, further comprising: The feature extracting means is
Means for dividing the other image data into a plurality of regions;
Means for extracting a connected area including the color set by the setting means in the plurality of areas;
7. The image search system according to claim 5, further comprising means for comparing an area of the connected region with an area corresponding to a shape set as a condition. The image search system according to any one of claims 4 to 7, further comprising means for storing data corresponding to an area corresponding to a color and shape set as the condition, wherein the data includes a plurality of the pixel data . Image retrieval system according to claims 1 to 8 comprising the imaging means, a video camera or digital still camera. An image search that searches for image data that meets the conditions set by the searcher from a plurality of image data that is continuously or intermittently captured by the image capturing means and is composed of a plurality of pixel data stored in the data storage means. A method,
A step of setting reference image data serving as a reference from among the plurality of image data, and an area corresponding to the color and shape serving as the conditions;
Comparing the reference image data in the plurality of image data and other image data other than the reference image data for each corresponding pixel of the reference image and the other image data ;
For each of the corresponding pixels, an area corresponding to the color and shape as the condition is searched for the other image data in which the difference between the reference image and the other image data compared by the comparison unit is a predetermined value or more . And an image search method including the steps. The image search method according to claim 10 , further comprising a step of storing a search result after the step of searching for an area corresponding to the color and shape . The image search method according to claim 10 or 11 , wherein the color setting in the setting step sets the hue, saturation, lightness, red, green, and blue of the color with numerical data. The step of comparing comprises:
Extracting a plurality of pixel data from the reference image data;
Extracting the pixel data at the same position as the pixel data extracted from the reference image data in the other image data;
Comparing the pixel data of the reference image data with the pixel data of the other image data;
The image search method according to claim 12 , comprising: The step of comparing the pixel data comprises:
A step of calculating a difference between numerical values of hue, saturation, and brightness included in pixel data;
Determining a difference between the reference image data and the other image data from a value obtained by taking the difference;
The image search method according to claim 13 , comprising: Searching for an area corresponding to the color and shape,
Dividing the other image data into a plurality of regions;
Including the color set in the setting step and extracting a plurality of connected regions; and
Comparing the area corresponding to the shape set in the setting step and the area of the connected region;
Image searching method according to claim 11 or 14 including.

Images