JP2005202938A - Image search apparatus and image search method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image search apparatus and an image search method which realize proper image search and effective search by easily designating a search object. <P>SOLUTION: The image search apparatus has; a moving area extraction part 102 for extracting a storage object moving area within an image; an area division part 104 for dividing the storage object moving area into storage object block areas; a representative color calculation part 105 for deriving a representative color of each of storage object block areas constituting the storage object moving area; a DB 106 for storing representative colors of respective storage object block areas constituting the storage object moving area; a search area designation part 108 for extracting a search object area within the image; an area division part 109 for dividing the search object area into search object block areas; a representative color calculation part 110 for deriving a representative color of each of search object block areas constituting the search object area; and a comparison part 111 for comparing representative colors of respective storage object block areas constituting the storage object moving area which are stored in the DB 106, and representative colors of respective search object block areas constituting the search object area with each other and performing output according to the results of comparison. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video search apparatus and a video search method mainly used for video surveillance.

  Conventionally, various video search devices (a monitoring system and a personal identification device) used for video monitoring applications such as personal identification have been proposed (see, for example, Patent Documents 1 and 2).

  A block diagram of a conventional monitoring system is shown in FIG. The monitoring system illustrated in FIG. 14 includes a frame memory 512 that records a monitoring image captured by the imaging device 502 installed at a predetermined position in the digital recorder 501 via the A / D conversion unit 511, and a frame memory 512. The monitoring images recorded in the screen are sequentially read out, and at least a part of the read monitoring image is designated according to the operation of the operation unit 517 by the user, and further, the monitoring image having movement in the designated region is searched. The processing control unit 516 that performs the search, and the monitoring image in the frame memory 512 searched by the processing control unit 516 are input via the data compression unit 513, the write buffer 514, the data recording unit 515, the read buffer 518, and the data decompression unit 519. Display memory 520, D / A conversion unit 521 and display device 505 to be reproduced. To. In this monitoring system, only videos that have moved in the area designated by the operation of the operation unit 517 by the user are searched, extracted, and displayed at high speed.

  FIG. 15 shows the operation of the conventional personal identification device. The personal identification device shown in FIG. 15 acquires an image input unit 611 that inputs a face image of a person to be identified, and a face image input by the image input unit 611 via a primary face normalization unit 612. The feature point extraction unit 613 that extracts the feature points of the face image, the standard face image feature point database unit 614 that records the feature points of the standard face image, and the feature points coincide with the feature points of the standard face image. The reference face image feature point database unit 616 that records the deformed reference face image for each person, and the feature points match the feature points of the standard face image recorded in the standard face image feature point database unit 614. As described above, the secondary face normalization unit 615 that deforms the face image input by the image input unit 611 and the face image normalized by the secondary face normalization unit 615 and the reference face image database unit 616 are recorded. An image correlation calculation unit 617 that sequentially obtains a correlation with the reference face image, and a determination unit 618 that identifies a person corresponding to the face image input by the image input unit 611 based on the correlation location obtained by the image correlation calculation unit 617. And have. This personal identification device searches for a reference face image having the highest correlation with an input face image, and makes it possible to perform personal identification based on the searched reference face image. Robust (robust) personal identification against changes in

  In the conventional video search system, for example, the following is known as a method for searching for a desired person image using color information. One is to specify the entire image as a search target, extract color information from the image, compare the extracted color information with the color information extracted from the video data stored in the storage device, and compare similar images. How to get. In addition, when color information is directly specified as attribute information, the specified rectangular area is cut out from the displayed image by inputting the rectangular area on the image displayed on the monitor via the GUI. There is a method of extracting color information from the cut image and making it a search target. The “HSV space” described in this specification is disclosed in Non-Patent Document 1, and the “histogram intersection” is disclosed in Patent Document 3.

JP 2000-132669 A Japanese Patent Laid-Open No. 11-161791 JP 2004-252748 A (paragraphs 0012-0013) Supervised by Mikio Takagi and Yoshihisa Shimoda, "Image Analysis Handbook", The University of Tokyo Press, January 1991

  However, since the conventional monitoring system described above searches for an image to be reproduced based only on the presence or absence of motion, it is not always sufficient for video search applications for identifying specific objects such as personal identification. . On the other hand, when the above-described conventional personal identification device cannot acquire an image of a specific part such as the eyes, nose, and mouth of the face, it searches for the image and performs personal identification based on the search result. It is impossible. For this reason, it is required to perform video search appropriately.

  Further, in the conventional video search system, for example, in the method of designating the image itself as a search key, the search cannot be performed unless the image to be searched in advance is at hand. In addition, when color information is directly specified as attribute information, a method of specifying a search target area for extracting color information by inputting a rectangular area on an image displayed on a monitor via a GUI allows a user to select a color area. There are differences, and it may be difficult to always obtain similar search results.

  The present invention has been made to solve the conventional problems, and provides a video search apparatus and a video search method capable of appropriately performing a video search and easily performing an effective search by specifying a search target. The purpose is to provide.

  The video search apparatus of the present invention extracts a storage target moving area in a first video and outputs a storage target moving area video signal corresponding to the storage target moving area, and the storage target Based on a moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are output. Based on the target block area dividing means and the storage target block area video signal corresponding to each of the storage target block areas, a representative color of each of the storage target block areas constituting the storage target moving area is derived. An accumulation target block area representative color deriving means for outputting; an accumulation target block area representative color accumulating means for accumulating each representative color of the accumulation target block area; A search target area extracting means for extracting a search target area in the video of two and outputting a search target area video signal corresponding to the search target area; and searching the search target area based on the search target area video signal A search target block area dividing unit that divides the target block area and outputs a search target block area video signal corresponding to each of the search target block areas constituting the search target area, and corresponds to each of the search target block areas Search target block area representative color deriving means for deriving and outputting a representative color of each search target block area constituting the search target area based on the search target block area video signal, and the storage target block area A comparison in which each representative color is compared with each representative color of the block area to be searched, and an output corresponding to the comparison result is performed. A configuration and a stage.

  With this configuration, the video search is performed based on the motion in the video and the color of each block constituting the motion region. In other words, video search is performed not only based on the motion in the video but also based on the color of the region of the motion. Therefore, when searching for a video based only on the presence or absence of motion, or when searching for a video of a specific part of the target object The video search can be performed more appropriately than in the case where

  In the video search device of the present invention, the comparison unit derives a difference between each representative color of the accumulation target block area and each representative color of the search target block area as the comparison result. be able to.

  With this configuration, it is possible to search for an approximated color to the search target area from among a plurality of accumulation target moving areas, and thus it is possible to appropriately perform video search based on color.

  Further, in the video search device of the present invention, the comparison unit is configured to obtain a predetermined value from a value obtained by digitizing each representative color of the accumulation target block area and a value obtained by digitizing each representative color of the search target block area. The difference may be derived according to a rule.

  Further, the video search device of the present invention, based on the accumulation target moving area video signal, determines whether or not the accumulation target moving area satisfies a predetermined condition as a condition for being a person, The storage target moving area may be configured to include a person determination unit that outputs the storage target moving area video signal to the storage target block area dividing unit when the storage target moving area satisfies the predetermined condition.

  With this configuration, it is possible to store only the color of the accumulation target moving area corresponding to the person, and prevent the accumulation of unnecessary moving area colors other than the person when personal identification is performed. can do.

  In the video search device of the present invention, the accumulation target block area representative color deriving unit corresponds to an average value of values obtained by a conversion method for reducing the influence of luminance change on colors appearing in the accumulation target block area. One of the color and the color with the highest frequency of the obtained value may be derived as a representative color of the accumulation target block area.

  With this configuration, it is possible to appropriately derive the representative color of the accumulation target block area.

  In the video search device of the present invention, the search target block area representative color deriving unit corresponds to an average value of values obtained by a conversion method that reduces the influence of a luminance change on a color that appears in the search target block area. One of the color and the color with the highest frequency of the obtained value may be derived as a representative color of the search target block area.

  With this configuration, it is possible to appropriately derive the representative color of the search target block area.

  The video search method of the present invention is a video search method for searching for a moving object in a first video based on information in a second video, wherein a moving target moving area in the first video is determined. An accumulation target moving area extraction step for extracting and outputting a accumulation target moving area video signal corresponding to the accumulation target moving area, and based on the accumulation target moving area video signal, the accumulation target moving area is converted into an accumulation target block area. An accumulation target block area dividing step for dividing and outputting an accumulation target block area video signal corresponding to each of the accumulation target block areas constituting the accumulation target moving area, and the accumulation corresponding to each of the accumulation target block areas Accumulation target block area representative that derives and outputs a representative color of each of the accumulation target block areas constituting the accumulation target moving area based on the target block area video signal A derivation step, a storage target block region representative color storage control step of storing each representative color of the storage target block region in a storage target block region storage means, and extracting a search target region in the second video, A search target area extracting step for outputting a search target area video signal corresponding to the search target area, and dividing the search target area into search target block areas based on the search target area video signal, thereby configuring the search target area Based on the search target block region video signal corresponding to each of the search target block regions, the search target block region video signal outputting a search target block region video signal corresponding to each of the search target block regions A search target block for deriving and outputting a representative color of each of the search target block areas constituting the search target area. And a comparison step of comparing each representative color of the accumulation target block region with each representative color of the search target block region and performing output according to the comparison result. It becomes composition.

  In the video search method of the present invention, the comparison step derives a difference between each representative color of the accumulation target block area and each representative color of the search target block area as the comparison result. be able to.

  Further, the video search method of the present invention determines, based on the accumulation target moving area video signal, whether or not the accumulation target moving area satisfies a predetermined condition defined as a condition of being a person, A person determination step of outputting the accumulation target moving area video signal when the accumulation target moving area satisfies the predetermined condition, wherein the accumulation target block area dividing step includes the accumulation target moving area image in the person determining step; When a signal is output, the accumulation target moving area is divided into accumulation target block areas based on the accumulation target moving area video signal, and each accumulation target block area constituting the accumulation target moving area is handled. The accumulation target block area video signal to be output can be output.

  Further, the video search device of the present invention extracts a storage target moving area in a first video and outputs a storage target moving area video signal corresponding to the storage target moving area; Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and an accumulation target block area video signal corresponding to each of the accumulation target block areas constituting the accumulation target moving area is output. And a video signal included in each of the accumulation target block areas constituting the accumulation target moving area based on the accumulation target block area video signal corresponding to each of the accumulation target block areas. Storage target block area color information generating means for extracting and outputting a color distribution from the storage target block area, and storing target block area for storing color information of each of the storage target block areas. A search area color information storage means, a search target area extraction means for extracting a search target area in the second video, and outputting a search target area video signal corresponding to the search target area; and the search target area video signal A search target block region dividing means for dividing the search target region into search target block regions and outputting search target block region video signals corresponding to the search target block regions constituting the search target region; Search that extracts and outputs a color distribution from video signals included in each previous search target block area constituting the search target area based on the search target block area video signal corresponding to each of the search target block areas The target block area color information generation means, the color distribution of each of the accumulation target block areas, and the color distribution of the search target block area are compared. And, a structure having a comparing means for performing an output corresponding to the comparison result.

  With this configuration, the video search is performed based on the motion in the video and the color of each block constituting the motion region. In other words, since video search is performed not only based on the motion in the video but also based on the color distribution of the region of the motion, when searching for the video based only on the presence or absence of motion, Video search can be performed more appropriately than necessary.

  Further, in the video search device of the present invention, the comparison means calculates the degree of coincidence of the color appearance frequencies of each color distribution of the accumulation target block area and each color distribution of the search target block area as the comparison result. It can be set as the structure derived | led-out as.

  Further, in the video search device of the present invention, the comparing means calculates the degree of coincidence of the color appearance frequency between each color distribution of the accumulation target block area and one of the color distributions of the search target block area. It can be set as the structure derived | led-out as.

  Further, the video search device of the present invention determines whether or not the accumulation target moving area satisfies a predetermined condition defined as a condition that the accumulation target moving area is a person based on the accumulation target moving area video signal, The storage target moving area can be configured to include a person determination unit that outputs the storage target moving area video signal to the storage target block area dividing unit when the storage target moving area satisfies the predetermined condition.

  With this configuration, it is possible to search for a plurality of accumulation target moving areas that have similar ratios of the number of occurrences of each color in the search target area, so that video search based on colors can be performed appropriately.

  Further, the video search device of the present invention extracts a storage target moving area in a first video and outputs a storage target moving area video signal corresponding to the storage target moving area; Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and an accumulation target block area video signal corresponding to each of the accumulation target block areas constituting the accumulation target moving area is output. And a video signal included in each of the accumulation target block areas constituting the accumulation target moving area based on the accumulation target block area video signal corresponding to each of the accumulation target block areas. Storage target block area color information generating means for extracting and outputting a color distribution from the storage target block area, and storing target block area for storing color information of each of the storage target block areas. A search area color information storage unit, and a search target point in the second video is extracted, and color information of the search target point is extracted based on a search target area video signal corresponding to the search target point. Retrieval color information generating means for outputting, representative color information is generated from each color distribution of the accumulation target block area, the representative color information is compared with the search target color information, and the comparison result is determined. And a comparison unit that performs output.

  With this configuration, even when there is no image to be searched, only one point is specified from the color information image indicated by the search device, and the video search is performed not only based on the motion in the captured video but also based on the color of the region of the motion. Therefore, the video search can be performed more appropriately than when the video search is performed based only on the presence or absence of movement or when the video of the specific part of the target object is required for the video search. Can be easily specified.

  Further, in the video search device of the present invention, the comparison means uses the similarity of the appearance frequency of the color between the representative color information generated from each color distribution of the accumulation target block region and the color information of the search target point. Is derived as the comparison result.

  With this configuration, it is possible to search for a plurality of accumulation target moving regions that are similar to the color information of the search target point, so that it is possible to appropriately perform video search based on color.

  Further, the video search apparatus of the present invention extracts an accumulation target moving area in a video, and outputs an accumulation target moving area video signal corresponding to the accumulation target moving area, and the accumulation target moving area. Person motion scene determination means for identifying the same person from continuous frame images in the video based on the area video signal, outputting the moving area video signal to be accumulated in the previous period, and outputting a motion scene end signal when there is no movement And, based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and the accumulation target block area videos corresponding to the respective accumulation target block areas constituting the accumulation target moving area A storage target block region dividing means for outputting a signal, and a storage target block region video signal corresponding to each of the storage target block regions. A color feature amount is extracted from a video signal included in each of the accumulation target block areas constituting the accumulation target moving area for each person, and the movement scene end signal is received to each accumulation target block area for each person. Based on the accumulation target block area color information generating means for generating and outputting a corresponding color distribution and the accumulation target moving area video signal, a search target area is set for each person, and the motion scene end signal is received. Search area setting means for outputting a search target area video signal corresponding to the search target area for each person, color information of each storage target block area and the search target area video signal corresponding to the search target area An area color information storage means for storing, and a person representative image list that obtains the search target area video signal from the area color information storage means and outputs a display list. Display means, search target area color information generating means for generating and outputting a color distribution from a video signal included in the search target area based on the search target area video signal, and each color of the storage target block area Comparing the distribution with the color distribution of the search target area, and a comparison means for performing output according to the comparison result.

  With this configuration, the user can select a search target person while visually confirming one image for each person, so that the search target area can be easily specified. In addition, since a search is performed using a search target area generated in advance by selecting a person, it is possible to appropriately perform a video search based on color.

  Further, the video search apparatus of the present invention may be configured such that the accumulation target block area dividing means determines the division position according to the shape of a person.

  With this configuration, it becomes possible to selectively designate a person's clothing as a search target area, and not only a video can be searched for by color, but also a search based on further narrowed clothes can be performed.

  The video search method of the present invention is a video search method for searching for a moving object in a first video based on information in a second video, wherein a moving target moving area in the first video is determined. An accumulation target moving area extraction step for extracting and outputting a accumulation target moving area video signal corresponding to the accumulation target moving area, and based on the accumulation target moving area video signal, the accumulation target moving area is converted into an accumulation target block area. An accumulation target block area dividing step for dividing and outputting an accumulation target block area video signal corresponding to each of the accumulation target block areas constituting the accumulation target moving area, and the accumulation corresponding to each of the accumulation target block areas Accumulation that extracts and outputs a color distribution from the video signal included in each of the accumulation target block areas constituting the accumulation target moving area based on the target block area video signal Elephant block area color information generation step, accumulation target block area color information accumulation control step for accumulating the color information of each accumulation object block area in the accumulation object block area accumulation means, and search object area in the second video A search target area extraction step for extracting a search target area video signal corresponding to the search target area, and dividing the search target area into search target block areas based on the search target area video signal, A search target block region dividing step for outputting a search target block region video signal corresponding to each of the search target block regions constituting the search target region, and the search target block region video signal corresponding to each of the search target block regions Based on the video signal included in each previous search target block area constituting the search target area. The search target block region color information generation step for extracting and outputting the color distribution from the output, comparing each color distribution of the accumulation target block region with the color distribution of the search target block region, and according to the comparison result And a comparison step for outputting.

  Further, in the video search method of the present invention, the comparison step determines the degree of coincidence of the color appearance frequency between each color distribution of the accumulation target block area and each color distribution of the search target block area. It can be set as the structure derived | led-out as.

  Further, in the video search method of the present invention, the comparison step determines the degree of coincidence of the color appearance frequency between each color distribution of the accumulation target block area and one of the color distributions of the search target block area. It can be set as the structure derived | led-out as.

  Further, the video search method of the present invention determines, based on the accumulation target moving area video signal, whether or not the accumulation target moving area satisfies a predetermined condition defined as a condition of being a person, A person determination step of outputting the accumulation target moving area video signal when the accumulation target moving area satisfies the predetermined condition, wherein the accumulation target block area dividing step includes the accumulation target moving area image in the person determining step; When a signal is output, the accumulation target moving area is divided into accumulation target block areas based on the accumulation target moving area video signal, and each accumulation target block area constituting the accumulation target moving area is handled. The accumulation target block area video signal to be output can be output.

  The video search method of the present invention is a video search method for searching for a moving object in a first video based on information in a second video, wherein a moving target moving area in the first video is determined. An accumulation target moving area extraction step for extracting and outputting a accumulation target moving area video signal corresponding to the accumulation target moving area, and based on the accumulation target moving area video signal, the accumulation target moving area is converted into an accumulation target block area. An accumulation target block area dividing step for dividing and outputting an accumulation target block area video signal corresponding to each of the accumulation target block areas constituting the accumulation target moving area, and the accumulation corresponding to each of the accumulation target block areas Accumulation that extracts and outputs a color distribution from the video signal included in each of the accumulation target block areas constituting the accumulation target moving area based on the target block area video signal Elephant block area color information generation step, accumulation target block area color information accumulation control step for accumulating the color information of each accumulation target block area in the accumulation target block area accumulation means, and one search in the second video A search target color information generating step of extracting a target point and extracting and outputting color information of the search target point based on a search target area video signal corresponding to the search target point; and each of the storage target block areas A comparison step of generating representative color information from the color distribution, comparing the representative color information with the color information to be searched, and outputting according to the comparison result.

  Further, in the video search method of the present invention, the comparison step includes the similarity of the appearance frequency of the color between the representative color information generated from each color distribution of the accumulation target block region and the color information of the search target point. Can be derived as the comparison result.

  In addition, the video search method of the present invention includes a storage target motion area extraction step of extracting a storage target motion area in a video and outputting a storage target motion area video signal corresponding to the storage target motion area; A person motion scene determination step of identifying the same person from continuous frame images in the video based on the region video signal, outputting the moving region video signal to be accumulated in the previous period, and outputting a motion scene end signal when there is no movement And, based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and the accumulation target block area videos corresponding to the respective accumulation target block areas constituting the accumulation target moving area An accumulation target block area dividing step for outputting a signal, and the accumulation target block area video signal corresponding to each of the accumulation target block areas; Accordingly, a color feature amount is extracted from the video signal included in each of the accumulation target block areas constituting the accumulation target moving area for each same person, and each of the accumulations is received for each same person in response to a motion scene end signal. A storage target block region color information generation step for generating and outputting a color distribution corresponding to the target block region, and a search target region is set for each person based on the storage target moving region video signal, and the motion scene ends A search region setting step for receiving a signal and outputting a search target region video signal corresponding to the search target region for each person, and the search target corresponding to each color information of the storage target block region and the search target region A region color information storage control step for storing the region video signal in the region color information storage means, and acquiring the search target region video signal and outputting a display list A representative representative image list display step, a search target area color information generation step for generating and outputting a color distribution from a video signal included in the search target area based on the search target area video signal, and the storage target block The color distribution of each region is compared with the color distribution of the search target region, and a comparison step is performed for performing output according to the comparison result.

  The video search apparatus of the present invention is characterized in that there are a plurality of accumulation target moving areas. With this configuration, it is possible to simultaneously track and search for movements of a plurality of people.

  In the present invention, the video search is performed based on the motion in the video and the color of each block constituting the region of the motion. The video search can be performed more appropriately than when a video of a specific part of the object is required.

  In addition, the present invention can efficiently display a video of a person with similar clothing color simply by selecting the person image while visually confirming the person image displayed on the monitor when the user inputs a search target. It is possible to easily specify a search target for searching.

  Hereinafter, an image search apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 shows a block diagram of a video search apparatus according to the first embodiment of the present invention. In FIG. 1, a video search apparatus 100 captures a subject and generates and outputs a video signal. Image input units 101 a to 101 n (hereinafter, these image input units 101 a to 101 n are collectively referred to as “video input unit 101” as appropriate. The video signal from the corresponding video input unit 101 is input, the moving area (accumulation target moving area) in the video is extracted, and the video signal corresponding to the accumulating target moving area (accumulation target moving area video) A moving region extraction unit 102a to 102n that outputs a signal) (hereinafter, these moving region extraction units 102a to 102n are collectively referred to as a “moving region extraction unit 102” as appropriate) and an accumulation from the corresponding moving region extraction unit 102 The target moving area video signal is input, it is determined whether or not the accumulation target moving area satisfies a predetermined condition of being a person, and the condition of being a person is satisfied. The person determination means 103a to 103n (hereinafter, these person determination sections 103a to 103n are collectively referred to as “person determination section 103” as appropriate) for outputting the accumulation target moving area video signal in the case of The accumulation target moving area video signal from the unit 103 is input, the accumulation target moving area is divided into a plurality of block areas (accumulation target block areas), and video signals corresponding to the respective accumulation target block areas (accumulation target block area video signals) ) To output the region dividing units 104a to 104n (hereinafter, these region dividing units 104a to 104n are collectively referred to as “region dividing unit 104” as appropriate), and the respective accumulation target block region videos from the corresponding region dividing unit 104 A representative color calculation unit 105ano that inputs a signal and calculates and outputs a representative color of each accumulation target block area constituting the accumulation target moving area 105n (hereinafter, these representative color calculation units 105a to 105n are collectively referred to as “representative color calculation unit 105” as appropriate) and each accumulation target block area constituting the accumulation target moving area from each representative color calculation unit 105. A database (DB) 106 for storing representative colors, a keyboard 107 for a user to input instructions, and a video signal from the video input unit 101 are input, and a video to be searched according to an operation of the keyboard 107 by the user A search target area designating unit 108 for designating a predetermined area (search target area) and outputting a video signal (search target area video signal) corresponding to the search target area, and a search from the search target area designating unit 108 The target area video signal is input, and the search target area is divided into a plurality of block areas (search target block areas) to correspond to each search target block area. Region segmentation unit 109 that outputs a video signal (search target block region video signal) and a representative color of each search target block region constituting the search target region by inputting the search target block region video signal from the region segmentation unit 109 Representative color calculation unit 110 that calculates and outputs the representative color of each search target region block area constituting the search target area from the representative color calculation unit 110 and each accumulation target block constituting the accumulation target moving area in the DB 106 A comparison unit 111 that compares the representative colors of the regions and outputs a search result according to the comparison result, and a list display that generates and outputs a list of the search results in order to display the search results in the comparison unit 111 Unit 112, video selection unit 113 that selects and outputs a video signal to be recorded among video signals from each video input unit 101, and video from video selection unit 113 A compression unit 114 that compresses the video signal, a storage 115 that stores the compressed video signal from the compression unit 114, a video display instruction unit 116 that instructs video display in response to an operation of the keyboard 107 by the user, and a video display instruction An expansion unit 117 that expands the compressed video signal in the storage 115 in response to an instruction from the unit 116, and a display that displays a list of search results by the comparison unit 111 and a video corresponding to the video signal expanded by the expansion unit 117. Part 118.

  The operation of the video search apparatus 100 configured as described above will be described with reference to FIGS. First, the operation during accumulation of representative colors will be described with reference to FIG.

  The video input unit 101 shoots a subject and generates a video signal (S101). Further, the video input unit 101 outputs the generated video signal to the corresponding moving region extraction unit 102. For example, the video input unit 101a outputs the generated video signal to the moving region extraction unit 102a.

  When the video signal from the corresponding video input unit 101 is input, the moving region extraction unit 102 performs background difference processing for calculating a difference value between the video corresponding to the video signal and the background video stored in advance (S102). . Next, the moving region extraction unit 102 determines whether or not there is a motion in the video corresponding to the input video signal (S103). Specifically, the moving region extraction unit 102 determines that there is motion in the video if the difference value is larger than a predetermined value in the background difference processing in S102, and otherwise the motion is in the video. Judge that there is no. Note that the moving region extraction unit 102 performs processing other than background difference processing in S102, and determines whether or not there is motion in the video according to the result of processing other than background difference processing in S103. Also good.

  When there is no motion in the video corresponding to the video signal input to the moving region extraction unit 102, the operations after the subject shooting and video signal generation (S101) by the video input unit 101 are repeated.

  On the other hand, when there is a motion in the video corresponding to the video signal input to the motion region extraction unit 102, the motion region extraction unit 102 extracts a motion region (accumulation target motion region) in the video (S104). . Furthermore, the moving area extraction unit 102 outputs the accumulation target moving area video signal corresponding to the extracted accumulation target moving area to the corresponding person determination unit 103. For example, the moving region extraction unit 102a outputs an accumulation target moving region video signal corresponding to the extracted accumulation target moving region moving region to the human determination unit 103a.

  When the person determination unit 103 receives the accumulation target moving region video signal from the corresponding moving region extraction unit 102, the human determination unit 103 performs elliptical Hough processing on the image of the accumulation target moving region corresponding to the accumulation target moving region video signal. (S105). Next, the person determination unit 103 determines whether or not a condition that the accumulation target moving area corresponding to the input accumulation target moving area video signal is a person is satisfied (S106). Specifically, when the human determination unit 103 can detect an elliptical area that is likely to be a human face in the image of the accumulation target moving area by the ellipse hough processing in S105, the accumulation target moving area If it is determined that the condition of being a person is satisfied and an elliptical area that is likely to be a human face cannot be detected, the condition that the accumulation target moving area is a person is satisfied. Judge that there is no. In S105, the person determination unit 103 performs a process other than the elliptical Hough process (for example, a process for deriving the shape, size, etc. of the entire accumulation target moving area), and in S106, performs a process other than the elliptical Hough process. Depending on the result, it may be determined whether or not the condition that the accumulation target moving area is a person is satisfied.

  When the accumulation target moving area corresponding to the accumulation target moving area video signal does not satisfy the condition that the person is the person, the operation after the photographing of the subject and the generation of the video signal (S101) by the video input unit 101 is repeated. .

  On the other hand, if the condition that the accumulation target moving area corresponding to the accumulation target moving area video signal is a person is satisfied, the person determination unit 103 divides the input accumulation target moving area video signal into the corresponding area division. Output to the unit 104. For example, the person determination unit 103a outputs the input accumulation target moving area video signal to the area dividing unit 104a.

  The area dividing unit 104 receives the accumulation target moving area video signal from the corresponding person determination unit 103, and stores the accumulation target moving area corresponding to the accumulation target moving area video signal into a plurality of block areas (accumulation target block areas). Divide (S107). For example, when dividing the accumulation target moving area into four, the area dividing unit 104 counts the vertical pixels and the horizontal pixels of the accumulation target moving area, and calculates the vertical midpoint and the horizontal midpoint. Is identified. Further, the area dividing unit 104 divides the accumulation target moving area into two in the vertical direction with the middle point in the vertical direction as a division position, and divides the accumulation target moving area into two in the division direction with the middle point in the horizontal direction as a division position. The accumulation target moving area is divided into four accumulation target block areas. The number of divisions of the accumulation target moving area and the shape of the accumulation target block area are not particularly limited.

  Further, the region dividing unit 104 outputs a video signal (storage target block region video signal) corresponding to each storage target block region constituting the storage target moving region to the corresponding representative color calculation unit 105. For example, the area dividing unit 104a outputs the accumulation target block area video signal to the representative color calculating unit 105a.

  When the representative color calculation unit 105 inputs the accumulation target block region video signal corresponding to each accumulation target block region constituting the accumulation target moving region from the corresponding region division unit 104, the representative color calculation unit 105 based on the accumulation target block region video signal. Then, the representative color of each accumulation target block area constituting the accumulation target moving area is calculated (S108). Specifically, the representative color calculation unit 105 is determined according to a predetermined rule (a conversion rule that reduces the influence of luminance change so that the RGB color system is represented by the HSV color system) for colors appearing in the accumulation target block region. The color corresponding to the average value of the values or the color that appears most frequently in the accumulation target block area is calculated as the representative color of the accumulation target block area.

  Further, the representative color calculation unit 105 outputs the representative color of each accumulation target block area constituting the accumulation target moving area to the DB 106. The DB 106 accumulates the representative colors of the accumulation target block areas constituting the accumulation target moving area from the representative color calculation unit 105 (S109). The DB 106 associates with the representative color of each accumulation target block area constituting the accumulation target moving area, identifies the identification information (ID) of the camera 102 that shot the video including the accumulation target moving area, the shooting date and time, and the accumulation thereof. Thumbnail video data obtained by reducing the video corresponding to the target moving area may be accumulated.

  By repeating the processing of S101 to S109 described above, the representative color of each accumulation target block area constituting the accumulation target moving area is accumulated in the DB 106 for each of a plurality of accumulation target moving areas. .

Next, the operation at the time of video search will be described with reference to FIG.
When the user views an image displayed on the display unit 118 and operates the keyboard 107 to give an instruction (search instruction) for specifying a person in the image as a search target, the search area specifying unit 108 A search instruction by the user is accepted (S201). Next, the search area designation unit 108 inputs a video signal corresponding to the video displayed on the display unit 118 from the image input unit 101, and selects a moving area (search target area) designated by the user in the video. Extract (S202). Further, the search area designating unit 108 outputs a search target area video signal corresponding to the extracted search target area to the area dividing unit 109.

  The area dividing unit 109 receives the search target area video signal from the search area specifying unit 108, and divides the search target area corresponding to the search target area video signal into a plurality of block areas (search target block areas) (S203). ). For example, when dividing the search target area into four, the area dividing unit 109 counts the vertical pixels and the horizontal pixels of the search target area in the same manner as the above-described area dividing unit 104, and determines the middle of the vertical direction. Identify the point and the midpoint in the horizontal direction. Further, the area dividing unit 109 divides the search target area into two in the vertical direction with the vertical midpoint as the split position, and divides the search target area into two with the horizontal midpoint as the split position. The area is divided into four search target block areas. Note that the number of divisions of the search target area and the shape of the search target block area are not particularly limited. Further, the region dividing unit 109 outputs a video signal (search target block region video signal) corresponding to each search target block region constituting the search target region to the representative color calculation unit 110.

  When the representative color calculation unit 110 receives the search target block region video signal corresponding to each search target block region constituting the search target region from the region division unit 109, the representative color calculation unit 110 performs the search target based on the search target block region video signal. The representative color of each search target block area constituting the area is calculated (S204). Specifically, the representative color calculation unit 110, similar to the above-described representative color calculation unit 105, the color corresponding to the average value of the values determined according to a predetermined rule for the color appearing in the search target block area, and the search target block The color that appears most frequently in the area is calculated as the representative color of the search target block area. Further, the representative color calculation unit 110 outputs the representative color of each search target block area constituting the search target area to the comparison unit 111.

  When the comparison unit 111 inputs the representative color of each search target block area constituting the search target area from the representative color calculation unit 110, the comparison unit 111 represents the representative color of each storage target block area constituting one of the accumulation target moving areas from the DB 106. Is read (S205).

  Further, the comparison unit 111 represents a numerical value corresponding to the representative color of each accumulation target block area constituting the read accumulation target moving area and a numerical value corresponding to the representative color of each search target block area constituting the input search target area. Is calculated (S206).

  As the representative color, a numerical value corresponding to RGB color information or a numerical value corresponding to lightness, saturation, or the like is used. In this case, a numerical value is calculated such that the closer the color is, the smaller the difference in the numerical value is. For example, there are four accumulation target block areas (A1, A2, A3, A4) constituting the accumulation target moving area, the representative color value of the accumulation target block area A1 is a1, and the representative color of the accumulation target block area A2 is The numerical value is a2, the numerical value of the representative color of the storage target block area A3 is a3, and the numerical value of the representative color of the storage target block area A4 is a4. On the other hand, there are four search target block areas (storage) B1 corresponding to the target block area A1, B2 corresponding to the storage target block area A2, B3 corresponding to the storage target block area A3, B4 corresponding to the storage target block area A4), and the representative color of the search target block area B1 Is b1, the representative color value of the search target block area B2 is b2, the representative color value of the search target block area B3 is b3, the search target block area Numerical value of the representative color of the B4 consider the case for which it is given by the b4). In this case, the comparison unit 111 represents the numerical value of the representative color of each accumulation target block area with the coordinates (a1, a2, a3, a4) of the four-dimensional space, and represents the numerical value of the representative color of each search target block area to the four-dimensional space. It is expressed by space coordinates (b1, b2, b3, b4), and the Euclidean distance between these coordinates is calculated. Note that the comparison unit 111 may calculate a distance other than the Euclidean distance.

  Next, the comparison unit 111 calculates a numerical value corresponding to the representative color of each accumulation target block area constituting the calculated accumulation target moving area, and a numerical value corresponding to the representative color of each search target block area constituting the search target area. It is determined whether or not the distance is within a predetermined threshold (S207). Here, it is desirable that the user can freely set the threshold.

  When the distance between the numeric value corresponding to the representative color of each accumulation target block area constituting the accumulation target moving area and the numeric value corresponding to the representative color of each search target block area constituting the search target area is within a predetermined threshold The comparison unit 111 includes data related to the representative color of each accumulation target block area constituting the accumulation target moving area (for example, identification information (ID) of the camera 102 that has captured the video including the accumulation target moving area, shooting date and time). , Thumbnail video data obtained by reducing the video of the accumulation target moving area) is output to the list display unit 112 as a search result. The list display unit 112 adds data related to the representative color of each accumulation target block area constituting the accumulation target moving area from the comparison unit 111 to the search result list (S208).

  After the process of adding to the search result list by the list display unit 112 (S208), or the numerical value corresponding to the representative color of each accumulation target block area constituting the accumulation target moving area by the comparison unit 111 and the search target area After determining that the distance from the numerical value corresponding to the representative color of each search target block area exceeds a predetermined threshold (negative determination in S207), the comparison unit 111 configures all accumulation target moving areas in the DB 106. It is determined whether or not the representative color of each accumulation target block area is read (S209). When there is a representative color of each accumulation target block area that constitutes an accumulation target moving area that has not yet been read in the DB 106, each accumulation target block area that constitutes the accumulation target moving area that has not been read by the comparison unit 111 The operations after the reading of the representative color (S205) are repeated.

  On the other hand, when the comparison unit 111 reads the representative colors of the accumulation target block areas constituting all the accumulation target moving areas in the DB 106, operations after the search instruction reception (S201) by the search area designation unit 108 Is repeated. Further, the search result list generated by the list display unit 112 is displayed on the display unit 118, and the user can recognize the search result.

  Note that FIG. 3 illustrates the case where the user designates the moving area as the search target area. However, the search target person may be stationary and the stationary person area may be designated as the search target area. Similarly, video search can be performed. In this case, the search area designating unit 108 extracts an area (search target area) instructed by the user as in FIG. 3, and searches the search target area video signal corresponding to the extracted search target area into the area dividing unit 109. Output to.

  As described above, in the video search apparatus 100, when the moving area in the video shot by the video input unit 101 satisfies the condition that it is a person, the moving area is divided into a plurality of accumulation target block areas. Above, the representative color of each accumulation target block area is calculated and accumulated in the DB 106. In the video search, the search target area specified by the user is divided into a plurality of search target block areas, and the representative colors of each search target block area are calculated, and each search constituting the search target area is performed. Data related to the representative color of each accumulation target block area constituting the accumulation target area that approximates the representative color of the target block area is shown to the user as a search result. In other words, the video search is performed not only based on the motion in the captured video but also based on the color of the motion region. Therefore, when searching for a video based only on the presence or absence of motion, Video search can be performed more appropriately than when video is required.

  Next, FIG. 4 shows a block diagram of a video search apparatus according to the second embodiment of the present invention. In FIG. 4, the same components as those of the first embodiment are denoted by the same reference numerals as those of the video search apparatus 100, and the description thereof is omitted.

  The video search apparatus 400 inputs each accumulation target block area video signal from the corresponding area dividing unit 104, extracts the color feature amount of each accumulation target block area constituting the accumulation target moving area, and generates a color distribution. The output color information generation units 401a to 401n (hereinafter, these color information generation units 401a to 401n are collectively referred to as “color information generation unit 401” as appropriate) and the accumulation target moving area from each color information generation unit 401 are configured. The database (DB) 402 for storing the color distribution of each storage target block area to be input and each search target block video signal from the area dividing unit 109 are input, and the color feature amount of each search target block constituting the search target area is calculated. A color information generation unit 403 that extracts, generates and outputs a color distribution, and a color of each search target block area constituting the search target area from the color information generation unit 403 Comparing the color distribution of each storage object block region constituting the storage target motion area in the cloth and DB 402, and a comparator 404 for outputting a search result according to the comparison result.

  The operation of the video search apparatus 400 configured as described above will be described with reference to FIGS. First, the operation during color distribution accumulation will be described with reference to FIG. The same processing steps as those of the first embodiment are denoted by the same reference numerals, and the operation description thereof is simplified.

  The video input unit 101 shoots a subject and generates a video signal (S101). Further, the video input unit 101 outputs the generated video signal to the corresponding moving region extraction unit 102.

  When the video signal from the corresponding video input unit 101 is input, the moving region extraction unit 102 performs background difference processing for calculating a difference value between the video corresponding to the video signal and the background video stored in advance (S102). . Next, the moving region extraction unit 102 determines whether or not there is a motion in the video corresponding to the input video signal (S103).

  When there is no motion in the video corresponding to the video signal input to the moving region extraction unit 102, the operations after the subject shooting and video signal generation (S101) by the video input unit 101 are repeated.

  On the other hand, when there is a motion in the video corresponding to the video signal input to the motion region extraction unit 102, the motion region extraction unit 102 extracts a motion region (accumulation target motion region) in the video (S104). . Furthermore, the moving area extraction unit 102 outputs the accumulation target moving area video signal corresponding to the extracted accumulation target moving area to the corresponding person determination unit 103.

  When the person determination unit 103 receives the accumulation target moving region video signal from the corresponding moving region extraction unit 102, the human determination unit 103 performs elliptical Hough processing on the image of the accumulation target moving region corresponding to the accumulation target moving region video signal. (S105). Next, the person determination unit 103 determines whether or not a condition that the accumulation target moving area corresponding to the input accumulation target moving area video signal is a person is satisfied (S106).

  When the accumulation target moving area corresponding to the accumulation target moving area video signal does not satisfy the condition that the person is the person, the operation after the photographing of the subject and the generation of the video signal (S101) by the video input unit 101 is repeated. .

  On the other hand, if the condition that the accumulation target moving area corresponding to the accumulation target moving area video signal is a person is satisfied, the person determination unit 103 divides the input accumulation target moving area video signal into the corresponding area division. Output to the unit 104.

  The area dividing unit 104 receives the accumulation target moving area video signal from the corresponding person determination unit 103, and stores the accumulation target moving area corresponding to the accumulation target moving area video signal into a plurality of block areas (accumulation target block areas). Divide (S107).

  Further, the area dividing unit 104 outputs a video signal (accumulation target block area video signal) corresponding to each accumulation target block area constituting the accumulation target moving area to the corresponding color information generation unit 401.

  When the color information generation unit 401 receives the accumulation target block region video signal corresponding to each accumulation target block region constituting the accumulation target moving region from the corresponding region division unit 104, the color information generation unit 401, based on these accumulation target block region video signals. Then, color information of each accumulation target block area constituting the accumulation target moving area is generated (S501). Specifically, the color information generation unit 401 acquires the RGB value of each pixel of the accumulation target block region video as color information, converts it to the HSV space (see Non-Patent Document 1), and uses H and S. A histogram (hereinafter referred to as color distribution) is created. Although the color space is converted from RGB to HSV here, it can be used in other color spaces (for example, XYZ, YCrCb, etc.) by general usage (generating histograms on the XY plane and CrCb plane).

  Furthermore, the color information generation unit 401 outputs the color distribution of each accumulation target block area constituting the accumulation target moving area to the DB 402. The DB 402 accumulates the color distribution of each accumulation target block area constituting the accumulation target moving area from the color information generation unit 401 (S502). The DB 402 associates with the color distribution of each accumulation target block area constituting the accumulation target moving area, identifies the identification information (ID) of the camera 102 that shot the video including the accumulation target moving area, the shooting date and time, and the accumulation thereof. Thumbnail video data obtained by reducing the video corresponding to the target moving area may be accumulated.

  By repeating the processing of S101 to S107 and S501 and S502, the DB 402 accumulates the color distribution of each accumulation target block area constituting the accumulation target moving area for each of the plurality of accumulation target moving areas. Will be.

Next, an operation during video search will be described with reference to FIG.
When the user views an image displayed on the display unit 118 and operates the keyboard 107 to give an instruction (search instruction) for specifying a person in the image as a search target, the search area specifying unit 108 A search instruction by the user is accepted (S201). Next, the search area designation unit 108 inputs a video signal corresponding to the video displayed on the display unit 118 from the image input unit 101, and selects a moving area (search target area) designated by the user in the video. Extract (S202). Further, the search area designating unit 108 outputs a search target area video signal corresponding to the extracted search target area to the area dividing unit 109.

  The area dividing unit 109 receives the search target area video signal from the search area specifying unit 108, and divides the search target area corresponding to the search target area video signal into a plurality of block areas (search target block areas) (S203). ). Further, the region dividing unit 109 outputs a video signal (search target block region video signal) corresponding to each search target block region constituting the search target region to the color information generation unit 403.

  When the search target area video signal corresponding to the search target area from the area dividing unit 109 is input, the color information generation unit 403 generates a color distribution of the search target area based on these search target area video signals (S601). . Specifically, the color information generation unit 403 acquires the RGB value of each pixel of the accumulation target block area video as color information and converts it into the HSV space, as in the color information generation unit 401 described above, and converts the color distribution. create. Although the color space is converted from RGB to HSV here, it can be used in other color spaces (for example, XYZ, YCrCb, etc.) by general usage (generating histograms on the XY plane and CrCb plane). Further, the color information generation unit 403 outputs the color distribution of the search target area to the comparison unit 404.

  When the comparison unit 404 receives the color distribution of the search target region from the color information generation unit 403, the comparison unit 404 reads the color distribution of each storage target block region constituting any of the storage target moving regions from the DB 402 (S602).

  Further, the comparison unit 404 calculates the degree of coincidence by comparing the color distribution of each accumulation target block area constituting the read accumulation target moving area and the color distribution of each search target block area constituting the input search target area. (S603).

  Since the color distribution is a two-dimensional histogram using H and S, for example, a histogram intersection (see Patent Document 3) may be used to calculate the degree of coincidence. According to this method, the degree of coincidence when the color distributions are completely matched is a value obtained by integrating all frequencies included in the color distribution of each search target block area constituting the search target area. The coincidence becomes 0.

  The degree of coincidence between the color distribution of each accumulation target block area constituting the accumulation target moving area and the color distribution of each search target block area constituting the search target area is obtained, and these are collectively referred to as partial coincidence. The degree of coincidence is calculated by adding all the partial coincidences. Alternatively, it is also possible to calculate a partial matching degree by selecting a storage target block region to be compared according to the purpose, and calculate only the calculated partial matching degrees.

  Although a histogram intersection is used here, other methods for comparing two color distributions can be considered. For example, a color distribution position having a frequency greater than a preset threshold value is extracted, and these color distribution positions are extracted. Among them, a method of comparing the color distribution position and the frequency ratio is also conceivable.

  Next, the comparison unit 404 determines that the degree of coincidence between the calculated color distribution of each accumulation target block area constituting the accumulation target moving area and the color distribution of each search target block area constituting the search target area is within a predetermined threshold. It is determined whether or not (S604). Here, it is desirable that the user can freely set the threshold.

  When the degree of coincidence between the color distribution of each accumulation target block area constituting the accumulation target moving area and the color distribution of each search target block area constituting the search target area is within a predetermined threshold, the comparison unit 404 stores Data related to the accumulation target moving area having each accumulation target block area constituting the target area (for example, the identification information (ID) of the camera 102 that captured the video including the accumulation target moving area, the shooting date and time, the accumulation target movement) The thumbnail image data obtained by reducing the region image is output to the list display unit 112 as a search result. The list display unit 112 adds data related to the accumulation target moving area having the accumulation target block areas constituting the accumulation target area from the comparison unit 404 to the search result list (S605).

  After the addition processing to the search result list (S605) by the list display unit 112, or the color distribution of each accumulation target block area constituting the accumulation target moving area by the comparison unit 404 and each search object constituting the search target area After determining that the degree of coincidence with the color distribution of the block area exceeds a predetermined threshold value (No determination in S604), the comparison unit 404 determines each of the accumulation target block areas constituting all the accumulation target moving areas in the DB 402. It is determined whether or not the color distribution has been read (S606). When there is a color distribution of each accumulation target block area that constitutes an accumulation target moving area that has not yet been read out in the DB 402, each accumulation target block area that constitutes an accumulation target moving area that has not been read by the comparison unit 404 The operation after reading the color distribution (S602) is repeated.

  On the other hand, when the comparison unit 404 reads out the color distribution of each accumulation target block area constituting all the accumulation target moving areas in the DB 402, the operation after the search instruction reception (S201) by the search area designation unit 108 Is repeated. Further, the search result list generated by the list display unit 112 is displayed on the display unit 118, and the user can recognize the search result.

  Note that FIG. 6 illustrates the case where the user designates the moving area as the search target area. However, the search target person may be stationary and the stationary person area may be designated as the search target area. Similarly, video search can be performed. In this case, the search area designating unit 108 extracts an area (search target area) instructed by the user, as in FIG. 6, and outputs a search target area video signal corresponding to the extracted search target area as a color information generation unit. Output to 403.

  As described above, in the video search device 400, when the moving area in the video captured by the video input unit 101 satisfies the condition that it is a person, the moving area is divided into a plurality of accumulation target block areas. Above, the color distribution of each accumulation target block area is generated and accumulated in the DB 402. In the video search, the color distribution of the search target area specified by the user is generated and related to the storage target areas having the respective storage target block areas constituting the storage target area similar to the color distribution of the search target area. Data to be displayed is shown to the user as a search result. In other words, the video search is performed not only based on the motion in the captured video but also based on the color of the motion region. Therefore, when searching for a video based only on the presence or absence of motion, Video search can be performed more appropriately than when video is required.

  Next, FIG. 12 shows a video search apparatus according to the third embodiment of the present invention. In FIG. 12, the same components as those of the second embodiment are denoted by the same reference numerals as those of the video search device 400, and the description thereof is omitted.

  In FIG. 12, the video search apparatus 1100 according to the embodiment of the present invention includes a color information generation unit 1101 that extracts and outputs color information from colors to be searched according to the operation of the keyboard 107 by the user, The average color information of the continuous distribution area is generated from the color distribution of each accumulation target block constituting the accumulation target moving area, the search target color information from the color information generation unit 1101 and the average of each generated accumulation target block A comparison unit 1102 that compares the color information and outputs a search result corresponding to the comparison result;

  The operation of the video search apparatus 1100 configured as described above will be described with reference to FIGS. The same processing steps as those in the first and second embodiments are denoted by the same reference numerals, and the operation description thereof is simplified. Further, since the operation at the time of storing color information is the same as that of the video search device 400, the description with reference to FIG.

The operation at the time of video search will be described with reference to FIG.
When the user operates the keyboard 107 to output a search start request, the color information generation unit 1101 outputs a display request to the display unit 118 for the color information image held. The color information image may be a color palette used for color settings generally used by software on a PC, or a plurality of color information set in advance in the search device. When the user looks at the displayed color information image and gives an instruction (search instruction) to specify a color to be searched, the color information generation unit 1101 receives the search instruction from the user (S1201). The search instruction is performed by designating one point on the color information image. The color information generation unit 1101 acquires the RGB value of the pixel corresponding to one point designated by the user from the image displayed on the display unit 118, converts it to the HSV space, and generates color information (S1202). The result is output to the comparison unit 1102. When the search target color information from the color information generation unit 1101 is input, the comparison unit 1102 reads the color distribution of each accumulation target block area configuring any accumulation target moving area from the DB 402 (S602). Subsequently, the comparison unit 1102 extracts only color information having a frequency greater than a preset fixed value from the color distribution of each accumulation target block read from the DB 402, and further uses two-dimensional coordinates using H and S. Representative color information is generated in units of color distribution regions that are continuous above (S1203). Further, the comparison unit 1102 compares the input search target color information with the representative color information generated from the color distribution of each accumulation target block area constituting the read accumulation target moving area, and calculates the similarity (S1204). Since the search target color information and the representative color information are each one point on the two-dimensional coordinates using H and S, the distance between these two points may be obtained.

  Next, the comparison unit 1102 determines whether the similarity between the representative color information calculated from the color distribution of each accumulation target block area constituting the calculated accumulation target moving area and the search target color information is within a predetermined threshold. Is determined (S1205). When the similarity between the representative color information generated from the color distribution of each accumulation target block area constituting the accumulation target moving area and the search target color information is within a predetermined threshold, the comparison unit 1102 configures the accumulation target area. Data related to the accumulation target moving area having each accumulation target block area (for example, the identification information (ID) of the camera 102 that captured the video including the accumulation target moving area, the shooting date and time, and the video of the accumulation target moving area) The reduced thumbnail image data) is output to the list display unit 112 as a search result. The list display unit 112 adds data related to the accumulation target moving area having the accumulation target block areas constituting the accumulation target area from the comparison unit 1102 to the search result list (S605).

  After the addition processing to the search result list by the list display unit 112 (S605), or representative color information generated from the color distribution of each accumulation target block area constituting the accumulation target moving area by the comparison unit 1102, the search target color After determining that the coincidence similarity with the information exceeds a predetermined threshold value (No determination in S1205), the comparison unit 1102 determines the color distribution of each accumulation target block area constituting all accumulation target moving areas in the DB 402. It is determined whether the data has been read (S606). When there is a color distribution of each accumulation target block area that constitutes an accumulation target moving area that has not yet been read in the DB 402, each accumulation target block area that constitutes the accumulation target moving area that has not been read by the comparison unit 1102 The operation after reading the color distribution (S602) is repeated.

  On the other hand, when the comparison unit 1102 reads the color distribution of each accumulation target block area constituting all the accumulation target moving areas in the DB 402, the operation after the search instruction reception (S1201) by the color information generation unit 1101 Is repeated. Further, the search result list generated by the list display unit 112 is displayed on the display unit 118, and the user can recognize the search result.

  In FIG. 13, the case where the user designates a color palette or preset color information as a search target has been described. However, the DB 402 stores the color distribution of each accumulation target block area constituting the accumulation target area. When this is done, the color distribution may be integrated separately, and the color information with high frequency of occurrence may be selected from a higher fixed number in advance to generate the search target color information. The user selects one color information from the search target color information generated in this way and gives a search instruction.

  As described above, in the video search apparatus 1100, when the moving area in the video captured by the video input unit 101 satisfies the condition that it is a person, the moving area is divided into a plurality of accumulation target block areas. Above, the color distribution of each accumulation target block area is generated and accumulated in the DB 402. In video search, the storage target area having each storage target block area constituting the storage target area having a representative color similar to the search target color information by the user specifying one search target color information. Data related to is shown to the user as a search result. That is, even when there is no image to be searched, only one point is specified from the color information image indicated by the search device, and the video search is performed not only by the motion in the captured video but also by the color of the region of the motion. Therefore, video search can be performed more appropriately than when video search is performed based only on the presence or absence of motion, or when a video of a specific part of an object is required for video search.

  Next, a video search apparatus according to a fourth embodiment of the present invention is shown in FIG. Constituent elements similar to those in the first embodiment are denoted by the same reference numerals as those of the video search apparatus 100, and description thereof is omitted.

  In FIG. 7, the video search apparatus 700 according to the embodiment of the present invention inputs an accumulation target moving area video signal from the moving area extraction unit 102 corresponding to each moving object, and the accumulation target moving area is predetermined. It is determined whether or not the condition of being a person is satisfied, and if the condition of being a person is satisfied, the accumulation target moving area video signal is output for each moving object and the accumulation is continuously input. It is determined whether or not the target moving area meets a predetermined condition of the same person. If it is determined that the target moving area is the same person, the person is tracked and the person being tracked exists in the video. The person motion scene determination units 701a to 701n (hereinafter referred to as “person motion scene determination units 701a to 701n”, which output a motion scene end signal when they are not used, Each of the accumulation target block area video signals from the corresponding area dividing unit 104, and the color feature amount of each accumulation target block area constituting the accumulation target moving area is extracted for each person. When the motion scene end signal is input from the corresponding person motion scene determination unit 701, the color information generation units 702a to 702n that generate and output color distribution for each person (hereinafter, these color information generation units 702a to 702n) are stored. Are collectively referred to as “color information generation unit 702” as appropriate, and an accumulation target moving region video signal is input from the corresponding person motion scene determination unit 701, and whether or not a predetermined search target region setting condition is satisfied If the search target area setting condition is satisfied, the search target area video signal is stored for each person, and the corresponding person motion scene determination unit 70 is stored. When a motion scene end signal is input from 703a to 703n (hereinafter, these search region setting units 703a to 703n are collectively referred to as “search region setting unit 703”) that outputs search target region video signals, A database (DB) 704 for storing the color distribution for each person from the information generation unit 702 and the search target area video signal from each search area setting unit 703; a user instruction input unit 705 for a user to input an instruction; A person representative image list display unit 706 that reads a search target area video signal as a person representative image from the DB 704 in response to a search execution request by a user, generates a person representative image list, and outputs the person representative image list, and corresponds to the person representative image selected by the user. A search target selection unit 707 that reads out and outputs a search target region video signal from the DB 704, and a search target selection unit A color information generation unit 708 that inputs a search target region video signal from 707 to extract a color feature amount of the search target region, generates and outputs a color distribution, and a color distribution of the search target region from the color information generation unit 708 The comparison unit 709 compares the color distribution of each accumulation target block area constituting the accumulation target moving area in the DB 704 and outputs a search result according to the comparison result.

  The operation of the video search apparatus 700 configured as described above will be described with reference to FIGS. The same processing steps as those in the first and second embodiments are denoted by the same reference numerals, and the operation description thereof is simplified. First, the operation at the time of storing color information will be described with reference to FIGS.

  As shown in FIG. 8, the video input unit 101 shoots a subject and generates a video signal (S101). Further, the video input unit 101 outputs the generated video signal to the corresponding moving region extraction unit 102. When the video signal from the corresponding video input unit 101 is input, the moving region extraction unit 102 performs background difference processing for calculating a difference value between the video corresponding to the video signal and the background video stored in advance (S102). .

  Next, the moving region extraction unit 102 determines whether or not there is a motion in the video corresponding to the input video signal (S103). When there is no motion in the video corresponding to the video signal input to the motion region extraction unit 102, the motion region extraction unit 102 notifies the person motion scene determination unit 701 that there is no motion.

  When a signal notifying that there is no motion is input from the corresponding moving region extraction unit 102, the human motion scene determination unit 701 determines whether the tracking mode is set (S808).

  When it is determined that the mode is not the tracking mode, the operation after the photographing of the subject and the generation of the video signal (S101) by the video input unit 101 is repeated.

  On the other hand, when it is determined that the tracking mode is set, the human motion scene determination unit 701 outputs a motion scene end signal to the corresponding color information generation unit 702 and the corresponding search area setting unit 703.

  The DB 704 accumulates the color distribution corresponding to each accumulation target block area for each person output by the color information generation unit 702 and the search area setting unit 703, and further stores the search target area video corresponding to the search target area in association with each other. (S809). Also turn off the tracking mode.

  On the other hand, when there is a motion in the video corresponding to the video signal input to the motion region extraction unit 102, the motion region extraction unit 102 extracts all motion regions (accumulation target motion regions) in the video (S104). ). Furthermore, the moving region extraction unit 102 outputs the accumulation target moving region video signals corresponding to all the extracted accumulation target moving regions to the corresponding person movement scene determination unit 701.

  When all the accumulation target moving area video signals from the corresponding moving area extraction unit 102 are input, the human motion scene determination unit 701 generates an ellipse with respect to the accumulation target moving area video corresponding to the accumulation target moving area video signal. Hough processing is performed (S105). Next, it is determined whether or not the condition that the accumulation target moving area corresponding to all input accumulation target moving area video signals is a person is satisfied (S106).

  As shown in FIG. 9, if the condition that the accumulation target moving area corresponding to the accumulation target moving area video signal is not a person is satisfied, whether or not the processing for all accumulation target moving areas has been completed. Is determined (S810). When it is determined that the processing for all the accumulation target moving areas has been completed, the operation after the photographing of the subject and the generation of the video signal (S101) by the video input unit 101 is repeated. If it is determined that the processing for all the accumulation target moving areas has not been completed, the operations after the elliptical Hough process (S105) by the human motion scene determination unit 701 are repeated.

  On the other hand, if the condition that the accumulation target moving area corresponding to the accumulation target moving area video signal is a person is satisfied, the person motion scene determination unit 701 applies the continuously input accumulation target moving area video signal to the accumulation target moving area video signal. It is determined whether or not the corresponding person in the accumulation target moving area satisfies the condition that it is the same person as the person in the accumulation target moving area being tracked (S801). Specifically, using the stored location / movement information of the accumulation target area, the movement direction of the accumulation target moving area is within the preset movable range of the person, and the continuous movement direction has changed significantly. If not, it is determined that they are the same person.

  If it is determined that the condition of being the same person is not satisfied, tracking is started as a new tracking person (S802). Specifically, an ID for identifying a person is created, and the input barycentric coordinates of the accumulation target moving area are stored as position information in the video. Note that the barycentric coordinates of the accumulation target moving area may use other information as long as the position of the area in the video can be represented. Further, the tracking mode is turned on (S8021).

  On the other hand, if it is determined that the condition of being the same person is satisfied, the person motion scene determination unit 701 searches the input accumulation target moving region video signal for the corresponding region dividing unit 104 and the corresponding search region setting unit 703. The movement distance from the previous frame, the movement direction, the position information, etc. are stored as the person tracking information from the stored position information of the accumulation target moving area (S803).

  The area dividing unit 104 receives all the accumulation target moving area video signals from the corresponding person motion scene determination unit 701, and sets the accumulation target moving area corresponding to the accumulation target moving area video signal as a plurality of block areas (accumulation targets). It is divided into block areas) (S107). Further, the area dividing unit 104 outputs a video signal (accumulation target block area video signal) corresponding to each accumulation target block area constituting the accumulation target moving area to the corresponding color information generation unit 702.

  When the accumulation target block region video signal corresponding to each accumulation target block region constituting all the accumulation target moving regions from the corresponding region dividing unit 104 is input, the color information generation unit 702 receives the accumulation target block region video signal. Based on this, the color information of each accumulation target block area constituting the accumulation target moving area is extracted, and the color information is integrated for each accumulation target block area of the same person stored (S804). Specifically, the color information generation unit 702 obtains RGB values of each pixel in the accumulation target block area as color information, converts the RGB value into an HSV space, and uses H and S to generate a histogram (hereinafter referred to as color distribution). Further, the color distribution corresponding to a plurality of accumulation target block areas constituting the accumulation target moving area is integrated for each accumulation target block area for each same person who is being tracked. Although the color space is converted from RGB to HSV here, it can be used in other color spaces (for example, XYZ, YCrCb, etc.) by general usage (generating histograms on the XY plane and CrCb plane).

  Finally, the color information generation unit 702 stores a color distribution corresponding to each accumulation target block for each tracking person (S807). On the other hand, when the motion scene end signal is input from the corresponding person motion scene determination unit 701, the color information generation unit 702 normalizes the color distribution corresponding to each accumulation target block region for each tracked person and stores the DB 704. Output to.

  When all the accumulation target moving area video signals are input from the corresponding person motion scene determination unit 701, the search area setting unit 703 determines whether or not the search target area can be set based on a preset rule ( S805). Specifically, using the position and size of the accumulation target moving area corresponding to the input accumulation target moving area signal, it is determined whether or not the search target area can be set based on a preset threshold.

  When it is determined that the search target area cannot be set, the search area setting unit 703 terminates the process, and when it is determined that the search target area can be set, the search area setting unit 703 In step S806, the search target area is set. Finally, a search target area video signal corresponding to the search target area stored for each tracked person is stored (S807). At this time, the search target area video signal corresponding to the search target area that has already been stored is compared with the search target area video signal corresponding to the new search target area, and is close to a preset condition (for example, the search target area If the average of the luminance / saturation of the entire area video signal is closer to a preset value), the search area video signal corresponding to the best search area can always be saved. .

  By repeating the processing of S101 to S106 and S801 to S810 described above, the DB 704 stores the color distribution of each accumulation target block area constituting the accumulation target moving area and the search for each of the plurality of accumulation target moving areas. The search target area video corresponding to the target area is accumulated.

Next, an operation during video search will be described with reference to FIG.
When the user makes a search request using the user instruction input unit 705, the person representative image list display unit 706 receives the request from the user (S901). The person representative image list display unit 706 acquires a search target area video signal stored in the DB 704 as a person representative image, creates a list (person representative image list), and outputs the list to the display unit 118. The display unit 118 A representative image list is displayed. The user views the person representative image list displayed on the display unit 118 and inputs an instruction to select one of the person representative images from the user instruction input unit 705.

  When a user representative image selection instruction is input from the user instruction input unit 705, the search target selection unit 707 acquires and outputs a search target region and a search target region video signal corresponding to the person representative image selected by the user from the DB 704. (S902).

  When the search target region video signal is input from the search target selection unit 707, the color information generation unit 708 extracts color information of the search target region (S903). Specifically, the color information generation unit 702 obtains RGB values of each pixel in the accumulation target block area as color information, converts the RGB values into an HSV space, and creates a histogram (color distribution) using H and S. To do. Further, histograms corresponding to a plurality of accumulation target block areas constituting the accumulation target area for each same person being tracked are integrated for each accumulation target block area. Although the color space is converted from RGB to HSV here, it may be converted into another color space (for example, XYZ, YCrCb, etc.) to generate a histogram (XY plane, CrCb plane). Further, the color information generation unit 708 outputs the color distribution to the comparison unit 709.

  When the comparison unit 709 receives the color distribution of the search target region from the color information generation unit 708, the comparison unit 709 reads the color distribution of each storage target block region constituting any storage target moving region from the DB 704 (S602).

  Further, the comparison unit 709 compares the color distribution of each accumulation target block area constituting the read accumulation target moving area with the input color distribution of the search target area to calculate the degree of coincidence (S904).

  Since the color distribution is a two-dimensional histogram using H and S, for example, a histogram intersection may be used to calculate the degree of coincidence. When the color distributions completely match, the total frequencies included in the color distribution are integrated, and when the color distributions are not completely matched, the matching degree is zero.

  The degree of coincidence (partial coincidence) between the color distribution of each accumulation target block area constituting the accumulation target moving area and the color distribution of the search target area is obtained, and the degree of coincidence is calculated by adding all the partial coincidences. Here, the degree of coincidence is calculated by accumulating all the partial coincidences.However, according to the purpose, the accumulation target block area is selected to calculate the partial coincidence, and only the calculated partial coincidence is calculated. It is conceivable that the degree of coincidence is calculated by integration, or all partial coincidences are calculated, and the highest partial coincidence is selected as the coincidence.

  Next, the comparing unit 709 determines whether or not the degree of coincidence between the calculated color distribution of each accumulation target block area constituting the accumulation target moving area and the color distribution of the search target area is within a predetermined threshold. (S604). Here, it is desirable that the user can freely set the threshold.

  When the degree of coincidence between the color distribution of each accumulation target block area constituting the accumulation target moving area and the color distribution of the search target area is within a predetermined threshold, the comparison unit 709 displays each accumulation target constituting the accumulation target area. Data related to the accumulation target moving area having a block area (for example, identification information (ID) of the camera 102 that shot the video including the accumulation target moving area, shooting date and time, thumbnail video obtained by reducing the video of the accumulation target moving area) Is output to the list display unit 112 as a search result. The list display unit 112 adds data related to the accumulation target moving area having the accumulation target block areas constituting the accumulation target area from the comparison unit 709 to the search result list (S605).

  After the addition processing (S605) to the search result list by the list display unit 112, or the color distribution of each accumulation target block area constituting the accumulation target moving area by the comparison unit 709 and the color distribution of the search target area After the determination that the degree exceeds the predetermined threshold (negative determination in S604), the comparison unit 709 determines whether or not the color distribution of each accumulation target block area constituting all the accumulation target moving areas in the DB 704 has been read. Determination is made (S606). When there is a color distribution of each accumulation target block area that constitutes an accumulation target moving area that has not yet been read in the DB 704, each accumulation target block area that constitutes an accumulation target moving area that has not been read by the comparison unit 709. The operation after reading the color distribution (S602) is repeated.

  On the other hand, when the comparison unit 709 reads the color distribution of each accumulation target block area constituting all the accumulation target moving areas in the DB 704, the person representative image list display unit 706 receives a search instruction (S901) and thereafter. Is repeated. Further, the search result list generated by the list display unit 112 is displayed on the display unit 118, and the user can recognize the search result.

  As described above, in the video search apparatus 700, when the moving area in the video captured by the video input unit 101 satisfies the condition that it is a person, the moving area is divided into a plurality of accumulation target block areas. Above, the color information of each accumulation target block area is extracted, and if it is determined that they are the same person, the color distribution of the accumulation target block areas constituting each moving area of the series of motion scenes of that person is generated At the same time, the search target area video signal corresponding to the best search target area from the series of motion scenes of the person is stored in the DB 704. In video search, when a user designates a human image to be searched from the person representative image list displayed on the display unit 118, a color distribution of the human search target area is automatically generated, and the search target area Data related to the accumulation target area having each accumulation target block area that constitutes the accumulation target area similar to the color distribution is shown to the user as a search result. In other words, not only can the search be performed based on the motion in the captured video and the color of the motion area, but also the search is performed using the search target region that is set in only one of the human motion scenes. The video search can be performed more appropriately than when the video search is performed based only on the presence or absence of the image, or when the video of the specific part of the target object is required for the video search. Furthermore, the user designates a desired person image as the search target area without searching for the desired search target area while playing the video when setting the search area, or repeatedly selecting the search target area by repeatedly viewing the video of the same person. Thus, a stable search result can always be obtained without obtaining a different result each time the search is executed.

  In the above description, the search target selection unit 707 obtains the search target area corresponding to the person representative image designated by the user and the search target area video signal from the DB 704 and generates the color distribution during the video search. However, it is also possible to perform the same operation when the search area setting unit 703 performs generation of the color distribution of the search target area and stores the color distribution and the search target area video signal in the DB 704 in association with each other during video recording.

  In the first to fourth embodiments, as shown in FIG. 11, the region dividing unit 104 sets the accumulation target region output from the person motion scene determination unit 701 as the person region (a), and determines the person motion scene determination. By outputting the ellipse position and size obtained by the ellipse hough process by the unit 701, the area dividing unit 104 divides the area as shown in FIG. The block area video signal is output, and the color information generation unit 702 generates and outputs a color distribution for each accumulation target block area. Similarly, in the search area setting unit 703, as shown in FIG. 11, the search target area is divided and a search target block area video signal corresponding to the search target area is output. The color distribution and the search target block area video signal corresponding to the search target area are stored in association with each other. The comparison unit 709 compares the color distributions of the search target block areas with the corresponding storage target block areas, so that when the user selects a person representative image as a search target on the monitor, the head, upper body, lower body, shoes, etc. If a representative image is selected by specifying a selection item for a divided area, it is possible to perform a search based on a combination of a partial clothing color of a person or a clothing color of a plurality of people.

  In the fourth embodiment, the size of the accumulation target block may be fixed while the tracking mode is ON.

  As described above, the video search apparatus and the video search method according to the present invention can appropriately perform video search, and search targets for efficiently searching for videos of persons with similar clothing colors. It has the effect that it can be specified easily, and is useful as a video search apparatus and video search method used for video surveillance applications such as personal identification.

Block diagram of the video search apparatus in the first embodiment of the present invention Flowchart of operation at the time of accumulation of representative colors of the video search device in the first embodiment of the present invention Flowchart of the operation at the time of video search of the video search device in the first embodiment of the present invention Block diagram of a video search apparatus in the second embodiment of the present invention Flowchart of operation at the time of accumulation of representative colors of the video search apparatus in the second embodiment of the present invention Flowchart of the operation at the time of video search of the video search device in the second embodiment of the present invention The block diagram of the video search device in the 4th Embodiment of this invention Flowchart of operation at the time of accumulation of representative color of video search device in fourth embodiment of the present invention Flowchart of operation at the time of accumulation of representative color of video search device in fourth embodiment of the present invention Flowchart of the operation at the time of video search of the video search device in the fourth embodiment of the present invention (A) Schematic diagram showing a person area in the fourth embodiment of the present invention (b) Schematic diagram showing a person area divided into color information extraction units in the fourth embodiment of the present invention Block diagram of a video search apparatus in a third embodiment of the present invention Flowchart of the operation at the time of video search of the video search device in the third embodiment of the present invention Block diagram of a conventional monitoring system Block diagram of a conventional personal identification device

Explanation of symbols

100, 400, 700, 1100 Video search device 101a-101n Video input unit 102a-102n Moving region extraction unit 103a-103n Person determination unit 104a-104n, 109 Region division unit 105a-105n, 110 Representative color calculation unit 106, 402, 704 DB
107 Keyboard 108 Search area designation unit 111, 404, 709, 1102 Comparison unit 112 List display unit 113 Video selection unit 114 Compression unit 115 Storage 116 Video display instruction unit 117 Expanding unit 118 Display unit 401a to 401n, 403, 702a to 702n, 708, 1101 Color information generation unit 701a to 701n Human motion scene determination unit 703a to 703n Search area setting unit 705 User instruction input unit 706 Person representative image list display unit 707 Search target selection unit

Claims (23)

An accumulation target moving area extracting means for extracting an accumulation target moving area in the first video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing means for outputting;
A storage target block area representative that derives and outputs a representative color of each of the storage target block areas constituting the storage target moving area based on the storage target block area video signal corresponding to each of the storage target block areas Color derivation means;
Accumulation target block area representative color accumulation means for accumulating each representative color of the accumulation target block area;
Search target area extraction means for extracting a search target area in the second video and outputting a search target area video signal corresponding to the search target area;
A search for dividing the search target area into search target block areas based on the search target area video signal, and outputting search target block area video signals corresponding to the search target block areas constituting the search target area Target block area dividing means;
Retrieval block area representative color for deriving and outputting a representative color of each search block area constituting the search area based on the search block area video signal corresponding to each of the search block areas Deriving means;
An image search apparatus comprising: a comparison unit that compares each representative color of the accumulation target block area with each representative color of the search target block area and performs output according to the comparison result.   2. The video search apparatus according to claim 1, wherein the comparison means derives a difference between each representative color of the accumulation target block area and each representative color of the search target block area as the comparison result. .   Based on the accumulation target moving area video signal, it is determined whether or not the accumulation target moving area satisfies a predetermined condition as a condition for being a person, and the accumulation target moving area satisfies the predetermined condition. 3. The video search apparatus according to claim 1, further comprising a person determination unit that outputs the accumulation target moving area video signal to the accumulation target block area dividing unit when the condition is satisfied.   The accumulation target block region representative color deriving unit is configured to generate a color corresponding to an average value of values obtained by a conversion method for reducing an influence of a luminance change on a color appearing in the accumulation target block region, and a frequency of appearance of the obtained value. 4. The video search apparatus according to claim 1, wherein one of the highest colors is derived as a representative color of the accumulation target block area. 5.   The search target block area representative color deriving means includes a color corresponding to an average value of values obtained by a conversion method for reducing an influence of a luminance change on a color appearing in the search target block area, and a frequency at which the obtained value appears. 4. The video search apparatus according to claim 1, wherein one of the highest-colored colors is derived as a representative color of the search target block area. 5. A video search method for searching for a moving object in a first video based on information in a second video,
An accumulation target moving area extracting step of extracting an accumulation target moving area in the first video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing step to be output; and
A storage target block area representative that derives and outputs a representative color of each of the storage target block areas constituting the storage target moving area based on the storage target block area video signal corresponding to each of the storage target block areas A color derivation step;
An accumulation target block area representative color accumulation control step of accumulating each representative color of the accumulation target block area in an accumulation target block area accumulation unit;
A search target region extracting step of extracting a search target region in the second video and outputting a search target region video signal corresponding to the search target region;
A search for dividing the search target area into search target block areas based on the search target area video signal, and outputting search target block area video signals corresponding to the search target block areas constituting the search target area A target block area dividing step;
Retrieval block area representative color for deriving and outputting a representative color of each search block area constituting the search area based on the search block area video signal corresponding to each of the search block areas A derivation step;
A video search method comprising: a comparison step of comparing each representative color of the accumulation target block area with each representative color of the search target block area and performing output according to the comparison result.   The video search method according to claim 6, wherein the comparison step derives a difference between each representative color of the accumulation target block area and each representative color of the search target block area as the comparison result. . Based on the accumulation target moving area video signal, it is determined whether or not the accumulation target moving area satisfies a predetermined condition defined as a condition that the accumulation target moving area is a person, and the accumulation target moving area satisfies the predetermined condition A human determination step of outputting the accumulation target moving area video signal when
The accumulation target block area dividing step divides the accumulation target moving area into accumulation target block areas based on the accumulation target moving area video signal when the accumulation target moving area video signal is output in the human determination step. 8. The video search method according to claim 6, wherein a storage target block area video signal corresponding to each of the storage target block areas constituting the storage target moving area is output. An accumulation target moving area extracting means for extracting an accumulation target moving area in the first video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing means for outputting;
Based on the accumulation target block area video signal corresponding to each of the accumulation target block areas, a color distribution is extracted from the video signals included in the accumulation target block areas constituting the accumulation target moving area and output. Accumulation target block area color information generating means;
Storage target block area color information storage means for storing color information of each of the storage target block areas;
Search target area extraction means for extracting a search target area in the second video and outputting a search target area video signal corresponding to the search target area;
A search for dividing the search target area into search target block areas based on the search target area video signal, and outputting search target block area video signals corresponding to the search target block areas constituting the search target area Target block area dividing means;
Search that extracts and outputs a color distribution from video signals included in each previous search target block area constituting the search target area based on the search target block area video signal corresponding to each of the search target block areas Target block area color information generating means;
A video search apparatus comprising: comparing means for comparing each color distribution of the accumulation target block area and the color distribution of the search target block area and performing output according to the comparison result.   The comparison means derives a degree of coincidence of color appearance frequencies of each color distribution of the accumulation target block area and each color distribution of the search target block area as the comparison result. 9. The video search device according to 9.   The comparison means derives, as the comparison result, a degree of coincidence of color appearance frequencies between each color distribution of the accumulation target block area and one of the color distributions of the search target block area. 9. The video search device according to 9.   Based on the accumulation target moving area video signal, it is determined whether or not the accumulation target moving area satisfies a predetermined condition defined as a condition that the accumulation target moving area is a person, and the accumulation target moving area satisfies the predetermined condition 12. The video search apparatus according to claim 9, further comprising a person determination unit that outputs the accumulation target moving area video signal to the accumulation target block area dividing unit. An accumulation target moving area extracting means for extracting an accumulation target moving area in the first video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing means for outputting;
Based on the accumulation target block area video signal corresponding to each of the accumulation target block areas, a color distribution is extracted from the video signals included in the accumulation target block areas constituting the accumulation target moving area and output. Accumulation target block area color information generating means;
Storage target block area color information storage means for storing color information of each of the storage target block areas;
Search target color information generating means for extracting one search target point in the second video and extracting and outputting color information of the search target point based on a search target region video signal corresponding to the search target point When,
Comparing means that generates representative color information from each color distribution of the accumulation target block region, compares the representative color information with the search target color information, and performs output according to the comparison result. A featured video search device.   The comparison means derives, as the comparison result, the similarity of the appearance frequency of the color between the representative color information generated from each color distribution of the accumulation target block region and the color information of the search target point. The video search device according to claim 13. An accumulation target moving area extracting means for extracting an accumulation target moving area in a video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
A person who identifies the same person from the continuous frame images in the video based on the accumulation target moving area video signal, outputs the previous accumulation target moving area video signal, and outputs a motion scene end signal when there is no movement A motion scene determination means;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing means for outputting;
Based on the accumulation target block area video signal corresponding to each of the accumulation target block areas, a color feature amount is calculated from the video signal included in each of the accumulation target block areas constituting the accumulation target moving area for each same person. Storage target block area color information generating means for extracting and generating a color distribution corresponding to each of the storage target block areas for each same person in response to the motion scene end signal;
A search for setting a search target area for each person based on the accumulation target moving area video signal, and outputting a search target area video signal corresponding to the search target area for each same person in response to the motion scene end signal Region setting means;
Area color information storage means for storing the search target area video signal corresponding to each color information of the storage target block area and the search target area;
A person representative image list display means for acquiring the search target area video signal from the area color information storage means and outputting a display list;
Search target area color information generating means for generating and outputting a color distribution from a video signal included in the search target area based on the search target area video signal;
A video search apparatus comprising: a comparison unit that compares each color distribution of the accumulation target block area with the color distribution of the search target area and performs output according to the comparison result.   The video search apparatus according to claim 9, wherein the accumulation target block area dividing unit determines a division position based on a shape of a person. A video search method for searching for a moving object in a first video based on information in a second video,
An accumulation target moving area extracting step of extracting an accumulation target moving area in the first video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing step to be output; and
Based on the accumulation target block area video signal corresponding to each of the accumulation target block areas, a color distribution is extracted from the video signals included in the accumulation target block areas constituting the accumulation target moving area and output. An accumulation target block area color information generation step;
An accumulation target block area color information accumulation control step of accumulating each color information of the accumulation target block area in an accumulation target block area accumulation unit;
A search target region extracting step of extracting a search target region in the second video and outputting a search target region video signal corresponding to the search target region;
A search for dividing the search target area into search target block areas based on the search target area video signal, and outputting search target block area video signals corresponding to the search target block areas constituting the search target area A target block area dividing step;
Search that extracts and outputs a color distribution from video signals included in each previous search target block area constituting the search target area based on the search target block area video signal corresponding to each of the search target block areas A target block area color information generation step;
A video search method comprising: a comparison step of comparing each color distribution of the accumulation target block region with the color distribution of the search target block region and performing output according to the comparison result.   The comparison step derives a degree of coincidence of color appearance frequencies of each color distribution of the accumulation target block area and each color distribution of the search target block area as the comparison result. 17. The video search method according to 17.   The comparison step derives, as the comparison result, a degree of coincidence of color appearance frequencies between each color distribution of the accumulation target block area and one of the color distributions of the search target block area. 17. The video search method according to 17. Based on the accumulation target moving area video signal, it is determined whether or not the accumulation target moving area satisfies a predetermined condition defined as a condition that the accumulation target moving area is a person, and the accumulation target moving area satisfies the predetermined condition A human determination step of outputting the accumulation target moving area video signal when
The accumulation target block area dividing step divides the accumulation target moving area into accumulation target block areas based on the accumulation target moving area video signal when the accumulation target moving area video signal is output in the human determination step. 20. The video search method according to claim 17, wherein a storage target block area video signal corresponding to each of the storage target block areas constituting the storage target moving area is output. A video search method for searching for a moving object in a first video based on information in a second video,
An accumulation target moving area extracting step of extracting an accumulation target moving area in the first video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing step to be output; and
Based on the accumulation target block area video signal corresponding to each of the accumulation target block areas, a color distribution is extracted from the video signals included in the accumulation target block areas constituting the accumulation target moving area and output. An accumulation target block area color information generation step;
An accumulation target block area color information accumulation control step of accumulating each color information of the accumulation target block area in an accumulation target block area accumulation unit;
Search target color information generation for extracting one search target point in the second video and extracting and outputting color information of the search target point based on a search target region video signal corresponding to the search target point Steps,
A comparison step of generating representative color information from each color distribution of the accumulation target block region, comparing the representative color information with the search target color information, and performing output according to the comparison result. A featured video search method.   The comparison step derives the similarity of the appearance frequency of the representative color information generated from each color distribution of the accumulation target block region and the color information of the search target point as the comparison result. The video search method according to claim 21. An accumulation target moving area extracting step of extracting an accumulation target moving area in the video and outputting an accumulation target moving area video signal corresponding to the accumulation target moving area;
A person who identifies the same person from the continuous frame images in the video based on the accumulation target moving area video signal, outputs the previous accumulation target moving area video signal, and outputs a motion scene end signal when there is no movement A motion scene determination step;
Based on the accumulation target moving area video signal, the accumulation target moving area is divided into accumulation target block areas, and accumulation target block area video signals corresponding to the accumulation target block areas constituting the accumulation target moving area are obtained. An accumulation target block area dividing step to be output; and
Based on the accumulation target block area video signal corresponding to each of the accumulation target block areas, a color feature amount is calculated from the video signal included in each of the accumulation target block areas constituting the accumulation target moving area for each same person. An accumulation target block area color information generating step for extracting and generating a color distribution corresponding to each of the accumulation target block areas for each same person in response to the motion scene end signal;
A search for setting a search target area for each person based on the accumulation target moving area video signal, and outputting a search target area video signal corresponding to the search target area for each same person in response to the motion scene end signal An area setting step;
An area color information accumulation control step of accumulating in the area color information accumulation means the color information of each of the accumulation target block areas and the search target area video signal corresponding to the search target area;
A person representative image list display step of acquiring the search target area video signal and outputting a display list;
A search target area color information generation step for generating and outputting a color distribution from a video signal included in the search target area based on the search target area video signal;
A video search method comprising: a comparison step of comparing each color distribution of the accumulation target block region with the color distribution of the search target region and performing output according to the comparison result.

Images