JPH06223179A - Method and device for deciding whether or not there is specific object in color image and color moving picture cut retrieval display system using the device - Google Patents

Abstract

PURPOSE:To improve the reliability and performance of technology regarding color moving picture detection by preventing an omission of detection of a frame containing an object from a color moving picture. CONSTITUTION:Plural partial areas of similar colors are extracted as color groups from the color image of the specific object and a couple of adjacent color groups showing the feature of the specific object are selected among the extracted color groups; when more than a specific number of the couples of color groups are included in an optional color image, it is decided that the specific object is included in the optional color image, thus deciding whether or not the specific object is present in the color image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device such as a multimedia processing system or a video device, and a color image search technique in those devices, and more particularly to a color moving image in a color moving image editing work. The present invention relates to a method for determining the presence / absence of a specific object in a color image suitable for efficiently searching for a desired scene from an image, an apparatus therefor, and a color moving image cut search / display system using this apparatus.

[0002]

2. Description of the Related Art In recent years, moving image information has dramatically increased in importance as an information medium, and there is a demand for an editing support environment in which general people can easily create and utilize moving image information. In editing a moving image, a function of efficiently searching for a scene needed by an editor is required from many moving image materials.

[0003] In moving image retrieval, it is general to create an index of moving images by using a keyword or the like.
In this search, the search target can be designated relatively easily by inputting text information serving as an index. However, there is an essential problem that you cannot search using keywords that are not registered as an index. Especially for creative activities such as video editing, this is a big complaint for users.
There is also a problem that the work of registering the index must be performed by a human.

For the purpose of solving such a problem, for example, a technique proposed by the proposers of the present invention is described in "Information Processing Society of Japan, Vol. 33, N." edited by Information Processing Society of Japan.
o. 41992 "pp. 543-550, "Automatic Indexing and Object Retrieval in Color Video Images"
(Nagasaka, Tanaka). With this technology, all you have to do is specify one color image that contains the object you are looking for, and all the frames in which the object appears (a series of still images that form a moving image. Thirty
(Which is composed of one frame) in a color moving image. The details of this technique will be described below.

As a technique for determining whether or not a designated object exists in an image, there are various algorithms including pattern matching. However, in the search of a moving image, the posture and the position of the target object change, so that the size and shape of the target object in the frame change every moment. Therefore, it must be robust to such deformations. In addition, high speed is indispensable for processing a huge number of frames of a moving image.

The object retrieval technique described in the above-mentioned document is designed to satisfy the above-mentioned conditions. The algorithm is roughly divided into the following two basic processes. (A) A process of extracting characteristic information from an image (reference image) of a designated object. (B) A process of determining whether or not the extracted information is included in an arbitrary frame.

In the process (a), a pair of adjacent colors (color pair) on the color image is extracted as information for characterizing the image of the object. Such a color pair is local feature information of the object, and is information unique to the object image that does not change in size or orientation of the object. In the process (b), it is checked whether or not the reference color pair selected in the process (a) exists in the frame to be determined. At this time, if N (for example, about N = 16) reference color pairs that are the characteristic information of the target object are simultaneously present in the frame, it is determined that the target object is included in this frame. . Details of the process (a) will be described below with reference to FIG. 17, and details of the process (b) will be described with reference to FIG.

FIG. 17 is a flowchart showing an example of a conventional method of selecting characteristic information of a color moving image object.
The feature extraction process from the color reference image is performed as follows. First, a searcher is requested to specify one color image (hereinafter, simply referred to as an image) in which an object appears (step 17).
01), only the image area part of the object is cut out as a reference image (step 1702). Next, the reference image is divided into rectangular areas (hereinafter referred to as cells) (step 170).
3) The following processing is performed for each cell (step 170).
4). First, a color histogram is obtained (step 170).
5) All the colors (called cell colors) whose frequencies in the histogram are greater than the set threshold are extracted and registered in the list CC (steps 1706 to 1708). An example of the data structure of the list CC is shown in Table 2 below.

[Table 2]

Next, the following processing is performed for each cell on the reference image (step 1709). First, a pair (color pair) is created by selecting one color each from the cell color of a cell and the cell color of each adjacent cell (step 1710). Create this color pair for the number of cell colors that two cells have,
All combinations are registered in the list CP (step 171).
1). At this time, the cell color combination in each cell is also registered in the CP. Furthermore, N pairs of colors with the highest appearance frequency are selected from the list CP and registered in the list RCP (step 17).
12). These N color pairs are used as information that characterizes the object (referred to as reference color pairs). Tables 3 and 4 below show examples of the data structures of the list CP and the list RCP, respectively.

[Table 3]

[Table 4]

FIG. 18 is a flowchart showing an example of a conventional method for searching a color moving image object. This flowchart is for determining whether or not the frame to be determined includes the object characterized in FIG.
The frame to be judged is divided into cells (1801), the following processing is performed for each cell, and the cell color of each cell is obtained (step 1802). First, a color histogram is obtained (step 1803), and all colors (cell colors) whose frequency is higher than a set threshold value in the histogram are extracted and registered in the list CC (steps 1804 to 1806).

Next, the following processing is performed for each of the color pairs registered in the list RCP to obtain the number of appearances in the above frame (step 1807). First, each cell on the frame (step 1808) has one of two colors forming a color pair as a cell color (step 18).
09), and extracts all the cells having the other color of the color pair as the cell color of the cell itself or any of the eight adjacent cells (hereinafter, effective cell) (step 1810). , The total number of valid cells in the frame is calculated (step 1811).

If the number of valid cells exceeds the set threshold (step 1812), the color pair is counted as a valid color pair (step 1813), and the total number of valid color pairs is compared with the set threshold (step 1813). In step 1814), it is determined whether or not the object exists in the frame (steps 1815 and 1816).

FIG. 19 is an explanatory diagram showing a search example using the algorithm shown in FIG. In this figure, 19
Reference numeral 1 is a frame including an object (girl) serving as a reference for search, a partial area surrounded by a frame is a reference image of the object, 192 is a frame including the search object, and 193 is a search. It is a frame that does not include the object, 1
94 and 195 are distribution maps in which the presence of 16 types of reference color pairs is detected. In the example of this figure, the number of reference color pairs determined to exist is 14 in the distribution map 194 and 8 in the distribution map 195. Therefore, if the threshold value is set between 8 and 14, it can be correctly determined that the object exists in the distribution map 194 and does not exist in the distribution map 195.

However, as a result of experiments on various image samples, the conventional algorithm described above is shown in Table 1 below.
As can be seen from the data in the column marked as “conventional”, it was found that a large number of over-detection and over-detection occurred. As a result of visual observation during the experiment, the following two points are considered to be the most significant factors causing these malfunctions. (B) How to select the reference color pair. (B) The color change of the object due to the color temperature change of the light source. In the following, these two points will be considered in detail.

The influence of the selection method of the reference color pair of (A) will be described. Many reference images of an object can be divided into a plurality of partial areas of uniform color. Taking the woman shown in the image in the next FIG. 20 as an example, it is possible to consider partial regions mainly for black-colored, flesh-colored, and green-colored components for each component such as hair, face, and clothes. At this time, a pair of colors with low similarity such as a pair of skin color and green (hereinafter,
A different color pair) is the boundary between the elements (in this case, the boundary between the face and clothes)
Exists only in a very small area.

However, there are many pairs of similar colors (hereinafter, pairs of the same color) such as pairs of greenish colors depending on the size of each partial region. Therefore, when only the color pair having the highest appearance frequency is used as the reference color pair, the same color pair occupies the majority of the reference color pairs. Then, as will be described with reference to FIG. 20, the same-color pair often appears on the frame as compared with the different-color pair, even if the target object does not appear. In this case, in order to prevent omission of detection, the number of effective color pairs must be determined with a small threshold value, which results in increased over detection.

FIG. 20 is an explanatory diagram showing an example of reference color pairs extracted from a reference image. In this figure, for each of the 16 reference color pairs extracted from the reference image 291, the RGB color space coordinates 292, 293 of the two colors forming the pair are shown.
And the color difference 294 between the two colors is shown. RGB
The coordinate values of the color space coordinates 292, 293 represent the brightness of each of the RGB components and take values from 0 to 7. Also, the color difference 294
Is simply the distance between the RGB coordinates. The color pairs with a large color difference 294 are ranked 4th, 3rd, 5th, 8th and 10th.
There is only one, and the third color pair is the same achromatic color pair, and the color difference is close to human perception. From this result, it can be seen that the same color pairs occupy most of the reference color pairs.

The influence of the change in the color temperature of the light source (b) will be described. Even in the image where the same object appears, if the objects are under the light sources with different color temperatures,
Images of objects are represented in different colors. Therefore, it is not appropriate to use the reference color pair extracted from the reference image captured under a certain light source as it is for the determination of the frame captured under the light source having a different color temperature. This is one of the major causes of detection failure.

On the other hand, as for a moving image search technique, for example, there is one that detects a cut transition based on a change in correlation between adjacent frames of a moving image. And
Using such a technique, as shown in FIG.
Systems have been developed to improve the efficiency of video editing.

FIG. 21 is a block diagram showing the structure of a conventional color moving image cut search / display system. In the figure, reference numeral 2101 denotes a moving image reproduction device that includes devices such as a video tape recorder and a write-once laser disk player, and that converts video signals from these devices into digital image data (moving image) and outputs the digital image data. The device, 2102
The cut detection device 2103 for detecting the cut transition of the moving image sent from the moving image reproduction device 2101,
Based on the cut detecting operation of the cut detecting device 2202, an icon screen for each cut is generated from the moving image sent from the moving image reproducing device 2101, and the display 210 is displayed.
The cut display control device 2105 displayed on the display unit 4105 is the display 2104 generated by the cut display control device 2103.
It is a cut display screen composed of an icon group displayed above.

When the moving image is supplied from the moving image reproducing apparatus 2101, the cut detecting apparatus 2202 detects the cut transition based on the change in the correlation between adjacent frames of the moving image. The cut display control device 2103 displays the reduced images of the frames at the beginning of each cut as icons in order based on the operation of detecting the transition of each cut by the cut detection device 2202, and
The time length of each cut is recorded, and the time length is displayed as the thickness in the depth direction of each icon.

As described above, by displaying the icons each consisting of the time length of each cut and the image in order, the temporal structure of the video can be visualized, and the editing for each cut can be performed efficiently. It will be possible. However, this technique is a search with cuts as the minimum unit, and cannot support more detailed editing work for characters and the like.

[0023]

The problem to be solved by the prior art is that in the prior art, the color change of the object due to the selection of the reference color pair and the change of the color temperature of the light source in the technique of determining the frame including the specific object. Is not taken into consideration, and it is not possible to prevent detection omission of a frame containing an object from a color moving image, and in a color moving image cut search display system, editing of a color moving image is operated in cut units. It is limited and cannot support more detailed editing work. The object of the present invention is to solve the problems of these conventional techniques, to accurately detect a frame containing an object from a color moving image, and to improve the reliability and performance of the technology related to color moving image detection. Another object of the present invention is to provide a method for determining the presence or absence of a specific object in a color image and a device therefor capable of supporting a more detailed editing operation of a color moving image, and a color moving image cut search and display system using this device.

[0024]

In order to achieve the above object, a method and apparatus for determining the presence / absence of a specific object in a color image according to the present invention, and a color moving image cut search / display system using this apparatus include (1) From the color image of a specific object, a pair of adjacent colors that characterize this specific object is extracted, and an arbitrary color image is searched based on the extracted color pair. In the method for determining the presence / absence of a specific object in a color image for determining whether or not a specific object is included, a plurality of partial areas of similar colors are formed as a color group from the color image of the specific object. When a plurality of color groups are selected, a pair of adjacent color groups that shows a characteristic of a specific object is selected, and when the number of the color group pairs is larger than a predetermined number in an arbitrary color image, In this any color image And judging that contains a specific object. Further, (2) in the method for determining the presence or absence of a specific object in a color image according to (1) above,
As a pair of adjacent color groups that show the characteristics of a specific object, select a different color group pair that is a pair of color groups of different shades, and this different color group pair is more than a predetermined number in a given color image. When present, it is characterized in that it is determined that the specific object is included in this arbitrary color image. Also,
(3) In the method for determining the presence / absence of a specific target object in a color image according to (1) above, as a pair of adjacent color groups showing a characteristic of the specific target object, a different color group is a pair of color groups having different shades. A pair and an identical color group pair, which are pairs of color groups having the same hue, are selected, and if the same color group pair and the different color group pair are present in a color image more than a predetermined number, It is characterized in that it is determined that a specific object is included in the color image. (4) In the method for determining the presence / absence of a specific object in a color image according to (3) above, the presence / absence of a specific object in an arbitrary color image with the same color group pair is determined by the different color group pair. It is characterized in that it is performed based on a number larger than a predetermined number to be used. Also, (5)
In the specific object presence / absence determining method in the color image according to (3) or (4), the same color group pair other than the vicinity of the region in which the different color group pair is distributed is invalidated, and the different color group pair and It is characterized in that the presence or absence of a specific object in an arbitrary color image is determined by using the same color group pair in the vicinity of the area in which the different color group pair is distributed. (6) In the method of determining the presence or absence of a specific object in a color image according to (5), when selecting a different color group pair from the specific object, the first color group forming the different color group pair is selected. The direction to the second color group is obtained as the reference direction, and the area of the first color group forming the different color group pair existing in an arbitrary color image is expanded in the direction opposite to the reference direction to obtain the first mask area. Is expanded in the reference direction in the regions of the second color group forming the different color group pair existing in the arbitrary color image to generate the second mask regions, respectively. Presence or absence of a specific object in an arbitrary color image using the same color group pair having the same color tone as the first color group and the same color group pair having the same color tone as the second color group in the second mask area Is characterized in that (7) In the method for determining the presence or absence of a specific object in a color image according to any one of (1) to (6) above,
Based on the change in color temperature between the light source of the color image of the specific object and the light source of the arbitrary color image, the hue, saturation, brightness, etc. of the image of the specific object is corrected, and the corrected image is used. , Selecting a pair of color groups showing a characteristic of a specific object. Also, (8) above (1) to (7)
In the specific object presence determination method in the color image according to any one of the, based on the color temperature change between the light source of the color image of the specific object and the light source of the arbitrary color image, from the color image of the specific object The hue of each color group that forms a pair of adjacent color groups that are characteristic of the extracted particular object,
It is characterized in that colors such as saturation and lightness are corrected and the presence or absence of a specific object in an arbitrary color image is determined by using the corrected color group. Also, (9) above (1)
The method for determining the presence / absence of a specific object in a color image according to any one of (8) to (8), based on the color temperature change between the light source of the color image of the specific object and the light source of the arbitrary color image, The present invention is characterized in that colors such as hue, saturation, and lightness are corrected, and the presence or absence of a specific object in the corrected image is determined using the corrected image.
In addition, (10) a pair of adjacent colors that characterize the specific object is extracted from the color image of the specific object, and an arbitrary color image is searched based on the extracted color pair,
In the specific object presence / absence determining device in the color image for determining whether or not a specific object is included in the arbitrary color image, a partial area of a similar color is extracted from the color image of the specific object. A color group extraction unit that extracts a plurality of color groups, and a unique color selection unit that selects, from the plurality of color groups extracted by the color group extraction unit, a pair of adjacent color groups that show the characteristics of a specific object. , The pair of color groups selected in this unique color selection section is
When there are more than a predetermined number in an arbitrary color image, a presence / absence determining unit that determines that a specific object is included in the arbitrary color image is provided, and an arbitrary color group is used to determine whether It is characterized in that the presence or absence of a specific object in the color image is determined. (11) In the specific object presence / absence determining device in the color image according to (10), the user can arbitrarily select and combine the color image according to any one of (1) to (9) above. A determination method storage unit for storing the specific object presence / absence determination method is provided, and based on the stored contents of the determination method storage unit, the above (1) to (9)
The method for determining the presence / absence of a specific object in a color image according to any one of 1 to 3 above is used to determine the presence / absence of a specific object in an arbitrary color image. (12) A continuous color moving image is divided in units of cuts, and the color image of the first frame of each cut is set to the front for each divided cut, and the time length of each cut is set to the first frame. In a color moving image cut search display system displaying a cut display screen group consisting of a rectangular parallelepiped indicated by the depth from, the presence or absence of a specific object in the color image according to (10) or (11) above from within the cut. Based on the determination result by the determination device, a specific frame detection unit that detects a color moving image frame that includes a specific object, and the first of the continuous specific color moving image frame group detected by this specific frame detection unit. Instead of the color image of the first frame of the cut containing this frame, the color image of the frame is replaced by the corresponding image of this frame on the cut display screen. A certain frame image display unit for displaying the position of the depth provided,
An image of a color moving image frame including a specific object is preferentially displayed on the cut display screen. (13) In the color moving image cut search and display system according to (12), a rectangular parallelepiped corresponding to a time length of a continuous specific color moving image frame group based on the image displayed on the specific frame image display unit It is characterized in that a specific frame group display section for coloring or highlighting the depth of is provided, and a section in which a color moving image frame group including a specific target object in a cut is present is displayed. Also, (14) above (12), or
In the color moving image cut search display system according to any one of (13), the specific frame image display unit reduces and displays the image of the first frame of the frame group including the specific object. Also, (15) above (1
In the color moving image cut search display system according to any one of 2) to (14), the reduced image of the first frame of the frame group including the specific object and the time chart of this frame group are associated with each other. It is characterized in that an object screen display unit for displaying is provided. (16) In the color moving image cut search / display system according to (12), instead of the specific frame image display unit, a first of each of a plurality of frame groups each including a different specific object is provided.
A multiple object display unit that displays reduced frame images and graphically displays the time length of each frame group including each reduced image at a position in the depth direction corresponding to the cut display screen including this frame group. It is characterized in that the ratio of the time length of the color moving image frame group in which a plurality of specific objects are present is individually displayed in the entire color moving image. (17) In the color moving image cut search / display system according to any one of (14) to (16), a specific search target based on a user's selection operation for the plurality of displayed reduced images is performed. It is characterized in that a designated frame detection unit for detecting an object is provided, and display control of the cut display screen corresponding to the specific object detected by the designated frame detection unit is selectively performed.

[0025]

In the present invention, a group (color group) in which similar colors are collected is made into a pair (color group pair), and based on this color group pair, a frame containing an object is detected from a color moving image. To do. Further, the color of the color (unique color) peculiar to the image or the object is corrected, and then the frame is judged. As a result, it is possible to stably detect a pair of color groups even with respect to a difference in color temperature of the light source, and it is possible to greatly reduce omission of detection. Also, paying attention to the distribution characteristics of the pair of different color groups and the pair of the same color group, the pair of the same color group and the pair of different color groups are treated separately, and the former has a high threshold for determining an effective color pair. By setting, excessive detection can be reduced. Furthermore, in determining whether or not an object is present in a frame, it is possible to reduce excessive detection by also considering the adjacent direction between the color group distribution regions. By using such a specific object presence / absence determining technique in the color image, it is possible to search the color moving image in frame units more detailed than in cut units.

[0026]

Embodiments of the present invention will now be described in detail with reference to the drawings. 1 and 2 are flowcharts showing a first embodiment of a method of determining the presence or absence of a specific object in a color image according to the present invention, and FIG. 3 is a block diagram showing the configuration of an apparatus used for the execution. is there. In FIG. 3, 1 is a computer for determining the presence or absence of a specific object in a color image of the present invention, and 2 is a CR for displaying an output screen of the computer 1.
A display such as T, 3 is a moving image reproducing device such as an optical disk or a video deck, 4 is an A / D converter for converting an analog signal into a digital signal, and 5 is a connecting control signal between the moving image reproducing device 3 and the computer 1. Control line, 6 is an external storage device such as a hard disk, 7 is an input device such as a mouse, 8a to 8e are interfaces (described as I / F in the figure) for controlling connection between the computer 1 and peripheral devices, 9 Is a CPU that controls the processing of the computer 1.
(Central processing unit) 10 is a memory directly accessed by the CPU 9.

Video signals output from the moving image reproducing apparatus 3 are sequentially converted into digital image data by the A / D converter 4 and sent to the computer 1. Inside the computer 1, the digital image data enters the memory 10 via the interface 8b and is processed by the CPU 9 according to the program stored in the memory 10.

Each frame of the moving image handled by the moving image reproducing apparatus 3 is numbered (frame number) in order from the beginning of the moving image. By sending this frame number from the computer 1 to the moving image reproducing apparatus 3 via the control line 5,
The moving image of the scene is reproduced. It should be noted that various types of information are stored in the external storage device 6 as necessary for processing. Memory 10
The various types of lists created by the processing described below are stored in, and are referred to as necessary.

With such a configuration, in consideration of the following points, the processing relating to the method of determining the presence or absence of a specific object in a color image of the present invention is performed to avoid omission of detection. That is,
When the object moves from the sun to the shade, the colors that represent the object are mostly low-brightness colors (shadow colors). Collect them as one group (color group) and treat them as the same color. As a result, when the light source changes such as in the sun or in the shade, the total number of color groups is always secured to some degree. Moreover, pairs of such color groups (color group pairs) are also robust to such changes in the light source. Then, when the image area of the object is divided into partial areas of colors that are regarded as the same group, the color group becomes a feature amount that represents each partial area. Further, the adjacency relationship of the color groups is determined by the adjacency relationship between the partial areas represented by the color group. Hereinafter, the processing operation of the method for determining the presence / absence of a specific object in a color image of the present invention, particularly regarding a different color group pair
This will be described with reference to FIGS. 1 and 2.

First, the processing for forming the different color group pair for reference in FIG. 1 will be described. The searcher is made to specify one image showing the object (step 101), and a plurality of partial areas having substantially the same color are divided from the image area of the object (step 102). For example, when the object is a person, it is roughly divided into three partial areas for each of the same colors, hair, face, and clothes. The following process is performed for each of these partial regions r (step 103).

First, a color histogram is taken (step 1
04), N high-frequency colors among similar colors are sequentially selected and grouped to form a color group, and as shown in Table 5 below,
It is registered in the list CG (r) (step 105).

[Table 5] Then, these color groups are combined based on the adjacency relationship of the corresponding partial areas to form a pair (reference color group pair). That is, a pair of all adjacent partial regions (r1,
For r2) (step 106), the color group list CG
A pair of (r1) and CG (r2) is created and registered in the list RCGP of different color group pairs as shown in Table 6 (step 107).

[Table 6] In this way, for example, two pairs of hair and face and face and clothes are created.

Next, the search process according to the present invention using the different color group pairs thus obtained will be described with reference to FIG.
The frame to be judged is divided into cells (step 20
1) The following processing is performed on each cell to obtain the cell color of each cell (step 202). First, a color histogram is obtained (step 203), and all colors (cell colors) whose frequency is higher than the set threshold value are extracted (steps 204, 205) and registered in the list CC (step 206).

Next, the following processing is performed for each of the reference color group pairs registered in the list RCGP obtained in FIG. 1, and the reference color group pairs (CG (r1), C
The number of existing G (r2)) is calculated (step 207). First, of the cells C (x, y) on the frame (step 208), two color groups CG (r1), C that form a color pair are formed.
One of the colors belonging to one color group of G (r2) is set as a cell color (step 209), and one of the colors belonging to the other color group of the color group pair is added to the cell itself or its neighbor. The cell C (x, y) that any one of the eight cells has as a cell color is extracted as a valid cell (step 210), and the total number in the frame is obtained (step 211).

In each reference color group pair, if the number of effective cells in the frame is equal to or larger than the set threshold value (step 21)
2) The color group pair (CG (r1), CG (r2)) is counted as an effective color group pair (step 213). If the total number of counted effective color group pairs is equal to or larger than the set threshold value (step 214), it is determined that the object exists in the frame (step 215), and the total number of effective color group pairs is set to the set threshold value. If it is below, it is determined that the object does not exist (step 216).

In the continuous judgment experiment according to FIG. 1 and FIG. 2, the result of New 2 in the following Table 1 was obtained.

[Table 1] As shown in Table 1, the same tendency is shown in both the upper and lower stages, and it can be seen that the omission of detection can be greatly reduced by grouping the colors. However, with this determination method, over-detection is increased at the cost of being able to drastically reduce detection omissions. To reduce such over-detection, different color group pairs (pairs of cells of different color groups) and same color group pairs (pairs of cells of the same color group) are used, and the distribution of different color group pairs and the same color group pairs are used. It is effective to pay attention to the difference in the characteristics of the distribution of and to reflect it in the determination process.

That is, the effective cells of the pair of different color groups exist only around the boundary between the partial areas, but the effective cells of the pair of the same color group exist by the area of the partial area corresponding to the color group. Therefore, the number of effective cells of the same color group pair is much larger than that of the different color group pair, and if the same set threshold value is determined for the same color group pair and the different color group pair, excessive detection occurs. Therefore, as shown in FIG. 4 below, in the case of the same color group pair, the effective color group pair is determined using a higher cell number threshold than in the case of the different color group pair.

FIG. 4 is a flow chart showing a second embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention. By the same processing as steps 201 to 205 in FIG. 2, all colors (cell colors) whose frequency of the color histogram is greater than the set threshold value are extracted and registered in the list CC (step 401). Similarly, by the same processing as steps 207 to 213 in FIG. 2, the effective color group pair (CG (r1), CG (r) is set based on the effective cell setting threshold t1.
2)) is calculated and counted (steps 402, 40)
3). Then, the processing according to the second embodiment is performed next.

That is, each color group CG (r1), CG (r
The following processing is performed for each of (2) (step 4).
04). First, from each cell, a cell whose color registered in the color group CG is registered in the list CC (x, y) is extracted, counted as valid cells, and the number of valid cells of the same color group pair is obtained. (Steps 405-407). Then, based on the setting threshold value t2 that is larger than the setting threshold value t1 used in the process of step 403, it is determined whether or not it is an effective color group pair (step 408), and the color group CG determined to be the effective color group pair is determined.
Is counted (step 409). If the total number of effective color group pairs thus obtained is equal to or greater than the set threshold value t3 (step 410), it is determined that the object is present in the frame (step 411), and the effective color group is also determined. If the total number of pairs is less than or equal to the set threshold value, it is determined that the object does not exist (step 412).

The result of the continuous judgment experiment conducted in this manner is shown in the new item 3 of Table 1 above. Last new 2
In addition to the different color group pairs of, the same color group pairs of three types corresponding to hair, face, and clothes are included in the reference color group pair. From this, it can be seen that the number of over-detections is reduced while the number of missed detections is kept small. That is, it is clear that the effect of reducing the over-detection is obtained by setting the threshold value of the effective cells to be high in the same color group pair.

Next, a technique for further reducing the over-detection will be described. In a normal image, the target object does not rotate upside down, so in the target object image, the variation in the adjacent direction between the partial areas corresponding to each color group is
Almost subsides within a range of ± 45 degrees. Therefore, in the process of determining the presence of the pair of reference color groups, by adding the direction between the adjacent partial regions to the determination condition, the determination of the frame including the object can be made more reliable. In addition, to find the adjacent areas in a specific direction,
Various methods are conceivable, but here, in order to suppress an increase in processing time, a method as simple as possible is adopted.

For example, in the frame to be judged, the valid cells of the different color group pair are present only near the boundary between the distribution areas of the two color groups forming the different color group pair if they appear correctly. Conversely speaking, there should be no effective cells of the same color group pair in the area other than the vicinity of the distribution area of the different color group pair. Therefore, by paying attention to the effective cells of the pair of different color groups and masking and removing the effective cells of the pair of same color groups that are not in the vicinity of the effective cells, noise appearing in a region unrelated to the object can be reduced. When masking, the directionality of color group pairs is also taken into consideration in order to enhance the effect. Less than,
A third embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention in consideration of the directionality of a color group pair will be described with reference to FIGS. 5 and 6.

FIGS. 5 and 6 are flow charts showing a third embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention. First, the process of extracting the reference color group pair from the reference image will be described with reference to FIG. By the same processing as steps 101 to 107 in FIG. 1, a color group list C is obtained for each partial area r of the image area of the object.
The G (r) is registered (step 501). And
For all pairs (r1, r2) of adjacent partial areas, a pair of color groups CG (r1) and CG (r2) is created and registered in the list RCGP '(steps 502, 503).
At this time, the direction of the area of the other color group (hereinafter referred to as the reference direction) is extracted with the area of one color group forming the color group pair (CG (r1), CG (r2)) as the base point. , Registered in the list RCGP 'as shown in the following Table 7 (step 5
04). This step 504 is a newly added process,
This is a feature of the third embodiment. Note that the reference directions are expressed in all eight directions that are diagonally up, down, left and right. In this experiment using this example, the direction was manually specified.

[Table 7]

Next, the search processing by the reference color group pair with the direction thus obtained will be described with reference to FIG. Similar to the processing of step 401 in FIG. 4, all the colors (cell colors) whose frequency of the color histogram is greater than the set threshold are extracted by the same processing as steps 201 to 205 in FIG. 2 and registered in the list CC (step 601).
Similarly, similar to the processing in steps 402 and 403 in FIG. 4, the same processing as in steps 207 to 213 in FIG. 2 is performed, and the effective color group pair (CG
(R1), CG (r2)) are calculated and counted (steps 602 and 603). Further, step 404 of FIG.
Similar to the processing of 407, color groups CG (r1), CG
In each of (r2) (step 604), the colors registered in the color group CG are listed CC (x,
The cells registered in y) are extracted and counted as valid cells to obtain the number of valid cells of the same color group pair (steps 605 to 607).

After obtaining the number of effective cells for each cell C (x, y), the color group CG (r
1), effective cells are removed by a mask in the vicinity of the distribution area on the frame of CG (r2) (step 608).
Then, the total number of remaining valid cells is set threshold t2 (larger than the set threshold t1 used in the process of step 603).
If it is above (step 609), the color group CG is determined as an effective color group pair and counted (step 610).

If the total number of effective color group pairs thus obtained is equal to or greater than the set threshold value t3 (step 611), it is determined that the object is present in the frame (step 612), and The total number of effective color group pairs is the set threshold t3.
If it is below, it is determined that the object does not exist (step 613). In the third embodiment, step 60
The algorithm is exactly the same as that of FIG. 4 except that the masking process of 8 is added. The masking process in step 608 will be described below with reference to FIG.

FIG. 7 is an explanatory diagram showing a fourth embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention. In this figure, 71 is a reference color group pair obtained from the reference image in the process in FIG. 5, 72 is a distribution map of the main color group in the reference color group pair 71, 73 is a distribution map of the secondary color groups, and 74.
Is a distribution map of color group pairs, 75 is a direction mask obtained by expanding the distribution map of color group pairs in the direction opposite to the reference direction, and 76 is a direction mask obtained by expanding the distribution map of color group pairs in the reference direction. , 77 is a distribution map of the main color group from which noise has been removed using the directional mask 75, and 78 is a distribution map of the secondary color group from which noise has been removed using the directional mask 76.

The mask processing is performed as follows.
First, in the color group pair 74 (CG (r1), CG (r2)), starting from each effective cell thereof, in the reference direction, a length of m cells (here, 4 cells) and a width of the reference direction are set. A rectangular area corresponding to n cells (here, 2 × 2 cells) in a direction orthogonal to is obtained for all effective cells. That is, the direction mask 76 is obtained. Then, among the color groups forming the color group pair, the color group CG (r2) that is not the base point of the reference direction.
For each valid cell of, valid cells that are not included in any of the rectangular areas are invalidated and subtracted from the total number of valid cells. That is, the secondary color group distribution map 73 and the direction mask 7
An AND process with 6 is performed to obtain a secondary color group distribution map 78 from which noise is removed.

Next, the color group pair (CG (r1), CG (r
2)), starting from each valid cell, the length m cells (here, 4 cells) in the direction opposite to the reference direction, and the width n cells in the direction orthogonal to the reference direction (here, 2 × 2). Cell) to obtain the rectangular area. That is, the direction mask 75 is obtained. Then, among the color groups forming the color group pair, for each valid cell of the color group CG (r1) set as the reference point in the reference direction, valid cells that are not included in any of the rectangular regions obtained by the number of valid cells are selected. Invalidated and subtracted from the total number of valid cells. That is, the AND process of the distribution map 72 of the main color group and the direction mask 75 is performed to obtain the distribution map 77 of the secondary color group from which noise has been removed.

As described above, the result of the continuous judgment experiment in which the same color group pair other than the neighborhood of the region in which the different color group pairs are distributed is invalid is:
It becomes the one shown in New 4 in Table 1 above, and compared with New 3,
In both the upper and lower stages, the number of over-detections was greatly reduced while the detection omission remained zero. That is, it can be understood that the excessive detection can be greatly suppressed by adding the determination of the adjacent direction to the reference color group pair.

The embodiment described with reference to FIGS. 1 to 7 is a specific object presence / absence determining process based on a color group pair.
As shown in FIG. 8 below, by using different color pairs, the presence or absence of a specific object in a color image can be determined more reliably than in the related art using only the color pairs.

FIG. 8 is a flow chart showing a fifth embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention. In this embodiment, the list CP is obtained (step 801) by the conventional technique shown in FIG.
A feature is that a pair of different colors is selected from the list CP and used as a reference color pair (step 802). For example, when a person is the target, the different color pair is selected so as to be a combination of colors among the elements of hair, face, and clothes.

According to the processing steps of this experimental example, the result of continuous determination for the same frame as that of the conventional method is shown in the column 1 of Table 1 above. It can be seen that the addition of the process of appropriately selecting the pairs of different colors significantly reduces the over-detection as compared with the conventional technique.

In the description of the embodiments shown in FIGS. 1 to 8, the case where all the processes are executed by the program by the CPU has been described, but each process may be executed by the electronic circuit as necessary. it can. In addition, the reliability of the determination result can be further improved by performing color correction and freely selecting and combining the processing used in each of the embodiments according to the image. Hereinafter, such an embodiment will be described with reference to FIGS. 9 and 10.

FIG. 9 is a block diagram showing an embodiment of the structure relating to the present invention of the specific object presence / absence determining device in the color image of the present invention. The specific object presence / absence determining device 90 in the color image of the present embodiment controls input / output of color image information to / from a main control unit 91 that controls the processing operation of the entire device and a moving image reproducing device or a display device. An image input / output unit 92, a color correction unit 93 relating to the present invention, which performs color correction (for example, hue, saturation, lightness, etc.) on an input image, and a specification specified by the user from the input image. Object specifying unit 94 that cuts out the image area of the target object, and a color group extracting unit 95 according to the present invention that extracts a plurality of similar color partial areas from the specified image area of the target object as a color group. In the unique color selection unit 96 and the unique color selection unit 96 according to the present invention, which selects a pair of adjacent color groups showing a characteristic of a specific object from a plurality of color groups extracted by the color group extraction unit 95. Color to register the selected color group pair Versus the registration unit 97
Then, with reference to the color group pair registration unit 97, the presence / absence determination unit 98 according to the present invention for determining whether or not a particular target object is included in the input arbitrary image, and the color correction The determination method storage unit 99 stores the execution conditions and the processing contents selected and designated by the user, such as the determination processing methods described with reference to FIGS.

When an image (reference image) of a specific object is input to the image input / output unit 92, the color correction unit 93 corrects the color of this image based on the stored contents of the determination method storage unit 99. Output or output without color correction. In this color correction, for example, an auto white balance method widely used in video cameras is applied to keep this reference image at a specific value or its approximate value regardless of the light source.

With respect to the image from the color correction unit 93, the object specifying unit 94 cuts out the image area of the specific object specified by the user as a reference image. Then, the color group extraction unit 95
Extracts a plurality of similar color partial areas from the image area of the designated object as a color group and sends them to the unique color selection unit 96. The unique color selection unit 96 selects, from the color groups received from the color group extraction unit 95, pairs of adjacent color groups that indicate the characteristics of a specific object, and registers them in the color group pair registration unit 97. At this time, the unique color selection unit 96 also registers information such as a reference direction in the different color group pair in the color group pair registration unit 97 based on the storage content of the determination method storage unit 99.

Next, when the image to be judged is input to the image input / output unit 92, the color correction unit 93 corrects the reference image based on the stored contents of the judgment method storage unit 99. Correspondingly, the image is color-corrected or is output without color correction. Alternatively, the color correction unit 93 performs the color correction of the color group pair registered in the color group pair registration unit 97 corresponding to the light source of this image based on the storage content of the determination method storage unit 99, and the correction is performed. The color group pair
It is sent to the presence determination unit 98.

The presence / absence determining unit 98 uses the color group pair registered in the color group pair registering unit 97 or the color group pair color-corrected by the color correcting unit 93 to generate an image from the color correcting unit 93. , It is determined whether or not a specific object is included. At this time, the presence / absence determination unit 98 performs determination based on the storage content of the determination method storage unit 99, for example, for the pair of the same color or different color group and the different color or color group with different setting thresholds, Alternatively, the presence of a pair is determined by using the adjacent direction of the sense of distribution area of colors or color groups, and it is determined whether or not a specific object is included.

As described above, in the determination processing by the presence / absence determining unit 98, the color correcting unit 93 performs color correction of the image and the color group. For example, based on the reference image of the daytime object, the setting sun is set. Even when the same object below is searched for, that is, when the hue greatly changes due to the color temperature change of the light source, the accurate determination can be performed. Specifically, after extracting a color peculiar to the object from the reference image, this peculiar color is shifted according to the difference between the light source of the image to be searched and the light source of the reference image. As a result, the unique color of the target object can be correctly detected, and the presence / absence of the target object under a light source different from the reference image can be correctly determined.

FIG. 10 is a flow chart showing the processing operation according to the present invention of the specific object presence / absence determining device in the color image in FIG. First, when a reference image including a specific object is input to the image input / output unit 92 of FIG. 9, the color correction unit 93 of FIG. 9 is applied to the input reference image.
Thus, color correction is performed (step 1001). Then, from the color-corrected image, the object specifying unit 94 of FIG.
Then, the color group extraction unit 95 extracts a plurality of colors unique to a specific earth-saving material (step 1002), collects similar colors into a color group, and draws this color group as a specific color of a specific target. 9 color group pair registration unit 98 (step 100)
3).

Next, when the image to be judged is input to the image input / output unit 92 of FIG. 9, the color correction unit 93 of FIG. 9 corrects the color of the input judgment image or inputs it. The color group pair registration unit 9 in FIG. 9 based on the brightness of the image for determination
The color correction of the color group pair registered in No. 8 is performed (step 1
004). Then, the presence / absence determining unit 98 in FIG. 9 determines the pair of the same color or color group and the pair of different color or color group with different setting thresholds (step 1).
005), the existence of a pair is determined by using the adjoining direction between the distribution areas of the color or the color group (step 1006).
In this way, it is determined whether or not the unique colors appear in pairs in the image to be determined (step 100).
7).

In this embodiment, the determination method storage unit 99 shown in FIG.
Each process can be selected on the basis of the stored contents of the step 100, and the user can select the step 100 surrounded by a double frame in the figure.
Each of 1, 1003 to 1006 can be arbitrarily combined. Also, steps 1001 and 1004
As described in FIG. 9, the color correction in step 1 is performed by applying the method of auto white balance that is widely used in video cameras, and the colors of the same object in the reference image and the image to be determined are respectively used as light sources. It keeps a specific value or its approximate value. This also makes it possible to correctly determine the presence / absence of an object under a light source different from that of the reference image.

As described above with reference to FIGS. 1 to 10, in the method and apparatus for determining the presence / absence of a specific object in a color image according to this embodiment, a pair is formed by a group of similar colors. (Color group pair), based on this color group pair, the frame including the object is detected from the color moving image. This makes it possible to detect pairs of color groups in a stable manner even when the color temperature of the light source differs, and greatly reduces detection omissions when determining whether an object is present in the frame. Can be made. In addition, a threshold for determining an effective color pair is set higher for a pair of different color groups than a threshold for a pair of the same color group. This can reduce over-detection.
Further, by considering the adjoining direction between the color group distribution regions, it is possible to further reduce the excessive detection.

The method and apparatus for determining the presence or absence of a specific object in a color moving image according to the present invention are not limited to the embodiments described with reference to FIGS. Can be variously changed. For example,
In extracting the color group described above, only the colors that actually exist in the partial area of the reference image are grouped, but it is also possible to obtain similar colors in various color spaces by calculation and group them. Next, the color moving image cut search and display system of the present invention will be described with reference to FIGS.

FIG. 11 is a block diagram showing the first embodiment of the constitution relating to the present invention of the color moving image cut retrieval display system of the present invention. In the figure, 11 is a moving image reproducing apparatus such as an optical disc or a video deck, and 12 is a moving image reproducing apparatus 11 by the same processing as the specific object presence / absence determining apparatus 90 in the color moving image described in FIG. A frame determination device that determines whether or not a specific object is included in each frame of the continuous color moving image output from the The display is divided into cuts, and a cut display screen 14 of a rectangular parallelepiped formed of the cuts and the frames within the cuts is displayed for each divided cut.
The cut display control device 16 for displaying on the screen of No. 1, a specific frame detection unit 17 for detecting a frame containing a specific object and a cut based on the determination result by the frame determination device 12, 17 is the specific frame. Detector 16
It is a specific frame image display unit that generates and displays the cut display screen 14 and the target object screen 18 by using the frame including the specific target object detected in. 14a
Is a cut display screen displayed by the conventional color moving image cut search display system, in which the image of the first frame of each cut is displayed. The time length of each cut is represented by the depth (thickness) of the cut display screen 14a.

The cut display control device 13 utilizes the result of the determination processing of the specific object by the frame determination device 12 when displaying continuous color moving images output from the moving image reproducing device 11 in units of cuts. . That is, the specific frame detection unit 16 detects whether or not each cut includes a frame (specific frame) in which a specific object is present, based on the specification result of the frame determination device 12. Then, when generating the cut display screen for each cut, the specific frame image display unit 17 displays the image of the specific frame for the cut including the specific frame.

At this time, the specific frame is displayed at the corresponding position of the portion showing the cut time length on the cut display screen. In this embodiment, it is displayed at the depth m. For a cut that does not include a specific frame, the image of the first frame of this cut is displayed in the foreground as in the conventional technique. As described above, in the color moving image cut search display system of the present embodiment, the image of the specific frame including the specific object is preferentially displayed on the cut display screen. You can easily search for cuts that include.

FIG. 12 is a block diagram showing a second embodiment of the constitution relating to the present invention of the color moving image cut retrieval display system of the present invention. In the second embodiment, the specific frame group display unit 1 is added to the cut display control device 13 in FIG.
9 is provided to configure the cut display control device 13a, and a cut display screen 20 indicating the time length of a specific frame group that continuously includes a specific object is displayed.
The cut display control device 13a causes the specific frame detection unit 16 to perform a specific object determination process by the frame determination device 12 when displaying continuous color moving images output from the moving image reproduction device 11 in units of cuts. Based on the result, it is detected whether or not each cut includes a frame in which a specific object exists (specific frame).

When the specific frame image display unit 17 generates a cut display screen for each cut displaying the image of the specific frame, the specific frame group display unit 19
Therefore, after the image of the specific frame on this cut display screen, with the depth corresponding to the time length of the specific frame group,
Color or highlight. As described above, in the color moving image cut search and display system according to the second embodiment, the section in which the frame group including the specific object in the cut exists is displayed. With this, the user can easily confirm the amount of time of the frame in which the specific object in the searched cut is included.

FIG. 13 is a block diagram showing a third embodiment of the constitution relating to the present invention of the color moving image cut retrieval display system of the present invention. In the third embodiment, a display unit 21 is provided instead of the specific frame image display unit 17 of the cut display control device 13 in FIG. 11 to form a cut display control device 13b, and a specific target object is set. The images 22 to 24 of the included frame are reduced and displayed, and the cut display screens 25 to 29 in which the time lengths of the respective frame groups of the images 22 to 24 of the reduced frames are graphically displayed are displayed.

The cut display control device 13b causes the specific frame detection unit 16 to specify the specific object by the frame determination device 12 when displaying continuous color moving images output from the moving image reproducing device 11 in units of cuts. It is detected whether or not each cut includes a frame (specific frame) in which a specific target object exists, based on the determination processing result. Then, the display unit 21 causes the specific frame detection unit 1
The images 22 to 24 of the plurality of specific frames detected in 6 are reduced and displayed, and the cut display screen 2 of the rectangular parallelepiped is displayed.
Display 5-29. In the third embodiment, the display unit 21 highlights the time lengths of the specific frame groups corresponding to the images 22 to 24 on the upper surfaces of the cut display screens 25 to 29 with the display bars 22a to 24a. .

As described above, in the color moving image cut search and display system of the third embodiment, not only for a single object but also for a plurality of objects, at which position of which cut, it appears. . Then, draw a bar (straight line) in the depth (thickness) part of each cut with a color and expression format unique to the object, and according to the position and length of the bar,
Represents the appearing section of each object. This allows the user to understand at a glance which object appears where and how long it appears. Also, at a glance, the appearance relationship between the objects, for example, the frame portions where the objects shown in the images 22 to 24 appear at the same time,
The user can easily search for a plurality of specific objects.

FIG. 14 is a block diagram showing a fourth embodiment of the configuration relating to the present invention of the color moving image cut search / display system of the present invention. In the fourth embodiment, the cut display control device 13 in FIGS. 11 to 13 uses the reference screens 3 used to specify a plurality of types of target objects designated by the user.
1 to 33 and time chart screens 31a to 33a indicating time positions in continuous color moving images output from the moving image reproducing apparatus 11 of each specific frame extracted using each of the reference screens, The target screen display unit 34 displayed on the screen of the display 15, the reference screens 31 to 33 and the time chart screens 31a to 33a displayed by the target screen display unit 34 are specified by the user using the mouse 30 or the like. The specific frame detection unit 16 sends the frame to the frame determination device 12, and the specific frame image display unit 17 includes a designated frame detection unit 36 that highlights the corresponding cut display screen 35.

The operation of the fourth embodiment will be described with reference to FIGS. 15 and 16 below. FIG. 15 is an explanatory diagram showing a first example of screen contents of the display in FIG. This figure shows a state in which the user has not designated the mouse 30 of FIG. On this screen, the cut display screens 37 to 4 including the image of the first frame of each cut of the color moving image and the depth (thickness) indicating the time length are displayed.
1, and the reference screens 31 to 33 and the time chart screens 31a to 33a by the object screen display unit 34 of FIG. 14 are displayed. In this state, when the user designates the reference screen 32 using the mouse 30 of FIG. 14, the screen contents shown in FIG. 16 are changed.

FIG. 16 is an explanatory diagram showing a second example of the screen contents of the display in FIG. This figure shows an operation example when the user designates the reference screen 32 using the mouse 30 of FIG. 14. When the designated frame detection unit 36 of FIG. 14 detects the click information from the mouse 30 of FIG. 14, that is, the search instruction information of the reference screen 32, it activates the frame determination device 12 of FIG. The determination process of the frame to be started is started. Based on the determination result from the frame determination device 12 in FIG. 14, the specific frame detection unit 16 and the specific frame image display unit 17 in FIG. 14 switch the cut display screens 37 to 41 to the display based on the reference screen 32. That is, each of the display screens 37, 40, 41 in FIG. 15 is switched to the display screens 37a, 40a, 41a, and the reference screen 3
A screen appears displaying two specific objects. Further, the screen displaying the specific object of the reference screen 32 is displayed at the position of the corresponding time of each cut.

As described above with reference to FIGS. 1 to 16, according to the method for determining the presence / absence of a specific object in a color image, the apparatus therefor, and the color moving image search / display system using this apparatus, The user can detect a frame containing a specific object from a color moving image with high reliability without omission. It should be noted that the method for determining the presence or absence of a specific object in a color moving image, the apparatus thereof, and the color moving image search and display system using this apparatus according to the present invention are, for example, as shown in the description of FIGS. ~ Fig. 16
The present invention is not limited to the embodiment described by using, and various modifications can be made without departing from the scope of the invention.

[0077]

According to the present invention, it is possible to select a reference color pair,
A technology related to color moving image detection that can efficiently prevent omission of detection of a frame containing an object from a color moving image, which has occurred due to a color change of the object due to a change in color temperature of a light source. It is possible to improve the reliability and performance of the color moving image and to support more detailed editing work for the color moving image in the color moving image cut search display system.

[0078]

[Brief description of drawings]

FIG. 1 is a flowchart showing a first embodiment of a method for determining the presence / absence of a specific object in a color image according to the present invention.

FIG. 2 is a flowchart showing a first embodiment of a method for determining the presence / absence of a specific object in a color image according to the present invention.

FIG. 3 is a block diagram showing the configuration of an apparatus used for carrying out the method for determining the presence / absence of a specific object in a color image according to the present invention in FIGS. 1 and 2.

FIG. 4 is a flowchart showing a second embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention.

FIG. 5 is a flowchart showing a third embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention.

FIG. 6 is a flowchart showing a third embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention.

FIG. 7 is an explanatory diagram showing a fourth embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention.

FIG. 8 is a flowchart showing a fifth embodiment of the method for determining the presence / absence of a specific object in a color image according to the present invention.

FIG. 9 is a block diagram showing an example of a configuration according to the present invention of a specific object presence / absence determination device in a color image of the present invention.

10 is a flowchart showing a processing operation according to the present invention of the specific object presence / absence determination device in the color image in FIG.

FIG. 11 is a block diagram showing a first embodiment of a configuration relating to the present invention of a color moving image cut search / display system of the present invention.

FIG. 12 is a block diagram showing a second embodiment of the configuration relating to the present invention of the color moving image cut search / display system of the present invention.

FIG. 13 is a block diagram showing a third embodiment of the configuration according to the present invention of the color moving image cut search / display system of the present invention.

FIG. 14 is a block diagram showing a fourth embodiment of the configuration according to the present invention of the color moving image cut search / display system of the present invention.

15 is an explanatory diagram showing a first example of screen contents of the display in FIG.

16 is an explanatory diagram showing a second example of screen contents of the display in FIG.

FIG. 17 is a flowchart showing an example of a conventional method of selecting characteristic information of a color moving image target.

FIG. 18 is a flowchart showing a conventional method for searching a color moving image object.

19 is an explanatory diagram showing a search example using the algorithm shown in FIGS.

FIG. 20 is an explanatory diagram showing an example of reference color pairs extracted from a reference image.

FIG. 21 is a block diagram showing a configuration of a conventional color moving image cut search / display system.

[Explanation of symbols]

 1 Computer 2 Display 3 Moving Image Reproducing Device 4 A / D Converter 5 Control Line 6 External Storage Device 7 Input Device 8a to 8e Interface 9 CPU 10 Memory 11 Moving Image Reproducing Device 12 Frame Judging Device 13, 13a, 13b Cut Display Control Device 14, 14a Cut display screen 15 Display 16 Specific frame detection unit 17 Specific frame image display unit 18 Object screen 19 Specific frame group display unit 20 Cut display screen 21 Display unit 22 to 24 frame image 22a to 24a Each display bar 25 -29 Cut display screen 30 Mouse 31-33 Reference screen 31a-33a Time chart screen 34 Object screen display unit 35 Cut display screen 36 Designated frame detection unit 37-41 Cut display screen 37a, 40a, 41a Cut display screen 71 See Color group pair 72 Main color group distribution map 73 Secondary color group distribution map 74 Color group pair distribution map 75, 76 Directional mask 77 Noise-free main color group distribution map 78 Noise-free secondary color group distribution map Map 90 Specific object presence / absence determination device in color image 91 Main control unit 92 Image input / output unit 93 Color correction unit 94 Object designation unit 95 Color group extraction unit 96 Unique color selection unit 97 Color group pair registration unit 98 Presence / absence determination unit 99 determination method storage unit 191 frame including object 192 frame including search object 193 frame not including search object 194, 195 distribution map 291 reference image 292, 293 RGB color space coordinates 294 color difference 2101 moving image reproducing apparatus 2102 Cut detection device 2103 Cut display control device 2104 Display 2105 Cut display screen

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 6, 1993

[Procedure Amendment 1]

[Document name to be corrected] Drawing

[Name of item to be corrected] Fig. 19

[Correction method] Change

[Correction content]

FIG. 19

[Procedure Amendment 2]

[Document name to be corrected] Drawing

[Name of item to be corrected] Fig. 20

[Correction method] Change

[Correction content]

FIG. 20

Claims (17)

[Claims] 1. A color image of a specific object is extracted from adjacent color pairs indicating the characteristics of the specific object, and an arbitrary color image is searched based on the extracted color pair. In any color image, in the specific object presence determination method in the color image to determine whether the specific object is included, from the color image of the specific object,
A plurality of partial areas having similar colors are extracted as a color group, and a pair of adjacent color groups that shows the characteristic of the specific object is selected from the plurality of color groups. Method for determining the presence / absence of a specific target object in a color image, characterized in that it is determined that the specific object is included in the arbitrary color image when the number of color objects is greater than a predetermined number. . 2. The method for determining the presence / absence of a specific object in a color image according to claim 1, wherein the pair of adjacent color groups showing the characteristics of the specific object are pairs of color groups having different shades. Select a pair of different color groups, and determine that the specific object is included in the arbitrary color image when the number of pairs of different color groups is more than a predetermined number in the arbitrary color image. A method for determining the presence or absence of a specific object in a color image, which is characterized by the above. 3. The method for determining the presence or absence of a specific object in a color image according to claim 1, wherein the pair of adjacent color groups that characterize the specific object are pairs of color groups having different shades. The different color group pair and the same color group pair which are pairs of color groups having the same hue are selected, and the same color group pair and the different color group pair are
Presence or absence of a specific target object in a color image, characterized in that it is determined that the specific target object is included in the arbitrary color image when there are more than a predetermined number in the arbitrary color image Judgment method. 4. The method for determining the presence / absence of a specific object in a color image according to claim 3, wherein the presence / absence of the specific object in the arbitrary color image based on the same color group pair is determined.
A method for determining the presence or absence of a specific object in a color image, which is performed based on a number larger than the predetermined number used for the determination by the pair of different color groups. 5. The method for determining the presence / absence of a specific object in a color image according to claim 3 or 4, wherein the pair of same color groups other than the vicinity of the area in which the pair of different color groups is distributed is invalid. As a color image characterized by using the different color group pair and the same color group pair in the vicinity of the area in which the different color group pair is distributed, the presence or absence of the specific object in the arbitrary color image is determined. Method for determining the presence or absence of a specific object in 6. The method for determining the presence / absence of a specific object in a color image according to claim 5, wherein the first color group forming the different color group pair is selected when the different color group pair is selected from the specific object. From the first color group to the second color group is determined as a reference direction, and the region of the first color group forming the different color group pair existing in the arbitrary color image is expanded in the direction opposite to the reference direction. The first mask area is expanded in the reference direction with respect to the area of the second color group forming a different color group pair existing in the arbitrary color image to generate second mask areas, and the first mask area is generated. Using the same color group pair having the same color tone as the first color group in one mask area and the same color group pair having the same color tone as the second color group in the second mask area, Characterized by determining the presence / absence of the specific object in an arbitrary color image A method for determining the presence or absence of a specific object in a color image. 7. The method for determining the presence / absence of a specific object in a color image according to claim 1, wherein a light source for the color image of the specific object and a light source for the arbitrary color image are used. Based on the change in color temperature, the hue, saturation, lightness, etc. of the image of the specific object is corrected, and a pair of color groups showing the characteristics of the specific object is selected from the corrected image. A method for determining the presence or absence of a specific object in a color image, which is characterized by the above. 8. The method for determining the presence / absence of a specific object in a color image according to claim 1, wherein a light source for the color image of the specific object and a light source for the arbitrary color image are used. Based on the change in color temperature, the hue and saturation of each color group forming a pair of adjacent color groups showing the characteristics of the specific object extracted from the color image of the specific object,
The color such as brightness is corrected, and the corrected color group is used to
A method for determining the presence / absence of a specific object in a color image, comprising determining the presence / absence of the specific object in the arbitrary color image. 9. The method for determining the presence / absence of a specific object in a color image according to claim 1, wherein a light source for the color image of the specific object and a light source for the arbitrary color image are used. Based on the color temperature change, the hue, saturation, lightness, etc. of the arbitrary image is corrected, and the presence or absence of the specific object in the corrected image is determined using the corrected image. A method for determining the presence or absence of a specific object in a color image, which is characterized by the above. 10. A color image of a specific object is extracted from adjacent color pairs indicating the characteristics of the specific object, an arbitrary color image is searched based on the extracted color pair, In a specific object presence / absence determining device in a color image for determining whether or not the specific object is included in an arbitrary color image, a partial area of a similar color is obtained from the color image of the specific object. A color group extracting means for extracting a plurality of color groups as a color group, and an inherent color selection for selecting a pair of adjacent color groups showing the characteristics of the specific object from the plurality of color groups extracted by the color group extracting means. Means and the number of pairs of color groups selected by the proper color selecting means are present in the arbitrary color image in excess of a predetermined number, the specific object is included in the arbitrary color image. The presence / absence determination means for determining that An apparatus for determining the presence or absence of a specific object in a color image, which determines the presence or absence of the specific object in the arbitrary color image. 11. The specific object in the color image according to claim 1, wherein the specific object presence / absence determination device in the color image according to claim 10 is selected and combined by a user. A determination method storage means for storing the object presence / absence determination method is provided, and the specific object presence / absence determination method in the color image according to any one of claims 1 to 9 is based on the storage content of the determination method storage means, An apparatus for determining the presence or absence of a specific object in a color image, which determines the presence or absence of the specific object in the arbitrary color image. 12. A continuous color moving image is divided into cuts, and each cut is divided into first cuts.
In the color moving image cut search display system for displaying a cut display screen group consisting of a rectangular parallelepiped in which the color image of the frame is the front and the time length of each cut is the depth from the first frame, Item 1
0 or a specific frame detecting means for detecting a color moving image frame including the specific object based on the determination result by the specific object presence / absence determining device in the color image according to claim 11. Instead of the color image of the first frame of the continuous specific color moving image frame group detected by the specific frame detecting means, instead of the color image of the first frame of the cut including the frame,
A specific frame image display means for displaying at a depth position corresponding to the frame of the cut display screen is provided, and the image of the color moving image frame including the specific object is preferentially displayed on the cut display screen. A color moving image cut search display system characterized by being displayed on. 13. The color moving image cut retrieval display system according to claim 12, wherein the time length of the continuous specific color moving image frame group is based on the image displayed by the specific frame image display means. A color characterized by providing a specific frame group display means for coloring or highlighting the depth of the rectangular parallelepiped, and displaying a section in which a color moving image frame group including the specific object in the cut is present. Video cut search display system. 14. The color moving image cut retrieval display system according to claim 12 or 13, wherein the specific frame image display means is a first frame of a frame group including the specific object. A color moving image cut search display system characterized by displaying a reduced size image. 15. The color moving image cut retrieval display system according to claim 12, wherein a reduced image of a first frame of a frame group including the specific object and a time chart of the frame group. A color moving image cut search and display system, characterized in that an object screen display means for displaying and is associated with each other is provided. 16. The color moving image cut search and display system according to claim 12, wherein instead of the specific frame image display means, a first frame of each of a plurality of frame groups each including the different specific object is displayed. A plurality of object display means is provided for displaying the image in a reduced size and displaying the time length of each frame group including the reduced image in a graph at a position in the depth direction corresponding to the cut display screen including the frame group. A color moving image cut search and display system characterized by individually displaying a ratio of a time length of a color moving image frame group in which a plurality of specific objects exist in the entire color moving image. 17. The color moving image cut search and display system according to claim 14, wherein a specific search target is specified based on a user's selection operation for the plurality of displayed reduced images. A color moving image cut search display characterized in that designated frame detection means for detecting an object is provided, and display control of the cut display screen corresponding to a specific object detected by the designated frame detection means is selectively performed. system.