JPH11191109A - Device and method for retrieving object - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable the retrieval of a relevant object from a retrieval object image even when its position and size are different from a designated object. SOLUTION: A division part 108 divides a rectangular object area into n×m pieces of blocks. In the unit of (n) pieces of blocks, an extraction part 106 extracts profile data to a Y axis in the retrieval object image concerning the representative hue value of these blocks and detects a clamp area where the distribution is overlapped on the respective profile data of (n) pieces of relevant blocks. The extraction part 106 extracts profile data to an X axis concerning the representative hue value of (n) pieces of relevant blocks while defining the area on the retrieval object image in the detected clamp area as an object, and a retrieval part 109 detects a clamp area where the distribution is continuously adjacent on the respective relevant profile data. By performing this processing (m) times while switching the unit of (n) pieces of blocks, the designated object is retrieved from the retrieval object image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention mainly relates to a large amount of images,
The present invention relates to an object search apparatus and method suitable for efficiently searching for one frame of a target image or video from a video.

[0002]

2. Description of the Related Art In recent years, multimedia information-related computer systems have been used on an individual basis. In addition to conventional text, media such as voice, graphics, color images, and video can be easily used as media. It can be handled.

[0003] Here, there are a large number of free image materials on each site on the World Wide Web (WWW) of the Internet and on a paid image database. Recently, video has also been digitized,
It can be used as digital files such as 1, @ PEG2. The digitized video can be handled as a time-series file, and each frame (that is, frame image) can be handled in the same manner as a still image. Therefore,
In the following description, in order to avoid complication, video materials are also regarded as a kind of image materials, and are all referred to as image materials.

[0004] Now, the task of retrieving an image required by each individual, for example, an image suitable for each individual's taste and feeling, from a large amount of image materials (image data) is extremely burdensome. Conventionally, in order to facilitate image search work, a plurality of search target images reduced on one screen are simultaneously displayed to allow a user to select, or a sketch representing a feature of an image to be searched by the user. A technique has been applied in which an image can be presented by a user operation as a search condition or an image similar to a target image can be presented by a user operation as a search condition.

[0005]

However, in a conventional search method, for example, a method using so-called multi-image display, in which a plurality of reduced images are displayed on a screen at the same time and a user selects one of the reduced images is selected. If the number of pieces of image data exceeds a certain level, the number of display screens increases, and there is a problem that the search efficiency immediately drops. Furthermore, when the number of reduced screens increases, there is also a problem that the search must be performed while switching the reduced screens and watching for a long time.

Next, in a method of presenting a similar image using a simple sketch image, it is difficult to accurately present the image if there is no sense of drawing, and it is troublesome to draw such a pattern even if it is a sketch image. There is such a problem.

[0007] Further, an image similar to the intended image is stored as one image.
The method of presenting sheets is a search for images in which the atmosphere of the entire image is similar, that is, a so-called fuzzy search method. Of course,
When there is a characteristic in the upper, lower, left, and right sides of the image, an image similar to some extent can be searched. However, when a search is performed for a search object (object) existing in an arbitrary area in an image, only images corresponding to the same position and area size in the presented image are extracted as similar images. There is. Usually, objects such as people and cars to be searched are located in the image,
This problem cannot be ignored because the size can be changed freely.

The present invention has been made in view of the above circumstances, and has as its object to specify an object (search target) existing in an image in a system for handling a large amount of image material, and to specify the object in another image. An object search apparatus and method capable of extracting a similar object at any position in an image, and any size without limitation to the size of a presented object. To provide.

[0009]

According to the present invention, there is provided an object area designating means for setting an object area including a designated object in a designated image, and setting the set object area in one of an X-axis direction and a Y-axis direction. And an object area dividing means for dividing into n × m blocks by dividing into n equal parts in the direction of n and m in the other direction, and a representative of the blocks for each of n × m blocks on the designated image. A representative hue value determining means for extracting a hue value which becomes a first hue value, and first profile data on the X-axis corresponding to the representative hue value of each block from the search target image converted into the hue image and Y
Profile extraction means for extracting the second profile data to the axis, and the arrangement of the hues is determined based on the corresponding first profile data and each second profile data in units of n blocks. Object searching means for searching a specified object from a search target image by detecting a region having a distribution in which the representative hue values of the blocks correspond to each other. Here, the object area may be a rectangle containing the designated object (designated object), for example, a minimum rectangle containing the designated object, that is, a circumscribed rectangle.

As described above, from the search target image converted into the hue image, the first profile data on the X-axis and the first profile data on the Y-axis with respect to the representative hue value of each block are obtained for each of n × m blocks. 2 to extract the profile data
For each set of each of the n blocks in the set, the hue of the hue is determined based on the first profile data and the second profile data for the representative hue values of the n blocks in the set. By detecting a region having a distribution in which the arrangement of the representative hue values of the n blocks matches the arrangement, even if the position and the size are different from the designated object, a combination of the same hue two-dimensionally with the designated object An image portion of a pattern (hue pattern), that is, an image having a similar object can be searched.

Here, for each of the n blocks of the m sets, a region in which the arrangement of hues has a distribution in which the representative hue values of the n blocks coincide with each other, that is, a region of the same hue pattern as that of the designated object, is used as an efficient In order to perform a proper detection, the following may be performed.

First, profile data (one-dimensional profile) for one of the X-axis and the Y-axis for the representative hue value of each of the n × m blocks is extracted from the search target image. Then, in units of n blocks,
The distribution of the representative hue values of the n blocks on each profile data is compared, and an area (profile data of n (In the case of taking a profile with respect to an axis in a direction orthogonal to the direction of the arrangement of the blocks), or an area (profile data) that satisfies the condition that the distribution is continuously adjacent in the arrangement order of the n blocks. Is a profile of an axis in a direction parallel to the direction of the array of n blocks). An area that satisfies this condition may include an image portion where the arrangement of hues matches the representative hue values of the n blocks.

Therefore, if an area that meets this condition is detected, the other of the X-axis or Y-axis with respect to the representative hue value of the corresponding n blocks is targeted for the detected area in the search target image. , Ie, the profile data on the axis opposite to the previous case is extracted, and the distribution of the representative hue values of each of the n blocks on each profile data is compared. Is a region that satisfies the condition that the distribution is continuously adjacent in the arrangement order of the n blocks (in the case where the profile data is obtained by taking a profile with respect to an axis parallel to the direction of the arrangement of the n blocks). ) Or an area that satisfies the condition that the block areas overlap between each profile data (the direction in which the profile data is orthogonal to the direction of the array of n blocks) If those taking the profile relative to the axis) is detected. An area that satisfies this condition is an image portion where the arrangement of hues matches the representative hue values of the n blocks.

Therefore, by executing the above-described region detection involving comparison of distribution on each profile data for each of the n blocks of m sets, an image portion of the same hue pattern as the designated object, That is, it is possible to search for an image having a similar object.

Further, in order to efficiently perform the area detection accompanied by the comparison of the distribution on each profile data, when the object area is divided into n × m blocks, each block is labeled (for example, , Consecutive numbering)
Is performed, the relative positional relationship between the blocks is defined, and when the representative hue values of the n × m blocks are extracted, the representative hue values of the hue values in the array corresponding to the above-mentioned labeling order are used. It is preferable to create a value table (representative hue value combination pattern data). This way,
During object search, if hue values are extracted in the order of the arrangement of hue combination patterns shown in the representative hue value table, regardless of the size of the extracted objects and their positions on the image, it is easy to determine that they match the specified object. I can judge.

By the way, the type of image, for example, a monitoring image, may be discontinuous in the Y-axis direction unlike an animation image or a graphic image. Therefore, when performing an object search for this type of image, it is preferable to use profile data for the X axis first. That is, it is preferable to determine which of the first and second profile data is used first according to the attribute information indicating the type of the search target image.

When a representative hue value for each block is extracted, the hue value having the largest area in the block is preferably used as the representative hue value, and the hue value further has a certain width up and down. The hue value having that width may be used as a true representative hue value.

In this way, an object similar to the designated object can be efficiently searched, and the search level, that is, the level regarded as a similar object can be arbitrarily set by increasing or decreasing the fixed width. .

Further, when the hue value having the largest area in the block is determined as the representative hue value, the ratio of the area to the block is calculated. An operation of adding the hue values at both ends continuous with the maximum hue value to the representative hue value may be performed until the ratio exceeds the above-described predetermined ratio, so that the representative hue value has a width. At this time, both ends of the representative hue value may have a certain width.

In this manner, an object similar to the specified object can be efficiently searched. If the number of pixels at the end of the distribution (lump area) of the profile data is equal to or larger than a predetermined threshold, the end is set as a boundary of the distribution. The profile data may be extracted again by moving the boundary of the distribution to the position described above.

In this way, the accuracy of the object search can be improved. Further, when distributions having a width equal to or larger than a predetermined width on the profile data are adjacent to each other with an interval therebetween, the width of the representative hue value may be increased by up and down by a certain value, and the corresponding profile data may be extracted again. . By doing so, it is possible to automatically correct a discontinuous portion due to some factor, and it is possible to improve the accuracy of object search.

[0022]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of an object search device according to one embodiment of the present invention.

As shown in FIG. 1, the object search apparatus 100 includes an image storage unit 101, an image presentation unit 102, a user interface unit 103, a hue image conversion unit 104, an object area designation unit 105, and a profile extraction unit 10.
6, a representative hue value determination unit 107, an object area division unit 108, an object search unit 109, and an object search result display unit 110.

The image storage unit 101 in the object search device 100 stores (saves) a large number of images taken from WWW, an image database, or the like according to the purpose.
When searching for a large number of images stored in the image storage unit 101 using images, an image portion to be searched specified by the user via the user interface unit 103 (a search target image portion) Is displayed on a display device (not shown) such as a CRT display or a liquid crystal display by the image presentation unit 102. Then, the user specifies the position of the image displayed through the image presenting unit 102 via the user interface unit 103 by an operation using an input unit (pointing device) such as a mouse or a pen, thereby searching for the image. A region having a search target image portion (object), that is, an object region is defined as an object region designation unit 1
Set at 05. The image displayed on the image presentation unit 102 can be directly captured from outside in addition to the image storage unit 101. The image in which the object area is set according to the user's designation is called an object designation image.

The object area set by the object area designating section 105 is
Are divided into n × m blocks (BK). Each block is given unique block identification information, for example, a number (block number) in the raster direction in order from the upper left. In other words, each block is labeled, and the relative positional relationship between the blocks is defined. Here, n × m numbers of 1, 2, 3,... N × m are assigned. Here, the block to which the number i (i = 1 to n × m) is assigned is referred to as a block BKi.

On the other hand, an image to be searched is retrieved from the image storage unit 101. The hue image conversion unit 4 converts the entire image of the image in which the object area is set and the image to be searched into a hue image. The image storage unit 101 normally stores a color image in the RGB format, but the image handled by the present system is a hue image. By using a hue image, it is possible to perform stable color image processing that is less affected by image acquisition conditions. Note that, for an image in which an object area is set, only an image portion of the object area may be converted to a hue image.

The representative hue value determination unit 107 uses the hue image data after image conversion by the hue image conversion unit 104 and the coordinate information of each block in the object area divided by the object area division unit 108 to generate each block. A representative hue value is determined every time. This representative hue value is
Basically, the hue value occupies the largest area in each block. The determination of the representative hue value by the representative hue value determination unit 107 greatly affects a process of extracting the range of the chunk region in the profile data on the X axis and the Y axis, which will be described later. Therefore, when the ratio of the area occupied by the representative hue value to the entire block is less than a predetermined reference ratio (threshold), hue values adjacent to both ends of the representative hue value are also considered.

Based on the hue image after the image conversion by the hue image conversion unit 4, the profile extraction unit 106 provides profile data on the X axis for a specific hue value (that is, for each X coordinate having the hue value). Histogram data representing the number of pixels) and profile data for the Y axis (that is, histogram data representing the number of pixels for each Y coordinate having the hue value) are extracted. Here, using the coordinate information of each block obtained by the object area dividing unit 108 and the representative hue value of each block obtained by the representative hue value determining unit 107, the X-axis and the Y-axis of each representative hue value are This is to extract profile data.

The profile extracting unit 106 first determines whether to extract profile data on the Y axis or profile data on the Χ axis, depending on the attribute of the image to be searched. For example, in the case of an image using a clear color such as an animation image or graphic data (hereinafter referred to as an image of type a), profile data is firstly extracted on the Y axis. On the other hand, when a perspective image using a TV camera or the like such as a monitoring image or a natural color image (hereinafter, referred to as a type b image) is handled, a general characteristic of the image is binarization processing or the like. First, profile data is extracted to the Χ axis. This has an advantage that the range of the block area using the profile data can be easily obtained in the later processing. The profile data for the Y-axis and the profile data for the X-axis are each nxm. Each of the n × m pieces of profile data corresponds to the block number of each block obtained by dividing the object area (that is, the relative positional relationship between the blocks).

The object search unit 109 determines each block area range by applying specific conditions using the profile data for the X axis and the Y axis extracted by the profile extraction unit 106. That is, the object search unit 109 obtains a candidate of the range of the chunk region in the representative hue value for each block, and overlaps in units of n pieces of profile data on the Y axis (n pieces of profile data on the X axis). A chunk region that satisfies the condition of being adjacent at consecutive positions in units of data) is extracted. If the condition is met, the profile data on the X-axis (Y-axis) is extracted from the region on the image indicated by the range of the overlapped clump region, and n profile data on the X-axis are further extracted. A chunk region that meets the condition of being adjacent at consecutive positions in data units (overlapping in units of each of the n pieces of profile data on the Y axis) is extracted. By repeating the above series of processes m times, it is possible to determine whether or not it matches the set object area.

In the object search result display section 110,
An image in which an area matching the set object area is detected is displayed on the display device. Object search device 10
0 is controlled by the system control unit 200.

Next, the object search apparatus 100 shown in FIG.
2 and 3 show the object search process in
This will be described with reference to the flowchart of FIG. Now, specified by the user through the user interface unit 103,
For example, it is assumed that a color image (object designation image) in the RGB format is displayed on the display device by the image presentation unit 102.

In this state, the user operates the image presentation unit 1
For the object designation image displayed by 02, the position of the area where the image portion to be searched (object: object) exists is designated using a mouse or a pen (step F0).

In step F0, the coordinate information of the designated area is obtained by the object area designating unit 105, and the following processing is performed. First, (for example) a minimum rectangular area including the specified area, that is, a rectangular area circumscribing the specified area is determined by the object area specifying unit 105, and the rectangular area is an object to be actually searched. Set as an area.

Next, the rectangular object area set by the object area specifying unit 105 is divided by the object area dividing unit 108 into n equal parts in the X axis direction and m parts in the Y axis direction.
It is equally divided and divided into nxm blocks (BK1 to BKnm), and each block is assigned a unique number, for example, a continuous number.

FIG. 4 is a diagram for explaining an operation for setting an object area. In the figure, L1 indicates a contour drawn with a pen or the like along the contour of a person existing in the image. When the area indicated by the outline L1 is specified as an object area, the object area specifying unit 105 obtains the smallest rectangle including the outline L1 (a rectangle circumscribing the area surrounded by the outline L1). ,
The rectangular area is set as an object area to be actually searched.

Then, control is passed from the object area designation unit 105 to the object area division unit 108. The object area dividing unit 108 converts the object area set by the object area specifying unit 105, that is, the rectangle (circumscribed rectangle) including the contour L1 in FIG.
The block is divided into n × m blocks, for example, 3 × 3 (when n = m = 3) blocks BK, and each block BK is sequentially arranged in the raster direction from the upper left in the raster direction as shown in FIG.
Assign consecutive block numbers like K2,... BK9.

Next, control is passed from the object area dividing section 108 to the representative hue value determining section 107. The representative hue value determination unit 107 obtains a hue value representing each of the n × m blocks divided by the object area dividing unit 108, and represents a representative hue value in an array in the order of the block numbers of the block. A table (representative hue value combination pattern data) is created. That is, in the present embodiment, the data of each block is processed by the hue value. When calculating a hue value (representative hue value) representative of each block, it is necessary for the hue image conversion unit 104 to convert at least the image portion of the object area in the object designation image into a hue image.

The representative hue value table (representative hue value combination pattern data) created by the representative hue value determination unit 107 is J (H1, H2,... H) in the case of n = m = 3.
9) is expressed as: Here, Hi is a hue value representing the block (BKi) having the block number i.

As shown in FIG. 5, the hue value Hi is 60 degrees (YER: yellow) and 120 degrees (GRN: green) in a clockwise direction with the uppermost part being a reference angle, that is, 0 degree (RED: red). ,
180 degrees (CYN: cyan), 240 degrees (BLE: cyan)
Blue), 300 degrees (MAZ: magenta), 360 degrees (= 0)
This is data represented by an angle such as "degree: RED). That is, the hue value Hi is represented by angle information, and the range of possible values is 0 degrees ≦ Hi ≦ 360 degrees.

As a representative hue value of each block, a hue value having the largest area in the block is basically used.
However, in the present embodiment, the representative hue value has a certain width ± α. That is, assuming that the hue value at which the area is maximum is Hmax, the hue value in the range of Hmax-α to Hmax + α is set as the representative hue value. If the ratio of the area occupied by the hue value Hmax to the entire block is less than a predetermined reference ratio (threshold), the hue value Hmax is continuously applied to both ends of the block until the ratio exceeds the reference ratio. Hue values to be added. Assuming that the width between the continuous hue values at both ends of the hue value Hmax is β, the representative hue value has a width from Hmax− (α + β) to Hmax + (α + β). By giving the representative hue value width in this way,
It is possible to reliably extract a lump region in profile data to be described later on the X axis and profile data on the Y axis.

When the processing in step F0 described above is completed, the image pointer N for designating the image to be searched is
Is set to the initial value 1 (step F1). Then, in the next step F2, the image specified by the image pointer N, that is, the N-th image is input as a search target image from the image storage unit 101 to the hue image conversion unit 104, and the entire input search target image is input. The hue image conversion unit 104
Is converted into a hue image.

In the next step F3, the type of the search target image converted into the hue image by the profile extraction unit 106 is checked. Here, an artificial image such as an animation image or a graphic image (graphic image) as a search target image is determined as type a, and a natural color image or a camera image for surveillance use is determined as type b.
The profile extraction unit 106 determines which of the extraction of the profile data on the Y axis and the extraction of the profile data on the X axis is performed first, based on the determination result of the image type (image property).

As described above, the type (attribute) of the search target image
The reason for checking is as follows. First, in a perspective image such as a monitoring image, when image processing involving threshold processing, for example, binarization processing, is performed, there is a characteristic that the image is discontinuous in the vertical direction and is interrupted. Therefore, if the profile data for the Y axis is extracted first, there is a possibility that the profile data will be adversely affected in the process of processing the mass region of the profile data. Therefore, the type of the search target image is checked, and the profile data is extracted on the Y axis or X
It is arranged to determine which of the profile data is to be extracted to the axis first.

If the search target image is of type a, the profile extraction unit 106 targets the entire area of the search target image (hue image) to the n × m number of pixels determined by the representative hue value determination unit 107. Each block (BK1-BKnm)
The profile data (Y1 to Ynm) for the Y-axis is extracted for the representative hue value (step F4).

For example, when n = m = 3, the representative hue values H1 to H9 of each of the nine blocks BK1 to BK9 are used.
For each of the profile data Y1 to Y-axis.
Y9 is extracted. Here, the profile data Yi
Is histogram data representing the number of pixels for each X coordinate having the same hue value as the representative hue value Hi. Since the representative hue value Hi has a certain width as described above, the profile data Yi accurately indicates, for each X coordinate, the total number of pixels having any hue value included in the width.

The profile data Y1 to Y9 are converted into three blocks BK1 to BK3, BK4 to BK6, BK4 to BK6 each having a common Y coordinate range in the object search unit 109.
BK7 to BK9, that is, profile data Y
1 to Y3, Y4 to Y6, and Y7 to Y9 are used for each set. Therefore, in the present embodiment, the variable i is applied, and i =
Yi, Yi + 1 for each of 1, 1, i = 4, i = 7
, Yi + 2 are processed as follows.

First, the object search unit 109 determines the representative hue values Hi, Hi + of the blocks BKi, BKi + 1, BKi + 2.
Whether there is a portion where the lump area ranges overlap in the Y-axis direction in the profile data Yi, Yi + 1, Yi + 2 on the Y-axis for 1, Hi + 2 (the initial value of i is 1). No, that is, the profile data Yi, Yi + 1, Yi + 2
It is checked whether or not the coordinate range has a distribution (lump region range) (step F5).

If the profile data Yi, Yi + 1,
If there is no portion where the block area range overlaps in Yi + 2, it is determined at this point that there is no object (similar) to the object specified by the user in the corresponding search target image. (Step F18), the pointer N is incremented by 1 (Step F10),
For the image specified by the pointer N after the increment, that is, the next image, the processing after the step F2 is performed.

On the other hand, profile data Yi,
If there is a common Y coordinate range in which the block area ranges overlap in Yi + 1 and Yi + 2, the profile extraction unit 10
Reference numeral 6 denotes the representative hue values Hi and Hi + 1 of the blocks BKi, BKi + 1, and BKi + 2 in an area (indicated by Y coordinates Ys and Ye at both ends) of the Y coordinate range in the search target image.
, Hi + 2 profile data Xi to the Χ axis
, Xi + 1, Xi + 2 are extracted (step F6).

Then, the object search unit 109 extracts the block BK extracted by the profile extraction unit 106.
i, BKi + 1, BKi + 2, representative hue values Hi, Hi + 1, H
Profile data Xi, Xi + 1 on the X axis for i + 2
, Xi + 2, it is checked whether or not there is a block area range continuously adjacent in the X-axis direction (step F).
7).

If the profile data Xi, Xi + 1,
If there is no portion where the block area range is continuously adjacent in Xi + 2, it is determined at this point that no object matching the specified object exists in the corresponding search target image ( Step F18), pointer N
Is incremented by 1 (step F10), and the processing after step F2 is performed on the image specified by the pointer N after the increment, that is, the next image.

On the other hand, the profile data Xi,
If there is a part where the chunk region ranges are consecutively adjacent to each other in Xi + 1 and Xi + 2, three horizontal ({i-1) / 3} +1 objects in the object processed first Blocks BKi, BKi + 1, BKi + 2 (for the first processing,
Since i = 1, it is determined that BK1, BK2, BK3) matches the specified object, and
After incrementing the variable i by 3 (step F8), the three blocks B indicated by the variable i after the increment of 3
Ki, BKi + 1, BKi + 2 (if i = 4, BK4
, BK5, BK6) for the same search target image, the above-described processing after step F5 is performed. Therefore, if the conditions of steps F5 and F7 described above are met, the processing of steps F5, F6 and F7 is performed with i = 1 and i =
4, i = 7, that is, three times in total.

The processing in steps F5, F6, and F7 is i =
If the search has been performed three times, i.e., 1, i = 4, i = 7, the object search unit 109 sets the search target image to
Blocks BK1 to BK3, BK4 to BK6, BK7 to B
The image is displayed by the object search result display unit 110 assuming that an object that matches the designated object exists for any of K9, that is, as including an area that matches the object area set by the object area specifying unit 105. Display on the device.

By the way, as described above, i = 1 and i =
Each of the Y coordinate ranges where the lump area ranges overlap in the Y-axis direction, which are detected from the profile data Yi, Yi + 1, and Yi + 2 for each set of 4, 4 = 7, are not necessarily continuous in the Y-axis direction. And they are not necessarily next to each other. Also, i
= 1, i = 4, i = 7, each X coordinate, which is detected from the profile data Xi, Xi + 1, Xi + 2, and where the block area range is continuously adjacent in the X-axis direction. The ranges do not always overlap in the X-axis direction. Therefore, if the fact that the respective Y coordinate ranges are continuously adjacent in the Y-axis direction and that the respective X coordinate ranges overlap in the X-axis direction are added to the conditions for the designated object search, In the units of Bi, Bi + 1, and Bi + 2, the similar designator can also prevent an image in which the similar region is discretely present from being searched, and the search accuracy can be further improved. .

When the processing in steps F5, F6, and F7 is performed three times, i = 1, i = 4, and i = 7, the pointer N is incremented by 1 (step F1).
0), if the value of the pointer N after the increment does not reach, for example, the number of images specified by the user, the image specified by the pointer N after the increment, that is, the next image, Perform processing. The processing after step F2 is repeatedly performed until the value of the pointer N after the increment reaches the number of images specified by the user, and when the number of images specified by the user reaches the number, the series of processing ends.

In the above description, the case where the search target image is of type a has been described.
The same operation is performed except that the order in which the profile data on the X-axis and the profile data on the Y-axis are used is opposite to that in the case of the search target image of type a described above. Here, first, profile data Xi, Xi + 1, Xi + 2 on the X-axis for representative hue values Hi, Hi + 1, Hi + 2 of the blocks BKi, BKi + 1, BKi + 2 are used first. (Step F12), an X coordinate range in which the chunk regions are continuously adjacent in the X-axis direction is detected in Xi, Xi + 1, and Xi + 2 (Step F13). Next, profile data Yi, Yi + 1, and Yi + 2 for the Y axis are extracted for an area in the search target image that has the detected coordinate range in common (step F14), and overlap in the Y axis direction. An existing lump area is detected (step F15).

Next, of the above-described processing in the object search apparatus 100, a method of extracting a block area for each block of three profile data, which is the core of the present embodiment, and a processing procedure will be described with reference to FIG. Will be explained. Here, it is assumed that n = m = 3 and profile data for the Y-axis is obtained first.

First, for the representative hue values H1 to H9 of the blocks BK1 to BK9, nine Y1 to Y9 profile data on the Y axis for the image to be searched are extracted. Then, a set of (Y1, Y2, Y3), (Y
4, Y5, Y6) and (Y7, Y8, Y9) for each set of profile data Yi, Yi + 1, Yi + 2 (i =
A region Ri where lump regions (1, i = 4, i = 7) overlap in the Y-axis direction is extracted.

As shown in FIG. 6, in this example, the representative hue values H1, H2, H3 of the blocks BK1, BK2, BK3.
(Assumed to be RED, YER and GRN respectively)
, Y1 has lump areas YP1-1 and YP1-2, and Y2 has lump area YP2-1.
, YP2-2, and Y3 is a lump area YP3-1, YP3-
I have two. Here, the lump area YP1-1 of Y1
And the Y coordinate range (referred to as Ys1-Ye1) of the lump area YP2-1 and the lump area YP3-1 of Y3 are the same. That is,
The lump areas of YP1-1, YP2-1, and YP3-1 overlap in the Y-axis direction. A region R1 having a Y coordinate range of Ys1-Ye1 is extracted as a region on the image of the overlapping block region. This region R1 is a target region for extracting profile data on the X axis.

Similarly, the blocks BK4, BK5, BK6
Corresponding to the representative hue values H4, H5, H6 (Y4, Y5
, Y6), there is a lump area that overlaps in the Y-axis direction in a Y-coordinate range (Ys2−Ye2), and as an area of the overlapping lump area on the image, the Y-coordinate range is Ys2-Ye.
The region R2 of e2 is extracted. Blocks BK7, B
Even for the representative hue values H7, H8, H9 of K8, BK9 (Y7, Y8, Y9), the Y coordinate range (Y
A region R3 having a Y coordinate range of Ys3-Ye3 is extracted as a region of the overlapping mass region on the image. These regions R
2, R3 are also target areas for extracting profile data on the X axis.

The profile data extraction results on the Χ axis for the region R1 in the search target image for the representative hue values H1, H2, H3 of the blocks BK1, BK2, BK3 are X1, X2, X3. . Similarly, the representative hue values H4, H4, BK4, BK5,
The extraction results of the profile data on the Χ axis for the region R2 in the search target image for H5 and H6 are X4 and X
5 and X6, and the profile data extraction results on the Χ axis for the region R3 in the search target image for the representative hue values H7, H8 and H9 of the blocks BK7, BK8 and BK9 are X7, X8 and X9.

For profile data on the Χ axis,
For each set of (X1, X2, X3), (X4, X5, X6) and (X7, X8, X9), profile data Xi, Xi + 1, Xi + 2 (i = 1, i) = 4, i =
The range where the lump region of 7) is continuously adjacent in the X-axis direction is extracted. In the example of FIG. 6, four lump regions XP1-1 of X1 have
XP1-1 of XP1-4, XP2-1 of two mass regions XP2-1 and XP2-1 of X2, and of three mass regions XP3-1 to XP3-3 of X3. XP3-3. Other XP1-2 to XP1-4, XP2-2, XP3-1
, XP3-2 are excluded as noise regions.

By performing the same processing on the remaining (X4, X5, X6) pairs and (X7, X8, X9) pairs, the blocks can be extracted in the order of the representative hue value for each block. If possible, it is determined that the object to be searched has been extracted.

It should be noted that a threshold value may be set for judging the overlap of the lump areas of Yi, Yi + 1, and Yi + 2. In this case, the ratio of the overlapping portion of the lump area to the whole is obtained,
If the ratio is equal to or greater than a threshold (reference ratio), it is determined that they overlap. Similarly, the above Xi, Xi + 1, Xi + 2
It is preferable to provide a threshold value for judging that the chunk regions are continuously adjacent to each other. In this case, the ratio of each of the adjacent lump regions of Xi, Xi + 1, and Xi + 2 to the entire overlapping portion or gap portion is obtained, and if the ratio is equal to or less than a threshold value (reference ratio), it is determined that the adjacent regions are adjacent to each other. judge.

For the profile data, in order to clarify the boundary of the chunk region, if the number of pixels at the end of the chunk region is equal to or greater than a predetermined threshold value, the chunk region is set to the chunk. If the threshold value is less than the threshold value, the boundary of the lump area is moved to a position equal to or greater than the threshold value.

When a chunk region having a width equal to or larger than a predetermined width exists adjacent to each other with a space therebetween, the width of the representative hue value is increased up and down by a fixed value γ, that is, ± is increased by γ. By re-extracting the profile data, it is possible to correct the discontinuity of the area due to the influence of noise or the like.

If it is determined that there is no object corresponding to the specified object in the search target image, the search target image is rotated in units of 90 degrees before proceeding to the next image, and the most In this case, it is possible to broaden a substantial search target by performing a search process on a 90-degree rotated image, a 180-degree rotated image, and a 270-degree rotated image in the same procedure as described above. It is.

Next, in this embodiment, even if an object (an object similar to) specified by the user is located at an arbitrary position in the search target image, the object can be detected by the same processing procedure as described above. This will be described with reference to FIG.

Since the basic extraction method has already been described, only the points will be described here. First, it is assumed that the user-designated object area has a color pattern as indicated by I1 in FIG. In this case, BΚ1 (RED), BK2 (YER), BK existing at the lower left of the image
3 (GRΝ), BK4 (MAZ), BK5 (BLE),
BK6 (RED), BK7 (RED), BK8 (CY
Images having object areas such as N) and BK9 (BRW: brown) (representative hue values in parentheses) are searched from the image group stored in the image storage unit 101.

Now, consider a case in which a user-specified object is searched for the search target image I2. In the image I2, a red area exists above the image, and a user-specified object (an object similar to) exists at a position closer to the center right.

In this example, the result of extracting the profile data on the Y axis for the representative hue values of the blocks BK1 to BK9 in the object search device 100 of FIG. 1 is Y1 to Y9 shown in FIG. Here, Y1, Y2
, Y3, it is understood that YP1-2, YP2-1, and YP3-1 exist as patterns in which the lump areas overlap in the Y-axis direction. Y1 has YP1-1 in addition to YP1-2.
However, since the region overlapping with the block region YP1-1 does not exist on the profile data of Y2 and Y3, the block region YP1-1 is excluded as a noise region.

In this case, the chunk areas YP1-2, YP2-1, Y
An area R1 in the search target image I2 in which P3-1 overlaps in the Y-axis direction becomes a target of profile data extraction on the Χ axis for representative hue values of the blocks BK1, BK2, and BK3, and X1, X2, and X3 Each profile data is extracted. Similarly, a set of Y4, Y5, Y6 and Y5, Y5,
The set of Y6 and Y7 also has a lump area overlapping in the Y-axis direction, and the search target image I2 overlapping in the Y-axis direction.
, One of the blocks BK4, BK5, BK
6, the other is to extract the profile data of the blocks BK7, BK8, and BK9 on the Χ axis with respect to the representative hue values, respectively. X4, X5, X6, and X7, X8
, X9 are extracted.

Here, comparing the profile data of X1, X2, and X3, it is found that the chunk regions are XP1-1, XP2-1, X2-1 as patterns in which the chunk regions are continuously adjacent in the X-axis direction.
It turns out that P3-1 exists. X1 has XP1-
1, there is a lump area of XP1-2, but the lump area XP1-
2 is not present on the profile data of X2 and X3, and the block area XP
1-2 are excluded as noise regions.

In the above area determination, the condition that the ranges of the chunk regions are overlapped, or the condition that the ranges of the chunk regions overlap, is not determined based on numerical information such as the data amount on the profile data and the range width of the chunk region. Since the pattern is extracted under the condition that the ranges of the regions are continuously adjacent, it is possible to extract the designated object at any position in the image.

The above processing is performed for Y4, Y5, Y6 (X4, X
5, X6) and Y7, Y8, Y9 (X7, X8, X9
) In the same way,
It is possible to detect an object region at an arbitrary position in an image that matches the hue order of a specified object.

Next, in the present embodiment, even if the object designated by the user (an object similar to the object) is located at an arbitrary position in the search target image and has an arbitrary size, the same as described above is performed. The fact that the object can be detected in the processing procedure will be described with reference to FIG.

Since the basic extraction method has already been described, only the points will be described here. First, it is assumed that the user-specified object area has a color pattern as indicated by I11 in FIG. In this case, BΚ1 (RED), BK2 (YER), BK existing at the lower left of the image
3 (GRΝ), BK4 (MAZ), BK5 (BLE),
BK6 (RED), BK7 (RED), BK8 (CY
Images having object areas such as N) and BK9 (BRW) (representative hue values in parentheses) are searched from the image group stored in the image storage unit 101.

Now, consider a case where a user-specified object is searched for the search target image I12. In the image I12, a red area exists at the lower left of the image, and an object (similar to) designated by the user exists at the upper right of the image.

In this example, the result of extracting the profile data on the Y axis for the representative hue values of the blocks BK1 to BK9 in the object search device 100 of FIG. 1 is Y1 to Y9 shown in FIG. Here, Y1, Y2
, Y3, it is found that YP1-1, YP2-1, and YP3-1 exist as patterns in which the chunk regions overlap in the Y-axis direction. Y1 has YP1-2 in addition to YP1-1.
However, since the region overlapping with the block region YP1-2 does not exist on the profile data of Y2 and Y3, the block region YP1-2 is excluded as a noise region.

In this case, the chunk areas YP1-1, YP2-1, Y
An area R10 in the search target image I12 where P3-1 overlaps in the Y-axis direction becomes a target of profile data extraction on the Χ-axis for the representative hue values of the blocks BK1, BK2, and BK3, and X1, X2, and X3 Each profile data is extracted. Similarly, a set of Y4, Y5, Y6 and Y5, Y5,
The set of Y6 and Y7 also has a lump area overlapping in the Y-axis direction, and the search target image I12 overlapping in the Y-axis direction.
, One of the blocks BK4, BK5, BK
6, the other is to extract the profile data of the blocks BK7, BK8, and BK9 on the Χ axis with respect to the representative hue values, respectively. X4, X5, X6, and X7, X8
, X9 are extracted.

Here, when comparing the profile data of X1, X2, and X3, it is found that the block regions are expressed as patterns in which the chunk regions are continuously adjacent in the X-axis direction.
It turns out that P3-1 exists. X1 has XP1-
In addition to the block area XP1-1, there is a block area XP1-1.
Since the area continuously adjacent to 1 does not exist on the profile data of X2 and X3, the block area XP
1-1 is excluded as a noise region.

In the above-described determination of the area, the condition that the ranges of the block areas overlap or the condition that the ranges of the block areas overlap each other is not determined based on the numerical information such as the data amount on the profile data and the range width of the block area. Since the pattern is extracted under the condition that the ranges of the regions are continuously adjacent, it is necessary to extract the specified object regardless of the position in the image and the size of the object is arbitrary. Becomes possible.

The above processing is performed for Y4, Y5, Y6 (X4, X
5, X6) and Y7, Y8, Y9 (X7, X8, X9
) In the same way,
It is possible to detect an object area of an arbitrary size at an arbitrary position in an image that matches the hue order of a specified object.

In the above, the object area is set to n in the X-axis direction.
The case has been described in which the object area is equally divided into m blocks in the Y-axis direction and divided into n × m blocks. The same can be applied to the case of dividing into blocks. However, when this is applied to the above-described embodiment, since the coordinate range in the X-axis direction is processing in units of n blocks in common, for example, in the case of a type a image, the processing procedure is as follows. Become.

First, the profile data on the Y axis is extracted, and the range of the clump area whose distribution is continuously adjacent in the Y axis direction is detected. Next, profile data on the X-axis is extracted for the image portion within the detection range, and a lump area whose distribution overlaps in the X-axis direction is detected. This process is repeated m times while switching the unit of n blocks.

Similarly, if the image is of type b, X
The profile data for the axis is extracted, and the range of the lump area whose distribution overlaps in the X-axis direction is detected. Next, profile data for the Y-axis is extracted for the image portion within the detection range, and a mass region whose distribution is continuously adjacent in the Y-axis direction is detected. This process is repeated m times while switching the unit of n blocks.

In the above description, the type a image is
Although the processing is performed first after the extraction of the profile data on the Y axis, the processing may be performed first on the extraction of the profile data on the X axis as in the case of the image of type b.

Note that the object search apparatus 100 applied in the above-described embodiment performs an object search for executing a processing procedure as shown in the flowcharts of FIGS. 2 and 3 on a computer, for example, a personal computer. CD-ROM with processing program recorded
The present invention can also be realized by mounting a recording medium such as a ROM, a floppy disk, or a memory card, and reading and executing a program recorded on the recording medium by a personal computer. In addition, this program
It is also possible to supply by a communication medium such as a network.

[0090]

As described above in detail, according to the present invention, an object area including a designated object in a designated image is divided into n × m blocks, and
The first profile data on the X-axis and the second profile data on the Y-axis for the representative hue value of each of the n × m blocks are extracted, and for each of the n blocks of m sets, Based on each of the first and second profile data on the representative hue values, an area having a distribution of the hue arrangement and the coincidence distribution of the representative hue values of the n blocks is detected. Even if the position and size are different, it is possible to search for an image having an image portion of the combination pattern of the same hue two-dimensionally as the designated object.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the overall configuration of an object search device according to one embodiment of the present invention.

FIG. 2 is an exemplary flowchart illustrating a part of a flow of an object search process according to the embodiment;

FIG. 3 is an exemplary remaining flowchart illustrating the flow of an object search process according to the embodiment;

FIG. 4 is a view for explaining setting of an object area in an image, division of a block targeting the object area, and block numbering of each block.

FIG. 5 is a diagram showing a relationship between respective hue values when converted into a hue image.

FIG. 6 is a view for explaining a method of extracting a block area for each block of profile data.

FIG. 7 is a view for explaining that an object specified by a user can be detected even at an arbitrary position in another image.

FIG. 8 is a view for explaining that an object specified by a user can be detected even at an arbitrary position in another image and at an arbitrary size.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 object retrieval device 101 image storage unit 102 image presentation unit 104 hue image conversion unit 105 object region designation unit 106 profile extraction unit 107 representative hue value determination unit 108 object region division unit 109 object retrieval unit 110: Object search result display section

Claims (16)

[Claims] 1. An object area specifying means for setting an object area including a specified object in a specified image, and an object area set by the object area specifying means in one of an X-axis direction and a Y-axis direction. Object area dividing means for dividing into n × m blocks by equally dividing into n and m in the other direction, and for each of the n × m blocks on the designated image, a representative of the block Representative hue value determining means for extracting the following hue value: first profile data on the X axis and second profile on the Y axis for the representative hue value of each block from the search target image converted into the hue image Profile extracting means for extracting data; and a corresponding one of the first profile data and a corresponding one of the second profiles in units of n blocks. Object search means for searching for a specified object from the search target image by detecting an area having a distribution in which the arrangement of hues matches the arrangement of the representative hue values of the n blocks based on the wheel data. An object search device, comprising: 2. An object area specifying means for setting an object area including a specified object in a specified image, and an object area set by the object area specifying means in one of an X-axis direction and a Y-axis direction. Object area dividing means for dividing into n × m blocks by equally dividing into n and m in the other direction, and for each of the n × m blocks on the designated image, a representative of the block Representative hue value determining means for extracting the following hue value: first profile data on the X axis and second profile on the Y axis for the representative hue value of each block from the search target image converted into the hue image A profile extraction unit for extracting data; and a whole region of the search target image by the profile extraction unit in units of n blocks. Are extracted based on each one of the first profile data or the second profile data corresponding to the n blocks, and the arrangement of the hue is the n A region including an image part that may possibly match the representative hue value of the block of the block is detected, and the detected n regions in the search target image are subjected to the n blocks extracted by the profile extraction unit. Detection of a region where the arrangement of hues matches the representative hue values of the n blocks in accordance with each of the other one of the corresponding first profile data or the second profile data. Performing an object search by searching for the designated object from the search target image. Object retrieval apparatus. 3. An object area specifying means for setting an object area including a specified object in a specified image, and an object area set by the object area specifying means in one of an X-axis direction and a Y-axis direction. An object area dividing unit that divides into n × m blocks by dividing into n equal parts and m equal parts in the other direction, and labeling the respective blocks to define a relative positional relationship between the respective blocks. For each of the n × m blocks on the designated image, a representative hue value of the block is extracted, and a representative hue value table including hue values in an array corresponding to the labeled order is created. A representative hue value determining unit, and a representative hue value of each block indicated by the representative hue value table from the search target image converted into the hue image. A profile extraction unit for extracting first profile data on the X axis and second profile data on the Y axis, and for each of the n blocks, the profile extraction unit extracts all of the search target images. Based on each of the first profile data or the second profile data of each of the first profile data and the second profile data corresponding to the n blocks extracted for the region, the arrangement of the hues is the representative. A region that may include an image portion that matches the arrangement of the representative hue values of the n blocks indicated in the hue value table is detected, and the detected region in the search target image is extracted by the profile extraction unit. The first profile data or the second profile data corresponding to the n blocks to be processed. Based on each of the other profile data, by detecting a region in which the arrangement of hues matches the arrangement of the representative hue values of the n blocks shown in the representative hue value table, An object search device comprising: an object search means for searching for a specified object. 4. The object search means of the first profile data or the second profile data corresponding to the n blocks for the entire area of the search target image. A common coordinate range in the first direction in which the distribution of profile data on the axis in the first direction orthogonal to the arrangement direction of the n blocks overlaps in the first direction is detected. Of the first profile data or the second profile data corresponding to the n blocks in the area within the search target image in the set coordinate range, in the arrangement direction of the n blocks Between the respective profile data along the axis in the second direction parallel to the distribution of the n blocks is detected in the order in which the blocks are consecutively adjacent in the second direction. 4. The object search device according to claim 2, wherein 5. The object search means of the first profile data or the second profile data corresponding to the n blocks for the entire area of the search target image. Between each of the profile data on the axis in the first direction parallel to the arrangement direction of the n blocks, the distribution in the order of arrangement of the n blocks is continuous in the first direction. The coordinate range of the area in the first direction is detected, and each of the first profile data or each of the first profile data corresponding to the n blocks for the area in the search target image in the detected coordinate range is detected. Of the two profile data, the distribution of the profile data on the axis in the second direction orthogonal to the arrangement direction of the n blocks overlaps in the second direction. 4. The object search device according to claim 2, wherein the detection of the area is performed. 6. The method according to claim 1, wherein the object search unit determines which one of the first and second profile data is used first according to attribute information indicating a type of the search target image. 4. The object search device according to claim 2, wherein: 7. The representative hue value determining means obtains a hue value having the largest area in a block, assigns the hue value a certain width up and down, and sets the hue value as a representative hue value of the block. The object search device according to any one of claims 1 to 3, wherein: 8. The representative hue value determining means sets a hue value having the largest area in the block as a representative hue value, and when a ratio of the area to the block is equal to or less than a predetermined ratio, the representative hue value is determined. 4. An operation of adding the hue values at both ends which are continuous with the maximum hue value to the representative hue value until the predetermined hue value is exceeded, so that the representative hue value has a width. An object search device according to any one of the above. 9. The object search apparatus according to claim 8, wherein said representative hue value determination means has a further fixed width above and below said representative hue value. 10. The profile extraction means, if the number of pixels at the end of the distribution of the first and second profile data is equal to or greater than a predetermined threshold value, determines the end of the distribution. The method according to claim 1, wherein when the value is less than the threshold value, the boundary of the distribution is moved to a position equal to or greater than the threshold value, and the first and second profile data are extracted again. 3. The object search device according to any one of 3. 11. The profile extracting means increases or decreases the width of a representative hue value when a distribution having a width equal to or more than a predetermined width exists on the first or second profile data at intervals. 4. The object search device according to claim 1, wherein the corresponding profile data is re-extracted by expanding the profile data by a predetermined value. 12. The object search means, when a specified object cannot be searched from the search target image, performing a search again by rotating the search target image in units of 90 degrees with 270 degrees as an upper limit. The object search device according to any one of claims 1 to 3, wherein: 13. A first step of setting an object area including a specified object in a specified image, and dividing the set object area into n equal parts in one of an X-axis direction and a Y-axis direction, and the other. A second step of dividing the block into n × m blocks by dividing the block into m equal parts in the direction of, and extracting a representative hue value of each of the n × m blocks on the designated image A third step, a fourth step of converting the search target image into a hue image, and an X-axis or a Y-axis with respect to the representative hue value of each block for the entire region of the search target image converted into the hue image. A fifth step of extracting first profile data for one of the axes, and, based on each of the first profile data corresponding to the n blocks, the arrangement of the hue is n A sixth step of detecting an area that may include an image part that matches the arrangement of the representative hue values of the blocks of the blocks, and the image part that matches the arrangement of the representative hue values of the n blocks in the sixth step. When an area that may be included is detected, the detected area in the search target image is set as a target and the other of the X axis or the Y axis with respect to the representative hue value of the n blocks is moved to the other axis. A seventh step of extracting the second profile data of the above, and based on each of the second profile data corresponding to the n blocks extracted in the seventh step, the arrangement of the hue is the number of the n An eighth step of detecting a region in which the representative hue values of the blocks are aligned with each other, and performing the sixth to eighth steps by switching the n blocks in units of the n blocks. Object search method characterized by comprising a ninth step of repeating. 14. When the first profile data corresponding to the n blocks is profile data on an axis in a first direction orthogonal to the arrangement direction of the n blocks, In a sixth step, a common coordinate range in the first direction in which distributions of the respective first profile data overlap each other in the first direction is detected, and in the seventh step, the first A second direction parallel to the arrangement direction of the n blocks on the X-axis or the Y-axis with respect to the representative hue value of the n blocks for an area in the search target image in the coordinate range in the direction of Extracting the second profile data on the axis of the nth block, and in the eighth step, between the respective second profile data on the axis in the second direction parallel to the arrangement direction of the n blocks. The n 14. The object search method according to claim 13, wherein an area whose distribution is continuously adjacent in the second direction in the arrangement order of the blocks is detected. 15. The method according to claim 15, wherein the first profile data corresponding to the n blocks is profile data on an axis in a first direction parallel to an arrangement direction of the n blocks. In the step (6), the distribution in the first direction of the continuous area adjacent to the distribution of the n blocks in the first direction in the order of arrangement of the n blocks among the respective first profile data. In the seventh step, an area within the search target image in the coordinate range in the first direction is set as an object, and an X-axis or a Y-axis of a representative hue value of the n blocks is determined. Extracting the second profile data on the axis in the second direction orthogonal to the arrangement direction of the n blocks. In the eighth step, a second profile data orthogonal to the arrangement direction of the n blocks is extracted. To axis of direction Detecting a common area whose distribution is overlapped in the second direction among the respective second profile data.
Object search method described. 16. A first step of setting an object area including a specified object in a specified image, and dividing the set object area into n equal parts in one of an X-axis direction and a Y-axis direction, and the other. A second step of dividing into n × m blocks by dividing into m equal parts in the direction of, and extracting a representative hue value of each of the n × m blocks on the designated image. A third step, a fourth step of converting the search target image into a hue image, and an X-axis or a Y-axis with respect to the representative hue value of each block for the entire region of the search target image converted into the hue image. A fifth step of extracting first profile data for one of the axes, and an arrangement of hues based on each of the first profile data corresponding to n blocks. Performing a detection of a region that may include an image portion that matches the arrangement of the representative hue values of the n blocks, and the arrangement of the representative hue values of the n blocks in the sixth step. When a region that may include a matching image portion is detected, the detected region in the search target image is set as a target and the X-axis or the Y-axis with respect to the representative hue value of the n blocks is used. A seventh step of extracting second profile data to the other axis of, and a hue arrangement based on each of the second profile data corresponding to the n blocks extracted in the seventh step. An eighth step of detecting a region in which the representative hue values of the n blocks coincide with each other, and switching the n blocks in units of the n blocks while performing the switching of the n blocks. And a ninth step of repeating m times the eighth step, a computer readable recording medium recording a program to be executed by a computer.