JP2010033464A - Image retrieval device, image retrieving method, information processing program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce throughput required for retrieval in an image retrieval system having a partial matching function. <P>SOLUTION: This image retrieval device for calculating similarity between an image to be retrieved and an image as a retrieval condition to execute image retrieval has: an image processing part 102 for acquiring the image to be retrieved, extracting a partial image showing a portion of the acquired image, generating numerical string information on the basis of the partial image, and generating subtraction numerical string information by excluding the numerical string information generated on the basis of the partial image from the numerical string information generated on the basis of the image to be retrieved; and a similarity calculating part 103 for calculating the similarity on the basis of the generated subtraction numerical string information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image search device, an image search method, an information processing program, and a recording medium, and more particularly to a search method in image search.

  Search technology for electronic data or search result display technology has become an increasingly important technology because of the increase in the number of search results due to an increase in the amount of information to be searched. This is because the information that is sought is buried in a large amount of search results, making it difficult to find. As such a search technique, an image search method for inputting an image as a search image and extracting an image similar to the input image from an already stored search image has been proposed (for example, Patent Documents). 1).

An image search technique such as that disclosed in Patent Document 1 is used, for example, for searching published patent publications. For example, when searching for a drawing included in a document such as a published patent publication, the drawing is set as a search key, and a document including a drawing similar to the drawing is extracted. In addition, in the case of searching for image information using image information as a search key in a database storing a plurality of image information, the same can be used.
JP 2002-245087 A

  In a conventional image search as disclosed in Patent Document 1, the entire image set as a search key (hereinafter referred to as a key image) and the entire image to be searched (hereinafter referred to as a search target image) are similar. It is assumed that the degree is judged. For this reason, even if the search target image includes an image similar to the key image, the degree of similarity is not calculated high in relation to the whole image.

  On the other hand, the inventor of the present application invented a method for extracting a plurality of regions such as a document region, a photographic region, a table region, and the like from an image and matching the images for each region, and filed an application (special feature). Application No. 2007-022108). In the above method, numerical sequence data such as a histogram is generated for each extracted region, and image matching is performed based on the numerical sequence data. Thereby, an image search can be performed for a part of the key image and the search target image. Such matching by a part of an image is particularly effective in trademark search or the like.

  Generally, an image search system such as a trademark search is provided by a client / server system via a network. In such an online search system, it is important to shorten the response time for the operation of the client. However, in the method according to the above application, it is necessary to calculate numerical string data for a plurality of regions included in one image. Therefore, even if the calculation amount of each numerical sequence data is small, if the amount of search target images becomes enormous, the total calculation amount required for calculating the numerical sequence data also becomes enormous. It is not realistic to operate a system that requires an enormous amount of processing as a client / server system.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to reduce the processing amount required for searching in an image search system that enables partial matching.

  In order to solve the above-described problem, the invention according to claim 1 is an image search device that performs an image search by calculating a similarity between an image to be searched and an image of a search condition, wherein the image to be searched Alternatively, numerical value sequence information is generated based on a processing target image that is one of the search condition images, and the similarity is calculated by generating feature information of the image based on the generated numerical sequence information. A processing target image acquisition unit that acquires a processing target image that is one of the search target image or the search condition image, and a partial image that generates a partial image indicating a part of the processing target image A generation unit, a numerical sequence information generation unit that generates numerical sequence information based on the partial image, and a numerical sequence generated based on the partial image from the numerical sequence information generated based on the processing target image And subtracting the numerical sequence information generating unit which generates a subtraction numeric sequence information except broadcast, and having a similarity calculation section for calculating the degree of similarity based on the generated subtraction numerical string information.

  Moreover, the invention according to claim 2 is the image search device according to claim 1, wherein the partial image generation unit uses a region where characters are displayed in the processing target image as a part of the processing target image. The partial image is generated.

  According to a third aspect of the present invention, in the image search device according to the first or second aspect, the partial image generation unit extracts a part of the processing target image to generate the partial image. Features.

  According to a fourth aspect of the present invention, in the image search device according to the first or second aspect, the partial image generation unit may display an image in which only a part of the processing target image is displayed in the processing target image. The partial image is generated as the partial image.

  According to a fifth aspect of the present invention, in the image search device according to any one of the first to fourth aspects, the numerical sequence information generating unit projects the pixels constituting the processing target image in a one-dimensional manner. A projection histogram is generated, and the numerical string information is generated based on the generated projection histogram.

  According to a sixth aspect of the present invention, in the image search device according to any one of the first to fifth aspects, the processing target image is a binary image, and the numerical sequence information generating unit The numerical column information is generated by counting pixel components for at least one of a column and a row of pixels constituting the pixel column.

  In addition, according to a seventh aspect of the present invention, in the image search device according to any one of the first to sixth aspects, the partial image generation unit is configured to select one of the processing target images based on pixels constituting the processing target image. It is characterized by determining a part.

  Moreover, the invention according to claim 8 is the image search device according to claim 7, wherein the partial image generation unit extracts a connected component in a pixel constituting the processing target image, and extracts the connected component to the extracted connected component. A part of the processing target image is determined on the basis of this.

  The invention according to claim 9 is the image search device according to claim 8, wherein the processing target image is a binary image, and the partial image generation unit is continuous in pixels constituting the processing target image. The same pixel component is extracted as the connected component.

  The invention described in claim 10 is the image search device according to claim 8, wherein the processing target image is a chromatic color image, and the partial image generation unit is configured to include pixels in the processing target image. A plurality of pixels that are continuous and have similar colors are extracted as the connected components.

  The invention according to claim 11 is an image search method for performing an image search by calculating a similarity between the search target image and the search condition image, wherein the search target image or the search condition image The numerical value sequence information is generated based on a processing target image that is one of the images, and the similarity is calculated by generating feature information of the image based on the generated numerical sequence information. The target image acquisition unit acquires a processing target image that is either the search target image or the search condition image, and the partial image generation unit generates a partial image indicating a part of the processing target image. The numerical sequence information generation unit generates numerical sequence information based on the partial image, and the subtraction numerical sequence information generation unit generates based on the partial image from the numerical sequence information generated based on the processing target image. Is Except for the numerical sequence information to generate a subtraction numeric sequence information, the similarity calculating unit, and calculates the degree of similarity based on the generated subtraction numeric string information.

  The invention according to claim 12 is an information processing program that causes an information processing apparatus to function as an image search apparatus that calculates a similarity between an image to be searched and an image of a search condition and executes an image search. The image search device generates numerical sequence information based on a processing target image which is either the search target image or the search condition image, and the image feature based on the generated numerical sequence information. Calculating the similarity by generating information, obtaining a processing target image which is either the search target image or the search condition image, and a part of the processing target image. A step of generating a partial image to be shown; a step of generating numerical sequence information based on the partial image; and the portion from the numerical sequence information generated based on the processing target image Causing the information processing apparatus to execute a step of generating subtraction numerical sequence information excluding the numerical sequence information generated based on an image and a step of calculating the similarity based on the generated subtraction numerical sequence information It is characterized by that.

  The invention according to claim 13 is a recording medium, wherein the information processing program according to claim 12 is recorded in a format readable by the information processing apparatus.

  According to one aspect of the present invention, it is possible to reduce the amount of processing required for a search in an image search system that enables partial matching.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, as an example of an image search apparatus, an apparatus that searches search target image information stored in advance based on an input image and extracts an image similar to the input image will be described as an example. .

  FIG. 1 is a diagram illustrating an example of an operation mode of an image search system according to the present embodiment. As shown in FIG. 1, the image search system according to the present embodiment includes an image search device 1, a client device 2, and an image information DB 200. The image search device 1, the client device 2, and the image information DB 200 are connected via the network 3.

  The client device 2 is configured by a general information processing device such as a PC (Personal Computer). For example, the client device 2 can be configured by a mobile communication terminal such as a mobile phone, an MFP (Multi Function Peripheral), or the like. The image search device 1 is connected to the client device 2 via a network, and searches for image information stored in the image information DB 200 in response to a search request from the client device 2. The image information DB 200 stores a plurality of pieces of image information as search target information. The image information DB 200 is configured by a nonvolatile storage medium such as an HDD (Hard Disk Drive).

  As shown in FIG. 1, in this embodiment, an example in which the image information DB 200 is provided separately from the image search device 1 will be described. However, the image information DB 200 may be configured inside the image search device 1. Is possible. Furthermore, the image information DB 200, the image search device 1, and the client device 2 may be configured as a single device.

  Next, a hardware configuration of the image search device 1 according to the present embodiment will be described. FIG. 2 is a block diagram illustrating a hardware configuration of the image search apparatus 1 according to the present embodiment. As shown in FIG. 2, the image search apparatus 1 according to the present embodiment has the same configuration as an information processing terminal such as a general server or a PC (Personal Computer).

  That is, in the image search apparatus 1 according to the present embodiment, a CPU (Central Processing Unit) 10, a RAM (Random Access Memory) 20, a ROM (Read Only Memory) 30, an HDD 40, and an I / F 50 are connected via a bus 80. ing. Further, an LCD (Liquid Crystal Display) 60 and an operation unit 70 are connected to the I / F 50.

  The CPU 10 is a calculation means and controls the operation of the entire image search apparatus 1. The RAM 20 is a volatile storage medium capable of reading and writing information at high speed, and is used as a work area when the CPU 10 processes information. The ROM 30 is a read-only nonvolatile storage medium and stores a program such as firmware. The HDD 40 is a non-volatile storage medium that can read and write information, and stores an OS (Operating System), various control programs, application programs, and the like.

  The I / F 50 connects and controls the bus 80 and various hardware and networks. The LCD 60 is a visual user interface for the user to check the state of the image search device 1. The operation unit 70 is a user interface for the user to input information to the image search apparatus 1 such as a keyboard and a mouse.

  As described with reference to FIG. 1, the image search apparatus 1 according to the present embodiment is operated as a server. Therefore, user interfaces such as the LCD 60 and the operation unit 70 can be omitted. In addition, as long as the image search device 1 is a terminal having an information processing function, any realization mode may be used. For example, a general PC, MFP, etc. can be considered.

  In such a hardware configuration, a program stored in a storage medium such as the ROM 30, the HDD 40, or an optical disk (not shown) is read into the RAM 20, and operates according to the control of the CPU 10, thereby configuring a software control unit. A functional block for realizing the functions of the image search apparatus 1 according to the present embodiment is configured by a combination of the software control unit configured in this way and hardware.

  Next, functional blocks of the image search device 1 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram illustrating an image information DB 200 that stores functional blocks of the image search device 1 according to the present embodiment and image information to be searched by the image search device 1. As illustrated in FIG. 3, the image search apparatus 1 according to the present embodiment includes a search control unit 100, an information input unit 110, a network I / F 120, and a display unit 130.

  The information input unit 110 is configured to allow a user to operate the image search apparatus 1 and input information to the search control unit 100, and is realized by the I / F 50 and the operation unit 70 illustrated in FIG. The network I / F 120 is an interface for the image search apparatus 1 to acquire information via the network or transmit information via the network, and is realized by the I / F 50 illustrated in FIG. Specifically, it is realized by, for example, an Ethernet (registered trademark) connection interface or a USB (Universal Serial Bus) connection interface.

  The display unit 130 is configured to display the operation state of the image search apparatus 1, search results, and the like, and is realized by the I / F 50 and the LCD 60 shown in FIG. The search control unit 100 is configured to perform the search function of the image search apparatus 1 according to the present embodiment, and includes an input image information acquisition unit 101, an image processing unit 102, a similarity calculation unit 103, and an extraction result processing unit 104.

  The input image information acquisition unit 101 uses the image information input by the user via the information input unit 110 or the image information input via the network via the network I / F 120 as input image information, that is, a condition image that serves as a search condition. Get as. The input image information acquisition unit 101 is configured by a program loaded in the RAM 20 shown in FIG.

  The image processing unit 102 extracts a part of the search target image stored in the condition image acquired by the input image information acquisition unit 101 or the image information DB 200 as a partial image. Further, the image processing unit 102 generates numerical string information based on the condition image, the search target image, and the extracted partial image. The image processing unit 102 according to the present embodiment generates a projection histogram of pixels as numerical string information. The projection histogram generation process will be described in detail later.

  Further, the image processing unit 102 performs a subtraction process between the numerical sequence information generated based on the entire image of either the condition image or the search target image and the numerical sequence information generated based on the partial image extracted from the entire image. And generate subtraction numerical information. This will also be described in detail later. Then, the image processing unit 102 generates feature information indicating the feature of the image based on the generated numerical sequence information and subtracted numerical sequence information. The feature information according to the present embodiment is symbol string information obtained by converting the numeric string information into a symbol based on a predetermined rule.

  The similarity calculation unit 103 calculates the similarity between the condition image, the search target image, and the partial image based on the feature information generated by the image processing unit 102. Then, the similarity calculation unit 103 generates information (hereinafter referred to as similarity list information) in which the search target image or partial image is associated with the similarity of the image. The similarity calculation unit 103 is configured by a program loaded in the RAM 20 illustrated in FIG. 2 operating according to the control of the CPU 10. The similarity calculation unit 103 compares feature information between the condition image and the search target image or partial image, and calculates the similarity. As a method for calculating the similarity, an existing image search technique can be used.

  The extraction result processing unit 104 displays the similarity list information generated by the similarity calculation unit 103 on the display unit 130 or generates and outputs display information to be displayed on the display unit of the client device 2. That is, the extraction result processing unit 104 functions as a display information generation unit that generates display information for displaying the calculation result of the similarity. The display information generation unit is configured by a program loaded in the RAM 20 shown in FIG. When the display unit 130 is used, the display information is used as information for displaying an image extracted as a similar image.

  In such an image search device 1, the present embodiment is characterized in the processing by the image processing unit 102. Hereinafter, the operation of the image search apparatus 1 according to the present embodiment will be described with reference to the drawings. FIG. 4 is a sequence diagram showing a search operation in the image search system according to this embodiment.

  As shown in FIG. 4, when searching for image information registered in the image information DB 200, first, the user operates the client device 2 to display information for displaying a search image input screen in the image search device 1. A request is made (S401). In response to the request from the client device 2, the image search device 1 transmits information for displaying the search image input screen (S402).

  Upon acquiring the search image input screen from the image search device 1, the client device 2 displays the search image input screen (S403). Here, the search image input screen is a screen for inputting search image information. Hereinafter, in the description of the present embodiment, a case where the user operates the client device 2 and uses the function of the image search device 1 will be described as an example.

  In accordance with the operation by the user, the client device 2 transmits information on the input image input through the search image input screen (hereinafter referred to as input image information) to the image search device 1 (S404). An example of input image information input in this embodiment is shown in FIG. The input image information transmitted to the image search device 1 is input from the network I / F 120 to the image search device 1 and acquired by the input image information acquisition unit 101 of the search control unit 100 (S405). The input image information acquired by the image search device 1 in S405 is used as a condition image.

  The image search device 1 that has acquired the input image information as a condition image extracts a partial image from the condition image or the search target image (S406). The partial image is a partial image of the condition image or the search target image. Then, the image search device 1 generates numerical string information and feature information based on the condition image, the search target image, and the partial image, and calculates the similarity (S407). As described above, similarity list information is generated as a result of calculating the similarity. The processing of S406 and S407 will be described in detail later.

  When the image search device 1 generates similarity list information as a result of the process of S407, the image search device 1 extracts an image whose similarity exceeds a predetermined threshold as a similar image. That is, the similarity calculation unit 103 functions as an image information extraction unit. The image information extraction unit is configured by a program loaded in the RAM 20 shown in FIG.

  The threshold value is a value set to determine whether or not the search target image is a similar image based on the calculated similarity. That is, it is information for determining whether the information stored in the image information DB 200 or the region image extracted therefrom corresponds to a similar image. In other words, the value is set to exclude an image whose characteristics do not match the condition image from the extraction result.

  The threshold is set according to the similarity calculation method in S407. For example, in the feature amount comparison process in S407, an image whose features do not match is calculated as a similarity of “0%”, and an image whose features match even a little is calculated as a similarity which is not “0%”. When the calculation mode is used, the threshold value is set as “0%”.

  In addition, the threshold value may be set to a value other than “0%” depending on the similarity calculation method in S407. Further, the threshold value may be set to extract an image having a high degree of matching between the condition image and the feature, in addition to setting an image whose feature does not match the condition image from the extraction result. In this case, the above threshold is set to a higher value.

  When the similar image is extracted, the image search device 1 determines the display order of the extracted similar image. Then, display information for displaying the extraction result is generated and transmitted to the client apparatus 2 via the network I / F 120 (S408). In step S408, the image search device 1 determines the order in which similar images are displayed in the order of the calculated similarity. The client device 2 displays the extraction result on the display unit based on the display information received from the image search device 1 (S409), and ends the process.

  Next, detailed processing in S406 and S407 in FIG. 4 will be described with reference to the flowchart in FIG. In the present embodiment, a case where the similarity between the condition image shown in FIG. 5 and a part of the search target images included in the image information DB 200 is calculated will be described as an example. When the designated image information is acquired in S405 of FIG. 4, the image processing unit 102 acquires an image (hereinafter, referred to as a processing target image) that is a processing target of region extraction and numerical string information (S601). That is, the image processing unit 102 functions as a processing target image acquisition unit. The processing target image acquisition unit is configured by a program loaded in the RAM 20 illustrated in FIG. In the present embodiment, the search target image is acquired from the image information DB 200.

  An example of an image acquired by the image processing unit 102 from the image information DB 200 in S601 is shown in FIG. In the present embodiment, an example will be described in which a monochrome image is processed as shown in FIG. FIG. 8 is a diagram illustrating an example of information stored in the image information DB 200. As shown in FIG. 8, the image information DB 200 includes, in addition to information on each image, an ID for uniquely identifying each image, and a numerical string information on each image (hereinafter referred to as a target image ID). Is stored in association with The numerical string information will be described in detail later. In step S <b> 601, the image processing unit 102 also acquires numerical string information illustrated in FIG. 8 together with the processing target image.

  When the image processing unit 102 acquires the processing target image, the image processing unit 102 analyzes the pixels of the acquired processing target image and extracts a circumscribed rectangle of the connected black pixels (S602). FIG. 9 is a diagram illustrating the processing result of S602. As shown in FIG. 9, circumscribed rectangles of connected black pixels are extracted. Next, the image processing unit 102 analyzes the rectangle extracted in S602 and determines a character area (S603). In step S <b> 603, the image processing unit 102 determines whether each rectangle is a character or a part of a character based on the interval between adjacent rectangles or the size of the rectangle.

Furthermore, the image processing unit 102 determines a character area based on the arrangement relationship between adjacent rectangles. FIG. 10 shows the determination result of the character area. In the present embodiment, as shown in FIG. 10, a rectangular portion displayed as “parking prohibited” is determined as a character area. In the present embodiment, as shown in FIG. 10, the character area is specified by the coordinates of the upper left end and the lower right end of the rectangle. The image processing unit 102 holds information on the coordinates “(X ₁ , Y ₁ )” and “(X ₂ , Y ₂ )” shown in FIG.

It should be noted that various known methods can be used to determine the character area from the above rectangular extraction, and thus detailed description thereof will be omitted. When determining the character area, the image processing unit 102 extracts the character area as a partial image (S604). That is, the image processing unit 102 functions as a partial image generation unit. The partial image generation unit is configured by a program loaded in the RAM 20 shown in FIG. FIG. 11 shows an image extracted as a partial image in the present embodiment. As shown in FIG. 11, in the present embodiment, only the character region determined by the coordinates “(X ₁ , Y ₁ )” and “(X ₂ , Y ₂ )” is extracted as a partial image.

  When the partial image is extracted, the image processing unit 102 generates numerical sequence information (hereinafter referred to as partial numerical sequence information) based on the extracted partial image (S605). That is, the image processing unit 102 functions as a numerical string information generation unit. The numerical string information generation unit is configured by a program loaded in the RAM 20 shown in FIG.

  FIG. 12 shows an example of how numeric string information is generated in S605. FIG. 12 is a diagram illustrating an example of pixel information of the partial image processed in S605 and numerical value string information generated from the pixel information. In the pixel information of FIG. 12, “1” is displayed for black pixels and “0” for white pixels. As illustrated in FIG. 12, the image processing unit 102 calculates the total value of black pixels in a pixel column, and generates numerical value column information. That is, in step S605, the image processing unit 102 calculates a projection histogram of the partial image, and extracts numerical string information from the projection histogram.

  When the partial numerical sequence information is generated, the image processing unit 102 executes a process of subtracting the partial numerical sequence information from the numerical sequence information of the processing target image acquired from the image information DB 200 together with the processing target image in S601 (S606). That is, the image processing unit 102 functions as a subtraction numerical value string information generation unit. The subtraction numerical string information generation unit is configured by a program loaded in the RAM 20 shown in FIG.

  With reference to FIGS. 13A to 13C, the processing of S606 will be described. FIG. 13A is a diagram showing a histogram of the processing target image shown in FIG. 7 and its numeric string information (hereinafter referred to as overall numeric string information), that is, a projection histogram. In the histograms shown in FIGS. 13A to 13C, the vertical axis represents the total number of black pixels. In the histograms shown in FIGS. 13A to 13C, the horizontal axis indicates a pixel row in the processing target image.

FIG. 13B is a diagram illustrating a histogram of the numerical value string information of the region image extracted in S604 and the partial image generated in S605. In step S606, the image processing unit 102 subtracts the value of the partial numerical sequence information for each corresponding column from the entire numerical sequence information. At this time, the image processing unit 102 refers to the coordinates “(X ₁ , Y ₁ )” and “(X ₂ , Y ₂ )” shown in FIG. Identify

  Thereby, numerical sequence information (hereinafter referred to as subtracted numerical sequence information) as shown in FIG. 13C is generated. By subtracting the value of the partial numerical sequence information from the entire numerical sequence information, the numerical sequence information of the portion excluding the partial image from the processing target image is obtained. That is, as shown in FIG. 13C, it is possible to obtain numerical value string information of a portion of the sign (hereinafter referred to as a subtraction image) excluding the region where “parking prohibited” is displayed from the processing target image.

  When generating the subtraction value string information, the image processing unit 102 generates feature information based on each of the whole value string information, the partial value string information, and the subtraction value string information (S607). The image processing unit 102 also calculates feature information of the condition image in the process of S607. When calculating the feature information of the condition image, the feature information is generated after the numerical sequence information (hereinafter referred to as the conditional numerical sequence information) is generated based on the condition image in the same manner as described above.

  Various existing methods can be used for generating the feature information. For example, there are a method for generating a symbol based on the magnitude relationship between adjacent numerical values in the numerical sequence information, and a method for quantizing the numerical sequence information. There is also a method of smoothing the numerical string information by calculating a moving average. Furthermore, the numerical string information itself can be used as feature information.

  When the image processing unit 102 calculates the feature information through the process of S607, the image processing unit 102 inputs the calculated feature information to the similarity calculation unit 103. The similarity calculation unit 103 calculates the similarity based on the feature information acquired from the image processing unit 102 (S608). In this embodiment, the similarity between the condition image shown in FIG. 5 and the images shown in FIGS. 13A to 13C, that is, the processing target image, the partial image, and the subtraction image is calculated. As described above, the similarity calculation unit 103 compares each feature information, and calculates the similarity by quantifying the similarity.

  In the present embodiment, as shown in FIGS. 5 and 13C, the condition image and the subtraction image are substantially the same image. Thereby, the similarity of the subtraction image shown in FIG.13 (c) is calculated highly. That is, an image search can be executed for a part of the search target image. The similarity calculation unit 103 employs the highest value among the similarities calculated for each of the processing target image, the partial image, and the subtraction image illustrated in FIGS. To do.

  After inputting the feature information to the similarity calculation unit 103, the image processing unit 102 checks whether or not the processing of S601 to S608 has been executed for all the search target images stored in the image information DB 200 (S609). ). If the processing has been completed for all the search target images (S609 / Yes), the image processing unit 102 and the similarity calculation unit 103 end the processing, and proceed to S408 in FIG. On the other hand, if the processing has not been completed for all the search target images (S609 / No), the image processing unit 102 repeats the processing from S601.

  In the image search device 1 according to the present embodiment, the partial numeric string information of the partial image extracted as a character area is calculated, and the partial numeric string information is subtracted from the entire numeric string information to thereby obtain the numeric string of the remaining area. Information can be obtained. That is, by calculating the numerical value string information for one area, it is possible to obtain numerical value string information for two areas. Although a process called subtraction is separately required, since the subtraction process is a process with a low load, there is no significant influence on the purpose of reducing the processing amount.

  As described above, the image search apparatus according to the present embodiment can reduce the processing amount required for the search in the image search system that enables partial matching. The image search system according to this embodiment is particularly effective as a trademark search system. In the trademark search, as shown in FIG. 7, there are many images in which graphics and characters are combined as search targets. This is because by applying the image search system according to the present embodiment, it is possible to provide a search system that enables partial matching in a realistic processing time.

  Further, as an example of the image information stored in the image information DB 200 that is a search target, information on a drawing included in a patent document such as a published patent publication can be considered. This is because the drawings included in the patent document are displayed in black and white and are suitable for the above-described example of the projection histogram. In addition, drawings included in the patent document may have explanatory characters displayed in the drawings. This is because partial matching is required in such a case.

  In the above description, as illustrated in FIG. 8, the example in which the entire numerical value string information is stored in the image information DB 200 has been described. Thereby, it becomes possible to reduce the processing amount required in the response to the search request from the user. In addition, the entire numerical value string information may be calculated by the image processing unit 102 during the processing shown in FIG. Thereby, the amount of information stored in the image information DB 200 can be reduced.

  In the above embodiment, as described with reference to FIG. 12, the number of black pixels is counted to extract the numerical value string information. In addition, the numerical value string information may be extracted by counting white pixels. In addition, when a search is executed based on a color image other than a monochrome binary image, the same processing as described above can be applied by binarizing the image. In addition, when performing a search based on a color image, in addition to the case of binarizing the image, the similarity of the colors between adjacent pixels is determined. It is possible to execute the same processing as in FIG.

  Further, in the above embodiment, as described with reference to FIG. 12, the example in which the pixels are counted for each pixel column has been described. In addition, the pixels may be counted for each pixel row, or the count results of both the pixel column and the pixel row may be used.

  Moreover, in the said embodiment, as shown to FIG. 10, FIG. 11, the example which extracts a character area as a partial image was demonstrated. In addition, a graphic region may be extracted as a region image. However, since various methods have already been proposed for extracting a character area, the present embodiment can be readily realized by extracting a character area. In addition, an area where a graphic or the like is displayed has various characteristics depending on the graphic, whereas a character area has certain characteristics. Therefore, by setting the area to be extracted as the area image as the character area, it is possible to improve the accuracy of area extraction.

  Further, in the above-described embodiment, as described in FIG. 6, the example in which the search target image stored in the image information DB 200 is the processing target image has been described. In addition, an input image input by the user, that is, a condition image may be used as the processing target image. This makes it possible to execute a matching search between a part of the input image and the search target image stored in the image information DB 200. Furthermore, both the condition image and the search target image may be set as processing target images. This makes it possible to execute a matching search between part of the input image and part of the search target image.

In the above embodiment, as described with reference to FIG. 11, only the character region determined by the coordinates “(X ₁ , Y ₁ )” and “(X ₂ , Y ₂ )” is extracted as a partial image. An example was explained. In addition, as shown in FIG. 14, an image in which information in an area other than the area determined as a character area is deleted may be generated as a partial image. In this case, the partial image is the same size as the entire image. Therefore, the number of numerical values in the entire numerical value string information and the partial numerical value string information is also the same. As a result, in the subtraction process shown in FIGS. 13A to 13C, it is not necessary to specify a range in which the partial numerical value string information is subtracted in the entire numerical value string information, and the partial numerical value string information is simply obtained from the entire numerical value string information. Subtract.

It is a figure which shows the operation | use form of the image search system which concerns on embodiment of this invention. It is a block diagram which shows typically the hardware constitutions of the image search device which concerns on embodiment of this invention. It is a block diagram which shows the function structure of the image search device which concerns on embodiment of this invention. It is a sequence diagram which shows operation | movement of the image search system which concerns on embodiment of this invention. It is a figure which shows the example of the condition image which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement of area | region extraction and similarity calculation which concerns on embodiment of this invention. It is a figure which shows the example of the search object image as a process target image which concerns on embodiment of this invention. It is a figure which shows the example of the information stored in image information DB which concerns on embodiment of this invention. It is a figure which shows the extraction result of the circumscribed rectangle which concerns on embodiment of this invention. It is a figure which shows the determination result of the character area which concerns on embodiment of this invention. It is a figure which shows the example of the partial image which concerns on embodiment of this invention. It is a figure which shows the production | generation aspect of the numerical value string information which concerns on embodiment of this invention. It is a figure which shows the aspect of the subtraction process which concerns on embodiment of this invention. It is a figure which shows the example of the partial image which concerns on other embodiment of this invention.

Explanation of symbols

1 Image Retrieval Device 2 Client Device 10 CPU
20 RAM
30 ROM
40 HDD
50 I / F
60 LCD
70 Operation unit 80 Bus 100 Search control unit 101 Input image information acquisition unit 102 Image processing unit 103 Similarity calculation unit 104 Extraction result processing unit 110 Information input unit 120 Network I / F
130 Display unit 200 Image information DB

Claims (13)

An image search device that calculates a similarity between an image to be searched and an image of a search condition and executes an image search,
By generating numerical sequence information based on a processing target image that is either the search target image or the search condition image, and generating feature information of the image based on the generated numerical sequence information To calculate the similarity,
A processing target image acquisition unit that acquires a processing target image that is either the search target image or the search condition image;
A partial image generation unit that generates a partial image indicating a part of the processing target image;
A numerical value string information generating unit for generating numerical value string information based on the partial image;
A subtraction value string information generation unit that generates subtraction value string information by removing the value string information generated based on the partial image from the value string information generated based on the processing target image;
An image search apparatus comprising: a similarity calculation unit that calculates the similarity based on the generated subtraction numerical value string information.   The image search device according to claim 1, wherein the partial image generation unit generates the partial image by using a region in which characters are displayed in the processing target image as a part of the processing target image.   The image search device according to claim 1, wherein the partial image generation unit generates the partial image by extracting a part of the processing target image.   The image search device according to claim 1, wherein the partial image generation unit generates an image in which only a part of the processing target image is displayed in the processing target image as the partial image.   The numerical sequence information generation unit generates a projection histogram by projecting pixels constituting the processing target image in one dimension, and generates the numerical sequence information based on the generated projection histogram. The image search apparatus according to claim 1. The processing target image is a binary image,
The numerical value column information generation unit generates the numerical value column information by counting pixel components for at least one of a column and a row of pixels constituting the processing target image. The image search device according to any one of the above.   The image search device according to claim 1, wherein the partial image generation unit determines a part of the processing target image based on pixels constituting the processing target image.   The said partial image generation part extracts a connection component in the pixel which comprises the said process target image, and determines a part of a process target image based on the extracted said connection component, It is characterized by the above-mentioned. Image search device. The processing target image is a binary image,
The image search device according to claim 8, wherein the partial image generation unit extracts the same pixel component continuous in pixels constituting the processing target image as the connected component. The processing target image is a chromatic image,
9. The partial image generation unit, as a connected component, extracts a plurality of continuous pixels that are similar in color among pixels constituting the processing target image. The image search device described in 1. An image search method for performing image search by calculating a similarity between an image to be searched and an image of a search condition,
By generating numerical sequence information based on a processing target image that is either the search target image or the search condition image, and generating feature information of the image based on the generated numerical sequence information To calculate the similarity,
The processing target image acquisition unit acquires a processing target image that is either the search target image or the search condition image,
The partial image generation unit generates a partial image indicating a part of the processing target image,
The numerical sequence information generation unit generates numerical sequence information based on the partial image,
The subtraction numerical sequence information generation unit generates subtraction numerical sequence information by removing the numerical sequence information generated based on the partial image from the numerical sequence information generated based on the processing target image,
An image search method, wherein the similarity calculation unit calculates the similarity based on the generated subtraction value string information. An information processing program for causing an information processing apparatus to function as an image search apparatus that calculates an similarity between an image to be searched and an image of a search condition, and executes an image search,
The image search device generates numerical sequence information based on a processing target image that is either the search target image or the search condition image, and the image feature information based on the generated numerical sequence information The similarity is calculated by generating
Obtaining a processing target image that is either the search target image or the search condition image;
Generating a partial image indicating a part of the processing target image;
Generating numerical sequence information based on the partial image;
Generating subtraction numerical sequence information by excluding the numerical sequence information generated based on the partial image from the numerical sequence information generated based on the processing target image;
An information processing program causing the information processing apparatus to execute the step of calculating the similarity based on the generated subtraction numerical value string information.   13. A recording medium in which the information processing program according to claim 12 is recorded in a format readable by an information processing apparatus.

Images