JP2013050502A - Advertisement display control device, advertisement display control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To effectively display advertisement on the basis of a plurality of pieces of image data extracted as image retrieval results.SOLUTION: In a server 1 (advertisement display control device), retrieval result image data output according to image retrieval processing requests from a client terminal 2 is identified, an advertisement database where advertisement information and a plurality of feature quantities are associated with each other and stored is referred to, advertisement information associated with a feature quantity matching the feature quantity of retrieval result image data is retrieved, the retrieved advertisement information is determined as advertisement to be displayed together with the retrieval result image data, and the display position of advertisement is determined according to display mode of the retrieval result image data and is displayed on the client terminal 2.

Description

  The present invention relates to an advertisement display control device, an advertisement display control method, and a program.

  Common elements of multiple image data extracted as search results when searching for similar image data by setting multiple image data held by individuals or multiple people with common preferences as a search query An advertisement display method is known in which an advertisement corresponding to the common element is extracted from an advertisement database and displayed (for example, Patent Document 1).

  According to the advertisement display method shown in Patent Document 1, a common element of a plurality of image data that cannot be obtained from a single image data is extracted, and the extracted common element is shared between individuals or a plurality of persons. The display of an effective advertisement is realized by analogy with common tastes.

JP 2006-106404 A (summary, paragraph 0003, etc.)

  In the advertisement display method disclosed in Patent Document 1, image data owned by an individual or a plurality of persons having a common preference is used as a search target, and a display advertisement and a feature of the advertisement are associated on a one-to-one basis. It is set to be a relationship. By the way, when searching for similar image data on the Internet or the like, image data possessed by an unspecified number of persons is included in the search target. Therefore, when an image search is performed on image data that includes various preferences, the common elements that appear in the search result image data do not necessarily match the searcher's preferences. Therefore, when searching for similar image data on the Internet or the like by the advertisement display method disclosed in Patent Document 1, an advertisement reflecting the searcher's preference is not always displayed, and an effective advertisement is displayed. It is unclear whether or not the display can be realized.

  Further, in the advertisement display method disclosed in Patent Document 1, since the association between the advertisement and the feature of the advertisement is set to have a one-to-one relationship, one feature associated with the advertisement is provided. There is no opportunity for an advertisement to be displayed except when they match as a common element between a plurality of image data. That is, as described above, although it is unclear whether the elements common to the search result image match the searcher's preference, the advertisement display opportunities are very limited.

  Further, according to the advertisement display method disclosed in Patent Document 1, when there is a margin in the display area of the image search result, the larger the common element value, the larger the advertisement to be displayed. Depending on the positional relationship between the image data to be displayed and the advertisement displayed on the same screen, the advertisement display may not always be effective.

  The present invention has been made in view of at least one of the above-described problems, and an advertisement display control device and an advertisement display control method for effectively displaying an advertisement based on a plurality of image data extracted as an image search result And to provide a program.

  An advertisement display control apparatus according to one aspect of the present invention is specified by a search result image specifying unit that specifies a plurality of search result image data output in response to a search processing request for image data, and a search result image specifying unit. A feature amount calculating means for calculating feature amounts of a plurality of search result image data; a feature amount calculated by the feature amount calculating means with reference to an advertisement database in which advertisement information and a plurality of feature amounts are stored in association with each other; Display advertisement determining means for searching for advertisement information associated with a matching feature quantity and determining the searched advertisement information as an advertisement to be displayed together with a plurality of search result image data, and a display mode of the plurality of search result image data Accordingly, the display position determining means for determining the display position of the advertisement determined by the display advertisement determining means, and the advertisement determined by the display advertisement determining means are displayed. Those comprising an advertisement display control means for displaying on the determined display position by the position determination means.

  By configuring the advertisement display control device as described above, an advertisement corresponding to the feature amount of a plurality of image data extracted as a search result is selected, and the advertisement is displayed at a position in consideration of the display mode of the search result image. Can be made. In addition, since a plurality of feature quantities are associated with the advertisement, if one of them is in common with the feature quantity of the search result image data, it may be selected as an advertisement to be displayed. It is possible to provide a new advertisement display opportunity to the advertiser.

  In addition, in the advertisement database, area information indicating a feature vector space corresponding to a product to be advertised or a service is associated with each advertisement information as a feature quantity, and the feature quantity calculation means includes a plurality of search result image data. The average feature amount is calculated from the feature amount, and the display advertisement determining means refers to the advertisement database and searches the advertisement information associated with the area information indicating the feature vector space including the average feature amount. The advertisement information is preferably determined as an advertisement to be displayed together with a plurality of search result image data.

  Since the advertisement display control device has the above-described configuration, an advertisement associated with a feature vector space including a value obtained by averaging a plurality of feature amounts obtained from search result image data is selected. It is possible to display an advertisement for a product or service having an element common to. As a result, for example, when an advertisement is displayed together with a search image result when an advertisement is displayed along with a search result image data or a product having a common visual feature such as a color histogram, texture, shape, and edge feature of the search result image data. Even if the user is not particularly conscious of the visual characteristics, the line of sight is easily guided to the display advertisement, and a more effective advertisement display can be realized.

  In addition, in the advertisement database, keyword information related to an advertiser's product or service is associated with each advertisement information as a feature amount, and a feature amount calculation means is associated with each of a plurality of search result image data. The display advertisement determining means searches the advertisement information associated with the keyword information that matches the metadata most frequently aggregated by the feature quantity calculating means by referring to the advertisement database. The advertisement information is preferably determined as an advertisement to be displayed together with a plurality of search result image data.

  By configuring the advertisement display control device as described above, the metadata obtained from a plurality of search result image data is aggregated, and the advertisement associated with the keyword that matches the metadata with the highest number of appearances is displayed. Therefore, it is possible to display an advertisement for a product or service that is common to a keyword estimated to be most relevant to a plurality of search result image data.

  In addition, in the advertisement database, keyword information related to an advertiser's product or service is associated with each advertisement information as a feature amount, and a feature amount calculation means is associated with each of a plurality of search result image data. The display advertisement determining means selects keyword information that matches each metadata according to the number of totals of each metadata by the feature quantity calculating means, and refers to the advertisement database to select the selected metadata. It is preferable that advertisement information associated with the keyword information is searched and the searched advertisement information is determined as an advertisement to be displayed together with a plurality of search result image data.

  By configuring the advertisement display control device as described above, metadata obtained from a plurality of search result image data is aggregated, and advertisements associated with keywords corresponding to the total number of metadata are displayed. It is possible to appropriately select keywords corresponding to the total number of items, and display advertisements of products or services presumed to be related to the selected keywords.

  The image data search processing request is a similar image search processing request based on a combination of keyword information and search image data specified by the user, and the advertisement database relates to an advertiser's product or service as a feature amount. Keyword information to be associated with each advertisement information, the search result image specifying means specifies a plurality of search result image data output as search results according to the similar image search processing request, and the feature amount calculating means includes: All the keyword information associated with the search result image data is extracted and aggregated, and the keyword information specified by the user is added with a predetermined weighted value, and the display advertisement determining means calculates the feature amount. Aggregate value of each metadata by means and numerical value with predetermined weighting of keyword information specified by user The largest of the added values is determined as keyword information for selecting an advertisement, and the advertisement database is searched for the advertisement information associated with the determined keyword information. The advertisement information is preferably determined as an advertisement to be displayed together with a plurality of search result image data.

  Since the advertisement display control device has the above-described configuration, the search keyword set by the user as a search condition is also given a predetermined weight and used as a determination material when selecting a display advertisement. There is an effect that advertisements for products or services are more easily selected and displayed.

  In addition, the present invention can be realized not only in the advertisement display control apparatus but also in an advertisement display control method having each process executed by each unit constituting the advertisement display control apparatus. Furthermore, the present invention can also be realized in a program that causes a computer to execute functions corresponding to the respective units constituting the advertisement display control device. Therefore, the advertisement display control method and program that are one aspect of the present invention can also achieve the same effects as the advertisement display control apparatus described above.

It is explanatory drawing which shows schematic structure of the server and client terminal which concern on 1st Embodiment of this invention. It is explanatory drawing which shows typically the function of the internal structure of the server shown in FIG. 1 with a functional block diagram. It is a figure which shows an example of the information memorize | stored in the image database shown in FIG. It is a figure explaining the concept of the cluster produced | generated within the image database shown in FIG. It is a figure which shows an example of the information memorize | stored in the advertisement database shown in FIG. It is explanatory drawing which shows the various programs and modules stored in the memory with which the server shown in FIG. 1 is provided. It is explanatory drawing which shows the detail of the advertisement search module shown in FIG. It is explanatory drawing which shows typically the internal structure of the client terminal shown in FIG. 1 with a functional block diagram. It is explanatory drawing which shows the various programs and modules stored in the memory with which the client terminal shown in FIG. 1 is provided. It is a flowchart which shows the image search process performed between the client terminal shown in FIG. 1, and a server. It is a flowchart which shows the detail of the advertisement matching process shown in FIG.10 S16, and a display position control process. It is explanatory drawing which shows notionally the matching process shown in FIG. It is a figure which shows an example of the advertisement display displayed by the process shown in FIG. It is a figure which shows an example of the advertisement display displayed by the process shown in FIG. It is a figure which shows an example of the advertisement display displayed by the process shown in FIG. It is a flowchart which shows the advertisement matching process in 2nd Embodiment of this invention. It is the figure which showed notionally the advertisement matching process shown in FIG. It is a flowchart which shows the advertisement matching process in 3rd Embodiment of this invention. It is a flowchart which shows the image search process in 4th Embodiment of this invention. It is a flowchart which shows the detail of the advertisement matching process shown to step S56 shown in FIG. It is the figure which showed notionally the advertisement matching process shown in FIG.

  Hereinafter, an advertisement display control device, an advertisement display control method, and a program according to the present invention will be described based on each embodiment with reference to the drawings. The advertisement display control method according to the present invention will be described using the operation of the advertisement display control apparatus as an example, and the program according to the present invention will be described using a program installed in the advertisement display control apparatus as an example.

[First Embodiment]
FIG. 1 is an explanatory diagram showing a schematic configuration of a server 1 and a client terminal 2 according to the first embodiment of the present invention. The server 1 and the client terminal 2 are connected via a network 3 so that bidirectional communication is possible. The network 3 may be any of the Internet, an intranet, a WAN (Wide Area Network), a LAN (Local Area Network), and the like.

  The server 1 stores a large number of image data existing on a computer connected via the network 3, performs similar image data search in response to a search request from the client terminal 2, and extracts it as a search result. This is an example of an advertisement display control apparatus that selects and transmits the image data that has been displayed and advertisement data suitable for the image data. In the server 1 of the present embodiment, the apparatus has both the image search processing function and the advertisement matching processing function. However, the image search processing function may be realized by another computer.

(Configuration of server 1)
FIG. 2 is an explanatory diagram schematically showing functions of the internal configuration of the server 1 shown in FIG. 1 in a functional block diagram.

  As shown in FIG. 2, the server 1 includes a central processing unit (CPU) 11, a memory 12, a storage device 13, and an input / output interface 14 that are communicably connected to each other.

  The CPU 11 executes an OS (not shown), various programs, and modules stored in the memory 12.

  The memory 12 stores programs and modules executed by the CPU 11 in a nonvolatile manner, and has a volatile work area where the programs and modules are expanded when the CPU 11 executes processes. As the memory 12, for example, a semiconductor memory device such as a read-only memory that stores a program or the like in a nonvolatile manner or a random access memory that provides a volatile work area when executing the program can be used.

  The storage device 13 is configured by one or a plurality of mass storage devices such as a hard disk drive and a flash memory drive. In the storage device 13, an image database 13A and an advertisement database 13B are constructed.

  In the image database 13A, an image index is constructed by clustering in order to shorten the time required to search for image data, and is configured by associating a plurality of image data, belonging clusters, and a plurality of feature amounts of each image data. ing. Here, the plurality of feature amounts of the image data include, for example, the average brightness, the minimum brightness, the maximum brightness, the hue, the size of the object (product, figure, animal, human face, etc.) included in the image data, It consists of multiple types of values such as position. Note that these feature quantities can be obtained by methods well known to those skilled in the art, and thus description thereof is omitted.

  FIG. 3 is a diagram illustrating an example of information stored in the image database 13A illustrated in FIG. As shown in FIG. 3, the affiliation cluster to which each image data belongs and the image feature amount are stored. Although not shown, each cluster is associated with a representative value of the image feature amount of each image data (for example, an average value of the feature amount of each image data). The image search is performed in units of, and thereafter, comparison with each image data belonging to a similar representative value is performed.

  FIG. 4 is a diagram for explaining the concept of clusters generated in the image database 13A shown in FIG. In the example shown in FIG. 4, the image data is classified based on whether or not a face is included in the image data based on the normalized image feature amount. For clusters including faces, the normalized feature quantity indicating the size of the face is greater than or equal to a predetermined value (face size large in FIG. 4) or less than a predetermined value (face size small in FIG. 4). Are further classified. For a cluster having a large face size, whether the face is a child's face or an adult's face (age in FIG. 4) is further classified based on, for example, a feature amount related to the organ position of the face. A cluster having a small face size is further classified as to whether it is a landscape scene based on whether the normalized feature amount indicating the hue of blue or green is a predetermined value or more. Then, the scene scene cluster is further classified based on whether or not the hue shows blue, and the cluster whose hue is not blue is further classified based on whether or not the hue shows green. Clusters that do not include a face are classified based on whether or not they are landscapes. For clusters that are not landscape scenes, the saturation is greater than or equal to a predetermined value (high saturation) or less than a predetermined value (saturation low). Further classification based on whether there is. Note that the configuration and classification of the clusters illustrated in FIG. 4 are merely examples, and the present invention is not limited to this, and a plurality of clusters may be configured, or classification different from that illustrated in FIG.

  Clustering is executed by the server 1 at the initial timing of creating an image index for the image database 13A, or at the timing when a predetermined amount of new image data is added to the image database 13A. In addition to the hierarchical clustering method as described above, non-hierarchical clustering methods such as a K-means method and a self-organizing map (SOM) are also known as clustering methods. It may be adopted.

  The advertisement database 13B is constructed by associating advertisement data and a plurality of feature quantities of the advertisement data. Here, the plurality of feature amounts of the advertisement data are, for example, the average brightness, the minimum brightness, the maximum brightness, the hue, and the object included in the image data indicating the product and service of the advertisement, as in the image database 13A described above. In addition to a plurality of types of values such as the size and position (products, figures, animals, human faces, etc.), it is composed of advertising products, keyword information related to services, and the like. Note that the multiple feature values of the advertisement data are the display conditions for the advertisement, so they do not necessarily have to be derived from the feature values of the image data indicating the product or service of the advertisement. Values specified and set by the advertiser Or a keyword or the like.

  FIG. 5 is a diagram illustrating an example of information stored in the advertisement database 13B illustrated in FIG. As shown in FIG. 5, a vector space feature quantity, keyword 1, and keyword 2 are associated as a plurality of feature quantities for each advertisement data. The vector space feature amount is obtained by, for example, extracting an object included in image data indicating an advertisement product or service and deriving the image feature amount of the object. For example, the color and brightness distribution in the object, the direction of the contour line, the average color and average brightness, the histogram of hue and brightness, etc. are shown numerically. It is expressed by a combination (vector). The keyword 1 stores, for example, the product of the advertisement, the name of the object included in the image data indicating the service, or character information included as tag information associated with the image data. The keyword 2 stores, for example, information on the name of an advertisement, specific store name handling the service, and the like. In the example shown in FIG. 5, only the image data is displayed as the advertisement data, but it may be text data (character information).

  The storage device 13 may function as a data storage unit that stores only a plurality of image data and advertisement data. In this case, the feature amount of each image data and the feature amount of each advertisement data stored in the storage device 13 are stored in the memory 12.

  Furthermore, in this embodiment, the storage device 13 is described as an example of a database that simply stores image data, advertisement data, and their feature quantities. However, the storage device 13 has a search processing function and an advertisement matching processing function. It is good also as a stand-alone database system which provides the control part provided, and outputs advertisement data with the image data of a search result according to the search processing request from the outside. In this case, a database system is arranged outside the server 1, and a search request and a search result are transmitted / received between the two via the input / output interface 14. In either case, the database search program is stored in the memory 12 and executed by the CPU 11.

  The input / output interface 14 receives a search request and transmits a search result with an external device, for example, the client terminal 2 according to a known communication protocol. Although not shown, the input / output interface 14 may have a function as an operation unit such as a keyboard and a function as a display unit such as a display.

  FIG. 6 is an explanatory diagram showing various programs and modules stored in the memory 12 included in the server 1 shown in FIG. As shown in FIG. 6, the memory 12 stores an image search program SP2 and an advertisement matching program SP3.

  The image search program SP2 is a program for searching for image data from the storage device 13, and includes a search image data acquisition module SM21 and an image search module SM22.

  The search image data acquisition module SM21 is a module for acquiring search image data set as a search condition transmitted from the client terminal 2 via the network 3.

  The image search module SM22 is a module for searching image data similar to or matching the search image data from the image database 13A using the search image data acquired by the search image data acquisition module SM21.

  FIG. 7 is an explanatory diagram showing details of the advertisement search module SM22 shown in FIG. As shown in FIG. 7, the image search module SM22 further includes a search image feature amount acquisition module SM23 and a similarity calculation module SM24 as submodules.

  The search image feature amount acquisition module SM23 is a module that acquires the feature amount of the search image data acquired by the search image data acquisition module SM21.

  The similarity calculation module SM24 includes the feature amount of the search image data acquired by the search image feature amount acquisition module SM23, the representative value of each cluster stored in the image database 13A, and the image feature amount of image data belonging to each cluster. Is used to calculate the similarity between the search image data and the image data to be searched (that is, the image data stored in the storage device 13).

  The advertisement matching program SP3 is a program for determining advertisement data to be displayed according to the image search result of the image search program SP2, and includes an advertisement acquisition module SM31, an advertisement search module SM32, and a display position control module SM33.

  The advertisement acquisition module SM31 is a module executed for acquiring advertisement data (image data, text data) acquired from an advertiser. Note that the advertisement data acquired from the advertiser may be received and acquired from a computer (not shown) connected via the network 3, or acquired by connecting an external storage medium to the server 1. You may do it.

  The advertisement search module SM32 is a module for searching and determining advertisement data to be displayed as an advertisement. As submodules, the search result image specifying module SM34, the search result image feature amount calculating module SM35, the advertisement feature amount acquiring module SM36, A fitness calculation module SM37 is further provided.

  The search result image specifying module SM34 is a module for specifying one or a plurality of image data output as a search result by the image search program SP2. The search result image specifying module SM34 is an example of a search result image specifying unit in the claims.

  The search result image feature amount calculation module SM35 is a module that calculates a feature amount when matching with advertisement data using the feature amount of the image data specified by the search result image specifying module SM34. Note that the search result image feature quantity calculation module SM35 is an example of a feature quantity calculation unit in the claims.

  The advertisement data feature amount acquisition module SM36 is a module that acquires a plurality of feature amounts of advertisement data stored in the advertisement database 13B.

  The suitability calculation module SM37 uses the feature amount calculated by the search result image feature amount calculation module SM34 and the feature amount of the advertisement data acquired by the advertisement data feature amount acquisition module SM35 to perform the search image data and the advertisement data. It is a module that determines the advertisement to be calculated and displayed. The fitness level calculation module SM37 is an example of a display advertisement determination unit in the claims.

  The display position control module SM33 is a module that controls and determines the advertisement display position determined by the fitness calculation module SM37 with respect to the image search result searched by the image search program SP2. The display position control module SM33 is an example of the advertisement display control means in the claims.

(Configuration of client terminal 2)
FIG. 8 is an explanatory diagram schematically showing the internal configuration of the client terminal 2 shown in FIG. 1 using a functional block diagram.

  The client terminal 2 includes a central processing unit (CPU) 21, a memory 22, a storage device 23, and an input / output interface 24 that are communicably connected to each other.

  The CPU 21 executes an OS (not shown), various programs, and modules stored in the memory 21.

  The memory 21 stores programs and modules executed by the CPU 21 in a nonvolatile manner, and has a volatile work area where the programs and modules are expanded when the CPU 21 executes processes. As the memory 21, for example, a semiconductor storage device such as a read only memory that stores a program or the like in a non-volatile manner, or a random access memory that provides a volatile work area when the program is executed can be used.

  The storage device 23 is configured by one or more mass storage devices such as a hard disk drive and a flash memory drive. The storage device 13 stores image data set as a search condition by the user.

  The input / output interface 24 has a function of transmitting a search request and receiving a search result with an external device, for example, the server 1 according to a known communication protocol. Although not shown, the input / output interface 24 has a function as an operation unit for the user to input an image search instruction to the server 1 and can be realized by, for example, a button or a wheel. The input / output interface 24 has a function as a display unit, and displays an image based on the searched image data and advertisement data for the user, and displays various information for the user.

  FIG. 9 is an explanatory diagram showing various programs and modules stored in the memory 22 included in the client terminal 2 shown in FIG.

  The memory 21 includes an image search request program CP1 and a Web browser program CP2.

  The image search request program CP1 is a program for requesting the server 1 to perform an image search process, and includes an image data identification module CM11, a search request module CM12, and a search result acquisition module CM13.

  The image data specifying module CM11 is a module for specifying image data to be searched.

  The search request module CM12 is a module that transmits the image data specified by the image data specifying module CM11 and the search request to the server 1.

  The search result acquisition module CM13 is a module that acquires one or more image data and advertisement data as search results from the server 1.

  The Web browser program CP2 is a program for displaying the search result on the display unit using the data acquired by the search result acquisition module CM13.

(Image search processing)
FIG. 10 is a flowchart showing image search processing executed between the client terminal 2 and the server 1 shown in FIG.

  When the image (search image) specified by the user is selected (START), the CPU 21 of the client terminal 2 specifies search image data corresponding to the search image (step S10). The search image data is, for example, image data D stored in the storage device 13 and is selected by the user via an operation unit (not shown).

  The CPU 21 transmits the search image data to the server 1 as a search processing request (search query) (step S11). Specifically, the data is transmitted to the server 1 via the input / output interface 24 and the network 3.

  When the CPU 11 of the server 1 receives the image search processing request based on the search image data in step S11 from the client terminal 2, the CPU 11 executes the image data acquisition module SM21 to acquire the search image data (step S12). Specifically, the search image data is taken into the server 1 via the input / output interface 14 and stored in the memory 12.

  When acquiring the search image data in step S12, the CPU 11 executes the search image feature amount acquisition module SM23 to acquire a plurality of feature amounts of the search image data (step S13).

  When step S13 is completed, the CPU 11 executes the similarity calculation module SM24 to search the image database 13A for a plurality of clusters that are similar to or match the search image data. The image data that matches or is similar to the search image data is searched from the plurality of image data belonging to the cluster obtained by (Step S14).

  Specifically, the similarity is calculated using the value of each feature value of the acquired search image data and the attribute (representative value of each feature value) associated with each cluster. The similarity is, for example, the distance between the feature amount of the search image data calculated by a distance calculation method such as the Euclidean distance or the Mahalanobis distance and the representative value associated with each cluster, that is, the search image data Judgment is made using the distance between the multi-element vectors represented by the feature quantity and the representative value of each cluster, and the shorter the obtained distance, the more similar it is judged. Then, the above-described similarity is calculated again using each feature amount of each image data belonging to the cluster determined to be similar and each feature amount of the search image data.

  Note that the method for selecting a plurality of image data is similar in that a distance as a threshold is determined in advance and image data included in the threshold is selected, or the number of selected image data is set in advance. Up to the number of selected image data may be selected in order from the top of the image data. It should be noted that the calculation method of each distance and the calculation formula of the inner product for calculating the similarity are well known to those skilled in the art, and thus description thereof is omitted.

  When the search for similar / matching image data is completed in step S14, the CPU 11 executes the advertisement matching program SP3, and executes the advertisement matching process and the display control process (step S15). Details of the advertisement matching process and the display control process will be described later.

  The CPU 11 transmits the plurality of image data and advertisement data searched in step S14 and step S15 to the client terminal 2 that is the transmission source of the search request (step S16).

  When the CPU 21 of the client terminal 2 receives a plurality of image data and advertisement data by transmission in step S16, the CPU 21 executes the search result acquisition module CM13 and stores it in the memory 21 (step S17).

  When completing step S17, the CPU 21 executes the web browser program CP2, displays the image data stored in the memory 21 on the display unit (step S18), and ends the image search process (END).

(Advertising matching processing and display position control processing)
FIG. 11 is a flowchart showing details of the advertisement matching process and the display position control process shown in step S16 of FIG.

  When step S14 in FIG. 10 is completed, the CPU 11 executes the feature amount acquisition module SM12 to acquire a plurality of feature amounts of the search result image data selected as image data similar to the search image data (step S21). ).

  The feature amount used in the present embodiment is, for example, the brightness of the image data or the brightness histogram in a partial area of the image data, texture, presence / absence of an object, object size, hue, and the like. Is used. From the luminance histogram, the average luminance value, the maximum luminance value, and the minimum luminance value are acquired, and the hue is the average value of each of R, G, and B components in the image data, or the total number of pixels constituting the image data. The ratio of the number of R, G, and B component pixels that occupy a predetermined value or more is acquired. Since a method of acquiring a luminance histogram, hue, and texture as a feature amount is a method well known to those skilled in the art, a description thereof will be omitted. The presence / absence of an object, that is, the detection of an object and the size of the object can be determined by, for example, a pattern matching method using a model shape, a method of specifying a hue region of a color unique to an object, or a method of detecting a feature of an object. Each can be used alone or in combination.

  The CPU 11 repeatedly executes step S21 until the feature amount is acquired from all the image data output as the search result (step S22: No). When the CPU 11 finishes obtaining the feature amounts for all the image data (step S22: Yes), the CPU 11 obtains an average value of the feature amounts of the image data (step S23).

  The CPU 11 repeatedly executes step S23 until the average value of each feature amount is obtained from all the image data output as search results (step S24: No). When the CPU 11 finishes calculating the average value of each feature amount from all the image data output as the search result (step S24: Yes), the CPU 11 performs a matching process between the average value of the image data and the feature amount of the advertisement data ( Step S25). Note that the number of advertisement data to be selected by matching may be determined in descending order of suitability according to the display mode of the similar image search result screen.

  The matching process in step S25 can be performed in the same manner as the part of the image search process described above. Specifically, the matching process is performed using the average value of each feature amount calculated from the acquired search result image data and the value of each feature amount of the advertisement data. The matching process is performed by, for example, the distance between the average value of each feature amount of the search image data and the value of each feature amount of the advertisement data, which is calculated by a distance calculation method such as the Euclidean distance or the Mahalanobis distance, that is, the search image. Judgment is made using the distance between the multi-vectors represented by the feature value of the data and the value of each feature value of the advertisement data, and the shorter the obtained distance, the more suitable the display advertisement. Alternatively, the average value of each feature quantity of the search image data and the value of each feature quantity of the advertisement data are made into a multi-factor vector extracted by frequency decomposition by wavelet transform or the like, and the inner product of these multi-factor vectors is obtained and judged. May be. In this case, it can be determined that the closer the inner product value is to 1, the more suitable the display advertisement for the search result image data is. It should be noted that a calculation method for each distance and an arithmetic expression for the inner product for calculating the degree of fitness are well known to those skilled in the art, and a description thereof will be omitted.

  When the CPU 11 determines the advertisement data suitable for the search result image in step S25, the CPU 11 controls the display position of the advertisement data according to the display position of the search result image to determine the display position (step S26). A display example in which the display position is controlled will be described later. When step S26 of FIG. 11 is completed, the CPU 11 proceeds to the process of step S16 of FIG.

  FIG. 12 is an explanatory diagram conceptually showing the matching process shown in FIG. In the example shown in FIG. 12, apple image data is set as search image data. Then, when a plurality of similar image data is extracted based on the above-described image search processing of FIG. 10 and FIG. . In the example illustrated in FIG. 12, two feature amounts (assuming feature amount A and feature amount B) are schematically illustrated. However, it is accurately performed in two or more dimensions. And the average value of all these feature vectors is calculated | required, and suitable advertisement data are searched from advertisement database 13B. As a result, advertisement data suitable for the advertisement display area is selected and displayed.

  13 to 15 are diagrams illustrating an example of advertisement display displayed by the process illustrated in FIG. As shown in FIG. 13, in the specific advertisement display area 100, three advertisements (advertisement C1, advertisement C2, and advertisement C3) are displayed. This is a display that a plurality of advertisements determined to have a high fitness level from the average feature amount of the search result image data are displayed in a predetermined number (3 in the case of FIG. 13) in order from the highest fitness level. Control is in progress.

  Further, display control as shown in FIG. 14 may be performed. For example, the same number of similar image data may be displayed in each row where the search result image data is displayed. In FIG. 14, the same number of apple images and strawberry images are displayed in region A and region B. In such a case, the image displayed on the left side is more similar to the search image than the image displayed on the right side, so an advertisement that matches the feature amount of the image data displayed on the left side is displayed. , Display at the same height. As a result, an advertisement corresponding to an image having a high degree of similarity to the search image data is displayed at the same height, so that there is a higher possibility that an advertisement that matches the preference of the user who performed the search is displayed. Become.

  Further, display control as shown in FIG. 15 may be performed. That is, the search result image data related to the display advertisement is surrounded by dotted lines (dotted line a, dotted line b, and dotted line c in FIG. 15), and advertisements related to these are displayed in the specific advertisement display area 100. It may be emphasized. In that case, the color of each dotted line surrounding the search result image data and the color of the character of each advertisement display related to the search result image surrounded by the dotted line may be the same color. Although not shown, when the advertisement display is displayed as an image, the image may be surrounded by a line of the same color.

  As described above, the advertisement display position control method as shown in FIGS. 13 to 15 creates a Web page in which the display position or display mode is changed using Java (registered trademark) script or the like when displaying the search result image. This is possible. The advertisements shown in FIGS. 13 to 15 are only text information, but an image indicating a product to be advertised or a service may be displayed.

(Effect of 1st Embodiment)
As described above, the CPU 11 of the server 1 in the first embodiment specifies a plurality of search result image data output in response to a search processing request for image data, and features of the specified plurality of search result image data. As an advertisement for referring to the advertisement database 13B, searching for advertisement information associated with a feature quantity that matches the calculated feature quantity, and displaying the searched advertisement information together with a plurality of search result image data The display position of the advertisement is determined according to the display mode of the plurality of search result image data.

  Therefore, it is possible to select an advertisement corresponding to the feature amount of a plurality of image data extracted as a search result and display the advertisement at a position in consideration of the display mode of the search result image. In addition, since a plurality of feature quantities are associated with the advertisement, if one of them is in common with the feature quantity of the search result image data, it may be selected as an advertisement to be displayed. It is possible to provide a new advertisement display opportunity to the advertiser.

  In particular, in the advertisement database 13B of the server 1, area information indicating a feature vector space corresponding to a product to be advertised or a service is associated with each advertisement information as a feature amount. Then, the search result image feature amount calculation module SM35 of the server 1 calculates an average feature amount from the feature amounts of the plurality of search result image data, and the advertisement search module SM32 indicates a feature vector space including the average feature amount. The advertisement information associated with the information is determined as an advertisement to be displayed together with a plurality of search result image data.

  Therefore, the feature quantity obtained from a plurality of search result image data can be averaged, and an advertisement associated with the area information indicating the feature vector space including the averaged feature quantity can be displayed. As a result, it is possible to display an advertisement suitable for a plurality of image data to be displayed.

[Second Embodiment]
In the first embodiment described above, the advertisement matching process is performed by averaging the feature amounts of the image data extracted as the search results. However, in the second embodiment, the metadata is aggregated as the feature amounts of the image data. Select ads that match the most metadata. The metadata here is, for example, tag information included in image data in the case of image data in EXIF (Exchangeable Image File Format) format. Since the configuration of each device in the second embodiment is the same as the configuration of each device in the first embodiment, illustration is omitted, but the association between image data and metadata included in the image data is associated with the image database. In addition, the advertisement database is associated at least in the advertisement database with the advertisement data and the keyword information (such as character information representing the color and shape of the product) related to the products and services placed in the advertisement. That is, in the image database according to the second embodiment, one or more metadata of image data is associated with each image data as the feature amount of the image data, and the advertisement database includes the advertiser as the feature amount of the advertisement. One or a plurality of keyword information related to the product or service is associated with each advertisement information.

  FIG. 16 is a flowchart showing an advertisement matching process in the second embodiment of the present invention. The advertisement matching process shown in FIG. 16 is the same as the advertisement matching process (FIG. 11) of the first embodiment, but differs in the following points.

  That is, in the process corresponding to step S21 in FIG. 11, the search result image feature amount calculation module in the second embodiment acquires metadata as the feature amount of the image data (step S31), and corresponds to step S22 in FIG. In the processing to be performed, it is determined whether or not the acquisition of metadata has been completed from all the image data (step S32). In the processing corresponding to steps S23 and S24 in FIG. Is specified (step S33).

  In the process corresponding to step S25 in FIG. 11, the advertisement search module in the second embodiment specifies advertisement data associated with the keyword information that matches the metadata most frequently counted in step S33 (step S33). S34), a display position control process corresponding to step S26 of FIG. 11 is performed (step S35).

  FIG. 17 is a diagram conceptually illustrating the advertisement display in the image search in the second embodiment of the present invention. In the example shown in FIG. 17, apple image data is set as search image data. Then, when a plurality of similar image data is extracted based on the image search process described above, the tag information associated with each of these images is aggregated, and the most aggregated tag information is set as keyword information. The matching advertisement is displayed with reference to the advertisement database. In the example shown in FIG. 17, since there are the most tags called T-shirts, advertisement data related to T-shirts is selected and displayed.

  As described above, the search result image feature amount calculation module according to the second embodiment aggregates the metadata associated with each of the plurality of search result image data, and the advertisement search module according to the second embodiment includes the search result image feature. Since the advertisement information associated with the keyword information that matches the most aggregated metadata by the quantity calculation module is determined as an advertisement to be displayed together with a plurality of search result image data, the plurality of search result image data It is possible to display an advertisement corresponding to the keyword that appears most frequently.

[Third Embodiment]
In the second embodiment described above, the metadata of the image data is aggregated, and the process of selecting the advertisement that matches the most metadata is performed. In the third embodiment, the advertisement matching is performed according to the number of metadata aggregation. Process. The configuration of each device in the third embodiment is the same as the configuration of each device in the first embodiment and the second embodiment, and how to hold the data of the image database and the advertisement database in the third embodiment is as follows. Since it is the same as the image database and the advertisement database in the second embodiment, illustration is omitted.

  FIG. 18 is a flowchart showing an advertisement matching process in the third embodiment of the present invention. The process shown in FIG. 17 is the same as the advertisement matching process (FIG. 16) shown in the second embodiment, but differs in the following points.

  That is, in the process corresponding to step S32 in FIG. 16, the search result image feature amount calculation module in the third embodiment selects metadata according to the total number of acquired metadata (step S43). In the process corresponding to step S34 in FIG. 16, the advertisement search module in the third embodiment identifies advertisement information associated with keyword information that matches the metadata selected in step S43 (step S44). ).

  As described above, the search result image feature amount calculation module in the third embodiment totals the metadata associated with each of the plurality of search result image data, and the advertisement search module performs the search result image feature in the third embodiment. The advertisement information associated with the keyword information corresponding to the total number of each metadata by the amount calculation module is selected and determined as an advertisement to be displayed together with a plurality of search result image data.

  Note that “according to the total number of each metadata” means that, for example, the top two keyword information of the total number is selected, or the keyword information that is two or more total values is selected. It is intended that free setting according to the total number is possible. Therefore, it is possible to display an advertisement corresponding to the number of keywords frequently obtained from a plurality of search result image data.

[Fourth Embodiment]
The image search processing request in the first to third embodiments described above is based on the premise that image data is specified as a search condition. In the fourth embodiment, two combinations of keyword information and search image data specified by the user are used. The similar image search process is performed by the above. It should be noted that the configuration of each device, the way of holding data in the image database and the advertisement database in the fourth embodiment are the same as those in the third embodiment, and are not shown.

  FIG. 19 is a flowchart showing image search processing according to the fourth embodiment of the present invention. The process shown in FIG. 18 is the same as the image search process (FIG. 9) shown in the first embodiment, but differs in the following points.

  That is, in the process corresponding to step S10 in FIG. 10, search image data and search keyword information corresponding to the search image specified by the user are specified (step S50), and a search processing request (search) is performed in the process corresponding to step S11. As a query, search image data and search keyword information are transmitted to the server 1 (step S51).

  When the CPU 11 of the server 1 receives an image search processing request based on the search image data and search keyword information in step S51 from the client terminal 2 in the process corresponding to step S12 in FIG. 10, the image data acquisition module in the fourth embodiment. To obtain search image data and search keyword information (step S52). Specifically, search image data and search keyword information are taken into the server 1 via the input / output interface 14 and stored in the memory 12.

  In the process corresponding to step S13 of FIG. 10, when the CPU 11 acquires the search image data and the search keyword information in step S52, the CPU 11 executes the search image feature amount acquisition module in the fourth embodiment, and executes a plurality of search image data. A feature amount is acquired (step S53).

  When step S53 is completed in the process corresponding to step S14 in FIG. 10, the CPU 11 executes the similarity calculation module in the fourth embodiment, and obtains a similarity to the combination of the search image data and the search keyword information from the image database 13A. Image data that matches or is similar to the search image data and the search keyword is searched from the plurality of matching clusters and the plurality of image data belonging to the cluster obtained by the search (step S54). Similarity or coincidence is determined based on keyword matching or mismatching in addition to the similarity between each feature amount of the search image data and each feature amount of each image data belonging to each cluster. The keyword is an auxiliary role of similarity. Basically, the higher the similarity value, the more similar / matched it is determined, and a plurality of image data having the same similarity is extracted. In the case of a match, the order of similarity is determined by the presence or absence of keywords.

  In the step corresponding to step S16 in FIG. 10, when the search of the image data is completed in step S55, the CPU 11 executes the advertisement matching program in the fourth embodiment, and performs the advertisement matching process and the display position control process (steps). S55). In addition, although the process of step S56-S58 is performed as a process equivalent to step S16-S18 of FIG. 10, since it is the same, description is abbreviate | omitted.

  FIG. 20 is a flowchart showing details of the advertisement matching process shown in step S56 shown in FIG. The process shown in FIG. 19 is the same as the advertisement matching process (FIG. 17) shown in the third embodiment, but differs in the following points.

  That is, in the process corresponding to step S42 in FIG. 17, the search result image feature amount calculation module in the fourth embodiment compares all the acquired metadata counts with the values obtained by weighting the search keyword information. The keyword is selected (step S62). In the process corresponding to step S43 in FIG. 17, the advertisement search module in the fourth embodiment identifies advertisement information associated with keyword information that matches the keyword selected in step S62 (step S63). .

  FIG. 21 is a diagram conceptually showing the matching process shown in FIG. In the example shown in FIG. 21, the image data of the aquarium is set as the search image data, and “aquarium” is set as the search keyword. When a plurality of similar image data is extracted based on the above-described image search process, tag information associated with each of these images is aggregated. In the example shown in FIG. 21, as tag information, two “water tanks”, three “aquatic plants”, and one “funa” are extracted. Here, the tag extracted from the search result image because aquatic plants were displayed in the image data set as the search image even though “aquarium” was selected as the keyword of interest to the user. As a result, a lot of “aquatic plants” were counted as information. However, in this embodiment, since “aquarium” set as the search keyword is added with 1.5 as a predetermined weight, “aquarium” is finally selected as the most tabulated tag information. . As a result, it is possible to display an advertisement derived from a keyword that the user is interested in. That is, depending on the feature amount of the search image data, there is a problem that a keyword from a viewpoint different from the keyword that the user is interested in is selected when determining the advertisement data. Such a problem is solved by adding a predetermined weight so that the keyword of interest is selected with priority over other keywords.

(Effect of the embodiment corresponding to claim 5)
As described above, the search result image specifying module according to the fourth embodiment specifies a plurality of search result image data output as search results in response to a similar image search processing request based on a combination of user search keyword information and image data. The search result image feature quantity calculation module in the fourth embodiment performs predetermined weighting on the search keyword information, and totals the metadata associated with the search result image data, thereby performing advertisement search in the fourth embodiment. When the module compares the total number of each metadata by the search result image feature amount calculation module in the fourth embodiment and the value after a predetermined weight is given to the search keyword information, and selects the advertisement information The keyword information of the keyword is determined, and the determined keyword information is associated with Advertisement information in the fourth embodiment is selected from the advertisement database and determined as an advertisement to be displayed together with a plurality of search result image data. Therefore, the search keyword set by the user as a search condition is determined from the search result image. By giving weight to the obtained keyword, there is an effect that an advertisement of a product or service that the user is interested in can be displayed more easily.

[Other Embodiments]
In addition, said each embodiment is only an illustration, and this invention includes what is illustrated below and other embodiment other than that. For example, in the second embodiment and the fourth embodiment, advertisement data associated with keyword information that matches the most tag information is selected and displayed. However, not only the most tag information but also the top three, You may make it display to predetermined order, such as 10 cases.

  In addition, the advertisement data included in the advertisement database 13B is mapped in advance for each cluster constructed in the advertisement database 13B (associating information indicating which cluster is mapped), and the search image data When a similar / matching cluster is extracted during the image search process, advertisement data mapped to the cluster may be determined as a display advertisement. This makes it possible to perform advertisement matching processing integrated with image search processing, thereby enabling more efficient advertisement display.

  In each of the above embodiments, it is assumed that image data is input as a search processing request (search query). However, the present invention can also be applied to advertisement display control for search result image data of a search processing request using a keyword.

  Further, the present invention includes configurations in which the functions disclosed in the above-described embodiments and other embodiments are mutually replaced, configurations in which combinations are changed, and the like.

1 ... Server (an example of an advertisement display control device)
11 ... CPU
DESCRIPTION OF SYMBOLS 12 ... Memory 13 ... Storage device 13A ... Image database 13B ... Advertising database 14 ... Input / output interface 2 ... Client terminal 21 ... CPU
DESCRIPTION OF SYMBOLS 22 ... Memory 23 ... Storage device 24 ... Input / output interface CP1 ... Image search request program CP2 ... Web browser program CM11 ... Image data identification module CM12 ... Search request module CM13 ... Search result acquisition module SP2 ... Image search program SM21 ... Search image data Acquisition module SM22 ... Image search module SM23 ... Search image feature quantity acquisition module SM24 ... Similarity calculation module SP3 ... Advertisement matching program (example of program)
SM31 ... Advertisement acquisition module SM32 ... Advertisement search module SM33 ... Display position control module (an example of advertisement display control means)
SM34 ... Search result image specifying module (an example of search result image specifying means)
SM35 ... Search result image feature amount calculation module (an example of feature amount calculation means)
SM36 ... Advertisement feature amount acquisition module SM37 ... Conformity calculation module (an example of display advertisement determination means)

Claims (7)

Search result image specifying means for specifying a plurality of search result image data output in response to a search processing request for image data;
Feature quantity calculating means for calculating feature quantities of the plurality of search result image data specified by the search result image specifying means;
Refer to an advertisement database in which advertisement information and a plurality of feature quantities are stored in association with each other, search for the advertisement information associated with a feature quantity that matches the feature quantity calculated by the feature quantity calculation unit, and search Display advertisement determining means for determining the advertisement information that has been displayed as an advertisement to be displayed together with the plurality of search result image data;
Display position determining means for determining a display position of the advertisement determined by the display advertisement determining means according to a display mode of the plurality of search result image data;
Advertisement display control means for displaying the advertisement determined by the display advertisement determination means at the display position determined by the display position determination means;
An advertisement display control device comprising: The advertisement display control device according to claim 1,
In the advertisement database, as the feature amount, product information to be advertised, or region information indicating a feature vector space corresponding to a service is associated with each advertisement information,
The feature amount calculating means includes:
An average feature amount is calculated from the feature amounts of the plurality of search result image data,
The display advertisement determining means includes:
With reference to the advertisement database, the advertisement information associated with the area information indicating the feature vector space including the average feature amount is searched, and the searched advertisement information is displayed together with the plurality of search result image data. Decide as an ad,
An advertisement display control device characterized by the above. The advertisement display control device according to claim 1,
In the advertisement database, keyword information related to an advertiser's product or service is associated with each advertisement information as the feature amount,
The feature amount calculating means includes:
Aggregating metadata associated with each of the plurality of search result image data,
The display advertisement determining means includes:
The advertisement database is searched for advertisement information associated with the keyword information that matches the most aggregated metadata by the feature amount calculation means, and the searched advertisement information is searched for the plurality of search result images. Decide as an ad to display with the data,
An advertisement display control device characterized by the above. The advertisement display control device according to claim 1,
In the advertisement database, keyword information related to an advertiser's product or service is associated with each advertisement information as the feature amount,
The feature amount calculating means includes:
Aggregating metadata associated with each of the plurality of search result image data,
The display advertisement determining means includes:
According to the total number of each metadata by the feature amount calculation means, select keyword information that matches each metadata,
Referring to the advertisement database, searching for advertisement information associated with the selected keyword information, and determining the searched advertisement information as an advertisement to be displayed together with the plurality of search result image data;
An advertisement display control device characterized by the above. The advertisement display control device according to claim 1,
The image data search processing request is a similar image search processing request by a combination of keyword information and search image data specified by the user,
In the advertisement database, keyword information related to an advertiser's product or service is associated with each advertisement information as the feature amount,
The search result image specifying means includes
Identifying a plurality of search result image data output as search results in response to the similar image search processing request,
The feature amount calculating means includes:
All the keyword information associated with the search result image data is extracted and aggregated, and for the keyword information specified by the user, a numerical value with a predetermined weight is added,
The display advertisement determining means includes:
The maximum value among the sum of the total value of each metadata by the feature amount calculation means and the numerical value to which the keyword information specified by the user is weighted is determined as the keyword information when selecting the advertisement. And
The advertisement database is searched for advertisement information associated with the determined keyword information, and the searched advertisement information is determined as an advertisement to be displayed together with the plurality of search result image data.
An advertisement display control device characterized by the above. A search result image specifying step for specifying a plurality of search result image data output in response to a search processing request for image data;
A feature amount calculating step for calculating a feature amount of the plurality of search result image data specified by the search result image specifying step;
Refer to an advertisement database in which advertisement information and a plurality of feature amounts are stored in association with each other, search for the advertisement information associated with a feature amount that matches the feature amount calculated by the feature amount calculating step, and search A display advertisement determining step for determining the advertisement information that has been displayed as an advertisement to be displayed together with the plurality of search result image data;
A display position determining step for determining a display position of the advertisement determined by the display advertisement determining step according to a display mode of the plurality of search result image data;
An advertisement display control step for displaying the advertisement determined by the display advertisement determination step at the display position determined by the display position determination step;
An advertisement display control method characterized by comprising: Computer
Search result image specifying means for specifying a plurality of search result image data output in response to a search processing request for image data;
Feature quantity calculating means for calculating feature quantities of the plurality of search result image data specified by the search result image specifying means;
Refer to an advertisement database in which advertisement information and a plurality of feature quantities are stored in association with each other, search for the advertisement information associated with a feature quantity that matches the feature quantity calculated by the feature quantity calculation unit, and search Display advertisement determining means for determining the advertisement information that has been displayed as an advertisement to be displayed together with the plurality of search result image data;
Display position determining means for determining a display position of the advertisement determined by the display advertisement determining means according to a display mode of the plurality of search result image data;
Advertisement display control means for displaying the advertisement determined by the display advertisement determination means at the display position determined by the display position determination means;
Program to function as.