WO2009081936A1

WO2009081936A1 - Advertisement management system, advertisement management server, advertisement management method, program, and browse client

Info

Publication number: WO2009081936A1
Application number: PCT/JP2008/073425
Authority: WO
Inventors: Takuya Miyata; Shunji Uchida; Takuya Minagawa; Tetsuji Masuda
Original assignee: J-Magic Inc.
Priority date: 2007-12-25
Filing date: 2008-12-24
Publication date: 2009-07-02
Also published as: JP2011090348A

Abstract

An advertisement management system which can efficiently manage a structure for obtaining advertisement information related to image data with high flexibility and independence. When a front server (30) receives image data (92) from a registered client (10), it transmits the received data to an advertisement management server (50) and an image ID corresponding to a similar image with respect to the image data (92) is obtained in the advertisement server (50). The advertisement management server (50) decides an object ID which is associated with the image ID and represents an object that the image data indicates and transmits the object ID to the front server (30). The front server (30) stores the object ID as data of a web page in which the image data (92) is displayed and transmits the object ID to the advertisement management server (50) in order to request advertisement data when a browse client (20) browses the web page. The advertisement management server (50) obtains the advertisement data by using the object ID and provides the advertisement data to the front server (30).

Description

Advertisement management system, advertisement management server, advertisement management method, program, and browsing client

The present invention relates to an advertisement management server that determines advertisement information related to image data and outputs the advertisement information.

In recent years, with the spread of mobile phones and personal computers (PCs), lifestyles such as browsing web pages via the Internet using these devices, searching for necessary information, and trying to communicate with other users It is becoming established.

Many cameras are equipped with a camera so that users can easily shoot still images and videos anytime. Against this background, an increasing number of websites have adopted a mechanism for publishing and storing images received from users to other users. Also, with the widespread use of digital (still) cameras, digital video cameras, broadband networks, and the like, images taken with these camera devices are often uploaded to the website via a personal computer.

Examples of such Web sites that handle images include blog sites that publish diary-like descriptions made up of text and images uploaded by users to the user and other users, and (generally) support communication between specific users SNS (Social Network Service) site that receives texts and images from each user and provides these information to the specific user, stores images uploaded by the users, and easily browses them like albums There are album sites and photo sharing sites that are electronically presented to the user or other users in various forms, and bulletin board sites where users post texts and images for exchanging opinions on a predetermined theme.

On the other hand, as access to websites increases, mechanisms for providing advertisement information to users of websites have attracted a great deal of attention, and some of them have already been implemented. The first example is a mechanism for displaying advertisement information together with a search result in a search site for searching a Web site. When a search keyword for searching a website is input from a search user, the search management system that realizes such a mechanism displays a list of websites related to the search keyword as a search result to the search user. The advertisement information related to the search keyword is displayed, for example, on the side or upper / lower part of the search result list. The advertisement information displayed in this way is registered in advance by the advertiser in association with a predetermined keyword or the like, and whether or not to display the advertisement information depends on the display condition set by the advertiser, the search keyword, The search management system makes a determination based on the similarity and relevance to the keyword set by the advertiser, the content of the advertiser's Web page, and the like.

A second example of providing advertisement information to users of a website relates to a blog management system that manages and manages the blog site described above. The blog management system displays advertisement information related to a keyword extracted from an article (text information) posted by a user to a posting user or a browsing user. Like the search management system, the advertising information is registered in advance by the advertiser in association with a predetermined keyword, etc. Whether or not to display the advertising information depends on the display conditions set by the advertiser and the posted articles. Judgment is made based on the similarity and relevance between the extracted keyword and the keyword set by the advertiser.

As described above, in the conventional website, the advertisement information selected based on the keyword extracted from the search keyword or the posted article is displayed to the search user, the browsing user, etc., as described above, the image data Recently, a number of Web sites have been proposed, and several mechanisms for distributing information related to image data to image data providers have been proposed.

For example, in Patent Document 1, image data that is most similar to image data received from a mobile phone is detected, and related information directly associated with the detected image data is displayed on the mobile phone that has transmitted the image data. An information providing apparatus to be provided is disclosed.
JP 2005-165461 A

In Patent Document 2, image data received from a mobile terminal is compared with a large number of image data, keywords associated with similar image data are acquired as recognition results, and an advertisement DB is searched using the acquired keywords. An advertisement information providing device that distributes advertisement information obtained by doing this to a mobile terminal that has transmitted the image data is disclosed.
JP 2004-185591 A

However, the related art including the information providing apparatus of Patent Document 1 and the advertisement information providing apparatus of Patent Document 2 has the following problems. The first problem is a problem in managing related information and keywords. In the above-described conventional apparatus, related information and keywords are associated with each piece of image data to be stored for comparison with image data received from a mobile phone. The same related information and keywords are associated with a plurality of pieces of image data representing objects and meanings. This may impede efficient management, such as when newly adding related information or keywords or when maintaining information.

The second problem is an accuracy problem when judging the similarity of image data. In the information providing apparatus, the advertisement information providing apparatus, and the like, similar image data is determined by a single method, that is, only by one criterion, and the similarity of the image data is determined from a plurality of criteria by a plurality of criteria. It is not configured like this. By using various comparison methods and image data of various genres, it is possible to determine similarity with higher accuracy.

The third problem is that the scope of providing advertisement information is limited. The information providing apparatus and the advertisement information providing apparatus are configured to receive image data from a mobile phone and distribute the obtained related information and advertising information directly to the mobile device. Information is basically provided to only one user. Therefore, with such a configuration, only a very small advertising effect can be obtained. Originally, it is desirable to provide advertisement information with effective contents to many users who have a certain interest.

The fourth problem is that it cannot be adopted when advertising information is to be provided to many users on a blog site, SNS site, album site, bulletin board site, etc. due to the above-mentioned problem of the conventional apparatus. is there. Further, in these conventional apparatuses, the server that manages and operates each site cannot perform flexible control such as specifying a judgment standard related to image similarity, a source (type) of advertisement information, a number, and the like. .

Therefore, an object of the present invention is to separately manage image data to be compared when image data is received, and related information and keywords corresponding to these comparison target image data by associating them with a plurality of databases. To provide an advertisement management system, an advertisement management server, an advertisement management method, a program, and a browsing client that can be managed efficiently with flexibility and independence.

A further object of the present invention is to compare a plurality of different criteria, for example, different similarity determination methods, different comparison target image data, when comparing received image data with a lot of stored comparison target image data. An object of the present invention is to provide an advertisement management system, an advertisement management server, an advertisement management method, a program, and a browsing client that can determine the degree of similarity in a multifaceted manner by using a set.

Another object of the present invention is to provide an advertisement management system, an advertisement management server, an advertisement management method, and a program capable of providing related information and advertisement information related to image data to many browsing users who browse the image data. , And to provide a browsing client.

Furthermore, an object of the present invention can be used by a server that manages and operates a blog site, an SNS site, an album site, a bulletin board site, and the like, and advertisement information related to image data uploaded to the site is stored in the server. In addition, advertisement management that can flexibly change the criteria for determining image similarity and easily change the target and number of advertisement information according to the designation by the server. A system, an advertisement management server, an advertisement management method, a program, and a browsing client are provided.

A first embodiment of the present invention is an advertisement management system including a front server and an advertisement management server connected to the front server via a network, and the front server receives image data from a registered client In addition, image data transmitting means for transmitting the image data to the advertisement management server, editing the advertisement information based on the advertisement data related to the image data received from the advertisement management server, and browsing the advertisement information The advertisement management server transmits an image corresponding to an image similar to the image data by performing an image recognition process when receiving the image data from the front server. Image recognition processing means for determining an ID, and the image ID or an identifier associated with the image ID A keyword acquisition unit that acquires a keyword related to the image data, and searches for advertisement data based on the keyword acquired by the keyword acquisition unit, and the advertisement data obtained as a search result is used as the front server. Advertising data transmission means for transmitting to the network.

With such a configuration of the present invention, if a blog or SNS server is configured with a front server, related information and advertisement information related to image data can be provided to many browsing users who browse the image data. .

The second embodiment of the present invention further comprises an identifier transmission / reception means in the first embodiment described above, and the identification transmission / reception means, when the image ID is determined by the image recognition processing means, Alternatively, the identifier is transmitted to the front server so as to be stored in the front server, and when the front server receives a browsing request regarding the image data from the browsing client, the identifier stored in the front server is stored. An image ID or the identifier is received from the front server, and the image ID or the identifier is provided to the keyword acquisition means for acquiring the keyword.

According to a third embodiment of the present invention, in the first embodiment, the recognition method instruction data includes information on an image recognition method, and the image recognition processing means of the advertisement management server stores the recognition method instruction data. When received, the image ID is determined according to the recognition method instruction data.

According to a fourth embodiment of the present invention, in the first embodiment described above, the advertisement acquisition instruction data includes information related to an advertisement data acquisition method, and the advertisement data transmission unit of the advertisement management server includes the advertisement acquisition instruction data. Is received, the advertisement data is searched according to the advertisement acquisition instruction data.

According to a fifth embodiment of the present invention, in the second embodiment, the front server further includes a Web page generation unit, and the Web page generation unit receives the image ID or the identifier from the advertisement management server. Is transmitted to the front server, the transmitted image ID or the identifier is stored as Web page data.

According to a sixth embodiment of the present invention, in the first embodiment, the keyword acquisition unit acquires a keyword by searching a database using the image ID or the identifier, and the database acquires the image. It is configured to be switched according to the ID or the identifier.

According to a seventh embodiment of the present invention, in the first embodiment described above, the image recognition processing unit determines the image ID for each of the image data by a plurality of recognition methods or a plurality of determination criteria. Is done.

According to an eighth embodiment of the present invention, there is provided image data receiving means for receiving image data, image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data, and the image ID Alternatively, using an identifier associated with the image ID, a keyword acquisition unit that acquires a keyword related to the image data, and searches advertisement data based on the keyword acquired by the keyword acquisition unit, An advertisement management server having advertisement data transmitting means for transmitting the obtained advertisement data to a predetermined computer, wherein the image recognition processing means uses a plurality of recognition methods or a plurality of judgment criteria for each of the image data. It is configured to determine the image ID.

With such a configuration of the present invention, if a blog or SNS server is configured with a front server, related information and advertisement information related to image data can be provided to many browsing users who browse the image data. . Further, when comparing received image data with a large amount of stored comparison target image data, a plurality of different criteria, for example, different similarity determination methods and different sets of comparison target image data are used. The similarity can be determined.

According to a ninth embodiment of the present invention, in the eighth embodiment, the recognition method instruction data includes information on an image recognition method, and the image recognition processing means receives the recognition method instruction data. The image ID is determined according to the recognition method instruction data.

In a tenth embodiment of the present invention, in the above-described eighth embodiment, when the advertisement acquisition instruction data includes information related to an advertisement data acquisition method, and the advertisement data transmission means receives the advertisement acquisition instruction data The advertisement data is searched according to the advertisement acquisition instruction data.

In an eleventh embodiment of the present invention, in the eighth embodiment, the keyword acquisition unit acquires a keyword by searching a database using the image ID or the identifier, and the database acquires the image. It is configured to be switched according to the ID or the identifier.

A twelfth embodiment of the present invention is an advertisement management method executed in an advertisement management system including a front server and an advertisement management server connected to the front server via a network. An image data transmission step of transmitting the image data to the advertisement management server when receiving the image data from the advertisement management server, and based on the advertisement data related to the image data received from the advertisement management server in the front server An advertisement information transmission step of editing advertisement information and transmitting the advertisement information to a browsing client; and when the image data is received from the front server in the advertisement management server, image recognition processing is performed to perform the image recognition An image for determining an image ID corresponding to an image similar to data A keyword processing step of acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID in the advertisement management server; and An advertisement data transmitting step of searching advertisement data based on the keyword acquired in the acquiring step and transmitting the advertisement data obtained as a search result to the front server is configured.

In a thirteenth embodiment of the present invention, an image data receiving step for receiving image data, an image recognition processing step for performing image recognition processing to determine an image ID corresponding to an image similar to the image data, and the image ID Alternatively, using an identifier associated with the image ID, a keyword acquisition step of acquiring a keyword related to the image data, and searching advertisement data based on the keyword acquired by the keyword acquisition step, as a search result An advertisement data transmitting step of transmitting the obtained advertisement data to a predetermined computer, wherein the image recognition processing step is performed by using a plurality of recognition methods or a plurality of determination criteria for each of the image data. It is configured to determine the image ID.

A fourteenth embodiment of the present invention is a program for realizing an advertisement management system including a front server and an advertisement management server connected to the front server via a network. When receiving the image data, the image data transmitting means for transmitting the image data to the advertisement management server, and the advertisement information is edited based on the advertisement data related to the image data received from the advertisement management server. , Function as an advertisement information transmitting means for transmitting the advertisement information to the browsing client, and when the image management server receives the image data from the front server, performs image recognition processing and resembles the image data Image recognition processing means for determining an image ID corresponding to an image to be processed, the image ID Or, using an identifier associated with the image ID, keyword acquisition means for acquiring a keyword related to the image data, and searching for advertisement data based on the keyword acquired by the keyword acquisition means It is a program that functions as advertisement data transmission means for transmitting the advertisement data obtained as a result to the front server.

A fifteenth embodiment of the present invention is a program for realizing an advertisement management server. The advertisement management server performs image data receiving means for receiving image data, and performs image recognition processing to generate an image similar to the image data. Image recognition processing means for determining a corresponding image ID, keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID, and acquisition by the keyword acquisition means The advertisement data is searched based on the keyword, and the advertisement data obtained as a search result is made to function as an advertisement data transmitting means for transmitting to a predetermined computer, and the image recognition processing means for each of the image data The image ID is determined by a plurality of recognition methods or a plurality of judgment criteria. Is a program to cormorants function.

According to a sixteenth embodiment of the present invention, in response to a request from a browsing user, a browsing request transmitting means for transmitting a browsing request for a web page designated by the browsing user to a front server, and a response message to the browsing request Message receiving means for receiving from a front server, and when the Web page includes image data, advertisement information displayed on the Web page in relation to the image data depends on the type of the image data The browsing client is configured to be searched based on the keyword of the selected database.

A seventeenth embodiment of the present invention is an advertisement management system including a front server and an advertisement management server connected to the front server via a network, and the front server receives image data from a registered client In addition, image data transmitting means for transmitting the image data to the advertisement management server, editing the advertisement information based on the advertisement data related to the image data received from the advertisement management server, and browsing the advertisement information The advertisement management server transmits an image corresponding to an image similar to the image data by performing an image recognition process when receiving the image data from the front server. Image recognition processing means for determining an ID, and the image ID or an identification associated with the image ID A keyword acquisition unit that acquires a keyword related to the image data, and a keyword selection unit that selects at least one keyword from the keywords acquired by the keyword acquisition unit according to a predetermined selection criterion; And advertisement data transmitting means for searching for advertisement data based on the keyword obtained by the keyword selecting means and transmitting the advertisement data obtained as a search result to the front server.

According to an eighteenth embodiment of the present invention, image data receiving means for receiving image data, image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data, and the image ID Alternatively, using an identifier associated with the image ID, a keyword acquisition unit that acquires a keyword related to the image data, and at least one keyword among the keywords acquired by the keyword acquisition unit is a predetermined keyword. Keyword selection means for selecting according to selection criteria; advertisement data transmission means for searching for advertisement data based on the keyword obtained by the keyword selection means and transmitting the advertisement data obtained as a search result to a predetermined computer; And the image recognition processing means includes the image For each over data, configured to determine the image ID by a plurality of recognition methods or criteria.

A nineteenth embodiment of the present invention is an advertisement management system including a front server and an advertisement management server connected to the front server via a network, and the front server receives image data from a registered client In addition, image data transmitting means for transmitting the image data to the advertisement management server, editing the advertisement information based on the advertisement data related to the image data received from the advertisement management server, and browsing the advertisement information The advertisement management server transmits an image corresponding to an image similar to the image data by performing an image recognition process when receiving the image data from the front server. Image recognition processing means for determining an ID, and the image ID or an identification associated with the image ID The keyword acquisition means for acquiring a keyword related to the image data, the keyword acquired by the keyword acquisition means, the media information representing the media related to the image data, and the attribute of the image data Keyword conversion means for converting based on at least one of the attribute information to be represented; search for advertisement data based on the keyword obtained by the keyword conversion means; and the advertisement data obtained as a search result for the front server Advertising data transmission means for transmitting to the network.

According to a twentieth embodiment of the present invention, image data receiving means for receiving image data, image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data, and the image ID Alternatively, using an identifier associated with the image ID, a keyword acquisition unit that acquires a keyword related to the image data, and a medium that represents the medium that the image data is related to the keyword acquired by the keyword acquisition unit Based on at least one of information and attribute information representing the attribute of the image data, keyword conversion means for conversion, and advertisement data is searched based on the keyword obtained by the keyword conversion means, and obtained as a search result Advertising data sender for transmitting the received advertising data to a predetermined computer An advertisement management server having the bets, the image recognition processing unit, for each of the image data, configured to determine the image ID by a plurality of recognition methods or criteria.

A twenty-first embodiment of the present invention is an advertisement management system including a front server and an advertisement management server connected to the front server via a network, and the front server receives image data from a registered client In addition, image data transmitting means for transmitting the image data to the advertisement management server, editing the advertisement information based on the advertisement data related to the image data received from the advertisement management server, and browsing the advertisement information The advertisement management server transmits an image corresponding to an image similar to the image data by performing an image recognition process when receiving the image data from the front server. Image recognition processing means for determining an ID, and the image ID or an identification associated with the image ID The keyword acquisition means for acquiring a keyword related to the image data, the keyword acquired by the keyword acquisition means, the media information representing the media related to the image data, and the attribute of the image data Searching for advertisement data based on at least one of the keyword conversion means to convert based on at least one of the attribute information to represent, the keyword obtained by the keyword conversion means, the media information and the attribute information; And an advertisement data transmitting means for transmitting the advertisement data obtained as a search result to the front server.

According to a twenty-second embodiment of the present invention, image data receiving means for receiving image data, image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data, and the image ID Alternatively, using an identifier associated with the image ID, a keyword acquisition unit that acquires a keyword related to the image data, and a medium that represents the medium that the image data is related to the keyword acquired by the keyword acquisition unit Information and at least one of keyword conversion means for conversion based on at least one of attribute information representing the attribute of the image data, the keyword obtained by the keyword conversion means, the media information, and the attribute information Before searching for ad data based on search results An advertisement management server having advertisement data transmitting means for transmitting advertisement data to a predetermined computer, wherein the image recognition processing means assigns the image ID to each of the image data by a plurality of recognition methods or a plurality of judgment criteria. Configured to determine.

The advertisement management system according to the present invention efficiently manages image data to be compared when image data is received and related information and keywords corresponding to these image data to be compared with high flexibility and independence. be able to. The comparison target image data and the related information / keyword are separately managed by a plurality of databases and associated in multiple stages.

Further, when the received image data is compared with a lot of stored image data to be compared by the advertisement management system, a plurality of different criteria, for example, different similarity determination methods, different comparison target image data By using sets, it is possible to determine the degree of similarity in a multifaceted manner.

Further, the advertisement management system can provide related information and advertisement information related to the image data to many browsing users who browse the image data. The advertisement management system is provided in a form that can be used by a server that manages and operates a blog site, an SNS site, an album site, a bulletin board site, etc., and receives advertisement information related to image data uploaded to the site. Can be provided to a server, and according to the designation provided from the server, the criteria for determining image similarity can be flexibly changed, and the target and number of advertisement information can be easily changed. .

It is an approximate line figure showing an outline of an advertising management system concerning one embodiment (the 1st embodiment) of the present invention. It is a functional block diagram of the advertisement management system concerning one embodiment of the present invention. It is a functional block diagram of the advertisement management server in the advertisement management system concerning one embodiment of the present invention. In the advertisement management system concerning one embodiment of the present invention, it is an approximate line figure showing GUI at the time of registering image data with a registration client. In the advertisement management system concerning one embodiment of the present invention, it is an approximate line figure showing GUI at the time of registering image data with a registration client. In the advertisement management system concerning one embodiment of the present invention, it is an approximate line figure showing an example which displayed image data and advertisement data with a browsing client. In an advertisement management system concerning one embodiment of the present invention, it is an approximate line figure showing a flow of processing in a registration client at the time of image contribution, a front server, and an advertisement management server. It is a flowchart which shows the process sequence in the advertisement management server at the time of image contribution in the advertisement management system which concerns on one Embodiment of this invention. It is a basic diagram which shows the example of the recognition method instruction | indication data utilized with the advertisement management system which concerns on one Embodiment of this invention. In an advertisement management system concerning one embodiment of the present invention, it is an approximate line figure showing a flow of processing in a browsing client at the time of image browsing, a front server, an advertisement management server, etc. It is a flowchart which shows the process sequence in the advertisement management server at the time of image browsing in the advertisement management system which concerns on one Embodiment of this invention. In the advertisement management system which concerns on one Embodiment of this invention, it is a basic diagram which showed notionally the process sequence from image recognition to keyword acquisition. It is a basic diagram which illustrated the composition of the image identification DB used in the advertisement management system concerning one embodiment of the present invention, and the contents of the image data storage area. It is a basic diagram which showed another structural example of image identification DB utilized with the advertisement management system which concerns on one Embodiment of this invention, and another structural example of keyword DB. It is a basic diagram which showed another structural example of keyword DB utilized with the advertisement management system which concerns on one Embodiment of this invention. 6 is a flowchart illustrating an image recognition procedure in the advertisement management system according to the embodiment of the present invention. 6 is a flowchart illustrating an image recognition procedure in the advertisement management system according to the embodiment of the present invention. In the advertisement management system concerning one embodiment of the present invention, it is an approximate line figure which illustrated a selection pattern of image ID. It is a basic diagram which showed the example of a structure of advertisement listing DB utilized with the advertisement management system which concerns on one Embodiment of this invention. It is a basic diagram which shows the example of the hardware constitutions of the computer which comprises the advertisement management server of the advertisement management system which concerns on one Embodiment of this invention. It is a basic diagram which shows the example of the hardware constitutions of the portable apparatus utilized in the advertisement management system which concerns on one Embodiment of this invention. It is a basic diagram for demonstrating the process in the advertisement management system which concerns on the 2nd Embodiment of this invention. It is a basic diagram for demonstrating the process in the advertisement management system which concerns on the 3rd Embodiment of this invention. It is a basic diagram for demonstrating the process in the advertisement management system which concerns on the 4th Embodiment of this invention. It is a basic diagram for demonstrating the process in the advertisement management system which concerns on the 5th Embodiment of this invention.

Explanation of symbols

1 advertisement management system 10 registration client 20 browsing client 30 front server 50 advertisement management server 300 computer 301 CPU
302 Memory 303 Audio Output Device 304 Network Interface 305 Display Controller 306 Display 307 Input Device Interface 308 Keyboard 309 Mouse 310 External Storage Device 311 External Recording Medium Drive Device 312 Bus 320 Network 330 External Recording Medium

First, an outline of an advertisement management system according to one embodiment (first embodiment) of the present invention will be described with reference to FIG. The advertisement management system 1 of the present invention includes a registration client 10, a browsing client 20, a front server 30, an advertisement management server 50, and a network 90 that interconnects these.

As shown in FIG. 1, the registration client 10 is a portable device such as a mobile phone 10a or a computer such as a personal computer 10b, and registers the image data 92 in the front server 30 via the network 90 (ie, posting or uploading). To do. The registration client 10 may also be another type of portable terminal such as a smartphone or PDA. For example, the image data 92 is still image data in a format such as JPEG or PNG, but may be moving image data recorded in the MPEG format. When the mobile phone 10a is a mobile phone with a built-in imaging device (so-called camera-equipped mobile phone) that has been widely used in recent years, the captured image data 92 is usually stored in an internal memory or an external recording medium of the mobile phone 10a. The image data 92 can be uploaded to the front server 30 as it is.

Image data 92 photographed with a digital (still) camera or digital video camera, and image data 92 obtained through other routes are uploaded to the front server 30 via the mobile phone 10a or the personal computer 10b.

As shown in FIG. 1, the browsing client 20 is a portable device such as a mobile phone 20a or a computer such as a personal computer 20b, and browses image data 92 and advertisement data 96 provided from the front server 30 via the network 90. To do. The browsing client 20 may also be other types of portable terminals such as smartphones and PDAs. In FIG. 1, for convenience of explanation, the registration client 10 and the browsing client 20 are shown as separate ones. However, in some cases, the registered user himself / herself browses the image data 92 registered by the registration client 10. One device constitutes both the registration client 10 and the browsing client 20.

The browsing of the image data 92 and the advertisement data 96 is normally displayed on the display device of the browsing client 20 by a browser (Web browser) operating on the browsing client 20. In response to a request from the Web browser, the front server 30 sends a Web page including a description (description in HTML etc.) for displaying the image data 92 and the advertisement data 96 to the browsing client 20 using a protocol such as HTTP (HyperText Transfer Protocol). Send.

The front server 30 is a server that manages and operates, for example, a blog site, an SNS site, an album site, a bulletin board site, and the like, and basically has a function of a Web server. Although the front server 30 is represented as one computer in FIG. 1, it may be configured to share functions with a plurality of computers. In the case of a plurality of computers, each computer is connected via a public network or a private network as necessary.

In the advertisement management system 1 of the present invention, advertisement data 96 related to the image data 92 is provided, which is realized by two processes. The first process is a process in which, when the image data 92 is registered from the registration client 10, the advertisement management server 50 obtains the target ID 104 related to the image data 92 by performing the image recognition process. This target ID 104 is then provided to the front server 30 where it is stored, for example, in data (HTML data) of a predetermined Web page (a keyword such as the OCR text 110 may be stored).

In the second process, when the Web page is browsed from the browsing client 20, the advertisement management server 50 obtains the related advertisement data 96 based on the target ID 104 stored in the Web page. It is. The advertisement data 96 is then provided to the front server 30 where it is incorporated to be displayed on the web page requested to be viewed.

When the advertisement data 96 includes link information associated with the URL of the advertiser site or the like, the link information is displayed on the Web page, and the user of the browsing client 20 clicks on the link information. The advertiser site can be displayed on the display device of the browsing client 20.

As described above, by dividing the processing into two processes, it is possible to reduce the load on the computer such that the processing is performed at a stretch until the advertisement data 96 is obtained every time the image data 92 is registered. When the association with the advertisement data 96 changes frequently, the change can be dynamically reflected.

In this example, once the registration data 10 is registered by the registration client 10, the object ID 104 is obtained from the image data 92, and then the advertisement data 96 is obtained when the browsing client 20 browses the Web page. The advertisement data 96 related to the image data 92 is displayed through a two-stage process so that the image data 92 is registered by the registration client 10 or viewed. When the client 20 browses the Web page, the two processes can be performed continuously.

The front server 30 receives the image data 92 from the registration client 10 and registers it in the storage device 40 of the front server 30 when the user of the registration client 10 registers the image data 92 by an operation such as posting a photograph. The information is stored and transmitted to the advertisement management server 50 via the network 90. At this time, in addition to the image data 92, the front server 30 can transmit data for instructing a recognition method of the image data and recognition method instruction data 94 to the advertisement management server 50.

When the advertisement management server 50 receives the image data 92 and the recognition method instruction data 94 from the front server 30, the advertisement management server 50 performs image recognition processing on the image data 92 by the method specified by the recognition method instruction data 94. When the recognition method instruction data 94 is not transmitted from the front server 30, for example, the image recognition processing of the image data 92 is performed by a predetermined method.

Thereafter, when the target ID 104 corresponding to the image data 92 is acquired by the advertisement management server 50, the front server 30 receives the target ID 104 from the advertisement management server 50, and adds this to the data of the Web page and stores it. The data of the Web page is transmitted to the registration client 10 while being stored in the device 40.

When there is a web page browsing request from the browsing client 20, the front server 30 transmits the target ID 104 (corresponding to the image data 92) added to the data of the web page to the advertisement management server 50 via the network 90. To do. At this time, in addition to the target ID 104, the front server 30 provides data for indicating the original server (the user agent or address corresponding to the server) from which the advertisement data 96 is acquired, the number of advertisement data 96 to be acquired, and the like. The acquisition instruction data 95 can be transmitted to the advertisement management server 50.

When the advertisement management server 50 receives the target ID 104 and the advertisement acquisition instruction data 95 from the front server 30, the advertisement management server 50 acquires the advertisement data 96 for the target ID 104 by the method specified by the advertisement acquisition instruction data 95. When the advertisement acquisition instruction data 95 is not transmitted from the front server 30, the advertisement data 96 is acquired by a predetermined method, for example. The front server 30 receives the advertisement data 96 from the advertisement management server 50, updates the data of the web page so that the content of the advertisement data 96 is displayed on the web page, and stores the data of the web page in the storage device 40. And the web page data is transmitted to the browsing client 20.

Although the advertisement management server 50 is represented as one computer in FIG. 1, the advertisement management server 50 may be configured to share functions with a plurality of computers. For example, the image recognition process can be configured to function on different computers by a predetermined method. In the case of a plurality of computers, the computers are connected to each other via a public network or a private network as necessary. Furthermore, a configuration is possible in which the advertisement management server 50 is integrated with the front server 30 and the functions of both servers are performed by a single computer.

The network 90 is a wide area connection network such as the Internet, but is not limited thereto. For example, the network 90 can be configured with a LAN, a WAN (Wide Area Network) including a LAN and a dedicated line, an intranet, an extranet, various other topologies, and combinations thereof. When the registration client 10 or the browsing client 20 is a stationary personal computer, it is generally connected to the network 90 via a LAN cable or other cable, a router (modem), an ISP server, or the like. In the case of a portable terminal such as a mobile phone, the connection path to the network 90 includes a wireless communication network.

The advertisement data 96 includes data for representing advertisement information. Here, the advertising information includes various information such as information that is directly or indirectly related to sales of products and services, and information for image enhancement and recognition of products, services, and companies. Moreover, various data such as text, still images, moving images, audio, URLs (link information associated with URLs of advertiser sites, etc.) can be included.

In the example of FIG. 1, the front server 30 is configured to relay various data exchanges between the registration client 10 and the browsing client 20 and the advertisement management server 50, but the front server 30 is omitted ( Alternatively, the front server 30 and the advertisement management server 50 may be integrated), and the advertisement management server 50 may be configured to directly communicate with the registration client 10 and the browsing client 20.

In this example, the recognition method instruction data 94 is configured to be transmitted from the front server 30 together with the image data 92. However, the recognition method instruction data 94 can be transmitted at an arbitrary timing instead of the image data 92. The receiving advertisement management server 50 can perform control such that the image recognition processing after the recognition method instruction data 94 is received follows the instruction content of the recognition method instruction data 94. Further, the recognition method instruction data 94 can be transmitted from a computer other than the front server 30. Similarly, the advertisement acquisition instruction data 95 can be transmitted from various computers at various timings.

Further, in this example, the advertisement management server 50 is configured to transmit the advertisement data 96 to the front server 30 that has received the image data 92. However, as long as the advertisement management server 50 is connected via the network, the advertisement management server 50 responds to the designation. Can be sent to any computer.

Next, the functional outline of the advertisement management system 1 according to an embodiment of the present invention will be described with reference to the functional block diagrams of FIGS.

First, with reference to FIG. 2, an outline of functions of the registration client 10, the browsing client 20, and the front server 30 will be described. The registration client 10 includes a Web page display unit 11, a mail transmission unit 12, and a network I / F (interface) unit 13. The browsing client 20 includes a Web page display unit 21 and a network I / F (interface) unit 22.

The front server 30 includes a web access control unit 31, a content management unit 32, a web page generation / registration unit 33, an image data registration control unit 34, a web page browsing control unit 35, and a network I / F (interface) unit 36. The storage device 40 of the front server 30 stores Web page data 41 and related files 42.

The Web page display unit 11 of the registration client 10 is substantially a so-called Web browser. For example, by reading HTML data or the like in which a predetermined tag is described, text, still images, moving images, and the like are stored in the registration client 10. Output in the window of the Web browser displayed on the display device. When the registered user performs an operation for instructing registration of the image data 92 in the window of the Web browser displayed on the display device of the registration client 10, the mail transmission unit 12 is activated and the screen of the mail application is displayed. 10 display devices. Therefore, when the registered user designates the image data 92 as an attached file and issues a transmission instruction, the image data 92 is transmitted to the front server 30 via the network I / F unit 13 and the network 90 as an attached file of a mail. And stored as a related file 42.

Registration of the image data 92 can also be performed via a Web application displayed by a Web browser executed by the Web page display unit 11. For example, if the registration client 10 is the personal computer 10b, the registration operation includes key input from the keyboard, button display using the mouse (button-shaped mark displayed on the display device), and the like. On the other hand, if the registration client 10 is a type such as the mobile phone 10a, input or button display is selected using a push button.

The network I / F unit 13 of the registration client 10 controls data error check, transmission / reception timing, and the like when transmitting / receiving data to / from another computer connected to the network 90. For example, data transmission / reception between the registration client 10 and the front server 30 is also performed via the network I / F unit 13. In many cases, such functions are typically provided in portable devices and computers that connect to the network 90 to transmit and receive data. In addition to the registration client 10, the browsing client 20, the front server 30, and the advertisement management server 50 are also the same. It has the function of. Therefore, the description of the network I / F unit is omitted hereinafter.

The web page display unit 21 of the browsing client 20 has the same function as the web page display unit 11 of the registration client 10 described above. Image data 92 and advertisement data 96 are displayed in a web browser window displayed on the display device of the browsing client 20. When the advertisement data 96 includes link information associated with the URL of the advertiser site or the like, when the browsing user performs an operation of selecting the link information, the display in the web browser window is selected. The web page is switched to the URL corresponding to the link information. The operation for instructing selection is the same as the operation for instructing registration at the registration client 10.

The Web access control unit 31 of the front server 30 receives a request from the registration client 10 or the browsing client 20 using a protocol such as HTTP. When there is a request for a predetermined Web page, the Web page is acquired from the Web page data 41 and transmitted to the requesting client via HTTP (when there is image data 92 and advertisement data 96). As a result, a desired Web page is displayed on the display device of the registration client 10 or the browsing client 20. Such data exchange with each client is similar to the function of a conventional Web server. Further, here, it is assumed that the Web access control unit 31 also performs mail reception management. Further, communication between the registration client 10 or the browsing client 20 and the front server 30 is performed via the network I / F unit 36.

The content management unit 32 manages necessary data according to the control of the web access control unit 31. More specifically, when the registration instruction of the image data 92 is issued from the registration client 10 by sending a Web application or an attached file of an email, a file name is set in the received image data 92 as necessary, This is stored as a related file 42. If the web page requested from the browsing client 20 includes a description for displaying the image data 92 and the advertisement data 96, these data are acquired from the related file 42 and sent to the web access control unit 31. provide.

The web page generation / registration unit 33 dynamically updates / edits the web page requested from the registration client 10 or the browsing client 20 according to the control of the web access control unit 31. On the other hand, when there is an instruction from the registration client 10 to create a new web page (for example, when a new diary page is posted on a blog site), the web page is generated, and this is performed the next time. It is stored as Web page data 41 for the purpose of correction and browsing from other browsing users.

The image data registration control unit 34 transmits the image data 92 to the advertisement management server 50 via the network 90 when the image data 92 is registered from the registration client 10. At this time, the image data registration control unit 34 can be configured to transmit the recognition method instruction data 94 together with the image data 92 to the advertisement management server 50 based on the designation of the front server 30 by the site manager. The recognition method instruction data 94 is data for instructing how the advertisement management server 50 performs the image recognition processing of the image data 92. In addition to this, the front server 30, for example, information on the registered user who registered the image data 92 (user attribute information), the classification and genre to which the image data 92 belongs, registration date information of the image data 92, and registration of the image data 92 Information related to the registered user, the image data 92, and the registration operation, such as information on the location where the registration has been made, can be provided to the advertisement management server 50.

Further, when the target ID 104 is transmitted from the advertisement management server 50, the image data registration control unit 34 controls the Web page generation / registration unit 33 to display the corresponding image data 92 with the target ID 104. Therefore, it is added to the Web page generated for this purpose (corresponding to the image data 92). For example, the Web page data is modified so as to describe the target ID 104 in association with a tag for displaying the corresponding image data 92 or a predetermined tag. The Web page corrected in this way is stored in the storage device 40 as Web page data 41. A keyword such as the OCR text 110 may be transmitted from the advertisement management server 50 together with the target ID 104 or alone. In this case, such a keyword is similarly displayed on the Web page displaying the corresponding image data 92. Incorporated.

When the browsing client 20 browses a web page, the web page browsing control unit 35 extracts the target ID 104 described in the data of the web page and transmits it to the advertisement management server 50. Thereafter, when the advertisement data 96 is transmitted from the advertisement management server 50, it is stored in the related file 42. The advertisement data 96 is provided in, for example, an XML format. The web page browsing control unit 35 further controls the web page generation / registration unit 33 to update the web page data so that the advertisement information represented by the received advertisement data 96 is displayed on the web page requested to be browsed. To do.

At this time, the Web page generation / registration unit 33 can place the advertisement information (text, image, link information) included in the advertisement data 96 at an arbitrary page position according to the setting of the front server 30. In addition, a plurality of pieces of advertisement information (or a plurality of pieces of advertisement data 96) may be provided for one image data 92. In that case, the pieces of advertisement information can be arranged in an arbitrary position and order. In some cases, ranking information (for example, an order in which the affiliate condition is advantageous or an order in which the degree of association with the image data 92 is high) is assigned to the plurality of advertisement information from the advertisement management server 50. According to the user's setting, the front server 30 can control the arrangement of each advertisement information according to the order.

In this example, it is shown that an image is posted by a Web browser operating on the registration client 10 (by a Web application), but image posting processing is performed by various other applications of the registration client 10. Can be done. Similarly, the advertisement information can be displayed by various applications other than the Web browser operating on the browsing client 20.

Next, with reference to FIG. 3, a functional overview of the advertisement management server 50 will be described. The advertisement management server 50 includes a process control unit 51, an image data reception unit 52, an image recognition processing unit 53, a target ID acquisition unit 54, a target ID transmission / reception unit 55, a keyword search unit 56, an advertisement data search unit 57, and an advertisement data transmission unit. 58, a log acquisition unit 59, and a network I / F unit 60. The image recognition processing unit 53 further includes a first image recognition unit 53a, a second image recognition unit 53b, and a third image recognition unit 53c, and each image recognition unit uses a different image recognition engine. Further, the storage device 70 of the advertisement management server 50 includes an image identification DB 71, an object designation DB 73, a table designation DB 74, and a keyword DB 75.

The process control unit 51 of the advertisement management server 50 manages the entire process in the advertisement management server 50, and controls the order and timing of the processes performed by the function units (52 to 59).

The image data receiving unit 52 receives the image data 92 from the front server 30 via the processing control unit 51. Further, when the recognition method instruction data 94 is transmitted from the front server 30, the data is also received. Thereafter, the image data 92 is transferred to the image recognition processing unit 53 by the processing control unit 51, where image recognition processing is performed with reference to the image identification DB 71.

The image recognition processing unit 53 determines which image the image data 92 is similar to or includes what information is included based on a plurality of criteria, and obtains an image ID corresponding to the similar image. . If the image recognition engine is OCR, OCR text is required. In this example, the first image recognizing unit 53a and the second image recognizing unit 53b determine the degree of similarity or the like with different determination methods and different comparison target image data, and the third image recognizing unit 53c performs image data using OCR. Text data is acquired directly from 92. In the example of FIG. 3, three different image recognition units are prepared, but four or more image recognition units may be provided. Even if only one image recognition engine is used, it is possible to perform image recognition based on a plurality of standards by changing parameters or the like during image recognition processing. When the recognition method instruction data 94 is received from the front server 30, the recognition method is set according to the content of the instruction.

In this embodiment, an example in which the image recognition engine of the third image recognition unit 53c is OCR is shown. However, other recognition methods that directly acquire text (keywords) from the image data 92 can be employed. . For example, in the case of an image recognition engine that performs face detection, when image data 92 showing a person is input, the presence of “face” is recognized and the text “face” is returned. In this case, the text data “face” can be directly used as a keyword for acquiring an advertisement. Further, if a technique called face authentication is used, it is possible to acquire even the name of the person (if the face is registered in the DB). In addition to human faces, there are various image recognition engines for recognizing predetermined “categories” such as cars, hands, and animals, including those at the research level. The category name obtained by such category recognition can be used as an advertisement acquisition keyword.

The image recognition processing in the advertisement management system 1 of the present invention includes processing for determining similarity to the image data 92 using comparison target image data stored in a DB or the like, and processing for obtaining text directly from the image data 92. Depending on the application, one or both treatments are used. In this example, in the processing of the first image recognition unit 53a and the second image recognition unit 53b, the image data to be compared (its feature amount) is associated with each unique image ID, and each image ID is 1 Each object is associated with an object ID 104 representing an idea.

The target ID acquisition unit 54 refers to the target specification DB 73 and acquires a target ID corresponding to the image ID obtained by the image recognition processing unit 53.

When the registration client 10 registers the image data 92, the target ID transmission / reception unit 55 transmits the target ID obtained by the target ID acquisition unit 54 to the front server 30 and displays the image data 92 from the browsing client 20. When the web page is browsed, the target ID associated with the image data 92 is received from the front server 30. The target ID transmission / reception unit 55 may receive the advertisement acquisition instruction data 95 along with the reception of the target ID 104.

In the keyword search unit 56, when the target ID is received by the target ID transmission / reception unit 55, the keyword is determined based on the received target ID with reference to the table designation DB 74 and the keyword DB 75. Here, a table for searching for keywords is set for each target ID. Such multi-level association is performed by a plurality of databases. The keyword search unit 56 determines a keyword by searching such a plurality of databases. In this example, a plurality of keywords may be determined for one image data 92.

The advertisement data search unit 57 acquires the advertisement information using the existing advertisement distribution system 150 based on the keyword determined by the keyword search unit 56. When the target ID transmission / reception unit 55 receives the advertisement acquisition instruction data 95, the advertisement information is acquired according to this instruction. For example, the advertisement acquisition instruction data 95 includes an identifier (user agent or address) of the advertisement distribution system 150 or server used for obtaining the advertisement information, the number of advertisement information to be acquired (the number of lower and upper limits), and the like. May be included.

The advertisement distribution system 150 is an external system that, when given a keyword, returns advertisement information related to the keyword. The advertisement distribution system 150 may return a plurality of pieces of advertisement information for one keyword. Each advertisement information includes commentary on products and services, sales promotion information, URLs of sales sites, CM images, and the like. In addition, as shown in FIG. 3, advertisement information related to the keyword may be acquired using an advertisement listing DB 76 that is uniquely prepared and stored in the storage device 70.

The advertisement data transmission unit 58 transmits the advertisement data 96 acquired by the advertisement data search unit 57 to the front server 30. Transmission of the image data 92 to the advertisement management server 50 and reception of the advertisement data 96 in the front server 30 are provided in the form of, for example, a Web API. The advertisement data 96 can include a plurality of advertisement information. In addition, as described above, when there are a plurality of pieces of advertisement information, rank information indicating a rank regarding the advertisement information may be included.

Next, an example of a GUI when image data 92 is registered (uploaded to the front server 30) by the mail transmission unit 12 of the registration client 10 will be described with reference to FIGS. This example is for posting an article on a blog site for the mobile phone 10a, and the mail transmitting unit 12 substantially uses the mail application of the registration client 10 (mobile phone 10a) to generate image data 92. Is attached to the mail as an attached file, thereby transmitting the image data 92 to the front server 30. Also, in this example, text is simultaneously posted by entering text in the body of the mail.

FIG. 4A shows a mail creation instruction screen 200 displayed on the display device of the mobile phone 10a. The mail creation instruction screen 200 is displayed by, for example, pressing a “post article” button with a push button or the like of the mobile phone 10a in a state where the blog site is accessed by the web page display unit 11 (web browser). Is displayed. In the mail creation instruction screen 200, a destination input area 201, a subject input area 202, an attached file input area 203, a text input designation link 204, and a button display unit 205 are displayed. ”Button,“ Previous page ”button,“ Enter ”button,“ Submenu ”button, and“ Next page ”button are displayed.

In the destination input area 201, the image data 92 and the e-mail address of the blog site that receives the posted article are designated in advance or by input of a registered user. In addition, the text “sunset” is input to the title input area 202 by the registered user. When the attached file input area 203 is selected with a push button or the like of the mobile phone 10a, a pop-up window and an attached file selection screen 210 shown in FIG. 4B are displayed. The attached file selection screen 210 displays a file type list 211 that displays a plurality of file types 212 indicating the type of file to be attached, such as “1 image (still image) attachment”, “2 melody attachment”, etc. ing. Here, when the file type 212 of “1 image (still image) attachment” is pressed with a push button or the like of the mobile phone 10a, an image selection screen 220 shown in FIG. 4C is displayed on the display device of the mobile phone 10a. .

The image selection screen 220 displays a thumbnail image 221, a file name input area 222, and a button display unit 223 corresponding to a plurality of image data 92 stored in the memory of the mobile phone 10a. In “223”, a “OK” button, a “Confirm” button, and a “Submenu” button are displayed. Here, when the image data 92 transmitted by the registered user is selected by a push button (movement button) of the mobile phone 10a, the file name of the selected image data 92 in the mobile phone 10a is input to the file name input area 222. . Thereafter, when the “OK” button is selected, as shown in FIG. 5A, the display device of the mobile phone 10a returns to the display of the mail creation instruction screen 200 again. In FIG. 4A, “(no attachment)” is displayed. In the attached file input area 203, the file name of the image data 92 input to the file name input area 222 of FIG. 4C is displayed.

Here, when the text input designation link 204 is selected, the display on the display device of the mobile phone 10a changes to a display as shown in FIG. 5B. As can be seen from a comparison with FIG. 5A, the text input area 231 is newly displayed by selecting the text input designation link 204, and the registered user inputs text information there. When the input of the text information is completed, the “Send” button is selected, whereby the image data 92 stored in the memory of the mobile phone 10a and the posted article (text information) input by the registered user are displayed on the front server. 30. Also, by this operation, the transmitted image data 92 is used for displaying an image of a blog post article, and text information is used for a comment portion of the blog post article.

In this example, the image data 92 is transmitted to the front server 30 by using the attachment function of the mail application provided as a standard in the mobile phone 10a. However, the image data 92 is transmitted by various other methods. be able to. For example, by selecting button display according to the instruction of the Web page displayed by the Web page display unit 11, the image data 92 stored in the memory of the mobile phone 10a is designated and uploaded to the front server 30. It is also possible to construct a simple web application.

In this example, the text (article) posted on the blog site is input to the body of the email and transmitted to the front server 30, but this can also be performed by various existing methods. For example, it is possible to construct a Web application that inputs text into a predetermined input area of a Web page and uploads it to the front server 30.

4 and 5 show an example in which the image data 92 is registered in the front server 30 using the mobile phone 10a. However, as described with reference to FIG. 1, various types of devices such as a stationary personal computer are used. The image data 92 can be registered using a computer. Here, an example is shown in which the image data 92 taken by the camera of the mobile phone 10a and stored in the memory of the mobile phone 10a is transmitted as it is to the front server 30, but a digital camera or digital video camera is separately provided. An image photographed by the above-mentioned method or an image obtained by other means can be transmitted as image data 92 to the front server 30 via the mobile phone 10a or the personal computer 10b.

Next, referring to FIG. 6, an example in which the Web page display unit 21 of the browsing client 20 displays the image data 92 and the advertisement data 96 provided from the front server 30 on the display device of the browsing client 20. Will be described. In this example, it is assumed that an article posted by the GUI shown in FIGS. 4 and 5 is displayed on the mobile phone 20a of another browsing user on the blog site for mobile phones.

FIG. 6 shows a blog article display screen 250 displayed on the display device of the mobile phone 20a. The blog article display screen 250 is displayed, for example, when a specific article is selected by pressing the push button of the mobile phone 20a while the blog site is being accessed by the web page display unit 21 (web browser). . The blog article display screen 250 includes a blog title 251, an article title 252, a display image 253, a first advertisement 254, a second advertisement 255, a blog text display area 256, position information 257, a user designation tag 258, a third advertisement 259, A content editing instruction link 260, a photo deletion instruction link 261, and a comment display area 262 are displayed.

Among the items displayed on the blog article display screen 250, the blog title 251, article title 252, display image 253, blog text display area 256, content editing instruction link 260, photo deletion instruction link 261, and comment display area 262 are as follows: It is an item commonly found on previous blog sites. Among these, the contents displayed in the article title 252, the display image 253, and the blog text display area 256 are contents registered by the series of operations shown in FIGS. 4 and 5. Is stored in the Web page data 41 of the front server 30 and the related file 42 and then downloaded and displayed in response to a request from the Web page display unit 21. The display image 253 is obtained by displaying the image data 92 registered by the registration client 10 as it is or with a different resolution and quality. In the comment display area 262, comments posted on this article by the viewing user are displayed.

The first advertisement 254, the second advertisement 255, and the third advertisement 259 are advertisement information determined by the advertisement management server 50 based on the image data 92 registered by the registration client 10. The advertisement management server 50 may output a plurality of advertisement information regarding one image data 92, and transmits it to the front server 30 as advertisement data 96. The front server 30 edits the web page to display the received advertisement information on the web page, stores the web page in the web page data 41, and transmits the web page to the requesting client in response to a request from the web browser or the like. .

At least one of the advertisement management server 50 and the front server 30 can reduce the number of obtained advertisement information and the content of each advertisement information based on a predetermined rule. In the example of FIG. 6, three pieces of advertisement information are displayed, but the front server 30 is based on its own rules or based on the rank information (when rank information is given from the advertisement management server 50). The display mode such as the display order, display position, and display size of the advertisement information can be determined.

In the position information 257, for example, information on the location of the registered client 10 when an article is posted by the registered client 10 is displayed. Such position information can be acquired by GPS (Global Positioning System), and is provided from the registration client 10 to the front server 30 when an article is posted. In recent years, many portable devices with built-in GPS have been sold, and when such a device is the registered client 10, the information can be acquired and transmitted relatively easily.

The user designation tag 258 is a tag designated when the registered user posts an article by the registered client 10. The tag is a keyword for representing the classification and genre of the content of the posted article. When a registered user posts an article, the registered user determines which genre the article belongs to and designates a related tag. The registered user can specify a plurality of such tags for each posted article, but it is also possible not to specify them. Moreover, the keyword used for a tag may be freely set by a registered user.

FIG. 6 shows an example of displaying the image data 92 and the advertisement data 96 provided from the front server 30 using the mobile phone 20a. However, as described with reference to FIG. The display can be performed using various types of computers.

Up to this point, the related advertisement data 96 has been provided and displayed based on the image data 92 provided in the blog system through FIGS. 4 to 6, but the image data 92 is provided from the registration client 10. As long as it is done, the idea of the present invention can be applied to any other system. For example, the present invention can be applied to a system that operates and manages an SNS site, an album site, and a bulletin board site.

Next, a first process in the advertisement management system 1, that is, a process when the image data 92 is registered from the registration client 10 will be described with reference to FIG. FIG. 7 shows what processing is performed in the registration client 10, the front server 30, and the advertisement management server 50, and what data is exchanged between the nodes.

First, the image data 92 is transmitted to the front server 30 by posting the image in the registration client 10. When receiving the image data 92 from the registration client 10, the front server 30 transmits a target ID acquisition request to the advertisement management server 50. This target ID acquisition request is for acquiring a target ID corresponding to the image data 92. The target ID acquisition request includes, for example, authentication data such as a service ID and a service key, and image data 92 for confirming whether the connection source (front server) is a legitimate node. Further, authentication data and image data 92 can be included in the request by a description method in XML format. At this time, the image data 92 can be specified to include the data itself, but the address (URI or the like) of the image data 92 can also be specified as text data.

The front server 30 can specify the recognition method instruction data 94 in the target ID acquisition request and provide it to the advertisement management server 50.

Next, the advertisement management server 50 that has received the target ID acquisition request performs image recognition on the image data 92 by the image recognition processing unit 53 and obtains the image ID 102. Thereafter, the target ID acquisition unit 54 obtains the target ID 104 based on the image ID 102 and transmits this to the front server 30 as a response to the target ID acquisition request. In this response, if the authentication data is not correct, an error message is returned. In addition, a plurality of target IDs may be returned for one image data 92. If the recognition engine includes OCR, the OCR text 110 is output.

Each expiration date can be set for each target ID. This is because there is a possibility that different image IDs may be returned depending on the time even for the same image data 92 because the images to be compared are updated daily, and the target ID determined based on such image IDs. This is to limit the effectiveness of the system within a certain period.

The front server 30 that has received the response to the target ID acquisition request incorporates the received target ID into a Web page related to the posted image data 92. At this time, if each target ID has an expiration date, the information is also stored in the Web page in association with the corresponding target ID. The front server 30 periodically checks the expiration date, and transmits a target ID acquisition request (to the advertisement management server 50) again for the image data 92 of the target ID that has passed the expiration date. Can be refreshed with the latest information.

The Web page in which the target ID is embedded in the front server 30 is transmitted to the registration client 10. Here, on the display device of the registration client 10, the Web in which the registered image data 92 is indicated as a response to image posting. A page is displayed. The advertisement data 96 is not yet displayed on the Web page at this time.

FIG. 8 is an example of a processing outline of the advertisement management server 50 in the case shown in FIG. First, in step S <b> 10, the image data receiving unit 52 of the advertisement management server 50 receives the image data 92 from the front server 30 through a target ID acquisition request. Further, when the recognition method instruction data 94 is transmitted from the front server 30, the data is also received.

After that, if there is the recognition method instruction data 94 in step S11, the image recognition process is executed according to this. The recognition method instruction data 94 is, for example, data as shown in FIG. FIG. 9 shows three examples of the recognition method instruction data 94 from FIGS. 9A to 9C. In this example, the recognition method instruction data 94 includes data for designating a recognition engine, use / non-use, use conditions, and use DB for each of the three image recognition units.

Here, for example, when the image data 92 is given, the first image recognition engine 85a returns one image ID related to the image data having the highest similarity among the similarities exceeding the first threshold value (first image). If nothing exceeds the threshold, nothing is returned). For example, the second image recognition engine 85b determines the similarity by a method different from that of the first image recognition engine 85a, and when the image data 92 is given, the second threshold is different from the first threshold or the first threshold. Returns a plurality of image IDs related to image data having a degree of similarity exceeding the threshold (if there is no image data exceeding the threshold, nothing is returned). The third image recognition engine 85c is, for example, an OCR engine and returns text information recognized from the image data 92 when the image data 92 is given.

9A, the first image recognition unit 53a shown in FIG. 3 uses the first image recognition engine 85a (No. 1 in FIG. 9A), and the second image recognition unit 53b receives the first recognition method instruction data 94 shown in FIG. The second image recognition engine 85b is used (No. 2 in FIG. 9A), and the third image recognition unit 53c uses the third image recognition engine 85c (No. 3 in FIG. 9A). For the image recognition engine, use / non-use, use conditions, and use DB are defined. Use / non-use specifies whether to use the corresponding image recognition engine and under what start conditions. In the example of FIG. 9A, 1 (sequential use) is set for all the image recognition engines, which uses all the image recognition engines, and the first image recognition engine 85b is the first image recognition engine 85a. The third image recognition engine 85c starts after the process of the second image recognition engine 85b ends.

The use condition specifies when to use the corresponding image recognition engine. In the example of FIG. 9A, the first image recognition engine 85a is 0 (none), and the second image recognition engine 85b is used when there is no similar image that satisfies the condition in the first image recognition engine 85a. The third image recognition engine 85c is used when there is no similar image that satisfies the condition in the second image recognition engine 85c.

The use DB designates the type and version of a database (image identification DB 71) for specifying a set of image data used as a comparison target when the image data 92 is given. In the example of FIG. 9A, the first image recognition engine 85a uses an image identification DB 71 called DB003, and the second image recognition engine 85b uses an image identification DB 71 called DB019. Since the third image recognition engine 85c is an OCR engine in this example, there is no designation of the image identification DB 71, but a specific data set (for example, learning data) used for recognition by the OCR engine is designated here. You can also. Thus, since the administrator of the front server 30 can designate the image identification DB 71 used in the image recognition engine each time, for example, the image data 92 posted on a DVD-related blog site is enriched with DVD images. By specifying that the database to be used is used, more accurate image recognition can be expected.

In the example shown in FIG. 9B, image recognition is performed in three sequential steps, but the second image recognition engine 85b is designated to be used in two ways. Unlike the pattern in FIG. 9A, the second image recognition engine 85b (No. 2 in FIG. 9B) that is used first performs recognition processing even when there is a similar image that satisfies the conditions in the first image recognition engine 85a. Execute. The second image recognition engine 85b (No. 3 in FIG. 9B) for the second time separates the image identification DB 71 when there is no similar image that satisfies the conditions in the recognition process in the first second image recognition engine 85b. The recognition process is performed again instead of the one (DB042).

In the example shown in FIG. 9C, the specification is such that an image recognition engine can be selected, and the image recognition engine (E003) used by the first image recognition unit 53a and the image used by the third image recognition unit 53c. It is specified that the recognition process in the recognition engine (E008) is performed in parallel.

Referring back to the flowchart of FIG. Here, it is assumed that the designation shown in FIG. 9A is received as the recognition method instruction data 94 in step S10, and in step S11, image recognition processing is performed according to this designation. Next, in step S12, it is determined whether either the image ID 102 or the OCR text 110 is obtained as a result of the image recognition process. If neither is obtained (NO in step S12), the process proceeds to step S16, where predetermined advertisement data 96 is acquired. As described above, in the process of step S16, when there is no image similar to the image data 92 or text information recognized by the OCR from the image data 92, predetermined advertisement information is provided and displayed on the browsing client 20. However, it is also possible to take a method of not providing advertisement data or providing information indicating that a similar image or the like is not found.

If either an image ID or text is obtained in step S12 (YES in step S12), the process proceeds to step S13, and it is determined whether a similar (matching) image ID 102 is obtained in the image recognition process. . When a similar image ID 102 is obtained (YES in step S13), in step S14, the target designation DB 73 is searched with the image ID, and the target ID 104 corresponding to the image ID 102 is acquired.

Next, in step S15, the target ID 104 is transmitted to the front server 30 as a response to the target ID acquisition request. If there is OCR text, it is also sent. The response data includes a description in XML format, for example.

Next, a second process in the advertisement management system 1, that is, a process when a Web page is browsed from the browsing client 20 will be described with reference to FIG. FIG. 10 shows what processing is performed in the browsing client 20, the front server 30, the advertisement management server 50, the advertisement distribution system 150, and the advertiser site 160, and what data is exchanged between the nodes. It expresses how.

First, when the browsing client 20 designates a page with a blog, a Web page browsing request is transmitted to the front server 30. When the front server 30 receives a Web page browsing request from the browsing client 20, the front server 30 transmits an advertisement acquisition request to the advertisement management server 50. This advertisement acquisition request is for acquiring advertisement data 96 corresponding to the target ID. The advertisement acquisition request includes, for example, authentication data such as a service ID and a service key, and a target ID 104 for confirming whether the connection source (front server) is a legitimate node.

The target ID 104 is stored in the Web page data in association with the image data 92 of the Web page requested to be browsed, and a plurality of target IDs 104 can exist for one image data 92. In some cases, the OCR text is stored in the web page. In this case, the OCR text is included in the advertisement acquisition request. Authentication data and target ID 104 can be included in the request by a description method in XML format. Further, the advertisement acquisition request can include identification information of the destination from which the advertisement data 96 is acquired and designation of the number of advertisements to be acquired.

Next, the advertisement management server 50 that has received the advertisement acquisition request uses the keyword search unit 56 to determine the keyword 106 with reference to the table designation DB 74 and the keyword DB 75 based on the target ID 104 received by the request. When the OCR text is received, the OCR text is set as the keyword 106.

Thereafter, the advertisement data search unit 57 provides the keyword 106 to the advertisement distribution system 150 and receives the advertisement data 96 related to the keyword 106. Upon receiving the advertisement data 96, the advertisement management server 50 transmits it to the front server 30 as a response to the advertisement acquisition request. As the response data, one or a plurality of advertisement data 96 is described in, for example, an XML format.

The front server 30 that has received the response to the advertisement acquisition request generates a Web page so that the received advertisement data 96 is displayed, and transmits the generated Web page as a response to the Web page browsing request from the browsing client 20. To do.

If there is link information associated with the URL of the advertiser site or the like in the advertisement data 96, the display is switched to the Web page of the advertiser site 160 if the browsing user clicks on this. What is shown in the lower part of FIG. 10 is a flow when the browsing user clicks the link information of the advertisement. In this example, by clicking the link information of the advertisement, the address setting of the HTTP redirect is set such that the advertisement management server 50 is once accessed and the log is recorded there. By such log recording, the advertisement management server 50 can grasp how much actual access has been made to which advertisement data 96. In the redirect setting in this case, the browsing user 20 displays the web page of the advertiser site 160 on the browsing client 20 without being aware that the click is recorded in the log. It is also possible to display the Web page to be displayed on the browsing client 20 and to explicitly indicate to the browsing user that log recording has been performed or that the advertisement management server 50 has been accessed. Further, without providing such a redirect setting, the page of the advertiser site 160 can be displayed as it is without log recording.

FIG. 11 is an example of a processing outline of the advertisement management server 50 in the case shown in FIG. First, in step S20, an advertisement acquisition request is received from the front server 30, and a target ID and OCR text are received. In step S21, it is determined whether the reception request includes the target ID. If the text is an OCR text (NO in step S21), the process proceeds to step S24, where the OCR text is processed.

If it is the target ID (YES in step S21), in step S22, the table designation DB is searched by the target ID, and the table ID corresponding to the target ID 104 is acquired. Thereafter, in step S23, using the acquired target ID, the table specified by the acquired table ID is searched, and one or more keywords are acquired.

In step S24, it is determined whether text is obtained by image recognition by OCR. If the OCR text has been obtained (YES in step S24), the process proceeds to step S25, where the OCR text is set as a keyword. If the keyword is obtained even by acquiring the image ID, the setting is made so as to include both keywords. At this timing, any keyword can be deleted using a predetermined rule.

If it is determined in step S24 that the OCR text is not obtained (NO in step S24), the process of step S25 is skipped and the process proceeds to step S26. In step S26, the keyword is provided to the existing advertisement distribution system 150, and advertisement data 96 including one or more pieces of advertisement information related to the keyword is acquired from the system. Finally, in step S27, the advertisement data 96 acquired in step S26 is transmitted to the front server 30.

Next, the process from image recognition to keyword acquisition will be described in more detail with reference to FIG. 12 and FIG. FIG. 12 is a diagram conceptually showing processing from image recognition to keyword acquisition. The upper part of FIG. 12 shows a method for acquiring the image ID 102 and the OCR text 110 by the image recognition process, and the lower part shows a method for acquiring the keyword 106 from the image ID 102 through the target ID 104. .

The processing control unit 51 of the advertisement management server 50 uses the first image recognition unit 53a, the second image recognition unit 53b, and the third image recognition unit 53c according to the content of the recognition method instruction data 94 received by the image data reception unit 52. Control the image recognition process. For example, the first image recognition unit 53a uses the first image recognition engine 85a to obtain the feature amount of the image data 92, and then compares the feature amount with the comparison target image data stored in the image identification DB 85a. One of the image IDs 102 corresponding to the image data having the smallest difference, that is, the highest similarity among the features having a difference between the feature amounts equal to or smaller than a predetermined threshold (for example, the first threshold) is compared. return. In this example, the image data corresponding to each image ID 102 stored in the image identification DB 85a is data prepared for CD recognition. Here, the feature amount is used as a reference for determining the similarity of image data, but the similarity is determined using various criteria such as other criteria or a combination of other criteria and feature values. can do.

Here, with reference to FIG. 13, the configuration of the image identification DB 71 and the image data storage area in which the image data related thereto are stored will be described. FIG. 13 shows the configuration of the image identification DB 71. The image identification DB 71 stores the image ID 102 and the feature amount data of the image data corresponding to the image ID 102 in a one-to-one correspondence. The feature amount data is data representing the features of the image, and is stored as, for example, binary data or metadata such as XML. Although the image identification DB 71 is described as a tabular database here, various other storage forms / access forms are conceivable.

The feature amount data stored in the image identification DB 71 is obtained by performing feature amount calculation for each image data. For example, by calculating the feature amount for the image data with the image file name “001.jpg”, the feature amount data “D001” is obtained, and the feature amount is calculated for the image data with the image file name “002.jpg”. Thus, the feature amount data “D002” is obtained. Here, “D001” and “D002” are file names, and the data itself is stored in a file indicated by the file name. The file name usually has path information including a drive name and a directory name, or URI information that can identify a resource on the Internet, but is omitted here for convenience of explanation.

For example, if "001.jpg", "002.jpg", and "003.jpg" are actually images that represent the same object at different angles, these are "GH001", "GH002", and "GH003", respectively. Are associated with different image IDs.

Referring to FIG. 12 again, with this configuration of the image identification DB 71, the first image recognition engine 85a reads the feature amount data from the image identification DB 85a and sequentially compares the feature amount data with the feature amount data of the image data 92 to determine the similarity. To do. Then, in the recognition process for all the feature amount data (corresponding to the image data) stored in the image identification DB 85a, one image ID 102a corresponding to the image data with the highest similarity is selected and returned to the processing control unit 51. .

Also, in order to determine the degree of similarity regarding a moving image, a frame of the moving image is taken out as a still image, and feature amount data between the still image and comparison target image data is compared. Which frame is to be extracted from a moving image composed of a plurality of frames is determined by a predetermined rule. For example, a method is also possible in which the timing at which the scene changes greatly is obtained quantitatively and the frame at this time is extracted. Further, when a plurality of frames are taken out, the degree of similarity can be determined for each frame, a plurality of image IDs 102 can be acquired, and the advertisement data 96 can be acquired. When a plurality of image IDs 102 are obtained, selection can be made according to the degree of similarity, the content and attributes of the original still image, and the like.

The second image recognition unit 53b uses the second image recognition engine 85b. For example, the second image recognition engine 85b determines the degree of similarity by a method different from that of the first image recognition engine 85a. When the image data 92 is given, the difference in the feature amount is determined by the first image recognition engine 85a. A plurality of image IDs 102 that are equal to or less than a predetermined threshold (first threshold) or a different threshold (second threshold) are returned.

The second image recognition engine 85b reads the feature amount data from the image identification DB 71h and sequentially compares the feature amount data with the feature amount data of the image data 92 to determine the similarity. Then, the second image recognition engine 85b recognizes all the feature amount data stored in the image identification DB 71h (image data corresponding thereto), and the image ID 102b corresponding to the image data having a similarity equal to or less than a predetermined threshold value. Are selected and returned to the processing control unit 51.

The third image recognition unit 53c uses, for example, a third image recognition engine 85c such as OCR. When the image data 92 is given, the third image recognition engine 85c returns text recognized from the image. When the text information is obtained by the recognition processing, the third image recognition engine 85c returns this to the processing control unit 51 as the OCR text 110. As described above, such image recognition processing for obtaining text data directly from the image data 92 is not limited to OCR. There is also processing for recognizing the relationship with a certain category from the image data 92 and returning the category name. In an example of such category recognition processing, the feature of the category to be recognized is “learned” by the program itself without using the DB so that the category can be recognized. For example, in the process of recognizing a face, a program that gradually reacts to a “human face-like image” is created by using an algorithm called “machine learning” for many human face images.

Next, the target ID acquisition unit 54 acquires the target ID 104. The target designation DB 73 is searched using the image ID 102a obtained by the first image recognition unit 53a and the image ID 102b obtained by the second image recognition unit 53b as keys. The target designation DB 73 defines a correspondence relationship between the image ID 102 and the target ID 104, and represents what the image ID 102 is targeted for. In the example illustrated in FIG. 12, the image represented by the image ID “1” is associated with a certain artist's CD (target ID “1”). When the same CD is shot at different angles, it is determined that the image ID is “2”, and the images are distinguished as different images. However, in the object designation DB 73, these image IDs have the same object ID “1”. Therefore, the image IDs “1” and “2” are finally associated with the same CD. Thus, when different images represent substantially the same target, the same target ID 104 is associated.

When it is determined that the similarity between the image data 92 and the comparison target image data is extremely high, the image recognition unit such as the first image recognition unit 53a stores the image data 92 and The feature amount data of the image data 92 and the image ID 102 corresponding to the image data 92 are newly registered in the identification DB 85a. Further, the newly registered image ID 102 and the target ID associated with the similar image data are associated with each other and registered in the target designation DB 73. Thus, the image data 92 itself registered by the registered user can be effectively used for determining the target ID 104 of the image data 92 registered thereafter.

Next, the keyword search unit 56 acquires the keyword 106. When the target ID 104 is acquired, the table designation DB 74 is searched using the target ID 104 as a key. The table designation DB 74 defines a correspondence relationship between the target ID 104 and a table to be searched for acquiring the keyword 106 with the target ID 104. In the example shown in FIG. 12, the table ID “T1” and the table ID “T2” are used for the target ID “1” and the target ID “2”, and the table ID “T11” is used for the target ID “4”. ”Table is defined to be used. Thus, the table specification DB 74 can switch the table used for acquiring the keyword 106 to an optimum one.

Further, when there are a plurality of usable tables as in the case of the target ID “1” and the target ID “2”, the keyword 106 is acquired using both of them, but either one or one of them is acquired. It is also possible to select and use one table.

Next, the table represented by the table ID obtained by searching the table designation DB 74 is searched using the target ID 104 as a key, and the keyword 106 is acquired. In the example of FIG. 12, in order to acquire the keyword 106 for the target ID “1”, the table ID “T1” (table T1) and the table ID “T2” (table T2) are used. In this example, since the target ID “1” corresponds to the genre of CD, the table T1 is a table related to CD products provided by the company A, and the table T2 is a table related to CD products provided by the company B. The target ID “1” is converted into a product ID of each company by the target ID conversion unit 77 as necessary, and the table T1 and the table T2 are searched using the product ID as a key. For example, the target ID “1” is converted into the product ID “A3” for the search of the table T1, and the items of the title “C” and the artist “CC” associated with the product ID are acquired as the keywords 106. Is done. Similarly, the target ID “1” is converted into the product ID “B3” for the search of the table T2, and the items of the title “G” and the label “FFF” associated with the product ID are used as the keywords 106. To be acquired.

As in the table T1 and the table T2 shown in FIG. 12, even if the tables are related to the same genre, they may have different items for the same product. By using both tables, a wider range of keywords 106 can be obtained. Can do. The table T1 and the table T2 can include various information related to the product, such as the song lyrics, composition, name of the performer, commentary / criticism information, release date, and the like. All or part of the item can be acquired as the keyword 106.

Also, the target ID 104 not only corresponds to products and services, but also corresponds to various concepts. For example, the target ID “4” relates to a landscape. In this case, the table T11 is a table including keywords related to the landscape. In this example, the table T11 is searched using the target ID as a key as it is. When the object ID “4” is searched, “sky”, “evening”, and “cloud” are acquired as related words. The keyword search unit 56 acquires all or part of the related words as the keywords 106.

In the example shown in FIG. 12, one or a plurality of tables are provided for keyword search for each target ID 104. However, the table designation DB 74 is omitted and only one table for keyword search is provided in total. A simpler configuration without it can also be used.

The target ID 104, the product ID converted by the target ID conversion unit 77, and the like are identifiers associated with the image ID 102 by the target specification DB 73, the table specification DB 74, and the target ID conversion unit 77. In the present invention, having an identifier associated with such an image ID 102 is also one of the features, which facilitates design flexibility and partial addition / modification of data. However, the image ID 102 and the keyword DB 75 can be directly linked without using such an identifier. Furthermore, the keyword DB 75 and the table can be switched according to the image ID 102. For example, when the target ID 104 is not used, the target ID transmission / reception unit 55 transmits / receives the image ID 102 instead of the target ID 104, and the target ID acquisition unit 54 becomes unnecessary.

The keyword DB 75 used for acquiring the keyword 106 can be configured to be designated from the registration client 10, the browsing client 20, the front server 30, and the like. Thus, there are cases where more effective keywords can be acquired by specifying the keyword DB 75 from a position close to the end user. For example, when it is clear that a web page of a blog is related to a certain theme, there is often a certain tendency in the images posted on the web page. Then, when the administrator of the front server 30 that operates and manages the blog obtains the advertisement data 96 related to the image data 92 from the advertisement management server 50, the administrator specifies the keyword DB 75 that is close to the theme, and is effective as a result. The advertisement data 96 can be obtained.

FIG. 14 is an example of another DB used in processing from image recognition to keyword acquisition. 14A shows the configuration of the image identification DB 71 ′. Unlike the image identification DB 71 shown in FIG. 13, different feature amount data (image data thereof) is associated with the same image ID. . For example, image data having feature amount data “D001”, “D002”, and “D003” is substantially associated with one image ID “GH001”. For this reason, when the same product is photographed from different angles, it is associated with one image ID by the image identification DB 71 ′. If the image identification DB 71 ′ having such a configuration is prepared instead of the image identification DB 71, it can play a part of the target designation DB 73 (see FIG. 12).

FIG. 14B is another configuration example for the keyword DB 75. In the keyword DB 75 ′, related words corresponding to the target ID 104 are collectively stored regardless of the target genre to which the target ID 104 is related. For example, the target ID “1” is related to a CD, and the related word is a group name, an album title, or the like. On the other hand, the target ID “3” is related to the landscape, and words such as “sea” and “blue” are set as related words.

FIG. 15A shows still another configuration example of the keyword DB 75. In the keyword DB 75 ″, the storage method of the related word corresponding to the target ID 104 changes according to the genre of the target related to the target ID 104. For example, the target ID “1” is related to a CD, and the related word includes product information including a product name and an artist, and related information other than the product information. On the other hand, the target ID “19” is related to the landscape, and as related information, words such as “sky” and “evening” are set.

The keyword DB 75 '' of FIG. 15B has a configuration in which a plurality of tables of the keyword DB 75 shown in FIG. 12 are integrated into one. When the keyword DB having such an integrated configuration is prepared, the table determination process using the table designation DB 74 (see FIG. 12) is not necessary.

Next, with reference to FIG. 16 and FIG. 17, a procedure in which the processing control unit 51 controls the image recognition processing using each image recognition unit and each image recognition engine will be described in more detail. As described above, when there is the recognition method instruction data 94, the process control unit 51 controls the image recognition process according to this designation. If not, the process is controlled according to a predetermined procedure. Here, the case where the pattern recognition method instruction data 94 shown in FIG. 9A is provided will be described as an example. That is, the first image recognition engine 85a to the third image recognition engine 85c are sequentially executed, and when the image ID 102 is acquired by the previously executed image recognition engine, the process ends.

16 and 17 are mainly composed of three columns of display, the processing of the processing control unit 51 in the left column, the processing of each image recognition unit in the center column, and the image recognition engine in the right column. Each process is described. First, in step S30, the process control unit 51 checks whether or not the first image recognition engine 85a can be used based on the recognition method instruction data 94. If it cannot be used (NO in step S30), the process proceeds to step S36 to check the next image recognition engine. If it is usable (YES in step S30), the process proceeds to step S31, and the process of the first image recognition unit 53a is activated.

The activated first image recognition unit 53a provides the received image data 92 to the first image recognition engine 85a in step S32. Then, the first image recognition engine 85a refers to the image identification DB 71 (more specifically, the DB 003 designated by the recognition method instruction data 94) in step S33, and the feature amount data of the image data 92 and the image identification DB 71. Are compared with the feature amount data stored in each of them, and a difference (similarity) between the feature amount data is obtained. Next, in step S34, the first image recognition engine 85a sets the first image ID 102 corresponding to the image data whose difference is equal to or smaller than the first threshold value to the first (highest similarity) one. It returns to the image recognition part 53a. If there is no value below the first threshold, 0 is returned. Thereafter, the first image recognition unit 53a passes the image ID 102 received from the first image recognition engine 85a to the process control unit 51 in step S35. Although depending on the function of the first image recognition engine 85a, the first image recognition unit 53a may be configured to determine the similarity in step S34.

Next, in step S36, the process control unit 51 checks whether or not the second image recognition engine 85b can be used based on the recognition method instruction data 94. If it cannot be used (NO in step S36), the process proceeds to step S50 in FIG. 13 to check the next image recognition engine. If it can be used (YES in step S36), the process proceeds to step S37, where it is checked whether the image ID has already been obtained in the previous image recognition process. If already obtained (NO in step S37), the image recognition process is terminated. If the image ID is not obtained (YES in step S37), the process proceeds to step S38, and the process of the second image recognition unit 53b is activated.

The activated second image recognition unit 53b provides the received image data 92 to the second image recognition engine 85b in step S39. Then, the second image recognition engine 85b refers to the image identification DB 71 (more specifically, DB 019 specified by the recognition method instruction data 94) in step S40, and the feature amount data of the image data 92 and the image identification DB 71. Are compared with the feature amount data stored in each of them, and a difference (similarity) between the feature amount data is obtained. Next, in step S41, the second image recognition engine 85b returns all the image IDs 102 corresponding to the image data whose difference is equal to or smaller than the second threshold value to the second image recognition unit 53b. If there is no value below the second threshold, 0 is returned. After that, when receiving a plurality of image IDs 102 in step S42, the second image recognition unit 53b selects a part or all of the image IDs 102 according to a predetermined rule. Next, the second image recognition unit 53b passes the selected image ID 102 to the processing control unit 51 in step S43.

Here, the second threshold value may be the same value as the first threshold value used in the first image recognition engine 85a. Naturally, as the threshold value decreases, only the image ID of image data matched with a high degree of similarity is returned. Further, although depending on the function of the second image recognition engine 85b, it is possible to configure the second image recognition engine 85b to perform the selection process in step S42, and conversely, the similarity determination in step S41 is performed. It is also possible for the second image recognition unit 53b to perform it.

Next, in step S50 shown in FIG. 17, the processing control unit 51 checks whether or not the third image recognition engine 85c can be used by the recognition method instruction data 94. If it cannot be used (NO in step S50), the image recognition process is terminated. If it can be used (YES in step S50), the process proceeds to step S51, where it is checked whether the image ID has already been obtained in the previous image recognition process. If already obtained (NO in step S51), the image recognition process is terminated. When the image ID is not obtained (YES in step S51), the process proceeds to step S52, and the process of the third image recognition unit 53c is activated.

The activated third image recognition unit 53c provides the received image data 92 to the third image recognition engine 85c in step S53. Here, the third image recognition engine 85c is an OCR engine. In step S54, the third image recognition engine 85c performs image recognition processing on the image data 92, and if there is recognizable text information, returns it to the third image recognition unit 53c as OCR text 110. Thereafter, the third image recognition unit 53c passes the OCR text 110 to the processing control unit 51 in step S55.

The recognition method instruction data 94 has various designated variations other than those illustrated in FIG. 9, and various image recognition engines exist. Here, only an example of a procedure for performing a plurality of image recognition processes is shown, and the idea of the present invention is not limited to such an example. There are a number of patterns regarding the selection and combination of image recognition engines to be used and the manner of use of each image recognition engine.

Next, with reference to FIG. 18, various cases are assumed regarding which image ID is finally selected according to the content specified by the recognition method instruction data 94 and the image ID acquisition pattern. The selection of the image ID 102 is performed by the processing control unit 51, and the image ID 102 thus selected is used for keyword search.

FIG. 18A shows a case where the first image recognition engine 85a and the second image recognition engine 85b are usable in the recognition method instruction data 94 and the image ID 102 cannot be obtained by the first image recognition engine 85a. In this pattern, the image recognition engine 85b is set to be used. Here, when there is image data in which the difference between the feature data is equal to or less than the first threshold (similarity to the image data 92), the first image recognition engine 85a has only one image ID 102 corresponding to the image data. The second image recognition engine 85b returns all image IDs corresponding to the image data when there is image data in which the difference between the feature data is equal to or less than the second threshold (similarity to the image data 92). It is a specification to return.

In FIG. 18A, the image ID 102 acquired by the first image recognition engine 85a is selected with the highest priority, and the image ID 102 acquired by the second image recognition engine 85b is the image ID corresponding to the image data having the highest similarity. Is selected. Therefore, in this case, only one image ID 102 is selected. In case A1, since the image ID “001” is acquired by the first image recognition engine 85a, this image ID is selected. In case A3, since the three image IDs “003”, “004”, and “005” have been acquired by the second image recognition engine 85b, the image ID “004” having the highest similarity is selected.

18B, the same recognition method instruction data 94 as in FIG. 18A is set. Here, the image ID 102 acquired by the first image recognition engine 85a is selected with the highest priority, and all of the image IDs 102 acquired by the second image recognition engine 85b are selected. Accordingly, since the image ID “001” is acquired by the first image recognition engine 85a in the case B1, this image ID “001” is selected. In the case B2, one image ID “002” is selected by the second image recognition engine 85b. ”Is acquired, and this image ID“ 002 ”is selected. In the case B3, the second image recognition engine 85b has acquired three image IDs“ 003 ”,“ 004 ”, and“ 005 ”. All IDs “003”, “004”, and “005” are selected.

FIG. 18C shows that in the recognition method instruction data 94, the first image recognition engine 85a and the second image recognition engine 85b can be used, and the image ID cannot be acquired by the first image recognition engine 85a. In this pattern, the second image recognition engine 85b is set to be used sequentially or simultaneously. Here, all the image IDs 102 acquired by the first image recognition engine 85a and the second image recognition engine 85b are selected. Therefore, in the case C1, the image ID “001” is acquired by the first image recognition engine 85a and the image ID “002” is acquired by the second image recognition engine 85b. Therefore, the image IDs “001” and “002” are selected. In the case C2, since the image ID “001” is acquired by the first image recognition engine 85a and the image IDs “003” and “004” are acquired by the second image recognition engine 85b, the image IDs “001” and “004” are acquired. “003” and “004” are selected.

In FIG. 18D, the same recognition method instruction data 94 as in FIG. 18C is set. Here, the image ID 102 acquired by the first image recognition engine 85a is preferentially selected, and up to two image IDs 102 acquired by the second image recognition engine 85b are selected on the basis of similarity. . Accordingly, in case D2, the image ID “001” is acquired by the first image recognition engine 85a and the image IDs “003”, “004”, and “005” are acquired by the second image recognition engine 85b. Among the image IDs “003”, “004”, and “005”, “004” and “005” having high similarity are selected. In case D3, the image ID 102 is not acquired by the first image recognition engine 85a and the image IDs “003”, “004”, and “005” are acquired by the second image recognition engine 85b. , “004” is selected.

As described above, there are many variations on the selection method of the image ID 102 acquired by the image recognition engine. In the present invention, the selection of the image ID 102 can be determined based on the acquired image recognition engine, the similarity of each image ID 102, the upper limit number of selections, and the like.

Next, the configuration of the advertisement listing DB 76 that can be used in place of the advertisement distribution system 150 to acquire the advertisement data 96 in the advertisement management server 50 according to the present invention will be described with reference to FIG. . The advertisement listing DB 76 of FIG. 19A is configured to include an advertisement service ID, advertisement text, URL, link text, e-mail address, etc. for each advertisement service.

The advertisement data search unit 57 of the advertisement management server 50 searches the advertisement text of each advertisement service stored in the advertisement listing DB 76 using the keyword acquired by the keyword search unit 56 as a search keyword, and at least a part of the advertisement text. When the search keyword matches or is similar, the advertisement service ID, advertisement text, URL, link text, and e-mail address of the advertisement service are extracted as advertisement information. Whether a part of the advertisement text and the search keyword are similar can be determined using an existing method. By setting a predetermined condition using the determination method, a set of a plurality of advertisement information can be extracted from one keyword. When there are a plurality of keywords acquired by the keyword search unit 56, the above search is repeated, each is extracted as advertisement information, and these are passed as advertisement data 96 to the advertisement data transmission unit 58.

The advertisement content stored in the advertisement listing DB 76 is basically provided by each advertiser, advertising agency, and the like. In this example, the items of the advertisement listing DB 76 are set as described above, but various other information can be stored. Further, the items to be extracted as the advertisement information need not be all items of the advertisement listing DB 76, and a part thereof can be omitted as necessary.

The advertisement text, the link text, and the URL included in the advertisement information are provided to the browsing client 20 via the front server 30. For example, the first advertisement 254, the second advertisement 255, and the third advertisement 259 in FIG. Displayed in various ways. Here, the underlined portion is the link text, and the text information without the underline is the advertisement text. When the browsing user selects the link text by using a push button or the like of the mobile phone 20a, the Web page display unit 21 is specified by a URL associated with the link text (this URL is provided as advertisement information). The web page is displayed on the display device of the browsing client 20. Advertisers can expect effective sales promotion and image improvement by making this Web page a page for advertisements, reservations / sales, explanations of products and services.

Although not shown in the example of FIG. 6, an e-mail address can also be displayed on the display device of the browsing client 20, and when this is selected by a push button or the like of the mobile phone 20a, a mail application is started. E-mail can be easily transmitted to the e-mail address.

19B is configured to include an advertisement service ID, advertisement text, search metadata, URL, link text, e-mail address, etc. for each advertisement service. It differs from the advertisement listing DB 76 of FIG. 19A in that it includes search metadata.

The advertisement data search unit 57 of the advertisement management server 50 searches the search metadata for each advertisement service stored in the advertisement listing DB 76 ′ using the keyword acquired by the keyword search unit 56 as a search keyword, and searches for the search metadata. When the search keyword matches or is similar to at least a part of the URL, the advertisement service ID, advertisement text, URL, link text, and e-mail address of the advertisement service are extracted as advertisement information. Whether one search metadata and a search keyword are similar can be determined using an existing method. Moreover, it can comprise so that the set of several advertisement information may be extracted from one keyword.

The search metadata is words or phrases extracted from advertisement texts, related web pages, and the like for use in such search by a search keyword. An advertiser, an advertising agency, or the like can also create this search metadata. In this example, when the advertisement service ID is “PR0011”, the search metadata is “sunset, sky, travel,...”, But one record is formed for each word. May be. For example, one record is created for each of the search metadata “sunset”, “sky”, and “travel”, and the same (or different) advertisement service ID is assigned to these records.

Next, an example of the hardware configuration of a computer used as the advertisement management server 50 of the advertisement management system 1 of the present invention will be described with reference to FIG. However, the computer 300 in FIG. 20 merely illustrates a typical configuration of a computer that functions as the advertisement management server 50. The personal computer 10b as the registration client 10, the personal computer 20b as the browsing client 20, and each computer of the front server 30 are also configured by the same components.

A computer 300 includes a CPU (Central Processing Unit) 301, a memory 302, an audio output device 303, a network interface 304, a display controller 305, a display 306, an input device interface 307, a keyboard 308, a mouse 309, an external storage device 310, and an external recording medium. It includes a drive 311 and a bus 312 that connects these components to each other.

The CPU 301 controls the operation of each component of the computer 300, and executes the main function of the advertisement management system 1 according to the present invention under the control of the OS. For example, in the advertisement management server 50, the processing control unit 51, the image data receiving unit 52, the image recognition processing unit 53, the target ID acquisition unit 54, the target ID transmission / reception unit 55, the keyword search unit 56, the advertisement data search unit 57, the advertisement The execution of each function of the data transmission unit 58, the log acquisition unit 59, and the network I / F unit 60 is controlled. In the front server 30, each of the web access control unit 31, the content management unit 32, the web page generation / registration unit 33, the image data registration control unit 34, the web page browsing control unit 35, and the network I / F unit 36. Control the execution of functions.

The memory 302 is usually composed of a ROM (Read Only Memory) that is a nonvolatile memory and a RAM (Random Access Memory) that is a volatile memory. The ROM stores a program executed when the computer 300 is started up. The RAM temporarily stores programs for realizing the functions of the advertisement management server 50 executed by the CPU 301 and data used during the execution of these programs.

The audio output device 303 is, for example, a device such as a speaker, and receives audio data from a video player or the like and outputs audio.

The network interface 304 is an interface for connecting to the network 320. The network 320 corresponds to, for example, the network 90 illustrated in FIG.

The display controller 305 is a dedicated controller for actually processing a drawing command issued by the CPU 301. The drawing data processed by the display controller 305 is once written in the graphic memory and then output to the display 306. The display 306 is a display device configured by, for example, an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube).

The input device interface 307 receives a signal input from the keyboard 308 or the mouse 309 and transmits a predetermined command to the CPU 301 according to the signal pattern.

The external storage device 310 is, for example, a storage device such as a hard disk drive (HDD), in which the above-described programs and data are recorded, and loaded from there to the RAM of the memory 302 as necessary at the time of execution. Is done. For example, the storage device 70 of the advertisement management server 50 corresponds to such a storage device.

The external recording medium driving device 311 accesses the recording surface of a portable external recording medium 330 such as a CD (Compact Disc), MO (Magneto-Optical Disc), DVD (Digital Versatile Disc), etc., and is recorded there. It is a device that reads data. The external recording medium 330 can also record the program of the advertisement management server 50 and the program of the front server 30 for realizing the advertisement management system 1 according to the present invention. Data recorded in the external recording medium 330 is stored in the external storage device 310 via the external recording medium driving device 311. If it is a program, it is loaded into the RAM of the memory 302 at the time of execution.

Other distribution forms of the program for realizing the advertisement management system 1 according to the present invention include a route that is stored in the external storage device 310 via a network 320 and a network interface 304 from a predetermined server on the network. Is also possible. The program stored in this manner is loaded into the RAM of the memory 302 at the time of execution and executed in the same manner as described above.

As described above, the configuration of such a computer is basically common to the advertisement management server 50, the front server 30, the personal computer 10b, and the personal computer 20b, but the server computers such as the advertisement management server 50 and the front server 30 are the same. The audio output device 303, the display controller 305, the display 306, the input device interface 307, and the keyboard 308 are not essential components.

Next, an example of the hardware configuration of the mobile phone 10a as the registration client 10 and the mobile phone 20a as the browsing client 20 that can be used in the advertisement management system 1 of the present invention will be described with reference to FIG. However, the mobile device 400 in FIG. 21 merely illustrates a typical configuration of the mobile phone 10a or the mobile phone 20a.

The portable device 400 includes a CPU (Central Processing Unit) 401, a memory 402, a camera 403, a GPS control unit 404, a GPS antenna 405, a wireless signal processing unit 406, a wireless communication antenna 407, an audio control unit 408, a microphone 409, a speaker. 410, a display control unit 411, a display 412, an input device interface 413, a push button 414, an external recording medium interface 415, and a bus 416 for connecting these components to each other.

The CPU 401 controls the operation of each component of the portable device 400 and executes each function under the control of the OS. For example, in the case of the mobile phone 10 a that functions as the registration client 10, execution of each function of the Web page display unit 11, the mail transmission unit 12, and the network I / F unit 13 is controlled. Further, in the case of the mobile phone 20 a that functions as the browsing client 20, execution of each function of the Web page display unit 21 and the network I / F unit 22 is controlled.

The memory 402 is generally composed of a ROM (Read Only Memory) which is a nonvolatile memory or a semiconductor memory called a flash memory. The ROM stores a program executed when the portable device 400 is activated. The flash memory stores a program executed by the CPU 401 for realizing each function and photographed image data 92.

The camera 403 is a small imaging device built in the portable device 400, and there is a type in which the imaging element is a CCD or a CMOS. In recent years, there are many things that can shoot not only still images but also moving images. The photographed data is stored as image data 92 in a flash memory or the like.

The GPS control unit 404 receives signals from a plurality of GPSs via the GPS antenna 405 and checks the position of the mobile device 400. The position information thus obtained is stored in a flash memory or the like and used in a program as necessary. The wireless signal processing unit 406 communicates with the base station via the wireless communication antenna 407, transmits / receives call data to / from other devices, and performs web communication with other terminals (via the Internet). Control the sending and receiving of page and email data. The portable device 400 is connected to the network 90 illustrated in FIG. 1 via the wireless signal processing unit 406 and the wireless communication antenna 407.

The audio control unit 408 controls the microphone 409 and the speaker 410 to realize a wireless communication call, and controls to receive audio data from a music player or a video player and output the audio.

The display control unit 411 processes drawing data issued by the CPU 401 and outputs it to the display 412. The display 412 is a display device configured by, for example, an LCD (Liquid Crystal Display).

The input device interface 413 receives a signal that the push button 414 of the mobile device has been pressed, and transmits a predetermined command to the CPU 301 according to the signal. The push button 414 of the portable device includes various function buttons such as a movement button in addition to a numeric button capable of inputting text such as numerals and characters.

The external recording medium interface 415 is a device that accesses the external recording medium 420 and reads data recorded therein. The external recording medium 420 is, for example, a portable flash memory.

Next, an advertisement management system according to another embodiment of the present invention will be described with reference to FIGS. FIG. 22 to FIG. 25 show the relationship between the function unit that executes the process and the data, respectively. The advertisement management system according to another embodiment has basically the same configuration as that of the first embodiment except for the points described below. First, an advertisement management system according to the second embodiment will be described with reference to FIG. The advertisement management system according to the second embodiment includes a keyword selection unit 81 that selects an optimum keyword from among a plurality of keywords obtained by the processing of the second image recognition engine 85b and the keyword search unit 56. . As shown in FIG. 22, as in the first embodiment, one or a plurality of image IDs 102b are output from the second image recognition engine 85b to which one image data 92 has been input, and the keywords that have received these image IDs 102b. The search unit 56 searches for and outputs one or more keywords 106 related to the image ID 102b using the keyword DB 75 or the like. In FIG. 22, for convenience, the keywords 106 are shown in a table format for each image ID, but this simply means that eight keywords are output. However, when the keyword selection unit 81 employs the image ID as a selection criterion, each keyword is provided to the keyword selection unit 81 in association with the corresponding image ID.

In the example of FIG. 22, the image ID 102b is acquired from the second image recognition engine 85b. However, as shown in FIG. 12, the image ID 102a is acquired from the first image recognition engine 85a, An image ID may be acquired from the image recognition engine. Further, for example, as shown in FIG. 12, the keyword search unit 56 can search and obtain a keyword using the target specification DB 73, the target ID conversion unit 77, the table specification DB 74, etc. in addition to the keyword DB 75.

Thereafter, several keywords are selected by the keyword selection unit 81 from the searched keywords 106. Normally, one or more keywords are selected, but if all the keywords do not satisfy the selection criteria, a situation may occur in which no keywords are selected. In the example of FIG. 22, eight keywords are provided to the keyword selection unit 81. In summary for each image ID, keywords “sky”, “evening”, and “cloud” are provided for image ID = 001, and keywords “blue”, “sky”, and “cloud” are provided for image ID = 002. For the image ID = 003, keywords “sky” and “sea” are provided.

Here, for example, the keyword selection unit 81 selects two keywords in descending order of frequency (appearance) across all image IDs. When such a selection criterion is adopted, the keyword with the highest frequency is “sky” (3 times), the keyword with the next highest frequency is cloud (twice), and “sky” and “cloud” are the keywords. Is selected. It is also possible to adopt a selection criterion such as selecting only one keyword with the highest frequency. If there are multiple keywords with the same frequency under such selection criteria, the number of selected keywords itself may be changed to select all of them, In order not to change the number, one of these keywords can be selected at random. In the above case where there are a plurality of keywords having the same frequency, the similarity (the degree to which the image corresponding to the image ID is similar to the original image data 92) is input together with the image ID to make the image more similar. It can also be designed to select keywords for selected images.

Further, the input keywords are grouped and aggregated according to a predetermined concept, or keywords having common characters (characters having a predetermined number of characters) are grouped and aggregated. You can also choose. Thus, the criteria for selecting a keyword can be set based on various viewpoints as well as the frequency of the keyword.

In the example of FIG. 22, the keyword “empty” is output as the selected keyword 107 by the keyword selection unit 81 as a result of adopting a selection criterion such as selecting only one keyword with the highest frequency. The keyword “empty” output in this way is sent to the advertisement data search unit 57 as in the advertisement management system of the first embodiment, where the advertisement listing DB 76 is used or advertisement distribution based on the keyword 107. Advertisement data 96 is obtained via the system 150.

Next, an advertisement management system according to the third embodiment will be described with reference to FIG. The advertisement management system according to the third embodiment converts the obtained keyword according to the media information regarding the image data 92 when the keyword is obtained by the processing of the second image recognition engine 85b and the keyword search unit 56. It is comprised so that it may do. Through a process similar to that of the second embodiment, after the keyword 106 is provided to the keyword selection unit 81, the keyword 107 selected by the keyword selection unit 81 is provided to the keyword conversion unit 82. Here, as the keyword 107, a keyword that does not pass through the processing by the keyword selection unit 81 may be provided to the keyword conversion unit 82. For example, as shown in FIG. 12, an OCR text 110 that is an OCR output as shown by the third image recognition engine 85 c can be used as the keyword 107. Here, the keyword selection process by the keyword selection unit 81 can be omitted.

Here, further, the keyword conversion unit 82 is provided with media information 108 that is media information related to the image data 92. The media information 108 indicates what kind of media the image data 92 that is the original image is related to. For example, a site where the image data 92 is uploaded (or viewed) It is information which shows. In the example of FIG. 23, it is assumed that the photo sharing site A is the media 1 and the SNS (Social Networking Service) site B is the media 2.

The media information 108 is received via, for example, the browsing client 20, the front server 30, and the image data receiving unit 52 of the advertisement management server 50, and finally provided to the keyword converting unit 82.

The keyword conversion unit 82 converts the input keyword 107 into the keyword 109 using the input media information 108 and the keyword conversion DB 83. The keyword conversion DB 83 stores association data indicating which keyword the input keyword 107 is converted into for each media information 108. For example, when the keyword conversion DB 83 has the contents as shown in FIG. 23, the keyword conversion unit 82 receives the keyword “mountain” when the keyword “mountain” and the media information 108 indicating the photo sharing site A of the medium 1 are input. And the keyword “mountain” and the media information 108 indicating the SNS site B of the media 2 are input, the keyword “climbing” is output. As can be seen from this example, the same keyword may be output without being converted as a result. However, for convenience, such processing is also referred to as “conversion” of the keyword.

The converted keyword 109 output in this way is sent to the advertisement data search unit 57 in the same manner as in the advertisement management system of the first embodiment, where the advertisement listing DB 76 is used based on the keyword 107, or the advertisement. The advertisement data 96 is acquired via the distribution system 150. In order to acquire advertisement data according to the genre and nature of the site where the image data 92 is uploaded (or browsed), the age group of the user, etc. by adopting the configuration of the advertisement management system according to the third embodiment. You can adjust and change the keywords of to more optimal ones.

Furthermore, the third embodiment shown in FIG. 23 can be modified to display the advertisement data 96 in a different manner on the web page of the browsing client 20. For example, in the first advertisement 254, the second advertisement 255, or the third advertisement 259 shown in FIG. 6, at least a part of the advertisement data 96 is directly displayed as it is. (3 or 5) titles (categories) are displayed side by side as link information, and when one of the link information is clicked, etc., it is acquired using a keyword related to the title corresponding to the link information. Advertising data is displayed.

In the modification of the third embodiment described above, a title conversion unit 84 (not shown) is added to the configuration shown in FIG. When the title conversion unit 84 receives the converted keyword 109 output from the keyword conversion unit 82, the title conversion unit 84 acquires a title corresponding to the keyword 109 using a title conversion DB (not shown). In the title conversion DB, association data indicating which title the input keyword 109 is converted into for each media information 108 is stored. In this example, the keyword 109 is associated with a predetermined title according to the media information 108. However, the keyword 109 may be associated with a predetermined title by the media information 108 and attribute information 112 described later. Here, an example in which the converted keyword 109 is input to the title conversion unit 84 in the third embodiment illustrated in FIG. 23 has been described. However, in the second embodiment illustrated in FIG. The keyword 107 can be input, and further, in the fourth embodiment described later with reference to FIG. 24, the converted keyword 111 is input, or in the fifth embodiment described later with reference to FIG. It can also be configured to input a later keyword 109.

The title acquired by the title conversion unit 84 is transmitted to the front server 30 in association with the keyword 109 used for the conversion. A plurality of such title / keyword pairs can be generated and transmitted to the front server 30. Thereafter, the Web page generation / registration unit 33 of the front server 30 generates a Web page so that the title transmitted to the front server 30 is displayed as the first advertisement 254, for example. As described above, when a plurality of sets of titles and keywords are transmitted to the front server 30, a Web page is generated so that all (or a part) of the titles are displayed. The Web page generated in this way is displayed by a browser or the like in response to a user operation of the browsing client 20. Here, the title displayed on the web page is displayed as link information, and when one of the titles is clicked, the keyword associated with the title is directly or via the front server 30. This is provided to the advertisement data search unit 57 of the advertisement management server 50.

When the keyword is provided to the advertisement data search unit 57, the advertisement data search unit 57 acquires the advertisement data 96 related to the keyword from the keyword (using the advertisement listing DB 76 and the advertisement distribution system 150). When the advertisement data 96 is acquired, the advertisement data transmission unit 58 generates a web page so that the advertisement data 96 is displayed, and transmits the web page to the browsing client 20. Upon receiving this Web page, the browsing client 20 displays the page using a browser or the like.

The advertisement data 96 displayed on the web page may include a plurality of advertisements. In that case, each advertisement is displayed as a list (list display). Link information is included. Further, it is possible to generate a web page so that a part of the advertisement data 96 is selected and displayed according to a predetermined criterion. On the other hand, the number of titles displayed on a Web page can be specified by the site of the Web page. For example, at one site, three titles are displayed in a horizontal row, and at another site, five titles are displayed.

Further, the advertisement data transmission unit 58 transmits the advertisement data 96 to the front server 30, where the Web page generation / registration unit 33 of the front server 30 generates a Web page for displaying a list of the advertisement data 96, It can also be designed to be sent to the browsing client 20.

As described above, the advertisement data 96 related to the corresponding keyword can be displayed from the title acquired by the title conversion unit 84. However, this method is merely an example, and the same effect can be obtained by various other methods. Can be played. For example, the following modifications can be considered. The title acquired by the title conversion unit 84 is transmitted to the front server 30 in association with the advertisement data 96 obtained from the keyword corresponding to the title (using the advertisement listing DB 76 and the advertisement distribution system 150). . The web page generation / registration unit 33 of the front server 30 generates a web page so that three or five titles are displayed side by side as the first advertisement 254, for example. Further, the title displayed on the Web page by the browsing client 20 is displayed as link information, and when one of the titles is clicked, the advertisement data 96 associated with the title is browsed from the front server 30. It is downloaded to the client 20 and displayed on the web page.

Next, an advertisement management system according to the fourth embodiment will be described with reference to FIG. When the keyword is obtained by the processing of the second image recognition engine 85b and the keyword search unit 56, the advertisement management system according to the fourth embodiment uses the keyword according to the media information and the attribute information regarding the image data 92. Is configured to convert. Through a process similar to that of the second embodiment, after the keyword 106 is provided to the keyword selection unit 81, the keyword 107 selected by the keyword selection unit 81 is provided to the keyword conversion unit 82A. Here, as the keyword 107, a keyword that does not pass through the processing by the keyword selection unit 81 may be provided to the keyword conversion unit 82 </ b> A. For example, as shown in FIG. 12, an OCR text 110 that is an OCR output as shown by the third image recognition engine 85 c can be used as the keyword 107. Here, the keyword selection process by the keyword selection unit 81 can be omitted.

Here, further, media information 108 and attribute information 112, which are media information related to the image data 92, are provided to the keyword conversion unit 82A. The media information 108 is the same as the media information 108 described in the third embodiment. The attribute information 112 is information indicating the attribute of the image data 92, and may include image attribute information 112A that is attribute information obtained from the image data, such as image analysis by an image recognition engine, and other attribute information 112B. Image attribute information 112A includes face / person attributes (eg, age, gender, facial expression (eg, emotion)), landscape (eg, mountain, sea, sky, cloudy, sunny, etc.), and relationships among multiple objects (eg, , Recognizing the relationship that two people are a parent, child, classmate, couple, or lover from the age, gender, posture, etc. included in the image), location information (for example, location information of the image data, building , Position information obtained from images such as intersections and road signs, etc.), similarity (for example, the degree of similarity of the image corresponding to the image ID to the original image data 92), and the like. On the other hand, as the other attribute information 112B, position information (for example, position information of a terminal of a posting or browsing user (such position information can be grasped by GPS or the like), time information (for example, photographing time of image data, Upload time, browsing time, etc.) It is also possible to manage a single attribute by combining a plurality of the above-mentioned attributes, for example, “male of teenagers” combining age and gender. , Three attributes consisting of “male in their 20s” and “male in their 30s” may be obtained from the image data.

The media information 108 is received via the image data receiving unit 52 of the browsing client 20, the front server 30, and the advertisement management server 50, for example, as in the third embodiment. The image attribute information 112A is provided from, for example, the second image recognition engine 85b that has performed image recognition processing of the image data 92, other image recognition engines, and the like, and the other attribute information 112B is the same as the media information 108. For example, it is provided via the browsing client 20, the front server 30, and the image data receiving unit 52 of the advertisement management server 50.

When the keyword 107, the media information 108, and the attribute information 112 are provided, the keyword conversion unit 82A converts the keyword 107 using the keyword conversion DB 83A. In the keyword conversion DB 83A, association data indicating which keyword the input keyword 107 is converted into for each media information 108 and attribute information 112 is stored. For example, when the keyword conversion DB 83A has the contents as shown in FIG. 24, the keyword conversion unit 82A has the keyword “mountain”, media information 108 indicating the photo sharing site A of the media 1, and attribute 1 (for example, age When the attribute information 112 indicating “low child” is input, the keyword “climbing” is output, the keyword “mountain”, the media information 108 indicating the SNS site B of the medium 2, and the attribute information indicating attribute 2 (for example, clear) When 112 is input, the keyword “outdoor” is output. Further, as a result, the same keyword may be output without being converted. However, for convenience, such processing is also referred to as keyword conversion. In addition, a plurality of attribute information 112 can be provided hierarchically in the keyword conversion DB 83A. For example, regarding the age, three attributes “10's and below”, “20's”, and “30 and above” are set, and “male” and “female” are set for each of the attributes. By configuring the keyword conversion DB 83A in this way, the input keyword 107 is converted into a predetermined keyword for each of the media information 108 and the plurality of attribute information 112.

The converted keyword 111 output in this way is sent to the advertisement data search unit 57 in the same manner as the advertisement management system of the first embodiment, where the advertisement listing DB 76 is used based on the keyword 107 or the advertisement. The advertisement data 96 is acquired via the distribution system 150. By adopting a configuration like the advertisement management system according to the fourth embodiment, the image data 92 is read not only from the genre and nature of the site where the image data 92 is uploaded (or browsed), from the user's age group, but also from the image itself. The keyword for acquiring advertisement data can be adjusted and changed to a more optimal one based on image attribute information that can be used and other attribute information related to image uploading and browsing.

Next, an advertisement management system according to the fifth embodiment will be described with reference to FIG. The advertisement management system according to the fifth embodiment converts the keyword according to the media information about the image data 92 when the keyword is obtained by the processing of the second image recognition engine 85b and the keyword search unit 56. The attribute information is used when the advertisement data is acquired. In this example, the process proceeds in the same procedure as in the third embodiment shown in FIG. 23 until the converted keyword 109 is obtained. The keyword conversion DB 83 has the same configuration. Here, the keyword selection process by the keyword selection unit 81 may be omitted.

After the keyword 109 is obtained, the converted keyword 109 and attribute information 112 are provided to the advertisement data search unit 57A. The attribute information 112 is the same as that described in the fourth embodiment shown in FIG. The advertisement data search unit 57A transmits the keyword 109 and the attribute information 112 to the advertisement listing DB 76 or the advertisement distribution system 150. Then, the advertisement listing DB 76 or the advertisement distribution system 150 returns the advertisement data 96 corresponding to the contents of the keyword 109 and the attribute information 112 to the advertisement data search unit 57. At this time, in the advertisement listing DB 76 or the advertisement distribution system 150, there is a possibility that a search key is added, and there is a case where a data structure change, data addition, a search processing logic change or the like is required.

In this example, the keyword 109 and the attribute information 112 converted according to the media information 108 are configured to be provided to the advertisement listing DB 76 or the advertisement distribution system 150. First, the keyword conversion unit 82 uses the attribute information. The keyword conversion according to 112 is performed, and then the converted keyword 109 may be provided to the advertisement listing DB 76 or the advertisement distribution system 150 together with the media information 108. Further, the keyword conversion unit 82 performs keyword conversion according to the media information 108 and the attribute information 112, and then provides the media information 108 and the attribute information 112 together with the converted keyword 109, and the advertisement listing DB 76 or the advertisement The advertisement data 96 can be acquired using the distribution system 150. In this case, the media information 108 used for keyword conversion and the media information 108 used for advertisement data acquisition need not be exactly the same information. The same applies to the attribute information 112.

Further, the attribute information 112 (image attribute information 112A or other attribute information 112B) that has not been used for the conversion is converted into a keyword converted according to either the image attribute information 112A or the other attribute information 112B among the attribute information 112. , And the media information 108 may be provided to the advertisement listing DB 76 or the advertisement distribution system 150.

When image data is received by the advertisement management system according to the present invention, related information and keywords corresponding to the image data can be effectively acquired, and thus can be applied to various uses using information related to the image data. .

Claims

In an advertisement management system including a front server and an advertisement management server connected to the front server via a network,
The front server is
Image data transmitting means for transmitting the image data to the advertisement management server when receiving image data from a registered client;
Advertisement information transmitting means for editing advertisement information based on advertisement data related to the image data received from the advertisement management server and transmitting the advertisement information to a browsing client;
The advertisement management server
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data when the image data is received from the front server;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
An advertisement management system comprising: advertisement data transmitting means for searching advertisement data based on the keyword acquired by the keyword acquiring means and transmitting the advertisement data obtained as a search result to the front server. .
The advertisement management system according to claim 1,
Furthermore, it has an identifier transmission / reception means,
The identification transmitting / receiving means includes
When the image ID is determined by the image recognition processing means, the image ID or the identifier is transmitted to the front server in order to be stored in the front server,
When the front server receives a browsing request regarding the image data from the browsing client, the image ID or the identifier stored in the front server is received from the front server, and the image ID or the identifier is An advertisement management system provided to the keyword acquisition means for acquiring a keyword.
The advertisement management system according to claim 1,
The recognition method instruction data includes information on the image recognition method,
The image management processing unit of the advertisement management server determines the image ID according to the recognition method instruction data when the recognition method instruction data is received.
The advertisement management system according to claim 1,
The advertisement acquisition instruction data includes information on how to acquire advertisement data,
When the advertisement acquisition instruction data is received, the advertisement data transmission unit of the advertisement management server searches the advertisement data according to the advertisement acquisition instruction data.
The advertisement management system according to claim 2,
The front server further includes web page generation means,
When the image ID or the identifier is transmitted from the advertisement management server to the front server, the web page generation unit stores the transmitted image ID or the identifier as data of a web page. A featured advertising management system.
The advertisement management system according to claim 1,
The keyword acquisition means acquires a keyword by searching a database using the image ID or the identifier, and the database is switched according to the image ID or the identifier. .
The advertisement management system according to claim 1,
The advertisement management system, wherein the image recognition processing means determines the image ID for each of the image data by a plurality of recognition methods or a plurality of judgment criteria.
Image data receiving means for receiving image data;
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
Advertisement data transmission means for searching advertisement data based on the keyword acquired by the keyword acquisition means and transmitting the advertisement data obtained as a search result to a predetermined computer;
The advertisement management server, wherein the image recognition processing means determines the image ID for each of the image data by a plurality of recognition methods or a plurality of judgment criteria.
The advertisement management server according to claim 8,
The recognition method instruction data includes information on the image recognition method,
The image management processing unit, when receiving the recognition method instruction data, determines the image ID according to the recognition method instruction data.
The advertisement management server according to claim 8,
The advertisement acquisition instruction data includes information on how to acquire advertisement data,
The advertisement data transmitting means, when receiving the advertisement acquisition instruction data, searches for the advertisement data according to the advertisement acquisition instruction data.
The advertisement management server according to claim 8,
The keyword acquisition means acquires a keyword by searching a database using the image ID or the identifier, and the database is switched according to the image ID or the identifier. .
An advertisement management method executed in an advertisement management system including a front server and an advertisement management server connected to the front server via a network,
In the front server, when image data is received from a registered client, an image data transmission step of transmitting the image data to the advertisement management server;
In the front server, an advertisement information transmission step of editing advertisement information based on advertisement data related to the image data received from the advertisement management server and transmitting the advertisement information to a browsing client;
In the advertisement management server, when the image data is received from the front server, an image recognition process step of determining an image ID corresponding to an image similar to the image data by performing an image recognition process;
In the advertisement management server, a keyword acquisition step of acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
The advertisement management server includes an advertisement data transmission step of searching advertisement data based on the keyword acquired in the keyword acquisition step and transmitting the advertisement data obtained as a search result to the front server. A featured advertisement management method.
An image data receiving step for receiving image data;
An image recognition processing step of performing an image recognition process to determine an image ID corresponding to an image similar to the image data;
A keyword acquisition step of acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
An advertisement data transmission step of searching advertisement data based on the keyword acquired by the keyword acquisition step and transmitting the advertisement data obtained as a search result to a predetermined computer;
The advertisement management method executed by an advertisement management server, wherein the image recognition processing step determines the image ID for each of the image data by a plurality of recognition methods or a plurality of judgment criteria.
A program for realizing an advertisement management system including a front server and an advertisement management server connected to the front server via a network,
In the front server,
Image data transmitting means for transmitting the image data to the advertisement management server when receiving image data from a registered client; and
Editing advertisement information based on advertisement data related to the image data received from the advertisement management server, and functioning as advertisement information transmitting means for transmitting the advertisement information to a browsing client;
In the advertisement management server,
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data when the image data is received from the front server;
A keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID; and
A program that searches for advertisement data based on the keyword acquired by the keyword acquisition unit and functions as an advertisement data transmission unit that transmits the advertisement data obtained as a search result to the front server.
A program for realizing an advertisement management server,
In the ad management server,
Image data receiving means for receiving image data;
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data;
A keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID; and
Search advertisement data based on the keyword acquired by the keyword acquisition means, and function as advertisement data transmission means for transmitting the advertisement data obtained as a search result to a predetermined computer,
A program causing the image recognition processing unit to function to determine the image ID for each of the image data by a plurality of recognition methods or a plurality of determination criteria.
In response to a request from a browsing user, a browsing request transmission unit that transmits a browsing request for a web page designated by the browsing user to a front server;
Message receiving means for receiving a response message to the browsing request from the front server;
When the Web page includes image data, advertisement information displayed on the Web page related to the image data is searched based on a database keyword selected according to the type of the image data. A browsing client characterized by being a thing.
In an advertisement management system including a front server and an advertisement management server connected to the front server via a network,
The front server is
Image data transmitting means for transmitting the image data to the advertisement management server when receiving image data from a registered client;
Advertisement information transmitting means for editing advertisement information based on advertisement data related to the image data received from the advertisement management server and transmitting the advertisement information to a browsing client;
The advertisement management server
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data when the image data is received from the front server;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
Keyword selection means for selecting at least one keyword from the keywords acquired by the keyword acquisition means according to a predetermined selection criterion;
An advertisement management system comprising: advertisement data transmitting means for searching for advertisement data based on the keyword obtained by the keyword selecting means and transmitting the advertisement data obtained as a search result to the front server. .
Image data receiving means for receiving image data;
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
Keyword selection means for selecting at least one keyword from the keywords acquired by the keyword acquisition means according to a predetermined selection criterion;
Advertisement data transmission means for searching advertisement data based on the keyword obtained by the keyword selection means and transmitting the advertisement data obtained as a search result to a predetermined computer;
The advertisement management server, wherein the image recognition processing means determines the image ID for each of the image data by a plurality of recognition methods or a plurality of judgment criteria.
In an advertisement management system including a front server and an advertisement management server connected to the front server via a network,
The front server is
Image data transmitting means for transmitting the image data to the advertisement management server when receiving image data from a registered client;
Advertisement information transmitting means for editing advertisement information based on advertisement data related to the image data received from the advertisement management server and transmitting the advertisement information to a browsing client;
The advertisement management server
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data when the image data is received from the front server;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
A keyword conversion means for converting the keyword acquired by the keyword acquisition means based on at least one of media information representing a medium related to the image data and attribute information representing an attribute of the image data;
An advertisement management system comprising: advertisement data transmission means for searching for advertisement data based on the keyword obtained by the keyword conversion means and transmitting the advertisement data obtained as a search result to the front server. .
Image data receiving means for receiving image data;
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
A keyword conversion means for converting the keyword acquired by the keyword acquisition means based on at least one of media information representing a medium related to the image data and attribute information representing an attribute of the image data;
Advertisement data transmission means for searching advertisement data based on the keyword obtained by the keyword conversion means and transmitting the advertisement data obtained as a search result to a predetermined computer;
The advertisement management server, wherein the image recognition processing means determines the image ID for each of the image data by a plurality of recognition methods or a plurality of judgment criteria.
In an advertisement management system including a front server and an advertisement management server connected to the front server via a network,
The front server is
Image data transmitting means for transmitting the image data to the advertisement management server when receiving image data from a registered client;
Advertisement information transmitting means for editing advertisement information based on advertisement data related to the image data received from the advertisement management server and transmitting the advertisement information to a browsing client;
The advertisement management server
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data when the image data is received from the front server;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
A keyword conversion means for converting the keyword acquired by the keyword acquisition means based on at least one of media information representing a medium related to the image data and attribute information representing an attribute of the image data;
Advertisement data for searching advertisement data based on the keyword obtained by the keyword conversion means and at least one of the media information and the attribute information, and transmitting the advertisement data obtained as a search result to the front server And an advertisement management system.
Image data receiving means for receiving image data;
Image recognition processing means for performing image recognition processing to determine an image ID corresponding to an image similar to the image data;
Keyword acquisition means for acquiring a keyword related to the image data using the image ID or an identifier associated with the image ID;
A keyword conversion means for converting the keyword acquired by the keyword acquisition means based on at least one of media information representing a medium related to the image data and attribute information representing an attribute of the image data;
Advertisement data that searches advertisement data based on the keyword obtained by the keyword conversion means and at least one of the media information and the attribute information, and transmits the advertisement data obtained as a search result to a predetermined computer Transmission means,
The advertisement management server, wherein the image recognition processing means determines the image ID for each of the image data by a plurality of recognition methods or a plurality of judgment criteria.