WO2016125307A1 - Information delivery device and information delivery program - Google Patents

Abstract

Provided are an information delivery device 3 and an information delivery program capable of providing advertisements by utilizing image categorization information. The information delivery device 3 has: a storage unit 32 which stores, in association with user identification information, categories to which image data belong; a categorization part 331 which, upon acquiring the image data and the user identification information from a user terminal, determines the categories by means of image recognition processing of the acquired image data and registers the determined categories in the storage unit in association with the user identification information; an analysis part 332 which analyzes user preferences on the basis of the categories associated with the user identification information; and an information provision part 333 which provides information to the user on the basis of the analyzed user preferences.

Description

Information distribution apparatus and information distribution program

The present invention relates to an information distribution apparatus and an information distribution program.

In recent years, not only camera devices such as digital cameras but also various devices such as mobile phones, smartphones and tablets have been equipped with camera functions. The captured image data is stored in the storage unit of those terminals, and the stored image data is displayed in a list, for example, arranged in time series at the time of imaging.

A technique is known in which subjects are classified into predetermined categories by so-called image recognition processing that analyzes predetermined feature amounts related to image data, and image data is organized and displayed for each category to which the subjects belong (for example, patents). Reference 1).

JP 2007-133746 A

It is useful for a user who browses to display a categorized image that has been taken and stored, such as making it easier to find a desired image. Also, no method has been proposed for effectively utilizing information on how images collected and stored by a specific user have been categorized for purposes other than image categorization.

The purpose of the information distribution apparatus and the information distribution program described in the present application is to provide information to a user using image categorization information.

The information distribution device stores a category to which image data belongs in association with user identification information, acquires image data and user identification information from a user terminal, and performs image recognition processing on the acquired image data to perform category recognition. A categorizing unit that associates the determined category with the user identification information and registers the category in the storage unit, an analysis unit that analyzes user preferences based on the category associated with the user identification information, and the analyzed user And an information providing unit for providing information to the user based on the preference.

The information distribution program is an information distribution program for a computer having a storage unit that stores a category to which image data belongs in association with user identification information, and acquires image data and user identification information from a user terminal. Image recognition processing is performed on the data to determine the category to which the image data belongs, the determined category is associated with the user identification information and registered in the storage unit, and the user preference is based on the category associated with the user identification information. And the computer is caused to provide information to the user based on the analyzed user preference.

According to the above information distribution apparatus and information distribution program, it is possible to effectively provide information with strong user preferences.

The objects and advantages of the invention will be realized and obtained by means of the elements and combinations particularly pointed out in the appended claims. Both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

It is a figure which shows an example of schematic structure of the information delivery system. 2 is a diagram illustrating an example of a schematic configuration of a mobile terminal 2. FIG. It is a figure which shows an example of the data structure regarding a specific image. It is a figure which shows an example of the hierarchical structure of the category at the time of categorizing image data. 2 is a diagram illustrating an example of a schematic configuration of a server 3. FIG. It is a figure which shows an example of the data structure regarding a specific user. It is a figure which shows an example of the operation | movement sequence of an image classification. It is a figure which shows an example of the login screen 1100 demonstrated by step S100 of FIG. It is a figure which shows an example of the image browsing screen 1200 demonstrated by step S104 of FIG. It is a figure which shows an example of the selection screen 1300 of the image classification request | requirement demonstrated by step S105 of FIG. It is a figure which shows an example of the image classification screen 1400 demonstrated by step S111 of FIG. It is a figure which shows an example of the operation | movement sequence of advertisement provision. FIG. 10 is a diagram illustrating an example of the advertisement described in step S <b> 204 in FIG. 9. It is a figure which shows the other example of the advertisement demonstrated by step S204 of FIG. It is a figure which shows the further another example of the advertisement demonstrated by step S204 of FIG. It is a figure which shows an example of the data structure regarding the specific image used by 2nd Embodiment. It is a figure which shows an example of the data structure regarding the specific user used by 2nd Embodiment. It is a figure which shows an example of a shop information table. It is a figure which shows an example of a menu information table. It is a figure which shows an example of the operation | movement sequence of detailed information registration. It is a figure which shows an example of the flow of the extraction process of shop name relevant information. It is an example of the detailed information registration screen 3100 demonstrated by step S309 of FIG. It is an example of the detailed information registration screen 3200 demonstrated by step S315 of FIG. It is an example of the detailed information screen 3300 demonstrated by step S318 of FIG. It is a figure which shows an example of the display of an advertisement. It is a figure which shows an example of the display of an advertisement.

[First Embodiment]
FIG. 1 is a diagram illustrating an example of a schematic configuration of an information distribution system 1.

The information distribution system 1 includes at least one portable terminal 2 and a server 3. The portable terminal 2 and the server 3 are connected to each other via a communication network, and are connected to each other via, for example, a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7. The program executed on the mobile terminal 2 and the program executed on the server 3 communicate with each other using a communication protocol such as a hypertext transfer protocol (Hypertext Transfer Protocol, HTTP).

FIG. 2 is a diagram illustrating an example of a schematic configuration of the mobile terminal 2.

The portable terminal 2 includes a terminal communication unit 21, a terminal storage unit 22, an operation unit 23, a display unit 24, an imaging unit 25, a GPS sensor 26, a terminal processing unit 27, and the like. As the mobile terminal 2, a multi-function mobile phone (so-called “smart phone”) is assumed, but the mobile terminal 2 is not limited to this. The mobile terminal 2 is not limited as long as the present invention can be applied. For example, a personal computer (PC), a mobile phone (so-called “feature phone”), a personal digital assistant (Personal Digital Assistant, PDA), a mobile game machine, and mobile music. A player, a tablet PC, or the like may be used.

The terminal communication unit 21 includes a communication interface circuit including an antenna whose sensitivity band is a predetermined frequency band, and connects the mobile terminal 2 to the wireless communication network. The terminal communication unit 21 establishes a wireless signal line using the WCDMA (Wideband Code Multiple Access) method or the like with the base station 4 via the channel assigned by the base station 4. Communicate between the two. The terminal communication unit 21 outputs the data supplied from the terminal processing unit 27 to the server 3 or the like. Further, the terminal communication unit 21 supplies data acquired from the server 3 or the like to the terminal processing unit 27.

The terminal storage unit 22 includes, for example, at least one of a semiconductor memory, a magnetic disk device, and an optical disk device. The terminal storage unit 22 stores an operating system program, a driver program, an application program, data, and the like used for processing in the terminal processing unit 27. For example, the terminal storage unit 22 stores an input device driver program for controlling the operation unit 23, an output device driver program for controlling the display unit 24, and the like as driver programs. The terminal storage unit 22 stores a program for acquiring and displaying image data as an application program. Further, the terminal storage unit 22 stores image data and bibliographic information accompanying the image data. Further, the terminal storage unit 22 may temporarily store temporary data related to a predetermined process.

The operation unit 23 may be any device as long as the operation of the mobile terminal 2 is possible, for example, a touch pad, a keyboard, or the like. The user can input characters, numbers, and the like using the operation unit 23. When operated by the user, the operation unit 23 generates a signal corresponding to the operation. The generated signal is supplied to the terminal processing unit 27 as a user instruction.

The display unit 24 may be any device that can display video, images, and the like, and is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The display unit 24 displays an image or the like corresponding to the image data supplied from the terminal processing unit 27.

The imaging unit 25 may be any device capable of imaging and reading a QR code (registered trademark), and includes, for example, a lens and an imaging element.

The GPS sensor 26 is an example of a position sensor, is connected to a communication satellite, and acquires position information (GPS (Global Positioning System) information) including latitude information and longitude information. Note that the mobile terminal 2 does not necessarily include the GPS sensor 26. Further, the position information may be position information other than GPS information. For example, the position information may be position information specified based on the intensity of radio waves received by the position sensor of the mobile terminal 2 from a plurality of base stations. Alternatively, the position information may be position information acquired by the mobile terminal 2 by iBeacon (registered trademark).

The terminal processing unit 27 includes one or a plurality of processors and their peripheral circuits. The terminal processing unit 27 controls the overall operation of the mobile terminal 2 and is, for example, a CPU (Central Processing Unit). The terminal processing unit 27 includes a terminal communication unit 21, a display unit 24, an imaging unit 25, and a program so that the program stored in the terminal storage unit 22 is executed in an appropriate procedure according to the operation of the operation unit 23. The operation of the GPS sensor 26 and the like is controlled. The terminal processing unit 27 executes processing based on programs (operating system program, driver program, application program, etc.) stored in the terminal storage unit 22. The terminal processing unit 27 can execute a plurality of programs (such as application programs) in parallel.

The terminal processing unit 27 includes an image processing unit 271, an image browsing unit 272, an advertisement display unit 273, and a QR code processing unit 274. Each of these units is a functional module realized by a program executed by a processor included in the terminal processing unit 27. Or these each part may be mounted in the portable terminal 2 as firmware.

The image processing unit 271 generates image data and bibliographic information related to the image data based on the imaging by the imaging unit 25. The bibliographic information includes user ID, image type, imaging date and time, and the like. The bibliographic information may include other information such as Exif (Exchangeable image file format) information. Image data and bibliographic information are stored in the terminal storage unit 22.

The image browsing unit 272 displays an image browsing screen on the display unit 24 based on the program stored in the terminal storage unit 22. The image browsing screen can be a screen of various forms such as a list display screen of thumbnail images based on image data and a display screen of specific image data alone.

When the advertisement display unit 273 acquires the advertisement data from the server 3, the advertisement display unit 273 displays the advertisement on the display unit 24.

The QR code processing unit 274 is an example of a code information processing unit, and associates information related to the QR code read by the imaging unit 25 with image data stored in the terminal storage unit 22 by a predetermined method. In this embodiment, the case where the code information is a QR code will be described. However, the code information may be other code information such as a two-dimensional barcode or a one-dimensional barcode other than the QR code.

FIG. 3 is a diagram illustrating an example of a data structure related to a specific image.

The data structure shown in FIG. 3 shows a series of data linked to the image ID stored in the terminal storage unit 22. The series of data includes, for example, a user ID, an image type, an imaging date and time, a category to which the image belongs, and image data. The series of data described above is an example, and in addition, Exif information, metadata related to other images, and the like may be included as data related to images.

Image types include “imaging”, “download”, “capture”, etc., but are not limited to these. “Imaging” indicates image data captured by the imaging unit 25 of the mobile terminal 2. “Download” indicates that the image data is downloaded from the network to which the mobile terminal 2 is connected. “Capture” indicates image data generated by capturing an arbitrary display on the display unit 24 of the mobile terminal 2.

FIG. 4 is a diagram illustrating an example of a hierarchical structure of categories when image data is categorized.

As shown in FIG. 4, a plurality of categories form one tree structure. “Image” has, for example, subcategories of “person”, “meal”, “landscape”, and “other”.

“Person” has, for example, “male” and “female” subcategories, and “male” and “female” are, for example, “under 10 years old”, “teens”, “20s”, and “30”, respectively. It has a subcategory of age, such as “Yen”. “Meal” has, for example, subcategories of “Japanese food”, “Western food”, and “Chinese food”. “Japanese food” has, for example, lower categories such as “sushi”, “soba”, and “yakitori”. “Western food” has, for example, subcategories such as “pasta”, “salad”, and “pizza”. “Chinese Chinese” has, for example, subcategories such as “ramen”, “gyoza”, and “spring roll”. “Scenery” has, for example, subordinate categories of “mountain”, “river”, and “sea”. The “facility” has, for example, subcategories of “tower”, “shrines and churches”, and “theme park”.

The above-described categorization method is an example, and categorization can be performed by any other method. For example, categories such as “Italian”, “French”, and “Spanish cuisine” may be included in the lower category of “Western food”. Further, the category may form a structure other than a tree structure.

FIG. 5 is a diagram illustrating an example of a schematic configuration of the server 3.

The server 3 includes a server communication unit 31, a server storage unit 32, and a server processing unit 33. The server 3 categorizes the image data in response to a request from the mobile terminal 2 and outputs the categorized image data to the mobile terminal 2.

The server communication unit 31 includes a communication interface circuit for transmitting and receiving data via the Internet 7 and communicates with the mobile terminal 2.

The server storage unit 32 includes, for example, at least one of a magnetic tape device, a magnetic disk device, and an optical disk device, and an operating system program, a driver program, an application program used for processing in the server processing unit 33, Store data etc. Further, the server storage unit 32 stores advertisement data as data. The advertisement data includes advertisement data displayed on the display unit 24 of the mobile terminal 2 and a URL related to a web page related to the advertisement object. The server storage unit 32 further includes a buffer for temporarily storing temporary data related to a predetermined process.

The server processing unit 33 includes one or a plurality of processors and their peripheral circuits, and controls the overall operation of the server 3 in an integrated manner. For example, the server processing unit 33 includes a CPU. In addition, the server processing unit 33 controls the operation of the server communication unit 31 and the like by an appropriate procedure based on programs (operating system program, driver program, application program, etc.) stored in the server storage unit 32. Furthermore, the server processing unit 33 may execute a plurality of programs (such as application programs) in parallel.

The server processing unit 33 includes a categorizing unit 331, a user preference analyzing unit 332, an advertisement providing unit 333, a related information extracting unit 334, and the like. These are functional modules realized by a program executed by a processor included in the server processing unit 33. These may be installed in the server 3 as firmware.

The categorizing unit 331 obtains an image classification request from the portable terminal 2 via the server communication unit 31, and uses one of the categories to which each image data belongs, for example, the lowest category shown in FIG. Determine one category.

As a known image recognition processing method, a method of categorizing an image using a boundary line described in Japanese Patent Application Laid-Open No. 2007-133746 can be used. In this method, the image is divided into a plurality of regions, the boundary lines of each region are extracted, the figure composed of the extracted boundary lines is compared with the figure registered in advance for each category, and both are approximated. The category is determined based on whether or not For example, when a figure composed of boundary lines extracted from an image approximates a figure (for example, a triangle figure indicating a mountain) registered in advance for the category “landscape”, the image data Are classified into the “landscape” category.

In the above method, the category can be determined using the color. In that case, color information of each part of the image is extracted, the extracted color is compared with a color registered in advance for each category, and the category is determined based on whether or not the two are approximate. For example, when the color extracted from the image approximates a color registered in advance for the category of “person” (for example, yellow indicating skin color), the image data is classified into the category of “person”. Classify into:

Also, as a known image recognition processing method, a method of categorizing images using the face recognition processing described in JP-T-2014-517371 can be used. This method is based on geometric analysis (examine prominent features), optical analysis (statistical approach that removes unnecessary elements from images and compares them with templates to eliminate inconsistencies), etc. Determine the category. By the face recognition process, the sex of the subject and the age such as “under 10 years old”, “10s”, “20s”, etc. can be determined.

The user preference analysis unit 332 analyzes the user preference based on the number of images by category and the number of browsing advertisements by category, which will be described later. Here, the user preference is a concept representing the user's hobbies, preferences, interests, etc. with respect to things.

The user preference analysis method, for example, supposes that the category with the largest number is the user preference by adding the number of images by category and the number of advertisements browsed by category for each category. For example, in FIG. 6, the category of “pasta” has 28 category-specific images and 10 category-specific advertisement views, and the total number of these is 38. As a result of performing this operation for all categories, for example, if the number 38 relating to the “pasta” category is maximum, the user preference is regarded as “pasta”. The above analysis method is an example, and the degree of weighting in the user preference analysis may be changed depending on the number of images by category and the number of advertisements browsed by category, or other analysis methods may be used.

The advertisement providing unit 333 is an example of an information providing unit. Based on the user preference analyzed by the user preference analyzing unit 332, the advertisement providing unit 333 obtains related advertisement data from the advertisement data stored in the server storage unit 32. Extract.

The related information extraction unit 334 extracts related information related to the image data based on the acquired GPS information or QR code information. Related information will be described later.

FIG. 6 is a diagram illustrating an example of a data structure related to a specific user.

The data structure shown in FIG. 6 indicates a series of data associated with the user ID stored in the server storage unit 32. The series of data includes, for example, member type, name, sex, date of birth, image information, number of images by category, number of browsing advertisements by category, and the like. The series of data described above is an example, and may include information regarding other users.

The membership type indicates the type of the user as a member. For example, “paid”, “free”, and “premium” (indicating a member who pays an additional membership fee in addition to a regular fee-based membership fee) Etc.

The image information includes an image ID associated with the user ID and a subject category related to the image ID. Therefore, the server storage unit 32 stores the category to which the image belongs in association with the user identification information.

The number of images by category represents the number of image data belonging to each category. For example, the display of “pasta (28)” indicates that there are 28 images belonging to the “pasta” category among images collected by the user. Further, for example, the display of “female in twenties (15)” indicates that, among the images taken by the user, images belonging to the category of “20s”, which is a subcategory of “female”, is a person in the image. It shows that there are 15 pieces.

The number of advertisements browsed by category indicates the number of times the user has selected display of a screen guided by advertisement for each category. For example, the display of “pasta (10)” indicates that the user has selected an advertisement belonging to the category “pasta” and browsed a screen guided by the advertisement 10 times.

FIG. 7 is a diagram showing an example of an operation sequence for image classification.

The operation sequence described below is based on a program stored in the terminal storage unit 22 and the server storage unit 32 in advance, and each element of the mobile terminal 2 and the server 3 is mainly performed by the terminal processing unit 27 and the server processing unit 33. It is executed in cooperation with. Note that the mobile terminal 2 may be automatically set to an environment in which the following operations can be executed by launching a specific application.

First, the terminal processing unit 27 of the mobile terminal 2 displays a login screen on the display unit 24 (step S100). Next, when the user inputs the user ID and password on the login screen by the operation of the operation unit 23 and selects execution of login, the terminal processing unit 27 logs in including the user ID and password via the terminal communication unit 21. The request is output to the server 3 (step S101). When the login request is acquired via the server communication unit 31, the server processing unit 33 of the server 3 performs user authentication based on the acquired user ID and password (step S102). When the user authentication is permitted, the server processing unit 33 outputs an authentication permission notification to the mobile terminal 2 (step S103).

When the authentication permission notification is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 displays an image browsing screen on the display unit 24 based on the image data stored in the terminal storage unit 22 (step). S104). Next, when the user operates the operation unit 23 to select an image classification on the image browsing screen, the terminal processing unit 27 sends the image data and the image classification request stored in the terminal storage unit 22 to the terminal communication unit. 21 to the server 3 (step S105). Note that the image data output to the server 3 in step S105 may be one image data or a plurality of image data among a set of image data stored in the terminal storage unit 22 by user selection or the like. It may be.

When the image classification request is acquired via the server communication unit 31, the categorizing unit 331 processes each acquired image data using the above-described image recognition processing method, and determines a category to which each image data belongs (step S106). . Next, the categorizing unit 331 updates the image information associated with the user stored in the server storage unit 32, and registers a category for each image ID (step S107). Next, the categorizing unit 331 aggregates the number of image data for each category, and updates the number of images by category stored in the server storage unit 32 (step S108). Next, the server processing unit 33 outputs category data related to each image data to the mobile terminal 2 via the server communication unit 31 (step S109).

When the category data related to each image data is acquired via the terminal communication unit 21, the terminal processing unit 27 of the portable terminal 2 registers the category of each image data stored in the terminal storage unit 22 (step S110). Next, the terminal processing unit 27 displays an image classification screen for displaying image data for each category on the display unit 24 based on the image data and the category stored in the terminal storage unit 22 (step S111), and a series of steps. End the operation sequence.

FIG. 8A is a diagram showing an example of the login screen 1100 described in step S100 of FIG.

In the center of the login screen 1100, a user ID input unit 1101, a password input unit 1102, and a login determination icon 1103 are arranged. When the login determination icon 1103 is selected, the terminal processing unit 27 of the mobile terminal 2 outputs a login request including the input user ID and password to the server 3.

FIG. 8B is a diagram showing an example of the image browsing screen 1200 described in step S104 of FIG.

In the image browsing screen 1200 shown in FIG. 8B, images 1201 are arranged in 3 rows × 3 columns. The image 1201 is generated by capturing an image captured by the user with the imaging unit 25 of the mobile terminal 2, an image downloaded from a network to which the mobile terminal 2 is connected, or an arbitrary display on the display unit 24 of the mobile terminal 2. It is an image to be. The images 1201 are arranged in the order of 3 rows × 3 columns from the top to the bottom and from the left to the right in order from the slowest time of image capture, download, or capture. The image browsing screen 1200 is an example, and the display method of the image 1201 is not limited to this display method.

FIG. 8C is a diagram showing an example of the image classification request selection screen 1300 described in step S105 of FIG.

When the user performs a predetermined operation using the operation unit 23, the image browsing unit 272 of the mobile terminal 2 displays a pop-up 1301 "Do you want to organize images in the camera roll?" If “Yes” 1302 is selected, the terminal processing unit 27 outputs the one or more image data selected by the user and the image classification request to the server 3 via the terminal communication unit 21. If “No” 1303 is selected, the image classification process is terminated.

FIG. 8D is a diagram showing an example of the image classification screen 1400 described in step S111 of FIG.

After acquiring the category data from the server 3 via the terminal communication unit 21 and registering the category, the terminal processing unit 27 displays the image classification screen 1400. In the image classification screen 1400, for example, according to the tree structure of FIG. 4, a category name 1401 and an image 1402 classified into the category are displayed, for example, in chronological order. FIG. 8D shows an image classification screen 1400 related to the “pasta” category as an example. In the category name 1401, notations “cooking” and “western food”, which are upper categories of “pasta”, are displayed together with the number of images included in the category. For example, the display of “Cooking (45)> Western food (25)> Pasta (10)” shown in FIG. 8D has 45 images of “Cooking”, of which 25 images are “Western food”. Furthermore, it shows that there are 10 “pasta” images.

FIG. 9 is a diagram showing an example of an operation sequence for providing advertisements.

The operation sequence described below is based on a program stored in the terminal storage unit 22 and the server storage unit 32 in advance, and each element of the mobile terminal 2 and the server 3 is mainly performed by the terminal processing unit 27 and the server processing unit 33. It is executed in cooperation with.

As an example, the following processing starts when the server processing unit 33 of the server 3 outputs the category data relating to each image data to the mobile terminal 2 via the server communication unit 31 as described above (step S109). Is done. However, the process is not limited to this, and the process may be started at an arbitrary timing.

The user preference analysis unit 332 of the server 3 analyzes the user preference by the analysis method described above based on the number of images by category and the number of advertisements browsed by category (step S201).

Next, the advertisement providing unit 333 extracts, from the advertisement data stored in the server storage unit 32, data related to the advertisement related to the analyzed user preference based on the analyzed user preference. (Step S202). Next, the advertisement providing unit 333 outputs the extracted advertisement data to the mobile terminal 2 (step S203). The advertisement providing unit 333 increases the frequency of providing advertisements, increases the display area of advertisements, or the size of displayed characters as the number of image data related to categories (number of images by category) increases. Larger advertising data may be distributed.

When the terminal processing unit 27 of the mobile terminal 2 acquires the advertising data via the terminal communication unit 21, the terminal processing unit 27 displays the advertising based on the advertising data on the display unit 24 (step S204). Thereby, the advertisement with strong user's preference can be effectively provided. Next, when the user selects to view the advertisement by an operation such as pressing an area where the advertisement is displayed, the terminal processing unit 27 starts up the browser and indicates the URL related to the selected advertisement. A page browsing request is output to an external server (not shown) (step S205).

Next, the terminal processing unit 27 outputs identification information related to the selected advertisement to the server 3 (step S206). Next, the user preference analysis unit 332 updates the number of classified advertisement browsing stored in the server storage unit 32 based on the identification information related to the advertisement acquired from the mobile terminal 2 via the server communication unit 31. (Step S207) The operation sequence for providing advertisements is terminated.

FIG. 10A is a diagram illustrating an example of the advertisement described in step S204 of FIG.

The advertisement 2101 is displayed at the center of the standby screen 2100 displayed on the display unit 24. In the advertisement 2101, an explanation of a new menu provided by the store is displayed in text. Note that the display area of the advertisement 2101 may increase as the number of image data items related to the category (the number of images by category) increases. The menu provided by the store is an example of a product provided by the store. Therefore, the description of the new product provided by the store may be displayed as text.

FIG. 10B is a diagram showing another example of the advertisement described in step S204 of FIG.

The advertisement 2201 is displayed at the bottom of the image classification screen 2200 displayed on the display unit 24. In the advertisement 2201, the contents of the new menu provided by the store are displayed in text. Note that the display area of the advertisement 2201 may increase as the number of image data items related to the category (the number of images by category) increases.

FIG. 10C is a diagram showing still another example of the advertisement described in step S204 of FIG.

The advertisement 2301 is displayed along with the image 2302 on the image classification screen 2300 displayed on the display unit 24. In the advertisement 2301, the name of the new menu provided by the store is displayed together with the characters “PR” in the upper left. Note that the size of characters displayed in the advertisement 2301 may increase as the number of image data related to the category (the number of images by category) increases.

In addition, the advertisements 2101, 2201, and 2301 are examples, and the display mode of the advertisements is not limited to the above. For example, the advertisement may be a display including various contents such as an image and a moving image in addition to text information. Further, the advertisement is not limited to the one provided in the restaurant according to the analyzed user preference, and may be an advertisement provided by any store, business type, organization, or the like. For example, advertisements related to baby goods and children's clothing will be provided to users with many infant images, and advertisements related to outdoor goods will be provided to users with many images such as mountains and the sea. Will be done. In addition, for example, for a user with many images of women in their 20s, fashion or health advertising related to women in their 20s may be provided. Or, for users with many images such as towers, shrines and temples, or theme parks, advertisements and the like regarding sightseeing spots will be provided.

[Second Embodiment]
In the second embodiment, detailed information can be registered for each image data. Here, the detailed information refers to detailed information that more specifically develops the category of image data. For example, for the “meal” category, “store name”, “menu name”, and the like can be used as detailed information. Detailed information can be arbitrarily set according to the category of image data.

Further, in the second embodiment, related information related to specific image data can be extracted based on GPS information or QR code information as candidates for detailed information. Here, the related information is information that is a candidate for detailed information, and includes a set of facility information related to facilities having a predetermined relationship with bibliographic information such as GPS information or QR code information (including a case where there is one information). ). For example, a set of store names of one or a plurality of stores within a certain distance range from GPS information related to specific image data is related information related to the image data. Alternatively, the store name related to the QR code read within a predetermined time before the imaging date and time of specific image data is related information related to the image data. Here, “store name” and “menu name” are examples of facility information, and in the second embodiment, a case where the facility information is “store name” and “menu name” will be described. The “store name” and “menu name” may be other facility information, for example, “place name”, “product name”, “tourist place name”, “explanation regarding tourist spot”, and the like. In the second embodiment, related information is extracted using either GPS information or QR code information. However, related information may be extracted using both GPS information and QR code.

In the second embodiment, the portable terminal 2 and the server 3 described in the first embodiment are used as they are.

FIG. 11 is a diagram illustrating an example of a data structure related to a specific image used in the second embodiment.

The data structure shown in FIG. 11 shows a series of data linked to the image ID stored in the terminal storage unit 22. Compared with the data structure shown in FIG. 3, the data structure shown in FIG. 11 further includes GPS information, QR code information, a store name, a menu name, and the like.

The terminal processing unit 27 acquires the GPS information acquired by the GPS sensor 26 when image capturing is performed by the image capturing unit 25, and stores the GPS information in the terminal storage unit 22 in association with the image data.

When the QR code is read by the imaging unit 25, the QR code processing unit 274 stores information related to the QR code (for example, identification information of the QR code) and the reading date and time in the buffer of the terminal storage unit 22. The buffer stores only information related to the QR code with the latest reading date and time and the reading date and time. Further, when the image data is generated by the image processing unit 271, the QR code processing unit 274 stores the information related to the QR code and the reading date / time in the terminal storage unit 22 in association with the image data. Therefore, all the image data stored in the terminal storage unit 22 is associated with information on the latest QR code among QR codes read before the imaging date and time.

The store name is an example of the detailed information described above, and is, for example, the name of the store where the object of the image data is provided. The menu name is an example of the detailed information described above, and is, for example, the menu name of the object.

FIG. 12 is a diagram illustrating an example of a data structure related to a specific user used in the second embodiment.

12 shows a series of data associated with the user ID stored in the server storage unit 32. The data structure shown in FIG. Compared with the data structure shown in FIG. 6, the data structure shown in FIG. 12 further includes a store ID and a menu ID associated with the image ID, and includes the number of images by price range. The number of images by price range represents the number of image data belonging to each price range. For example, “1,000 yen (25)” is displayed when images belonging to a price range of 1,000 yen (1,000 to 1,999 yen) among images collected by the user are displayed. It shows that there are 25.

The number of images by price range may be the basis for user preference analysis. The number of images by price range represents how much the user pays for, for example, the food of a restaurant, and the user's hobbies, preferences, interests, etc. are the number of images by category and the number of browsing advertisements by category. It can be said that it represents from a different point of view. Other arbitrary parameters may be set as the basis of user preference analysis, such as the number of images by price range. Based on arbitrary items included in a store information table and menu information table, which will be described later, and other tables, parameters serving as a basis for user preference analysis can be set.

FIG. 13A is a diagram illustrating an example of a store information table.

The store information table is stored in the server storage unit 32 of the server 3. In the store information table, information such as store name, address, QR code ID (QR code identification information), menu information (menu ID) and the like is stored in association with the store ID. The QR code is installed in, for example, a store. The QR code may be unique to the store. The QR code may be the same for affiliate stores. Furthermore, the QR code may include table identification information given for each table used when the user has a meal in a store or the like.

FIG. 13B is a diagram showing an example of the menu information table.

The menu information table is stored in the server storage unit 32 of the server 3. In the menu information table, information such as a category, a menu name, a price, store image data (a menu image provided by the store) and the like are stored in association with the menu ID.

FIG. 14 is a diagram illustrating an example of an operation sequence for registering detailed information.

The operation sequence described below is based on a program stored in the terminal storage unit 22 and the server storage unit 32 in advance, and each element of the mobile terminal 2 and the server 3 is mainly performed by the terminal processing unit 27 and the server processing unit 33. It is executed in cooperation with.

First, the terminal processing unit 27 of the mobile terminal 2 displays a login screen on the display unit 24 (step S300). Next, when the user inputs the user ID and password on the login screen by the operation of the operation unit 23 and selects execution of login, the terminal processing unit 27 logs in including the user ID and password via the terminal communication unit 21. The request is output to the server 3 (step S301). When the login request is acquired via the server communication unit 21, the server processing unit 33 of the server 3 performs user authentication based on the acquired user ID and password (step S302). When the user authentication is permitted, the server processing unit 33 outputs an authentication permission notification to the mobile terminal 2 (step S303). For example, when the detailed information registration process is performed as a continuation of the image classification process described with reference to FIG. 7, the above user authentication process (steps S300 to S303) is not performed.

When the authentication permission notification is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 displays an image classification screen on the display unit 24 (step S304). Next, when the user selects specific image data, the terminal processing unit 27 displays a detailed information registration screen of the selected image data on the display unit 24 (step S305). Next, the terminal processing unit 27 outputs a detailed information registration request including the user ID, selected image data, bibliographic information, and the like to the server 3 (step S306). The bibliographic information includes GPS information and QR code information.

When the detailed information registration request is acquired via the server communication unit 21, the related information extraction unit 334 of the server 3 refers to the store information table based on the bibliographic information acquired from the mobile terminal 2, and related information related to the store name. (Store name related information) is extracted (step S307). The extraction of store name related information will be described later. Next, the server processing unit 33 of the server 3 outputs the extracted store name related information data to the mobile terminal 2 (step S308).

When the store name related information data is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 displays the acquired store name related information on the detailed information registration screen (step S309). Next, when the user operates the operation unit 23 to select a specific store name from the store name related information on the detailed information registration screen, the terminal processing unit 27 selects the store name selection result via the terminal communication unit 21. Is associated with the image ID and output to the server 3 (step S310). Next, the terminal processing unit 27 registers the store name of the image data stored in the terminal storage unit 22 (step S311).

When the selection result of the store name associated with the image ID is acquired via the server communication unit 31, the related information extraction unit 334 refers to the store information table and based on the acquired selection result of the store name, the server storage unit The store ID of 32 is registered in association with the image ID (step S312). Next, the related information extraction unit 334 of the server 3 refers to the store information table and the menu information table, and extracts related information related to the menu name (menu name related information) (step S313). Specifically, the related information extraction unit 334 selects a menu name having the same category as the category registered in step S107 among the menu names of the menu ID related to the store ID registered in step S308. Extract as information. Next, the server processing unit 33 of the server 3 outputs the menu name related information data to the portable terminal 2 (step S314).

When the menu name related information data is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 displays the menu name related information on the detailed information registration screen (step S315). Next, when the user operates the operation unit 23 to select a specific menu name from the menu name related information on the detailed information registration screen, the terminal processing unit 27 selects the menu name via the terminal communication unit 21. The result is linked to the image ID and output to the server 3 (step S316). Next, the terminal processing unit 27 registers the menu name of the image data stored in the terminal storage unit 22 (step S317). Next, the terminal processing unit 27 displays a detailed information screen on which the category of image data, the store name, the menu name, and the like are displayed on the display unit 24 (step S318).

When the selection result of the menu name associated with the image ID is acquired via the server communication unit 21, the server processing unit 33 refers to the store information table and the menu information table and based on the acquired menu name selection result. The menu ID in the server storage unit 32 is registered in association with the image ID (step S319). Next, the categorizing unit 331 of the server 3 refers to the menu information table, totals the number of image data for each price range of the menu with the menu ID linked to the image ID, and stores it in the server storage unit 32. The number of images classified by price range is updated (step S320), and the detailed information registration operation sequence is terminated.

FIG. 15 is a diagram illustrating an example of a flow of processing for extracting store name related information.
Hereinafter, with reference to FIG. 15, the flow of the extraction process of the store name related information in step S307 will be described.

First, the related information extraction unit 334 of the server 3 acquires bibliographic information including GPS information or QR code information via the server communication unit 21 (step S400).

Next, the related information extraction unit 334 refers to the bibliographic information and determines whether the imaging date / time is included within a predetermined time after the reading date / time of the QR code information (step S401). For example, in the case of the data shown in FIG. 11, the imaging date and time is 19:20 on January 16, 2015, and the reading date and time is 19:02 on January 16, 2015. The predetermined time is, for example, 1 hour. In this case, the predetermined time after the reading date / time is from January 16, 2015 at 19:02 to January 16, 2015 at 20:02. The imaging date and time (January 16, 2015 19:20) is within a predetermined time after the reading date and time. Therefore, in the case of the data shown in FIG. 11, the related information extraction unit 334 determines that the imaging date / time is included within a predetermined time after the reading date / time. The predetermined time can be arbitrarily set.

In step S401, when it is determined that the imaging date / time is included within a predetermined time after the reading date / time (step S401; Yes), the related information extraction unit 334 creates a store information table based on the QR code information of the bibliographic information. Referring to, the store name related to the QR code ID is extracted as related information (step S402), and the store name related information extraction process ends.

On the other hand, when it is determined in step S401 that the imaging date / time is not included within the predetermined time after the reading date / time (step S401; No), the related information extraction unit 334 stores store information based on the GPS information of the bibliographic information. With reference to the table, the store name of the store within a certain distance range is extracted as related information from the GPS information (step S403), and the store name related information extraction process is completed.

FIG. 16A is an example of the detailed information registration screen 3100 described in step S309 of FIG.

As shown in FIG. 16A, in the detailed information registration screen 3100, a store name column 3101 and a menu name column 3102 are displayed. In the lower part of the menu name column 3102, store name related information 3104 is displayed together with a question sentence 3103 that says “Where is the store used?”. A store name can be selected by pressing a specific store name area in the store name related information 3104. A search unit 3105 capable of searching for an arbitrary store is displayed below the store name related information 3104.

FIG. 16B is an example of the detailed information registration screen 3200 described in step S315 of FIG.

As shown in FIG. 16B, the selected store name 3206 is displayed in the store name column 3201 on the detailed information registration screen. In the lower part of the menu name column 3202, a menu sentence related information 3204 is displayed together with a question sentence 3203 “What is the menu name here?”. A menu name can be selected by pressing a specific menu name area in the menu name related information 3204. A search unit 3205 that can search for an arbitrary menu is displayed below the related information 3204 of the menu name.

FIG. 16C is an example of the detailed information screen 3300 described in step S318 of FIG.

As shown in FIG. 16C, in the detailed information screen 3300, a category 3302 to which image data belongs is displayed in the category column 3301. The selected store name 3304 is displayed in the store name column 3303. In the menu name column 3305, the selected menu name 3306 is displayed.

FIGS. 17A and 17B are diagrams showing an example of advertisement display.

In the second embodiment, an advertisement related to an image may be displayed in a form associated with the image. That is, the advertisement providing unit 333 may add related information related to the image data as advertisement information to the advertisement data extracted in step S202 described above. For example, first, the related information extraction unit 334 refers to the store information table and extracts the store name and / or menu information related to the store included in a predetermined range from the imaging location based on the GPS information related to the image data. Then, the advertisement providing unit 333 superimposes the extracted store name and / or menu information on the image data as advertiser information, and outputs the superimposed image data to the mobile terminal 2. Alternatively, first, the related information extraction unit 334 refers to the store information table and the menu information table, and stores the store name and / or the menu in a specific store based on the QR code read within a predetermined time from the imaging time related to the image data. Extract information. Then, the advertisement providing unit 333 superimposes the extracted store name and / or menu information on the image data as information related to the advertiser, and outputs the superimposed image data to the mobile terminal 2. If it does in this way, the terminal processing part 27 of the portable terminal 2 can display the information regarding the shop relevant to the provided image data as the said advertisement. Here, since the menu information in the store is an example of the product information in the store, the menu information may be replaced with the product information.

For example, as shown in FIG. 17A, the terminal processing unit 27 of the mobile terminal 2 displays an advertisement 4101A so as to overlap the upper left image 4101 of the category list screen 4100. Here, the advertisement 4101A is a store name and menu information related to the image data. The advertiser of the advertisement 4101 </ b> A is a store where the user has captured the image 4101. Similarly, in the image 4102, an advertisement 4102 </ b> A whose advertiser is the store where the user captured the image 4102 is displayed.

FIG. 17B shows an example of an advertisement displayed on the detailed information screen 4200. The detailed information screen 4200 is a screen that displays image data and bibliographic information related to the image 4201. Here, the advertiser of the advertisement 4201 </ b> A is a store “Ristorante □□” where the user images the image 4201.

According to the advertisement display mode as described above, for example, the advertisement of a store that has been visited in the past can be displayed on the image captured at the store, and the user can display the store at the store. The possibility of revisiting will increase. In this way, by displaying the advertisement related to the specific image in a manner associated with the image, it is possible to more effectively appeal the advertisement to the user.

In addition, when superimposing and synthesizing the store name and / or menu information on the image data as information related to the advertiser, the advertisement providing unit 333 may superimpose and synthesize the store name and / or the menu information. . Thereby, the user can confirm the image data of the part which has overlapped with store name and / or menu information.

Note that information may be provided to the user based on the price range associated with the user ID. For example, the user preference analysis unit 332 refers to the server storage unit 32 and specifies a price range in which the number of price range pixels associated with the user ID is the maximum or a predetermined number or more. Then, the advertisement providing unit 333 may refer to the store information table and the menu information table, specify the store of the specified price range, and output the advertisement of the specified store to the mobile terminal 2. Thereby, the advertisement with strong user's preference can be effectively provided. Here, the store of the specified price range may be a store related to the user preference analyzed by the user preference analysis unit 332 in the first embodiment.

[Other Embodiments]
In the first embodiment and the second embodiment, the server processing unit 33 of the server 3 classifies image data acquired from the mobile terminal 2. However, the terminal processing unit 27 of the mobile terminal 2 may classify the image data and output information related to a category such as the number of images for each category to the server 3.

In the first embodiment and the second embodiment, the advertisement providing unit 333 outputs the advertisement to the mobile terminal 2 based on the analyzed user preference as an example of the information providing unit. However, instead of the advertisement providing unit 333, the information providing unit may provide information to a contact associated with the user identification information based on the analyzed user preference. For example, the contact address associated with the user identification information may be an e-mail address, a mailing list, or an e-mail magazine.

In the second embodiment, a store name related to a store included in a predetermined range is extracted from an imaging location based on GPS information related to image data, and the extracted store name is superimposed and synthesized on the image data. The output image data is output to the portable terminal 2. However, since the store name is an example of facility information, the store name may be replaced with other facility information, for example, a description of a sightseeing spot name and / or a sightseeing spot. Specifically, when a user photographs an animal at a zoo, the facility information includes a sightseeing place name (zoo name) and / or a description about the sightseeing place (type of animal).

In the second embodiment, the store name based on the QR code read within a predetermined time from the imaging time related to the image data is extracted, and the extracted store name is superimposed and synthesized on the image data. The image data is output to the mobile terminal 2. However, since the store name is an example of facility information, the store name may be replaced with other facility information, for example, a description of a sightseeing spot name and / or a sightseeing spot. Specifically, when a user photographs an animal at a zoo, the facility information includes a sightseeing place name (zoo name) and / or a description about the sightseeing place (type of animal). In this case, the QR code is installed, for example, in or around the sightseeing spot. Alternatively, the QR code may be printed on a paper of a sightseeing map read by the QR code processing unit 274 in or around the sightseeing spot. Further, the QR code may be unique to a tourist spot.

A computer program for causing a computer to realize the functions of the terminal processing unit 27 and the server processing unit 33 may be provided in a form recorded on a computer-readable recording medium such as a magnetic recording medium or an optical recording medium. Good.

It should be understood by those skilled in the art that various changes, substitutions, and modifications can be made thereto without departing from the spirit and scope of the present invention, and the embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 1 Information delivery system 2 Portable terminal 21 Terminal communication part 22 Terminal memory | storage part 23 Operation part 24 Display part 25 Imaging part 26 GPS sensor 27 Terminal processing part 271 Image processing part 272 Image browsing part 273 Advertisement display part 274 QR code processing part 3 Server 31 Server communication unit 32 Server storage unit 33 Server processing unit 331 Categorizing unit 332 User preference analysis unit 333 Advertisement providing unit 334 Related information extraction unit 4 Base station 5 Mobile communication network 6 Gateway 7 Internet

Claims (10)

1. A storage unit for storing the category to which the image data belongs in association with the user identification information;
   The image data and the user identification information are acquired from a user terminal, the category is determined by performing image recognition processing on the acquired image data, and the determined category is associated with the user identification information and stored. A categorizing section to be registered in the section,
   An analysis unit that analyzes user preferences based on the category associated with the user identification information;
   Based on the analyzed user preferences, an information providing unit that provides information to the user;
   An information distribution apparatus comprising:
2. The information distribution device according to claim 1, wherein the categories form a tree structure.
3. A related information extracting unit that extracts related information related to the image data based on position information related to the image data;
   The information distribution apparatus according to claim 1, wherein the information providing unit provides the information to the user based on the extracted related information.
4. The related information is facility information relating to a facility included within a predetermined range from an imaging location based on the position information,
   The information distribution apparatus according to claim 3, wherein the information providing unit combines the extracted related information with the acquired image data, and provides the combined image data to the user.
5. A related information extracting unit that extracts related information related to the image data based on code information read before imaging related to the image data;
   The information distribution apparatus according to claim 1, wherein the information providing unit provides the information to the user based on the extracted related information.
6. The related information is facility information related to a facility based on the code information read within a predetermined time from an imaging time related to the image data,
   The information distribution apparatus according to claim 5, wherein the information providing unit combines the extracted related information with the acquired image data, and provides the combined image data to the user.
7. The information providing unit according to any one of claims 1 to 6, wherein the information providing unit increases the information providing frequency from the information providing unit or increases the display of the information as the number of the image data related to the category increases. The information distribution apparatus according to one item.
8. The information according to any one of claims 1 to 7, wherein the analysis unit acquires the number of advertisements viewed by a user and analyzes the user preference based on the acquired number of advertisements viewed and the category. Distribution device.
9. The categorizing unit performs face recognition processing on the acquired image data to determine a gender and age category of the person shown in the image data,
   9. The information providing unit according to claim 1, wherein the information providing unit provides the information to the user based on the gender and the age category of the person shown in the determined image data. The information distribution apparatus described.
10. An information distribution program for a computer having a storage unit for storing a category to which image data belongs in association with user identification information,
    Obtaining the image data and the user identification information from a user terminal;
    By performing image recognition processing on the acquired image data, a category to which the image data belongs is determined,
    The determined category is associated with the user identification information and registered in the storage unit,
    Analyzing user preferences based on the category associated with the user identification information,
    Causing the computer to provide information to a user based on the analyzed user preferences,
    An information distribution program characterized by that.

Images