JP6780691B2 - Information distribution device and information distribution program - Google Patents

Description

The present invention relates to an information distribution device and an information distribution program.

In recent years, not only camera devices such as digital cameras, but also various devices such as mobile phones, smartphones, and tablets are equipped with camera functions. The captured image data is stored in the storage unit of those terminals, and the stored image data is displayed in a list, for example, arranged in chronological order at the time of imaging.

A technique is known in which subjects are classified into predetermined categories by so-called image recognition processing that analyzes a predetermined feature amount related to the image data, and the image data is arranged and displayed according to the category to which the subject belongs (for example, a patent). Document 1).

JP-A-2007-133746

Categorizing and displaying the captured images is useful for users who browse the images, such as making it easier to find the desired image. Further, no method has been proposed to effectively utilize the information on how the images taken and stored by a specific user are categorized for purposes other than the purpose of categorizing the images.

An object of the information distribution device and the information distribution program described in the present application is to provide information to a user by using categorized information of images.

The information distribution device has a storage unit that stores the category to which the image data belongs in association with the user identification information, and the category by acquiring the image data and the user identification information from the user terminal and performing image recognition processing on the acquired image data. A categorization unit that determines and registers the determined category in the storage unit by associating it with the user identification information, an analysis unit that analyzes user preferences based on the category associated with the user identification information, and an analyzed user. It has an information providing unit that provides information to the user based on the preference.

The information distribution program is an information distribution program of a computer having a storage unit that stores the category to which the image data belongs by associating it with the user identification information. The image data and the user identification information are acquired from the user terminal, and the acquired image. By performing image recognition processing on the data, the category to which the image data belongs is determined, the determined category is linked to the user identification information and registered in the storage unit, and the user preference is based on the category linked to the user identification information. Is analyzed, and the computer is made to provide information to the user based on the analyzed user preference.

According to the above-mentioned information distribution device and information distribution program, it has become possible to effectively provide information having a strong user preference.

The objects and effects of the present invention will be recognized and obtained specifically by using the components and combinations pointed out in the claims. Both the general description described above and the detailed description below are exemplary and descriptive and do not limit the invention as described in the claims.

It is a figure which shows an example of the schematic structure of the information distribution system 1. It is a figure which shows an example of the schematic structure of the mobile terminal 2. It is a figure which shows an example of the data structure about a specific image. It is a figure which shows an example of the hierarchical structure of a category when image data is categorized. It is a figure which shows an example of the schematic structure of the server 3. It is a figure which shows an example of the data structure about a specific user. It is a figure which shows an example of the operation sequence of image classification. It is a figure which shows an example of the login screen 1100 described in step S100 of FIG. It is a figure which shows an example of the image viewing screen 1200 described in step S104 of FIG. It is a figure which shows an example of the image classification request selection screen 1300 described in step S105 of FIG. It is a figure which shows an example of the image classification screen 1400 described in step S111 of FIG. It is a figure which shows an example of the operation sequence of the provision of advertisement. It is a figure which shows an example of the advertisement which was explained in step S204 of FIG. It is a figure which shows the other example of the advertisement described by the step S204 of FIG. 9 is a diagram showing still another example of the advertisement described in step S204 of FIG. It is a figure which shows an example of the data structure about a specific image used in 2nd Embodiment. It is a figure which shows an example of the data structure about a specific user used in 2nd Embodiment. It is a figure which shows an example of a store information table. It is a figure which shows an example of a menu information table. It is a figure which shows an example of the operation sequence of detailed information registration. It is a figure which shows an example of the flow of the extraction process of the store name related information. This is an example of the detailed information registration screen 3100 described in step S309 of FIG. This is an example of the detailed information registration screen 3200 described in step S315 of FIG. This is an example of the detailed information screen 3300 described in step S318 of FIG. It is a figure which shows an example of the display of an advertisement. It is a figure which shows an example of the display of an advertisement.

[First Embodiment]
FIG. 1 is a diagram showing an example of a schematic configuration of an information distribution system 1.

The information distribution system 1 includes at least one mobile terminal 2 and a server 3. The mobile terminal 2 and the server 3 are connected to each other via a communication network, and are connected to each other via, for example, a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7. The program executed on the mobile terminal 2 and the program executed on the server 3 communicate with each other using a communication protocol such as a hypertext transfer protocol (HTTP).

FIG. 2 is a diagram showing an example of a schematic configuration of the mobile terminal 2.

The mobile terminal 2 includes a terminal communication unit 21, a terminal storage unit 22, an operation unit 23, a display unit 24, an imaging unit 25, a GPS sensor 26, a terminal processing unit 27, and the like. As the mobile terminal 2, a multifunctional mobile phone (so-called "smartphone") is assumed, but the present invention is not limited to this. The mobile terminal 2 may be a personal computer (PC), a mobile phone (so-called "feature phone"), a mobile information terminal (Personal Digital Assistant, PDA), a portable game machine, and portable music, as long as the present invention can be applied. It may be a player, a tablet PC, or the like.

The terminal communication unit 21 includes a communication interface circuit including an antenna having a predetermined frequency band as a sensitive band, and connects the mobile terminal 2 to a wireless communication network. The terminal communication unit 21 establishes a wireless signal line with the base station 4 by the WCDMA (registered trademark) (Wideband Code Division Multiple Access) method or the like via the channel assigned by the base station 4, and connects with the base station 4. Communicate between. The terminal communication unit 21 outputs the data supplied from the terminal processing unit 27 to the server 3 or the like. Further, the terminal communication unit 21 supplies the data acquired from the server 3 or the like to the terminal processing unit 27.

The terminal storage unit 22 includes, for example, at least one of a semiconductor memory, a magnetic disk device, and an optical disk device. The terminal storage unit 22 stores operating system programs, driver programs, application programs, data, and the like used for processing in the terminal processing unit 27. For example, the terminal storage unit 22 stores, as a driver program, an input device driver program that controls the operation unit 23, an output device driver program that controls the display unit 24, and the like. Further, the terminal storage unit 22 stores a program or the like for acquiring and displaying image data as an application program. In addition, the terminal storage unit 22 stores image data and bibliographic information associated therewith as data. Further, the terminal storage unit 22 may temporarily store temporary data related to a predetermined process.

The operation unit 23 may be any device as long as the mobile terminal 2 can be operated, for example, a touch pad, a keyboard, or the like. The user can input characters, numbers, etc. using the operation unit 23. When the operation unit 23 is operated by the user, the operation unit 23 generates a signal corresponding to the operation. The generated signal is supplied to the terminal processing unit 27 as a user's instruction.

The display unit 24 may be any device as long as it can display an image, an image, or the like, and is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The display unit 24 displays an image or the like corresponding to the image data supplied from the terminal processing unit 27.

The image pickup unit 25 may be any device as long as it can capture images and read the QR code (registered trademark), and includes, for example, a lens and an image pickup device.

The GPS sensor 26 is an example of a position sensor, and is connected to a communication satellite to acquire position information (GPS (Global Positioning System) information) including latitude information and longitude information. The mobile terminal 2 does not necessarily have to be equipped with the GPS sensor 26. Further, the position information may be position information other than GPS information. For example, the position information may be position information specified based on the intensity of radio waves received by the position sensors of the mobile terminal 2 from a plurality of base stations. Alternatively, the location information may be the location information acquired by the mobile terminal 2 by iBeacon (registered trademark).

The terminal processing unit 27 includes one or a plurality of processors and peripheral circuits thereof. The terminal processing unit 27 comprehensively controls the overall operation of the mobile terminal 2, and is, for example, a CPU (Central Processing Unit). The terminal processing unit 27 has the terminal communication unit 21, the display unit 24, the image pickup unit 25, and the terminal communication unit 27 so that the program stored in the terminal storage unit 22 is executed in an appropriate procedure according to the operation of the operation unit 23 and the like. It controls the operation of the GPS sensor 26 and the like. The terminal processing unit 27 executes processing based on a program (operating system program, driver program, application program, etc.) stored in the terminal storage unit 22. Further, the terminal processing unit 27 can execute a plurality of programs (application programs and the like) in parallel.

The terminal processing unit 27 includes an image processing unit 271, an image viewing unit 272, an advertisement display unit 273, and a QR code processing unit 274. Each of these units is a functional module realized by a program executed by a processor included in the terminal processing unit 27. Alternatively, each of these parts may be mounted on the mobile terminal 2 as firmware.

The image processing unit 271 generates image data and bibliographic information related to the image data based on the image captured by the image capturing unit 25. The bibliographic information includes a user ID, an image type, an imaging date and time, and the like. The bibliographic information may include other information such as Exif (Exif image file format) information. The image data and the bibliographic information are stored in the terminal storage unit 22.

The image viewing unit 272 displays the image viewing screen on the display unit 24 based on the program stored in the terminal storage unit 22. The image viewing screen can be a screen of various forms such as a list display screen of thumbnail images based on image data and a display screen of a specific image data alone.

When the advertisement display unit 273 acquires the advertisement data from the server 3, the advertisement display unit 273 displays the advertisement on the display unit 24.

The QR code processing unit 274 is an example of a code information processing unit, and associates information about a QR code read by the imaging unit 25 with image data stored in the terminal storage unit 22 by a predetermined method. In the present embodiment, the case where the code information is a QR code will be described, but the code information may be other code information such as a two-dimensional bar code or a one-dimensional bar code other than the QR code.

FIG. 3 is a diagram showing an example of a data structure relating to a specific image.

The data structure shown in FIG. 3 shows a series of data associated with the image ID stored in the terminal storage unit 22. The series of data includes, for example, a user ID, an image type, an imaging date and time, a category to which an image belongs, image data, and the like. The above series of data is an example, and may also include Exif information, other image-related metadata, and the like as data related to images.

Image types include, but are not limited to, "capture", "download", "capture", and the like. “Imaging” indicates that the image data is captured by the imaging unit 25 of the mobile terminal 2. “Download” indicates that the image data is downloaded from the network to which the mobile terminal 2 is connected. “Capture” indicates that the image data is generated by capturing an arbitrary display on the display unit 24 of the mobile terminal 2.

FIG. 4 is a diagram showing an example of a hierarchical structure of categories when image data is categorized.

As shown in FIG. 4, a plurality of categories form one tree structure. "Image" has, for example, subcategories of "person", "meal", "landscape", and "other".

"Person" has, for example, subcategories of "male" and "female", and "male" and "female" are, for example, "under 10", "teen", "20", and "30", respectively. It has a subcategory of age such as "dai". "Meal" has, for example, subcategories of "Japanese food", "Western food", and "Chinese food". "Japanese food" has subcategories such as "sushi", "soba", and "yakitori". "Western food" has subcategories such as "pasta", "salad", and "pizza". "Chinese food" has subcategories such as "ramen", "dumplings", and "spring rolls". "Landscape" has, for example, subcategories of "mountains", "rivers", and "sea". “Facilities” have, for example, subcategories of “towers”, “shrines / temples / churches”, and “theme parks”.

The above-mentioned categorization method is an example, and categorization can be performed by any other method. For example, the subcategories of "Western food" may include categories such as "Italian", "French", and "Spanish food". Moreover, the category may form a structure other than the tree structure.

FIG. 5 is a diagram showing an example of a schematic configuration of the server 3.

The server 3 includes a server communication unit 31, a server storage unit 32, and a server processing unit 33. The server 3 categorizes the image data in response to the request from the mobile terminal 2, and outputs the categorized image data to the mobile terminal 2.

The server communication unit 31 includes a communication interface circuit for transmitting and receiving data via the Internet 7, and communicates with the mobile terminal 2.

The server storage unit 32 includes, for example, at least one of a magnetic tape device, a magnetic disk device, and an optical disk device, and an operating system program, a driver program, and an application program used for processing in the server processing unit 33. Store data etc. In addition, the server storage unit 32 stores advertising data as data. The advertising data includes advertising data displayed on the display unit 24 of the mobile terminal 2 and a URL related to a web page related to the target of the advertising. Further, the server storage unit 32 further includes a buffer for temporarily storing temporary data related to a predetermined process.

The server processing unit 33 includes one or a plurality of processors and peripheral circuits thereof, and collectively controls the overall operation of the server 3, and is composed of, for example, a CPU. Further, the server processing unit 33 controls the operation of the server communication unit 31 and the like by an appropriate procedure based on the programs (operating system program, driver program, application program, etc.) stored in the server storage unit 32. Further, the server processing unit 33 may execute a plurality of programs (application programs and the like) in parallel.

The server processing unit 33 includes a categorization unit 331, a user preference analysis unit 332, an advertisement provision unit 333, a related information extraction unit 334, and the like. These are functional modules realized by a program executed by the processor included in the server processing unit 33. These may be implemented in the server 3 as firmware.

The categorization unit 331 acquires an image classification request from the mobile terminal 2 via the server communication unit 31, and uses a known image recognition processing method to assign each image data to a category, for example, one of the lowest categories shown in FIG. Determine one category.

As a known image recognition processing method, a method of categorizing an image using the boundary line described in JP-A-2007-133746 can be used. In this method, the image is divided into a plurality of areas, the boundary line of each area is extracted, and the figure composed of the extracted boundary line is compared with the figure registered in advance for each category, and both are approximated. The category is judged based on whether or not it is done. For example, when the figure composed of the boundary line extracted from the image is close to the figure registered in advance for the "landscape" category (for example, the triangular figure indicating a mountain), the image data. In the "landscape" category.

In the above method, the color can be used to determine the category. In that case, the color information of each part of the image is extracted, the extracted color is compared with the color registered in advance for each category, and the category is determined based on whether or not the two are similar. For example, when the color extracted from the image is close to the color registered in advance for the "person" category (for example, yellow indicating the skin color), the image data is assigned to the "person" category. Classify into.

Further, as a known image recognition processing method, a method of categorizing an image by using the face recognition processing described in Japanese Patent Application Laid-Open No. 2014-517371 can be used. This method is based on geometric analysis (examining prominent features), optical analysis (a statistical approach that removes unnecessary elements from the image and compares the value with the template to exclude discrepancies). , Judge the category. By the face recognition process, the gender of the subject and the age of "under 10 years old", "teens", "20s" and the like can be determined.

The user preference analysis unit 332 analyzes the user preference based on the number of images by category and the number of advertisement views by category, which will be described later. Here, the user preference is a concept that expresses a user's hobby, preference, interest, etc. with respect to a thing.

As for the analysis method of user preference, for example, the number of images by category and the number of advertisement views by category are added for each category, and the category with the largest number is regarded as user preference. For example, in FIG. 6, in the category of "pasta", the number of images by category is 28, the number of advertisement views by category is 10, and the total number of these is 38. As a result of performing the operation for all categories, for example, if the number 38 related to the "pasta" category is the maximum, the user preference is considered to be "pasta". The above analysis method is an example, and the degree of weighting in the user preference analysis may be changed depending on the number of images by category and the number of advertisement views by category, or another analysis method may be used.

The advertisement providing unit 333 is an example of the information providing unit, and based on the user preference analyzed by the user preference analysis unit 332, the related advertisement data is obtained from the advertisement data stored in the server storage unit 32. Extract.

The related information extraction unit 334 extracts related information related to the image data based on the acquired GPS information or QR code information. Related information will be described later.

FIG. 6 is a diagram showing an example of a data structure relating to a specific user.

The data structure shown in FIG. 6 shows a series of data associated with the user ID stored in the server storage unit 32. The series of data includes, for example, member type, name, gender, date of birth, image information, number of images by category, number of advertisement views by category, and the like. The above series of data is an example and may include information about other users.

The membership type represents the type of user as a member, for example, "paid", "free", and "premium" (indicating members who pay an additional membership fee in addition to the regular membership fee for paid members). Etc. are included.

The image information includes an image ID associated with the user ID and a category of the subject related to the image ID. Therefore, the server storage unit 32 stores the category to which the image belongs in association with the user identification information.

The number of images by category represents the number of image data belonging to each category. For example, the display of "pasta (28)" indicates that there are 28 images belonging to the "pasta" category among the images taken by the user. Further, for example, in the display of "female in her twenties (15)", among the images taken by the user, the image in which the person in the image belongs to the category of "female" is a subcategory of "female". It shows that there are 15 pieces.

The number of times the advertisement is viewed by category indicates the number of times the user selects the display of the screen guided by the advertisement for each category. For example, the display of "pasta (10)" indicates that the user has selected an advertisement belonging to the category of "pasta" and has browsed the screen guided by the advertisement 10 times.

FIG. 7 is a diagram showing an example of an operation sequence of image classification.

The operation sequence described below is based on a program stored in the terminal storage unit 22 and the server storage unit 32 in advance, and is mainly composed of the terminal processing unit 27 and the server processing unit 33, and each element of the mobile terminal 2 and the server 3. Performed in collaboration with. On the mobile terminal 2 side, by launching a specific application, the environment may be automatically set so that the following operations can be executed.

First, the terminal processing unit 27 of the mobile terminal 2 causes the display unit 24 to display the login screen (step S100). Next, when the user inputs the user ID and password on the login screen by operating the operation unit 23 and selects login execution, the terminal processing unit 27 logs in including the user ID and password via the terminal communication unit 21. The request is output to the server 3 (step S101). When the login request is acquired via the server communication unit 31, the server processing unit 33 of the server 3 authenticates the user based on the acquired user ID and password (step S102). When the user authentication is permitted, the server processing unit 33 outputs the authentication permission notification to the mobile terminal 2 (step S103).

When the authentication authorization notification is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 causes the display unit 24 to display the image viewing screen based on the image data stored in the terminal storage unit 22 (step). S104). Next, when the user operates the operation unit 23 to select image classification on the image viewing screen, the terminal processing unit 27 sends the image data and the image classification request stored in the terminal storage unit 22 to the terminal communication unit. Output to the server 3 via 21 (step S105). The image data output to the server 3 in step S105 may be one image data among the set of image data stored in the terminal storage unit 22 by the user's selection or the like, or a plurality of image data. It may be.

When the image classification request is acquired via the server communication unit 31, the categorization unit 331 processes each acquired image data by using the image recognition processing method described above, and determines the category to which each image data belongs (step S106). .. Next, the categorization unit 331 updates the image information associated with the user stored in the server storage unit 32, and registers the category for each image ID (step S107). Next, the categorization unit 331 totals the number of image data for each category and updates the number of images for each category stored in the server storage unit 32 (step S108). Next, the server processing unit 33 outputs the data of the category related to each image data to the mobile terminal 2 via the server communication unit 31 (step S109).

When the data of the category related to each image data is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 registers the category of each image data stored in the terminal storage unit 22 (step S110). Next, the terminal processing unit 27 displays an image classification screen for displaying image data for each category on the display unit 24 based on the image data and categories stored in the terminal storage unit 22 (step S111), and a series of series. End the operation sequence.

FIG. 8A is a diagram showing an example of the login screen 1100 described in step S100 of FIG. 7.

A user ID input unit 1101, a password input unit 1102, and a login decision icon 1103 are arranged in the center of the login screen 1100. When the login decision icon 1103 is selected, the terminal processing unit 27 of the mobile terminal 2 outputs a login request including the input user ID and password to the server 3.

FIG. 8B is a diagram showing an example of the image viewing screen 1200 described in step S104 of FIG. 7.

In the image viewing screen 1200 shown in FIG. 8B, the images 1201 are arranged in 3 rows × 3 columns. The image 1201 is generated by capturing an image captured by the image pickup unit 25 of the mobile terminal 2, an image downloaded from the network to which the mobile terminal 2 is connected, or an arbitrary display on the display unit 24 of the mobile terminal 2. It is an image to be done. The images 1201 are arranged from top to bottom and from left to right in 3 rows × 3 columns in order from the one with the latest image capture, download, or capture. The image viewing screen 1200 is an example, and the display method of the image 1201 is not limited to this display method.

FIG. 8C is a diagram showing an example of the image classification request selection screen 1300 described in step S105 of FIG. 7.

When the user performs a predetermined operation by the operation unit 23, the image viewing unit 272 of the mobile terminal 2 displays a pop-up 1301 asking "Do you want to organize the images in the camera roll?". When "Yes" 1302 is selected, the terminal processing unit 27 outputs one or more image data and an image classification request selected by the user to the server 3 via the terminal communication unit 21. When "No" 1303 is selected, the image classification process ends.

FIG. 8D is a diagram showing an example of the image classification screen 1400 described in step S111 of FIG.

After acquiring the category data from the server 3 via the terminal communication unit 21 and registering the category, the terminal processing unit 27 displays the image classification screen 1400. On the image classification screen 1400, for example, according to the tree structure of FIG. 4, the category names 1401 and the images 1402 classified into the categories are displayed in chronological order, for example. As an example, FIG. 8D shows an image classification screen 1400 related to the "pasta" category. In the category name 1401, the notation of "cooking" and "Western food", which are higher categories of "pasta", is displayed together with the number of images included in the category. For example, in the display of "cooking (45)> western food (25)> pasta (10)" shown in FIG. 8D, there are 45 images of "cooking", of which 25 are images of "western food". Furthermore, it shows that there are 10 images of "pasta" among them.

FIG. 9 is a diagram showing an example of an operation sequence of providing advertisements.

The operation sequence described below is based on a program stored in the terminal storage unit 22 and the server storage unit 32 in advance, and is mainly composed of the terminal processing unit 27 and the server processing unit 33, and each element of the mobile terminal 2 and the server 3. Performed in collaboration with.

As an example, the following processing starts when the server processing unit 33 of the server 3 outputs the data of the category related to each image data to the mobile terminal 2 via the server communication unit 31 (step S109), as described above. Will be done. However, the present invention is not limited to this, and the process may be started at any timing.

The user preference analysis unit 332 of the server 3 analyzes the user preference by the analysis method described above based on the number of images by category and the number of advertisement views by category (step S201).

Next, the advertisement providing unit 333 extracts data related to the analyzed user preference from the advertisement data stored in the server storage unit 32 based on the analyzed user preference. (Step S202). Next, the advertising promotion providing unit 333 outputs the extracted advertising data to the mobile terminal 2 (step S203). In addition, the advertisement providing unit 333 increases the frequency of providing advertisements, and determines the display area of advertisements or the size of displayed characters as the number of image data related to categories (the number of images by category) increases. You may also distribute the enlarged advertising data.

When the terminal processing unit 27 of the mobile terminal 2 acquires the advertisement data via the terminal communication unit 21, the terminal processing unit 27 displays the advertisement on the display unit 24 based on the advertisement data (step S204). As a result, it is possible to effectively provide advertisements with strong user preferences. Next, when the user selects to view the advertisement by an operation such as pressing the area where the advertisement is displayed, the terminal processing unit 27 launches the browser and the URL related to the selected advertisement indicates. The page browsing request is output to an external server (not shown) (step S205).

Next, the terminal processing unit 27 outputs the identification information related to the selected advertisement to the server 3 (step S206). Next, the user preference analysis unit 332 updates the number of advertisement views by category stored in the server storage unit 32 based on the identification information related to the advertisement acquired from the mobile terminal 2 via the server communication unit 31. (Step S207), the operation sequence of providing the advertisement is terminated.

FIG. 10A is a diagram showing an example of the advertisement described in step S204 of FIG.

The advertisement 2101 is displayed in the center of the standby screen 2100 displayed on the display unit 24. In the advertisement 2101, the explanation of the new menu provided by the store is displayed in text. The display area of the advertisement 2101 may increase as the number of image data related to the category (the number of images by category) increases. The menu provided by the store is an example of the product provided by the store. Therefore, the description of the new product provided by the store may be displayed in text.

FIG. 10B is a diagram showing another example of the advertisement described in step S204 of FIG.

The advertisement 2201 is displayed at the lower part of the image classification screen 2200 displayed on the display unit 24. In the advertisement 2201, the content of the new menu provided by the store is displayed in text. The display area of the advertisement 2201 may increase as the number of image data related to the category (the number of images by category) increases.

FIG. 10C is a diagram showing still another example of the advertisement described in step S204 of FIG.

The advertisement 2301 is displayed side by side with the image 2302 on the image classification screen 2300 displayed on the display unit 24. In the advertisement 2301, the name of the new menu provided by the store is displayed along with the letters "PR" on the upper left. The size of the characters displayed in the advertisement 2301 may increase as the number of image data related to the category (the number of images by category) increases.

The advertisements 2101, 2201, and 2301 are examples, and the display mode of the advertisement is not limited to the above. The advertisement may be displayed including various contents such as images and videos in addition to text information. Further, the advertisement is not limited to the one provided at the restaurant according to the analyzed user preference, but may be an advertisement provided by any store, industry, group, or the like. For example, advertisements related to baby products and children's clothing will be provided to users who have many images of infants, and advertisements related to outdoor products will be provided to users who have many images of mountains and the sea. Will be done. Further, for example, a fashion or health advertisement related to a woman in her twenties may be provided to a user who has many images of a woman in her twenties. Alternatively, for users with many images such as towers, shrines / churches, or theme parks, advertisements related to tourist spots will be provided.

[Second Embodiment]
In the second embodiment, detailed information can be registered for each image data. Here, the detailed information refers to detailed information that expands the category of image data more specifically. For example, for the category of "meal", "store name", "menu name", and the like can be used as detailed information. The detailed information can be arbitrarily set according to the category of the image data and the like.

Further, in the second embodiment, as a candidate for detailed information, related information related to specific image data can be extracted based on GPS information, QR code information, or the like. Here, the related information is information that is a candidate for detailed information, and is a set of facility information related to a facility having a predetermined relationship with bibliographic information such as GPS information or QR code information (including a case where there is only one piece of information). ). For example, a set of store names of one or more stores within a certain distance range from GPS information related to specific image data is related information related to the image data. Alternatively, the store name related to the QR code read within a predetermined time before the imaging date and time of the specific image data is the related information related to the image data. Here, the "store name" and the "menu name" are examples of facility information, and the second embodiment describes the case where the facility information is the "store name" and the "menu name". The "store name" and "menu name" may be other facility information, for example, "place name", "product name", "sightseeing spot name", "explanation of tourist spot", or the like. Further, in the second embodiment, the related information is extracted using either the GPS information or the QR code information, but the related information may be extracted using both the GPS information and the QR code.

In the second embodiment, the mobile terminal 2 and the server 3 described in the first embodiment will be used as they are.

FIG. 11 is a diagram showing an example of a data structure relating to a specific image used in the second embodiment.

The data structure shown in FIG. 11 shows a series of data associated with the image ID stored in the terminal storage unit 22. The data structure shown in FIG. 11 further includes GPS information, QR code information, a store name, a menu name, and the like, as compared with the data structure shown in FIG.

When the image pickup unit 25 performs imaging, the terminal processing unit 27 acquires the GPS information acquired by the GPS sensor 26, associates it with the image data, and stores it in the terminal storage unit 22.

When the QR code is read by the imaging unit 25, the QR code processing unit 274 stores information related to the QR code (for example, QR code identification information) and the reading date and time in the buffer of the terminal storage unit 22. Only the information related to the QR code having the latest read date and time and the read date and time are stored in the buffer. When the image data is generated by the image processing unit 271, the QR code processing unit 274 further stores the information related to the QR code and the reading date and time thereof in the terminal storage unit 22 in association with the image data. Therefore, all the image data stored in the terminal storage unit 22 is associated with the latest QR code information among the QR codes read before the imaging date and time.

The store name is an example of the above-mentioned detailed information, for example, a store name to which an object of image data is provided. The menu name is an example of the above-mentioned detailed information, for example, the menu name of the object.

FIG. 12 is a diagram showing an example of a data structure relating to a specific user used in the second embodiment.

The data structure shown in FIG. 12 shows a series of data associated with the user ID stored in the server storage unit 32. Compared with the data structure shown in FIG. 6, the data structure shown in FIG. 12 is further associated with the store ID and the menu ID to the image ID, and includes the number of images by price range. The number of images by price range represents the number of image data belonging to each price range. For example, in the display of "1,000 yen (25)", among the images taken by the user, the image whose price belongs to the price range of 1,000 yen (1,000 to 1,999 yen) is displayed. It shows that there are 25.

The number of images by price range may be used as the basis for analyzing user preferences. The number of images by price range indicates, for example, how much the user pays for cooking at a restaurant, and the number of images by category and the number of advertisement views by category indicate the user's hobbies, tastes, interests, etc. It can be said that it is expressed from a different point of view. Other parameters that are the basis of the analysis of user preference, such as the number of images for each price range, may be set arbitrarily. Parameters that are the basis of user preference analysis can be set based on the store information table and menu information table, which will be described later, and arbitrary items included in other tables.

FIG. 13A is a diagram showing an example of a store information table.

The store information table is stored in the server storage unit 32 of the server 3. In the store information table, information such as a store name, an address, a QR code ID (QR code identification information), and menu information (menu ID) is stored in association with the store ID. The QR code is installed in the store, for example. The QR code may be unique to the store. Further, the QR code may be the same among affiliated stores. Further, the QR code may include table identification information given for each table used when the user eats in a store or the like.

FIG. 13B is a diagram showing an example of the menu information table.

The menu information table is stored in the server storage unit 32 of the server 3. In the menu information table, information such as a category, a menu name, a price, and store image data (menu image provided by the store) is stored in association with the menu ID.

FIG. 14 is a diagram showing an example of an operation sequence of detailed information registration.

The operation sequence described below is based on a program stored in the terminal storage unit 22 and the server storage unit 32 in advance, and is mainly composed of the terminal processing unit 27 and the server processing unit 33, and each element of the mobile terminal 2 and the server 3. Performed in collaboration with.

First, the terminal processing unit 27 of the mobile terminal 2 causes the display unit 24 to display the login screen (step S300). Next, when the user inputs the user ID and password on the login screen by operating the operation unit 23 and selects login execution, the terminal processing unit 27 logs in including the user ID and password via the terminal communication unit 21. The request is output to the server 3 (step S301). When the login request is acquired via the server communication unit 21, the server processing unit 33 of the server 3 authenticates the user based on the acquired user ID and password (step S302). When the user authentication is permitted, the server processing unit 33 outputs the authentication permission notification to the mobile terminal 2 (step S303). For example, when the detailed information registration process is performed as a continuation of the image classification process described with reference to FIG. 7, the above user authentication processes (steps S300 to S303) are not performed.

When the authentication authorization notification is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 displays the image classification screen on the display unit 24 (step S304). Next, when the user selects specific image data, the terminal processing unit 27 causes the display unit 24 to display the detailed information registration screen of the selected image data (step S305). Next, the terminal processing unit 27 outputs a detailed information registration request including the user ID, the selected image data, the bibliographic information, and the like to the server 3 (step S306). The bibliographic information includes GPS information, QR code information, and the like.

When the detailed information registration request is acquired via the server communication unit 21, the related information extraction unit 334 of the server 3 refers to the store information table and related information related to the store name based on the bibliographic information acquired from the mobile terminal 2. (Store name related information) is extracted (step S307). The extraction of store name-related information will be described later. Next, the server processing unit 33 of the server 3 outputs the extracted store name-related information data to the mobile terminal 2 (step S308).

When the data of the store name-related information is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 displays the acquired store name-related information on the detailed information registration screen (step S309). Next, when the user selects a specific store name from the store name related information on the detailed information registration screen by operating the operation unit 23, the terminal processing unit 27 selects the store name via the terminal communication unit 21. Is linked to the image ID and output to the server 3 (step S310). Next, the terminal processing unit 27 registers the store name of the image data stored in the terminal storage unit 22 (step S311).

When the selection result of the store name associated with the image ID is acquired via the server communication unit 31, the related information extraction unit 334 refers to the store information table and the server storage unit is based on the acquired store name selection result. The 32 store IDs are associated with the image ID and registered (step S312). Next, the related information extraction unit 334 of the server 3 refers to the store information table and the menu information table, and extracts related information (menu name related information) related to the menu name (step S313). Specifically, the related information extraction unit 334 relates the menu name having the same category as the category registered in step S107 among the menu names of the menu ID related to the store ID registered in step S308 to the menu name. Extract as information. Next, the server processing unit 33 of the server 3 outputs the data of the menu name related information to the mobile terminal 2 (step S314).

When the data of the menu name-related information is acquired via the terminal communication unit 21, the terminal processing unit 27 of the mobile terminal 2 displays the menu name-related information on the detailed information registration screen (step S315). Next, when the user operates the operation unit 23 to select a specific menu name from the menu name related information on the detailed information registration screen, the terminal processing unit 27 selects the menu name via the terminal communication unit 21. The result is associated with the image ID and output to the server 3 (step S316). Next, the terminal processing unit 27 registers the menu name of the image data stored in the terminal storage unit 22 (step S317). Next, the terminal processing unit 27 displays a detailed information screen on which the image data category, store name, menu name, and the like are displayed on the display unit 24 (step S318).

When the selection result of the menu name associated with the image ID is acquired via the server communication unit 21, the server processing unit 33 refers to the store information table and the menu information table and is based on the selection result of the acquired menu name. The menu ID of the server storage unit 32 is associated with the image ID and registered (step S319). Next, the categorization unit 331 of the server 3 refers to the menu information table, aggregates the number of image data for each price range of the menu of the menu ID associated with the image ID, and stores it in the server storage unit 32. The number of images for each price range is updated (step S320), and the operation sequence for registering detailed information is terminated.

FIG. 15 is a diagram showing an example of a flow of extraction processing of store name related information.
Hereinafter, the flow of the store name-related information extraction process in step S307 will be described with reference to FIG.

First, the related information extraction unit 334 of the server 3 acquires bibliographic information including GPS information or QR code information via the server communication unit 21 (step S400).

Next, the related information extraction unit 334 determines whether or not the imaging date and time is included within a predetermined time after the reading date and time of the QR code information with reference to the bibliographic information (step S401). For example, in the case of the data shown in FIG. 11, the imaging date and time is 19:20 on January 16, 2015, and the reading date and time is 19:02 on January 16, 2015. The predetermined time is, for example, one hour. In this case, the predetermined time after the reading date and time is from 19:02 on January 16, 2015 to 20:02 on January 16, 2015. The imaging date and time (19:20 on January 16, 2015) is within a predetermined time after the reading date and time. Therefore, in the case of the data shown in FIG. 11, the related information extraction unit 334 determines that the imaging date and time is included in a predetermined time after the reading date and time. The predetermined time can be set arbitrarily.

In step S401, when it is determined that the imaging date and time is included within a predetermined time after the reading date and time (step S401; Yes), the related information extraction unit 334 displays the store information table based on the QR code information of the bibliographic information. With reference to this, the store name related to the QR code ID is extracted as related information (step S402), and the extraction process of the store name related information is completed.

On the other hand, in step S401, if it is determined that the imaging date and time is not included within a predetermined time after the reading date and time (step S401; No), the related information extraction unit 334 will perform store information based on the GPS information of the bibliographic information. With reference to the table, the store name of the store within a certain distance range is extracted as related information from the GPS information (step S403), and the extraction process of the store name related information is completed.

FIG. 16A is an example of the detailed information registration screen 3100 described in step S309 of FIG.

As shown in FIG. 16A, on the detailed information registration screen 3100, the store name column 3101 and the menu name column 3102 are displayed. At the bottom of the menu name column 3102, the related information 3104 of the store name is displayed together with the question sentence 3103 "Is the store you used here?". The store name can be selected by pressing the area of the specific store name in the related information 3104 of the store name. At the bottom of the related information 3104 of the store name, a search unit 3105 that can search for any store is displayed.

FIG. 16B is an example of the detailed information registration screen 3200 described in step S315 of FIG.

As shown in FIG. 16B, on the detailed information registration screen, the selected store name 3206 is displayed in the store name column 3201. At the bottom of the menu name column 3202, the related information 3204 of the menu name is displayed together with the question sentence 3203 "Is the menu name here?". The menu name can be selected by pressing the area of the specific menu name in the related information 3204 of the menu name. At the bottom of the related information 3204 of the menu name, a search unit 3205 that can search an arbitrary menu is displayed.

FIG. 16C is an example of the detailed information screen 3300 described in step S318 of FIG.

As shown in FIG. 16C, on the detailed information screen 3300, the category 3302 to which the image data belongs is displayed in the category column 3301. The selected store name 3304 is displayed in the store name column 3303. The selected menu name 3306 is displayed in the menu name column 3305.

17A and 17B are diagrams showing an example of display of advertisement.

In the second embodiment, the advertisement related to the image may be displayed in the manner associated with the image. That is, the advertising promotion providing unit 333 may add related information related to the image data as the information of the advertising advertiser to the advertising data extracted in step S202 or the like described above. For example, first, the related information extraction unit 334 refers to the store information table and extracts the store name and / or menu information related to the store included in the predetermined range from the imaging location based on the GPS information related to the image data. Then, the advertisement providing unit 333 superimposes and synthesizes the extracted store name and / or menu information on the image data as the information of the advertiser, and outputs the superimposed and synthesized image data to the mobile terminal 2. Alternatively, first, the related information extraction unit 334 refers to the store information table and the menu information table, and the store name and / or the menu in a specific store based on the QR code read within a predetermined time from the imaging time related to the image data. Extract information. Then, the advertisement providing unit 333 superimposes and synthesizes the extracted store name and / or menu information on the image data as information about the advertiser, and outputs the superimposed and synthesized image data to the mobile terminal 2. In this way, the terminal processing unit 27 of the mobile terminal 2 can display information about the store related to the provided image data as the advertisement. Since the menu information in the store is an example of the product information in the store, the menu information may be replaced with the product information.

For example, as shown in FIG. 17A, the terminal processing unit 27 of the mobile terminal 2 displays the advertisement 4101A so as to overlap the image 4101 on the upper left side of the category list screen 4100. Here, the advertisement 4101A is the store name and menu information related to the image data. Further, the advertiser of the advertisement 4101A is a store in which the user has captured the image 4101. Similarly, the image 4102 displays the advertisement 4102A whose advertiser is the store in which the user has imaged the image 4102.

FIG. 17B shows an example of advertising displayed on the detailed information screen 4200. The detailed information screen 4200 is a screen for displaying image data and bibliographic information related to image 4201. Here, the advertiser of the advertisement 4201A is the store "Ristorante □□" in which the user imaged the image 4201.

According to the above-mentioned display mode of the advertisement, for example, the advertisement of the store that has been visited in the past can be displayed on the image captured at the store, and the user can display the advertisement at the store. The chances of visiting again increase. In this way, by displaying the advertisement related to the specific image in the manner associated with the image, it is possible to more effectively appeal the advertisement to the user.

Further, when superimposing and synthesizing the store name and / or menu information on the image data as information about the advertiser, the advertising providing unit 333 may superimpose and synthesize the store name and / or menu information in a semi-transparent manner. .. As a result, the user can confirm the image data of the portion that overlaps with the store name and / or the menu information.

Information may be provided to the user based on the price range associated with the user ID. For example, the user preference analysis unit 332 refers to the server storage unit 32 and specifies a price range in which the number of price range pixels associated with the user ID is the maximum or a predetermined number or more. Then, the advertisement providing unit 333 may refer to the store information table and the menu information table, specify the store in the specified price range, and output the advertisement of the specified store to the mobile terminal 2. As a result, it is possible to effectively provide advertisements with strong user preferences. Here, the store in the specified price range may be a store related to the user preference analyzed by the user preference analysis unit 332 in the first embodiment.

[Other Embodiments]
In the first embodiment and the second embodiment, the server processing unit 33 of the server 3 classifies the image data acquired from the mobile terminal 2. However, the terminal processing unit 27 of the mobile terminal 2 may classify the image data and output information related to the category such as the number of images by category to the server 3.

In the first embodiment and the second embodiment, as an example of the information providing unit, the advertisement providing unit 333 outputs the advertisement to the mobile terminal 2 based on the analyzed user preference. However, instead of the advertisement providing unit 333, the information providing unit may provide the information to the contact linked to the user identification information based on the analyzed user preference. For example, the contact information associated with the user identification information may be an e-mail address, a mailing list, an e-mail newsletter, or the like.

In the second embodiment, the store name related to the store included in the predetermined range is extracted from the imaging location based on the GPS information related to the image data, and the extracted store name is superimposed and synthesized on the image data, and then superimposed and synthesized. The image data was output to the mobile terminal 2. However, since the store name is an example of facility information, the store name may be replaced with other facility information, for example, a tourist spot name and / or a description of the tourist spot. Specifically, when the user photographs an animal at the zoo, the facility information includes a tourist spot name (zoo name) and / or an explanation about the tourist spot (animal type).

Further, in the second embodiment, the store name based on the QR code read within a predetermined time from the imaging time related to the image data is extracted, and the extracted store name is superimposed and synthesized on the image data, and then superimposed and synthesized. The image data is output to the mobile terminal 2. However, since the store name is an example of facility information, the store name may be replaced with other facility information, for example, a tourist spot name and / or a description of the tourist spot. Specifically, when the user photographs an animal at the zoo, the facility information includes a tourist spot name (zoo name) and / or an explanation about the tourist spot (animal type). In this case, the QR code is installed in or around the tourist spot, for example. Alternatively, the QR code may be printed on the paper surface of the tourist map read by the QR code processing unit 274 in or around the tourist spot. In addition, the QR code may be unique to the tourist destination.

A computer program for realizing each function of the terminal processing unit 27 and the server processing unit 33 on a computer may be provided in a form recorded on a computer-readable recording medium such as a magnetic recording medium or an optical recording medium. Good.

It will be appreciated by those skilled in the art that various modifications, substitutions, and modifications can be made to this without departing from the spirit and scope of the invention, and embodiments may be combined as appropriate.

1 Information distribution system 2 Mobile terminal 21 Terminal communication unit 22 Terminal storage unit 23 Operation unit 24 Display unit 25 Imaging unit 26 GPS sensor 27 Terminal processing unit 271 Image processing unit 272 Image viewing unit 273 Advertisement display unit 274 QR code processing unit 3 Server 31 Server communication unit 32 Server storage unit 33 Server processing unit 331 Categorization unit 332 User preference analysis unit 333 Advertisement provision unit 334 Related information extraction unit 4 Base station 5 Mobile communication network 6 Gateway 7 Internet

Claims (8)

A storage unit that stores the category to which the image data belongs by associating it with the user identification information,
The image data and the user identification information are acquired from the user terminal, the category is determined by performing image recognition processing on the acquired image data, and the determined category is associated with the user identification information and stored. The categorization department to be registered in the department and
An analysis unit that analyzes user preferences based on the category associated with the user identification information,
Based on the analyzed user preference , information is provided to the user so that the larger the number of the image data related to the category, the higher the frequency of providing information , or the larger the display area or the displayed characters. Information provision department that provides
An information distribution device characterized by having. It further has a related information extraction unit that extracts related information related to the image data based on the position information related to the image data.
The information distribution device according to claim 1, wherein the information providing unit provides the information to the user based on the extracted related information. The related information is facility information relating to a facility included within a predetermined range from the imaging location based on the location information.
The information distribution device according to claim 2 , wherein the information providing unit synthesizes the extracted related information with the acquired image data and provides the synthesized image data to the user. It further has a related information extraction unit that extracts related information related to the image data based on the code information read before the imaging of the image data.
The information distribution device according to any one of claims 1 to 3, wherein the information providing unit provides the information to the user based on the extracted related information. The related information is facility information related to a facility based on the code information read within a predetermined time from the imaging time related to the image data.
The information distribution device according to claim 4 , wherein the information providing unit synthesizes the extracted related information with the acquired image data and provides the synthesized image data to the user. The information according to any one of claims 1 to 5 , wherein the analysis unit acquires the number of advertisement views viewed by the user and analyzes the user preference based on the acquired number of advertisement views and the category. Delivery device. The categorization unit determines the gender and age categories of the person reflected in the image data by performing face recognition processing on the acquired image data.
The information providing unit provides the information to the user based on the category of the gender and the age of the person reflected in the determined image data, according to any one of claims 1 to 6. The information distribution device described. An information distribution program for a computer having a storage unit that stores a category to which image data belongs in association with user identification information.
The image data and the user identification information are acquired from the user terminal, and the image data and the user identification information are acquired from the user terminal.
By performing image recognition processing on the acquired image data, the category to which the image data belongs is determined.
The determined category is associated with the user identification information and registered in the storage unit.
The user preference is analyzed based on the category associated with the user identification information, and the user preference is analyzed.
Based on the analyzed user preference , information is provided to the user so that the larger the number of the image data related to the category, the more frequently the information is provided , or the larger the display area or the displayed characters. To cause the computer to perform
An information distribution program characterized by this.