JP6998914B2 - Information processing systems, programs, and information processing methods - Google Patents

Description

The present invention relates to an information processing system, a program, and an information processing method, and more particularly to an information processing system for evaluating an image posted on social media.

In recent years, users have been posting images of photographs taken by smartphones to social media and sharing their interests and images with other users. One of the purposes of the user is to obtain sympathy for the posted image by the reaction from another user (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2018-120575

By the way, users tend to be conscious of the appearance of images to be captured in order to expect more reactions from other users. Users who are conscious of the appearance of images take a plurality of pictures for posting on social media and post images carefully selected from the images of the plurality of pictures. In addition, a user who is conscious of the appearance of an image often asks a friend of the user to evaluate which of a plurality of photographs has a good appearance.

However, no system is known for evaluating images posted on social media.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an information processing system or the like capable of predicting an evaluation of an image posted on social media.

The information processing system according to the first aspect of the present disclosure includes a type acquisition unit, a number acquisition unit, and a prediction unit. The type acquisition unit acquires the type of the image. The above image is included in the image data. The above-mentioned number acquisition unit acquires the number data. The above-mentioned number data includes the number of reactions and the number of specific reactions. The above-mentioned number of reactions is the number of cases for the contents that the other users have more reactions when the user posts a plurality of contents to the social media. The specific number of responses is the number of responses by other users for each type of predetermined image posted on social media. The prediction unit predicts the evaluation of the image based on the type of the image and the number data.

The information processing system according to the first aspect of the present disclosure can predict the evaluation of an image posted on social media.

The information processing system according to the second aspect of the present disclosure further includes a number estimation unit in the first aspect. The number estimation unit estimates the number of potential cases according to the number of followers. The above potential number indicates the possibility that other users will react to the content that the user posts on social media. The prediction unit acquires the predicted number of predicted reactions to the image from the evaluation of the image, the number of potential cases, and the number of followers of a predetermined user.

The information processing system according to the second aspect of the present disclosure can acquire a predicted number of cases for which how many other users will react to an image posted on social media.

The information processing system according to the third aspect of the present disclosure includes a terminal device and a server device in the second aspect. The terminal device includes an image pickup device, an input device, a communication device, and a display device. The image pickup apparatus acquires the image data of the image taken. The input device inputs the number of followers of the predetermined user. The communication device transmits the image data. The communication device transmits the number of followers of the predetermined user. The communication device receives the predicted number. The display device displays the predicted number. The server device has a server communication unit and a server storage unit in addition to the type acquisition unit, the number acquisition unit, the prediction unit, and the number estimation unit. The server communication unit receives the image data. The server communication unit receives the number of followers of the predetermined user. The server communication unit transmits the predicted number of cases. The server storage unit stores the program. The program causes the prediction unit to function by machine learning.

The information processing system according to the third aspect of the present disclosure can display the predicted number of cases in which the reaction to the image posted on the social media is predicted by the terminal device that communicates with the server device.

In the information processing system according to the fourth aspect of the present disclosure, in any one of the first to third aspects, the image data includes a still image or a moving image.

The information processing system according to the fourth aspect of the present disclosure can predict the evaluation of a still image or a moving image.

In the information processing system according to the fifth aspect of the present disclosure, in any one of the first to fourth aspects, the type acquisition unit determines the attributes of the captured object and the number of the objects in the image. get.

In the information processing system according to the fifth aspect of the present disclosure, it is possible to predict the evaluation of an image according to the attribute of the object and the number of the objects.

In the information processing system according to the sixth aspect of the present disclosure, in the fifth aspect, the attribute of the object indicates a person's gender, animal type, scenery, or food.

In the information processing system according to the sixth aspect of the present disclosure, it is possible to predict the evaluation of an image according to the gender of a person, the type of animal, the scenery, or the food as the attributes of the object.

The program according to the seventh aspect of the present disclosure causes the computer to function as a type acquisition unit, a number acquisition unit, a number estimation unit, and a prediction unit. The type acquisition unit acquires the type of the image included in the image data. The above-mentioned number acquisition unit acquires the number data. The above-mentioned number data includes the number of reactions and the number of specific reactions. The above-mentioned number of reactions is the number of cases for the contents that the other users have more reactions when the user posts a plurality of contents to the social media. The specific number of responses is the number of responses by the other users for each type of predetermined image posted on social media. The number estimation unit estimates the number of potential cases according to the number of followers. The potential number is a number indicating the possibility that the other user reacts to the content posted on the social media by a predetermined user. Based on the type of the image, the number data, and the potential number, the prediction unit acquires the predicted number of predicted reactions to the image from the number of followers of the predetermined user.

The program according to the seventh aspect of the present disclosure can acquire a predicted number of cases for which how many other users will react to an image posted on social media.

The information processing method according to the eighth aspect of the present disclosure includes a type acquisition step, a number acquisition step, a number estimation step, and a prediction step. The type acquisition step acquires the type of the image included in the image data. The above number acquisition step acquires the number data. The above-mentioned number data includes the number of reactions and the number of specific reactions. The above-mentioned number of reactions is the number of cases for the contents that the other users have more reactions when the user posts a plurality of contents to the social media. The specific number of responses is the number of responses by the other users for each type of predetermined image posted on social media. In the number estimation step, the potential number indicating the possibility that the other user reacts to the content posted on the social media by the predetermined user is acquired according to the number of followers. In the prediction step, based on the type of the image, the number data, and the potential number, the predicted number of predicted reactions to the image is acquired from the number of followers of the predetermined user.

In the information processing method according to the eighth aspect of the present disclosure, it is possible to obtain a predicted number of cases in which the number of other users who react to the image posted on the social media is predicted.

The information processing system and the like of the present invention can predict the evaluation of an image posted on social media.

FIG. 1 is a block diagram showing an information processing system of the present embodiment. FIG. 2 is an explanatory diagram showing an example of a method of acquiring an image type by machine learning. FIG. 3 is an explanatory diagram showing a learning model of machine learning that generates model parameters. FIG. 4 is an explanatory diagram showing machine learning in a learning model using model parameters. FIG. 5 is a flowchart showing the processing of the terminal device and the server device according to the present embodiment. FIG. 6 is a schematic diagram showing a home screen of the terminal device according to the present embodiment. FIG. 7 is a schematic diagram showing an input screen of the terminal device according to the present embodiment. FIG. 8 is a schematic diagram showing a forecast result screen of the terminal device according to the present embodiment.

Hereinafter, the information processing system 10 of the present embodiment will be described. As shown in FIG. 1, the information processing system 10 includes a terminal device 1 and a server device 2. The terminal device 1 is configured to be able to communicate with the server device 2.

The terminal device 1 includes an image pickup device 12, an input device 13, a communication device 14, and a display device 15. The image pickup device 12 is configured to be able to acquire image data of the captured image. The input device 13 is configured to be able to input the number of followers of a predetermined user. The communication device 14 is configured to be able to transmit image data and the number of followers of a predetermined user.

The server device 2 includes a server communication unit 21, a server control unit 22, and a server storage unit 23. The server communication unit 21 is configured to receive image data and receive the number of followers of a predetermined user. The server control unit 22 has a type acquisition unit 231, a number acquisition unit 232, a prediction unit 233, and a number estimation unit 234. The server storage unit 23 stores a program that causes the prediction unit 233 to function by machine learning.

The type acquisition unit 231 acquires the type of the image. The image is included in the image data. The number acquisition unit 232 acquires the number data. The number data includes the number of reactions and the number of specific reactions. The number of responses is the number of responses to the content that the other user responded to more when the user posted a plurality of contents to social media. The specific number of responses is the number of responses by other users for each type of predetermined image posted on social media. The prediction unit 233 predicts the evaluation of the image based on the type of the image and the number data.

The number estimation unit 234 estimates the number of potential cases according to the number of followers. The potential number indicates the possibility that other users will react to the content that the user posts on social media. The prediction unit 233 is configured to be able to acquire the predicted number of cases predicted to react to the image from the number of followers of a predetermined user in addition to the evaluation and the number of potential cases for the image.

In the server device 2, the server communication unit 21 is configured to be able to transmit the predicted number of cases. In the terminal device 1, the communication device 14 receives the predicted number of cases. The display device 15 is configured to be able to display the predicted number of cases.

The information processing system 10 of the present embodiment can predict the evaluation of an image posted on social media. The information processing system 10 of the present embodiment further predicts how many other users will react to the image posted on the social media by the terminal device 1 that communicates with the server device 2. The number of cases can be displayed.

The social media is composed of, for example, an external information processing device 3 capable of communicating with the terminal device 1. Examples of social media include SNS (Social Networking Service). The SNS is configured, for example, to allow the user to post the content of a photo, video or document. Examples of the SNS include Twitter (registered trademark), Facebook (registered trademark), Instagram (registered trademark), Google+ (registered trademark), Flickr (registered trademark), or YouTube (registered trademark). Social media is configured to allow you to enter ratings for the content of other users. Social media can, for example, select "Like" for content of interest or interest as another user's rating. Social media is configured to display the cumulative value of the number of "likes" selected.

In social media, the greater the number of "likes", which is the cumulative value of the number of "likes" selected, the higher the interest or interest of other users in the content posted by the user, and the higher the evaluation. Can be said to be high. In other words, the number of "likes" can be said to be the number of responses from other users. Here, among the user's contents, the number with the largest number of "likes" is referred to as the maximum number of "likes". In social media, a follower is a person who grasps the update status of a specific user and follows the activity of the user. Followers can subscribe to social media. In social media, the number of followers who follow the activity of a specific user is called the number of followers. The social media is configured so that the number of followers of registered followers can be confirmed from the terminal device 1.

Hereinafter, each configuration in the information processing system 10 of the present embodiment will be described more specifically.

As shown in FIG. 1, the terminal device 1 includes a storage device 16 and a control device 17 in addition to an image pickup device 12, an input device 13, a communication device 14, and a display device 15. The terminal device 1 is a terminal device used by the user. Examples of the terminal device 1 include mobile phones, cameras, video cameras, tablets, laptop computers, and smartphones.

The information processing system 10 according to the present embodiment is configured so that the terminal device 1 used by the user and the server device 2 can communicate with each other. The terminal device 1 and the server device 2 are connected by wireless communication, for example, via a public communication network 50. In FIG. 1, a double-headed arrow illustrates the flow of information in which communication is performed via the public communication network 50.

The image pickup device 12 is configured as, for example, a camera including an image pickup lens, a diaphragm, a shutter, an image pickup element, and an image processing engine. Examples of the image pickup element include a CCD (Charge Coupled Device) image sensor. The image processing engine acquires image data by performing image processing on the data acquired using the CCD image sensor. The image processing engine is composed of semiconductor elements. The image data is understood in a broad sense including the imaging data, the imaging data obtained by processing the imaging data, and the metadata related to the imaging data.

The input device 13 is configured to be able to input characters, numbers, or image data. Examples of the input device 13 include a touch panel integrally configured with the display device 15, a voice input device capable of inputting by voice, or a physical key. The input device 13 is not limited to the configuration of the touch panel integrally configured with the display device 15, and may be simply configured with the input unit. Examples of the touch panel touch sensor of the input device 13 include a capacitance type, a resistance film type, a surface acoustic wave type, an infrared type, and an electromagnetic induction type structure.

The communication device 14 is configured to be able to communicate with the server communication unit 21 of the server device 2 by using the public communication network 50. The communication device 14 may be configured to enable not only public communication using the public communication network 50 but also wired communication, for example. The communication device 14 may be configured to perform infrared communication, Wi-Fi (registered trademark), Bluetooth (registered trademark), or wireless communication corresponding to the communication standard of BLE.

The communication device 14 has a transmission unit 141 and a reception unit 142. The transmission unit 141 is configured to be able to transmit the input data to the server device 2. The receiving unit 142 is configured so that the server device 2 can receive the evaluation of the predicted image by machine learning based on the input data. The receiving unit 142 is further configured to be able to receive the predicted number of cases predicted by the server device 2. For example, it is preferable that the communication device 14 performs user authentication using a user ID (identification) in advance in order to specify which user has the predicted number of images transmitted from the terminal device 1. The user ID may be the terminal ID of the terminal device 1.

The display device 15 is configured to be able to display an image included in the image data captured by the image pickup device 12. Further, the display device 15 is configured to be able to display an evaluation on the image. Further, the display device 15 is configured to be able to display the predicted number of cases in which the reaction to the image is predicted. Examples of the display device 15 include a touch panel integrally configured with the input device 13. The display device 15 is not limited to the configuration of the touch panel integrally configured with the input device 13, and may be merely a configuration of the display unit. Examples of the display device 15 include a liquid crystal display device and an organic EL (Electro Luminescence) display device.

The storage device 16 has a volatile storage device in which information disappears when the power is turned off, and a non-volatile storage device in which information does not disappear even when the power is turned off. The volatile storage device temporarily stores the information processed by the control device 17. Examples of the volatile storage device include RAM (Random Access Memory). The non-volatile storage device is a storage that can store various information processing programs and various information. Examples of the non-volatile storage device include ROM (Read On Memory). As the ROM, for example, a flash memory or an HDD (Hard Disk Drive) is used. The storage device 16 is configured to be able to store image data captured by the image pickup device 12.

The control device 17 controls various information processing operations. The control device 17 is electrically connected to the image pickup device 12, the input device 13, the communication device 14, the display device 15, and the storage device 16. The control device 17 is configured to be able to control the image pickup device 12, the input device 13, the communication device 14, and the display device 15 based on the information processing program stored in the storage device 16. The control device 17 is composed of, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit).

Next, the server device 2 shown in FIG. 1 will be described. In the server device 2, the server control unit 22 is electrically connected to the server communication unit 21 and the server storage unit 23.

The server communication unit 21 is configured to be able to communicate with the communication device 14 of the terminal device 1 by using the public communication network 50. The server communication unit 21 may be configured to perform not only public communication using the public communication network 50 but also wired communication, for example. The server communication unit 21 and the communication device 14 may be configured to perform infrared communication, Wi-Fi (registered trademark), Bluetooth (registered trademark), or wireless communication corresponding to the communication standard of BLE.

The server communication unit 21 has a server reception unit 211 and a server transmission unit 212. The server receiving unit 211 is configured to be able to receive image data and information on the number of followers of a predetermined user from the terminal device 1. The server transmission unit 212 is configured to be able to transmit an evaluation of an image predicted by machine learning by the server control unit 22 to the terminal device 1. Examples of the evaluation for the image include information on the evaluation value in which the evaluation for the image is displayed as a percentage.

The server control unit 22 is configured so that the type acquisition unit 231, the number acquisition unit 232, and the prediction unit 233 function. The server control unit 22 may be configured so that the number estimation unit 234 functions in addition to the type acquisition unit 231, the number acquisition unit 232, and the prediction unit 233. The server control unit 22 can drive the type acquisition unit 231 or the prediction unit 233 by, for example, operating a machine learning program stored in the server storage unit 23. The server control unit 22 is composed of, for example, a CPU, an MPU, or a GPU (Graphics Processing Unit).

The type acquisition unit 231 is configured to use image data as input data and perform deep learning processing. As shown in FIG. 2, deep learning is a kind of machine learning method using a multi-layer neural network 60. FIG. 2 illustrates a method in which the type acquisition unit 231 acquires an image type by deep learning. Deep learning has an input layer 61, an intermediate layer 62, and an output layer 63. The intermediate layer 62 is also called a hidden layer. The intermediate layer 62 may be composed of two or more layers. The intermediate layer 62 tends to be unlearned if the number of layers is too small. The intermediate layer 62 tends to be overfitted if the number of layers is too large. In deep learning, when image data is input to the input layer 61 as input data, output data is output from the output layer 63. The output data represents a prediction result of which type of image the image of the image data is classified into. In deep learning, a plurality of pairs of input data and teacher data for the input data are prepared in advance.

The type acquisition unit 231 processes the teacher data by the input layer 61, the intermediate layer 62, and the output layer 63. That is, the type acquisition unit 231 dynamically generates and learns the optimum feature amount for the input image data, and performs forward propagation that performs arithmetic processing by forward information propagation. In FIG. 2, the direction of forward propagation is indicated by a thick single arrow.

Further, when learning is performed, the type acquisition unit 231 performs back propagation in which arithmetic processing is performed by information propagation in the reverse direction by giving information on what kind of image the output data is. In FIG. 2, the direction of back propagation is indicated by a thick single arrow. In machine learning, the type acquisition unit 231 evaluates an output error between the input data used for learning as teacher data and the output data. It is preferable that the type acquisition unit 231 sequentially optimizes the parameters of each layer and each node in machine learning from the output error by back propagation. The type acquisition unit 231 can gradually bring the parameters of each layer closer to the optimum values by learning. When the image data is input to the input layer 61, the type acquisition unit 231 can acquire the image type from the image data by using the parameters adjusted based on the result of machine learning.

The type acquisition unit 231 may appropriately set the number of layers in order to acquire the image type from the image data. The artificial neuron outputs the sum of the output of the previous layer multiplied by the parameter. The output data of the artificial neuron is controlled by the activation function, and non-linearity is added. As the activation function, for example, a softmax function, a sigmoid function, or a Gaussian function can be adopted.

The type acquisition unit 231 is given information on M correct answer data t1 to tM indicating which type of image the image data is for the obtained output data y1 to yM. When the information of tM is given from the correct answer data t1 and the back propagation is performed, the type acquisition unit 231 performs machine learning to adjust the sequential parameter to the optimum value. In other words, the type acquisition unit 231 can evaluate the deviation between the output data and the correct answer data by back propagation and optimize the parameters. As the software used for machine learning, for example, OpenCV, Numpy, Matplotlib, or Chainer can be adopted.

In the type acquisition unit 231, for example, the objects included in the image are 7: 1 male: M, 1 female: W, multiple people: S, food: F, animal: P, scenery: B, and others: O. It is configured so that it can be determined which of the two types is classified. The type acquisition unit 231 is not limited to the case where it is provided in the server device 2, and may be configured by an external server. When the type acquisition unit 231 acquires an image type from an external server, the type acquisition unit 231 functions as an input unit in which data indicating the image type is input. In other words, the server device 2 may transfer the image data to an external information processing device and classify what is reflected in the image by the image determination machine of the information processing device.

The type acquisition unit 231 can also be configured using, for example, TensorFlow. TensorFlow is an open source platform for machine learning. TensorFlow can recognize an object appearing in an image included in image data. When the image data is input, the type acquisition unit 231 can output what the possibility of the object included in the image data is and what percentage. When the type acquisition unit 231 discriminates the image data in which the leopard is photographed by machine learning, for example, the possibility of the leopard is 77%, the possibility of the jaguar is 14%, the possibility of the cheetah is 6%, and the snow. It can be output that the possibility of Leopard is 2% and the possibility of Egyptian cat is 1%.

The type acquisition unit 231 can also output, for example, some men in their teens. In the information processing system 10 of the present embodiment, the type acquisition unit 231 is configured to be able to output the difference in gender whether the object is male or female. The type acquisition unit 231 is configured to output the difference in the number of objects, whether the object is singular or plural.

Next, the number acquisition unit 232 is configured to be able to acquire the number data including the number of reactions and the specific number of reactions. The number of responses is the number of responses to the content that the other user responded to more when the user posted a plurality of contents to social media. The specific number of responses is the number of responses by other users for each type of predetermined image posted on social media. For example, the number acquisition unit 232 may acquire the number data from the evaluation on social media for the image of the image data for which the type of the image has been acquired. For example, the number acquisition unit 232 may set the number of “likes” that the user who posted the image to the social media receives the most from other users in the past as the number of reactions. The number of responses can be said to be the maximum number of “likes” that has the largest number of “likes”. Further, the number acquisition unit 232 may set, for example, the number of “likes” to the image posted on the social media as the specific number of reactions.

Since the prediction unit 233 performs machine learning, teacher data is first given as input data. The prediction unit 233 can use performance-based machine learning. The teacher data is, for example, as shown in FIG. 3, a learning image data (x), a value (y1) obtained by dividing the number of “likes” by the maximum number of “likes”, and an image type (y2). It is data. As the learning image data (x), for example, image data posted on social media in the past can be used. As the learning image data (x), for example, an image posted on social media during a predetermined period can be used. The predetermined period may be, for example, 3 months. When using an image posted on social media as the learning image data (x), it is preferable to use image data having a predetermined number of followers or more.

At the value (y1) obtained by dividing the number of "likes" by the maximum number of "likes", the number of "likes" is a specific reaction in which another user reacts to the learning image data (x) posted on social media. It can be used as the number of cases. In the value (y1) obtained by dividing the number of "likes" by the maximum number of "likes", the maximum number of "likes" should be used as the number of responses that other users have received more when posting to social media. Can be done. The number of responses is preferably the maximum number of "likes" when the user posts the content to social media in a predetermined period. That is, the value (y1) obtained by dividing the number of "likes" by the maximum number of "likes" can be said to be an evaluation of the image for which the number of "likes" has been acquired. In FIG. 3, the value (y1) obtained by dividing the number of “likes” by the maximum number of “likes” is shown by the number of “likes” / the maximum number of “likes” (y1).

The image type (y2) may be determined in advance according to a predetermined standard. As a predetermined standard, for example, it may be determined based on whether a person, an animal, a landscape, or food is reflected as an object in the learning image data (x). Further, when a person is shown, the learning image data (x) may be determined based on the gender and the number of the person.

For example, the prediction unit 233 can acquire image data for three months posted on social media and use it as a plurality of image data to generate a model parameter (θ) for machine learning. In the prediction unit 233, for example, in addition to the image data for three months acquired from social media, the number of "likes" indicating the evaluation of the image and the maximum number of "likes" are input. The prediction unit 233 assigns a score of the image included in the image data. The prediction unit 233 is set so that the maximum evaluation is given when the image is set to 100 points.

When the server device 2 generates a learning model, the number of "likes" and the maximum number of "likes" are input as attributes used for image discrimination, if the actual values displayed on social media are input. good. The maximum number of "likes" may be calculated not only by the measured value but also by a predetermined statistical process to determine the possibility that the maximum number of "likes" will be given to the number of followers. The maximum number of "likes" is not only calculated by a predetermined statistical process for the possibility that the maximum number of "likes" will be given to the number of followers, but also the possibility that the maximum number of "likes" will be given to the number of followers. May be calculated by machine learning.

As shown in FIG. 3, the prediction unit 233 uses the learning image data (x) and the value obtained by dividing the number of “likes” by the maximum number of “likes” (y1) as inputs during learning of the learning model. , The three data of the image type (y2) are used. As the learning model, three types of image data (x), a value (y1) obtained by dividing the number of “likes” by the maximum number of “likes”, and an image type (y2) are input. When three data are input, the training model outputs the model parameter (θ) used at the time of prediction. It is preferable that the prediction unit 233 is configured so that the updated model parameter (θ) is stored in the server storage unit 23 after the machine learning is completed.

When the model parameter (θ) is created from the teacher data, the prediction unit 233 saves the model parameter (θ) used at the time of prediction as, for example, a file in the HDF5 file format. Since the prediction unit 233 processes the file in the HDF5 file format with TensorFlow, it is preferable to convert it into the pb file format with the library of KerasToTensorFlow.

Next, as shown in FIG. 4, the prediction unit 233 is a machine based on the image file (X) of the image selected by the user and the teacher data in order to predict the evaluation of the image selected by the user. The trained model parameter (θ) and are input. The prediction unit 233 performs machine learning of the image file (X) using the learning model. The prediction unit 233 performs machine learning with the image file (X) of the image selected by the user and the model parameter (θ) machine-learned based on the teacher data, and the evaluation value (Y1) of the image selected by the user. ) Is output. The prediction unit 233 is configured to output the type of image in the image file (X) in addition to the evaluation value (Y1) by machine learning with the image file (X) and the model parameter (θ). May be good.

The server device 2 is configured to output the number of "likes" and the evaluation of the image as a result of machine learning. There is basically a correlation between the number of likes and the evaluation of images. The server device 2 is configured so that when the maximum number of "likes" is reached, the evaluation value of the evaluation for the image becomes 100 points. In other words, the server device 2 can display 100 photographs that are likely to exceed the maximum number of "likes" so far. For example, when the number of followers of the server device 2 is 1000, the number of "likes" is calculated to be 300 by machine learning. The number estimation unit 234 is machine learning how many followers the number of "likes" will be. The number estimation unit 234 may calculate from statistical data how many likes the number of followers is.

The number estimation unit 234 is configured so that the maximum number of “likes” can be calculated by machine learning when there are many followers. The number estimation unit 234 is configured not only to calculate the maximum number of "likes" from the number of followers by machine learning, but also to statistically calculate how many "likes" are attached from the number of followers. You may. For example, statistically, the number estimation unit 234 can calculate the maximum number of “likes” when the number of followers is 1000, and the maximum number of “likes” when the number of followers is 900.

In the information processing system 10 of the present embodiment, in the server device 2, the number estimation unit 234 may calculate the maximum number of “likes” from the statistical data in a predetermined number of followers. The server device 2 associates the image data and the number of “likes” received by the server communication unit 21 from the specific social media within a preset period with the learning data, and stores the data in the database of the server storage unit 23. You may.

For example, in machine learning, the number estimation unit 234 can calculate a numerical value estimated that the maximum number of “likes” is 350 when the number of followers of social media is 1000. The number estimation unit 234 can calculate a numerical value estimated to have a maximum of 350 likes when the number of followers of social media is 1000 by statistical processing as well as machine learning. Statistical processing is processed so that the correlation between the number of followers and the maximum number of "likes" can be obtained. The relationship between the number of followers and the maximum number of "likes" tends to increase as the number of followers increases. However, the relationship between the number of followers and the maximum number of “likes” is that if the number of followers is small, the maximum number of “likes” may increase. The relationship between the number of followers and the maximum number of "likes" is, for example, when the number of followers is as small as about 100, the number of "likes" is 80. The relationship between the number of followers and the maximum number of "likes" is that if the number of followers is large, the maximum number of "likes" may not increase proportionally.

The relationship between the number of followers and the maximum number of "likes" is, for example, when the number of followers is about 5,000, the number of "likes" does not reach 5,000, and the number of "likes" reaches about 800. There is a possibility of becoming. The relationship between the number of followers and the maximum number of likes is thought to be because the relationship with the user becomes weaker as the number of followers increases.

The server storage unit 23 has a volatile storage device in which information disappears when the power is turned off, and a non-volatile storage device in which information does not disappear even when the power is turned off. The volatile storage device temporarily stores the information processed by the server control unit 22. Examples of the volatile storage device include RAM. The non-volatile storage device is a storage that can store various information processing programs and various information. Examples of the non-volatile storage device include ROM. The non-volatile storage device is specifically a storage that can store a program for machine learning.

Next, the operation of the entire information processing system 10 according to the present embodiment will be described. First, the relationship between the terminal device 1 and the server device 2 will be described.

In the information processing system 10 according to the present embodiment, the terminal device 1 starts the software storage as shown in FIG. 5 by activating the application software stored in the storage device 16 (step 100). In the following, the step may be displayed as S.

Next, the information processing system 10 selects an image for which the user predicts the evaluation of a plurality of images by using the input device 13 of the terminal device 1 (S101).

Further, the input device 13 of the terminal device 1 inputs information on the number of followers of a predetermined user (S102). Either of the image selection and the input of the number of followers may be input first.

Next, the terminal device 1 receives the prediction start of transmitting the image data of the selected image and the information of the number of followers of a predetermined user in the input device 13 (S103). When the input device 13 receives the prediction start for transmitting the image data and the information on the number of followers, the terminal device 1 transmits the image data and the information on the number of followers from the communication device 14 to the server device 2 (S104).

If the terminal device 1 receives the prediction result of the evaluation for the image predicted based on the image data and the information of the number of followers from the server device 2 (S105), the terminal device 1 displays the prediction result on the display device 15 (S106). ..

Next, the operation of the server device 2 that has received the image data and the information on the number of followers from the terminal device 1 will be described.

The server device 2 is in a state where the image can be evaluated in advance. The server device 2 receives image data and information on the number of followers from the terminal device 1 in the process performed by the server device 2 (S201). The server device 2 stores the received image data and information on the number of followers in the server storage unit 23. When the server device 2 receives the image data and the information on the number of followers, the server device 2 starts predicting the evaluation of the image (S202).
The server device 2 acquires the prediction result of the evaluation of the image by the machine learning program whose operation is controlled by the server control unit 22. The server device 2 transmits the acquired prediction result to the terminal device 1 (S203).

Hereinafter, in the information processing system 10 according to the present embodiment, the screen transition in the terminal device 1 will be described.

In the information processing system 10 according to the present embodiment, as shown in FIG. 6, the application icon 156 is displayed on the home screen 150 of the display device 15 of the terminal device 1. When the user touch-operates the application icon 156 of the home screen 150, the terminal device 1 senses the user's touch operation by the input device 13 integrated with the display device 15. In the terminal device 1, the input device 13 senses the user's touch operation and activates the application software stored in the storage device 16.

In the information processing system 10 according to the present embodiment, the terminal device 1 is composed of a smartphone. Examples of the touch operation include a tap operation, a flick operation, a slide operation, and a swipe operation. The earpiece 18 of the mobile phone and the camera 191 of the image pickup apparatus 12 are provided on the front surface of the smartphone. On the front of the smartphone, a return button 133 for returning to the previous state, a home button 134 for returning to the home screen, and a menu button 135 for displaying a list of application software are configured to be displayed on the display device 15.

In the terminal device 1, when the control device 17 activates the application software stored in the storage device 16, the selection screen 151 shown in FIG. 7 is displayed on the display device 15. On the selection screen 151, when the image data of the plurality of photographs 152 captured by the user is stored in the storage device 16 in advance, the plurality of photographs 152 are displayed in a matrix from the upper left to the lower right in the order of newest image data.

In the terminal device 1, a check mark 159 is attached to the photo 152 of which the evaluation of the image is to be predicted among the plurality of photos 152 when the user performs a touch operation. In the information processing system 10 of the present embodiment, for example, in the case of a configuration including a touch panel, the input device 13 is selected by touch operation by touching the photo 152 displayed on the display device 15. The image data of the photograph 152 can be flagged and stored in the storage device 16.

The user can input the number of followers in the input box 131. In the terminal device 1, if the check mark 159 and the number of followers are input and the check start button 132 is touch-operated, the prediction start is accepted. That is, the terminal device 1 causes the user to select a plurality of photographs 152 displayed on the display device 15 so that each of the image data indicating the selected photographs 152 is transmitted from the communication device 14 to the server device 2. It is configured.

Next, in the terminal device 1, when the prediction result is received from the server device 2, the result screen 153 shown in FIG. 8 is displayed on the display device 15. On the result screen 153, a plurality of photographs 152 predicted to be evaluated by the server device 2 are displayed side by side below the display device 15. On the result screen 153, of the plurality of photographs 152 displayed side by side, the same photograph 152 as the central photograph 152 is enlarged and displayed above the display device 15. In FIG. 8, three photographs 152 are displayed side by side below the result screen 153. On the result screen 153, the predicted number 154 based on the evaluation value (Y1) of the enlarged photo 152 is displayed between the enlarged photo 152 and the three side-by-side photographs 152. Further, on the result screen 153, the evaluation score 155 of the evaluation value (Y1) is displayed at the lower left of each of the side-by-side photographs 152. The user can guess which photo 152 should be posted on social media to get more "likes" just by looking at the evaluation score 155.

That is, the terminal device 1 is configured to display an image that is the same as the enlarged image data but has a smaller scale and is smaller on the display device 15 below the enlarged image. In the terminal device 1, other images are displayed on the display device 15 on the left and right of the scaled image. The terminal device 1 may be displayed on the display device 15 so that the image moves according to the touch operation of the user. In the terminal device 1, for example, in response to a user's slide operation, an image matching the image displayed at the lower center of the result screen 153 is enlarged and displayed on the display device 15 above the result screen 153.

Further, the terminal device 1 is configured so that when the user slides three side-by-side photographs 152 in the left-right direction, the photographs 152 move in the left-right direction in response to the slide operation. The result screen 153 is configured such that the same photo 152 is enlarged and displayed above the result screen 153 so as to correspond to the side-by-side center photo 152 according to the movement of the photo 152. The terminal device 1 displays the image of the enlarged photograph 152, and displays the expected number of “likes” for the enlarged image in the predicted number 154 based on the evaluation value (Y1).

On the result screen 153, a save icon 157 and a post icon 158 are provided in the lower right of the enlarged photo 152. The save icon 157 is configured so that among the results of the plurality of photos 152, the photo 152 that the user wants to post on social media can be saved again in the storage device 16.

When the user selects a photo 152 to be posted on social media from a plurality of photos 152, if there are a plurality of similar photos 152, it is unknown which photo 152 has a high evaluation value (Y1). May become. In the terminal device 1, a plurality of photographs 152 are stored in the storage device 16 in chronological order. The terminal device 1 can display the new photo 152 in a time-series manner at a higher position that is easy for the user to see by re-storing the photo 152 that the user wants to post on social media from the similar photo 152.

The information processing system 10 is not limited to a configuration in which the image data captured by the terminal device 1 is transmitted to the server device 2 and the evaluation value (Y1) predicted by the server device 2 is displayed on the terminal device 1. The information processing system 10 can also be evaluated at a timing different from that of shooting, for example. The information processing system 10 may evaluate an image directly from a personal computer, for example, on social media.

As described above, in the information processing system 10 of the present embodiment, in the server device 2, the program causes the computer to function as the type acquisition unit 231, the number acquisition unit 232, the number estimation unit 234, and the prediction unit 233. The type acquisition unit 231 acquires the type of the image included in the image data. The number acquisition unit 232 acquires the number data. The number data includes the number of reactions and the number of specific reactions. The number of responses is the number of responses to the content that the other user responded to more when the user posted a plurality of contents to social media. The specific number of responses is the number of responses by other users for each type of predetermined image posted on social media. The number estimation unit 234 estimates the number of potential cases according to the number of followers. The potential number is a number indicating the possibility that another user will react to the content posted by a predetermined user on social media. The prediction unit 233 can acquire the predicted number of cases in which the reaction to the image is predicted from the number of followers of a predetermined user based on the type of the image, the number of cases data, and the number of potential cases.

Further, the information processing method of the information processing system 10 of the present embodiment includes a type acquisition step, a number acquisition step, a number estimation step, and a prediction step. The type acquisition step acquires the type of the image included in the image data. The number acquisition step acquires the number data. The number data includes the number of reactions and the number of specific reactions. The number of responses is the number of responses to the content that the other user responded to more when the user posted a plurality of contents to social media. The specific number of responses is the number of responses by other users for each type of predetermined image posted on social media. In the case number estimation step, the number of potential cases indicating the possibility that another user reacts to the content posted on the social media by a predetermined user is acquired according to the number of followers. The prediction step acquires the predicted number of cases in which the reaction to the image is predicted from the number of followers of a predetermined user based on the type of the image, the number data, and the number of potential cases.

Hereinafter, a hash tag can also be used as another information processing system 10 according to the embodiment.

In the information processing system 10, the hashtag attached to the content posted on the social media may be used to improve the accuracy of image classification. The server device 2 may collect the hashtag described in the text in which the photograph 152 is uploaded, in addition to the image data, the number of “likes” and the number of followers.

Hashtags are also words arbitrarily entered by the user. The hashtag may be a word, a combination of words, or a component including a subject and a predicate. Hashtags are used in the external information processing device 3 that constitutes social media. Hashtags are used as tags for social media posts.

The hashtag has, for example, an arbitrary word attached after the hash mark. For example, if a user wants to see various posts about "cake", he can search by typing "#cake" in a social media search engine and use it to list the latest posts about cake. can. The server device 2 does not have the word "cake" on the social media, and can use the hashtag of the cake only for the image to identify that the image posted on the social media is related to the cake. In other words, the server device 2 uses hashtags to relatively easily classify an image posted on social media as a specific image that reflects the user's interests and interests. Can be done.

Hashtags are composed of text data. Hashtags contain phrases. The phrase may be one word or a collection of a plurality of words. Hashtags are converted into a format that can handle text data mechanically. Hashtags are decomposed into multiple word units by morphological analysis. Here, morphological analysis can classify hashtags into the smallest units that have meaning in natural language. In the morphological analysis, it is also possible to acquire the part-of-speech information of the classified morphemes. In the morphological analysis, for example, it is possible to acquire information on whether or not the word included in the hashtag is a noun, an adjective, a verb, or an adverb.

For morphological analysis, for example, a morphological analysis engine such as MeCab can be used. MeCab can be used with a named entity dictionary. Examples of named entity dictionaries include system dictionaries such as Neologd.

The type acquisition unit 231 can determine the type of image data from the hashtag. The type acquisition unit 231 may use the hashtag acquired from the social media to determine the type of image data. The type acquisition unit 231 determines, for example, the hashtag in the text input to the social media. The type acquisition unit 231 performs morphological analysis of the text, and extracts the hashtag and the hashtag separated by a space before and after the wording continuous with the hash mark. Similarly, a hash tag may be input to the prediction unit 233. The prediction unit 233 can be used to determine the type of image included in the image data input from the hashtag.

Further, as yet another information processing system 10 according to the embodiment, instead of evaluating the still image, the moving image may be evaluated.

For the moving image, for example, the change of the object can be extracted by performing the detection process of the object. For example, when the server device 2 predicts the number of "likes" of a moving image, the information processing system 10 can further improve the prediction of the number of "likes" by using voice information as incidental information. In the case of moving images, the information processing system 10 can predict the number of "likes" for each predetermined frame rate. The image can be processed at a predetermined frame rate of, for example, 1 second and 10 frames. In the case of moving images, the information processing system 10 can predict the number of "likes" as a set of images. In the case of a moving image, the information processing system 10 may extract an object in a predetermined frame and predict the number of “likes” from the movement of the extracted object.

Each process, or each function of the above-described embodiment, may be realized by centralized processing by a single device or a single system, or may be distributed processing by a plurality of devices or a plurality of systems. It may be realized by. Each component of the embodiment may be configured by dedicated hardware. Each component of the embodiment may be realized by executing a program for a component that can be realized by software.

Each component of the above-described embodiment may be realized, for example, by the CPU executing a software program recorded on a recording medium. The program may be executed by being downloaded from the server, or may be executed by reading the program recorded on a predetermined recording medium. The number of computers that execute the program may be singular or plural. That is, the information processing system 10 of the embodiment may be configured to perform centralized processing or may be configured to perform distributed processing.

It should be noted that the present invention can be carried out in various other forms without departing from its spirit, gist or main characteristics. The above embodiments are merely exemplary in all respects and should not be construed in a limited way. That is, by taking the above embodiment as an example, the present invention is an information retrieval device or an information processing system.

1 Terminal device 2 Server device 10 Information processing system 12 Image pickup device 13 Input device 14 Communication device 21 Server communication unit 23 Server storage unit 231 Type acquisition unit 232 Number acquisition unit 233 Prediction unit 234 Number estimation unit

Claims (8)

The type acquisition unit that acquires the type of image included in the image data, the number of reactions to a specific image that the other user responded more when the user posted multiple images to social media, and posting to social media. It has a number acquisition unit for acquiring number data including a specific reaction number in which another user has reacted for each type of a predetermined image, and a prediction unit for predicting evaluation of the image . The prediction unit performs machine learning using image data for evaluation of an image for predicting evaluation and model parameters machine-learned based on teacher data for the image of the image data for evaluation. The teacher data is the image data for learning, the value obtained by dividing the specific number of reactions of the image data for learning by the number of reactions to the image data for learning, and the learning. An information processing system characterized by including data of an image type indicated by image data . Further provided with a number estimation unit that estimates the number of potential cases that indicate the possibility that another user may react to the image of the image data for evaluation that a predetermined user plans to post on social media according to the number of followers. The prediction unit determines the reaction of the image data for evaluation to the image from the evaluation value indicating the evaluation of the image data for evaluation as a percentage, the number of potential cases, and the number of followers of a predetermined user. The information processing system according to claim 1, wherein the predicted number of predicted cases is acquired. The terminal device includes a terminal device and a server device, and the terminal device includes an image pickup device that acquires image data of the captured image, an input device that inputs the number of followers of the predetermined user, and an image of the captured image. The server device includes a communication device that transmits data and the number of followers of the predetermined user and receives the predicted number, and a display device that displays the predicted number. The server device includes the type acquisition unit and the number of cases. In addition to the acquisition unit, the prediction unit, and the number estimation unit, a server communication unit that receives image data of the captured image and the number of followers of the predetermined user and transmits the prediction number, and machine learning. The information processing system according to claim 2, further comprising a server storage unit that stores a program that functions as the prediction unit that acquires the predicted number of cases . The information processing system according to any one of claims 1 to 3, wherein the image data for evaluation includes a still image or a moving image. The information processing system according to claim 3 or 4, wherein the type acquisition unit acquires the attribute of the object captured in the image of the image data for evaluation and the number of the objects. The information processing system according to claim 5, wherein the attribute of the object is a person's gender, an animal, a landscape, or food. The type acquisition unit that acquires the type of image contained in the image data on the computer, the number of reactions to a specific image that the other user responded more when the user posted multiple images to social media, and social The number of cases to acquire data including the specific number of reactions that other users have reacted to for each type of predetermined image posted on the media. It functions as a case number estimation unit that estimates the number of potential cases that indicate the possibility that another user may react to the image of the image data for evaluation, and a prediction unit that predicts the evaluation of the image, and the prediction unit . Is machine-learned using the evaluation image data of the image for which the evaluation is predicted and the model parameters machine-learned based on the teacher data. The evaluation is predicted, and the teacher data is a value obtained by dividing the specific reaction number of the learning image data and the learning image data by the reaction number to the learning image data, and the learning image. The prediction unit includes data of the type of image indicated by the data, and the prediction unit includes an evaluation value indicating the evaluation of the image data for evaluation as a percentage, the number of potential cases, and the number of followers of a predetermined user. A program for acquiring the predicted number of cases in which the reaction of the image data for evaluation to the image is predicted . The type acquisition step to acquire the type of image included in the image data, the number of reactions to a specific image that other users responded more when the user posted multiple images to social media, and posting to social media. For evaluation of the number acquisition step to acquire the number data including the specific number of reactions that other users responded to for each type of predetermined image, and the evaluation that the predetermined user plans to post to social media according to the number of followers . The prediction step includes a number estimation step for estimating the number of potential cases indicating the possibility that another user may react to the image of the image data of the above , and a prediction step for predicting the evaluation of the image. By machine learning using the image data for evaluation of the image for which evaluation is predicted and the model parameter trained by machine based on the teacher data, the evaluation of the image of the image data for evaluation is predicted. The teacher data includes an image data for learning, a value obtained by dividing the specific number of reactions of the image data for learning by the number of reactions to the image data for learning, and an image indicated by the image data for learning. The prediction step includes the type of data, and the prediction step is based on the evaluation value obtained by displaying the evaluation of the image data for evaluation as a percentage, the number of potential cases, and the number of followers of a predetermined user. An information processing method for acquiring the predicted number of predicted reactions of image data to an image .

Images