JP6569183B2 - Information processing apparatus, method, and program - Google Patents

Description

  Embodiments described herein relate generally to an information processing apparatus, a method, and a program.

  In many electronic commerce (EC), recommended products are introduced to registered members. For example, the preference of the member is estimated from the purchase history, and a product that matches the estimated preference is specified. Also in the distribution of Internet advertisements, preference is estimated based on the browsing history with a portable information terminal such as a smartphone, and advertisements such as banners that match the estimated preference are displayed together with the content.

  The purchase history directly reflects the subjective and positive intention of the person who purchases the purchase history, and analyzing the preference based on the purchase history may narrow the preference range. On the other hand, the browsing history may include a negative intention of the person browsing the browsing history, and the preference tendency obtained based on the browsing history may include an error component and have low accuracy.

  An object is to provide an information processing apparatus and a program that can acquire an appropriate preference tendency with high accuracy.

  The information processing apparatus according to the present embodiment receives, from the server, content data including an image posted by a contributor to a server that provides an SNS service, an EC service, or a video posting browsing service and a contributor comment by a contributor. Download and analyze the author's preference trends. The extraction means extracts images and poster comments from the content data. The language analysis means performs language processing on the contributor comment to extract nouns, adjectives, and adverbs. The preference estimating means estimates the poster's subjective preference polarity and degree based on the extracted adjectives or adverbs. In order to classify the types related to the object into multiple hierarchies and discriminate lower types from the types of the respective hierarchies, a plurality of classifiers are individually prepared for each of the plurality of hierarchies and the plurality of types. A specific classifier is selected from a plurality of classifiers based on the noun extracted from the poster comment. The preference information generating means generates the preference information about the poster by associating the estimated subjective preference polarity and degree with the lower-level type regarding the object in the image identified by the selected classifier. .

FIG. 1 is an overall configuration diagram including a preference information generation processing device (information processing device) according to the present embodiment. FIG. 2 is a configuration diagram of the information processing apparatus according to the present embodiment. FIG. 3 is a functional block diagram of the information processing apparatus of FIG. FIG. 4 is a flowchart showing an operation procedure of information processing in the present embodiment. FIG. 5 is a diagram showing a hierarchy of a plurality of discriminators constituting the image analysis unit of FIG. FIG. 6 is a diagram illustrating an example of a table used for preference polarity analysis in the preference analysis unit of FIG. 3. FIG. 7 is a diagram illustrating an example of a table used for preference degree analysis by the preference analysis unit of FIG. 3. FIG. 8 is a supplementary explanatory diagram of still image extraction processing by the moving image analysis unit of FIG. 3.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the present embodiment, a social networking service (hereinafter referred to as “SNS”) as a community-type service that promotes and supports a connection between people, and a service that sells products and services on a website via a network such as the Internet. Images (including photographs (still images) and videos composed of a series of still images) such as EC (electronic commerce) services, YouTube (registered trademark) video posting browsing services, etc. and accompanying comments (Text) is related to Internet services that are posted as content. Here, an SNS service using “Facebook (registered trademark)”, “Twitter (registered trademark)”, and “Instagram (registered trademark)” as representative examples will be described. These SNSs are high-community services on the Internet that allow users to post comments about their profile, hobbies, etc., along with their own comments. A user who submits an image is particularly referred to as an “image contributor”, and a user who browses the image is referred to as a “viewer” to distinguish between the two. A comment posted by an image contributor himself is also referred to as an “image contributor comment”, and a comment posted by a viewer is referred to as a “viewer comment” to distinguish between the two.

  As shown in FIG. 1, the information processing apparatus 20 in this embodiment is connected to a public communication line network (Internet line) 30. The Internet line 30 is connected to an SNS server 10 and an information terminal 40 owned by a user such as a PC, a smartphone, a PHS, a PDA, or a tablet terminal of an SNS user such as an image poster or a viewer. The SNS user can receive the SNS service provided by the SNS server 10 by communicating with the SNS server 10 through the information terminal 40 of the SNS user. An image contributor can upload content (profile, image, comment) to the SNS server 10 via his / her information terminal 40. The viewer can browse the content of the image contributor uploaded to the SNS server 10 via his / her information terminal 40 and can appropriately upload a comment on the image.

  FIG. 2 is a block diagram showing a configuration of the information processing apparatus 20 in the present embodiment. The information processing apparatus 20 is realized by a computer, for example, and includes a processor 201, a RAM 202, a ROM 203, an input controller 204, a video controller 205, an I / O controller 206, a communication device 207, an input device 210, a display 211, and a storage device 212. Have Note that the information processing apparatus 20 may be realized as a control unit 300 constituting a computer including a processor 201, a RAM 202, a ROM 203, an input controller 204, a video controller 205, an I / O controller 206, and a communication device 207.

  The processor 201 comprehensively controls each device and controller connected to the system bus. The processor 201 can be configured by, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 201 executes programs stored in the RAM 202 to realize various functions. The processor 201 executes a process for specifying the position of a specific part such as a human knee shown in the image by executing the image processing program according to the present embodiment.

  The RAM 202 functions as a main memory, work area, and the like for the processor 201. The processor 201 implements various operations by loading a program or the like necessary for execution of processing from the ROM 203 or the storage device 212 to the RAM 202 and executing the loaded program. The ROM 203 or the storage device 212 includes various programs necessary for realizing various functions such as a basic input output system (BIOS) executed by the processor 201, an operating system program (OS), and a preference analysis processing program according to the present embodiment. Various files processed by various programs, various data, and the like are stored.

  An input controller 204 controls input from an input device 210 such as a keyboard (KB), a pointing device such as a mouse or a touch panel. The video controller 205 controls display of a display such as an LCD (Liquid Crystal Display) under the control of the processor 201. The I / O controller 206 controls access to the storage device 212. The storage device 212 stores, for example, an image file (still image or moving image) that is a target of the part specifying process by the image processing program. An image file in which a human body to be subjected to the part specifying process is captured is input from the external electronic device 400 through the communication device 207, for example. The communication device 207 controls communication with the external SNS server 10 via the Internet line 30.

  FIG. 3 shows a functional configuration of the information processing apparatus 20 according to the present embodiment. FIG. 4 shows an operation procedure of preference analysis processing in the present embodiment. The information processing apparatus 20 according to the present embodiment downloads content data including an image posted by the image poster and a comment by the image poster from the SNS server 10 via the communication device 301 (S1). Along with these images and comments by image contributors, the image contributor's profile, comments posted by the viewer who viewed the images and comments, audio data, and the like are downloaded as appropriate. The separator 303 separates an image, an image contributor's comment, an image contributor's profile, a viewer's comment, and audio data from the downloaded content data based on the area and tag according to each application (S2). If the image data is a photograph (still image), it is supplied to the object extraction processing unit 307. If the image data is a moving image, the image data is supplied to the moving image analysis unit 305. The moving image analysis unit 305 extracts one specific still image from a series of still images constituting the moving image. For example, as shown in FIG. 8, a still image at the time when the number of actions from the viewer, for example, “Like” or a comment reaches the maximum number is selected from a series of still images constituting the moving image. The selected still image is sent to the object extraction processing unit 307.

  The language analysis unit 313 extracts a noun, an adjective, and an adverb as part of speech by subjecting the image contributor comment, the viewer comment, and the text converted from the voice data to text mining processing (S4). In addition, when a comment includes a design such as a pictograph or a stamp, a symbol, or a symbol, they are extracted as an adjective. In most cases, the noun extracted from the image contributor's comment represents the type of the object of interest in the image. The noun extracted from the image contributor's comment (a code representing it) is supplied to the discriminator selection unit 315, and the adjective and adverb (the code representing it) are supplied to the preference analysis unit 325.

  The object extraction processing unit 307 extracts and analyzes an object from the supplied image (S5). This process is typically applied with a convolutional neural network, which determines the type of object (major classification), the position of the object (partition within the frame), and the size of the object area (ratio of the number of pixels in the object area to the frame). Is done. For example, in the example of FIG. 3, three objects are included in the image, and the type of the first object is “cat”, which is located in the central section, and the size class is “large”. Similarly, the second object is of type “Dog”, is located in the lower middle section, the size class is “Small”, and the third object is of type “Cat”, which is in the lower left section. Located and the size class is “narrow”. The object information is supplied to the partial image cutout unit 317, and the partial image cutout unit 317 cuts out three partial images from the original image based on the object information (S6). The clipped partial image is supplied to the image selection unit 319.

  The object information is supplied to the discriminator selection unit 315. The discriminator selection unit 315 selects one of the plurality of discriminators 323-1 to 323-n constituting the image analysis unit 323 that matches the type of object identified as the noun analyzed by the language analysis unit 313. A selection command for selection is output to the image analysis unit 323 (S7). In the classifier selection unit 315, a noun analyzed by the language analysis unit 313 is selected as a noun, for example, based on the position and size of the object information, and a plurality of, here, one object of interest is selected from three objects, A selection command for selecting a partial image corresponding to the object of interest is output to the image selection unit 319. When one of the objects extracted by the object extraction processing unit 307 matches the type of the object identified as the noun analyzed by the language analysis unit 313, the object is selected as the target object. When two or more of the objects extracted by the object extraction processing unit 307 match the type of object identified as a noun analyzed by the language analysis unit 313, The object at the center or the closest position is selected as the object of interest. Further, when there are a plurality of objects whose positions in the image frame are closest to the center, the object occupying the largest area is selected as the target object.

  The image selection unit 319 selects a partial image related to the object of interest according to the selection command. The selected partial image is preprocessed by the preprocessing unit 321 and then supplied to the image analysis unit 323. Typically, as the preprocessing, resolution normalization processing and contrast normalization processing are exemplified in order to stabilize the accuracy of the image analysis result of the image analysis unit 323, but of course, the processing is not limited to these processing.

  The image analysis unit 323 includes a plurality of discriminators 323-1 to 323-n. The plurality of discriminators 323-1 to 323-n are typically configured by a convolutional neural network system. For machine learning when creating a convolutional neural network, a large number of actual teacher images are prepared and learning is repeated. As is well known, the convolutional neural network extracts a feature amount from an image by performing convolution with a filter on a two-dimensional image. A convolutional neural network is composed of a multilayer network that repeats convolution and pooling. In a convolutional neural network, a filter coefficient (weight) effective for identification constituting a convolutional layer is learned using a large amount of data such as a large amount of learning images. The coefficient (weight) is obtained by performing learning to acquire invariance with respect to various deformations by repeating convolution by a filter and pooling that collects reactions in a certain region using a large amount of data. Note that the discrimination performance of the convolutional neural network depends on the filters constituting the convolutional layer. A filter is prepared for each type of object.

  Here, as shown in FIG. 5, the object types are divided into multiple layers such as a large classification type, a middle classification type, and a small classification type, and classifiers 323-1 to 323-n are individually provided for each hierarchical type. Are prepared, and each of the discriminators 323-1 to 323-n identifies the type on the lower classification. For example, if the discriminator 323-1 is related to “animal”, the type of medium classification such as “dog” or “cat” is discriminated. If the discriminator 323-1 is related to “cat”, it is lower than the sled. A classification type such as “Persian” or “American short hair” is identified (S8).

  If “animal” is extracted from the comment by the language analysis unit 313, the classifier 323-1 related to the large category “animal” is selected, and the selected classifier 323-1 analyzes the partial image, for example, “ Identify "Cat". Further, when “cat” is extracted from the comment by the language analysis unit 313, the discriminator 323-2 regarding the “category” of the middle class is selected, and the selected discriminator 323-2 analyzes the partial image, for example, “ Identify "American Shorthair".

  The preference analysis unit 325 generates a table for determining preference polarity (positive (plus) / negative (minus)) for a large number of adjectives as illustrated in FIGS. 6 (a) and 6 (b). In addition, each gender is created and stored in advance, and the adjective extracted by the language analysis unit 313 is referred to a table selected according to age and gender in the image contributor's profile. A typical preference polarity is estimated (S9). Further, as illustrated in FIG. 7, the preference analysis unit 325 creates and stores in advance a table for determining the degree of preference (level) for a large number of adverbs, and the language analysis unit 313 extracts the table. By querying the adverb, the subjective preference degree of the image contributor is estimated (S9). These tables preferably include not only adjectives but also pictograms and stamps. The table should be added and updated from time to time.

  The lower hierarchy type of the object identified by the image analysis unit 323, the preference polarity and the preference level analyzed by the preference analysis unit 325 are integrated into the image poster profile by the profile integration processing unit 327 (S10). For example, if the type of the object identified by analyzing the image is “American short hair”, the preference polarity is “positive” as the preference analysis result, and the preference level is level 3, the user ID, the user name, the age, “American short hair / positive / level 3” as preference information is associated with user information such as sex. That is, the user (image contributor) can recognize a preference tendency that “I like American short hair very strongly”.

  When there is a viewer comment in the content data, the language analysis unit 313 performs a text mining process on the viewer comment in the same manner as the image contributor comment, extracts adjectives (including pictograms) and adverbs, and extracts these adjectives and adverbs. Based on this, the preference analysis unit 325 estimates the preference polarity and the degree of preference. The preference polarity and preference level based on these viewer comments are integrated into the user profile as the objective preference polarity and preference level of other companies related to the image poster.

  Note that the preference analysis unit 325 may be a method of estimating the preference polarity and the degree of preference by preparing a table in advance and inquiring it, or may be configured by a table neural network system.

  The feature of this embodiment is that the preference object is identified from the image posted by the image contributor and the viewer, the preference polarity and the degree of preference are determined from the comments attached to the image, and the preference information is integrated by integrating them. Is to generate. The preference information acquired in this way is very useful as more natural and reliable information with less excessive subjectivity and error vibration than a method of acquiring preference information from purchase history and browsing history.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 20 ... Information processing apparatus 301 ... Communication apparatus 303 ... Separator 305 ... Movie analysis part 307 ... Object extraction process part 313 ... Language analysis part 315 ... Discriminator selection part 317 ... Partial image extraction part 319 ... an image selection unit, 321 ... a preprocessing unit, 323 ... an image analysis unit, 325 ... a preference analysis unit, 327 ... a profile integration processing unit.

Claims (13)

Content data including an image posted by a contributor to a server that provides an SNS service, an EC service, or a video posting browsing service and a contributor comment by the contributor attached thereto are downloaded from the server, and the contributor's preferences In an information processing device that analyzes trends,
Extracting means for extracting the image and the contributor comment from the content data;
Linguistic processing means for extracting the nouns, adjectives, adverbs by language-processing the contributor comments;
Preference estimation means for estimating the polarity and degree of subjective preference of the poster based on the extracted adjectives or adverbs;
In order to classify the types related to objects into multiple layers and identify lower-level types from the types of each layer, a plurality of classifiers individually prepared for each of a plurality of layers and a plurality of types,
Means for selecting a specific classifier from the plurality of classifiers based on nouns extracted from the contributor comments;
Preference information for generating preference information about the contributor by associating the estimated subjective preference polarity and degree with a lower type of the object in the image identified by the selected classifier An information processing apparatus comprising generation means. The content data includes a viewer comment by a viewer,
The extraction means extracts the viewer comment from the content data,
The language analysis means performs language processing on the viewer comment to extract an adjective and an adverb,
The preference estimation means estimates the polarity and degree of objective preference regarding the poster viewed from the viewer based on an adjective or adverb extracted from the viewer comment,
The information processing apparatus according to claim 1, wherein the preference information generation unit generates the preference information from the polarity and degree of the subjective preference and the polarity and degree of the objective preference.   The information processing apparatus according to claim 1, wherein the preference information generation unit integrates the preference information into the poster's profile downloaded from the server.   The information processing apparatus according to claim 1, wherein the preference estimation unit estimates the polarity and degree of the subjective preference of the poster by referring to the age and sex in the poster profile downloaded from the server. A partial image cutout unit that cuts out a partial image including the object included in the image;
The information processing apparatus according to claim 1, wherein the selected classifier identifies a type of an object in the image for the partial image. When the image includes a plurality of objects, the partial image cutout means cuts out a plurality of partial images,
The information processing apparatus according to claim 5, further comprising an image selection unit that selects the specific partial image from the plurality of partial images according to a position of the noun or the object extracted from the contributor comment.   The information processing apparatus according to claim 1, wherein the discriminator is configured by a convolutional neural network.   The information processing apparatus according to claim 1, wherein the preference estimation unit is configured by a neural network. The image is a moving image, further comprising means for selecting a specific still image based on the number of actions from the viewer from a plurality of still images constituting the moving image,
The information processing apparatus according to claim 1, wherein the selected discriminator identifies a lower type related to an object included in the selected still image. The content data includes audio data,
Voice recognition means for converting the voice data into text by voice recognition processing;
The information processing apparatus according to claim 1, wherein the language analysis unit performs language processing on the converted text to extract nouns, adjectives, and adverbs. Content data including an image posted by a contributor to a server that provides an SNS service, an EC service, or a video posting browsing service and a contributor comment by the contributor attached thereto are downloaded from the server, and the contributor's preferences In an information processing device that analyzes trends,
Means for analyzing the image to identify the type of object in the image;
An information processing apparatus comprising: means for estimating a preference tendency for the identified object type of the poster based on the poster comment. Content data including an image posted by a contributor to a server that provides an SNS service, an EC service, or a video posting browsing service and a contributor comment by the contributor attached thereto are downloaded from the server, and the contributor's preferences In a preference trend analysis method for analyzing trends,
Extracting the image and the contributor comment from the content data;
The poster comment is processed in language to extract nouns, adjectives, adverbs,
Estimating the subjective preference polarity and degree of the poster based on the extracted adjectives or adverbs,
In order to classify the types related to the object into multiple hierarchies and to identify lower types from the types of the respective hierarchies, the plurality of classifiers individually prepared for each of the plural hierarchies and the plural types, the post Select a specific classifier based on the noun extracted from the comment
A preference tendency to generate preference information about the contributor by associating the estimated subjective preference polarity and degree to a lower type of the object in the image identified by the selected classifier analysis method. Computer
Means for downloading content data including an image posted by a contributor to a server that provides an SNS service, an EC service, or a video posting browsing service, and a contributor comment by the contributor attached thereto from the server;
Means for extracting the image and the contributor comment from the content data;
An output for extracting nouns, adjectives, adverbs by language-processing the contributor comments;
Means for estimating the polarity and degree of subjective preferences of the contributor based on the extracted adjectives or adverbs;
In order to classify the types related to the object into multiple hierarchies and to identify lower types from the types of the respective hierarchies, the plurality of classifiers individually prepared for each of the plural hierarchies and the plural types, the post Means for selecting a specific classifier based on nouns extracted from the person's comments;
As means for generating preference information about the contributor by associating the estimated subjective preference polarity and degree with a lower type of the object in the image identified by the selected classifier Preference trend analysis program to make it function.

Images