JP6337183B1 - Text extraction device, comment posting device, comment posting support device, playback terminal, and context vector calculation device - Google Patents

Abstract

A text group suggesting a context similar to a target text group is extracted. A text extraction apparatus includes a context vector calculation unit, a similarity calculation unit, and a search unit. The context vector calculation unit 102 calculates a first context vector of a target text group including a plurality of texts using a machine-learned neural network. The similarity calculation unit 104 calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector. The search unit 107 searches for one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups. [Selection] Figure 1

Description

  The present invention relates to a technique for extracting text suitable for the context of content.

  Conventionally, in some moving image distribution services, a user can post a comment on a moving image. The comment is superimposed on the moving image and contributes to enhancing the sense of unity between users who view the same moving image. Also, the number of comments posted on a moving image can be used as one index for evaluating the excitement (popularity) of the moving image.

  A moving image that has just been distributed may have a small number of comments and lack of excitement, and the number of playback may be difficult to increase. In addition, users can easily post comments on videos that have already been posted with a lot of comments, but users tend to hesitate to post comments on videos that have not yet been posted. It is. The user typically inputs text by directly operating an input device such as a keyboard (including a software keyboard), a mouse, a touch screen, or a remote controller, and posts a comment. Therefore, the user may not actively post comments unless the input device can be comfortably operated.

  For example, if a standard comment such as “www” or “B” is automatically generated and posted, the number of comments in the moving image can be easily increased. However, a comment suitable for the context of the moving image that invites the user's empathy rather than such a bland comment is required.

  Patent Document 1 discloses a technique for generating a vector from a normal comment for target moving image data and extracting another normal comment similar to the vector ([0042] to [0043]). It is also disclosed to extract a regular comment for moving image data different from the target moving image data ([0050]).

JP2015-220610A

  An object of the present invention is to extract a text group suggesting a context similar to the target text group.

  According to the first aspect of the present invention, the text extraction device includes a context vector calculation unit, a similarity calculation unit, and a search unit. The context vector calculation unit calculates a first context vector of a target text group including a plurality of texts using a machine-learned neural network. The similarity calculation unit calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector. The search unit searches for one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups.

  According to the second aspect of the present invention, the comment posting device includes a context vector calculation unit, a similarity calculation unit, a search unit, and a transmission unit. The context vector calculation unit calculates a first context vector of a target text group including a plurality of texts associated with content using a machine-learned neural network. The similarity calculation unit calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector. The search unit searches for one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups. The transmission unit transmits at least one of the texts included in the similar text group for posting as a comment associated with the content.

  According to the third aspect of the present invention, the comment posting support device includes a context vector calculation unit, a similarity calculation unit, a search unit, and a transmission unit. The context vector calculation unit calculates a first context vector of a target text group including a plurality of texts associated with content using a machine-learned neural network. The similarity calculation unit calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector. The search unit searches for one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups. The transmission unit transmits at least one of the texts included in the similar text group as a postable comment to a playback terminal that plays back the content.

  According to the fourth aspect of the present invention, the reproduction terminal includes a reception unit, a reproduction unit, a context vector calculation unit, a similarity calculation unit, a search unit, and a transmission unit. The receiving unit receives content. The playback unit plays back content. The context vector calculation unit calculates a first context vector of a target text group including a plurality of texts associated with content using a machine-learned neural network. The similarity calculation unit calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector. The search unit searches for one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups. The transmission unit transmits at least one of the texts included in the similar text group for posting as a comment associated with the content.

  According to the fifth aspect of the present invention, the reproduction terminal includes a reception unit, a reproduction unit, a context vector calculation unit, a similarity calculation unit, a search unit, an output unit, and a transmission unit. The receiving unit receives content. The playback unit plays back content. The context vector calculation unit calculates a first context vector of a target text group including a plurality of texts associated with content using a machine-learned neural network. The similarity calculation unit calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector. The search unit searches for one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups. The output unit outputs at least one of the texts included in the similar text group as a postable comment together with the reproduced content. The transmission unit transmits the postable comment selected by the user as a comment associated with the content.

  According to the sixth aspect of the present invention, the context vector calculation device includes a context vector calculation unit. The context vector calculation unit calculates a context vector of a target text group including a plurality of texts using a machine-learned neural network. In the neural network, a result of machine learning performed using a plurality of learning data is set. Each of the plurality of learning data includes a text group pair as input data, and teacher data indicating whether the text group pair suggests a similar context or a similar context.

  According to the present invention, it is possible to extract a text group that suggests a context similar to the target text group.

The block diagram which illustrates the text extraction device concerning an embodiment. The figure which shows the example of a display screen in a user's reproduction | regeneration terminal. The figure which illustrates the object text group extracted from the display screen of FIG. Explanatory drawing of the context vector of the object text group calculated by the context vector calculation part of FIG. The figure which illustrates the similar text extracted based on the object text group of FIG. The figure which illustrates the similar text group extracted based on the object text group of FIG. The flowchart which illustrates operation | movement of the text extraction apparatus of FIG. The block diagram which illustrates the comment contribution apparatus containing the text extraction apparatus of FIG. FIG. 2 is a block diagram illustrating a playback terminal including the text extraction device of FIG. 1. FIG. 10 is a block diagram illustrating a content distribution system including the comment posting device in FIG. 8 and the playback terminal in FIG. 9.

  Hereinafter, embodiments will be described with reference to the drawings. Hereinafter, elements that are the same as or similar to elements already described are denoted by the same or similar reference numerals, and redundant descriptions are basically omitted.

(Embodiment)
A text extraction apparatus according to an embodiment of the present invention uses a machine-learned NN (Neural Network) to quantify a context suggested by an input text group including a plurality of texts (hereinafter referred to as a target text group). Compute the context vector

  In the present embodiment, the text may mean an arbitrary information unit composed of characters such as a word, a phrase, a sentence, and a sentence.

  The target text group can be extracted from, for example, a plurality of comments displayed in synchronization with the reproduction of content distributed in a content distribution service, typically a moving image.

  Here, being displayed in synchronization with the reproduction of the content means that the comment is displayed at a temporal position (time) determined by the reproduction position of the content. The text extracting device searches for a candidate text group having a context vector similar to the context vector of the target text group (hereinafter referred to as a similar text group).

  Thereby, this text extraction apparatus can extract the similar text group which suggests a context similar to the target text group from the plurality of candidate text groups. The extracted similar text group may be used, for example, to automatically post as a new comment to the extracted content of the target text group, or can be posted to support comment posting by a user who views the content It may be used as a comment.

  In addition to the comment or instead of the comment, the same processing may be performed on text related to metadata attached to content such as a title, an explanation, a tag, and a category. However, the title, description, tag, and category are assigned to the entire content, whereas the comment is assigned to a limited part of the content, so the local context of the content is reflected. Easy to be.

  As illustrated in FIG. 1, the text extraction apparatus 100 according to the embodiment includes a text group input unit 101, a context vector calculation unit 102, an NN 103, a similarity calculation unit 104, and a candidate text group storage unit 105. And a similarity storage unit 106 and a similar text group search unit 107.

  The text extraction apparatus 100 is typically a computer, but is not limited thereto. The text extraction apparatus 100 includes a processor for calculating a context vector, calculating a similarity, searching for a similar text group, and the like, for example, a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), in order to implement such processing. A memory for temporarily storing a program executed by the processor and data used by the processor;

  The text extraction device 100 can further use a communication device for connecting to a network, an input device for receiving user input, and an auxiliary storage device for storing programs or data. These communication device, input device, and auxiliary storage device may be built in the text extraction device 100 or may be externally attached to the text extraction device 100.

  The communication device communicates with external devices such as a Web server, a content distribution server, and a comment distribution server via a network.

  The input device may be, for example, a keyboard, a mouse, a numeric keypad, a remote controller, a microphone, or the like, or may have a display device function such as a touch screen. User input may typically be text input or voice input. The voice input can be converted into text using ASR (Automatic Speech Recognition).

  The auxiliary storage device stores, for example, a program or data. The auxiliary storage device is preferably a non-volatile storage medium such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive). The auxiliary storage device may be a file server connected to the text extraction device 100 via a network.

  The text group input unit 101 acquires the target text group and sends it to the context vector calculation unit 102. The text group input unit 101 can acquire the target text group by the following methods (1) to (3) or other methods. The text group input unit 101 may be the above-described input device or communication device, or an interface with these, or the above-described processor and memory.

  (1) The text group input unit 101 may acquire a text group input by the user by operating the above-described input device, for example. If the similar text group extracted based on the target text group is used for automatic posting or used as a postable comment for supporting comment posting, in addition to the comment group directly input, Comments that suggest similar contexts can be posted. Thereby, the user can post many comments suitable for the context of the content he / she wants to excite with a light input load.

  (2) The text group input unit 101 may acquire, for example, at least a part of a comment group displayed in synchronization with the reproduction of the content as the target text group during the reproduction of the content. The text group input unit 101 reads a comment group from, for example, a buffer (memory). If the similar text group extracted based on the target text group is used as a postable comment for supporting comment posting, a postable comment suitable for the context of the content being reproduced can be presented to the user. Thereby, even if a user does not think about a sentence by himself / herself, he / she can post a comment only by selecting a favorite postable comment presented. That is, the user's psychological hurdle for comment posting can be lowered.

  The text group that can be extracted may be limited to a comment displayed at a position (display time) included in a time window set based on the reproduction position of the content at the time of extraction. A group of text extracted in this way is likely to suggest a context within this time window of the content.

  FIG. 2 shows an example of a display screen on the user's playback terminal. In the moving image display area, a comment is displayed in addition to the moving image. Each comment has a playback position (display time) at which the comment is displayed. In the example of FIG. 2, an area for displaying a table in which comments and their display times can be listed is prepared on the right side of the moving image display area. On the upper side of the moving image display area, for example, an area for displaying metadata such as a moving image title, a description, a moving image category indicating the type of moving image, a moving image tag that is a keyword assigned to the moving image by the user is prepared Yes. On the other hand, an area for displaying a seek bar indicating the current playback position of the moving image and a button for playback control such as playback / stop is prepared below the moving image display area.

  In the example of FIG. 2, if a time window is set for 5 seconds immediately before the current playback time (00:01:00), for example, the text group input unit 101, as illustrated in FIG. 4 comments, “Wow”, “Number of attacks at night”, “Waste Charge Soldier Domeme w” and “What is your turn?” Can be extracted as the target text group. Note that “sports”, which is a moving image category, may be included as a part of the target text group, or may be treated as additional content attribute information different from the target text group.

  (3) The text group input unit 101 receives, for example, information specifying a specific content distributed in the content distribution service, acquires a text group associated with the content from the comment distribution server (for example, receives it via a network). Also good. If a similar text group extracted based on such a target text group is used for automatic posting, it is possible to post a number of comments suitable for the context of the particular content and excite them. The specific content may be, for example, newly posted content, content where the number of comments is sluggish, content where the number of comments has started to decline, and the like.

  The text group that can be extracted may be limited to a comment displayed at a position (display time) included in a time window set based on a specific reproduction position of the content. A group of text extracted in this way is likely to suggest a context within this time window of the content. The specific playback position may be explicitly specified together with the specific content, may be determined in advance, or may be automatically determined based on, for example, the distribution of the number of comments.

  The context vector calculation unit 102 receives the target text group from the text group input unit 101, and calculates the context vector of the target text group using the machine-learned NN 103. For example, the context vector calculation unit 102 gives each text included in the target text group to the NN 103, receives the context vector for each text from the NN 103, and synthesizes them to calculate the context vector of the target text group. May be. The context vector calculation unit 102 sends the context vector of the target text group to the similarity calculation unit 104. The context vector calculation unit 102 may be the aforementioned processor and memory.

  NN 103 is a neural network in which a learning result (learned model) of machine learning for acquiring the ability to convert input text into a context vector is set. This machine learning will be described later. The NN 103 may be the aforementioned processor (usually a GPU) and memory.

  Here, it is assumed that the target text group T is a set of n texts t [1], t [2], ..., t [n]. n is an arbitrary natural number equal to or greater than 2, and may be a variable value. In the example of FIG. 3, n = 4. The texts t [1], t [2],..., T [n] that are elements of the target text group T are respectively given to the machine-learned NN 103 (g) as shown in FIG. , Context vectors g [1], g [2],..., G [n]. The context vector is a d-dimensional real vector, and d can be determined by design.

  The context vector calculation unit 102 calculates the context vector g [T] of the target text group T by synthesizing these context vectors g [1], g [2],..., G [n]. Combining refers to an operation that integrates a plurality of context vectors into one. Specifically, the sum of a plurality of context vectors may be simply calculated, or a convolution operation may be performed using a plurality of context vectors. In the following description, the context vector g [T] of the target text group T may be referred to as a first context vector for convenience.

  The similarity calculation unit 104 receives the first context vector from the context vector calculation unit 102 and reads the context vectors of a plurality (group M) of candidate text groups from the candidate text group storage unit 105. In the following description, the context vector of the candidate text group may be referred to as a second context vector for convenience.

M, which is the number of candidate text groups, is an arbitrary natural number greater than or equal to 2, but it is preferable that it is large to some extent because it defines the search range of similar text groups. These candidate text groups can be collected by, for example, extracting a text group associated with a large amount of content in advance by the same extraction method as the target text group. The number of texts included in each candidate text group is an arbitrary natural number, may be disjoint, or all may be the same. Further, the context vectors of these candidate text groups can be calculated in advance by the same calculation method as that for the first context vector. These candidate text groups S [1], S [2],..., S [M] and their context vectors g [S [1]], g [S [2]],. With [M]], a corpus for searching for similar text groups is constructed as follows.
Corpus = {(S [1], g [S [1]]), (S [2], g [S [2]]), ..., (S [M], g [S [M]] )}
The similarity calculation unit 104 calculates the similarity of each of the M second context vectors with respect to the first context vector. Similarity may be any index for evaluating similarity between vectors, typically cosine similarity, or Euclidean distance between normalized context vectors. The cosine similarity of the second context vector g [S [i]] of the i-th candidate text group with respect to the first context vector g [T] is cosine (arg (g [T], g [S [i ]])). i is an arbitrary natural number of M or less. The similarity calculation unit 104 associates the similarity between the first context vector and the second context vector of the candidate text group with information for identifying each candidate text group (for example, the value of i), and a similarity storage unit Save to 106. The similarity calculation unit 104 may be the above-described processor and memory.

  Note that it is not essential to calculate the similarity for all of the M second context vectors. For example, Local Sensitive Hashing (LSH) is known as an algorithm for rapidly extracting a pair of similar instances from a large amount of data. This algorithm uses a hash function defined to input a vector and output a scalar value. This hash function has a feature that the same scalar value is easily returned when a similar vector is input. Therefore, scalar values for the M second context vectors are calculated in advance and stored in the candidate text group storage unit 105, and the scalar value of the first context vector is calculated using a hash function. The second context vector associated with the same scalar value may be extracted and then given to the similarity calculation unit 104. The similarity calculation unit 104 can greatly reduce the amount of calculation by calculating the similarity only for the extracted second context vector.

  The above-mentioned corpus is constructed in the candidate text group storage unit 105, and the corpus or the second context vector is read out by the similarity calculation unit 104. The candidate text group corresponding to the similar text group is read by the similar text group search unit 107. The candidate text group storage unit 105 may be the auxiliary storage device described above.

  In the similarity storage unit 106, the similarity (and information for identifying the corresponding candidate text group) is written by the similarity calculation unit 104, and this is read by the similar text group search unit 107. The similarity storage unit 106 may be the memory described above.

  Based on the similarity stored in the similarity storage unit 106, the similar text group search unit 107 searches for a similar text group that is a candidate text group similar to the target text group and the context vector. Specifically, the similar text group search unit 107 may search for a candidate text group that is most similar with respect to the target text group and the context vector, or a predetermined number of candidate text groups in order of similarity with respect to the target text group and the context vector. Or a candidate text group whose similarity satisfies a predetermined numerical condition may be searched.

  It should be noted that depending on the definition of the similarity, the smaller the similarity, the more similar the first context vector and the second context vector may mean. Specifically, the cosine similarity means that the larger the value is, the more similar the first context vector and the second context vector are, but the Euclidean distance between the normalized context vectors is A smaller value means that the first context vector and the second context vector are more similar.

  The similar text group search unit 107 reads one or more searched similar text groups from the candidate text group storage unit 105 and outputs them. The similar text group search unit 107 may be the above-described processor and memory.

  The output similar text group may be used for automatically posting as a new comment to the content extracted from the target text group, or as a postable comment for supporting comment posting by a user viewing the content May be used.

  A similar text group extracted based on the target text group in FIG. 3, more specifically a similar comment group, is illustrated in FIG. 6. In FIG. 6, five similar text groups are sorted in ascending order of the Euclidean distance between the normalized context vectors, that is, in an order similar to the target text group and the context vector.

  Note that instead of the candidate text group, the search may be performed in units of candidate text. As a result, it is possible to extract similar text that is candidate text similar to the target text group and the context vector. This is equivalent to the case where the number of texts included in each candidate text group is all one. Alternatively, once a similar text group is extracted, similar text may be further extracted from text belonging to the similar text group. Thereby, compared with the case where it searches by candidate text unit, the calculation amount accompanying the calculation of similarity and the search of similar text can be reduced.

  FIG. 5 illustrates similar texts extracted based on the target text group in FIG. 3, more specifically similar comments. In FIG. 5, five similar texts are sorted in ascending order of the Euclidean distance between the normalized context vectors, that is, in an order similar to the target text group and the context vector.

  In the example of FIGS. 5 and 6, the candidate text extracted from the content associated with the category “game” or “entertainment” different from the category “sports” associated with the content (moving image) from which the target text group is extracted. Group) is extracted as similar text (group). As described above, by using the similarity regarding the context vector as a reference, even candidate text (group) extracted from moving images with different categories can be used as a candidate for similar text (group).

  Hereinafter, machine learning for acquiring the ability to convert input text into a context vector will be described. The NN that executes this machine learning may be the same as or different from the NN 103. In any case, the learning result of the machine learning is set in the NN 103.

  In this machine learning, when the first text group and the second text group suggest a similar context, the context vector of the first text group and the context vector of the second text group are similar. Thus, for example, the aim is to calculate the context vector so that the cosine similarity is sufficiently larger than 0. On the other hand, when this machine learning suggests a context in which the first text group and the second text group are not similar, the context vector of the first text group and the context vector of the second text group The context vector is calculated so that the cosine similarity is sufficiently smaller than 1, for example.

  In order to achieve such a learning goal, a pair of text groups with similar suggested contexts is used as input data, and a similarity value (1 for cosine similarity) indicating that both text groups are similar is used. Similarity value (cosine similarity) indicating that the first learning data (possible data) used as teacher data and a text group whose context suggested by them is not similar are input data, and both text groups are dissimilar. If the degree is 0, machine learning is performed using second learning data (negative data) using 0) as teacher data. In order to allow the machine learning to calculate a valid context vector, it is preferable to arrange a sufficient number of each of the first learning data and the second learning data, but one of the learning data is sufficiently arranged. Even if it is not done, a corresponding effect can be expected.

  The input data included in the first learning data and the second learning data is, for example, an enormous amount of comment data accumulated by a provider of a moving image distribution service that allows a user to post a comment on a moving image. Can be extracted from.

  Pairs of text groups with similar suggested contexts can be extracted from comments posted on the same content. Note that the context can change depending on the playback position of the content. Therefore, by extracting a comment in the vicinity of an arbitrary playback position of the content and dividing it into two groups, pairs of text groups that are likely to suggest similar contexts can be obtained. Alternatively, pairs of text groups with similar suggested contexts can be extracted from comments posted on two content associated with the same metadata (eg, belonging to the same category).

  The pairs of text groups whose dissimilar contexts are similar can be extracted from comments posted on two contents respectively associated with different metadata (for example, belonging to different categories), for example.

  In either case, the comments are not crossed between the extracted text groups.

  By performing such machine learning, the learning target NN is posted to, for example, a pair of text groups extracted from comments posted to the same content, or two contents associated with the same metadata. It is possible to calculate a context vector so that the context vectors of both text groups are similar to a pair of text groups extracted from the comment. On the other hand, this NN has a context so that the context vectors of both text groups are not similar to a pair of text groups extracted from comments posted for two contents respectively associated with different metadata, for example. The vector can be calculated.

Hereinafter, an operation example of the text extraction apparatus 100 will be described with reference to FIG.
First, the text group input unit 101 acquires a target text group (step S201). The target text group can be obtained by various methods as described above. The context vector calculation unit 102 uses the machine-learned NN 103 to calculate the first context vector of the target text group acquired in step S201 (step S202).

  The similarity calculation unit 104 calculates the similarity of each of the second context vectors of the plurality of candidate text groups with respect to the first context vector calculated in step S202 (step S203). Based on the similarity calculated in step S203, the similar text group search unit 107 searches for a similar text group that is a candidate text group similar to the target text group and the context vector (step S204), and the process ends.

  The text extraction apparatus 100 of FIG. 1 may be incorporated in the comment posting apparatus 300 as illustrated in FIG. This comment posting device 300 includes a receiving unit 301, a text extracting device 100, and a transmitting unit 302. This comment posting device 300 automatically generates and posts a comment associated with the content.

  The comment posting device 300 is typically a computer, but is not limited thereto. The comment posting apparatus 300 includes a text extraction process corresponding to the function of the text extraction apparatus in FIG. 1 and a processor that performs communication control, for example, a CPU and a GPU, a program executed by the processor to realize such a process, and And a memory for temporarily storing data used by the processor.

  The comment posting apparatus 300 can further use a communication apparatus for connecting to a network. The communication device communicates with external devices such as a reproduction terminal, a content distribution server, and a comment distribution server via a network. The communication device may be built in the comment posting device 300 or may be externally attached to the comment posting device 300.

  For example, the receiving unit 301 receives information for specifying content to be posted. Such content may be intentionally selected by the user or the operator of the content distribution service, or may be automatically selected according to some algorithm. The receiving unit 301 can also receive a plurality of texts associated with such content, that is, candidates for the target text group or the target text group itself. Here, the plurality of texts may include metadata such as comments, titles, explanations, tags, and categories associated with the content. Note that if the comment posting device 300 in FIG. 8 is incorporated in a server that distributes metadata such as comments attached to content, it is not necessary to receive the metadata from an external device. The receiving unit 301 sends information specifying the content to the text extraction device 100. The receiving unit 301 may be the communication device described above or an interface with the communication device.

  The text extraction apparatus 100 extracts a plurality of texts associated with the content specified by the information received from the receiving unit 301 as a target text group, and then searches for a similar text group that is similar to the target text group with respect to a context vector. The text extraction apparatus 100 sends at least one of the texts included in the similar text group to the transmission unit 302.

  The transmission unit 302 receives at least one text from the text extraction device 100 and transmits it as a comment associated with the content, for example, to a comment distribution server for posting. The transmission unit 302 may be the communication device described above or an interface with the communication device.

  Note that the transmission unit 302 may transmit at least one text received from the text extraction device 100 as a postable comment to a playback terminal that plays back content. According to such a modification, the comment posting device 300 in FIG. 8 can be configured as a comment posting support device that supports comment posting by the user.

  The text extraction apparatus 100 of FIG. 1 may be incorporated in the playback terminal 400 as illustrated in FIG. The playback terminal 400 includes a reception unit 401, a buffer 402, a playback unit 403, an output unit 404, a user input unit 405, a text extraction device 100, and a transmission unit 406.

  The playback terminal 400 includes (1) a function for automatically generating and posting a comment on the content based on the text input by the user regarding the content being played back, and (2) text input by the user regarding the content being played back, Or, based on the received text associated with the content being played, automatically generate a postable comment and output it together with the content, and / or actually post the postable comment according to the user's selection Prepare.

  The playback terminal 400 is typically a television receiver (including Internet television), a PC (Personal Computer), a mobile terminal (eg, tablet, smartphone, laptop, feature phone, portable game machine, digital music player, electronic Book reader, etc.), VR (Virtual Reality) terminal, and AR (Augmented Reality) terminal, but are not limited thereto.

  The playback terminal 400 performs content playback control, output control for an output device connected to the playback terminal 400, data processing for user input, text extraction processing corresponding to the function of the text extraction device in FIG. 1, communication control, and the like. It includes a processor to perform, for example, a CPU and a GPU, and a memory that temporarily stores a program executed by the processor and data used by the processor to realize such processing.

  The playback terminal 400 further uses a communication device for connecting to a network, an output device for outputting content, an input device for receiving user input, and an auxiliary storage device for storing programs or data. Is possible. These communication device, output device, input device, and auxiliary storage device may be built in the playback terminal 400 or may be externally attached to the playback terminal 400.

  The communication device communicates with external devices such as a Web server, a content distribution server, a comment distribution server, and a comment posting device 300 via a network.

  The output device may include a display device for displaying moving images, still images, text, and / or a speaker for outputting sound, music, and the like. Examples of the display device include a liquid crystal display, an organic EL (electroluminescence) display, and a CRT (Cathode Ray Tube) display. The display device displays display data including content. Note that the display device may have a function of an input device like a touch screen.

  The input device may be, for example, a keyboard, a mouse, a numeric keypad, a remote controller, or the like, or may have a display device function like a touch screen. User input may typically be a tap, a click, a specific key press, and the like. In addition, user input includes, for example, voice captured by a microphone, biological data detected by a biological sensor (for example, body temperature, facial expression, etc.), GPS (Global Positioning System) or position data identified by base station information, an acceleration sensor The user action estimated based on the acceleration data detected by (for example, the playback terminal 400 is swung) may be included.

  The auxiliary storage device stores, for example, a program or data. The auxiliary storage device is preferably a non-volatile storage medium such as an HDD or an SSD. The auxiliary storage device may be a file server connected to the playback terminal 400 via a network.

  The receiving unit 401 receives content to be reproduced from, for example, a content distribution server. The content to be played back may be intentionally selected by the user or may be automatically selected according to some algorithm. The receiving unit 401 sends the received content to the buffer 402. The receiving unit 401 may further receive metadata attached to the content to be played back, for example, from a comment distribution server or other metadata distribution server. This metadata may include text such as comments, titles, descriptions, tags, categories, etc. associated with the content to be played. When receiving the metadata, the receiving unit 401 sends the received metadata to the buffer 402. The receiving unit 401 may be the communication device described above or an interface with the communication device.

  The buffer 402 temporarily stores data from the receiving unit 401, that is, received content and / or received metadata. The buffer 402 sends the stored data to the playback unit 403 in response to a request from the playback unit 403. At this time, the buffer 402 may send the stored metadata to the text extraction apparatus 100 as well. The buffer 402 can be the memory described above.

  The playback unit 403 reads data from the buffer 402 as necessary. The playback unit 403 plays back the content read from the buffer 402 and sends the played content to the output unit 404 at a predetermined timing. Content playback includes, for example, decoding of content such as encoded video data and audio data, generation of display data based on the decoded video data and metadata, such as a process of combining a comment with the decoded video data, etc. May be included. The playback unit 403 may receive a user input related to content playback control from the user input unit 405, and may playback / stop the content or jump the content playback state to a designated playback position in response to the user input. . The playback unit 403 may be the above-described processor and memory.

  The output unit 404 outputs the content reproduced from the reproduction unit 403, for example, display data and / or audio data. Furthermore, the output unit 404 may receive at least one text as a postable comment from the text extraction device 100 and output it together with the content. The output unit 404 may be the above-described output device or an interface with the output device.

  The user input unit 405 receives various user inputs. For example, the user input unit 405 may receive a user input related to content playback control and send it to the playback unit 403. The user input unit 405 may receive a user input for posting a text associated with the content being reproduced, and may send this to the transmission unit 406. Such user input is not limited to direct text input, and may be selection input to a postable comment output by the output unit 404. Further, the user input unit 405 may send the text directly input by the user for posting as text associated with the content being played back to the text extraction apparatus 100 as a target text group candidate. The user input unit 405 may be the above-described input device or an interface with the input device.

  The text extraction device 100 may receive the metadata attached to the content being played back from the buffer 402, or receives the text directly input by the user for posting as text associated with the content from the user input unit 405. May be. In any case, the text extraction apparatus 100 searches for a similar text group that is similar to the target text group with respect to the context vector, and sends at least one of the texts included in the similar text group to the output unit 404 as a postable comment. Or send it to the transmission unit 406 for comment posting.

  The transmission unit 406 receives at least one text from the text extraction device 100 and transmits it to a comment distribution server, for example, in order to post it as a comment associated with the content being reproduced. Further, the transmission unit 406 may receive text directly input by the user, and transmit the text to a comment distribution server or other metadata distribution server, for example, in order to post it as text associated with the content being reproduced. . The transmission unit 406 may be the communication device described above or an interface with the communication device.

  The comment posting apparatus 300 in FIG. 8 and the playback terminal 400 in FIG. 9 can be incorporated into the content distribution system illustrated in FIG. This content distribution system includes a comment posting device 300, a playback terminal 400, a Web server 501, a content distribution server 502, and a comment distribution server 503.

  The apparatuses in FIG. 10 are connected to each other via a network and can transmit and receive data to and from each other. Note that the number of devices in FIG. 10 is not limited to one and may be plural. The server configuration illustrated in FIG. 10 is merely an example, and the function of one server may be shared by a plurality of devices, or the functions of a plurality of servers may be assigned to one device.

  When accessed from the playback terminal 400 via the network, the Web server 501 provides the playback terminal 400 with a web page including a list in which content that can be distributed from the content distribution system of FIG. 10 is listed.

  The user of the playback terminal 400 selects content desired to be distributed from a list in the web page displayed on the playback terminal 400 by performing a predetermined operation such as tap, click, or pressing a specific key, for example. . When the reproduction terminal 400 detects such a predetermined operation, the reproduction terminal 400 transmits a distribution request for the content selected by the user to the Web server 501 via the network.

  When the Web server 501 receives a content distribution request from the playback terminal 400 via the network, the Web server 501 receives information (typically, URL (Uniform Resource Locator)) for acquiring the content, and a comment (and Information (typically URL) for acquiring (/ other metadata) is returned to the playback terminal 400 via the network.

  When the playback terminal 400 receives the above-described information from the Web server 501 via the network, the playback terminal 400 accesses the content distribution server 502 via the network using the information for acquiring the content and uses the information for acquiring the comment. The comment delivery server 503 is accessed via the network.

  When the content distribution server 502 and the comment distribution server 503 receive content and comment distribution requests from the playback terminal 400 via the network, respectively, the content distribution server 502 and the comment distribution server 503 receive the content and comments associated with the information used by the playback terminal 400. Over the network.

  Upon receiving a content distribution request from the playback terminal 400 via the network, the content distribution server 502 identifies the content that is the target of the distribution request and distributes the content to the playback terminal 400 via the network. The content distribution server 502 typically stores a large amount of recorded moving image data in association with information (typically URLs) for acquiring each content in the storage unit.

  Note that the content distribution server 502 may have a live distribution function. That is, the content distribution server 502 sequentially receives moving image data transmitted from a transmission source (not shown) (for example, a computer connected to a video camera) via a network, and performs the video processing as it is or after performing necessary processing. The image data may be distributed to the playback terminal 400 that is playing back via a network. The text extraction apparatus 100 can also handle text associated with such live-distributed content as a target text group. In addition, the comment posting device 300 and the playback terminal 400 can automatically post or support a comment associated with the live delivery content.

  When the comment distribution server 503 receives a comment data distribution request from the playback terminal 400 via the network, the comment distribution server 503 identifies the comment that is the target of the distribution request and distributes it to the playback terminal 400 via the network. The comment distribution server 503 stores a comment associated with each of the contents that can be distributed from the content distribution server 502 and information (typically, a URL) for acquiring the comment in association with each other.

  When the comment distribution server 503 receives a new comment posted by the playback terminal 400 or the comment posting device 300 via the network, the comment distribution server 503 adds the new comment and updates the comment associated with the content. The comment distribution server 503 may automatically distribute the updated comment via the network to the playback terminal 400 that has already distributed the comment before the update.

  The playback terminal 400 displays a web page provided from the web server 501 via the network, detects an operation of selecting content made by the user while the web page is displayed, and sends a delivery request for the content to the web server 501. Over the network.

  The playback terminal 400 receives information for acquiring the content and information for acquiring a comment associated with the content from the Web server 501 via the network, and uses the information to transmit the content distribution server 502 and the comment. Each distribution server 503 is accessed via a network.

  Note that the latest comment data associated with the content may be distributed from the comment distribution server 503 while the reproduction terminal 400 is reproducing the content. In this case, the playback terminal 400 may discard the old comment and perform content playback control using the latest comment.

  The comment posting device 300 automatically posts a comment to the comment distribution server 503 in response to a request from the playback terminal 400 or another device.

  In the content distribution system of FIG. 10, the comment posting device 300 and the playback terminal 400 may be omitted, simplified, or modified. For example, the comment posting device 300 may be omitted or may be modified to the above-described comment posting support device. In the latter case, the comment posting support device may send a postable comment to the playback terminal 400 in response to a request from the playback terminal 400 or another device. In addition, the playback terminal 400 includes (1) the function of automatically generating and posting a comment on the content based on the text input by the user regarding the content being played back, and (2) the user regarding the content being played back. A function that automatically generates a postable comment based on the text entered by or received text associated with the content being played and outputs it together with the content, and actually posts the postable comment according to the user's selection, One or both of them may not be provided.

  As described above, the text extraction apparatus according to the embodiment calculates a context vector that quantifies a context suggested by a target text group including a plurality of texts. Then, the text extraction device searches for a similar text group having a context vector similar to the context vector of the target text group among the plurality of candidate text groups. Therefore, according to this text extraction device, a similar text group suggesting a context similar to the target text group can be extracted from a plurality of candidate text groups. The extracted similar text group may be used, for example, to automatically post as a new comment to the extracted content of the target text group, or can be posted to support comment posting by a user who views the content It may be used as a comment.

(Modification 1)
The text extraction apparatus according to the embodiment can also be used to check whether the context suggested by the target text group is similar to the context suggested by the text of interest. In such a modification, the context vector of the text of interest is used as the second context vector. For example, if the similarity is greater than or equal to a threshold, the context suggested by the target text group is similar to the context suggested by the text of interest. Determined. The text of interest may be, for example, an NG word or other regulated text. If the target text group suggests a context similar to the interest text, notify the operator of the content distribution service, refuse to post the target text group, or if the target text group has already been posted This may be deleted, or use of the content distribution service by the posting user, for example, server access may be restricted.

(Modification 2)
Using content attribute information, determine a comment that is automatically posted as a new comment on the content, determine a postable comment that is presented to the user to support comment posting by the user viewing the content, The first context vector may be corrected. The content attribute information includes local features of the content (for example, a specific person, character, thing, place, etc. It may be whether a specific music piece, melody, sound effect, or the like is flowing), or which part of the entire content corresponds to the time position (for example, immediately after the start or just before the end). Also, metadata such as tags and categories can be used as content attribute information here.

(Modification 3)
It is also possible to configure the context vector calculation device by paying attention to the context vector calculation unit and NN in the text extraction device according to the embodiment. According to this context vector calculation device, it is possible to calculate a context vector that quantifies the context formed by an arbitrary text group. The context vector may be used for various applications such as automatic classification of large-scale comments and automatic analysis of the context structure of content.

  The above-described embodiments are merely specific examples for helping understanding of the concept of the present invention, and are not intended to limit the scope of the present invention. The embodiment can add, delete, or convert various components without departing from the gist of the present invention.

  The various functional units described in the above embodiments may be realized by using a circuit. The circuit may be a dedicated circuit that realizes a specific function, or may be a general-purpose circuit such as a processor.

  At least a part of the processing of each of the above embodiments can also be realized by using a general-purpose computer as basic hardware. A program for realizing the above processing may be provided by being stored in a computer-readable recording medium. The program is stored in the recording medium as an installable file or an executable file. Examples of the recording medium include a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), and a semiconductor memory. The recording medium may be any recording medium as long as it can store the program and can be read by the computer. The program for realizing the above processing may be stored on a computer (server) connected to a network such as the Internet and downloaded to the computer (client) via the network.

DESCRIPTION OF SYMBOLS 100 ... Text extraction apparatus 101 ... Text group input part 102 ... Context vector calculation part 103 ... NN
104: Similarity calculation unit 105 ... Candidate text group storage unit 106 ... Similarity storage unit 107 ... Similar text group search unit 300 ... Comment posting device 301, 401 ... Reception unit 302 , 406 ... Transmitter 400 ... Playback terminal 402 ... Buffer 403 ... Playback unit 404 ... Output unit 405 ... User input unit 501 ... Web server 502 ... Content distribution server 503: Comment distribution server

Claims (12)

A context vector calculation unit that calculates a first context vector of a target text group including a plurality of mutually independent texts using a machine-learned neural network;
A similarity calculation unit that calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector;
A search unit that searches one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups; And
The plurality of texts included in the target text group are texts associated with first content,
The one or more texts included in each of the candidate text groups are texts associated with the first content or content different from the first content.
Text extraction device.   The context vector calculation unit applies each of a plurality of texts included in the target text group to the neural network, calculates a context vector for each text, and synthesizes the context vector for each text, thereby combining the first text. The text extraction apparatus according to claim 1, wherein a context vector is calculated.   The at least one of the texts included in the target text group is extracted from at least one comment whose display time is determined in synchronization with the reproduction of the first content. Text extractor.   At least one of the texts included in the target text group has a specific reproduction position of the first content among at least one comment whose display time is determined in synchronization with the reproduction of the first content. The text extracting device according to claim 1, wherein the text extracting device is extracted from a comment whose display time is determined within a time window set as a reference.   When the candidate text group corresponding to the second context vector that is most similar to the first context vector includes a plurality of texts, the search unit includes a context vector for each text included in the candidate text group. 5. The method according to claim 1, further searching for a thing most similar to the first context vector to obtain text corresponding to the context vector most similar to the first context vector. 6. Text extraction device. At least one of the texts included in the target text group is text related to content,
The content attribute information is used to correct the first context vector;
The text extraction device according to any one of claims 1 to 5. A context vector calculation unit that calculates a first context vector of a target text group including a plurality of texts associated with content using a machine-learned neural network;
A similarity calculation unit that calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector;
A search unit that searches one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups;
A comment posting device comprising: a transmission unit that transmits at least one of the texts included in the similar text group as a comment associated with the content. A context vector calculation unit that calculates a first context vector of a target text group including a plurality of texts associated with content using a machine-learned neural network;
A similarity calculation unit that calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector;
A search unit that searches one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups;
A comment posting support apparatus comprising: a transmission unit that transmits at least one of texts included in the similar text group as a postable comment to a playback terminal that plays back the content. A receiving unit for receiving content;
A playback unit for playing back the content;
A context vector calculation unit that calculates a first context vector of a target text group including a plurality of texts associated with the content using a machine-learned neural network;
A similarity calculation unit that calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector;
A search unit that searches one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups;
A playback terminal comprising: a transmitting unit that transmits at least one of the texts included in the similar text group as a comment associated with the content. A receiving unit for receiving content;
A playback unit for playing back the content;
A context vector calculation unit that calculates a first context vector of a target text group including a plurality of texts associated with the content using a machine-learned neural network;
A similarity calculation unit that calculates the similarity between each of the second context vectors of the plurality of candidate text groups each including one or more texts and the first context vector;
A search unit that searches one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector based on the similarity, and obtains similar text groups;
An output unit that outputs at least one of the texts included in the similar text group as a postable comment together with the reproduced content;
A playback terminal comprising: a transmission unit that transmits a postable comment selected by a user as a comment associated with the content. A context vector calculator that calculates a context vector of a target text group including a plurality of mutually independent texts using a machine-learned neural network;
The neural network is set with the result of machine learning performed using a plurality of learning data,
Wherein the plurality of training data, respectively, viewed contains a text group pair as input data, and a teacher data indicating whether said text group pairs or suggest the context of similar, or suggest context dissimilar,
The plurality of texts included in the target text group are texts associated with first content,
One or more texts included in each of the text groups as the input data are texts associated with the first content or content different from the first content.
Context vector calculator. Computer
Means for calculating a first context vector of a target text group including a plurality of mutually independent texts using a machine-learned neural network;
Means for calculating the similarity between each of the second context vectors of a plurality of candidate text groups each including one or more texts and the first context vector;
Based on the similarity, search for one or more candidate text groups respectively corresponding to one or more second context vectors similar to the first context vector, and function as means for obtaining a similar text group ,
The plurality of texts included in the target text group are texts associated with first content,
The one or more texts included in each of the candidate text groups are texts associated with the first content or content different from the first content.
Text extraction program.