JP6087704B2 - Communication service providing apparatus, communication service providing method, and program - Google Patents

Description

  The present invention relates to a communication service providing apparatus, a communication service providing method, and a program for estimating one scene suitable for communication from multimedia contents including an utterance.

  In general, video content and comic content are composed of utterances of characters that appear. Therefore, it is possible for a user to use the utterance of a character in these contents as his / her utterance in communication, and the affinity between communication and these contents is high. Already, in a bulletin board service or the like on the Internet, there is a case where a user uses one scene of these contents for communication in the form of an image or text without permission of the content holder.

  However, it has been a great effort for the user to find a scene that matches the development of communication from among a myriad of scenes of enormous content and use it for communication. In particular, since the comic content has a plurality of frames on one page and a plurality of utterances, it is difficult for the user to cut out one scene (one frame) and use it as his / her utterance for communication.

  Therefore, in the technique described in Patent Document 1, image processing is performed on a manga page and a frame border is recognized, so that one page is divided into a plurality of frames and one frame is set as one content. Data can be used in units. In the same document, a frame (content) is easily searched by extracting text in a balloon.

  On the other hand, as a system for searching for content, Google (registered trademark) image search is well known (for example, see Non-Patent Document 1). In Google image search, it is possible to search for images from text by creating an index by associating text (headings, image titles, explanations, etc.) existing around the images included in the content with image URLs. It is. Furthermore, a technique for improving the search function by extracting the features of the images themselves and associating the images has been proposed (see Non-Patent Document 2, for example).

JP 2011-238043 A

https: // www. Google. co. jp / imghp [Search April 4, 2013] Yushi Jing, and Scheme Baruja, "PageRank for Product Image Search", WWW 2008 / Referenced Track: Rich Media, 2008. [Search April 4, 2013]

  However, when one scene of video content or comic content is used for communication using the above-described technology, a search is performed using text included in one scene, and a user is appropriately selected from a large number of candidates obtained by the search. Work to select a scene is required. For this reason, smooth communication is difficult, and the real-time nature of communication is lost. In particular, on a narrow screen on a mobile phone, it is difficult to select an appropriate scene from a large number of candidates obtained by search, and smooth communication is almost impossible.

  Therefore, in view of the above problems, the present invention has an object of providing a communication service providing apparatus, a communication service providing method, and a program for estimating a scene suitable for the next utterance in communication from multimedia contents including the utterance. To do.

  The present invention proposes the following matters in order to solve the above problems. In addition, in order to make an understanding easy, although the code | symbol corresponding to embodiment of this invention is attached | subjected and demonstrated, it is not limited to this.

  (1) The present invention provides a communication service providing apparatus that provides a scene suitable for communication from multimedia content including utterance as content used for communication. In the communication service providing apparatus, the multimedia content including the utterance is uttered for each utterance. An utterance scene extraction unit that extracts an utterance text as a scene, and a sequence of utterance texts extracted by the utterance scene extraction unit as a communication sequence, and a sequence of utterance texts in a certain section on the content as a state sequence, An utterance learning means for learning a map from the state sequence to the action node as an utterance model, and an utterance model storage for storing the utterance model learned by the utterance learning means The communication history storage means for storing the communication utterance history, the communication utterance history stored in the communication history storage means, and the utterance model stored in the utterance model storage means, There is proposed a communication service providing apparatus comprising: a next utterance scene estimating means for estimating an utterance scene suitable for the next utterance in the communication.

  (2) In the communication service providing apparatus of (1), the present invention provides tag providing means for assigning attribute information of each utterance scene as a tag to each utterance scene extracted by the utterance scene extraction means, Tag storage means for storing the tag assigned to each utterance scene by the tag assigning means in association with each utterance scene extracted by the utterance scene extraction means, and the utterance learning means on the content An action tag node from a state tag sequence with a tag sequence assigned by the tag assigning unit corresponding to the utterance scene sequence in a section as a state tag sequence and a tag assigned to the next utterance scene in the interval as an action tag node The communication map is stored as the utterance model and stored in the communication history storage means. Next scene tag estimating means for estimating a tag to be assigned to a scene suitable for the next utterance of communication based on the utterance history of the application and the utterance model stored in the utterance model storage means, and the next scene tag And a tag scene search means for searching for an utterance scene suitable for the next utterance based on the tag estimated by the estimation means and the tag stored in the tag storage means. Proposing device.

  (3) In the communication service providing apparatus according to (2), the attribute information includes at least an utterance text of the utterance scene, an emotion of a character appearing in the utterance scene, and components of the utterance scene. We have proposed a communication service providing device.

  (4) The present invention proposes a communication service providing apparatus in which the utterance scene extracting means extracts spoken words and sound effects as the uttered text for the communication service providing apparatus of (1) to (3). doing.

  (5) In the communication service providing apparatus according to (1) to (4), the present invention provides, for each content, an utterance text sequence extracted by the utterance scene extraction unit for each content as a communication sequence. A sequence of utterance texts in a certain section as a state sequence, a next utterance scene in the section as an action node, a map from the state sequence to the action node as an utterance model, and the utterance model storage means, For each content, the utterance model generated by the utterance learning means is stored, and for each user, the usage history storage means for storing the history of multimedia content used by the user, and the user performing the communication, From the history stored in the usage history storage means, A speech candidate stored in association with the content candidate extracted by the content candidate extraction unit from a plurality of speech models stored in the speech model storage unit. Utterance model selection means, and the next utterance scene estimation means stores a scene suitable for the next utterance of the communication in the communication history storage means from among the utterance models selected by the utterance model selection means. The communication service providing apparatus is characterized in that the utterance scene suitable for the next utterance is estimated based on the utterance history of the communication.

  (6) In the communication service providing apparatus according to (5), the present invention uses each multimedia content as a basis based on the history stored in the usage history storage unit, and uses the number of uses of each multimedia content as a coefficient. Content history vector generating means for generating a content history vector for each user, and a user having a small distance from the user performing communication based on the content history vector generated by the content history vector generating means Content similar user extraction means for extracting as a similar user, and the content candidate extraction means stores the history stored in the usage history storage means based on the content similar user obtained by the content similar user extraction means Conte from It proposes a communication service providing device and extracting the tool candidate.

  (7) The present invention relates to the communication service providing apparatus according to (5) or (6), based on an utterance history storage unit that stores an utterance history for each user, and an utterance history stored in the utterance history storage unit. Based on the utterance history vector generation means for generating for each user an utterance history vector based on each word and the frequency of appearance of each word as a coefficient, and the utterance history vector generated by the utterance history vector generation means, Utterance similar user extraction means for extracting a user having a small distance from the communicating user as an utterance similar user, and the content candidate extraction means is determined by the utterance similar user extraction means. Based on the user, content candidates are extracted from the history stored in the usage history storage means. It has proposed a communication service providing device according to claim.

  (8) In the communication service providing apparatus according to (1) to (7), the present invention provides a plurality of utterance scenes suitable for the next utterance in the communication when the next utterance scene estimation unit estimates the plurality of utterance scenes. Providing a communication service providing apparatus, comprising: an image search for an utterance scene based on text received from a user performing the communication, and a means for narrowing down utterance scene candidates suitable for the next utterance doing.

  (9) In the communication service providing apparatus according to (1) to (8), the present invention performs the communication from the utterance scenes estimated by the next utterance scene estimation means to be suitable for the next utterance in the communication. Authentication means for authenticating whether or not the user has the right of the utterance scene selected by the user, and when the authentication means can authenticate, the user selects the other user performing the communication A communication providing apparatus characterized by transmitting a uttered scene is proposed.

  (10) The present invention is a communication service providing method in a communication service providing apparatus that provides a scene suitable for communication from multimedia content including speech as content used for communication, wherein the communication service providing apparatus includes: Multimedia content including utterance scene extraction means, utterance learning means, utterance model storage means, communication history storage means for storing the communication utterance history, and next utterance scene estimation means, wherein the utterance scene extraction means includes the utterance Is divided into utterance scenes in units of utterances, and the utterance text is extracted, and the utterance learning means communicates the series of utterance texts extracted in the first step. As a sequence of content, a sequence of utterance texts in a certain section on the content is used as a state sequence, the next utterance scene in the section is used as an action node, and a map from the state sequence to the action node is learned as an utterance model. The second utterance model storage means stores the utterance model learned in the second step, and the next utterance scene estimation means is stored in the communication history storage means. Proposing a communication service providing method comprising: a fourth step of estimating an utterance scene suitable for the next utterance in the communication based on an utterance history of communication and an utterance model stored in the utterance model storage means ing.

  (11) The present invention is a program for causing a computer to execute a communication service providing method in a communication service providing apparatus that provides a scene suitable for communication from multimedia content including speech as content used for communication. The communication service providing apparatus includes utterance scene extraction means, utterance learning means, utterance model storage means, communication history storage means for storing the communication utterance history, and next utterance scene estimation means, and the utterance scene extraction means includes The first step of dividing the multimedia content including the utterance as an utterance scene for each utterance and extracting the utterance text, and the utterance learning means includes the first step. The extracted sequence of utterance texts is used as a communication sequence, the sequence of utterance texts in a certain section on the content as a state sequence, the next utterance scene in the section as an action node, and a map from the state sequence to the action node. A second step of learning as an utterance model; a third step of storing the utterance model learned in the second step by the utterance model storage means; and a next utterance scene estimating means of the communication history storage means. A fourth step of estimating an utterance scene suitable for the next utterance in the communication based on the utterance history of the communication stored in the utterance and the utterance model stored in the utterance model storage means; Propose a program to run ing.

  According to the present invention, a scene suitable for the next utterance in communication can be estimated from multimedia contents including the utterance.

It is a figure which shows the structure of the communication service provision apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of authentication of the utterance scene by the authentication part which concerns on the 1st Embodiment of this invention. It is a figure which shows the speech model creation process flow which concerns on the 1st Embodiment of this invention. It is a figure which shows the next utterance scene estimation process flow which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the communication service provision apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the structure of the communication service provision apparatus which concerns on the 3rd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the constituent elements in the present embodiment can be appropriately replaced with existing constituent elements and the like, and various variations including combinations with other existing constituent elements are possible. Therefore, the description of the present embodiment does not limit the contents of the invention described in the claims.

<First Embodiment>
<Configuration of communication service providing device>
FIG. 1 is a diagram illustrating a configuration of a communication service providing apparatus 100 according to the first embodiment of the present invention. The communication service providing apparatus 100 according to the present embodiment is a scene suitable for the next utterance in communication based on the utterance model created from the utterance of each scene of multimedia content including utterance (hereinafter referred to as content for simplification). Is estimated from each scene of the content.

  Here, communication is communication performed via a network, and is, for example, LINE (registered trademark), Twitter (registered trademark), Facebook (registered trademark), or the like. Further, in the present embodiment, the communication service providing apparatus 100 is an apparatus independent from the user terminal and the server that provides the communication service, but may be realized by the user terminal and the server having the function.

  The communication service providing apparatus 100 can provide a user who performs communication with a scene that is estimated to be suitable for the next utterance in communication, and the user can easily use the content scene for communication. In addition, since the scenes presented to the user are narrowed down in advance, the user can easily select a scene, and as a result, smooth communication is possible.

  As shown in FIG. 1, the communication service providing apparatus 100 according to the present embodiment includes an utterance scene extraction unit 110, an utterance learning unit 120, an utterance model storage unit 130, a communication history storage unit 140, a next utterance scene estimation unit 150, and a narrowing down. Unit 160 and authentication unit 170.

  The utterance scene extraction unit 110 separates content including an utterance as an utterance scene for each utterance, and extracts an utterance text from the divided utterance scene. Here, the content including an utterance is a content including an image including an utterance, and is, for example, a video content or a comic content. The video content is a movie, animation, drama or the like.

  Specifically, when the content is video content, the utterance scene extraction unit 110 separates the utterance image as one utterance scene for each utterance, and converts the utterance corresponding to the divided utterance scene into text. In the case of comic content, using the technique described in Patent Document 1, one frame is segmented as one utterance scene, and the utterance text is extracted from the segmented utterance scene.

  Here, in the existing dialogue system represented by Non-Patent Document 2, spoken words such as stagnation and rephrasing, and sound effects such as onomatopoeia and filler were not included in the utterance text as meaningless utterances. . However, in the present invention, spoken words and sound effects are also included in the utterance text. This is because the state of emotions and utterance scenes can be expressed by spoken words and sound effects, and is meaningful when estimating utterance scenes suitable for communication.

  The content from which the utterance scene extraction unit 110 divides into utterance scenes and extracts the utterance text from the utterance scene is the content via the network when creating the utterance model used to estimate the utterance scene suitable for the next utterance of communication. Content acquired from a server under arbitrary or predetermined conditions, or content input by an administrator of the communication service providing apparatus 100 or the like.

  The utterance learning unit 120 treats the utterance text extracted by the utterance scene extraction unit 110 as a sequence of utterances in a chronological order of the utterance scenes of the extraction source, and utterances extracted from the utterance scenes in a certain section on the content A sequence of text is used as a state sequence, a next utterance scene in a certain section is used as an action node, and a map from the state sequence to the action node is learned as an utterance model.

  For a certain content, when the time series of utterance scenes are A, B, C, and D, and the utterance texts of each utterance scene are a, b, c, and d in order, for example, the utterance model is a → b → c → D.

  The utterance model storage unit 130 stores the utterance model generated by the utterance learning unit 120. Note that one utterance model may be created for each content item, or one utterance model may be created from a plurality of content items or a plurality of arbitrary content items that are frequently used by the user. Further, the utterance model may be created periodically, or may be created in accordance with an instruction from an administrator of the communication service providing apparatus 100 or the like.

  The communication history storage unit 140 stores the utterance history of one or more users who are performing communication. Specifically, the user identification information for identifying the user who made the utterance and the text of the user's utterance are associated with each other and stored for each communication. Note that the utterance history storage unit 330 may acquire and store the utterance scene from the server with which communication is performed each time the user utters, or provide the utterance scene from the user through the server with which communication is performed. When requested, the utterance history stored in the server may be acquired and stored. At this time, past utterance histories performed by one or more users may be acquired.

  The next utterance scene estimation unit 150 includes an utterance scene suitable for the next utterance in communication, a communication history storage unit 140 storing an utterance history so far in communication, and an utterance model storage unit 130 storing an utterance model. Estimate based on The next utterance scene estimation unit 150 estimates an utterance scene suitable for the next utterance in communication in response to receiving the next utterance scene request. Here, the next utterance scene request may be made voluntarily by the user, or may be automatically made in response to the end of the utterance of one user.

  As a method of estimating a scene suitable for the next utterance in communication, for example, an utterance scene suitable for the immediately preceding utterance is estimated by matching words between the utterance history so far in communication and the utterance text of the utterance model, and the immediately preceding utterance is estimated. There is a method for estimating the next scene of the utterance scene estimated to be suitable for the next utterance as a scene suitable for the next utterance. Specifically, words are extracted from utterance texts included in each scene and the preceding and following scenes, and the next scene of a scene with many words that match the immediately preceding utterance text is estimated as a scene suitable for the next utterance. Note that the immediately preceding utterance is not limited to one, but may be two immediately preceding utterances or three utterances.

  The next utterance scene estimation unit 150 estimates the utterance scene suitable for the next utterance by using the utterance model and the utterance history so far in the communication, so that the previous utterance is the same even if the previous utterance is the same. When the flow of the voices is different, different utterance scenes suitable for the flow of communication can be estimated. Further, when the utterance learning unit 120 learns an utterance model from the exchange of utterances between content characters using the technology proposed in Non-Patent Document 2, the next utterance scene estimation unit 150 performs communication consisting of a plurality of turns. Therefore, it is possible to estimate an utterance scene suitable for the next utterance.

  By providing the user who is performing communication with the utterance scene that is estimated to be suitable for the next utterance in communication by the next utterance scene estimation unit 150, the user can easily use the utterance scene that matches the next utterance. And smooth communication using the utterance scene.

  When the next utterance scene estimation unit 150 estimates a plurality of utterance scenes, the narrowing-down unit 160 performs an image search based on the text received from the user and narrows down the utterance scenes to be provided to the user as the next utterance scene. . Thereby, the number of utterance scenes provided to the user can be narrowed down, and smoother communication becomes possible. As an image search performed by the narrowing-down unit 160, for example, the technique described in Non-Patent Document 1 can be used.

  The user may narrow down the utterance scene to be provided to the user by inputting a keyword or the like of the next utterance before the user utters the next utterance, or every time the user inputs the character of the next utterance. Candidates may be narrowed down.

  The authentication unit 170 authenticates whether or not the user has the right of the utterance scene selected by the user who is performing communication from the utterance scenes estimated by the next utterance scene estimation unit 150. If the authentication unit 170 is authenticated, the authentication unit 170 transmits the utterance scene selected by the user to the other user who is performing communication. On the other hand, if the authentication unit 170 fails to authenticate, the authentication unit 170 indicates that the user has no right. Provide notifications to encourage purchases.

  In the present invention, it is possible to define rights not only for the entire content, but also for a part of the content that is subdivided in units of utterance scenes, chapters, or the like. As a result, it is possible to acquire rights only for necessary portions, and it is possible to flexibly respond to user requests.

  An example of authentication of an utterance scene by the authentication unit 170 will be described with reference to FIG. In this description, it is assumed that the service server that provides the communication service has the function of the communication service providing apparatus 100.

  (A) is an authentication method when the utterance scene provided by the service server, or a chapter or content including the utterance scene is used as the next utterance scene.

  First, the service server accepts selection of an utterance scene to be used as the next utterance scene from the utterance user from the utterance scenes estimated by the next utterance scene estimation unit 150. Next, the service server sends the utterance scene selected by the user, the chapter including the utterance scene, the content identification information of the content, the user identification information of at least one of the utterance user and the reception user, or the like to the content holder. Next, based on the information received from the service server, the content holder transmits the utterance scene selected by the user, or the chapter and content right information including the utterance scene to the service server.

  Then, the service server determines whether or not the user has the right of the utterance scene selected by the utterance user based on the right information received from the content holder. When the utterance user has the right of the selected utterance scene, the utterance scene is transmitted to the receiving user. On the other hand, when the utterance user does not have the right of the selected utterance scene, the service server charges the utterance user for a fee necessary for purchasing the right, and in response to the payment from the utterance user, the service server Transmits the utterance scene to the receiving user. At the same time, the service server transmits the right information of the utterance scene for which the fee has been paid to the content holder.

  (B) is an authentication method in the case of using an utterance scene or a chapter or content including the utterance scene, which has a right in the terminal of the utterance user.

  First, if the user terminal of the utterance user holds the right of the utterance scene selected by the utterance user from the utterance scenes estimated by the next utterance scene estimation unit 150, the DRM information to be retained, User identification information or the like of at least one of the speaking user and the receiving user is transmitted to the service server. Next, the service server transmits the received DRM information, identification information of the talking user and the receiving user, and the like to the content holder. Next, based on the information received from the service server, the content holder transmits the utterance scene selected by the user, or the chapter and content right information including the utterance scene to the service server.

  Next, based on the right information received from the content holder, the service server determines whether or not the utterance user has a right necessary to transmit to the receiving user for the utterance scene selected by the user. If the utterance user has the necessary right for the utterance scene selected, the utterance scene is transmitted to the receiving user. On the other hand, if the utterance user does not have the necessary rights for the utterance scene selected, the service server charges the utterance user for the fee necessary for right purchase, and in response to payment from the utterance user, The service server transmits the utterance scene to the receiving user. At the same time, the service server transmits the right information of the utterance scene for which the fee has been paid to the content holder.

  Note that the communication service providing apparatus 100 does not include the authentication unit 170, and uses the existing system to authenticate the right of the utterance scene selected by the utterance user from the utterance scenes estimated by the next utterance scene estimation unit 150. Also good. When the right of the utterance scene selected by the uttering user is authenticated in the existing system, the content is transmitted from the content holder to the service server, and the service server transmits the received content to the receiving user.

<Communication service processing flow>
The communication service process according to the first embodiment of the present invention includes an utterance model creation process and a next utterance scene estimation process. FIG. 3 is a diagram showing an utterance model creation processing flow according to the first embodiment of the present invention.

  First, in step S1, the utterance scene extraction unit 110 divides content into utterance scenes in units of utterances.

  Next, in step S2, the utterance scene extraction unit 110 extracts the utterance text from the utterance scene cut out in step S1.

  Next, in step S3, the utterance scene extraction unit 110 determines whether or not the utterance text has been extracted from all the utterance scenes cut out in step S1. If the utterance text is extracted from all utterance scenes (YES), the process proceeds to step S4. If the utterance text is not extracted from all utterance scenes (NO), the process returns to step S2.

  Next, in step S4, an utterance model is learned from the utterance scene cut out in step S1 and the utterance text extracted in step S2.

  Next, in step S5, the utterance model learned in step S4 is stored in the utterance model storage unit 130.

  FIG. 4 is a diagram showing a next utterance scene estimation processing flow according to the first embodiment of the present invention.

  First, in step S <b> 11, the next utterance scene estimation unit 150 acquires an utterance history in communication from the communication history storage unit 140.

  Next, in step S <b> 12, the next utterance scene estimation unit 150 acquires an utterance model from the utterance model storage unit 130.

  Next, in step S13, the next utterance scene estimation unit 150 estimates an utterance scene suitable for the next utterance in communication based on the utterance history acquired in step S11 and the utterance model acquired in step S12.

  As described above, according to the present embodiment, a scene suitable for the next utterance in communication can be estimated based on the utterance model learned from multimedia content including the utterance and the utterance history of communication. it can. As a result, the user can easily use the utterance scene suitable for the next utterance by providing the utterance scene estimated to be suitable for the next utterance in the communication to the user who is performing the communication. Smooth communication is possible.

<Second Embodiment>
A second embodiment of the present invention will be described with reference to FIG. Note that the communication service providing apparatus according to the present embodiment estimates the next utterance scene in which communication is possible based on the attribute information of the utterance scene. In addition, about the component which attaches | subjects the same code | symbol as 1st Embodiment, since it has the same function, the detailed description is abbreviate | omitted.

<Configuration of communication service providing device>
FIG. 5 is a diagram showing a configuration of a communication service providing apparatus 200 according to the second embodiment of the present invention. As shown in FIG. 5, in this embodiment, the communication service providing apparatus 200 includes an utterance scene extraction unit 110, a tag addition unit 210, a tag storage unit 220, an utterance learning unit 121, an utterance model storage unit 130, and a communication history storage unit. 140, a next utterance scene tag estimation unit 230, and a tag scene search unit 240.

  The tag assignment unit 210 assigns the attribute information of each utterance scene as a tag to each utterance scene extracted by the utterance scene extraction unit 110. Here, the attribute information of each utterance scene includes at least the utterance text of the utterance scene, the emotion of the character appearing in the utterance scene, and the constituent elements of the utterance scene. Further, the constituent elements of the utterance scene are the stage of the story (for example, one of the start and end), the characters appearing, the screen configuration such as the position and size of the character, and the background such as the school and the seaside. The attribute information may be automatically acquired by image analysis of the utterance scene or may be manually acquired from the utterance scene.

  The tag storage unit 220 stores the tag assigned to each utterance scene by the tag assigning unit 210 in association with each utterance scene extracted by the utterance scene extraction unit 110.

  The utterance learning unit 121 corresponds to the time series of the utterance scenes in a certain section on the content, the time series of the tags attached by the tag assignment unit 210, the tag attached to the utterance scene next to the section. As an action tag node, a map from a state tag sequence to an action tag node is learned as an utterance model.

  The next utterance scene tag estimation unit 230 has a tag given to a scene suitable for the next utterance in communication, an utterance history storage unit 330 that stores a history of utterances in communication, and an utterance that stores an utterance model. Estimation is performed based on the model storage unit 130. The next utterance scene estimation unit 150 estimates a scene suitable for the next utterance in communication in response to receiving the utterance scene request. Here, the utterance scene request may be made voluntarily by the user, or may be automatically made in response to the end of the utterance of one user.

  The tag scene search unit 240 searches a tag stored in the tag storage unit 220 for a tag that matches the tag estimated by the next utterance scene tag estimation unit 230. Then, the tag scene search unit 240 estimates an utterance scene to which the searched tag is assigned as an utterance scene suitable for the next utterance. Thereby, the utterance scene suitable for the next utterance in communication can be estimated from the tag of the attribute information of the utterance scene.

  As described above, according to the present embodiment, the utterance scene suitable for the next utterance in communication is estimated from the attribute information of the utterance scene. As a result, since the attribute information is information that abstracts the utterance scene, it is possible to estimate not only the utterance scene that best matches the flow of communication, but also the utterance scene that roughly matches as the utterance scene suitable for the next utterance. it can. As a result, an utterance scene different from the user's expectation is also provided, and the range of utterance scene choices used for communication can be expanded.

<Third Embodiment>
A third embodiment of the present invention will be described with reference to FIG. Note that the communication service providing apparatus according to the present embodiment selects an utterance model generated from specific content from a plurality of utterance models, and estimates a next utterance scene from the selected utterance model. In addition, about the component which attaches | subjects the same code | symbol as 1st Embodiment, since it has the same function, the detailed description is abbreviate | omitted.

<Configuration of communication service providing device>
FIG. 6 is a diagram illustrating a configuration of a communication service providing apparatus 300 according to the third embodiment of the present invention. As shown in FIG. 6, in this embodiment, the communication service providing apparatus 300 includes an utterance scene extraction unit 110, an utterance learning unit 122, an utterance model storage unit 132, a usage history storage unit 310, a content history vector generation unit 320, an utterance. It includes a history storage unit 330, an utterance history vector generation unit 340, a similar user extraction unit 350, a content candidate extraction unit 360, an utterance model selection unit 370, and a next utterance scene estimation unit 152.

  The utterance learning unit 122 treats, for each content, a sequence in which the utterance text extracted by the utterance scene extraction unit 110 is arranged in chronological order of the extraction source utterance scene as an utterance sequence, and from the utterance scene in a certain section on the content. A series of extracted utterance texts is used as a state sequence, a next utterance scene in a certain section is used as an action node, and a map from the state sequence to the action node is learned as an utterance model.

  The utterance model storage unit 132 stores the utterance model for each content generated by the utterance learning unit 122 for each content. Note that the utterance model may be created periodically or in response to an instruction from the administrator of the communication service providing apparatus 300 or the like.

  The usage history storage unit 310 stores, for each user, a history of content used by the user. The usage history storage unit 310 stores, for example, content identification information and the number of uses in association with user identification information.

  Based on the history stored in the usage history storage unit 310, the content history vector generation unit 320 generates, for each user, a content history vector based on each content and using the number of times each content is used as a coefficient. The content history vector can be expressed by equation (1).

  The utterance history storage unit 330 stores an utterance history for each user. Specifically, the utterance history storage unit 330 stores utterances in communication performed by the user in the past in association with the identification information of the user.

  Based on the utterance history stored in the utterance history storage unit 330, the utterance history vector generation unit 340 generates, for each user, an utterance history vector that uses each word as a base and uses the appearance frequency of each word as a coefficient. The utterance history vector can be expressed by equation (2).

  Based on the content history vector generated by the content history vector generation unit 320, the similar user extraction unit 350 extracts a user having a small distance from the communicating user as a similar user. Specifically, the degree of similarity with other users is calculated by equation (3), and the user with the lowest degree of similarity is set as a content similar user.

  Also, the similar user extraction unit 350 extracts a user having a small distance from the user performing communication as a similar user based on the utterance history vector generated by the utterance history vector generation unit 340. Specifically, similar to the case of extracting similar users based on the utterance history vector, the degree of similarity with other users is calculated by equation (3), and the user with the lowest degree of similarity is set as the utterance similar user.

  The content candidate extraction unit 360 extracts content candidates from the history stored in the usage history storage unit 310 for the user who is performing communication. Specifically, the content candidate extraction unit 360 extracts content that is frequently used by the user who is performing communication based on the history stored in the usage history storage unit 310.

  Further, the content candidate extraction unit 360 extracts content candidates from the history stored in the usage history storage unit 310 based on the content similar users obtained by the similar user extraction unit 350. Specifically, the content candidate extraction unit 360 extracts content that is frequently used by content-similar users based on the history stored in the usage history storage unit 310.

  Further, the content candidate extraction unit 360 extracts content candidates from the history stored in the usage history storage unit 310 based on the utterance similar user obtained by the similar user extraction unit 350. Specifically, the content candidate extraction unit 360 extracts content that is frequently used by utterance-like users based on the history stored in the usage history storage unit 310.

  The utterance model selection unit 370 selects an utterance model stored in association with the content candidate extracted by the content candidate extraction unit 360 from the plurality of utterance models stored in the utterance model storage unit 130.

  The next utterance scene estimation unit 152 selects the utterance scene suitable for the next utterance in communication, the utterance history storage unit 330 storing the utterance history so far in communication, and the utterance model selected by the utterance model selection unit 370. Estimation is performed based on the stored utterance model storage unit 130. The next utterance scene estimation unit 152 estimates an utterance scene suitable for the next utterance in communication in response to receiving the utterance scene request. Here, the utterance scene request may be made voluntarily by the user, or may be automatically made in response to the end of the utterance of one user. The scene estimation method suitable for the next utterance in communication is the same as that in the first embodiment.

  As described above, according to the present embodiment, the content that is frequently used by the user is the content that the user likes, and is likely to be used for communication. Therefore, by using the utterance model learned from the content frequently used by the user for estimation of the next utterance scene, the utterance scene included in the content frequently used by the user can be presented to the user as an utterance scene suitable for the next utterance, The estimation accuracy of the utterance scene suitable for the next utterance can be improved.

  In addition, by using an utterance model learned from content frequently used by similar users whose content and utterances are similar to the user for estimating the next utterance scene, content not used by the user and less frequently used However, since the utterance scene included in the content estimated to meet the user's preference can be presented to the user as the utterance scene suitable for the next utterance, an utterance scene different from the user's expectation is also provided, and the range of options of the user is expanded. be able to.

  Note that the communication service providing apparatus of the present invention can be realized by recording the processing of the communication service providing apparatus on a computer-readable recording medium, causing the device to read and execute the program recorded on the recording medium. . The computer system here includes an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW (World Wide Web) system is used. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the embodiments, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 100 Communication service provision apparatus 110 Utterance scene extraction part 120 Utterance learning part 130 Utterance model memory | storage part 140 Communication history memory | storage part 150 Next utterance scene estimation part 160 Narrowing part 170 Authentication part

Claims (11)

In a communication service providing apparatus that provides a scene suitable for communication from multimedia content including speech as content used for communication,
The multimedia content including the utterance is segmented as an utterance scene in one utterance unit, and an utterance scene extracting means for extracting an utterance text;
The sequence of utterance texts extracted by the utterance scene extraction means is used as a communication sequence, the sequence of utterance texts in a certain section on the content as a state sequence, the next utterance scene in the section as an action node, and the state sequence. An utterance learning means for learning a map to an action node as an utterance model;
Utterance model storage means for storing the utterance model learned by the utterance learning means;
Communication history storage means for storing the communication utterance history;
Next utterance scene estimation for estimating an utterance scene suitable for the next utterance in the communication based on the utterance history of the communication stored in the communication history storage means and the utterance model stored in the utterance model storage means Means,
A communication service providing apparatus comprising: Tag giving means for giving, as a tag, attribute information of each utterance scene to each utterance scene extracted by the utterance scene extracting means;
Tag storage means for storing a tag assigned to each utterance scene by the tag assignment means in association with each utterance scene extracted by the utterance scene extraction means;
With
The utterance learning means behaves with a tag series assigned by the tag assignment means corresponding to the utterance scene series in a certain section on the content as a state tag sequence, and a tag attached to the next utterance scene in the section. As a tag node, a map from a state tag sequence to an action tag node is learned as the utterance model,
Based on the utterance history of the communication stored in the communication history storage means and the utterance model stored in the utterance model storage means, a tag attached to a scene suitable for the next utterance of the communication is estimated. Next scene tag estimation means,
The tag scene search means for searching for an utterance scene suitable for the next utterance based on the tag estimated by the next scene tag estimation means and the tag stored in the tag storage means;
The communication service providing apparatus according to claim 1, further comprising:   The communication service providing apparatus according to claim 2, wherein the attribute information includes at least an utterance text of the utterance scene, an emotion of a character appearing in the utterance scene, and a component of the utterance scene.   4. The communication service providing apparatus according to claim 1, wherein the utterance scene extracting unit extracts a spoken word and a sound effect as the utterance text. For each content, the utterance learning means uses the utterance text sequence extracted by the utterance scene extraction means as a communication sequence, the utterance text sequence in a certain section on the content as a state sequence, and the next utterance scene in the section As an action node, learning a map from the state sequence to the action node as an utterance model,
The utterance model storage means stores the utterance model generated by the utterance learning means for each content,
Usage history storage means for storing a history of multimedia content used by the user for each user;
Content candidate extraction means for extracting content candidates from the history stored in the usage history storage means for the user performing the communication;
An utterance model selection means for selecting an utterance model stored in association with the content candidate extracted by the content candidate extraction means from a plurality of utterance models stored in the utterance model storage means;
With
The next utterance scene estimation unit is configured to select a scene suitable for the next utterance of the communication from the utterance model selected by the utterance model selection unit based on the utterance history of the communication stored in the communication history storage unit. The communication service providing apparatus according to claim 1, wherein an utterance scene suitable for the next utterance is estimated. Content history vector generation means for generating for each user a content history vector based on each multimedia content and based on the history stored in the use history storage means, with the number of uses of each multimedia content as a coefficient; ,
Content similar user extraction means for extracting, as a content similar user, a user having a small distance from the user performing communication based on the content history vector generated by the content history vector generation means;
With
6. The content candidate extracting unit extracts a content candidate from a history stored in a usage history storage unit based on the content similar user obtained by the content similar user extracting unit. The communication service providing apparatus described. Utterance history storage means for storing the utterance history for each user;
Based on the utterance history stored in the utterance history storage means, an utterance history vector generation means for generating for each user an utterance history vector with each word as a base and an appearance frequency of each word as a coefficient;
Based on the utterance history vector generated by the utterance history vector generation means, an utterance similar user extraction means for extracting a user having a small distance from the user performing the communication as an utterance similar user;
With
6. The content candidate extracting unit extracts a content candidate from a history stored in the usage history storage unit based on the utterance similar user obtained by the utterance similar user extracting unit. Or the communication service provision apparatus of Claim 6. When a plurality of utterance scenes suitable for the next utterance in the communication are estimated by the next utterance scene estimation means, an image search is performed on the plurality of utterance scenes based on text received from the user performing the communication. was carried out, the communication service providing device according to any one of claims 1 to 7, characterized in that it comprises a narrowing means for narrowing the candidate speech scenes suitable for the next utterance. Authentication is performed as to whether or not the user has the right of the utterance scene selected by the user performing the communication from the utterance scenes estimated to be suitable for the next utterance in the communication by the next utterance scene estimation means. With authentication means,
The communication service according to any one of claims 1 to 8, wherein when the authentication unit can authenticate, the utterance scene selected by the user is transmitted to another user who is performing the communication. Providing device. A communication service providing method in a communication service providing apparatus that provides a scene suitable for communication from multimedia content including speech as content used for communication,
The communication service providing apparatus includes an utterance scene extraction means, an utterance learning means, an utterance model storage means, a communication history storage means for storing the utterance history of the communication, and a next utterance scene estimation means,
A first step in which the utterance scene extraction means separates the multimedia content including the utterance as an utterance scene into one utterance unit, and extracts an utterance text;
The utterance learning means uses the sequence of utterance texts extracted in the first step as a communication sequence, the sequence of utterance texts in a certain section on the content as a state sequence, and the next utterance scene in the section as an action node. A second step of learning a map from the state sequence to the action node as an utterance model;
A third step in which the utterance model storage means stores the utterance model learned in the second step;
The utterance suitable for the next utterance in the communication based on the utterance history of the communication stored in the communication history storage means and the utterance model stored in the utterance model storage means. A fourth step of estimating the scene;
A communication service providing method comprising: A program for causing a computer to execute a communication service providing method in a communication service providing apparatus that provides a scene suitable for communication from multimedia content including an utterance as content used for communication,
The communication service providing apparatus includes an utterance scene extraction means, an utterance learning means, an utterance model storage means, a communication history storage means for storing the utterance history of the communication, and a next utterance scene estimation means,
A first step in which the utterance scene extraction means separates the multimedia content including the utterance as an utterance scene into one utterance unit, and extracts an utterance text;
The utterance learning means uses the sequence of utterance texts extracted in the first step as a communication sequence, the sequence of utterance texts in a certain section on the content as a state sequence, and the next utterance scene in the section as an action node. A second step of learning a map from the state sequence to the action node as an utterance model;
A third step in which the utterance model storage means stores the utterance model learned in the second step;
The utterance suitable for the next utterance in the communication based on the utterance history of the communication stored in the communication history storage means and the utterance model stored in the utterance model storage means. A fourth step of estimating the scene;
A program that causes a computer to execute.

Images