JP2022131963A - Summary generation device and summary generation method - Google Patents

Abstract

To provide a summary generation device capable of generating an easy-to-understand summary according to interruption positions.SOLUTION: A summary generation device 10 that generates a summary of a target content includes a processor 11. The processor determines a first range in the target content from a reproduction interruption position of the target content (first determination processing 111), calculates from the first range an index value of sentences included in a second range which is at least partially different from the first range, and which is a region from which a summary constituent sentence is extracted in the target content (calculation processing 113), and extracts the constituent sentences of the summary from the second range on the basis of the index values (extraction processing 114).SELECTED DRAWING: Figure 1

Description

There is an application for application of Article 30, Paragraph 2 of the Patent Act Disclosure fact 1: On June 8, 2020, the website of the paper of the 2020 Annual Conference of the Japanese Society for Artificial Intelligence (34th) (https:// drive.google.com/file/d/1GF6q7bimI8W7tTXw2DtJWb50gu1l_q1v/view?usp=sharing) Public Fact 2: The Institute of Electronics, Information and Communication Engineers Media Experience Virtual Environment Basics (MVE) on September 1, 2020 Research Society) Paper collection website (https://www.ieice.org/ken/user/index.php?cmd=login&back_url=https%3A%2F%2Fwww.ieice.org%2Fken%2Fpaper%2F20200908w1zT%2F )

The present disclosure relates to a summary generation device and a summary generation method.

Contents that have a story, such as dramas, novels, and comics, sometimes forget the previous playback contents when resuming playback after being interrupted. In that respect, it is composed of multiple episodes in chronological order, and at the beginning of the episode, the synopsis up to the previous episode, the preview of the next episode, etc. may be played.

In recent years, the reproduction of contents with stories such as dramas has changed from a passive form such as television to a proactive form such as online where a charge is made for the reproduction period such as a so-called subscription service.

When content is reproduced in such a form, the conventional style of presenting a summary only at the beginning of an episode does not present a summary when resuming reproduction from a position interrupted in the middle of the episode. Therefore, the summary may not be suitable for resuming playback.

In this respect, for example, Japanese Patent Application Laid-Open No. 2007-336085 (hereinafter referred to as Patent Document 1) discloses a method of generating an advance notice after the position where reproduction of content is interrupted.

Japanese Patent Application Laid-Open No. 2007-336085

However, when the playback is resumed from an arbitrary interrupted position, depending on the interrupted position, the sentences extracted from the previous playback range or the sentences extracted from the subsequent range may not be an easy-to-understand summary. Therefore, it is desirable to have a summary generation device and a summary generation method that can generate an easy-to-understand summary according to the interruption position.

Here, the summary generation device is a summary generation device that generates a summary of target content, and includes a processor. The processor determines a first range from the interruption position of the target content, and sets the index value of the sentence included in the second range, which is the range for extracting the constituent sentences of the summary, determined from the reproduction interruption position to the first range. is calculated from the range of , and a constituent sentence of the summary is extracted from the second range based on the index value.

Also, the method for generating a summary is a method for generating a summary of target content, in which a first range is determined from the interruption position of the target content, and sentences included in a second range, which is a range for extracting constituent sentences of the summary. from the first range, and extracting the constituent sentences of the summary from the second range based on the index value.

Further details are described as embodiments below.

FIG. 1 is a schematic diagram showing an example of the configuration of a summary generation device according to the first embodiment and an example of processing executed by the summary generation method according to the first embodiment. FIG. 2 is a diagram for explaining the configuration of content for which a summary is to be generated by the summary generation device. FIG. 3 is a diagram for explaining a method of determining a range for generating content summaries. FIG. 4 is a diagram for explaining the process of extracting summary constituent sentences. FIG. 5 is a flow chart showing an example of a summary generation method according to the first embodiment. FIG. 6 is a schematic diagram showing an example of the configuration of a summary generation device according to the second embodiment and an example of processing executed by the summary generation method according to the second embodiment. FIG. 7 is a flow chart showing an example of a summary generation method according to the second embodiment. FIG. 8 is a diagram for explaining a specific example of a method of generating a summary of a target range using a partial summary prepared for content as a gold standard. FIG. 9 is a diagram for explaining a specific example of a method of generating a summary of a target range using a partial summary prepared for content as a gold standard. FIG. 10 is a diagram for explaining a specific example of a method of generating a summary of a target range using a partial summary prepared for content as a gold standard. FIG. 11 is a schematic diagram showing an example of the configuration of a summary generation device according to the third embodiment and an example of processing executed by the summary generation method according to the third embodiment. FIG. 12 is a diagram showing an example of the appearance distribution in the content of constituent sentences of a partial summary generated for a certain content. FIG. 13 is a diagram showing a specific example of extraction data. FIG. 14 is a diagram showing a specific example of placement data.

<1. Outline of summary generation device and summary generation method>

(1) A summary generation device according to an embodiment is a summary generation device that generates a summary of target content, and includes a processor, and the processor determines a first range in target content from a playback interruption position of target content. Then, the index value of the sentence included in the second range, which is the range for extracting the constituent sentences of the summary in the target content and which is at least partially different from the first range, is determined from the playback interruption position, 1 range, and based on the index value, extract constituent sentences of the summary from the second range.

The target content has a story and is reproduced in chronological order. For example, videos such as movies, dramas, and animations, literary works such as novels, and performances such as lectures and classes are equivalent. Elements to be reproduced include text, music, images, etc., but the focus here is on text. The text may be serifs, sentences or nouns. The text may be accompanied by a piece of music or an image to be played at the same time.

A summary is a short edited version of a story within a certain range of content (hereinafter referred to as a summary target range). The abstract is either the primary abstract or the secondary abstract. The first summary is a so-called synopsis and the second summary is a so-called preview. The first summary refers to a summary that introduces a range before the time point at which playback is started (hereinafter referred to as the start position). A second summary refers to a summary that introduces the range after the starting position. The second summary may contain content beyond the start location.

The first range is a range used for calculating the index value of sentences included in the second range, and is a range within the target content. The first range is, for example, the expected playback range. The expected reproduction range is a range after the reproduction interruption position and a predetermined range from the reproduction interruption position. The predetermined range may be a preset range. The predetermined range may be determined based on the attributes of the user, the attributes of the target content, the user's playback tendency, and the like. The user's playback tendency is, for example, the user's playback behavior. Alternatively, an average value of general regeneration amounts may be used.

The reproduction interruption position is the position where the reproduction is interrupted. As an example, the playback interruption position coincides with the playback start position. In that case, the interruption position is a reference position for generating a summary.

The second range is the summary target range and is determined based on the playback break position. The second range may be different from, coincide with, or at least partially overlap with the first range.

The summary target range is, for example, the range from the beginning of the story to the end of the expected playback range determined based on the playback interruption position. The expected playback range is determined by the user's viewing/reading behavior, the average amount of general viewing/reading, or the provider's definition. A more understandable summary can be generated by appropriately setting the summary target range.

The index value is a value calculated from the first range for sentences included in the second range, and includes, for example, importance. By extracting the constituent sentences of the summary from the second range based on the index value calculated from the first range, the generated summary includes sentences that take into account the first range determined from the playback interruption position. shall include. This makes it possible to generate a digest suitable for resuming playback.

(2) Preferably, the index value includes a value obtained based on words included in the first range. A value obtained based on the words included in the first range is, for example, importance. As a result, the generated summary includes sentences that take into account the values obtained based on the words included in the first range.

(3) Preferably, the first range is a range after the reproduction interruption position and is a predicted reproduction range of the target content determined from the reproduction interruption position. As a result, it becomes possible to generate a summary that takes into consideration the playback content that is resumed after the interruption.

(4) Preferably, the second range is determined by whether the summary is the first summary or the second summary. The second summary is a summary generated with the second range including the position after the reproduction interruption position. The first summary is a synopsis and the second summary is a preview. Thus, the first summary includes sentences extracted from the range up to the playback interruption position. Therefore, a summary is generated that reminds the user of the content up to the point where the reproduction is interrupted and increases the desire to reproduce. Also, the second summary includes sentences after the playback interruption position. Therefore, a summary that increases the user's willingness to reproduce is generated.

(5) Preferably, extracting the constituent sentences of the summary includes sequentially extracting a plurality of sentences included in the constituent sentences from the second range based on the index value, and extracting the portion having the sequentially extracted sentences generating a set, wherein the index value includes a similarity between the sentences included in the second range and the subset; At this time, if the extraction based on the index value is to extract sentences with a high degree of similarity, the constituent sentences will be a set of sentences with unity in content, and the content of the summary sentence will tend to be clear. Also, if the extraction based on the index value is to extract sentences with a low degree of similarity, the constituent sentences will be a diverse set of sentences, and the summary will tend to have a well-balanced content.

(6) Preferably, the subset includes sentences extracted from a pre-prepared summary of the target content. As a result, a summary is generated using the constituent sentences of the summary prepared in advance for the target content. Therefore, the processing becomes easier than extracting all the constituent sentences based on the index value.

(7) Preferably, the processor is further configured to select the reference content based on the target content, and extracting the constituent sentences of the summary from the second range is associated with the reference content. extracting the constituent sentences of the summary from the second range with reference to the extracted data. The reference content is content different from the target content, and is content for which a summary has been prepared. The extraction data is data representing the tendency of the positions of the summary constituent sentences prepared in the reference content in the reference content. By extracting the constituent sentences of the summary from the second range using the extraction data, it is possible to easily generate a summary of the target content and to generate a summary that is as easy to understand as the reference content. .

(8) A method of generating a summary according to an embodiment is a method of generating a summary of target content, and is a method of generating a summary of target content in the summary generation device described in (1) to (7). As a result, a summary generated by the summary generation device described in (1) to (7) is obtained.

<2. Example of summary generation method and summary generation device>

[First embodiment]

A summary generation device 10 according to the present embodiment generates a content summary. Content handled in this embodiment has a story and is reproduced in chronological order. For example, videos such as movies, dramas, and animations, literary works such as novels, and performances such as lectures and classes are equivalent. Elements to be reproduced include text, music, images, etc., but the focus here is on text. The text may be serifs, sentences or nouns. The text may be accompanied by a piece of music or an image to be played at the same time.

A summary is a short edited version of a story within a certain range of content (hereinafter referred to as a summary target range). The summary is either a synopsis (first summary) or an advance notice (second summary). The synopsis refers to a summary that introduces the range before the point of time when playback starts (hereinafter referred to as the start position). Trailer refers to a summary that introduces the range after the starting position. The advance notice may include content beyond the start position.

Referring to FIG. 1, summary generation device 10 is configured by a computer having processor 11 and memory 12 . Processor 11 is, for example, a CPU. Memory 12 includes flash memory, EEPROM, ROM, RAM, and the like. Alternatively, the memory 12 may be a primary storage device or a secondary storage device.

The memory 12 stores a generation program 121 executed by the processor 11 . The processor 11 executes the abstract generating process by executing the generating program 121 . The summary generation processing refers to processing for generating a summary of content for which a summary is to be generated (hereinafter referred to as target content).

Memory 12 also stores one or more pieces of content information 122 . The content information 122 is information about target content and includes information about a reference position for generating a summary. As an example, the summary generation reference position is the interruption position, and in that case the content information 122 includes the interruption position information 21 . In the following description, it is assumed that the interrupt position matches the start position.

Content information 122 may include one or more summary information 22 . The summary information 22 is a summary prepared in advance for the target content, and the details will be described later.

All or at least part of the content information 122 may be stored in an external device of the summary generation device 10, such as the server 30 in FIG. In that case, the summary generation device 10 accesses the external device as necessary to read and use the content information 122 . Alternatively, the abstract generation device 10 may have the server 30 .

FIG. 1 shows an example in which a summary generation device 10 according to an embodiment also serves as a content reproduction device 15 . Note that it is not essential that the summary generation device 10 also serves as the reproduction device 15 . The summary generation device 10 may be installed in the playback device 15, or may acquire necessary information directly or indirectly from the playback device 15, or may be an independent device.

In the case of FIG. 1, the summary generation device 10 has an operation unit 17 that receives user operations related to reproduction. The digest generation device 10 also has a playback device 15 that plays back content.

The processor 11 executes a reproduction process 116 for reproducing the content specified by the reproduction device 15 according to the operation signal input from the operation unit 17 . The playback device 15 plays back the designated content according to the control signal from the processor 11 .

The reproduction process 116 includes a process of storing the interruption position information 21 indicating the position where the reproduction of the content was interrupted as the content information 122 of the reproduced content in the memory 12 . As a result, when the reproduction of content is interrupted by the reproduction device 15 , the interruption position information 21 for that content is stored in the memory 12 .

The summary generator 10 may have a display 14 . Display 14 is an example of an output device that outputs the generated summary. The output device may be one that performs other forms of output such as a speaker instead of or in addition to the display 14 . If the summary generation device 10 also serves as a content reproduction device, the display 14 is also an example of an output device for reproduced content.

The abstract generation device 10 may have a communication device 13 capable of communicating with other devices via a network such as the Internet. As an example, the communication device 13 may output the generated summary to another device. In that case, the communication device 13 is also an example of an output device that outputs the generated summary. Also, the content information 122 is stored in another device, and the summary generation device 10 may read the content information 122 from the other device by the communication device 13 accessing the other device.

As an example, the target content may consist of one or more seasons (terms), as shown in FIG. Each season makes up a story, and one or more seasons may make up a larger story as a whole.

Each season may be divided into a plurality of episodes (talks), for example. An episode is one complete story, and target content is assumed to be reproduced for each story. Specifically, Season 1 includes a plurality of episodes EP11, EP12, . . . EP1n. Season 2 includes multiple episodes EP21, EP22, . . . EP2n. Each episode contains one or more sentences Q, either spoken or written.

A partial summary may be prepared for each episode of the target content. A partial summary refers to a summary that covers the previous episode. Specifically, episode EP12 includes a partial summary AB12 whose summary target range is episode EP11, and episode EP1n includes a partial summary AB1n whose summary target range is episodes EP11 to EP1(n−1).

An example in which a partial summary is prepared for the target content will be used in the second and subsequent embodiments. Do not use partial summaries.

If the target content is played in units of episodes as expected, the partial summary is played before the episode is played. Therefore, the user can check the synopsis up to the previous episode before playing back the episode.

Specifically, sentences CS1, CS2, CS3, . Specifically, the constituent sentences CS1, CS2, CS3, .

With reference to FIG. 1 , the summary generating process executed by processor 11 of summary generating device 10 includes first determination process 111 . The first determination process 111 includes determining the expected reproduction range (first range) of the target content from the interruption position. The expected reproduction range is a range after the interruption position and a predetermined range from the interruption position. The predetermined range may be a preset range. As another example, the predetermined range may be determined based on user attributes, target content attributes, user playback tendencies, and the like. The user's playback tendency is, for example, the user's playback behavior. Alternatively, an average value of general regeneration amounts may be used.

The summary generation process includes a second decision process 112 . A second determination process 112 includes determining a summary target range (second range) for the target content. The summary target range may be at least partially different from the expected playback range. That at least a part of the range may be different may be a completely different range, or the range may partially overlap.

A summary target range is determined based at least on the break position. Preferably, the summary target range is determined based on both the break position and whether the summary to be generated is a synopsis or an advance preview. A method for determining a summary target range will be described with reference to FIG.

In FIG. 3, an arrow indicates content C, which is the target content, and indicates that the content is reproduced in the direction of the arrow, that is, from left to right in chronological order. A position P0, which is the starting point of the arrow in FIG. In other words, FIG. 3 shows how content C is played back from the beginning of a certain season. The partial summaries ABn and ABn+1 shown in FIG. 3 may not be prepared for the content C. It is assumed that these partial summaries are not included in the content C in the first embodiment.

Position P1 is the start position of episode n and position P4 is the end position of episode n. The position P2 is a position corresponding to the interruption position indicated by the interruption position information 21 included in the content information 122 of the content C, and corresponds to the next start position. The range H3 from the interruption position P2 to the position P3 of the content C is assumed to be the expected reproduction range determined by the first determination processing 111. FIG.

In a second determination process 112, the processor 11 determines a summary target range based on the interruption position P2. In this example, the processor 11 determines the scope of the summary based on both the position of the interruption and whether the summary to be generated is a synopsis or an advance notice.

When the summary to be generated is an advance notice, as an example, the processor 11 sets the range H4 from the start position P0 of the season to the position P3 after the interruption position P2 as the summary target range. As another example of the advance notice, the processor 11 may set the range H3 from the position P2 to the position P3, which matches the expected reproduction range, as the summary target range. That is, when the summary is an advance notice, the range to be summarized may include the range before the interruption position P2, or only the range after the interruption position P2. As a result, the generated announcement includes sentences extracted from the expected reproduction range. Therefore, the user can imagine the contents of the expected reproduction range, and the desire to reproduce is heightened.

Note that if the position P3 is too close to the position P4, that is, if the summary target range is up to the end of the episode, the possibility that the end of the episode will be included in the preview increases. That is, there is a possibility that it will become a so-called spoiler. Therefore, preferably, the summary target range when the summary is an advance notice is the range up to the position before the end position P4 of the episode.

When the summary to be generated is a synopsis, as an example, the processor 11 sets the range H5 from the start position P0 of the season to the discontinuation position P2 as the summary target range. As a result, the generated synopsis includes sentences extracted from the range up to the interruption position P2. Therefore, the user is reminded of the content up to the interruption position P2, and is motivated to reproduce.

The abstract generation process executed by processor 11 includes extraction process 114 . The extraction process 114 includes extracting sentences from the scope of the summary as constituent sentences of the summary. In the extraction process 114 , the processor 11 uses the extraction data 34 of the reference content selected by the selection process 111 .

The summary generation process includes calculation process 113 . Calculation processing 113 includes calculating the index value of each sentence included in the summary range from the expected reproduction range. The index value includes a value obtained based on words included in the expected playback range. Values obtained based on words included in the expected playback range are, for example, importance. Specifically, the processor 11 calculates the index value of each sentence included in the summary target range using the words included in the expected reproduction range.

Here, the degree of importance is a value representing the importance in the expected playback range of words included in each sentence included in the range to be summarized. A specific calculation method of importance is not limited. The degree of importance may be, for example, the appearance frequency of words. For example, it may be calculated that the higher the appearance frequency in the expected playback range of a phrase included in each sentence included in the summary target range, the higher the importance of that sentence.

Also, instead of or in addition to the frequency of appearance, the importance may be calculated based on the reproduction status of the phrase. Regarding the playback status of phrases, for phrases included in each sentence included in the summary target range, the excitement of voice and background sound (volume, range, etc.) during playback within the expected playback range, changes in the brightness and color of the video, and changes in the video refers to a specific object, etc.

As another example, the concept of so-called page rank may be used to calculate the degree of importance. That is, with regard to the words included in each sentence included in the range to be summarized, the more often the word is referred to in the expected reproduction range, the higher the importance of the sentence is calculated. A function stored in advance may be used for the calculation. As another example, the degree of importance may be calculated using the degree of relevance to other words in the expected playback range, or a combination thereof.

Preferably, the index value includes similarity with the subset. For example, a subset refers to a set of sentences sequentially extracted when a plurality of sentences to be constituent sentences are sequentially extracted from the summary target range by the extraction process 114 described later. The degree of similarity is, for example, the degree of similarity with respect to a subset. In this case, the index value is, for example, an MMR (Maximal Marginal Relevance) score calculated using importance and similarity.

Using the importance I(q) for each sentence q included in the summary target range and the similarity Sim(q, k) for the subset k, the processor 11 uses the following equation (1) for each sentence Calculate q's MMR score MMR(q). Note that the coefficient λ is a value of 0 or more and 1 or less. As an example, the coefficient λ is set to 0.5.
MMR(q)=λI(q)−(1−λ)Sim(q,k) (1)

As shown in Equation (1), the closer the coefficient λ is to 1, the more important the MMR score is, and the closer the coefficient λ is to 0, the more important is the similarity. When importance is placed on importance, the MMR score increases for sentences that include words and phrases with higher importance in the expected reproduction range. On the other hand, when the similarity is emphasized, a sentence with a lower similarity to the subset has a higher MMR score.

When extracting sentences with a high MMR score in extraction processing 114, which will be described later, sentences that include words and phrases with a high degree of importance in the expected playback range and that are not similar to the subset are more likely to be extracted. As a result, the constituent sentences can be related to the expected playback range and can be a diverse set of sentences.

The abstract generation process includes extraction process 114 . The extraction process 114 includes extracting sentences from the summary range as constituent sentences of the summary based on the index value of each sentence included in the summary range. It is assumed that the number of sentences used as constituent sentences is defined in advance. In this case, as an example, the processor 11 extracts, as constituent sentences, up to a specified number of sentences in descending order of the index value among the sentences included in the range to be summarized.

A specific example of the extraction processing 114 will be described with reference to FIG. In the example of FIG. 4, the MMR score is used as the index value. Assume that sentences q1 to q4 are included in the summary target range. In calculation processing 113, the importance of each of sentences q1 to q4 is calculated based on the words included in the expected reproduction range.

Referring to FIG. 4, at the start of extraction processing 114 when not even a single sentence is extracted as a constituent sentence, the subset is 0, so the similarity of each of sentences q1 to q4 with respect to the subset is calculated as 0. be. Therefore, at this time, the MMR scores of sentences q1 to q4 match the importance. When the magnitude relation of each MMR score is q2>q3>q1>q4, in the extraction process 114, the sentence q2 with the highest MMR score is extracted as a constituent sentence (step S1).

When the sentence q2 is extracted, the degree of similarity with the subset is calculated for each of the sentences q1, q3, and q4 included in the summary target range after extraction. The subset here is sentence q2. MMR scores of sentences q1, q3, and q4 are calculated using the calculated similarities (step S2). If the magnitude relation of each MMR score is q4>q3>q1, the sentence q4 with the highest MMR score is extracted as a constituent sentence in the extraction process 114 (step S3).

In extraction processing 114, processor 11 repeats the above processing until a specified number of sentences are extracted. As a result, sentences related to the expected playback range are extracted as constituent sentences, and the relationships between multiple sentences in the entire summary are also taken into consideration. When the MMR score is used as an index value, sentences with low similarity are likely to be extracted with respect to a subset of previously extracted sentences, so there is a high possibility of generating a well-balanced summary.

Note that the MMR score is an example of an index value using importance and similarity. As another example, an index value Iv obtained by adding the degree of similarity to the degree of importance may be used, as in Equation (2) below.
Iv(q)=λI(q)+(1−λ)Sim(q,k) (2)

When extracting sentences with a high index value Iv in extraction processing 114, which will be described later, sentences that include words and phrases with a high degree of importance in the expected playback range and that are similar to the subset are more likely to be extracted. As a result, the constituent sentences can be a set of sentences that are related to the expected playback range and have unity in content.

The summary generation process includes generation process 115 . Generation processing 115 includes arranging sentences extracted as constituent sentences in extraction processing 114 . Here, the arrangement method is not limited to a specific method. As an example, it may be arranged according to the order of appearance in the summary target range. Another example may be a method of arranging according to the magnitude of the calculated index value.

At this time, the processor 11 may group the plurality of sentences by using the similarity or contrast between the plurality of sentences and arrange them in units of groups. This allows a natural summary to be generated using multiple sentences, such as dialogues, when they are grouped.

A summary generation method according to the present embodiment will be described with reference to FIG. The process shown in the flowchart of FIG. 5 is a summary generation process according to the summary generation method according to the present embodiment, and is implemented by processor 11 executing generation program 121 . The processing in FIG. 5 is started when the reproducing device 15 reproduces the target content or receives a user operation instructing presentation of a summary.

Referring to FIG. 5, processor 11 reads the interruption position from content information 122 of the target content, and determines the expected reproduction range based on the interruption position (step S101). In step S101, as an example, the processor 11 sets a preset range from the interruption position as the expected reproduction range.

Also, the processor 11 determines a summary target range based on both the interruption position and whether the summary to be generated is a synopsis or an advance notice (step S103). Either step S101 or step S103 may be processed first.

Processor 11 calculates the importance of each sentence included in the summary range determined in step S103 based on the words included in the expected reproduction range determined in step S101 (step S105). In addition, the processor 11 uses the sentences already extracted as a summary constituent sentence as a subset, and calculates the similarity to the subset for each sentence that is not extracted as a summary constituent sentence among the sentences included in the summary target range ( step S107).

The processor 11 substitutes the degree of importance calculated in step S105 and the degree of similarity calculated in step S107 into the above equation (1), thereby extracting a summary constituent sentence from among the sentences included in the range to be summarized. An MMR score is calculated as an example of an index value for each sentence that has not been processed (step S109). Processor 11 then extracts a sentence with the highest MMR score as a constituent sentence (step S111).

If the number of sentences extracted as constituent sentences does not reach the specified number (NO in step S113), processor 11 repeats steps S107 to S111. As a result, sentences to be constituent sentences are sequentially extracted. Each time one sentence is extracted as a constituent sentence, the number of sentences to be a subset increases, and accordingly the similarity of unextracted sentences is recalculated. Therefore, the MMR score changes every time one sentence is extracted as a constituent sentence.

When the number of sentences extracted as constituent sentences reaches the prescribed number (YES in step S113), processor 11 arranges the extracted sentences to generate a summary (step S115).

[Second embodiment]

The summary generation device 10 according to the second embodiment generates a summary using a partial summary prepared for the target content in the summary generation process. In a second embodiment, processor 11 applies the Silver Standard Summary Algorithm (SSSA) for partial summarization. Regarding the application of SSSA, Yoshinori Yamanishi, Yoko Nishihara, and Daichi Kaneda, ``Generation of next notice of novel by application of partial summary and Silver Standard Summary Algorithm'', [online], June 9, 2020, The Japanese Society for Artificial Intelligence , [Retrieved on June 9, 2020], disclosed on the Internet <URL: https://doi.org/10.11517/pjsai.JSAI2020.0_3K5OS5b01>.

In SSSA-applied processing, the processor 11 uses a partial summary prepared for the target content as a gold standard. In this case, in the second embodiment, the summary generation process further includes a third decision process 117, as represented in FIG. A third determination process 117 includes determining a partial summary to be used for generating a summary among a plurality of partial summaries prepared for the target content as the gold standard, based on the interruption position.

In a third decision process 117, processor 11 converts the partial summaries associated with positions that are within the gold standard decision range according to the break position and whether the summary to be generated is a synopsis or a preview to gold. Decide on standard. As a result, partial summaries corresponding to episodes whose start position is close to the interrupt position are determined as the gold standard. A specific example is a partial summary corresponding to the episode following the episode to which the interruption position belongs. If the summary to be generated is an advance notice, as an example, the gold standard decision range is the range from the break point onwards.

A third decision process 117 includes determining a gold standard to silver standard summary depending on the position of the break and whether the summary to be generated is a synopsis or a preview. The silver standard abstract refers to the sentences that are used to generate the abstract among the constituent sentences of the gold standard. As an example, the processor 11 determines to be a silver standard summary a sentence whose appearance position in the target content is within the silver standard summary determination range from the interruption position, among the constituent sentences of the gold standard. If the summary to be generated is an advance notice, as an example, the silver standard summary decision range is the range from the interruption position to the future.

A control method according to the second embodiment will be described with reference to FIG. A specific example of the control method according to the second embodiment will be described with reference to FIGS. 3 and 8 to 10. FIG. 8 to 10 show an example of generating a summary with target content C of "Night on the Galactic Railroad" (quoted from Kenji Miyazawa's "Night on the Galactic Railroad" Aozora Bunko). Sentence numbers in the leftmost column are numbers assigned to all sentences in order from the beginning. Here, a case will be described in which the summary target range generates an advance notice including the range after the interruption position.

The flowchart of FIG. 7 differs from the flowchart of FIG. 5 showing a specific example of the abstract generation method according to the first embodiment in the processing of steps S201 and S203. That is, in the summary generation method according to the second embodiment, the processor 11 determines, as a gold standard, a partial summary to be used for generating the summary, out of the partial summaries prepared for the target content (step S201), Among the constituent sentences, the sentences used for generating the abstract are determined as the silver standard abstract (step S203).

In the example of FIG. 3, at position P1 of content C, a partial summary ABn corresponding to episode n is arranged. A partial summary ABn+1 corresponding to the next episode n+1 is placed at position P4. These partial summaries ABn and ABn+1 are shown in the summary information 22 included in the content information 122 of the content C. FIG.

The summary target range of the partial summary ABn is the range H1. That is, the partial summary ABn in this example is a synopsis of the episode n with the range from position P0 to position P1 to be summarized. The summary target range of partial summary ABn+1 is range H2. That is, the partial summary ABn+1 in this example is a synopsis of the episode n+1 with the range from position P0 to position P4 as the summary target range.

If the summary to be generated is an advance notice, as an example, the gold standard determination range is set to the range H6 from the interrupt position. In this case, in the third determination process 117, if the partial summary associated with the position in the range H6 from the interruption position P2 is determined as the gold standard, in the example of FIG. The partial summary associated with position P4 is determined as the gold standard.

If the summary to be generated is an advance notice, as an example, the silver standard summary decision range is set to the range H6 from the interruption position. In this case, the processor 11 determines, among the constituent sentences of the gold standard, sentences whose appearance position in the target content is within the range H6 beyond the interruption position to be the silver standard summary. Here, as an example, the range for determining the partial summary to be the gold standard and the range for determining the silver standard summary among the constituent sentences of the gold standard are the same range H6, but these ranges may be different. good.

Referring to FIG. 8, it is assumed that "Night on the Galactic Railroad" (quoted from Kenji Miyazawa's "Night on the Galactic Railroad" Aozora Bunko) has been read up to sentence number 130. In this case, sentence number 130 is stored in content information 122 as interruption position P2.

If the end of episode n to which sentence number 130 belongs is sentence number 191, sentence number 192 is position P4, which is the start position of the next episode n+1. In this case, for example, as shown in FIG. 8, the partial summary ABn+1 corresponding to the episode n+1 is arranged immediately before the sentence number 192 .

The range H2, which is the summary target range of the partial summary ABn+1, is from sentence number 1 to sentence number 191. Position P4 corresponding to partial summary ABn+1 is included in range H6 after interruption position P2. Therefore, processor 11 determines in a third decision process 117 that partial abstract ABn+1 is the gold standard.

FIG. 9 shows a specific example of a constituent sentence of a partial summary ABn+1. Referring to FIG. 9, the partial summary ABn+1 is, for example, eight sentences with sentence numbers 5, 21, 31, 55, 120, 163, 170, and 191 extracted from range H2 (sentence numbers 1 to 191). is a constituent sentence.

At this time, the group K1 with sentence numbers 5, 21, 31, 55 and 120 is extracted from before the interruption position P2, and the group K2 with sentence numbers 163, 170 and 191 is extracted after the position P2. That is, the group K2 is a group of sentences included in the range H6 after the interruption position P2, and the group K1 is a group of sentences not included. Therefore, processor 11 determines group K2 as a silver standard summary in a third decision process 117 .

Also in the abstract generation method according to the second embodiment, processor 11 generates a summary in the same manner as in the abstract generation method according to the first embodiment. That is, referring to FIG. 7, processor 11 selects the silver standard summaries (group K2: sentence numbers 163, 170, 191) determined in step S203 as a subset, and extracts the summaries of the sentences included in the range to be summarized. For each sentence that has not been extracted as a constituent sentence, the degree of similarity with respect to the subset is calculated (step S107).

Note that, as another example here, the processor 11 may extract one or more sentences that were not made into the silver standard summary (for example, group K1: sentence numbers 5, 21, 31, 55, and 120), or parts that are the gold standard. All the constituent sentences of the summary (for example, group K1+K2: sentence numbers 5, 21, 31, 55, 120, 163, 170, and 191) are extracted as a subset of the sentences included in the scope of the summary as the summary constituent sentences. For each sentence that does not have a similarity to this subset, a similarity score may be calculated.

Processor 11 uses the degree of importance and the degree of similarity to calculate an MMR score for each sentence that is not extracted as a summary constituent sentence among the sentences included in the summary target range (step S109). Processor 11 then extracts a sentence with the highest MMR score as a constituent sentence (step S111).

If the number of sentences extracted as constituent sentences does not reach the specified number (NO in step S113), processor 11 repeats steps S107 to S111.

As an example, assume that sentence numbers 131, 132, 133, 134, and 138 after sentence number 130, which is the reading end position, are extracted from the range to be summarized. When generating an advance notice, as shown in FIG. 10, the processor 11 adds newly extracted sentence numbers 131, 132, 133, 134 and 138 (group K3) are added to form the constituent sentences of the summary.

In FIG. 10, the composition sentences of the summary as an advance notice may include both the sentences before and after the interruption position P2. In other words, the constituent sentences of the summary as an advance notice are not limited to the sentences after the interruption position P2. For example, if even one sentence after the interruption position P2 is included in the constituent sentences, the summary may be treated as an advance notice.

By using the gold standard in this way, the number of sentences to be extracted is smaller than that of extracting all sentences of the constituent sentences, and processing is facilitated. That is, the number of times steps S107 to S113 are repeated can be reduced. Also, by determining the silver standard abstract from the gold standard and using it as a subset, it is possible to generate a more suitable abstract.

[Third embodiment]

The summary generation device 10 according to the third embodiment uses reference content in the summary generation process. The reference content refers to content other than the target content that is referred to when extracting the constituent sentences of the summary of the target content, and for which a summary has been prepared.

As an example, it is assumed that the summary generation device 10 acquires information about content used as reference content from another device. FIG. 11 is a diagram showing a summary generation device 10 according to the third embodiment. As shown in FIG. 11 , the summary generation device 10 according to the third embodiment can communicate with the server 30 as another device through the communication device 13 .

The server 30 stores content data 31A, 31B, . . . 31 associated with each of a plurality of contents. The content data 31A, 31B, . The fact that the content data 31 is associated with the content means that the content data 31 does not have to include the content itself, but includes information such as a name and an identifier indicating the content.

In the example of FIG. 11, the server 30 is an external device of the summary generation device 10 and is accessed by the communication device 13 via the network 70 . However, as another example, the server 30 may be a storage device installed in the summary generating device 10. FIG.

The content data 31 each include attributes 32 . The attribute 32 is information representing the characteristics of the story of the content, and is, for example, a value obtained by converting the sentences of the entire story into word vectors. Also, the attributes 32 may be meta-information such as genre, scriptwriter, season number, characteristics of the playback target, etc. instead of or in addition to word-vectorized values of the sentences of the entire story.

Content data 31 that can be used as reference content includes partial summaries 33 . FIG. 11 shows a case where content data 31 includes a plurality of partial summaries 33A, 33B, 33C, . The partial summaries 33A, 33B, 33C, .

Each content data 31 includes extraction data 34 . The extraction data 34 is data representing the tendency of the position of each constituent sentence of one or more partial summaries 33A, 33B, 33C, . . . in the content. The extraction data 34 will be specifically described with reference to FIG. 12 .

12A and 12B are diagrams showing the distribution in the actual contents A and B of the sentences constituting the partial summary in the actual contents A and B. FIG. The horizontal axis of FIG. 12 indicates the episode number EP, and the vertical axis indicates the extraction position PP from the summary target range of the constituent sentences. FIG. 12 plots the extraction position PP of each of the sentences constituting the corresponding partial summary for each episode.

Contents A and B are serial dramas that have been aired for three or more seasons and are popular all over the world. FIG. 12 shows the extraction position PP of sentences constituting partial summaries of all episodes of all three seasons, using up to three seasons out of all seasons of contents A and B, extracted from the summary target range. there is Partial summaries use sentences within 3 minutes of the beginning of each episode.

The extraction position PP(q) of sentence q extracted as a constituent sentence is obtained by the following procedure. That is, the expression (3) below obtains a value EPe(q) obtained by normalizing the appearance positions in all sentences (number Ne) included in episode e from which sentence q is extracted.
EPe(q)=Pe(q)/Ne (3)

The value EPe(q) is then used to obtain the absolute position AP(q) defined in equation (4) below.
AP(q)=EPe(q)+(e−1) (4)

Then, the extracted position PP(q), which is the past relative position, defined by the following equation (5) is obtained. The past relative position represents the relative position of the sentence q when the episode number e', which is the object of the partial summary with the sentence q as a constituent element, is set to 1.
PP(q)=AP(q)/(e'-1) (5)

As an example, if the absolute position AP of the constituent sentence of the partial summary corresponding to episode number 3 is 1.4591, the extraction position PP=0.7295 is obtained from equation (5). Representing the extracted position PP by the past relative position makes it possible to consider the relative positional relationship in the range from the start of the season to the position of the partial summary. As a result, it becomes possible to compare and consider the extraction position of the sentence beyond the contents.

The inventors, including the contents A and B, for a plurality of contents that are serial dramas that are highly popular in the world and have been aired for three seasons or more, have the composition sentences of the partial summaries in the same way as the contents A and B. We examined the distribution of the sentences in the content.

As a result, as shown in FIG. 12, the inventors have noticed a tendency for the extraction positions PP to concentrate at 0.0 and 1.0 for Season 1, common to multiple contents. The smaller the extraction position PP value (the closer it is to 0), the closer the extraction position is to the beginning of the summary target range, ie, the beginning of the season. A larger value (closer to 1) indicates that the extraction position is closer to the end of the summary target range, ie, the end of the episode immediately preceding the corresponding episode. Therefore, it was considered that, common to multiple contents, sentences used as constituent sentences for partial summaries tended to be biased toward the beginning and end of the scope of the summary for Season 1 (Examination 1).

In addition, the inventors noticed that, in common with a plurality of contents, the plot of the extraction position PP continuously descends to the right near the middle between 0.0 and 1.0. It was considered that each partial summary tends to use sentences at the same position in the corresponding season as constituent sentences (Consideration 2). In other words, it was considered that there is a tendency to use specific high-importance sentences in each partial summary in common with multiple contents.

Moreover, the inventors have noticed that the distribution tendencies of the extraction positions PP of the seasons 1 to 3 may differ between the contents A and B, as shown in FIG. It was considered that the tendency to extract sentences to be used as constituent sentences for partial summaries may differ from content to content and/or season to season (Consideration 3).

Based on considerations 1 and 2, the inventors divided the scope of the summary into three parts, the beginning, the middle, and the end. It shall be used as As an example, the beginning s, middle c, and end e range from the beginning of the season to 20%, 60%, and 20%, respectively.

Focusing on the episode 50 of the content A indicated by the dotted line in FIG. , and 20% at the end e. That is, the partial summary corresponding to episode 50 of content A consists of 50% sentences from the beginning s, 30% from the middle c, and 20% from the end e of the range from the start of season 3 to the end of episode 49. It is read that it was extracted and generated as a sentence.

The extraction data 34 represents the extraction ratio of the sentences constituting the partial summary corresponding to each episode from each segment. In other words, the extraction data 34 can be said to be data representing the tendency of the relative positions in the content of the sentences constituting the partial summary of the content.

As an example, the extraction data 34 can be presented in a tabular form as shown in FIG. That is, referring to FIG. 13, the extraction data 34 of the content A includes the beginning s , the middle stage c, and the final stage e. For example, for episode 50, 50%, 30%, and 20% are defined for the beginning s, middle c, and end e, respectively.

In addition, from consideration 3, the inventors decided to use the content suitable for the target content as the reference content for the summary generation process. Therefore, as shown in FIG. 11, the server 30 stores a plurality of content data 31A, 31B, .

Referring to FIG. 11, in the third embodiment, the summary generation process further includes selection process 118. FIG. The selection process 118 includes selecting content for reference. The content for reference is selected from content data 31A, 31B, . . .

In the selection process 118 , based on the attributes 32 included in the content data 31 , the content data 31 whose story characteristics are related to the target content are selected. In the selection process 118, as an example, content whose value obtained by word-vectorizing sentences of the entire story is within a predetermined range from the value of the target content is extracted as reference content. As another example, content matching or approximating at least one of the genre, script writer, season number, and characteristics of the person to be played is extracted as the reference content. good too.

In the third embodiment, in the extraction process 114, the processor 11 uses the extraction data 34 associated with the reference content to extract sentences that constitute the summary from the target content. Specifically, the processor 11 refers to the extraction ratio of the beginning s, middle c, and end e of the summary target range for the summary of the corresponding episode shown in the extraction data 34 associated with the reference content. . The applicable episode is an episode whose start position is near position P2, and in the case of the example of FIG. 6, it is episode n+1, for example. Also, in the case of this example, it may be episode n.

Assume that the extraction ratios of the beginning s, middle c, and end e of the summary target range for the summary ABn+1 of the applicable episode n+1 are 50%, 30%, and 20%, respectively, shown in FIG. In this case, processor 11 applies this ratio for the subject content to extract sentences based on MMR scores. In this example, if the number of constituent sentences is 10, 5, 3, and 2 sentences are extracted from the beginning s, middle c, and end e of the range to be summarized, respectively, based on the MMR score.

As a result, for the target content, a summary is generated by extracting sentences from the range of the target content with the same tendency as the partial summary prepared for the reference content having the story characteristic. Therefore, a summary can be easily generated.

The summary generation device 10 according to the third embodiment may further use placement data in the summary generation process. The placement data is an evaluation value of the discrepancy between the positional order of sentences in the content for which the summary is prepared and the placement order in the partial summary. The placement data 35 is, for example, the Jaro-Winkler distance.

In this case, as shown in FIG. 11, each of the content data 31 includes placement data 35 . The arrangement data 35 of the content A is, for example, the beginning s, middle c, and end e of the summary target range of the sentences constituting the partial summary for all episodes 1 to 69 of the content A up to season 3. shows the extraction ratio of For example, for episode 50, 50%, 30%, and 20% are defined for the beginning s, middle c, and end e, respectively.

The inventors have noticed that the arrangement order in the partial summary differs from the positional order in the content, that is, there is a mismatch in some cases. Therefore, for a plurality of contents including content A, the Jaro-Winkler distance was calculated and verified as an index value representing the discrepancy between the arrangement order in the partial summary and the appearance order in the content. The closer the Jaro-Winkler distance is to 1, the closer the arrangement order in the partial summary is to the positional order in the content, that is, the smaller the mismatch.

When the inventors investigated a large number of works and calculated the average value of the Jaro-Winkler distance for each season, it was found that most of the works ranged from 0.65 to 0.85. This indicates that sentences are often arranged in a different order than the content in the partial summary.

The greater the discrepancy between the arrangement order in the partial summary and the appearance order in the content, the more difficult it is for the summary to fully understand the content. Conversely, it is believed that the smaller the discrepancy, the easier it is for the content to be understood. Therefore, it is conceivable that they are used properly according to the attribute of the content.

In other words, in the case of content categories such as suspense, where it is preferable that the contents of the content are not completely understood by summarization, appropriate inconsistency is achieved depending on the scope of the summary, etc., so that the understanding of the content and the willingness to play are balanced. It was considered to be planned. Therefore, in the summary generation device 10 according to the third embodiment, the arrangement data 35 prepared for each content may be used to generate the summary of the target content.

The placement data 35 can be presented in a tabular format as shown in FIG. 14, for example. The placement data 35 in FIG. 14 shows, as an example, placement data (Jarro-Winkler distance) for seasons 1, 2, and 3 of contents A, B, and C, and all seasons. The Jaro-Winkler distance is 1 when the positional order of the sentences in the content completely matches the arrangement order in the partial summary, and 0 when there is no similarity. In addition, in the example of the placement data 35 in FIG. 14, the value of the placement data is shown for each season, but one value may be shown for each content.

In the summary generating apparatus 10 according to the third embodiment, the content placement data related to the target content is used for placement of sentences extracted as constituent sentences. As an example, data for placement of reference content is used. Alternatively, in the summary generation method according to the third embodiment, the arrangement order of the extracted sentences in the summary is matched with the appearance order in the target content, or arranged using the arrangement data of the related content. or may be selectable. The selection may be made, for example, by the summary generator.

Specifically, in the generating process 115 , the processor 11 arranges the plurality of sentences extracted as constituent sentences based on the arrangement data 35 . As an example, the processor 11 arranges the extracted sentences so that the Jaro-Winkler distance is the same as the Jaro-Winkler distance indicated in the placement data 35 of the reference content, or a value within a predetermined range. Sort by. As a result, the content of the target content can be easily grasped from the summary in the same manner as that of the reference content.

<3. Note>
The present invention is not limited to the above embodiments, and various modifications are possible. For example, at least two of the first to third embodiments may be combined.

10: Summary generation device 11: Processor 12: Memory 13: Communication device 14: Display 15: Playback device 17: Operation unit 21: Interruption position information 22: Summary information 30: Server 31: Content data 31A: Content data 31B: Content data 32: attribute 33: partial summary 33B: partial summary 33C: partial summary 34: extraction data 35: placement data 70: network 111: selection processing 112: second determination processing 113: calculation processing 114: extraction processing 115: generation Process 116: Playback process 117: Third decision process 118: Selection process 121: Generation program 122: Content information AB12: Partial summary AB1n: Partial summary ABn: Partial summary C: Content CS1: Composition sentence CS2: Composition sentence CS3: Composition Sentence EP10 : Episode EP11 : Episode EP12 : Episode EP1n : Episode EP21 : Episode EP22 : Episode H1 : Range H2 : Range H3 : Range H4 : Range H5 : Range H6 : Range K1 : Group K2 : Group K3 : Group P0 : Start position P2 : Interruption position PP : Extraction position q : Sentence q1 : Sentence q2 : Sentence q3 : Sentence q4 : Sentence

Claims (8)

A summary generation device for generating a summary of target content,
with a processor
The processor
determining a first range in the target content from a playback interruption position of the target content;
an index value of a sentence included in a second range, which is determined from the playback interruption position and which is a range for extracting the constituent sentences of the summary in the target content, and which is at least partially different from the first range; , calculated from said first range,
A summary generation device configured to extract constituent sentences of the summary from the second range based on the index value. 2. The summary generation device according to claim 1, wherein said index value includes a value obtained based on words included in said first range. 3. The summary generating apparatus according to claim 1, wherein the first range is a range after the reproduction interruption position and is an expected reproduction range of the target content determined from the reproduction interruption position. the second range is determined by whether the summary is a first summary or a second summary;
The first summary is a summary generated with a range that does not include after the reproduction interruption position as the second range,
The summary generation device according to any one of claims 1 to 3, wherein the second summary is a summary generated with a range including after the reproduction interruption position as the second range. Extracting the constituent sentences of the summary includes sequentially extracting a plurality of sentences included in the constituent sentences from the second range based on the index value, thereby obtaining a portion having the sequentially extracted sentences. generating a set;
5. The summary generation device according to any one of claims 1 to 4, wherein said index value includes a degree of similarity between sentences included in said second range and said subset. 6. The summary generation device according to claim 5, wherein said subset includes sentences extracted from a summary prepared in advance for said target content. The processor further
configured to select reference content based on the target content;
Extracting constituent sentences of the summary from the second range includes:
7. The summary generation according to any one of claims 1 to 6, comprising extracting the constituent sentences of the summary from the second range by referring to extraction data associated with the reference content. Device. A summary generation method for generating a summary of target content, comprising:
determining a first range in the target content from a playback interruption position of the target content;
calculating, from the first range, an index value of a sentence contained in a second range, which is a range from which the constituent sentences of the summary in the target content are extracted and which is at least partially different from the first range; ,
extracting constituent sentences of the summary from the second range based on the index value.

Images