JP2018206110A - Device for automatically generating interactive-sentence containing moving picture - Google Patents

Abstract

To provide a technique capable of generating a moving picture to which an interactive sentence is added in a device for automatically generating a moving picture from a plurality of still images.SOLUTION: A device for automatically generating an interactive-sentence containing moving picture according to the present invention computes a degree of image similarity between two still images in a plurality of still images input by an image input device. In addition, for each of the plurality of still images, the device extracts keywords for identifying objects included in each still image, and generates interactive texts related to the extracted keywords. Then, the device determines a reproduction order of a plurality of still images based on the degree of image similarity and a degree of linkage of the interactive sentences, and outputs the plurality of still images and subtitles or voices of the interactive sentences corresponding to the individual still images according to the reproduction order, thereby automatically generating an interactive-sentence containing moving picture.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an apparatus for automatically generating a moving image to which a dialogue sentence is assigned.

  In recent years, an apparatus for automatically generating a moving image from a plurality of still images has been developed. For example, in Patent Document 1, in a device that automatically generates a moving image from a plurality of still images, the narration of each still image is generated from the meta information of each still image, and the generated narration is associated with the still image. A technique for generating a narrated video has been proposed.

JP 2006-287521 A JP2013-101450A JP 2008-299493 A

  In the field of automatically generating moving images from multiple still images, there is an increasing need for the quality and diversity of the generated moving images. For example, moving images with interactive audio and subtitles added from multiple still images. A technology to generate is desired.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technology capable of generating a moving image to which a dialogue sentence is added in an apparatus that automatically generates a moving image from a plurality of still images. There is.

  An apparatus for automatically generating a dialogue sentence moving image according to the present invention is an apparatus that automatically generates a movie from a plurality of still images, and that automatically adds a dialogue sentence related to each still image to generate a movie. It is.

  More specifically, an apparatus for automatically generating a dialogue sentence moving image according to the present invention includes an image input unit for inputting a plurality of still images, and an image between two still images in the plurality of still images input by the image input unit. Image similarity calculating means for calculating the degree of similarity; keyword extracting means for extracting a keyword for identifying an object included in each still image for each of the plurality of still images input by the image input means; For each of a plurality of still images input by the image input means, a dialog sentence generating means for generating a dialog sentence related to the keyword extracted by the keyword extracting means, and an image similarity calculated by the image similarity calculating means The reproduction order of the plurality of still images based on the degree and the connection degree of the dialogue sentence generated by the dialogue sentence generation unit Includes a reproduction order determination means for determining, in accordance with the reproduction order determined by the reproduction order determination means, a reproduction means for outputting a subtitle or audio dialogue corresponding to the plurality of still images and the still image.

According to such an apparatus for automatically generating a dialogue sentence moving image, a dialogue sentence relating to an object included in each still image is automatically generated when a movie is generated from a plurality of still images. Then, the reproduction order of the still images is determined based on the image similarity degree and the connection degree of the dialogue sentence, and the still picture and the dialogue sentence corresponding to the still image are output according to the reproduction order. At that time, the dialogue sentence may be output in a subtitle format or may be output in an audio format. The moving image generated in this way has a high quality and rich variety as compared to a moving image generated considering only the continuity of images and a moving image to which a simple description of the image is added.

  Note that the “degree of connection of dialogue sentences” here is the degree of similarity of dialogue sentences between two still images. The degree of similarity of dialogue sentences between still images may be obtained, for example, by expressing a keyword of each still image as a vector and calculating a vector difference between the still images. Further, the degree of similarity of dialogue sentences between still images may be obtained by expressing words other than keywords included in the dialogue sentences of each still image as vectors and calculating vector differences between the still images. In these methods, the smaller the vector difference between still images, the higher the degree of interaction between the still images.

  Further, the reproduction order determining means in the present invention comprises a value obtained by multiplying the image similarity calculated by the image similarity calculating means by a first weighting factor, and a connection degree of the dialog sentence generated by the dialog sentence generating means. The cost value may be calculated by adding the value multiplied by the second weighting factor to the image, and the reproduction order of the plurality of still images may be determined based on the calculated cost value. According to such a configuration, it is possible to generate a more diverse moving image.

  Here, the reproduction order determination means includes the first weighting factor and the first weighting factor so that the weight of the connection degree of the dialogue sentence with respect to the degree of image similarity is larger when the time length of the generated moving image is long than when the time length is short. A second weighting factor may be determined. According to such a configuration, when the time length of the generated moving image is short, the moving image is generated with an emphasis on the degree of image similarity as compared to the degree of connection of the dialogue sentence, so that the image transition is uncomfortable. It is possible to generate a moving image with little. In addition, when the time length to be generated is long, a moving image is generated with an emphasis on the degree of connection of dialogue sentences compared to the degree of image similarity, and thus a moving image with high storyliness can be generated. Therefore, the quality of the moving image can be further improved.

  The reproduction order determining means determines the first weighting factor and the second weighting factor so that the weight of the connection degree of the dialogue sentence with respect to the image similarity degree changes in the middle of the generated moving image. Also good. For example, the first weighting coefficient and the second weighting coefficient may be determined so that the weight of the connection degree of the dialogue sentence with respect to the image similarity degree differs between the first half and the second half of the generated moving image. Further, the first weight coefficient and the second weight coefficient may be determined so that the weight of the connection degree of the dialogue sentence with respect to the image similarity degree is different in the early stage, middle stage, and end stage of the generated moving image. Note that the weight of the connection degree of the dialogue sentence with respect to the image similarity degree may be changed stepwise as described above, but may be changed continuously from the start to the end of the moving image. In this way, when the weight of the connection degree of the dialogue sentence with respect to the image similarity degree is changed in the middle of one moving picture, a more diverse moving picture can be generated.

  In addition, the dialog sentence generation means in the present invention may generate a plurality of dialog sentence candidates for one still image. In that case, the reproduction order determining means may select one dialogue sentence suitable for the still image from among the plurality of dialogue sentence candidates based on the appearance position of the word included in each dialogue sentence candidate. According to such a configuration, it is possible to assign a higher quality dialogue sentence to each still image.

The present invention can also be understood as a method for automatically generating a dialogue sentence moving image including at least a part of the above processing. For example, the method for automatically generating a dialogue sentence moving image includes a step of inputting a plurality of still images, a step of calculating an image similarity degree between two still images in the plurality of inputted still images, and a plurality of inputted plurality of still images. Extracting a keyword for identifying an object included in each still image for each of the still images; and generating a dialogue sentence related to the extracted keyword for each of the plurality of input still images. Determining a playback order of the plurality of still images based on the generated connection level of the dialogue sentences and the image similarity level, and determining the plurality of still images and each still image according to the determined playback order. Outputting a subtitle or audio of a corresponding dialog sentence.

  The present invention can also be understood as a program for realizing the above-described method for automatically generating a dialog sentence moving image and a recording medium on which the program is recorded. Each of the above means and processes can be combined with each other as much as possible to constitute the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the moving image to which the dialog sentence was added can be produced | generated in the apparatus which produces | generates a moving image automatically from several still images.

It is a block diagram which shows schematic structure of the automatic generation apparatus of the dialog sentence moving image concerning this invention. It is a figure which shows an example of the still image input by the image input device. It is a figure which shows an example of the method of specifying the object contained in a still image. It is a figure which shows an example of the table which shows the correlation with the feature word extracted from a dialogue sentence, and the reference | standard order of the appearance position in the dialogue sentence of a feature word. It is a figure which shows the operation | movement flow at the time of producing a dialogue sentence animation automatically by a computer.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The dimensions, materials, shapes, relative arrangements, and the like of the components described in the present embodiment are not intended to limit the technical scope of the invention to those unless otherwise specified.

  FIG. 1 is a block diagram showing a schematic configuration of an apparatus for automatically generating a dialogue sentence moving picture according to the present invention. As shown in FIG. 1, the automatic generation device 1 includes an image input device 2, an output device 3, and a computer 4. The image input device 2 is a device for inputting a still image, and is, for example, an image scanner or a digital camera. Note that the image input apparatus 2 may input a still image via a network, or may input a still image from a recording medium or the like. The output device 3 is a device that outputs a moving image with a dialogue sentence, and is configured by, for example, a display or a combination of a display and a speaker. The computer 4 is a personal computer or a workstation provided with a CPU, ROM, RAM, hard disk, user interface (for example, keyboard, mouse, touch panel) and the like.

  The computer 4 includes an image similarity calculation unit 40, a keyword extraction unit 41, a dialogue sentence generation unit 42, a reproduction order determination unit 43, and a reproduction unit 44. These functional units are realized, for example, when the CPU executes a program stored in a storage device such as a hard disk.

(Image similarity calculation unit 40)
The image similarity calculation unit 40 calculates, for each still image of a plurality of still images input by the image input device 2, the image similarity between the still image and another still image. This calculation process is performed for all combinations of two still images in a plurality of still images as moving image materials. For example, when three still images A, B, and C are input as the moving image material by the image input device 2, the image similarity in the combination of (A, B) and the image in the combination of (B, C) The similarity degree and the image similarity degree in the combination of (A, C) are calculated.

Note that the degree of image similarity may be evaluated using, for example, a difference between pixel values. In that case, 2
Assume that the smaller the difference in pixel value between still images, the higher the degree of image similarity between the two still images. Note that the degree of image similarity may be evaluated using an image feature amount other than the pixel value. The image similarity obtained in this way is stored in a storage device such as a RAM together with information for identifying a combination of two still images.

(Keyword extraction unit 41)
The keyword extraction unit 41 extracts a keyword for identifying a subject (object) included in each still image for each of a plurality of still images input by the image input device 2. For example, as shown in FIG. 2, when an object of a still image A is a red car, “red car” is extracted as a keyword of the still image A. It is assumed that a single still image includes a plurality of objects. Therefore, as shown in FIG. 3, each still image may be divided into a plurality of regions (four regions in the example shown in FIG. 3), and keywords may be extracted for each region. For example, as keywords of the still image B shown in FIG. 3, a plurality of keywords such as “red car”, “male”, and “traffic light” are extracted.

  For the keyword extraction as described above, a classifier created by machine learning can be used. Such a classifier prepares a lot of learning data consisting of combinations of still images and objects (“car”, “male”, “traffic light”, etc.) included in the still images, and applies existing machine learning algorithms. You can create it.

(Dialogue sentence generator 42)
The dialog sentence generation unit 42 generates a dialog sentence corresponding to each still image based on the keyword extracted by the keyword extraction unit 41. The “dialogue sentence” here is composed of a combination of a plurality of utterances in principle, but one dialogue sentence may be composed of one utterance. Furthermore, the “dialogue sentence” here is a dialogue sentence that includes a keyword extracted by the keyword extraction unit 41 or a word similar to the keyword in principle. For example, regarding the still image A illustrated in FIG. 2 described above, as a dialogue sentence including “red car” that is the keyword of the still image or “red car” similar to the keyword, “red car (red car) ) Is generated, and a dialogue sentence composed of a plurality of utterances such as “Like” is generated. In addition, regarding the still image B illustrated in FIG. 3 described above, “a man (man) near a signal seems to like a red car (red car)” and “I am a white car (white Create a dialogue sentence consisting of multiple utterances such as “I like the car”.

  Here, a classifier created by machine learning can be used to create the dialogue sentence as described above. Such a discriminator can be created by preparing a large number of learning data consisting of a combination of a keyword and a dialogue sentence related to the keyword, and applying an existing machine learning algorithm.

(Reproduction order determination unit 43)
The reproduction order determination unit 43 is input by the image input device 2 based on the image similarity degree calculated by the image similarity degree calculation unit 40 and the connection degree of dialogue sentences generated by the dialogue sentence generation unit 42. A reproduction order for continuously reproducing a plurality of still images as a moving image is determined. For example, the reproduction order determination unit 43 first calculates a cost value according to the following equation (1) for all combinations of two still images.
Cost value = w1 × (image similarity degree) + w2 × (dialog sentence connection degree) (1)
In the above equation (1), w1 is a first weighting factor, and w2 is a second weighting factor. These two coefficients w1 and w2 are coefficients that weight the transition of the still image and the transition of the dialogue sentence based on the subjective evaluation. These two coefficients w1 and w2 may be fixed values, but are changed according to the time length of the moving image to be generated (more specifically, the number of still images input by the image input device 2). It may be a variable value. At this time, when the time length of the generated moving image is long, w1 and w2 may be determined so as to place a weight on the connection degree of the dialogue sentence rather than the image similarity degree (w1 <w2). On the other hand, when the time length of the generated moving image is short, w1 and w2 may be determined so that the image similarity degree is weighted more than the connection degree of dialogue sentences (w1> w2). Further, the above two coefficients w1 and w2 may be changed in the middle of one moving image. For example, the coefficients w1 and w2 may be set to different values in the first half and the second half of the moving image. The coefficients w1 and w2 may be set to different values at the beginning, middle and end of the moving image. Furthermore, the coefficients w1 and w2 may be continuously changed from the start to the end of the moving image.

  The above-mentioned “degree of connection of dialogue sentences” is the degree of similarity of dialogue sentences between still images. The degree of similarity of dialogue sentences between still images may be obtained, for example, by expressing a keyword of each still image as a vector and calculating a vector difference between the still images. Further, the degree of similarity of dialogue sentences between still images may be obtained by expressing words other than keywords included in the dialogue sentences of each still image as vectors and calculating vector differences between the still images. In these methods, it is assumed that the smaller the vector difference between still images is, the higher the degree of connection of dialogue sentences between those still images is.

  When a plurality of dialogue sentence candidates for each still image are generated by the dialogue sentence generation unit 42, the reproduction order determination unit 43 performs the reproduction order determination process based on the above formula (1) before performing the reproduction order determination process. A process of selecting a dialogue sentence suitable for the still image from a plurality of dialogue sentence candidates is performed. Specifically, the reproduction order determination unit 43 first extracts feature words included in each dialogue sentence candidate. The “characteristic word” here is, in principle, a keyword extracted by the keyword extraction unit 41. When a characteristic word other than the keyword is included in the dialogue sentence, the word is a characteristic word. May be extracted as Next, the reproduction order determination unit 43 performs ranking based on the appearance position of each characteristic word. For example, when “signal”, “automobile”, and “male” are extracted as feature words of a dialogue sentence candidate “a car that is close to a signal is that man's?”, The dialogue sentence Since the appearance position of each feature word in the candidate is in the order of “signal”, “car”, “male”, the order of “signal” is “1”, the order of “car” is “2”, The ranking of “male” may be “3”. On the other hand, the reproduction order determination unit 43 specifies the reference order of each feature word. The “reference order” here is an average order of appearance positions of each feature word in a general dialogue sentence including the extracted feature words. Such a reference order is statistically obtained from many examples of dialogue sentences including characteristic words, and is stored in a storage device such as a ROM in a table format as shown in FIG. 4, for example. Then, the reproduction order determination unit 43 calculates the difference between the rank of each feature word and the reference rank, and calculates the sum of the differences, thereby obtaining the score of each dialogue sentence candidate. For example, when the score is calculated based on the reference ranking shown in FIG. 4 with respect to the above-mentioned dialogue sentence candidate “the car that is close to the signal belongs to that man”, the “signal” in the dialogue sentence candidate The difference between the ranking of “” and the reference ranking is “0”, the difference between the ranking of “automobile” in the dialogue candidate and the ranking is “1”, and the ranking of “male” in the dialogue candidate and the reference ranking Is “1”, and the sum of these differences is “2”. Therefore, the score of the dialogue sentence candidate “Is the car parked near the signal a thing of that man?” Is “2”. When the score is obtained for each of the plurality of dialogue sentence candidates assigned to each still image by such a method, the reproduction order determining unit 43 selects the dialogue sentence candidate having the lowest score among the plurality of dialogue sentence candidates. , And select as a dialogue sentence suitable for the still image. Then, it is assumed that the reproduction order determination unit 43 obtains the above-mentioned “degree of connection of dialogue sentences” based on the selected dialogue sentence.

  When the reproduction order determining unit 43 finishes executing the process for determining the reproduction order of still images and the process for selecting a dialogue sentence suitable for each still image by the above-described method, the reproduction order determining unit 43 associates the information with the dialogue sentence, Store in a storage device.

(Playback unit 44)
The playback unit 44 sequentially outputs a plurality of still images from the output device 3 in accordance with the playback order determined by the playback order determination unit 43, and causes the output device 3 to output a dialog sentence associated with each still image. At this time, if the dialogue sentence is subtitle format data, the dialogue sentence subtitle data may be output while the still image corresponding to each dialogue sentence is being outputted. Further, when the dialogue sentence is data in a voice format, the voice data of the dialogue sentence may be output while the still image corresponding to each dialogue sentence is being outputted.

  Next, a procedure for automatically generating a dialogue sentence moving image in the computer 4 of the present embodiment will be described with reference to FIG. FIG. 5 is a diagram showing an operation flow of the computer 4 when automatically generating a dialogue sentence moving image.

  The computer 4 inputs a plurality of still images from the image input device 2 (step S101). For example, when the image input device 2 is a digital camera, a plurality of still images captured by the digital camera are input. Note that the image input device 2 may store a plurality of still images previously captured by a digital camera in a storage device such as a hard disk and pick up the plurality of still images from the storage device. In that case, the user may arbitrarily select a plurality of still images from the storage device.

  When a plurality of still images are input by the image input device 2, the computer 4 calculates the degree of image similarity between the still image and another still image for each input still image (step S102). Here, as described above, the image similarity calculation unit 40 of the computer 4 calculates a difference such as a pixel value between each still image and another still image, and uses the difference as the image similarity. .

  The computer 4 extracts keywords from each of the plurality of still images input by the image input device 2 (step S103). Here, the keyword extraction unit 41 of the computer 4 specifies a characteristic object included in each still image and extracts a keyword for identifying the object. Specifically, as described above, the keyword extracting unit 41 extracts a keyword for identifying an object included in each still image by using a discriminator created by machine learning.

  When the keyword extraction process is completed for all the still images input by the image input device 2, the computer 4 generates a dialogue sentence corresponding to each still image based on the extracted keywords (step S104). Here, the dialog sentence generation unit 42 of the computer 4 generates a dialog sentence corresponding to each still image based on the keyword extracted by the keyword extraction unit 41. Specifically, the dialog sentence generation unit 42 generates a dialog sentence corresponding to the keyword by using a discriminator created by machine learning as described above.

  When the dialog sentence generation processing is completed for all the still images input by the image input device 2, the computer 4 determines the first weight coefficient w1 and the second weight coefficient w2 (step S105). Here, the reproduction order determination unit 43 of the computer 4 determines the first weighting coefficient w1 and the second weighting coefficient w2 based on the time length of the generated moving image. For example, when the time length of the generated moving image is long, the playback order determination unit 43 has a weight of the second weighting factor w2 with respect to the first weighting factor w1 as compared to the case where the time length of the generated moving image is short. The coefficients w1 and w2 are determined so as to increase.

The computer 4 executes the process of step S106 in parallel with the process of step S105. The process of step S106 is a process executed when a plurality of dialogue sentence candidates are generated for one still image, and a dialogue sentence suitable for the still image is selected from the plurality of dialogue sentence candidates. The process to select. This process is performed by the reproduction order determination unit 43 of the computer 4 as described above. Specifically, the reproduction order determination unit 43 first extracts feature words included in each dialogue sentence candidate, and performs ranking based on the appearance positions of these feature words. Next, the reproduction order determination unit 43 derives the reference rank of each feature word from the table as shown in FIG. 4, and calculates the sum of the differences between the rank of the feature word and the reference rank in each dialogue sentence candidate. The score of each dialogue sentence candidate is obtained. Then, the reproduction order determination unit 43 selects the dialogue sentence candidate having the smallest score among the plurality of dialogue sentence candidates as the dialogue sentence suitable for the still image.

  When the computer 4 finishes executing the processes of steps S105 to S106, it proceeds to the process of step S107. In the process of step S107, the computer 4 calculates the cost value between the still images using the first weighting coefficient w1 and the second weighting coefficient w2 determined in step S104. This calculation process is performed by the reproduction order determination unit 43 of the computer 4 as described above. Specifically, the reproduction order determination unit 43 first calculates the degree of connection of dialogue sentences between still images. As described above, this process is a process of calculating a vector difference between still images by expressing a keyword or the like included in the dialogue sentence of each still image as a vector. Next, the playback order determination unit 43 calculates a cost value based on the above-described equation (1) for each still image and another still image combination.

  When the cost value calculation processing is completed for all combinations of the two still images in the plurality of still images input by the image input device 2, the computer 4 reproduces the plurality of still images based on the cost values. The order is determined (step S108). This process is performed by the reproduction order determination unit 43 of the computer 4 as described above. For example, when determining a still image to be reproduced next to a certain still image A ′, first, a combination having the smallest cost value is identified from among combinations of the still image A ′ and other still images. Then, the still image on the other side in the specified combination may be determined as a still image to be reproduced next to the still image A ′. Note that the still image that is reproduced at the beginning of the generated moving image may be selected by the user, or may be selected by the computer 4 according to a predetermined algorithm. As a method of selecting the still image to be reproduced first by the computer 4, a method of selecting the still image with the oldest or newest shooting date from among a plurality of still images may be used.

  When the computer 4 finishes executing the process of step S108, the process proceeds to the process of step S109. In the process of step S109, the computer 4 outputs a plurality of still images and a dialog sentence associated with each still image from the output device 3 according to the reproduction order determined in the process of step S108, so that a dialog sentence moving image Play back.

  According to the procedure shown in FIG. 5, when a moving image is generated from a plurality of still images, a dialogue sentence related to an object included in each still image is automatically generated. Then, the reproduction order of the still images is determined based on the image similarity degree and the connection degree of the dialogue sentence, and the still picture and the dialogue sentence corresponding to the still image are output according to the reproduction order. The moving image generated in this way has a high quality and rich variety as compared to a moving image generated considering only the continuity of images and a moving image to which a simple description of the image is added.

  Further, according to the procedure shown in FIG. 5, when the time length of the generated moving image is long, the weight of the connection degree of the dialogue sentence with respect to the image similarity degree is increased compared to the case where the time length is short. Therefore, when the time length of the generated video is short, the video is generated with an emphasis on the degree of image similarity compared to the degree of dialogue sentence connection. Can be generated. On the other hand, when the time length to be generated is long, a moving image is generated with an emphasis on the degree of connection of dialogue sentences compared to the degree of image similarity, and thus a moving image with high storyliness can be generated. Therefore, the quality of the moving image can be further improved.

DESCRIPTION OF SYMBOLS 1 Automatic generator 2 Image input device 3 Output device 4 Computer 40 Image similarity calculation part 41 Keyword extraction part 42 Dialogue sentence generation part 43 Playback order determination part 44 Playback part

Claims (7)

Image input means for inputting a plurality of still images;
Image similarity calculation means for calculating the image similarity between two still images in a plurality of still images input by the image input means;
Keyword extraction means for extracting a keyword for identifying an object included in each still image for each of a plurality of still images input by the image input means;
Dialog sentence generation means for generating a dialog sentence related to the keyword extracted by the keyword extraction means for each of a plurality of still images input by the image input means;
Replay order determining means for determining the replay order of the plurality of still images based on the image similarity calculated by the image similarity calculating means and the connection degree of the dialog sentences generated by the dialog sentence generating means; ,
Reproducing means for outputting the plurality of still images and subtitles or voices of dialogue sentences corresponding to the still images in accordance with the reproduction order determined by the reproduction order determining means;
An apparatus for automatically generating a dialogue sentence moving image. The reproduction order determining means represents the keyword extracted by the keyword extracting means as a vector, and calculates a keyword vector difference between the two still images, thereby connecting the conversation sentences between the two still images. Seeking the degree,
The apparatus for automatically generating a dialogue sentence moving image according to claim 1.   The reproduction order determination unit is configured to add a second value to a value obtained by multiplying the image similarity calculated by the image similarity calculation unit by a first weighting factor and a connection degree of the dialog sentence generated by the dialog sentence generation unit. The interactive sentence moving image according to claim 1 or 2, wherein a cost value is calculated by adding a value multiplied by a weighting factor, and a reproduction order of the plurality of still images is determined based on the calculated cost value. Automatic generator.   The reproduction order determination means includes the first weighting factor and the second weighting factor so that the weight of the connection degree of the dialogue sentence to the image similarity degree becomes larger when the time length of the generated moving image is longer than when the time length is shorter. The apparatus for automatically generating an interactive sentence moving image according to claim 3, wherein the weighting coefficient is determined.   The reproduction order determining means determines the first weighting factor and the second weighting factor so that the weight of the connection degree of the dialogue sentence to the degree of image similarity changes in the middle of the generated moving image. Item 4. An apparatus for automatically generating a dialogue sentence moving image according to Item 3. The dialogue sentence generation means generates a plurality of dialogue sentence candidates related to the keyword extracted by the keyword extraction means for one still image,
The reproduction order determining unit selects one dialogue sentence suitable for the still image among the plurality of dialogue sentence candidates based on the appearance position of the word in each of the plurality of dialogue sentence candidates generated by the dialogue sentence generating unit. The apparatus for automatically generating an interactive sentence moving image according to any one of claims 1 to 5, wherein one is selected. Inputting a plurality of still images;
Calculating an image similarity between two still images in a plurality of input still images;
Extracting a keyword for identifying an object included in each still image for each of the plurality of input still images;
Generating a dialogue sentence related to the extracted keyword for each of the plurality of input still images;
Determining a playback order of the plurality of still images based on the generated connection degree of the dialogue sentence and the image similarity degree;
Outputting the plurality of still images and subtitles or voices of dialogues corresponding to the still images according to the determined reproduction order;
A method for automatically generating a dialogue sentence video including

Images