JP2011003991A - Information processor, operating method of the same, program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem that an output time of the whole slide show becomes too long when a reproduction time of a voice is extremely long compared with a display time of an image, in a conventional method where the display time is changed for each of a plurality of images according to an output time of a voice associated with each of the plurality of images.SOLUTION: This information processor includes a display means, an adjustment means, and an output control means. The display means displays each of a plurality of images for a fixed time according to a predetermined order, and can display a plurality of images simultaneously. The adjustment means adjusts a start time and a finish time of an output of a sound associated with each image within a range which does not exceed a time when each image is displayed not to output the sound associated with each of the plurality of images simultaneously. The output control means outputs the sound based on the result of adjustment by the adjustment means.

Description

  The present invention relates to a technique for digest reproduction of a plurality of images and sound associated with each image.

  There is a so-called slide show as a function for sequentially presenting still images one by one. In the past, it was used to give explanations by lecturers, and was used to explain the contents of pictures and diagrams manually.

  On the other hand, with the widespread use of digital cameras, software that presents music with background music such as BGM has appeared. Also, some digital cameras have a voice memo function, which allows you to make notes with a voice. When presenting a photo having audio data in a slide show in which the photo is switched every certain time, there is a problem that the audio is cut off in the middle if the audio data is longer than the display time of the photo. And in order to prevent that a sound cuts out on the way, when audio data is longer than display time, the technique of extending display time is proposed (for example, patent documents 1).

JP-A-10-145730

  In the method of changing the display time according to the length of the audio data as in Patent Document 1, the display time may be different for each slide, and the audio reproduction time is very long with respect to the image display time The playback time of the entire slide show becomes longer.

  Accordingly, an object of the present invention is to provide a technique that enables a viewer to grasp an outline of image data and audio data without requiring a very long reproduction time.

  In order to solve the above problems, an information processing apparatus of the present invention is an information processing apparatus capable of displaying each of a plurality of images on a display unit for a predetermined time according to a predetermined order and simultaneously displaying two or more images. In order to prevent the sounds associated with each of the plurality of images from being output at the same time, the start and end times of the sound associated with each image are set within a range not exceeding the time for which each image is displayed. Adjustment means for adjusting, and output control means for outputting sound based on the result of adjustment by the adjustment means.

  According to the present invention, it is possible for a viewer to grasp an outline of image data and audio data without requiring a very long reproduction time.

It is a figure which shows information processing apparatus. It is a flowchart which shows the process of initial arrangement | positioning. It is a flowchart which shows the process of position adjustment. It is a figure which shows an example of the image | video of a slide show. It is a figure which shows the symbol about the process of arrangement | positioning and a movement. It is explanatory drawing about the method of initial arrangement | positioning, and the judgment criterion of a movement. It is a figure which shows an example of a time chart. It is a figure which shows the example of reproduction | regeneration of a slide show. It is a figure which shows the example of reproduction | regeneration of a slide show. It is explanatory drawing about the method of initial arrangement | positioning, and the judgment criterion of a movement. It is a figure which shows an example of the time chart in the case of performing audio | voice area detection. It is a figure which shows an example of the cutting | disconnection method in the case of performing audio | voice area detection. It is a figure which shows the symbol about arrangement | positioning and a movement. It is explanatory drawing about the method of initial arrangement | positioning, and the judgment criterion of a movement. It is a flowchart of movement amount calculation. It is a figure which shows an example of a time chart. It is a figure which shows the reproduction frame setting of a moving image. It is a figure which shows the example of a reproduction frame setting of the moving image in the case of performing climax detection. It is a figure which shows the symbol about arrangement | positioning and a movement when not providing attention time. It is explanatory drawing about the method of initial arrangement | positioning, and the judgment criterion of a movement. It is a figure which shows the example of time chart preparation in case each video display time differs. It is a figure which shows the example of a time chart preparation in the case of changing reproduction time during video reproduction.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

Example 1
FIG. 1A is a functional block diagram of the information processing apparatus according to the first embodiment.

  The information processing apparatus includes an image storage unit 101, an image selection unit 102, a reproduction time adjustment unit 103, an audio editing unit 104, an image reproduction unit 105, an audio output unit 106, and an image display unit 107.

  Note that, for example, a digital camera is assumed to include the above-described units in one device.

  When the information processing apparatus is a digital camera, the information processing apparatus also includes an imaging unit 108, and has a function of displaying an image acquired by imaging by the imaging unit 108 on the image display unit 107.

  Further, an output interface 109 may be provided, and at this time, a function of displaying the image on the external display 110 via the output interface 109 is provided.

  Hereinafter, each element with which the information processing apparatus of a present Example is provided is demonstrated.

  The image storage unit 101 stores images such as still images and moving images. The image selection unit 102 selects an image to be displayed on the image display unit 107. The reproduction time adjustment unit 103 adjusts the timing for reproducing an image.

  The sound editing unit 104 edits sound data associated with an image as necessary. The image reproduction unit 105 reproduces (outputs) the image according to the timing adjusted by the reproduction time adjustment unit 103.

  The audio output unit 106 outputs audio associated with the image. The image display unit 107 displays the image output from the image reproduction unit 105.

FIG. 1B is a hardware configuration diagram of the information processing apparatus according to this embodiment.
In the figure, the information processing apparatus includes a central processing unit (CPU) 2801, a random access memory (RAM) 2802, and a read only memory (ROM) 2803.
The information processing apparatus includes a hard disk drive (HDD) 2804, a display 2805, a button 2806, and a speaker 2807.

  Hereinafter, each component with which the information processing apparatus of a present Example is provided is demonstrated.

The CPU 2801 controls the entire apparatus including the functions of the information processing apparatus. The RAM 2802 functions as a main storage device. The ROM 2803 stores programs and fixed data. The HDD 2804 stores programs and images.
Note that the program for realizing the function of the video playback device may be stored in the ROM 2803 or the HDD 2804.

  The display 2805 implements the image display unit 107. A button 2806 is used for instructing operations such as start / end / interruption / continuation of playback to the image playback unit 105. The speaker 2807 implements the audio output unit 106.

  FIG. 4A is an example of a slide show image reproduced by the information processing apparatus according to the present embodiment. In this example, all the images are still images, and images 2, 3, 4, 6, 8, 9 10 is associated with audio data.

Further, the images 1 to 10 are not displayed one by one, but are displayed for a fixed time while shifting the display start and end times, and a plurality of images are simultaneously displayed.
The voice data refers to audio data including a voice memo recorded at the time of image shooting, ambient environmental sounds, narration, sound effects, and music arbitrarily given after shooting.

  FIG. 8 is a diagram illustrating an example of an image displayed as a slide show on the image display unit 107. The image displayed on the image display unit 107 transitions in the order of 801 to 820 as time passes. Here, each image is displayed while moving from right to left.

  2 and 3 are flowcharts showing the operation for setting the timing for reproducing the audio data associated with the image.

  FIG. 2 is a flowchart for performing initial setting by calculating the display timing of each image.

  FIG. 3 is a flowchart for checking the reproduction timing of audio data associated with an initially set image and adjusting the reproduction timing so that the reproduction timing does not overlap.

  First, processing for setting the display timing of each image will be described with reference to the flowchart of FIG.

  In step S201, the image selection unit 102 selects images to be used for the slide show from the image storage unit 101 according to a predetermined order.

  Next, in step S202, the image selection unit 102 determines whether or not the selection of the image to be reproduced has been completed.

On the other hand, when an image to be reproduced is still selected and there is an image, in step S203, the reproduction time adjustment unit 103 sets the timing for displaying the image selected in step S201.
Note that the images displayed in order have a start time and an end time shifted so that the display times partially overlap, and the display timing set in step S203 corresponds to the initial setting.

  Next, in step S204, when the reproduction time adjustment unit 103 determines whether the selected image is a moving image and determines that the selected image is not a moving image, in step S205, the reproduction time adjustment unit 103 corresponds to the image. It is determined whether there is attached audio data.

  In step S205, when the reproduction time adjustment unit 103 determines that there is no audio data associated with the selected image, the process returns to step S201 to perform processing for the next image.

  On the other hand, when the reproduction time adjustment unit 103 determines in step S205 that there is audio data associated with the selected image, the process proceeds to step S206, and the reproduction time adjustment unit 103 sets the audio reproduction time.

  Here, with reference to FIGS. 5 (a) and 5 (b), symbols relating to the setting and changing method of the audio playback time will be described.

  FIG. 5A illustrates the variable symbols, and FIG. 5B illustrates the relationship between the image display time and the audio reproduction time.

  In this example, an attention display section is provided in which display is performed so that one of the images to be noticed can be recognized among the images displayed at the same time.

  The light gray band 501 indicates the display time of the mth image (from the time point PS (m) to the time point PE (m)), and the dark gray band 502 indicates the time of attention display of the mth image. (From time PAS (m) to time PAE (m)).

Further, a wavy line 503 indicates a sound reproduction time (from time VS (m) to time VE (m)) associated with the m-th image.
The time point LVE (m) is the end position of the sound reproduced immediately before the sound associated with the mth image, and the reproduction timing of the sound data associated with the mth image can be moved forward. Used to determine whether or not

Next, an initial arrangement method is shown in FIG. The following three types of examples are given as the initial arrangement method, but other methods may be used.
In FIG. 6, (m) representing the mth image assigned to PS or the like is omitted.

  (1) The center position of the audio data is matched with the center position of the attention time. (2) The end position of the audio data is matched with the end position of the attention time. (3) The start position of the audio data is matched with the start position of the attention time.

  Next, in step S206, the playback time adjustment unit 103 performs the initial placement process, and then proceeds to step S207. The playback time adjustment unit 103 displays the playback time of the audio data at the time when the audio data is aligned. It is judged whether the boundary before and after the time to display is exceeded.

Next, when it is determined in step S207 that the boundary is exceeded, in step S208, the audio editing unit 104 performs a process that does not reproduce the audio data of the portion exceeding the boundary, and the process returns to step S201.
The processing that does not reproduce the audio data in the portion beyond the boundary includes processing that specifies a part of the target audio data, processing that creates audio data corresponding to a part of the target audio data, and the like. .

On the other hand, if it is not determined in step S207 that the boundary is not exceeded, the process returns to step S201 without executing the process of step S208.
Note that the processing in step S207 and step S208 can be omitted. When step S207 and step S208 are omitted, adjustment is made so as not to exceed the image display time in steps S307, S316, S325, S328, and S333 in the flowchart of FIG.

  If it is determined in step S204 that the image is a moving image, the reproduction time adjustment unit 103 sets a moving image reproduction frame in step S209.

  In step S210, the reproduction time adjustment unit 103 determines whether there is audio data associated with the selected image. If there is no audio data, the process returns to step S201.

On the other hand, if it is determined in step S210 that there is audio data, the reproduction time adjustment unit 103 performs audio reproduction frame setting in step S2111 and returns to step S201.
Details of the processes in steps S209 and S210 will be described later.

  FIG. 7A is a time chart in the case where the display timing of the images to be subjected to the slide show of FIG. 4A and the reproduction timing of the audio data are set according to the flowchart of FIG.

  The display time of each image is t seconds, and the number of images that can be displayed simultaneously is N (5 in the present embodiment). It is arranged so as not to overlap with the attention time of the image.

In addition, the reproduction time of the sound associated with the image is also initially arranged in accordance with the image display time.
In this embodiment, as the method of initial arrangement of audio, (1) “Adjust the center position of audio data to the center position of time of interest” is adopted.

  However, overlapping portions are seen in the audio playback time, and if they are played as they are, the audio overlaps and it becomes difficult to hear.

  Therefore, according to the flowchart of FIG.

  In step S <b> 301, the video selection unit 102 selects an image to be reproduced from the video recording unit 101.

  Next, in step S302, the reproduction time adjustment unit 103 determines whether an image is selected in step S301. If no image is selected, the series of processing ends.

  On the other hand, if an image is selected, in step S303, the playback time adjustment unit 103 determines whether there is a sound (target sound) associated with the selected image. Hereinafter, the sound associated with the selected image is referred to as a target sound.

  Next, when it is determined in step S303 that the target voice does not exist, the process of step S308 is executed.

  On the other hand, when it is determined in step S303 that the target sound exists, in step S304, the reproduction time adjustment unit 103 determines whether the target sound overlaps the display start position of the image.

  If they overlap in step S304, the reproduction start timing of the target sound is forcibly moved backward to a timing that does not overlap with the image display start time in step S305.

  In step S306, the playback time adjustment unit 103 determines whether or not the end position of the target sound overlaps the display end position, and if so, executes the process of step S307.

In step S307, the voice editing unit 104 cuts behind the target voice at the display end position, and executes the process of step S308.
Note that the disconnection means designating a part of the audio reproduction time and generating another audio data corresponding to a part of the audio reproduction time. Alternatively, the reproduction start time and the end time of the audio data are associated with each other, and reproduction from the middle or halfway is performed.

If it is determined in step S306 that the image does not overlap, the process of step S308 is executed. If it is determined in step S304 that the image does not overlap with the image display start position, the process of step S308 is also executed.
Note that the processing from step S303 to step S307 can be omitted when the processing of step S207 and step S208 in FIG. 2 is performed.

In step S308, the previous position is set.
Note that the immediately preceding position is the next image display start position of the image selected in step S302 if it is determined in step S303 that no sound is present, and otherwise the image selected in step S302 is changed to the image selected in step S302. This is the end position of the associated voice. In addition, the sound data when there is sound becomes the immediately preceding sound for the subsequent processing.

  That is, the processes from step S301 to step S308 are processes related to the first image, and the processes after step S309 described later are processes related to the second and subsequent images.

  In step S <b> 309, the image selection unit 102 sequentially selects the second and subsequent images displayed from the image recording unit 101.

  In step S310, the playback time adjustment unit 103 determines whether or not the video to be played has ended. When the video has ended, the process of step S329 is executed.

  If an image exists, the process advances to step S311 to determine whether there is a sound (target sound) associated with the image selected by the reproduction time adjustment unit 103.

  If the playback time adjustment unit 103 determines in step S311 that there is no sound, the process proceeds to step S326.

  On the other hand, if the playback time adjustment unit 103 determines in step S311 that there is sound, it determines whether or not the target sound overlaps the previous position in step S312, and if it does not overlap, the process in step S317 is performed. Execute.

If it is determined in step S312 that the position overlaps with the immediately preceding position, in step S313, the reproduction time adjustment unit 103 determines whether or not there is an immediately preceding sound.
The immediately preceding sound is set in the immediately preceding position update process, and corresponds to the latest sound data associated with the image displayed until the image is displayed.

  In step S313, it is determined that there is no immediately preceding sound only when there is no sound data associated with the image displayed until the image is displayed. In this case, in step S314, the reproduction time adjustment unit 103 forcibly moves the target audio data backward to a position that does not overlap the previous position.

  Subsequently, the process of step S315 is executed.

  If the playback time adjustment unit 103 determines in step S313 that there is a previous sound, in step S318, the playback time adjustment unit 103 determines whether the previous sound moves forward.

  An example of a criterion for determining whether or not the immediately preceding voice can be moved forward is shown in FIG.

  The following four types of examples are given as criteria for determining whether or not it is possible to move forward, but other criteria may be used.

  (1) The audio end position VE is after the video attention start position PAS. (2) The audio end position VE is after the target center position PAM of the video. (3) The audio start position VS is after the video attention start position PAS. (4) The audio end position VE is after the video end position PAE.

  Under this criterion, the amount that can be moved is smaller in (4) than in (1).

  In addition, attention must be paid to the combination of the description of the initial arrangement and the movable standard.

  When (2) “match the end position of the audio data to the end position of the time of interest” is set in the initial arrangement, when (4) is selected as the forward movement reference, it is impossible to move forward.

  When (3) is set to “start audio data start position at the start position of the time of interest” in the initial arrangement, if (3) is selected as the forward movable reference, it is impossible to move forward. The item of the movable amount in FIG. 6B is a calculation formula for indicating the movable amount in each movement determination criterion.

  Note that (A) shows the result of calculating the amount by which the target voice can move on the basis of movement, and (B) shows the result of calculating the amount of the section where the immediately preceding voice and the target voice overlap. When (A)> (B), only (B) can be moved, and when (A) ≦ (B), only (A) can be moved.

  FIG. 7B shows an example in which the audio data associated with image number 2 that is the immediately preceding sound is moved forward when processing is performed on the audio data associated with image number 3. It is the figure shown typically.

  As a criterion for determining whether or not it is possible to move, (2) “the end position of the audio is behind the center position of interest of the video” is adopted.

  In this example, it is determined in step S318 that the audio data associated with the image number 2 is movable, and the process of step S319 is executed.

  In step S319, the reproduction time adjustment unit 103 moves the immediately preceding sound forward.

  The movement is performed by the movable amount.

  FIG. 7C is a diagram schematically showing the result of moving the audio data associated with the image number 2.

  Next, in step S320, the reproduction time adjustment unit 103 determines whether or not the target sound overlaps with the position immediately before the movement of the immediately preceding sound (in this case, the sound immediately before the movement).

  If it is determined in step S320 that they do not overlap, the process of step S315 is executed.

  On the other hand, if it is determined in step S320 that they overlap, in step S321, the playback time adjustment unit 103 determines whether the target audio data can be moved backward.

  Even when it is determined in step S318 that the immediately preceding voice cannot be moved forward, the process of step S321 is executed to determine whether or not the target voice data can be moved backward.

  FIG. 6C is a diagram schematically illustrating an example of a criterion for determining whether or not the reproduction time of audio data associated with an image can be moved to the rear of the reproduction time of the image.

  The following four types of examples are given as criteria for determining whether or not the vehicle can move backward, but other criteria may be used.

  (1) The audio start position VS is before the image attention end position PAE. (2) The voice start position VS is before the target center position PAM of the image. (3) The audio end position VE is before the video end position PAE. (4) The audio start position VS is before the video start position PAS.

  Under this criterion, the amount that can be moved is smaller in (4) than in (1). In addition, attention must be paid to the combination of the description of the initial arrangement and the movable standard. When (2) “Match audio data end position to end position of attention time” is selected in the initial arrangement, when (3) is selected as the backward movement reference, it is impossible to move backward.

  When (3) “match the start position of the audio data with the start position of the time of interest” is set in the initial arrangement, if (4) is selected as the backward movement reference, it is impossible to move backward.

  The movable amount is calculated within a range where the audio data does not exceed the display end position.

If it is determined in step S321 that the audio data can be moved backward, in step S322, the reproduction time adjustment unit 103 moves the target audio data backward.
The item “movable amount” in FIG. 6D is a calculation formula for indicating the movable amount in each movement determination criterion. The movement possible amount in the case of moving to the front of FIG. 6C is restricted by the end position of the immediately preceding voice, but the movement routine amount in the case of moving to the back of FIG. 6D is the end position of the immediately preceding sound. Not subject to restrictions. However, if it has moved as much as possible, there is a greater possibility that it will be necessary to move it forward again in subsequent processing. Therefore, the actual amount of movement is (A) the possible amount of movement, (B) the amount of the section where the immediately preceding voice and the target voice overlap, and if (A)> (B), the movement amount is (B), ) ≦ (B), move only (A).

  Next, in step S323, the playback time adjustment unit 103 determines whether or not the previous position overlaps with the target voice even after the target voice has moved, and if not, executes the process of step S315.

  On the other hand, if it is determined in step S323 that the target voice overlaps with the previous position even after the movement of the target voice, the process of step S324 is executed. Execute the process.

  FIG. 7D is a diagram schematically illustrating an example in which the immediately preceding voice is not movable forward and the target voice data is not movable backward.

  Here, the image number 4 is the target audio data.

  The sound data associated with the image number 3 that is the sound immediately before the image number 4 cannot be moved forward because it is immediately after the sound data associated with the image number 2.

  The criterion for determining whether or not it is possible to move backward is (2) “The start position of the sound is before the focus center position of the video”, but the end position of the sound data of image number 4 is the same as the display time. So I can't move back.

  In step S324, the reproduction time adjustment unit 103 determines whether it is possible to cut the back of the immediately preceding voice.

  FIG. 7E is a diagram schematically showing an example in which the portion behind the immediately preceding voice is not cuttable.

  Since the start position of the target audio data is before the start position of the immediately preceding sound, the immediately preceding sound cannot be disconnected.

  If it is intentionally disconnected, it is necessary to erase the previous voice and to cut to the previous voice.

  In this case, as a criterion for determining whether or not disconnection is possible, the immediately preceding voice is not lost.

  In addition to this, the end position of the immediately preceding voice may be after the attention start point, the end position of the immediately preceding sound may be after the attention center position, or the like may be used.

  If it is determined in step S324 that cutting is impossible, the process proceeds to step S314, and the target audio data is forcibly moved backward to a position where it does not overlap. An example is shown in FIG.

  In step 315, it is determined whether or not the end position of the target audio data overlaps the display end position of the image. If they overlap, the process of step S316 is executed.

  Next, in step S316, the voice editing unit 104 cuts behind the target voice data at the image display end position, and executes the process of step S317.

  FIG. 7G schematically shows an example before cutting, and FIG. 7H schematically shows an example after cutting.

If it is determined in step S324 that the back of the previous voice can be disconnected, in step S325, the playback time adjustment unit 103 cuts the back of the previous voice and executes the process of step S315.
FIG. 7 (i) shows an example when it is determined in step S321 that the target audio data can be moved backward.

  Since the audio data associated with the image number 6 that is the object of processing overlaps with the audio data associated with the image number 4, it moves within a movable range.

  An example after the movement is shown in FIG.

  This is because it is determined in step S323 that the target voice overlaps with the previous position after the movement of the target voice, and thus the process proceeds to step S324 and it is determined whether the back of the previous voice can be cut.

  In the example of FIG. 7 (j), it is determined that the back of the previous voice can be cut, and in step S325, the reproduction time adjustment unit 103 cuts the back of the previous voice.

  FIG. 7 (j) shows an example before cutting, and FIG. 7 (k) shows an example after cutting.

  In step S317, the reproduction time adjustment unit 103 sets the end position of the audio data associated with the current video as the immediately preceding position, and then repeats the process of step S309.

  In step S326, it is determined whether there is a previous voice. If there is no immediately preceding voice, the process of S309 is repeated. If there is an immediately preceding voice, the process proceeds to S327.

In step S327, it is determined whether or not the immediately preceding sound exceeds the display end position of the corresponding image. If it does not exceed, the process of step S309 is repeated. If it exceeds, the back end is cut so that the end position of the immediately preceding voice does not exceed the display end position, and the process of step S309 is repeated.
Note that the processing of steps S326 to S328 is necessary only when the processing of steps S207 and S208 is omitted in the flowchart of FIG. When the processes of steps S207 and S208 are performed in the flowchart of FIG. 2, the processes of steps S326 to S328 are not necessary.

  If it is determined in step S310 that the video to be played has ended, the process of step S329 is executed as the end process.

  In step S329, the reproduction time adjustment unit 103 determines whether or not the previous position overlaps the end position. If the position does not overlap, the process ends.

  On the other hand, if it is determined in step S329 that the previous position overlaps the end position, in step S330, the reproduction time adjustment unit 103 determines whether or not the previous sound moves forward.

  In the case of movement, in step S331, the reproduction time adjustment unit 103 moves the immediately preceding sound forward to execute the process of step S332, and in the case of not moving, executes the process of step S333.

  In step S332, the reproduction time adjustment unit 103 determines whether or not the immediately preceding position overlaps with the end even after the movement of the immediately preceding sound. If it is determined that the immediately preceding position and the end position do not overlap, the series of processing ends. .

If it is determined in step S332 that the previous position overlaps with the end even after the movement, in step S333, the reproduction time adjustment unit 103 cuts after the previous sound and ends.
Note that the processing from step S329 to the end is not necessary when the immediately preceding voice has passed the processing of steps S315 and S328. Further, it is not necessary even when the steps S207 and S208 are not omitted in the flowchart of FIG.

  The result of applying the above processing is shown in FIG. In FIG. 7 (l), there are no overlapping portions of the audio data.

  As described above, the flowchart of FIG. 3 creates a time chart for performing sound output control.

  According to the time chart created in the flowchart of FIG. 3, the image reproduction unit 105 reproduces a slide show, and outputs audio and images from the audio output unit 106 and the image display unit 107 as shown in FIG.

  The reproduced video is displayed while moving a still image from right to left, and 801 to 819 represent a part of the movement.

  The still image moving from the right to the left gradually increases to the central portion and becomes the maximum at the central portion. The maximum display time of this central portion is set as the attention time.

  The speech balloons displayed in FIG. 8 are for showing which video the audio associated with is reproduced, and the actual display may not be performed.

  As shown in the time chart of FIG. 7L, no audio data is associated with the image of image number 1, and no audio is reproduced in 801 and 802.

  The audio data of the image number 2 is arranged so as to exceed the boundary before and after the attention time.

Audio reproduction is started at a time 803 when the image number 2 is slightly before the attention display, and the sound reproduction is continued until 804 during the attention time and 805 after the attention time. In FIG. 8, 801 to 803 may be a discontinuous movement, or the image may be a continuous movement between 801 to 803 that moves, expands and contracts.
The display form of the image is not limited to the example of FIG.

  Another example of the image display form is shown in FIG.

  The display position of the still image is randomly arranged, the display starts at a small size, the size is increased while the position of the still image is fixed with the passage of time, the size is reduced when the attention time has passed, and the end position of the display time To erase.

  As shown in the time chart of FIG. 7L, no audio data is associated with the image of image number 1, and no audio is reproduced in 901 and 902.

  The audio data associated with the image number 2 is arranged so as to cross the boundary before and after the attention time.

  Audio reproduction is started at the time 903 when the image number 2 is slightly before the attention display, and the audio reproduction is continued until 904 during the attention time and 905 after the attention time.

  In FIGS. 5A, 5B and 6B, 6C, attention time is provided during the display time due to limitations on the arrangement position and movement range, and at least part of the audio data overlaps the attention time. However, the arrangement may be such that the audio data is within the display time without providing the attention time.

  Symbol explanations about the arrangement and movement method of the audio reproduction time in this case are shown in FIGS.

  FIG. 5C illustrates the variable symbols, and FIG. 5D illustrates the positional relationship.

  FIG. 10A shows an initial arrangement method when no attention time is provided, FIG. 10B shows a criterion for determining whether or not audio data can be moved forward, and a criterion for determining whether or not audio data can be moved backward. As shown in FIG.

  Eliminating the attention time broadens the range of audio position adjustment, but makes it difficult to determine which video is being played back. Therefore, in order to show a video in which sound is reproduced, a balloon display or a still image frame highlight display may be performed.

  In the example of the slide show video shown in FIG. 4A, the length of the audio data is targeted, and the sound state of the audio data is not considered. The voice data may include a section where the volume is low and the sound cannot be heard and a useless section such as noise, and the length of the voice data can be reduced by excluding these useless sounds. A block diagram for this purpose is shown in FIG.

  The same elements as those in FIG. 1A are denoted by the same reference numerals and description thereof is omitted.

  The voice segment detection unit 1208 detects a voice segment.

  There are various methods for detecting a voice interval. Judging by volume and detecting sections with a volume above a certain level, detecting sections that detect human voice and detecting only sections with human voice, and sound types using multiple sound models And the section is detected. Here, the method is not limited.

  FIG. 4B shows an example of each video of the slide show reproduced by the video reproduction device and the length of the audio data detected in the audio section.

  When speech section detection is performed using the speech data 1301, a non-speech section 1302 and a speech section 1303 are detected.

  With reference to FIGS. 5E and 5F, symbols relating to the arrangement and method of the audio playback time will be described.

  FIG. 5E illustrates the variable symbols, and FIG. 5F illustrates the positional relationship. Reference numeral 1401 denotes audio data. The non-speech section is white, the speech section is indicated by a wavy line, and the start position (VS) of the first speech section and the end position (VE) of the last speech section in the speech data are the start of speech. And the end position. In the example of FIG. 4B, the start position of the voice section 1303 is VS and the end position is VE. When there are a plurality of voice sections in the voice data, the start position of the first voice section is VS and the end position of the last voice section is VE.

  The description of the method of initial arrangement and the criteria for determining the movement will be omitted because VS and VE are the same as in FIG.

  FIG. 11A shows a time chart of the initial arrangement when the slide show of FIG. 4B is arranged according to the flowchart of FIG. There are a plurality of places where voices overlap, but it is a part where voice sections overlap that must be adjusted so as not to overlap.

FIG. 11B shows an example in which the processing of the flowchart of FIG. 3 is applied to FIG.
In FIG. 11 (b), there is an overlap of the non-voice section between the non-voice section of the video number 3 and the video number 4, but if the non-voice section is a silent section, the sound will be heard even if both voices are played back simultaneously. There is no duplication. When the non-speech section includes noise or the like, processing such as suppression of reproduction of the non-speech section is performed.

  Further, when cutting a voice that is too long, the voice is cut in a non-voice section. An example of the cutting method is shown in FIG. When the cutting position is a voice section as shown in FIG. 12A, there are two types of cutting methods, FIG. 12B and FIG. 12C. FIG. 12B shows a case where the cut position is shifted to the immediately preceding non-voice section when the overlapping portion is a voice section. This prevents the sound from being cut off in the middle of a word. In FIG. 12C, when the overlapping portion is a voice section, only the voice sections are collected within the time before the cutting position, and the volume of the reproduced voice sections is increased to cut.

  Therefore, unlike slideshows that display images one by one, multiple images are displayed with their start and end times shifted, so that the display time for one image can be reduced without changing the overall duration of the slideshow. Can be long. By adjusting the positional relationship between the previous and next audio data within the video display time, the upper limit for playback of long audio data is used by using the time when there is video without audio data or when there is video with short audio data. The time can be lengthened.

  In addition, a plurality of images can be seen at the same time, and the reproduced sound is for one of these images, and the sound of the invisible image is not reproduced.

  In addition, since all the sounds of the plurality of displayed videos are reproduced with the start position and the end position adjusted, at least some of the sounds are reproduced for all the images.

(Modification 1 of Example 1)
In the first embodiment, the position is adjusted so that the audio reproduction times do not overlap, but a time width in which a part of the audio reproduction times overlap may be provided.

  In this time range, an acoustic effect may be applied in which the front volume is gradually reduced and turned off, and the rear volume is gradually increased.

(Modification 2 of Example 1)
In the first embodiment, the image data is arranged and adjusted sequentially from the beginning, but may be arranged and adjusted sequentially from the end.

(Modification 3 of Example 1)
In the first embodiment, the possibility of movement is determined only for the immediately preceding data. However, the possibility of movement may be recursively determined such that the immediately preceding data is further determined with respect to the immediately preceding data. .

(Modification 4 of Example 1)
In addition to speech balloons and video frame highlighting so that the video in which the audio described in the first embodiment is reproduced can be seen with eyes, the audio is output with a plurality of channels shifted in phase so that it can be seen with the ear. You may do it.

(Modification 5 of Example 1)
In the first embodiment, the non-speech section is omitted and the speech section to be played back is shortened. However, the playback time may be shortened using speech speed conversion for the speech section as well.

  In addition, the middle voice segment may be omitted and the start part and end part may be reproduced without fail.

(Modification 6 of Example 1)
In the first embodiment, when voice recognition is applied to a voice section for characterization and the playback time of the voice section is shortened, the voice recognition result character may be displayed during image display.

(Example 2)
The first embodiment is an example in which the video is a still image, but the second embodiment is an example in which at least a part includes a moving image. FIG. 4C shows an example of each video of the slide show reproduced by the video reproduction device. Video number 2 and video number 7 are moving images with sound, video number 2 is relatively short, and video number 7 is relatively long.

  When the video is a moving image, the playback time of the moving image is also an arrangement target of the time chart.

  In the case of a moving image with sound, it is necessary to take the reproduction timing of the sound and the moving image, and therefore the reproduced content of the sound cannot be moved independently of the reproduced content of the moving image. The moving image playback content and the audio playback content are moved simultaneously. If the content cannot be moved, the playback frame for starting and ending playback is moved.

  The block diagram of the second embodiment of the present invention is the same as the block diagram of FIG.

  Although the flowcharts are the same as those in FIGS. 2 and 3, a supplementary explanation will be given for the processing for moving images.

  Steps S209 to S211 in the flowchart of FIG. 2 are initial arrangement processing when the video is a moving image. In step S209, a moving image playback frame is set. FIG. 13 shows a symbol explanation for the arrangement and movement method of the reproduction time. FIG. 13A illustrates the variable symbols, and FIGS. 13B and 13C illustrate the positional relationship. FIG. 13B shows a case where the time of moving image data is shorter than the display time, and FIG. 13C shows a case where the time of moving image data is longer than the display time.

  Since the moving image data (IS, IE) may be longer than the display time (PS, PE), the time frame for reproducing the moving image in the display time is set as the moving image reproduction frame (MS, ME). ). In addition, audio playback frames (VS, VE) for playing back audio within the range specified by the moving image playback frame are provided. LVE is the end position of the immediately preceding voice and is not described in FIG. 5B, but is used to determine whether data can be moved forward.

  FIG. 14A and FIG. 14B show the initial arrangement method for setting the playback frame.

The following three types of moving image playback frame initial arrangement methods are listed, but other methods may be used.
1) Adjust the center position of the moving image to the center position of the attention time.
2) Match the end position of the moving image to the end position of the time of interest.
3) The start position of the moving image is matched with the start position of the attention time.

FIG. 14A shows an example in which the length of the moving image is shorter than the display time, and FIG. 14B shows an example in which the length of the moving image is longer than the display time. When placed at the initial arrangement position, the moving image playback frame is set so that the moving image playback is not performed for the portion exceeding the boundary of the display time.
Next, in step S210, it is determined whether there is audio data. If it is determined that there is audio data, an audio playback frame is set in step S211. As the initial maximum length of the audio reproduction frame, the following three types are assumed within the range of the moving image reproduction frame, but other initial lengths may be used.
1) Make the attention time length.
2) Match the playback time of the moving image.
3) Double the attention time.

  An example of setting the audio playback frame is also shown in FIGS. 14 (a) and 14 (b). Here, it is determined whether or not it is a moving image, and it is determined whether or not there is audio data. However, there is a possibility that audio that is not synchronized with the moving image is included. If the moving image itself has no sound to be synchronized and a voice memo or music is added to the silent moving image, the moving image is handled in the same way as the audio data added to the still image.

  After applying the processing of the flowchart of FIG. 2, in the example of the slide show video of FIG. 4C, the time chart of FIG. 16A is generated. Since the moving image of the video number 2 is relatively short, the moving image reproduction frame is the same as the size of the original moving image. Since the moving image of the video number 7 is relatively long, the moving image data is trimmed so that the moving image playback frame fits within the frame of the video display time. The playback frame for audio data in FIG. 16A is twice as long as the time of interest, and the audio playback frame for video number 2 is the same as the original size, but the audio playback frame for video number 7 is Trimmed to twice the attention time.

  In the process of the flowchart of FIG. 3, a process specific to a moving image is necessary in determining whether or not the immediately preceding sound in steps S318 and S327 moves forward and in determining whether or not it moves after step S321.

A flowchart of the movement amount calculation is shown in FIG. In step S2001, it is determined whether or not the image is a moving image. If it is determined that the image is not a moving image, the process proceeds to step S2006, where the movement amount calculation for the same audio reproduction time as described in the first embodiment is performed and the process ends.
If it is determined in step S2001 that the image is a moving image, the process advances to step S2002 to determine whether the moving image reproduction frame can be moved. Whether or not the moving image playback frame can be moved is determined by whether or not the moving image playback frame can move within the video display time. When the length of the moving image exceeds the boundary before and after the display time in the initial arrangement, the moving image playback frame is set in accordance with the display time, and movement beyond the boundary becomes impossible.

If it is determined in step S2002 that the moving image playback frame is movable, the process advances to step S2003 to calculate the moving image playback frame movement amount. FIGS. 14C and 14D show how to calculate the movement amount when the moving image playback frame is movable. FIG. 14C shows a case of moving forward, and FIG. 14D shows a case of moving backward. The following four types of examples are given as criteria for determining whether or not it is possible to move forward, but other criteria may be used.
1) The end position of the moving image playback frame is after the attention start position of the video.
2) The end position of the moving image playback frame is after the focus center position of the video.
3) The start position of the moving image playback frame is after the attention start position of the video.
4) The end position of the moving image playback frame is after the attention end position of the video.

  The movable amount is set within a range in which the audio data does not overlap with the audio immediately before the data and the moving image data does not protrude before the video display time. The movement amount is the smaller amount of the section overlapping the movable amount.

The following four types of examples are given as criteria for determining whether or not it is possible to move backward, but other criteria may be used.
1) The start position of the moving image playback frame is before the notice end position of the video.
2) The start position of the moving image playback frame is ahead of the center position of interest in the video.
3) The end position of the moving image playback frame is before the target end position of the video.
4) The start position of the moving image playback frame is before the notice start position of the video.

  The movable amount is set within a range in which the moving image data does not exceed the video display time boundary. The movement amount is the smaller amount of the section overlapping the movable amount.

  In the second embodiment, the moving image playback frame is limited within the video display time, and the moving image playback frame can be moved only when the moving image playback time is shorter than the video display time. The movement in this case means movement of data arrangement time. As shown in FIG. 16B, the moving image data and audio data of video number 2 can be moved forward.

  After calculating the moving amount of the moving image playback frame in step S2003, it is determined in step S2004 whether the moving amount is sufficient, and if it is sufficient, the process ends.

If it is determined in step S2004 that the amount of movement is not sufficient, or if it is determined in step S2002 that the moving image playback frame cannot be moved, the amount of movement of the audio playback frame is calculated in step S2005. To do. FIGS. 14E and 14F show a method for calculating the movable amount when the audio reproduction frame is movable. FIG. 14E shows the case of moving forward, and FIG. 14F shows the case of moving backward. The following four types of examples are given as criteria for determining whether or not it is possible to move forward, but other criteria may be used.
1) The end position of the audio playback frame is after the attention start position of the video.
2) The end position of the audio playback frame is after the focus center position of the video.
3) The start position of the audio playback frame is after the attention start position of the video.
4) The end position of the audio playback frame is after the target end position of the video.

  The movable amount is set within a range in which the audio reproduction frame does not overlap with the immediately preceding audio of the data and the audio reproduction frame does not exceed the moving image reproduction frame. The movement amount is the smaller amount of the section overlapping the movable amount.

The following four types of examples are given as criteria for determining whether or not it is possible to move backward, but other criteria may be used.
1) The start position of the audio playback frame is before the notice end position of the video.
2) The start position of the audio playback frame is before the focus center position of the video.
3) The end position of the audio playback frame is before the noticed end position of the video.
4) The start position of the audio playback frame is before the attention start position of the video.

  The movable amount is set so that the audio playback frame does not exceed the video display time boundary. The movement amount is the smaller amount of the section overlapping the movable amount.

  In the flowchart of FIG. 15, the type of movement and the movable amount are calculated. In steps S319, S328, and S322 of the flowchart of FIG. 3, movement based on the calculated result is performed.

  In steps S319 and S328, the immediately preceding voice is moved forward. FIG. 16C shows the result of moving the audio data of video number 2 in FIG. 16B forward.

  In step S322, the target audio data is moved backward. As shown in FIG. 16 (d), the audio reproduction frame of the video number 7 overlaps with the immediately preceding audio data, so the overlap must be adjusted, but the moving image data exceeds the boundary of the video display frame. In step S2002, the moving image playback frame cannot be moved. Therefore, in step S2005, the amount of movement of the audio reproduction frame is calculated, and the audio reproduction frame is moved based on the result. The state after the movement is shown in FIG. In FIG. 16 (e) after the movement, it is determined that the audio reproduction frame of the immediately preceding video number 7 is overlapped in the processing of the flowchart step S312 of FIG. From step S312, the process proceeds from step S312 to S313 and S318. Since video number 7 cannot be moved forward, the process proceeds to step S321. In step S321, it is determined whether video number 8 can be moved backward. In this case, since movement is possible, the audio data of the video number 8 is moved backward in step S322. In FIG. 16 (f) after the movement, since there is still an overlap with the audio reproduction frame of the immediately preceding video number 7, the process proceeds from step S323 to S324. In step S324, it is determined that cutting is possible, and the process advances to step S325 to cut the back of the audio reproduction frame of video number 7. The adjustment result is shown in FIG.

  With the time chart thus created, the moving image is played back when the video of the video number 7 starts to be displayed in the slide show, but the audio starts to be played after the attention time has entered. The sound reproduced at this time is synchronized with the moving image being reproduced. In addition, when the video of video number 2 is displayed in the slide show, the video is not displayed but the still image is displayed, and the video playback and the audio playback are started at the video playback time. The still image is displayed until the video display time ends after the reproduction of the moving image and the sound ends. When the time of moving image data is shorter than the video display time, it may be supplemented with a still image as described above, or may be reproduced so as to repeat the preceding and following moving images.

In the initial arrangement method shown in FIGS. 14A and 14B, the portion where the moving image is reproduced is determined by the overall positional relationship regardless of the content of the moving image. FIG. 22 shows a setting example of the moving image playback frame in the method of aligning the center position. In this example, since the video display time is shorter than the time of the moving image data, only the middle part is reproduced. If there is a scene that seems to be important at the end of the moving image, it may be missed.
In order to prevent this, a means for detecting the climax scene and arranging it so as to always display the location is provided. A block diagram of the information processing apparatus at this time is shown in FIG.

  Reference numeral 2308 denotes climax detection means for detecting a climax scene in the moving image.

  101 to 107 are the same as those in the block diagram (a) of FIG.

  There are various methods for climax detection. A flag that indicates the climax part is manually assigned in advance to detect the flag part, a part that automatically analyzes the voice to detect a part such as applause or cheer, and a part that automatically analyzes the video to detect movement. For example, it detects the part of the image. Here, the method is not limited.

  FIG. 18 shows an example in which a climax scene is detected and set. When the last part of the moving image is detected as a climax scene, the position is matched with the end position of the attention time, and the moving image reproduction frame and the audio reproduction frame are set before that. In this case, the moving image is reproduced from the middle of the actual data when the video display is started. The audio playback starts before the attention time starts, and the playback of the moving image and the sound ends when the attention time ends. For example, a still image at the end of the moving image may be displayed during the time from the end of the playback of the moving image to the end of the video display time.

  Note that voice segment detection may also be applied to audio data of moving images, and voice segments and non-speech segments may be separated for setting and cutting of a voice playback frame. In that case, the speech section detection unit 1208 in FIG. 1C is added to the block diagram in FIG.

  Further, the audio data may be arranged within the display time without providing the attention time. 19 and 20 show examples of the initial arrangement of the audio reproduction time and the determination criterion for movement for that purpose. FIG. 19 is a symbol explanation for the arrangement and movement method of the reproduction time. FIG. 19A illustrates the variable symbols, and FIGS. 19B and 19C illustrate the positional relationship. FIG. 19B shows a case where the time of moving image data is shorter than the display time, and FIG. 19C shows a case where the time of moving image data is longer than the display time. In the initial arrangement, FIG. 20A shows an example in which the time of moving image data is shorter than the video display time, and FIG. 20B shows an example in which the time is long. When the moving image data time is shorter than the video display time, FIG. 20C shows a criterion for determining whether or not it is possible to move forward, and FIG. 20D shows a criterion for determining whether or not it can be moved backward. When the time of moving image data is longer than the video display time, FIG. 20 (e) shows a criterion for determining whether or not it can be moved forward, and FIG. 20 (f) shows a criterion for determining whether or not it can be moved backward.

  In the above example, the video display time of each video is the same, but the video display time may be designated for each video. The designation method may be manually input, or the importance of the video may be determined by analyzing voice or image and reflected in the fixed time. The method is not limited. FIG. 21 shows an example of creating a time chart when the video display time is different for each video. FIG. 21A shows the video display time and attention time preset for each video. FIG. 21B shows the initial arrangement of the audio data in FIG. 4A in accordance with this setting. Although there are some overlapping parts of the audio, there are some parts where the interval of the audio data is free because there is a long video display time.

  In the above example, the initial arrangement and the overlap adjustment are performed in separate flowcharts, but one arrangement may be arranged in the same flowchart and the adjustment may be repeated.

  In the above example, the video display time is determined in advance before the start of video playback, but means for changing the display time during video playback may be provided. A block diagram in this case is shown in FIG. Reference numeral 2901 denotes a display time changing unit. As the time change specifying means, numerical values may be input using the 2801 button in FIG. 1E, or the speed may be changed using a slider bar or a touch panel instead of the buttons.

  When changing the speed during video playback, a time chart is not created for all the videos in advance, but a method of adjusting the overlap by arranging one video is used. In addition, the audio data is not cut by the audio editing unit 104, but the reproduction time is controlled by the reproduction time adjustment unit 103. During normal playback, a time chart of a certain number of videos is created and played back with a preset display time. When a display time change operation is performed from 2901, a time chart is re-created for the video currently being displayed and the video just before the display for which the time chart has already been created, and video playback is continued.

  FIG. 22 shows an example of a time chart for changing the display time during the creation. FIG. 22A is a time chart created for the six videos from the beginning with the default display time. The time chart is updated at a timing including the currently displayed image and one immediately following image. When there is a change 3001 that doubles the display time when the video of video number 3 is at the position of interest and the video of video number 5 starts to be displayed, a time chart is displayed as shown in FIG. Update. In this example, the change instruction is not reflected immediately, but the change of the display time is reflected at the timing when the current video deviates from the target position. FIG. 22B is a time chart when the display of the video number 6 is started, and includes the reproduction time up to the video number 7. However, the change in display speed may be reflected immediately after.

  FIG. 22C shows an example in which the video is continued with the display time doubled and a time chart up to video number 9 is created. When there is a change 3002 that reduces the display time to 1/4 when the video number 6 is at the position of interest and the video number 8 starts to be displayed, a time chart as shown in FIG. Update.

In addition, when the display time is changed, if the display time is shorter than a certain time, the content may not be understood even if audio is played. When the display time becomes shorter than a certain time, the audio reproduction may be disabled. Or you may invalidate the display time change made shorter than fixed time.
Note that, when the display time is changed during video playback, the condition that the audio that has already started playback moves forward or cannot be moved later is added to the determination of whether or not movement is possible.

  Therefore, the voice data can be shortened by detecting the non-voice section and removing the front and rear. Still, when there is long audio data that does not fit within the display time of one video, the sound is prevented from being cut off in the middle of a word by cutting it in a non-voice segment.

  Further, by providing a climax detection means and aligning the climax position within the moving image display time, it is possible to prevent an important part of a long moving image from being missed when viewed in a slide show.

  Further, by providing means for designating the video display time for each video in advance, the display time can be adjusted according to the content of the video.

  In addition, by providing means for changing the display time during display video playback, the display speed can be changed during playback, and the audio playback time can be adjusted.

(Modification of Example 2)
The default moving image playback frame of the second embodiment maximizes the display time, but a value smaller than the display time may be given in advance.

(Other examples)
It is to be noted that the object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or the like, and the system or the like reads and executes the program codes stored in the storage medium. Can also be achieved.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, a CD-ROM or the like can be used.

  Further, based on the instruction of the program code, an OS (operating system) running on the computer may perform part or all of the actual processing, and the functions of the above-described embodiments may be realized by the processing. included.

Claims (9)

An information processing apparatus capable of displaying each of a plurality of images on a display unit according to a predetermined order for a certain period of time and simultaneously displaying two or more images,
Adjustment that adjusts the start and end times of the output of the sound associated with each image so that the sound associated with each of the images is not output at the same time so that the time for displaying each image is not exceeded. Means,
An information processing apparatus comprising: output control means for outputting sound based on a result of adjustment by the adjustment means. An information processing apparatus that displays a plurality of images according to a preset timing,
The time point PE (m) at which the display of the m (m> 0) -th image ends is after the time point PS (m + 1) at which the display of the m + 1-th image starts, and the time point at which the display of the m-th image starts. A point in time at which image display is started such that the time from PS (m) to PE (m) is equal to the time from PS (m + 1) to the point PE (m + 1) at which the display of the (m + 1) -th image ends. A determining means for determining the end point;
Acquisition means for acquiring data representing sound associated with each image;
The time when the reproduction of the sound associated with each image starts after the time when the display of the image starts, and the time when the reproduction of the sound associated with the image ends. A setting means for setting a time point at which sound reproduction starts and a time point to end so as to be before the time point when the display ends;
When the time point VS (m + 1) at which the reproduction of the sound associated with the m + 1st image is started is before the time point VE (m) at which the reproduction of the sound associated with the mth image is finished, VS (m + 1) Becomes the time after VE (m), and the time point VE (m + 1) at which the reproduction of the sound associated with the (m + 1) th image ends, the time point VS (m + 2) at which the reproduction of the sound associated with the (m + 2) th image starts. ) An information processing apparatus comprising: a changing unit that changes a time point of starting and ending reproduction of a sound associated with the (m + 1) th image so as to be before.   In the case where the changing unit has a time point VS (m + 1) at which the reproduction of the sound associated with the m + 1st image is started before a time point VE (m) at which the reproduction of the sound associated with the mth image is ended. , VE (m) is before VS (m + 1), and the time VS (m) at which the reproduction of the sound associated with the mth image is started is after the time at which the display of the mth image is started. The information processing apparatus according to claim 2, wherein the time point at which the reproduction of the sound associated with the mth image starts and the time point at which the sound ends are changed. An information processing apparatus that displays a plurality of images according to a preset timing,
The time point PS (m) at which the display of the m (m> 2) th image starts is before the time point PE (m−1) at which the display of the m−1th image is finished, and the m−1th image. The time from the point PS (m−1) to the point PE (m−1) at which display of the image is started is equal to the time from the point PS (m) to the point PE (m) at which the display of the m-th image is ended. Determining means for determining the start time and end time of image display,
Acquisition means for acquiring data representing sound associated with each image;
The time when the reproduction of the sound associated with each image starts after the time when the display of the image starts, and the time when the reproduction of the sound associated with the image ends. Setting means for setting a time point to start and end sound reproduction so as to be before the display ends;
When the time point VE (m−1) at which the reproduction of the sound associated with the (m−1) th image is finished is after the time point VS (m) at which the reproduction of the sound associated with the mth image is started, VE (m−1) is before VS (m), and the time point VS (m−1) at which the reproduction of the sound associated with the m−1th image is started is associated with the m−2th image. Change means for changing the time point at which the reproduction of the sound associated with the (m-1) th image is started and the time point at which the reproduction is finished so that the reproduction is after the time point VE (m-2) at which the reproduction of the generated sound is completed. Information processing apparatus provided. The determination means further includes a time point after the time point PS (m + 1) when PE (m) starts displaying the (m + 1) th image, a time period from PS (m) to PE (m), and PS (m + 1). To determine when to start and end the display of the image so that the time from the end of the display of the (m + 1) th image to the point of time PE (m + 1) is equal.
The changing means further includes a time point VS (m + 1) at which reproduction of the sound associated with the m + 1st image is started before a time point VE (m) at which reproduction of the sound associated with the mth image is terminated. In some cases, VS (m + 1) is after VE (m), and the reproduction of the sound associated with the m + 2nd image is the time point VE (m + 1) at which the reproduction of the sound associated with the m + 1st image is completed. 5. The information processing according to claim 4, wherein the playback start time and the playback end time of the sound associated with the (m + 1) th image are changed so as to be before the start time VS (m + 2). apparatus.   Furthermore, when VE (m + 1) becomes PE (m + 1) or later due to the change of the sound reproduction timing, the time for reproducing the sound associated with the m + 1st image is changed to the VS after the change. When (m + 1) to PE (m + 1) and VS (m-1) is before PS (m-1), the time for playing the sound associated with the m-1st image is PS ( The information processing apparatus according to any one of claims 2 to 5, further comprising a shortening unit configured to change from m-1) to VE (m-1) after being changed.   Furthermore, when the time which reproduces | regenerates a sound is shortened, the control means which changes the speed | rate which reproduces | regenerates a sound so that the sound corresponding to the data showing the said sound is reproduced | regenerated at the shortened time is provided. The information processing apparatus described. An operation method of an information processing apparatus capable of displaying each of a plurality of images on a display unit according to a predetermined order for a certain period of time and simultaneously displaying two or more images,
Adjustment that adjusts the start and end times of the output of the sound associated with each image so that the sound associated with each of the images is not output at the same time so that the time for displaying each image is not exceeded. Process,
An output control step of outputting a sound based on a result of the adjustment in the adjustment step.   A program for causing a computer to execute the operation method according to claim 8.

Images