JP2009253342A - Information processing device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reproduce information to be reproduced so that a user can easily understand the information when reproducing the information. <P>SOLUTION: A video output part 51 displays accompanying information, such as subtitles corresponding to video or comments, together with the video to a monitor as not shown in the figure. An accompanying information amount per minute measurement part 46 measures an amount of accompanying information per minute to be a data amount reproduced per minute. A reproduction speed calculation part 47 calculates reproduction speed of video and the accompanying information on the basis of the measured amount of accompanying information per minute. A reproduction control unit 48 controls reproduction of the video and the accompanying information at the reproduction speed calculated by the reproduction speed calculation part 47. This device is applicable, for example, to a reproduction device for reproducing AV streams. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method that make it possible for a user to easily reproduce information to be reproduced when information is reproduced.

  In recent years, digitalization of television broadcasting has progressed, and with the spread of DVD (Digital Versatile Disk) and BD (Blu-ray Disc (trademark)), not only watching and enjoying video content, Now, you can enjoy subtitles, related information, and detailed information that are played with the video content.

  In addition, comments written by other users on a predetermined part of video content shared on the Internet can be enjoyed together with the video content.

  In the manner of enjoying the video content as described above, the importance of the accompanying information accompanying the video content such as related information and comments rather than the main video content is increased, and the user can enjoy the accompanying information. Is starting to watch.

  However, the data amount of the accompanying information described above is not constant, and it is not guaranteed that the data amount is less than a predetermined ratio with respect to the data amount of the video content. When the user enjoys the accompanying information together with the video content, the accompanying information is deeper than the video content, and the user may take time to understand. I sometimes gave up.

  Therefore, a method has been proposed for controlling the stillness and reproduction of the main video in accordance with the update of the content of the sub-video such as subtitles (see, for example, Patent Document 1).

JP 2003-134472 A

  However, in the method of Patent Document 1, since the video is paused and reproduced repeatedly, the video and audio are reproduced in small pieces, which may affect the understanding of the video itself.

  The present invention has been made in view of such a situation. When information is reproduced, the information to be reproduced can be easily reproduced by the user.

  An information processing apparatus according to an aspect of the present invention includes a reproducing unit that reproduces second data together with first data, and a unit time data amount that is a data amount reproduced per unit time of the second data. Reproduction control for controlling reproduction of the first data and the second data by the reproduction unit based on a measurement unit for measurement and the unit time data amount of the second data measured by the measurement unit Means.

  The information processing apparatus is further provided with a playback speed calculating means for calculating the playback speed of the first data and the second data based on the unit time data amount of the second data, and the playback control The means can control the reproduction of the first data and the second data by the reproduction means at the reproduction speed calculated by the reproduction speed calculation means.

  The first data may be main data, and the second data may be sub data accompanying the main data.

  The first data may be video data, and the second data may be character data accompanying the video data.

  The information processing apparatus further includes storage means for storing correspondence information that associates the relationship between the unit time data amount of the second data and the reproduction speed, and the reproduction speed calculation means includes the storage means The reproduction speed can be calculated on the basis of the correspondence information stored in the above and the unit time data amount of the second data measured by the measuring means.

  The correspondence information may indicate a relationship in which the reproduction speed is inversely proportional to the unit time data amount of the second data.

  The information processing apparatus further includes an input unit that inputs an instruction to adjust the relationship indicated by the correspondence information, and the reproduction speed calculation unit includes the correspondence information instructed to be adjusted and the second data. The reproduction speed can be calculated based on the unit time data amount.

  The information processing apparatus is further provided with repetition number setting means for setting a repetition number of repetitions of reproduction of a predetermined section of the first data based on the unit time data amount of the second data, and the reproduction The control means can control the reproduction of the predetermined section of the first data and the second data for the number of repetitions at the reproduction speed calculated by the reproduction speed calculation means. .

  The first data may be recorded together with the second data on a predetermined recording medium.

  Either or both of the first data and the second data can be stored in a predetermined server on the network, and the information processing apparatus includes the first data and the second data. Communication means for acquiring both or one of the two data by communication with the predetermined server can be further provided.

  Each of the second data by the plurality of reproduction control means is attached to the information processing apparatus based on the unit time data amount of each of the second data measured by the plurality of measurement means. Display area setting means for setting a display area size of each video displayed based on the reproduction control of each of the first data; and the video having the display area size set by the display area setting means Compositing means for generating a synthesized video obtained by synthesizing each of them is further provided, and the reproducing means can reproduce the synthesized video.

  Each of the second data by the plurality of reproduction control means is attached to the information processing apparatus based on the unit time data amount of each of the second data measured by the plurality of measurement means. An overlap order setting means for setting the overlap order of each of the displayed videos based on the reproduction control of each of the first data, and a composite image obtained by combining each of the images in the overlap order set by the overlap order setting means Is further provided, and the reproducing means can reproduce the synthesized video.

  Each of the second data by the plurality of reproduction control means is attached to the information processing apparatus based on the unit time data amount of each of the second data measured by the plurality of measurement means. Volume setting means for setting the volume level of the audio data of the audio output corresponding to each video displayed based on the reproduction control of each of the first data, and the volume level set by the volume setting means Synthesis means for generating synthesized voice data obtained by synthesizing each of the voice data.

  An information processing method according to one aspect of the present invention includes a reproduction step of reproducing second data together with first data, and a unit time data amount that is a data amount of the second data reproduced per unit time. Based on the measurement step to be measured and the unit time data amount of the second data measured by the measurement means, the reproduction of the first data and the second data in the process of the reproduction step is controlled. A reproduction control step.

  In one aspect of the present invention, the second data is reproduced together with the first data, and a unit time data amount that is a data amount reproduced per unit time of the second data is measured and measured by the measuring unit. The reproduction of the first data and the second data is controlled based on the unit time data amount of the second data.

  According to one aspect of the present invention, when information is reproduced, the information to be reproduced can be reproduced in a manner that is easy for the user to understand.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example of an embodiment of a playback apparatus to which the present invention is applied.

  The playback device 11 shown in FIG. 1 reads video and audio obtained by reading and decoding an AV (Audio Visual) stream recorded in a recording medium (not shown) such as a DVD and encoded by a predetermined method. Play.

  In addition to the video and music, the playback device 11 plays back text data such as subtitles and comments corresponding to the video, related information related to the video and music, detailed information indicating details of the video and music, and the like. Subtitles, comments, related information, detailed information, and the like are information accompanying video and music, and are hereinafter referred to as accompanying information. The accompanying information is reproduced together with the video.

  The AV stream read out by the playback apparatus 11 is, for example, a moving picture experts group (MPEG) 2-PS (program stream) data format, and a video stream, an audio stream, and accompanying information encoded according to the MPEG2 system are MPEG2 It is multiplexed according to the system method.

  1 includes a separation unit 41, an audio decoding unit 42, a video decoding unit 43, an accompanying information decoding unit 44, a reference time generation unit 45, an accompanying information per minute measurement unit 46, a playback speed calculation unit 47, and playback control. Unit 48, audio output unit 49, video processing unit 50, and video output unit 51.

  The separation unit 41 inserts an AV stream read from a recording medium (not shown) into a video stream, an audio stream, encoded accompanying information (hereinafter referred to as encoded accompanying information as appropriate), and an AV stream. To separate the time information. The separation unit 41 supplies the audio stream separated from the AV stream to the audio decoding unit 42, the video stream to the video decoding unit 43, and the encoded accompanying information to the accompanying information decoding unit 44. Further, the separation unit 41 supplies the separated time information to the reference time generation unit 45 and the reproduction control unit 48.

  The audio decoding unit 42 has a memory (not shown) therein, and temporarily stores (buffers) the audio stream supplied from the separation unit 41 in the memory. The audio decoding unit 42 extracts the display time information from the audio stream and supplies it to the reproduction control unit 48. The audio decoding unit 42 decodes the buffered audio stream at a predetermined timing, and supplies the audio data obtained as a result to the audio output unit 49.

  The video decoding unit 43 has a memory (not shown) therein, and temporarily stores (buffers) the video stream supplied from the separation unit 41 in the memory. The video decoding unit 43 extracts display time information from the video stream and supplies it to the playback control unit 48. The video decoding unit 43 decodes the buffered video stream at a predetermined timing, and supplies the video data obtained as a result to the video processing unit 50.

  The accompanying information decoding unit 44 has a memory (not shown) therein, and temporarily stores (buffers) the encoded accompanying information supplied from the separating unit 41 in the memory. The accompanying information decoding unit 44 extracts display time information from the encoded accompanying information and supplies it to the reproduction control unit 48. The accompanying information decoding unit 44 decodes the buffered encoded accompanying information at a predetermined timing, and supplies the accompanying information obtained as a result to the video processing unit 50.

  The reference time generation unit 45 calibrates the 27 MHz clock based on the information for calibrating the system clock included in the time information supplied from the separation unit 41, generates a system clock, This is supplied to the measurement unit 46 and the reproduction control unit 48.

  Based on the system clock from the reference time generating unit 45, the accompanying information per minute measuring unit 46 uses the accompanying information decoding unit 44 to encode the data amount of encoded accompanying information decoded per unit time such as one minute (for example, (Hereinafter referred to as “accompanying information per minute”) is measured and supplied to the reproduction speed calculation unit 47.

  The unit of the accompanying information per minute is the number of bits that the encoded accompanying information is decoded per unit time, or when the accompanying information is character data, for example, the decoded information is reproduced (displayed) per unit time. The number of characters to be used.

  The reproduction control unit 48 controls reproduction of audio data, video data, and accompanying information decoded by the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44.

  More specifically, the playback control unit 48 includes display time information supplied from the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44, and a system clock supplied from the reference time generation unit 45. When they match, the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44 are instructed to perform decoding.

  That is, the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44 are video streams and audio streams buffered in their respective memories at the timing when decoding is instructed from the reproduction control unit 48. , And the encoding-accompanying information are respectively decoded.

  The playback speed calculation unit 47 calculates the playback speed of video data, audio data, and accompanying information to be played back under the control of the playback control unit 48 based on the amount of accompanying information per minute supplied from the accompanying information per minute measurement unit 46. To do.

  Further, the playback control unit 48 controls playback of audio data, video data, and accompanying information at the playback speed calculated by the playback speed calculation unit 47.

  More specifically, the playback control unit 48 sets the display time information and the system clock so that the playback speed of the audio data, the video data, and the accompanying information becomes the playback speed calculated by the playback speed calculation unit 47. The coincidence time interval is adjusted, and the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44 are instructed to perform decoding.

  The audio output unit 49 is connected to a speaker (not shown), and outputs audio corresponding to the audio data from the audio decoding unit 42 from the speaker.

  The video processing unit 50 synthesizes the video data from the video decoding unit 43 and the accompanying information from the accompanying information decoding unit 44 and supplies them to the video output unit 51 as one piece of video data.

  The video output unit 51 is connected to a monitor (not shown) that displays video, and displays video corresponding to the video data from the video processing unit 50 on the monitor.

  Next, a configuration example of the reproduction speed calculation unit 47 in FIG. 1 will be described with reference to the block diagram in FIG.

  The playback speed calculation unit 47 in FIG. 2 includes a playback speed adjustment amount calculation unit 71, a playback speed adjustment amount management unit 72, and an inverse ratio calculation unit 73.

  The playback speed adjustment amount calculation unit 71 performs playback speed adjustment based on the accompanying information per minute amount from the accompanying information per minute measurement unit 46 and the playback speed adjustment amount correspondence information stored in the playback speed adjustment amount management unit 72. The amount is calculated and supplied to the inverse ratio calculation unit 73. The reproduction speed adjustment amount correspondence information is information for associating the relationship between the accompanying information per minute amount and the reproduction speed adjustment amount indicating the degree of slowness of the reproduction speed, and details thereof will be described later.

  The inverse ratio calculation unit 73 calculates the inverse ratio (reciprocal number) of the reproduction speed adjustment amount supplied from the reproduction speed adjustment amount calculation unit 71 and supplies it to the reproduction control unit 48.

  That is, the playback control unit 48 represents the playback speed of audio data, video data, and accompanying information by the product of the standard playback speed (hereinafter simply referred to as the standard speed) and the inverse ratio of the playback speed adjustment amount. The time interval at which the display time information and the system clock coincide with each other is adjusted so as to achieve the playback speed, and the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44 are instructed to perform decoding.

  Next, with reference to the flowchart of FIG. 3, the stream reproduction processing of the reproduction apparatus 11 of FIG. 1 will be described.

  In step S11, the separation unit 41 separates the video stream, the audio stream, the encoded associated information, and the time information from the AV stream read from a recording medium (not shown). Then, the separation unit 41 supplies the audio stream to the audio decoding unit 42, the video stream to the video decoding unit 43, and the encoded accompanying information to the accompanying information decoding unit 44. Further, the separation unit 41 supplies the separated time information to the reference time generation unit 45 and the reproduction control unit 48.

  In step S12, the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44 are supplied from the demultiplexing unit 41 and buffered in their internal memories, the video stream, the audio stream, and the encoding. Each accompanying information is decoded.

  In step S <b> 13, the incidental information minute amount measuring unit 46 measures the incidental information minute amount that is the data amount of the incidental information decoded per minute in the incidental information decoding unit 44, and supplies it to the reproduction speed calculation unit 47. .

  In step S <b> 14, the reproduction speed calculation unit 47 determines whether or not the accompanying information per minute supplied from the accompanying information per minute measurement unit 46 is smaller than a predetermined value S.

  Here, the predetermined value S is a preset value, and if it is larger than that, it is difficult for the user to completely understand the accompanying information reproduced (displayed) together with the video at the standard speed, If the value is smaller than that, it represents the amount of incidental information per minute that the incidental information can be completely understood.

  If it is determined in step S14 that the amount per minute of the accompanying information is not smaller than the predetermined value S, the process proceeds to step S15. In step S15, the playback speed adjustment amount calculation unit 71 of the playback speed calculation unit 47, based on the accompanying information per minute amount and the playback speed adjustment amount correspondence information stored in the playback speed adjustment amount management unit 72, The playback speed adjustment amount is calculated.

  Here, with reference to FIG. 4, the relationship between the amount per minute of accompanying information and the reproduction speed adjustment amount indicated by the reproduction speed adjustment amount correspondence information will be described.

  In FIG. 4, the horizontal axis indicates the amount of accompanying information per minute, and the vertical axis indicates the reproduction speed adjustment amount.

  As shown in FIG. 4, when the amount of accompanying information per minute is 0 to S, the playback speed adjustment amount is 1, and when the amount of accompanying information per minute is not smaller than S, the playback speed adjustment amount is set to the amount of accompanying information per minute. The proportional relationship is shown.

  That is, in step S15, the reproduction speed adjustment amount calculation unit 71 performs the reproduction speed adjustment amount n according to the accompanying information minute amount N based on the relationship between the accompanying information minute amount and the reproduction speed adjustment amount shown in FIG. Is supplied to the inverse ratio calculation unit 73, and the process proceeds to step S16.

  In step S <b> 16, the inverse ratio calculation unit 73 calculates the inverse ratio (reciprocal) 1 / n of the reproduction speed adjustment amount n supplied from the reproduction speed adjustment amount calculation unit 71, and supplies it to the reproduction control unit 48.

  In step S17, the reproduction control unit 48 controls reproduction of video, audio, and accompanying information at 1 / n times the standard speed.

  Specifically, the playback control unit 48 sets the time interval at which the display time information and the system clock match so that the playback speed of the audio data, video data, and accompanying information is 1 / n times the standard speed. To instruct the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44 to perform decoding. For example, the reproduction control unit 48 employs one of n clocks of the system clock as the system clock, and when the system clock and the display time information match, the audio decoding unit 42, the video decoding unit 43, and The accompanying information decoding unit 44 is instructed to perform decoding. Then, the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44 receive the video stream, the audio stream, and the encoding accompanying information buffered in the memory included therein from the reproduction control unit 48. Decoding is performed at the timing when decoding is instructed.

  Then, the voice decoding unit 42 supplies the voice data obtained as a result of the decoding to the voice output unit 49, and the voice output unit 49 sends the voice corresponding to the voice data from the voice decoding unit 42 from a speaker (not shown). Output. The video decoding unit 43 and the accompanying information decoding unit 44 supply the video data and the accompanying information obtained as a result of the decoding to the video processing unit 50, respectively, and the video processing unit 50 combines the video data and the accompanying information. The single video data is supplied to the video output unit 51. The video output unit 51 displays a video corresponding to the video data from the video processing unit 50 on the monitor.

  On the other hand, if it is determined in step S14 that the amount of incidental information per minute is smaller than the predetermined value S, the process proceeds to step S18.

  In step S18, the playback speed adjustment amount calculation unit 71 of the playback speed calculation unit 47 sets the playback speed adjustment amount to 1 based on the relationship between the accompanying information per minute amount and the playback speed adjustment amount shown in FIG. It supplies to the ratio calculation part 73. The inverse ratio calculation unit 73 calculates the inverse ratio (reciprocal) of the playback speed adjustment amount, but since the inverse ratio of the playback speed adjustment amount that is 1, the playback control unit 48 uses the video at the standard speed. Control the playback of audio and accompanying information.

  Through the above processing, the playback device 11 can play back video, audio, and accompanying information at 1 / n times the standard speed in accordance with the amount N of accompanying information per minute.

  In this way, as the amount of accompanying information per minute increases, the playback speed becomes slower, so even if the amount of accompanying information that is difficult for the user to understand completely at standard speed is an AV stream displayed together with the video, Depending on the amount of accompanying information, the playback speed can be slowed down so that the user can easily understand the playback.

  Note that the relationship between the amount of incidental information per minute and the reproduction speed adjustment amount is not limited to the proportional relationship described with reference to FIG. 4. For example, as illustrated in FIG. In the case where the reproduction speed adjustment amount is 1, and the amount per minute of the accompanying information is larger than S, the reproduction speed adjustment amount may be increased step by step for each predetermined amount of the accompanying information. In this case, the reproduction speed is decreased in steps for each predetermined amount of accompanying information.

  Next, another configuration example of the playback device will be described with reference to FIG.

  In FIG. 6, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted as appropriate. That is, the playback apparatus 111 in FIG. 6 basically includes the playback repetition number calculation unit 141, and basically includes the playback control unit 142 in place of the playback control unit 48. It has the same configuration.

  In the playback device 111 of FIG. 6, the accompanying information per minute measurement unit 46 measures the accompanying information per minute amount in the accompanying information decoding unit 44 and supplies it to the playback speed calculation unit 47 and the playback repetition number calculation unit 141.

  The reproduction repetition number calculating unit 141 repeats reproduction of a predetermined section of the video and the accompanying information displayed under the control of the video processing unit 50 based on the accompanying information per minute amount supplied from the accompanying information per minute measuring unit 46. The number of reproduction repetitions is set and supplied to the reproduction control unit 142.

  Here, the predetermined section indicates, for example, a section (time) represented by start information that specifies the start time of a scene and duration information that specifies the playback time of the scene in seconds, which are included in the video stream. . That is, the predetermined section indicates a time during which a predetermined one scene is reproduced.

  The reproduction control unit 142 has the same function as the reproduction control unit 48 and has a predetermined interval corresponding to the number of reproduction repetitions supplied from the reproduction repetition number calculation unit 141 at the reproduction speed calculated by the reproduction speed calculation unit 47. The playback is controlled so as to repeat the playback.

  Next, with reference to the flowchart of FIG. 7, the playback processing of the stream of the playback device 111 of FIG. 6 will be described.

  The processes in steps S41 and S42 and steps S50 to S53 in the flowchart of FIG. 7 are the same as the processes in steps S11 and S12 and steps S15 to S18 in the flowchart of FIG.

  In step S <b> 43, the accompanying information per minute amount measuring unit 46 measures the accompanying information per minute amount decoded per minute in the accompanying information decoding unit 44, and supplies it to the reproduction speed calculating unit 47 and the reproduction repetition number calculating unit 141. .

  In step S <b> 44, the reproduction speed calculation unit 47 determines whether or not the accompanying information per minute supplied from the accompanying information per minute measurement unit 46 is smaller than a predetermined value S.

  If it is determined in step S44 that the amount of incidental information per minute is not smaller than the predetermined value S, the process proceeds to step S45, and the reproduction speed calculation unit 47 has the amount of incidental information per minute smaller than the predetermined value M. Judge whether or not there is.

  Here, the predetermined value M represents the amount of accompanying information per minute for which the playback speed determined by the corresponding playback speed adjustment amount m is the lower limit of the playback speed in the playback device 111.

  If it is determined in step S45 that the amount of incidental information per minute is not smaller than the predetermined value M, the process proceeds to step S46, and the reproduction speed adjustment amount calculation unit 71 of the reproduction speed calculation unit 47 determines the amount of incidental information per minute. Based on the reproduction speed adjustment amount correspondence information stored in the reproduction speed adjustment amount management unit 72, the reproduction speed adjustment amount is calculated.

  Here, with reference to FIG. 8, the relationship between the amount of accompanying information per minute and the amount of playback speed adjustment and the relationship between the amount of accompanying information per minute and the number of playback repetitions, which are indicated by the playback speed adjustment amount correspondence information, will be described.

  In FIG. 8, the horizontal axis represents the amount per minute of the accompanying information, the left vertical axis represents the playback speed adjustment amount represented by a bold line, and the right vertical axis represents the number of playback repetitions represented by the thick dotted line. ing.

  As shown in FIG. 8, when the amount of accompanying information per minute is 0 to S, the playback speed adjustment amount is 1, and when the amount of accompanying information per minute is larger than S, the playback speed adjustment amount is smaller than the amount of accompanying information per minute. Indicates a proportional relationship. When the amount of accompanying information per minute is larger than M, the reproduction speed adjustment amount is a constant value m with respect to the amount of accompanying information per minute.

  That is, in step S46, the playback speed adjustment amount calculation unit 71 sets the playback speed adjustment amount to m based on the relationship between the accompanying information per minute amount and the playback speed adjustment amount shown in FIG. To proceed to step S47.

  In step S <b> 47, the inverse ratio calculation unit 73 calculates the inverse ratio (reciprocal) 1 / m of the reproduction speed adjustment amount supplied from the reproduction speed adjustment amount calculation unit 71, and supplies it to the reproduction control unit 142.

  In step S48, the reproduction repetition number calculation unit 141 sets a reproduction repetition number for repeating reproduction of a predetermined scene of the video based on the amount of incidental information per minute supplied from the amount of incidental information measurement unit 46, and performs reproduction control. To the unit 142.

  Here, as shown in FIG. 8, when the amount of accompanying information per minute is 0 to M, the number of reproduction repetitions is 1, and when the amount of accompanying information per minute is larger than M and smaller than Z, the number of reproduction repetitions is 2. Further, when the amount of incidental information per minute is larger than Z, the number of reproduction repetitions is 3.

  That is, in step S48, the reproduction repetition number calculation unit 141, for example, has an amount of incidental information per minute greater than M and a value of Z based on the relationship between the amount of incidental information per minute and the number of reproduction repetitions shown in FIG. If it is smaller, the number of repetitions of reproduction is set to 2, supplied to the reproduction control unit 142, and the process proceeds to step S49.

  In step S49, the reproduction control unit 142 controls the reproduction so that the reproduction of the video, audio, and accompanying information in a predetermined section at 1 / m times the standard speed is repeated for the number of reproduction repetitions. For example, the playback control unit 48 controls the playback of video, audio, and accompanying information so that a predetermined scene is repeated twice at 1 / m times the standard speed.

  On the other hand, if it is determined in step S45 that the amount per minute of the accompanying information is smaller than the predetermined value M, the process proceeds to step S50, and finally the video, audio, and Accompanying information is reproduced.

  If it is determined in step S44 that the amount per minute of the accompanying information is smaller than the predetermined value S, the process proceeds to step S53, and finally the video, audio, and accompanying information are reproduced at the standard speed. The

  With the above processing, the playback device 111 plays back video, audio, and accompanying information at 1 / n times the standard speed according to the amount N of incidental information, and when the amount is larger than the amount M of incidental information. Thus, it is possible to repeatedly reproduce a predetermined section while maintaining the lower limit of the reproduction speed.

  In this way, since a predetermined section is repeatedly played at the slowest playback speed, an amount of accompanying information that is difficult for the user to understand completely is an AV stream displayed together with the video even at the slowest playback speed. However, it is possible to reproduce the content so that it can be easily understood by the user.

  In the above description, the section in which playback is repeated is the time during which a predetermined scene determined by start information and duration information is played back. However, for example, a predetermined time such as 5 seconds is set. Also good. Further, it may be an interval between scene changes detected using a change in the feature amount of the video, or an interval at which the context of the character string (character data) as accompanying information reproduced along with the video is switched. Also good.

  In FIG. 8, the reproduction speed for the accompanying information per minute larger than M is made constant. For example, as shown in FIG. 9, every time the number of reproduction repetitions increases according to the accompanying information per minute. In addition, the playback speed adjustment amount increased to m may be lowered to a predetermined value, that is, the playback speed that has been reduced to 1 / m times the standard speed may be increased to the predetermined playback speed.

  In the above description, the playback device reads an AV stream including video, audio, and accompanying information (multiplexed) from a recording medium (not shown). However, the playback device 211 shown in FIG. As described above, part or all of the information included in the AV stream may be read from the server 212 via a network such as the Internet 210.

  FIG. 11 shows an example of the configuration of a playback apparatus 211 configured to read a part of information included in the AV stream from the server 212.

  In FIG. 11, the same reference numerals are given to the portions corresponding to those in FIG. 1, and the description thereof will be omitted as appropriate. That is, the playback device 211 in FIG. 11 has basically the same configuration as the playback device 11 in FIG. 1 except that it includes a separation unit 241 instead of the separation unit 41 and a communication unit 242. It is what you have.

  In the playback device 211 of FIG. 11, the separation unit 241 separates the video stream, the audio stream, and the time information inserted in the AV stream from the AV stream read from the recording medium (not shown), The audio stream is supplied to the audio decoding unit 42, and the video stream is supplied to the video decoding unit 43. In addition, the separation unit 241 supplies the separated time information to the reference time generation unit 45 and the reproduction control unit 48.

  The communication unit 242 communicates with the server 212 via the Internet 210 to acquire accompanying information (encoded accompanying information) corresponding to an AV stream stored in a recording medium (not shown) from the server 212. And supplied to the accompanying information decoding unit 44.

  With such a configuration, even when an AV stream that does not include accompanying information is read and played, the playback device 211 reduces the playback speed according to the amount of accompanying information from the server 212 and understands the user. Can be played easily.

  In the configuration of FIG. 11, the accompanying information is acquired from the server 212, but an AV stream including video, audio, and accompanying information may be acquired from the server 212. Furthermore, an AV stream including video and audio and accompanying information (encoded accompanying information) may be acquired from different servers.

  11 is the same as the process described with reference to the flowchart of FIG. 3 except that the accompanying information is acquired from the server 212, and thus the description thereof is omitted. To do.

  In the above, an example in which the playback device adjusts the playback speed has been described, but the user himself / herself could not adjust the playback speed.

  Thus, with reference to FIG. 12, an example of the configuration of a playback apparatus that allows the user to adjust the playback speed will be described.

  In FIG. 12, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted as appropriate. That is, the playback apparatus 311 in FIG. 12 is basically provided with an operation input unit 341, except that a playback speed calculation unit 342 is provided instead of the playback speed calculation unit 47, basically the playback apparatus 11 in FIG. It has the same configuration as.

  The operation input unit 341 includes, for example, various operation buttons and dials. When an instruction to the playback device 311 is input, the operation input unit 341 is operated by the user, generates an operation signal corresponding to the operation content, and calculates a playback speed. To the unit 342.

  More specifically, the operation input unit 341 is operated by the user when an adjustment regarding the playback speed is instructed, generates an operation signal corresponding to the operation content, and supplies the operation signal to the playback speed calculation unit 342.

  The reproduction speed calculation unit 342 performs adjustment related to the calculated reproduction speed according to the operation signal from the operation input unit 341.

  Next, a configuration example of the reproduction speed calculation unit 342 in FIG. 12 will be described with reference to the block diagram in FIG.

  In FIG. 13, the same reference numerals are given to the portions corresponding to those in FIG. That is, the playback speed calculation unit 342 in FIG. 13 basically has the same configuration as the playback speed calculation unit 47 in FIG. 2 except that it includes an inverse ratio calculation unit 371 in place of the inverse ratio calculation unit 73. Is.

  The inverse ratio calculation unit 371 calculates an inverse ratio (reciprocal) of the reproduction speed adjustment amount supplied from the reproduction speed adjustment amount calculation unit 71, and determines a predetermined ratio according to the inverse ratio and an operation signal from the operation input unit 341. The product obtained as a result of the multiplication is supplied to the reproduction control unit 48.

  For example, the inverse ratio calculation unit 371 multiplies the inverse ratio 1 / n of the calculated reproduction speed adjustment amount and the speed adjustment coefficient α, and supplies the product α / n obtained as a result of the multiplication to the reproduction control unit 48. To do.

  When the speed adjustment coefficient α is less than 1, the product α / n is a value smaller than the inverse ratio 1 / n, and the playback speed is slower than 1 / n times the standard speed. When the speed adjustment coefficient α is larger than 1, the product α / n is larger than the inverse ratio 1 / n, and the reproduction speed is faster than 1 / n times the standard speed.

  Next, the playback speed calculation process of the playback speed calculation unit 342 in FIG. 13 will be described with reference to the flowchart in FIG.

  In step S61, the inverse ratio calculation unit 371 of the reproduction speed calculation unit 342 determines whether or not the operation input unit 341 is operated by the user and an instruction to adjust the speed is given.

  In step S61, when an instruction to adjust the speed is given, the process proceeds to step S62.

  In step S <b> 62, the inverse ratio calculation unit 371 operates the operation signal indicating that the inverse ratio 1 / n of the reproduction speed adjustment amount supplied from the reproduction speed adjustment amount calculation unit 71 and the speed adjustment from the operation input unit 341 are instructed. The product α / n obtained as a result of the multiplication is supplied to the reproduction control unit 48.

  In step S61, when the speed adjustment is not instructed, the process in step S61 is repeated until the speed adjustment is instructed.

  With the above processing, the playback speed can be adjusted by the speed adjustment coefficient α according to the user's operation. Therefore, even for a plurality of users having different speeds of understanding the same amount of accompanying information, Easy to understand and play.

  In the above description, the example of adjusting the inverse ratio based on the operation signal has been described. However, the playback speed adjustment amount may be further adjusted.

  FIG. 15 shows a configuration example of a playback speed calculation unit that also adjusts the playback speed adjustment amount.

  The playback speed calculation unit 342 in FIG. 15 includes the playback speed adjustment amount management unit 372 in place of the playback speed adjustment amount management unit 72 in the playback speed calculation unit 342 in FIG. The speed calculation unit 342 has the same configuration.

  The reproduction speed adjustment amount management unit 372 stores the reproduction speed adjustment amount correspondence information and changes the reproduction speed adjustment amount correspondence information according to the operation signal from the operation input unit 341.

  For example, in the relationship between the incidental information minute amount N and the reproduction speed adjustment amount n described in FIG. 4 indicated by the reproduction speed adjustment amount correspondence information, when the incidental information minute amount is larger than S, n = pN (p is Define the equation shown by (tilt). In this case, the reproduction speed adjustment amount management unit 372 multiplies the inclination p by a predetermined coefficient β (hereinafter referred to as an inclination adjustment coefficient β) corresponding to the operation signal from the operation input unit 341. That is, in FIG. 4, the relationship between the incidental information amount N and the reproduction speed adjustment amount n when the incidental information amount S is larger than n is represented by n = βpN.

  When the speed adjustment coefficient β is less than 1, the playback speed adjustment amount n with respect to the accompanying information per minute amount N is a smaller value when the slope is p, and the playback speed is faster than when the slope is p. When the speed adjustment coefficient β is larger than 1, the playback speed adjustment amount n with respect to the accompanying information per minute amount N is larger than the slope p, and the playback speed is slower than the slope p.

  Next, the playback speed calculation process of the playback speed calculator 342 in FIG. 15 will be described with reference to the flowchart in FIG.

  In step S81, the playback speed adjustment amount management unit 372 of the playback speed calculation unit 342 determines whether or not the operation input unit 341 has been operated by the user to instruct speed adjustment.

  In step S81, when the speed adjustment is instructed, the process proceeds to step S82.

  In step S82, the reproduction speed adjustment amount management unit 372 multiplies the inclination p in the reproduction speed adjustment amount correspondence information by the inclination adjustment coefficient β according to the operation signal from the operation input unit 341.

  After that, the playback speed adjustment amount calculation unit 71 obtains the playback speed adjustment amount n based on the relationship between the accompanying information per minute amount multiplied by the inclination adjustment coefficient β and the playback speed adjustment amount, and the inverse ratio calculation unit 371 The inverse ratio 1 / n is calculated. At this time, the inverse ratio calculation unit 371 may multiply the calculated inverse ratio 1 / n and the speed adjustment coefficient α in accordance with the operation signal from the operation input unit 341.

  In step S81, when the speed adjustment is not instructed, the process in step S81 is repeated until the speed adjustment is instructed.

  With the above processing, the finally obtained reproduction speed can be adjusted by the inclination adjustment coefficient β according to the user's operation. Therefore, for a plurality of users having different speeds of understanding the same amount of accompanying information Can be reproduced in a manner that is easy to understand for each user.

  The reproduction speed adjustment amount management unit 372 in FIG. 15 stores a plurality of pieces of reproduction speed adjustment amount correspondence information, and is referred to by the reproduction speed adjustment amount calculation unit 71 in accordance with an operation signal from the operation input unit 341. The speed adjustment amount correspondence information may be changed. For example, the playback speed adjustment amount management unit 372 stores playback speed adjustment amount correspondence information indicating the relationship described with reference to FIG. 4 and playback speed adjustment amount correspondence information indicating the relationship described with reference to FIG. One of the reproduction speed adjustment amount correspondence information may be selected in accordance with the operation signal from.

  Further, the playback speed adjustment amount management unit 372 in FIG. 15 stores the playback speed adjustment amount correspondence information, but stores new playback speed adjustment amount correspondence information according to the operation signal from the operation input unit 341. May be. For example, the reproduction speed adjustment amount management unit 372 stores reproduction speed adjustment amount correspondence information indicating the relationship described with reference to FIG. 4, and the reproduction indicating the relationship described with reference to FIG. 5 by an operation signal from the operation input unit 341. The speed adjustment amount correspondence information may be newly stored, and the reproduction may be controlled based on any reproduction speed adjustment amount correspondence information according to the user.

  As described above, the playback speed that is slowed according to the amount of accompanying information can be further adjusted according to the user's operation, so that it is easy for the user to understand according to the speed of understanding the accompanying information. Can be played.

  In the above, an example in which a single AV stream is handled has been described. However, the present invention can also be applied to an example in which a plurality of AV streams are reproduced, combined, and displayed.

  FIG. 17 shows a configuration example of an embodiment of a playback apparatus that plays back, combines, and displays a plurality of streams.

  The playback device 531 in FIG. 17 is obtained by reading and decoding a plurality of AV streams encoded by a predetermined method from a recording medium (not shown) such as a DVD or a server (not shown) on the network, for example. Play multiple videos with corresponding accompanying information.

  The playback device 531 in FIG. 17 includes playback processing units 541-1 to 541-4, a display area setting unit 542, a video composition unit 543, and a video output unit 544.

  Each of the reproduction processing units 541-1 to 541-4 separates and decodes an AV stream read from a recording medium (not shown) or a server (not shown), and supplies the obtained video data to the video synthesis unit 543. . In addition, the reproduction processing units 541-1 to 541-4 each measure the amount of accompanying information for the read AV stream and supply the measured amount to the display area setting unit 542.

In FIG. 17, the reproduction processing unit 541-1 supplies the video data of the video A ′ obtained by decoding the read stream A (hereinafter, also simply referred to as the video A ′) to the video synthesis unit 543. At the same time, the measured amount N _A of the accompanying information is supplied to the display area setting unit 542. Supplying reproduction processing section 541-2, the read image B 'obtained by decoding the stream B, and supplies to the video synthesis section 543, accompanying information for each quantity N _B measured, the display area setting unit 542 To do.

Similarly, the reproduction processing unit 541-3 supplies the video C ′ obtained by decoding the read stream C to the video synthesizing unit 543, and the measured incidental information amount N _C is displayed on the display area setting unit. 542. The reproduction processing unit 541-4 supplies the video D ′ obtained by decoding the read stream D to the video synthesis unit 543 and supplies the measured incidental information amount N _D to the display area setting unit 542. To do.

The display area setting unit 542 displays each of the images A ′ to D ′ displayed on the image output unit 544 based on the amounts of accompanying information N _{A to} N _D supplied from the reproduction processing units 541-1 to 541-4. The size of the display area is set and supplied to the video composition unit 543.

  The video composition unit 543, based on the sizes of the display areas of the videos A ′ to D ′ from the display area setting unit 542, the videos A ′ to D supplied from the reproduction processing units 541-1 to 541-4. 'Is set to the size of the set display area, and synthesized composite video data is generated. The video composition unit 543 supplies the generated composite video data to the video output unit 544.

  The video output unit 544 is connected to a monitor (not shown) that displays video, and displays a composite video corresponding to the composite video data from the video synthesis unit 543 on the monitor.

  Next, a configuration example of the reproduction processing units 541-1 to 541-4 in FIG. 17 will be described with reference to the block diagram in FIG. Note that the reproduction processing units 541-1 to 541-4 have the same configuration, and are referred to simply as the reproduction processing unit 541 when distinction is unnecessary.

  In FIG. 18, parts corresponding to those in FIG. 1 are denoted with the same reference numerals, and description thereof is omitted as appropriate. That is, the reproduction processing unit 541 in FIG. 18 basically has the same configuration as the reproduction device 11 in FIG. 1 except that the reproduction speed calculation unit 47, the audio output unit 49, and the video output unit 51 are deleted. Is.

  The audio decoding unit 42 decodes the audio stream of the AV stream, and supplies the audio data obtained as a result to an audio synthesis unit (not shown).

  The accompanying information per minute measuring unit 46 measures the accompanying information per minute based on the system clock from the reference time generating unit 45 and outputs it to the display area setting unit 542.

  The reproduction control unit 48 simply controls reproduction of audio data, video data, and accompanying information decoded by the audio decoding unit 42, the video decoding unit 43, and the accompanying information decoding unit 44.

  The video processing unit 50 synthesizes the video data from the video decoding unit 43 and the accompanying information from the accompanying information decoding unit 44 and supplies them to the video composition unit 543 as one piece of video data.

  Next, a configuration example of the display area setting unit 542 in FIG. 17 will be described with reference to the block diagram in FIG.

  The display area setting unit 542 in FIG. 19 includes a display area magnification calculation unit 551 and a display area magnification management unit 552.

The display area magnification calculation unit 551 includes the amounts N _{A to} N _D of accompanying information from the reproduction processing units 541-1 to 541-4 and the display area magnification correspondence information stored in the display area magnification management unit 552. Based on this, the display area magnification is calculated and supplied to the video composition unit 543. The display area magnification correspondence information is information that associates the amount of the accompanying information per minute and the display area magnification that represents the magnification with respect to the display area where the video is originally displayed, and details thereof will be described later.

  That is, the video composition unit 543 multiplies a window in which the videos A ′ to D ′ are displayed by a predetermined number based on the display area magnification for each of the videos A ′ to D ′ from the display area setting unit 542. A composite video is generated as a single video.

  Next, with reference to the flowchart of FIG. 20, the reproduction processing of a plurality of streams of the reproduction device 531 of FIG. 17 will be described.

  In the multi-stream playback process, the playback processing units 541-1 to 541-4 of the playback device 531 sequentially separate and decode the read AV streams.

  In step S111, the associated information amount measuring unit 46 of each of the reproduction processing units 541-1 to 541-4 measures the amount of associated information decoded per minute for the streams A to D, and displays the display area setting unit. 542.

In step S112, the display area setting unit 542 selects any one of the accompanying information per unit amounts N _{A to} N _D supplied from the accompanying information per unit amount measuring unit 46 of each of the reproduction processing units 541-1 to 541-4. It is determined whether or not the value is smaller than a predetermined value S.

  Here, if the predetermined value S is a value larger than that, the size of the window in which the image is displayed increases according to the accompanying information, and if the value S is smaller than that, the window size is the standard size. Represents the amount of accompanying information per minute that remains to be sized.

  If it is determined in step S112 that the amount of incidental information per minute is not smaller than the predetermined value S, the process proceeds to step S113, and the display area magnification calculation unit 551 of the display area setting unit 542 The display area magnification is calculated based on the display area magnification correspondence information stored in the display area magnification management unit 552.

  Here, with reference to FIG. 21, the relationship between the amount per incidental information and the display area magnification indicated by the display area magnification correspondence information will be described.

  In FIG. 21, the horizontal axis indicates the amount of incidental information per minute, and the vertical axis indicates the display area magnification.

  As shown in FIG. 21, when the incidental information minute amount is 0 to S, the display area magnification is 1, and when the incidental information minute amount is larger than S, the display area magnification is proportional to the incidental information minute amount. Show the relationship.

That is, in step S113, the display area magnification calculation unit 551 obtains the display area magnification n _w corresponding to the incidental information per-unit amount N based on the relationship between the per-accompanying information per unit amount and the display region magnification shown in FIG. The process proceeds to step S115.

  On the other hand, when it is determined in step S112 that the amount per minute of the accompanying information is smaller than the predetermined value S, the process proceeds to step S114.

In step S114, the display area magnification calculation unit 551 of the display area setting unit 542 sets the display area magnification _nw to 1 based on the relationship between the amount of incidental information and the display area magnification shown in FIG. The process proceeds to step S115.

In step S115, the display area setting unit 542 determines whether the display area magnification _nw has been obtained for all the streams (videos).

In step S115, for all the streams (video), if it is determined not to seek display area magnification n _w, the process returns to step S112, for all the streams (video), determine the display area magnification n _w Steps S112 through S115 are repeated.

On the other hand, in step S115, for all the streams (video), when it is determined that seeking display area magnification n _w, the display area setting unit 542, a video display area magnification n _w obtained for all streams (video) The data is supplied to the combining unit 543, and the process proceeds to step S116.

In step S116, the video composition unit 543 displays windows in which the videos A ′ to D ′ are displayed based on the display area magnification n _w for each of the videos A ′ to D ′ from the display area setting unit 542, respectively. _The combined video data that has been multiplied by _w and combined as one video is generated and supplied to the video output unit 544. The video output unit 544 displays a synthesized video corresponding to the synthesized video data from the video synthesizing unit 543 on a monitor (not shown).

With the above processing, for example, the display area magnification n _w corresponding to each of the accompanying information per minute amounts N _{A to} N _C supplied from the reproduction processing units 541-1 to 541-3 is 1, and the reproduction processing unit When the display area magnification n _w corresponding to the per-minute amount N _D of the accompanying information supplied from 541-4 is a value larger than 1, the playback device 531 displays the playback processing unit 541- as shown in FIG. A window W ₁ in which the video A ′ from 1 is displayed, a window W ₂ in which the video B ′ from the reproduction processing unit 541-2 is displayed, and a window W in which the video C ′ from the reproduction processing unit 541-3 is displayed. ₃ each size, and remains a standard size, the size of the window W ₄ video D from the reproduction processing unit 541-4 'is displayed, and the size of n _w times the standard size The composite video is connected to the video output unit 544 Can be displayed on Nita (not shown).

In addition, when the windows W _{1 to} W ₄ having a size _nw times all become so large that they cannot be displayed on a monitor (not shown) connected to the video output unit 544, respectively. window W ₁ to W ₄ of the magnitude corresponding to the ratio of the display area magnification n _w is may be displayed.

  Thus, as the amount of accompanying information per minute for a predetermined video increases, the size of the window in which the predetermined video is displayed in the composite video increases. For example, one scene of a video where comments are concentrated In this case, the user's attention can be drawn, and reproduction can be easily performed for the user.

  In the above, an example of adjusting the window size according to the amount of accompanying information per minute when reproducing a plurality of streams has been described. However, the visibility may be changed according to the amount of accompanying information per minute. Therefore, for example, the windows may be displayed in an overlapping manner and the order thereof may be adjusted.

  FIG. 23 shows an example of the configuration of an embodiment of a playback apparatus in which, when a plurality of streams are played back, the respective windows are displayed in an overlapping manner, and the order is adjusted according to the amount of accompanying information per minute.

  In FIG. 23, parts corresponding to those in FIG. 17 are denoted by the same reference numerals, and description thereof is omitted as appropriate. That is, the playback apparatus 561 in FIG. 23 basically has the same configuration as the playback apparatus 531 in FIG. 17 except that the display order setting section 542 is replaced with an overlap order setting section 571.

The stacking order setting unit 571 displays on a monitor (not shown) connected to the video output unit 544 based on the accompanying information per minute amounts N _{A to} N _D supplied from the reproduction processing units 541-1 to 541-4. The overlapping order for the images A ′ to D ′ to be set is set and supplied to the image composition unit 543.

  The video composition unit 543 sets the videos A ′ to D ′ supplied from the reproduction processing units 541-1 to 541-4 based on the overlay order of the videos A ′ to D ′ from the overlap order setting unit 571. The synthesized video data is generated by synthesizing in the overlapped order.

  Next, with reference to the flowchart of FIG. 24, the reproduction processing of a plurality of streams of the reproduction device 561 of FIG. 23 will be described.

  In the multi-stream playback process, the playback processing units 541-1 to 541-4 of the playback device 561 sequentially separate and decode the read AV streams.

  In step S141, the accompanying information amount measuring unit 46 of each of the reproduction processing units 541-1 to 541-4 measures the amount of accompanying information per minute decoded per minute for the streams A to D, and sets the overlapping order setting unit. 571.

In step S142, the stacking order setting unit 571 determines the video output unit according to the accompanying information per unit amounts N _{A to} N _D supplied from the accompanying information per unit amount measuring units 46 of the reproduction processing units 541-1 to 541-4. An overlapping order is set for the images A ′ to D ′ displayed on a monitor (not shown) connected to 544, and is supplied to the image composition unit 543.

More specifically, the stacking order setting unit 571 ranks the accompanying information per minute amounts N _{A to} N _D supplied from the reproduction processing units 541-1 to 541-4 in descending order of the data amount. The ranking is set as the overlapping order of the images A ′ to D ′ corresponding to the ranked amounts N _{A to} N _D of the accompanying information.

  In step S143, the video composition unit 543, based on the overlapping order of the videos A ′ to D ′ from the overlapping order setting unit 571, the videos A ′ to D supplied from the reproduction processing units 541-1 to 541-4. 'Is synthesized in the set overlapping order to generate synthesized video data, which is supplied to the video output unit 544. The video output unit 544 displays a synthesized video corresponding to the synthesized video data from the video synthesizing unit 543 on a monitor (not shown).

As a result of the above processing, for example, N _D > N _B = N _C > N _A is ranked for the incidental information per minute amounts N _{A to} N _D supplied from the reproduction processing units 541-1 to 541-4, respectively. In this case, as shown in FIG. 25, the playback device 561 combines the window W ₄ with the forefront, the window W ₂ and the window W ₃ on the back of the window W ₄ , and the window W ₁ on the back. Can be displayed.

  In this way, as the amount of accompanying information per minute for a predetermined video increases, the overlapping order of the predetermined video in the composite video becomes the foreground. Therefore, as in the example of FIG. In one scene, the user's attention can be drawn, and playback can be easily performed for the user.

  In the above description, when reproducing a plurality of streams, the example of adjusting the window size and the stacking order for each stream according to the amount of accompanying information per minute has been described. However, not only the window but also, for example, the volume is adjusted. It may be.

  FIG. 26 shows a configuration example of an embodiment of a playback apparatus in which the volume for each stream is adjusted when a plurality of streams are played back.

  The playback device 631 in FIG. 26 is obtained by reading and decoding a plurality of AV streams encoded by a predetermined method from a recording medium (not shown) such as a DVD or a server (not shown) on the network. Outputs audio corresponding to multiple images.

  The playback device 631 in FIG. 26 includes playback processing units 541-1 to 541-4, a volume setting unit 641, a voice synthesis unit 642, and a voice output unit 643.

  Each of the reproduction processing units 541-1 to 541-4 separates and decodes an AV stream read from a recording medium (not shown) or a server (not shown), and supplies the obtained voice data to the voice synthesis unit 642. . In addition, each of the reproduction processing units 541-1 to 541-4 measures the amount of incidental information for the read AV stream and supplies the measured amount to the volume setting unit 641.

In FIG. 26, the reproduction processing unit 541-1 uses the audio data of the audio A ″ obtained by decoding the read stream A (hereinafter also simply referred to as audio A ″) as an audio synthesis unit 642. supplying supplies, accompanying information for each quantity N _a measured, the volume setting unit 641. Supplying reproduction processing section 541-2, read audio B '' obtained by decoding the stream B, and supplies the voice synthesis unit 642, the associated information for each quantity N _B measured, the volume setting unit 641 To do.

Similarly, the reproduction processing unit 541-3 supplies the audio C ″ obtained by decoding the read stream C to the audio synthesis unit 642, and the measured accompanying information per minute amount N _C is supplied to the volume setting unit. 641. The reproduction processing unit 541-4 supplies the audio D '' obtained by decoding the read stream D to the audio synthesizing unit 642, and supplies the measured incidental information amount N _D to the volume setting unit 641. To do.

The volume setting unit 641 is configured to output audio A ″ to D ″ output to the audio output unit 643 based on the accompanying information per minute amounts N _{A to} N _D supplied from the reproduction processing units 541-1 to 541-4. A volume level of (speech data) is set and supplied to the speech synthesizer 642.

  The voice synthesizer 642, based on the volume levels of the voices A '' to D '' from the volume setting part 641, respectively, the voices A '' to D 'supplied from the reproduction processing units 541-1 to 541-4. 'Is set to each set volume level, and synthesized voice data is generated. The voice synthesis unit 642 supplies the generated synthesized voice data to the voice output unit 643.

  The audio output unit 643 is connected to a speaker (not shown), and outputs audio corresponding to the synthesized audio data from the audio synthesis unit 642 from the speaker.

  Next, a configuration example of the volume setting unit 641 in FIG. 26 will be described with reference to the block diagram in FIG.

  The volume setting unit 641 in FIG. 27 includes a volume level magnification calculation unit 651 and a volume level magnification management unit 652.

The volume level magnification calculation unit 651 uses the accompanying information per minute amounts N _{A to} N _D from the reproduction processing units 541-1 to 541-4 and the volume level magnification correspondence information stored in the volume level magnification management unit 652. Based on this, the volume level magnification is calculated and supplied to the speech synthesizer 642. The volume level magnification correspondence information is information associating the relationship between the amount of incidental information per minute and the volume level magnification representing the magnification with respect to the standard volume level at which the sound is originally output. Details thereof will be described later.

  That is, the voice synthesis unit 642 multiplies the volume levels of the voices A ″ to D ″ by a predetermined number based on the volume level magnification for each of the voices A ″ to D ″ from the volume setting unit 641. A synthesized voice synthesized as one voice is generated.

  Next, with reference to the flowchart of FIG. 28, a description will be given of a reproduction process of a plurality of streams of the reproduction device 631 of FIG.

  In the multi-stream playback process, the playback processing units 541-1 to 541-4 of the playback device 631 sequentially separate and decode the read AV streams.

  In step S171, the accompanying information amount measuring unit 46 of each of the reproduction processing units 541-1 to 541-4 measures the amount of accompanying information per minute decoded for each of the streams A to D, and the volume setting unit 641. To supply.

In step S172, the volume setting unit 641 performs predetermined processing on any one of the accompanying information per minute amounts N _{A to} N _D supplied from the accompanying information per minute measuring unit 46 of each of the reproduction processing units 541-1 to 541-4. It is determined whether or not the value is smaller than S.

  Here, if the predetermined value S is a value larger than that, the sound volume level becomes higher according to the accompanying information, and if the value S is smaller than that, the sound volume level remains at the standard sound volume level. Represents the amount of accompanying information per minute.

  If it is determined in step S172 that the amount of incidental information per minute is not smaller than the predetermined value S, the process proceeds to step S173, and the volume level magnification calculation unit 651 of the volume setting unit 641 Based on the volume level magnification correspondence information stored in the volume level magnification management unit 652, the volume level magnification is calculated.

  Here, with reference to FIG. 29, the relationship between the amount per minute of the accompanying information and the volume level magnification indicated by the volume level magnification correspondence information will be described.

  In FIG. 29, the horizontal axis indicates the amount per minute of the accompanying information, and the vertical axis indicates the volume level magnification.

  As shown in FIG. 29, when the incidental information minute amount is 0 to S, the volume level magnification is 1, and when the incidental information minute amount is larger than S, the volume level magnification is proportional to the incidental information minute amount. Show the relationship.

That is, in step S173, the volume level magnification calculation unit 651 obtains the volume level magnification n _v corresponding to the incidental information minute N based on the relationship between the incidental information minute and the volume level magnification shown in FIG. The process proceeds to step S175.

  On the other hand, if it is determined in step S172 that the amount per minute of the accompanying information is smaller than the predetermined value S, the process proceeds to step S174.

In step S174, the volume level magnification calculator 651 of the volume setting unit 641 sets the volume level magnification n _v to 1 based on the relationship between the accompanying information per minute and the volume level magnification shown in FIG. Proceed to step S175.

In step S175, the volume setting unit 641 determines whether or not the volume level magnification _nv has been obtained for all streams (sounds).

In step S175, for all the streams (audio), if it is determined not to seek volume level ratio n _v, the process returns to step S172, for all the streams (voice) to determine the volume level ratio n _v Steps S172 to S175 are repeated until the above.

On the other hand, in step S175, for all the streams (audio), if it is determined that determined the volume level ratio n _v, the volume setting unit 641, speech synthesis volume level ratio n _v obtained for all streams (voice) The process proceeds to step S176.

In step S176, the voice synthesis unit 642 sets the volume levels of the voices A ″ to D ″ based on the volume level magnification n _v for each of the voices A ″ to D ″ from the volume setting unit 641. The synthesized voice data is multiplied by n _v and synthesized as one voice, and is supplied to the voice output unit 643. The audio output unit 643 outputs audio corresponding to the synthesized audio data from the audio synthesis unit 642 from a speaker (not shown).

As a result of the above processing, the playback device 631, for example, as shown in FIG. 30, the video A ′ according to the amounts N _{A to} N _D of accompanying information supplied from the playback processing units 541-1 to 541-4. The volume a sound that is _nv times the volume level of the sound A ″ that is output corresponding to the sound volume B, and the sound of the volume b that is _nv times the volume level of the sound B ″ that is output corresponding to the video B ′ , sound volume level n _v multiplied by volume c of '' sound C which is output corresponding to 'video C, and the volume level of' audio D output in response to the 'video D n _v A synthesized voice obtained by synthesizing the voice of the doubled volume d can be output.

  Thus, as the amount of accompanying information per minute for a predetermined video increases, the volume level of the voice corresponding to the predetermined video in the synthesized voice increases. Therefore, as in the examples of FIGS. In one scene of concentrated video, the user's attention can be drawn, and the reproduction can be easily performed by the user.

  As described above, the playback apparatus plays back accompanying information together with video, measures the amount of accompanying information per minute, which is the amount of data to be played back per unit time, and based on the measured amount of accompanying information per minute. Since the playback of video and accompanying information is controlled, the amount of accompanying information that is difficult for the user to understand completely at standard speed depends on the amount of accompanying information, even in an AV stream that is displayed with the video. The reproduction speed can be slowed down so that the user can easily understand the reproduction.

  Further, although the embodiment in which the present invention is applied to a playback apparatus that plays back an AV stream has been described, the present invention can be applied to other information processing apparatuses having a function of playing back video content.

  In the above description, the playback target is an AV stream in which accompanying information is displayed together with video. However, a presentation application or the like having a slide show function for continuously playing back slides with figures and characters is displayed. It may be a target.

  A series of the above-described reproduction processes can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 31 is a block diagram illustrating a hardware configuration example of a computer that executes the above-described series of processing by a program.

  In a computer, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are connected to each other by a bus 804.

  An input / output interface 805 is further connected to the bus 804. The input / output interface 805 includes an input unit 806 made up of a keyboard, mouse, microphone, etc., an output unit 807 made up of a display, a speaker, etc., a storage unit 808 made up of a hard disk, nonvolatile memory, etc., and a communication unit 809 made up of a network interface, etc. A drive 810 for driving a removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is connected.

  In the computer configured as described above, the CPU 801 loads the program stored in the storage unit 808 into the RAM 803 via the input / output interface 805 and the bus 804 and executes the program, for example. Is performed.

  The program executed by the computer (CPU 801) is, for example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.), a magneto-optical disk, or a semiconductor. The program is recorded on a removable medium 811 that is a package medium including a memory or the like, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  The program can be installed in the storage unit 808 via the input / output interface 805 by attaching the removable medium 811 to the drive 810. The program can be received by the communication unit 809 via a wired or wireless transmission medium and installed in the storage unit 808. In addition, the program can be installed in the ROM 802 or the storage unit 808 in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a block diagram which shows the structural example of one Embodiment of the reproducing | regenerating apparatus to which this invention is applied. It is a block diagram which shows the structural example of the reproduction speed calculation part. 6 is a flowchart for explaining a stream reproduction process of the reproduction apparatus in FIG. 1. It is a figure explaining the relationship between the amount per minute of face accompanying information, and the reproduction speed adjustment amount. It is a figure explaining the relationship between the amount per minute of face accompanying information, and the reproduction speed adjustment amount. FIG. 38 is a block diagram illustrating another configuration example of the playback device. FIG. 7 is a flowchart for explaining a stream playback process of the playback apparatus of FIG. 6. FIG. It is a figure explaining the relationship between the amount per incidental information and the reproduction speed adjustment amount, and the relationship between the amount per incidental information and the number of reproduction repetitions. It is a figure explaining the relationship between the amount per incidental information and the reproduction speed adjustment amount, and the relationship between the amount per incidental information and the number of reproduction repetitions. It is a block diagram explaining the reproducing | regenerating apparatus connected to a network. [Fig. 38] Fig. 38 is a block diagram illustrating still another configuration example of the playback device. [Fig. 38] Fig. 38 is a block diagram illustrating still another configuration example of the playback device. It is a block diagram which shows the structural example of the reproduction speed calculation part of FIG. It is a flowchart explaining the reproduction speed calculation process of the reproduction speed calculation part of FIG. FIG. 13 is a block diagram illustrating another configuration example of the reproduction speed calculation unit in FIG. 12. 16 is a flowchart for explaining a reproduction speed calculation process of a reproduction speed calculation unit in FIG. 15. FIG. 38 is a block diagram illustrating a configuration example of another embodiment of a playback device. FIG. 18 is a block diagram illustrating a configuration example of a reproduction processing unit in FIG. 17. It is a block diagram which shows the structural example of the display area setting part of FIG. FIG. 18 is a flowchart for describing a reproduction process for a plurality of streams in the reproduction apparatus of FIG. 17. FIG. It is a figure explaining the relationship between the amount per incidental information and a display area magnification. It is a figure explaining the example of a display of a video output part. FIG. 38 is a block diagram illustrating another configuration example of the playback device. FIG. 24 is a flowchart for describing a reproduction process of a plurality of streams of the reproduction apparatus of FIG. It is a figure explaining the other example of a display of a video output part. [Fig. 38] Fig. 38 is a block diagram illustrating still another configuration example of the playback device. It is a block diagram which shows the structural example of the volume setting part of FIG. FIG. 27 is a flowchart for describing a reproduction process of a plurality of streams of the reproduction apparatus of FIG. 26. FIG. It is a figure explaining the relationship between the amount per minute of accompanying information, and a volume level magnification. It is a figure explaining the example of the audio | voice output of an audio | voice output part. It is a block diagram which shows the structural example of the hardware of the computer which functions as a reproducing | regenerating apparatus with which this invention is applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 Playback apparatus, 42 Audio | voice decoding part, 43 Video | video decoding part, 44 Accompanying information decoding part, 46 Accompanying information per minute measurement part, 47 Playback speed calculation part, 48 Playback control part, 71 Playback speed adjustment amount calculation part, 72 Playback speed Adjustment amount management unit, 73 Inverse ratio calculation unit, 141 Playback repetition number calculation unit, 211 Playback device, 242 Communication unit, 341 Operation input unit, 342 Playback speed calculation unit, 371 Inverse ratio calculation unit, 372 Playback speed adjustment amount management unit , 531 playback device, 541-1 to 541-4 playback processing unit, 542 display region setting unit, 543 video composition unit, 551 display region magnification calculation unit, 552 display region magnification management unit, 561 playback device, 571 stacking order setting unit , 631 Playback device 641 Volume setting unit, 642 Speech synthesis unit, 651 Volume level magnification Out section, 652 volume level magnification management unit

Claims (14)

Reproducing means for reproducing the second data together with the first data;
Measuring means for measuring a unit time data amount which is a data amount reproduced per unit time of the second data;
Information processing comprising: a reproduction control means for controlling reproduction of the first data and the second data by the reproduction means based on the unit time data amount of the second data measured by the measurement means apparatus. Replay speed calculating means for calculating the replay speed of the first data and the second data based on the unit time data amount of the second data;
The information processing apparatus according to claim 1, wherein the reproduction control unit controls reproduction of the first data and the second data by the reproduction unit at the reproduction speed calculated by the reproduction speed calculation unit. . The first data is main data;
The information processing apparatus according to claim 1, wherein the second data is sub-data accompanying the main data. The first data is video data;
The information processing apparatus according to claim 1, wherein the second data is character data attached to the video data. Storage means for storing correspondence information that associates the relationship between the unit time data amount of the second data and the reproduction speed;
The reproduction speed calculation means calculates the reproduction speed based on the correspondence information stored in the storage means and the unit time data amount of the second data measured by the measurement means. Item 3. The information processing device according to Item 2. The information processing apparatus according to claim 5, wherein the correspondence information indicates a relationship in which the reproduction speed is inversely proportional to the unit time data amount of the second data. An input means for inputting an instruction to adjust the relationship indicated by the correspondence information;
The information processing apparatus according to claim 5, wherein the reproduction speed calculation unit calculates the reproduction speed based on the correspondence information instructed to be adjusted and the unit time data amount of the second data. Repetitive number setting means for setting a repetitive number of repetitions of reproduction of a predetermined section of the first data based on the unit time data amount of the second data;
The reproduction control means controls reproduction of the predetermined section of the first data and the second data for the number of repetitions at the reproduction speed calculated by the reproduction speed calculation means. 2. The information processing apparatus according to 2. The information processing apparatus according to claim 2, wherein the first data is recorded on a predetermined recording medium together with the second data. The first data and / or the second data are stored in a predetermined server on the network,
The information processing apparatus according to claim 2, further comprising a communication unit configured to acquire both or one of the first data and the second data through communication with the predetermined server. Based on the unit time data amount of each of the second data measured by a plurality of the measuring means, each of the first data accompanied by each of the second data by the plurality of reproduction control means. Display area setting means for setting the size of the display area of each video displayed based on the reproduction control;
And further comprising a combining unit that generates a combined image obtained by combining each of the images having the size of the display region set by the display region setting unit.
The information processing apparatus according to claim 4, wherein the reproduction unit reproduces the synthesized video. Based on the unit time data amount of each of the second data measured by a plurality of the measuring means, each of the first data accompanied by each of the second data by the plurality of reproduction control means. A stacking order setting means for setting a stacking order of each of the images displayed based on the reproduction control;
And further comprising a composing means for generating a composite image obtained by composing each of the images in the overlap order set by the overlap order setting means,
The information processing apparatus according to claim 4, wherein the reproduction unit reproduces the synthesized video. Based on the unit time data amount of each of the second data measured by a plurality of the measuring means, each of the first data accompanied by each of the second data by the plurality of reproduction control means. Volume setting means for setting the volume level of the audio data of the audio output corresponding to each of the images displayed based on the reproduction control;
The information processing apparatus according to claim 4, further comprising: synthesis means for generating synthesized voice data obtained by synthesizing each of the voice data at the volume level set by the volume setting means. A reproduction step of reproducing the second data together with the first data;
A measurement step of measuring a unit time data amount that is a data amount reproduced per unit time of the second data;
A reproduction control step for controlling the reproduction of the first data and the second data in the processing of the reproduction step based on the unit time data amount of the second data measured by the measurement means. Information processing method.

Images