JP2006155299A - Information processor, information processing program and program recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To execute a plurality of information processing operations while synchronizing at mutually designated times only by describing an easy scenario. <P>SOLUTION: An operation execution part 3 acquires and executes a next operation on the basis of a scenario read from a scenario storage part 2, performs "queuing operation" if the operation is ended, but the operation of the other is not ended, performs "hastening operation" if the operation is not ended but the operation of the other is ended, and performs "general operation" if the operation is not ended, nor is the operation of the other. Thus, change in operation speed or queuing can be performed in the course of operation by performing the synchronization in the course of operation. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information processing apparatus, an information processing program, and a program recording medium that function as a multimedia user interface using multimedia such as computer graphics, speech synthesis, and music playback.

  As computers become more popular, more natural user interface technologies are needed. For example, in recent years, user interfaces using a plurality of different media, such as agent technology using a combination of animation and speech synthesis by CG (computer graphics), have been studied.

  Usually, in order to realize such a complicated user interface, processing such as CG and speech synthesis is often divided into independent modules and executed in parallel. The module mentioned here is not only a plurality of functional modules in a single program, but also a plurality of programs that are executed simultaneously in a single computer or a program that is distributedly executed by a plurality of computers connected via a network. Also good. Alternatively, it may be considered not a computer graphic but a robot or electronic device that performs a physical operation. Further, the plurality of modules are not necessarily different types such as an image and a voice, and may be the same module such as a voice synthesis and a voice synthesis.

  When processing such as CG and speech synthesis is divided into such a plurality of modules, it is necessary to synchronize the modules. In addition, for example, a technique for synchronizing a plurality of media has been studied, such as a lip sync technique in which the mouth of an animation character (agent) is moved naturally in accordance with speech synthesis. However, the lip sync technology unilaterally adjusts the image to the sound, and the movement stops around the mouth, so the agent walks and indicates the direction of the whole body. There is no synchronization with speech synthesis.

  As a technique for synchronizing operations other than the mouth, for example, there is a “synthetic audio and video synchronization control device” disclosed in Japanese Patent Laid-Open No. 2003-216173 (Patent Document 1). This Patent Document 1 discloses a method for controlling the motion of an animation character by CG based on the prosody information of speech synthesis. According to this technology, it is possible to walk an agent while saying “tech, tech, tech” by speech synthesis.

  However, the synthesized audio and video synchronization control device disclosed in the above-mentioned conventional Patent Document 1 has the following problems. That is, the action of the agent expressed by the CG animation depends on how much distance the agent moves on the screen and how fast it moves. The time required for speech synthesis depends on the amount of text to be synthesized and the speech rate. Then, it is necessary to adjust the operation time of the agent and the speech synthesis time so as to be synchronized. However, the agent's operation time and speech synthesis time can be adjusted to some extent, but there are limits to the range that can be adjusted in order to achieve natural operation and natural speech.

  Disregarding this, forcibly synchronizing to any one as in the case of the synthesized audio and video synchronization control device disclosed in Patent Document 1 causes unnaturalness. For example, in a certain scene, if the text for speech synthesis is short relative to the time required for the agent's operation, the next scene will start before the operation is completed, The operation becomes discontinuous, or the operation becomes unnaturally faster by forcibly adjusting it within the utterance time. On the other hand, if voice synthesis is combined with the operation side, if the voice synthesis is performed for a while after the voice synthesis ends, or if the voice rate is slowed down to match the operation time, the voice is unnaturally slowed. It becomes.

  In order to prevent these problems, it is necessary to design with sufficient consideration when creating actions and utterance texts. However, in the case of an agent whose utterance contents and action contents change according to the situation, it is not only impractical to create a scenario while adjusting the synchronization between the utterance text and the action contents for all situations. It is impossible to cope with various situations.

  Next, a case where a plurality of speech synthesis modules are synchronized will be described. When synchronizing a plurality of speech synthesis using a conventional speech synthesis module, there are the following problems. 10 to 12 show a scenario given to the speech synthesizer and a temporal flow of actual speech output when the scenario is synthesized. Here, the target output is to realize a situation in which the speakers A and B speak at the same time and the speaker B blocks the speech of the speaker A, as shown in FIG.

  FIG. 10 shows a case where scenarios are sequentially read out using a general speech synthesizer. Since speaker A and speaker B do not utter at the same time, speaker B utters “you are fine today,” and then speaker B utters “you are fine”. There is a problem that B does not give the impression that the voice of speaker A is blocked.

  FIG. 11 shows an example of a method for dividing a scenario and bringing the speech synthesis module for speaker A and the speech synthesis module for speaker B in parallel for each part to approach the target output. The speaker “B +” in the scenario indicates that the utterance is started simultaneously without waiting for the end of the immediately preceding utterance. In the example shown in the figure, speaker A's utterance is divided into “Today is sunny, today is” and “Sunny”, and the latter half of “Sunny” is “Sorry.” Speak in parallel. Then, “Yes” of the speaker A and “Please” of the speaker B are uttered in parallel. Although this method seems to fulfill the purpose at first glance, since the sentence is divided in the middle, the intonation of the voice to be uttered becomes strange. This is because, in the speech synthesis technique, intonation is generated in a series of sentence units, and natural intonation cannot be generated in the case of sentences that are divided in the middle.

FIG. 12 shows a method in which pause (pause) time is designated in the scenario and the speech synthesis modules of the speakers A and B are simultaneously executed. Here, it is assumed that the output speech from each speech synthesis module is automatically synthesized and output from the speaker. Here, in order to designate a pause, an instruction “<PAU = 1>” or “<PAU = 3>” is used, and the pause is designated with an absolute time of 1 second or 3 seconds. Thus, the desired output result is obtained by designating a pause and simultaneously executing two speech synthesis modules. However, in this method, when the content of the text changes, it is necessary to reset the pause time once again, and it is difficult to create scenarios corresponding to various situations. For this reason, there is a problem that it is not possible to cope with a user interface such as an agent whose action and utterance text change depending on the situation.
JP 2003-216173 A

  SUMMARY OF THE INVENTION An object of the present invention is to provide an information processing apparatus, an information processing program, and a program recording medium that can execute a plurality of information processing operations in synchronization with each other at a designated timing simply by describing a simple scenario. Is to provide.

In order to solve the above problems, an information processing apparatus according to the present invention provides:
Text information storage means for storing text information for instructing the operation content of the information processing operation;
Operation execution means for reading out the text information from the text information storage means, acquiring the operation content instructed by the read text information, and executing an information processing operation according to the acquired operation content;
Synchronization detection means for detecting synchronization information sent from another device in accordance with an instruction from the operation execution means,
The operation execution means is synchronized with the information processing operation executed by the other device based on the synchronization information acquired through the synchronization detection means at a preset position on the text information. The information processing operation is performed.

  According to the above configuration, when the information processing operation is executed in synchronization with the other device based on the same text information or different text information having mutual relations, the start time and the end time of the text information are It becomes possible to synchronize with other devices. In that case, it is not necessary to divide the text information or put a break in the text information in order to synchronize the information processing apparatus and the other apparatus. Therefore, when the information processing operation is a speech synthesis operation, intonation can be generated in units of sentences based on text information instructing the utterance content, and synthesized speech with natural intonation can be generated and output. .

  Furthermore, when the information processing apparatus is a speech synthesizer and the other apparatus is a CG apparatus, one character of speech synthesis may be assigned to one step of the CG character movement with respect to the text information. This can be realized without complicated settings.

In the information processing apparatus according to one embodiment,
In the text information, a synchronization command for specifying a position to be synchronized with the information processing operation executed by the other device is described.
When the position of the currently executed information processing operation on the text information reaches the synchronization command, the operation execution unit acquires the synchronization information via the synchronization detection unit and executes the other device. It synchronizes with the information processing operation.

  According to this embodiment, the operation of the information processing apparatus can be synchronized with the operation of the other apparatus at the position of the synchronization command described in the text information, and the designer can It is possible to freely set the above-mentioned location for synchronization. Further, even in such a case, when the information processing operation is a speech synthesis operation, the synthesized speech of natural intonation is excluded by excluding the synchronization command described in the text information from the text for speech synthesis. Can be generated and output.

In the information processing apparatus according to one embodiment,
The operation execution means compares the synchronization information acquired through the synchronization detection means with the synchronization instruction, and the currently executed information processing operation is compared with the information processing operation executed by the other device. Information processing operation performed by the other device by determining whether it is ahead, and performing a waiting operation to wait for the end of the information processing operation performed by the other device if it is ahead To synchronize with.

  According to this embodiment, it is determined whether the information processing operation of the information processing apparatus precedes the information processing operation of the other apparatus at the position of the synchronization command described in the text information. The information processing operation corresponding to the determination result is executed to synchronize with the information processing operation of the other device. Therefore, the information processing operations can be synchronized in an extremely natural state.

In the information processing apparatus according to one embodiment,
The operation execution means compares the synchronization information acquired through the synchronization detection means with the synchronization instruction, and the currently executed information processing operation is compared with the information processing operation executed by the other device. It is determined whether or not it is delayed, and if it is delayed, the information processing operation currently being executed is performed in a hurry to synchronize with the information processing operation executed by the other device. It has become.

  According to this embodiment, it is determined whether or not the information processing operation of the information processing apparatus is delayed with respect to the information processing operation of the other apparatus at the position of the synchronization command described in the text information. The information processing operation corresponding to the determination result is executed to synchronize with the information processing operation of the other device. Therefore, the information processing operations can be synchronized in an extremely natural state.

In the information processing apparatus according to one embodiment,
The operation execution means compares the synchronization information acquired through the synchronization detection means with the synchronization instruction, and the currently executed information processing operation is compared with the information processing operation executed by the other device. Judge whether it is ahead or behind, and if it is ahead, waits for the end of the information processing operation being executed by the other device, and if it is behind, it is currently running The information processing operation is performed in a hurry to synchronize with the information processing operation executed by the other device.

  According to this embodiment, it is determined whether the information processing operation of the information processing apparatus precedes or delays the information processing operation of the other apparatus at the position of the synchronization command described in the text information. Then, the information processing operation corresponding to the determination result is executed, and the information processing operation of the other device is synchronized. Therefore, the information processing operations can be synchronized in an extremely natural state.

In the information processing apparatus according to one embodiment,
The synchronization information obtained through the synchronization detection means includes information representing the predicted time to the next synchronization position,
The operation execution means determines whether the currently executed information processing operation is ahead of or behind the information processing operation executed by the other device. The degree of the preceding or delay is obtained based on the information representing the predicted time to the synchronization position, and the operation speed of the currently executed information processing operation is continuously switched according to this degree.

  According to this embodiment, the operation speed of the information processing operation that is currently being executed is continuously set according to the degree of advance or delay of the information processing operation that is currently being executed with respect to the information processing operation of the other device. Since switching is performed, synchronization with the information processing operation of the other apparatus can be achieved in a more natural manner.

In the information processing apparatus according to one embodiment,
The text information includes the position to be synchronized, and the information processing operation and delay to be executed when the information processing operation currently being executed precedes the information processing operation executed by the other device. A synchronization instruction that specifies at least one of the information processing operations to be executed is described.
When the position of the currently executed information processing operation on the text information reaches the synchronization command, the operation execution unit acquires the synchronization information via the synchronization detection unit, and the acquired synchronization information From the comparison with the synchronization command, it is determined whether the information processing operation currently being executed is ahead or behind the information processing operation executed by the other device, and the synchronization is performed according to the determination result. By performing the information processing operation specified by the instruction, the information processing operation executed by the other device is synchronized.

  According to this embodiment, the information processing operation that is executed when the information processing operation that is currently being executed precedes the information processing operation of the other device, and is delayed by the synchronization instruction. Since at least one of the information processing operations to be executed in advance is designated in advance, various operations can be set according to the scene to be synchronized. For example, if the voice synthesizer is in a hurry, you can stop the utterance midway, move the utterance speed up to the end, or move to the next scene while saying "Skip". It is possible to set an operation according to. Furthermore, in the case of the waiting operation, it is possible to switch between the operation of waiting while saying “U” and the operation of waiting while saying “So-”, so that a more natural expression is possible.

In the information processing apparatus according to one embodiment,
The text information includes a part that indicates an operation content of an information processing operation to be executed when the information processing operation currently being executed is synchronized with the information processing operation executed by the other device, and the currently executed information. At least one of the operation content of the information processing operation executed when the information processing operation precedes the information processing operation executed by the other device and the operation content of the information processing operation executed when the information processing operation is delayed. And a synchronization command that specifies the position to be synchronized is added.
When the position of the currently executed information processing operation on the text information reaches the synchronization command, the operation execution unit acquires the synchronization information via the synchronization detection unit, and the acquired synchronization information From the comparison with the synchronization command, it is determined whether the currently executed information processing operation is ahead or behind the information processing operation executed by the other device, and is synchronized with the text information. By performing the information processing operation instructed in the portion corresponding to the determination result, the information processing operation executed by the other device is synchronized.

  According to this embodiment, the information processing operation to be executed at least one of the case where the information processing operation currently being executed is ahead of or behind the information processing operation of the other device. Since the operation content is described as the text information, the designer can specify the operation content of the information processing operation to be executed when the currently executed information processing operation is synchronized with the information processing operation of the other device. The setting can be made without worrying about whether the information processing operation becomes longer or shorter than the information processing operation of other devices.

In the information processing apparatus according to one embodiment,
There are a plurality of the operation execution means, and each operation execution means sends the synchronization information to the synchronization detection means.
The synchronization detection means receives an instruction from each of the operation execution means and detects synchronization information sent from at least one of the operation execution means.

  According to this embodiment, since there are a plurality of the above-mentioned action execution means, for example, each action execution means includes a voice synthesis means for speaker A, a voice synthesis means for speaker B, and speakers A and B. It is possible to synchronize information processing operations with each other.

In the information processing apparatus according to one embodiment,
The information processed by the operation execution means is any one of image information, audio information, music information, device control information, and robot control information.

  According to this embodiment, the image information processing apparatus, the sound information processing apparatus, the music information processing apparatus, the equipment control information processing apparatus, and the robot control information processing apparatus can be described by simply describing simple text information. An information processing operation can be performed while synchronizing at a specified timing.

  For example, in the case of an animation synthesizer, it is possible to synchronize the actions of agents displayed on the screen. In the case of a speech synthesizer, the utterance content can be synchronized. Further, in the case of an audio playback device, it is possible to synchronize control such as playback start / stop, fade-in, and fade-out. In the case of a robot control device, it is possible to synchronize the timing of the physical operation of the robot. Further, in the case of a device control apparatus, it is possible to synchronize predetermined operations and setting changes.

The information processing program of the present invention is
Computer
The information processing apparatus according to the present invention is characterized by functioning as text information storage means, operation execution means, and synchronization detection means.

  According to the above configuration, when the information processing operation is performed while synchronizing with the other device based on the text information, the start time and the end time of the text information can be synchronized with the other device. become. In that case, it is not necessary to divide the text information or put a break in the text information in order to synchronize the information processing apparatus and the other apparatus.

The program recording medium of the present invention is
The information processing program of the present invention is recorded.

  According to the above configuration, when the information processing operation is performed while synchronizing with the other device based on the text information, the start time and the end time of the text information can be synchronized with the other device. become. In that case, it is not necessary to divide the text information or put a break in the text information in order to synchronize the information processing apparatus and the other apparatus.

  As is clear from the above, according to the present invention, the information executed by the other device based on the synchronization information acquired through the synchronization detection means at the preset position on the text information by the action execution means. Since the information processing operation is executed while synchronizing with the processing operation, in order to synchronize the information processing device with the other device, the text information is divided or a break is put in the text information. Without this, the start time and end time of the text information can be synchronized with the other device. Therefore, when the information processing operation is a speech synthesis operation, intonation can be generated in units of sentences based on text information instructing the utterance content, and synthesized speech with natural intonation can be generated and output. .

  In other words, it is possible to realize an information processing apparatus that executes a plurality of information processing operations while synchronizing with each other at a designated timing only by describing simple text information.

  Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments.

First Embodiment FIG. 1 is a block diagram illustrating a configuration of an information processing apparatus according to the present embodiment. The information processing apparatus 1 includes a scenario storage unit 2 that stores a scenario, an operation execution unit 3 that executes an information processing operation according to the scenario, an output unit 4 that outputs a result of the operation execution, and a request from the operation execution unit 3 In response to this, the synchronization information is exchanged with other information processing apparatuses to include the synchronization detection unit 5 that detects the synchronization information of the other information processing apparatuses. Here, the scenario is text information indicating the operation content of an information processing operation (hereinafter simply referred to as an operation) executed by the operation execution unit 3.

  Considering the case where the scenario is stored in the scenario storage unit 2 in advance, the means for inputting the scenario is not particularly described in FIG. However, since the scenario is generally given from the outside, a scenario input means for inputting the scenario may be provided separately. Further, when the information processing apparatus 1 is an animation composition apparatus, the output unit 4 corresponds to a graphics drawing driver, a D / A conversion apparatus, a display apparatus, and the like. Or, in the case of a voice synthesizer, a voice output driver, a D / A converter, an amplifier, a speaker, and the like correspond to this. Further, the synchronization detection unit 5 detects and acquires synchronization information such as whether processing by another information processing apparatus has been completed or where it has proceeded. Moreover, the process by the operation | movement execution part 3 is demonstrated below.

  FIG. 2 is a flowchart of the information processing operation executed by the operation execution unit 3. Hereinafter, the information processing operation for controlling the operation of the information processing apparatus 1 according to the scenario stored in the scenario storage unit 2 in synchronization with the processing operation by another information processing apparatus will be specifically described with reference to FIG. To do.

  In step S1, a scenario is retrieved from the scenario storage unit 2. In step S2, it is determined whether or not the scenario is completed by determining whether or not there is an unprocessed scenario in the scenario storage unit 2 based on the search result. As a result, if the scenario is not completed (that is, if there is an unprocessed scenario), the process proceeds to step S3. If the scenario is completed, the information processing operation is terminated. In step S3, an unprocessed scenario in the scenario storage unit 2 is read, and the next operation is acquired based on the read scenario. In step S4, it is determined whether or not the operation of the information processing apparatus 1 acquired in step S3 is completed. As a result, if completed, the process proceeds to step S5, and if not completed, the process proceeds to step S8.

  In step S5, the synchronization information indicating the progress of the operation related to the information processing apparatus 1 is sent to another information processing apparatus via the synchronization detection section 5, and the synchronization detection section 5 relates to the other information processing apparatus. Acquire synchronization information indicating the progress of the operation. In step S6, based on the acquired synchronization information of the other information processing apparatus, it is determined whether or not the operation of the other information processing apparatus has ended. As a result, if completed, the process returns to step S1 and shifts to a process relating to the next scenario. On the other hand, if not completed, the process proceeds to step S7. In step S7, the operation of the information processing operation is stopped for a predetermined first predetermined time, and a “waiting operation” is executed to wait for the operation of the other information processing apparatus to end. Thereafter, the process returns to step S4, and it is determined whether or not the operation of the information processing apparatus 1 based on the scenario read in step S3 is completed. However, in this case, since the operation of the information processing apparatus 1 based on the scenario is completed, the process proceeds to step S6 again. Thus, when it is determined in step S6 that the operation of the other information processing apparatus is finished, the process returns to step S1 and shifts to the process relating to the next scenario.

  In step S8, synchronization information indicating the progress of the operation related to the information processing apparatus 1 is sent to another information processing apparatus via the synchronization detection section 5, and the synchronization detection section 5 relates to the other information processing apparatus. Acquire synchronization information indicating the progress of the operation. In step S9, based on the acquired synchronization information of the other information processing apparatus, it is determined whether or not the operation of the other information processing apparatus has ended. As a result, if completed, the process proceeds to step S10, and if not completed, the process proceeds to step S11.

  In step S10, although the operation of the information processing apparatus 1 has not ended although the operation of the other information processing apparatus has ended, the operation speed of the information processing apparatus 1 is increased by a predetermined speed. The “rushing action” is executed for a predetermined second predetermined time. Thereafter, the process returns to step S4, and it is determined whether or not the operation of the information processing apparatus 1 based on the scenario read in step S3 is completed. In step S11, neither the operation of the information processing apparatus 1 nor the operation of the other information processing apparatus is completed. Therefore, the operation of the information processing apparatus 1 based on the scenario read in step S3 (hereinafter referred to as “normal operation”). Is executed for a predetermined third predetermined time. Thereafter, the process returns to step S4, and it is determined whether or not the operation of the information processing apparatus 1 based on the scenario read in step S3 is completed.

  Thus, it is determined that the operation of the information processing apparatus 1 is completed in step S4, the operation of the other information processing apparatus is determined to be completed in step S6, and is stored in the scenario storage unit 2 in step S2. When it is determined that all the operations related to the scenario are completed, the information processing operation is terminated.

  Here, in the waiting operation in step S7, the rushing operation in step S10, or the normal operation in step S11, the operation acquired based on the unprocessed scenario in step S3 is executed in one operation. Instead, as described above, the processing is performed by dividing into fine units of “first, second, third predetermined time”. In this case, for example, if the motion acquired based on the scenario is an animation motion that walks to the target position, the delimiter unit is a unit obtained by decomposing a walking motion such as a motion that advances one step. Or, for example, in the case of speech synthesis, instead of synthesizing one sentence acquired based on an unprocessed scenario at a time, it is divided into output speech of one character or several milliseconds to several hundred milliseconds. Are synthesized in units.

  By doing this, the progress of the operation based on the above scenario can be transmitted to other information processing devices, and the operation can be synchronized in the middle of the operation to change the operation speed or wait in the middle of the operation. It becomes possible.

  In the flowchart shown in FIG. 2, since the timing to synchronize with the other information processing apparatus is the timing at which each operation based on the read scenario is completed, “speaker A” shown in FIG. It is not possible to cope with the case where synchronization is established at a specific position set in the middle of each operation of the scenario, as in the case of “speaking of the utterance of utterance by the utterance of speaker B”. In such a case, the operation execution part 3 should just control operation | movement of this information processing apparatus 1 according to the flowchart of information processing operation shown in FIG.

  In step S21, a scenario is retrieved from the scenario storage unit 2. In step S22, it is determined whether or not the scenario is completed by determining whether or not there is an unprocessed scenario in the scenario storage unit 2 based on the search result. As a result, if the scenario is not completed (that is, if there is an unprocessed scenario), the process proceeds to step S23, and if the scenario is completed, the information processing operation is terminated. In step S23, an unprocessed scenario in the scenario storage unit 2 is read, and the next operation is acquired based on the read scenario. In step S24, it is determined whether or not the operation of the information processing apparatus 1 acquired in step S23 is completed. As a result, if completed, the process returns to step S21 and shifts to processing for the next scenario. On the other hand, if not completed, the process proceeds to step S25.

  In step S25, it is determined whether or not the synchronization position Tn described in the currently processed scenario has been reached. As a result, when it has reached, the process proceeds to step S26, and when it has not reached, the process proceeds to step S27. In step S26, the synchronization position Tn is sent to the other information processing apparatus via the synchronization detector 5. In step S27, the synchronization position Tm currently reached by another information processing apparatus is acquired via the synchronization detection unit 5. Here, “Tn, Tm” indicating the synchronization position is a marker that is described in the scenario A and scenario B and includes a numerical value that increases in the order of appearance.

  In step S28, the magnitudes of the numerical values included in the synchronization position Tn representing the position reached by the information processing apparatus 1 in the scenario to be processed and the synchronization position Tm representing the position reached by another information processing apparatus are compared. The However, if the synchronization position has not yet been reached, the above numerical value is set to “0”. As a result, if Tn> Tm (that is, the information processing apparatus 1 is ahead of other information processing apparatuses), the process proceeds to step S29, and if not, the process proceeds to step S30. In step S29, the operation of the information processing operation is stopped for a predetermined first predetermined time, and a “waiting operation” is executed to wait for the progress of the operation of the other information processing apparatus. After that, the process returns to step S24 to determine whether or not the operation of the information processing apparatus 1 based on the scenario read in step S23 is completed. However, in this case, since the operation of the information processing apparatus 1 based on the scenario is not completed and has reached the synchronization position Tn, the steps S25 to S28 are executed again. Then, the operation of the other information processing apparatus proceeds to update the synchronization position Tm. If it is determined in step S28 that Tn> Tm is not satisfied, the process proceeds to step S30.

  In step S30, the numerical values included in the synchronization position Tn on the information processing apparatus 1 side and the synchronization position Tm on the other information processing apparatus side are compared. As a result, if Tn <Tm (that is, the information processing apparatus 1 is behind the other information processing apparatuses), the process proceeds to step S31; otherwise, the process proceeds to step S32. In step S31, the “rushing operation” in which the operation speed of the information processing apparatus 1 is increased by a predetermined speed is executed for a predetermined second predetermined time. Thereafter, the process returns to step S24 to determine whether or not the operation of the information processing apparatus 1 based on the scenario read in step S23 is completed. However, in this case, since the operation of the information processing apparatus 1 based on the scenario is not completed, steps S25 to S28 and step S30 are further executed. Then, the operation of the information processing apparatus 1 proceeds and the synchronization position Tn is updated. If it is determined in step S28 that Tn <Tm is not satisfied (that is, Tn = Tm), the process proceeds to step S32. In step S32, the “normal operation” in which the operation acquired in step S23 is executed is executed for a predetermined third predetermined time. After that, the process returns to step S24 to determine whether or not the operation of the information processing apparatus 1 acquired in step S23 is completed.

  Thus, when it is determined in step S24 that the operation of the information processing apparatus 1 has been completed, and in step S22 it is determined that the scenario is complete because there is no unprocessed scenario stored in the scenario storage unit 2. The information processing operation is terminated.

  In the information processing operation shown in FIG. 3, three cases of “waiting operation”, “rushing operation”, and “normal operation” are shown as examples of operations on the information processing apparatus 1 side. It is not limited to. For example, the synchronization position Tn transmitted to the other information processing apparatus in step S26 in the flowchart shown in FIG. 3 and the synchronization position Tm from the other information processing apparatus acquired in step S27 are described in the above “synchronization position. It is possible to add not only the discrete synchronization information indicating whether or not there is, but also a value that predicts how much time is left until the next synchronization position. The information processing apparatus 1 that has received the synchronization information switches the degree of rushing in multiple steps, adds an operation to slow down, or adds the normal operation based on the value of the expected time to the next synchronization position. It is also possible to continuously switch the speed (including 0 = stop).

  Hereinafter, the case where the operation execution unit 3 controls the operation of the information processing apparatus 1 according to the flowchart of the information processing operation shown in FIG.

(Example 1)
4 shows a scenario (FIG. 4 (a)) and output for obtaining the output result of the speech synthesizer by speakers A and B similar to FIG. 11 according to the flowchart of the information processing operation shown in FIG. An example of synthesized speech (FIG. 4B) is shown. That is, the information processing apparatus shown in FIG. 1 is based on the scenario A “Today is sunny, today is <# 1> sunny”. <# 2> Yes. ” Is a sunny day Today is a sunny day, based on a speech synthesizer that synthesizes with the voice of speaker A and scenario B "<# 1> Already fine. Please silently <# 2>." It is either one of the speech synthesizer that synthesizes “It's fine. The other speech synthesizer is the other information processing apparatus in the explanatory text of the flowchart shown in FIG.

  In the following description, it is assumed that the speech synthesizer that synthesizes the voice of the speaker A is the information processing apparatus 1. In this case, the other information processing apparatus which is a voice synthesizer that synthesizes the voice of the speaker B also has the same configuration as the information processing apparatus 1. Although not described in the flowchart shown in FIG. 3, it is assumed that “Tn = 0”, that is, “<# 0>” is transmitted first.

  First, in step S23 of the flowchart shown in FIG. 3, the unprocessed scenario A “Today is fine, today is <# 1> fine weather. <# 2> Yes.” Is read from the scenario storage unit 2. Then, based on this scenario, the operation content for generating the synthesized speech “Today is sunny and today is sunny” is acquired and executed. Then, the synthesized speech up to the position <# 1> “Today is a fine day today” is output, and in step S26, another information processing device that synthesizes the voice of the speaker B via the synchronization detection unit 5 In response, the synchronization position Tn “<# 1>” is transmitted.

  Meanwhile, in the other information processing apparatuses described above, the synchronization position Tm “<# 1>” is present at the head of the read scenario B “<# 1> is already fine. While the position Tm “<# 1>” is sent to the information processing apparatus 1, the synchronization position Tn “<# 0>” from the information processing apparatus 1 is received and the synchronization position Tm and the synchronization position Tn are compared. . Then, since the numerical value “1” included in the synchronization position Tm “<# 1>” is larger than the numerical value “0” included in the synchronization position Tn, a “waiting operation” for stopping speech synthesis is performed. Yes. When the synchronization position Tn “<# 1>” is received from the information processing apparatus 1, Tm “<# 1>” = Tn “<# 1>”, so “already fine” Start generating and outputting a synthesized speech “silently”.

  Meanwhile, the information processing apparatus 1 receives the synchronization position Tm “<# 1>” from the other information processing apparatus in the step S27, and receives the synchronization position Tn “<# 1>” and the synchronization position Tm “<< 1”. > ". Since Tn “<# 1>” = Tm “<# 1>”, the synthesized speech “sunny sky” is continuously generated and output up to the position of <# 2>, and is output to other information processing apparatuses. The synchronization position Tn “<# 2>” is transmitted. After that, the synchronization position Tn “<# 2>” is compared with the synchronization position Tm “<# 1>”. Since Tn “<# 2>”> Tm “<# 1>”, the speech synthesis is stopped. Then, shift to “waiting operation”.

  In the meantime, the other information processing apparatus outputs the synthesized speech “This is fine, silent” until it reaches the synchronization position Tm “<# 2>”, and then outputs the synchronization position Tm “< # 2> ". Then, since the synchronization position Tm “<# 2>” = the synchronization position Tn “<# 2>”, the synthesized speech “Please” is continuously generated and output to the end.

  Meanwhile, the information processing apparatus 1 receives the synchronization position Tm “<# 2>” from the other information processing apparatus, and the synchronization position Tn “<# 2>” = the synchronization position Tm “<# 2>”. Therefore, the synthesized speech “Yes” is generated and output.

  In this way, it is possible to realize a situation in which the speakers A and B speak simultaneously and the speaker B blocks the speech of the speaker A by voice. In that case, both the information processing apparatus 1 and the other information processing apparatus generate intonation in units of sentences regardless of the presence or absence of the synchronization positions Tn and Tm inserted in the scenario. Also, the speech is synthesized without interruption at the synchronization positions Tn and Tm. Therefore, unlike the example shown in FIG. 11, it is possible to generate and output a synthesized voice with natural intonation. Similarly, synchronization is automatically established at the positions of the synchronization positions Tn and Tm just by inserting the synchronization positions Tn and Tm in the scenario of the speaker A and the scenario of the speaker B. Therefore, even if the utterance speeds of speakers A and B are set as different speech synthesis parameters, and even if the utterance speed is changed, a natural conversation is performed without changing the scenario. It is possible to output such synthesized speech.

  In the first embodiment, the “waiting operation” is not specified in both scenario A and scenario B, but a case where an operation of waiting without synthesizing anything is set. Further, the “rushing action” is not particularly set since it is not necessary in the scenario of this embodiment.

(Example 2 and Example 3)
The present embodiment is a case where the operation contents of “waiting operation” and “rush operation” are designated, and an example in which the operation is designated in the synchronization position will be described. Hereinafter, the synchronization position including the operation designation as described above is referred to as a “synchronization command”.

  In FIGS. 5 and 6, the operation is designated by the synchronization command <# 2> in the scenario of the speaker A. Here, in <# 2: eh / STOP>, “waiting operation” is designated before “/”, and “rushing action” is designated after “/”. It is specified that the utterance is interrupted (STOP) as an urgent action.

  In the second embodiment shown in FIG. 5, as shown in FIG. 5 (a), the contents of the scenario are the same as those in FIG. 4 (a). Therefore, before the second utterance “Yes” of the speaker A, the “waiting operation” is entered, and the synthesized voice “e” is inserted.

  On the other hand, in Example 3 shown in FIG. 6, as shown in FIG. 6 (a), the content of scenario A is longer than that in FIG. 4 (a). Therefore, the speaker B reaches the position of <# 2> first. Therefore, when the information processing apparatus on the speaker A side receives the synchronization position Tm “<# 2>” from the information processing apparatus on the speaker B side, the synchronization position Tn “<# 1>” and the synchronization position Tm “ <# 2> ”is switched to“ Hurry ”, and it is in the middle of the utterance“ Today is sunny ”, but it is interrupted by“ Today is sunny ”. Is synthesized.

  In this case, the position where the speaker A interrupts the speech is automatically adjusted depending on the difference in speech speed between the speaker A and the speaker B. Therefore, when creating a scenario, even if the actual speech rate between the speaker A and the speaker B cannot be predicted, a natural conversation can be realized by the synthesized speech.

  In addition, by specifying the operation contents of “waiting operation” and “rush operation” with the synchronization command <# 2: e / STOP> as described above, a sentence is added to or deleted from scenario A, and scenario B Even if the relative length with respect to becomes longer or shorter, “E” is automatically uttered and “waiting operation” is performed, or utterance is interrupted and “hurrying operation” is performed. Therefore, the scenario creator can change the scenario without worrying about the setting position of the synchronization position.

  In the second and third embodiments, the operation contents of both the “waiting operation” and the “rushing operation” are set, but only one of them can be set.

  In the first to third embodiments, the format of the synchronous instruction is “<#n>: n is a positive integer”. However, the format is not limited to this format. In short, any format can be used as long as it can be easily identified from the contents of the scenario and the order of appearance in the scenario can be determined.

  Further, in the first to third embodiments, the case where the information processing apparatus in this embodiment is two speech synthesizers of the speaker A and the speaker B has been described as an example. However, it is also conceivable to add the animation image generation apparatuses of speaker A and speaker B to the two speech synthesizers and synchronize each other. In this case, the synchronization detection unit 5 in FIG. 1 needs to detect synchronization for two or more information processing devices as other information processing devices. In that case, it becomes a problem which of the two or more information processing devices that detect synchronization is to preferentially detect the operation of the information processing device. In general, it is considered appropriate to match the information processing device with the slowest operation. However, this is not limited depending on the purpose, and it is up to the information processing device designer or scenario creator to select the information processing device. To be done, it is not decided at all. In that case, the name of the information processing device to be synchronized is described in the scenario, and the synchronization detection unit 5 can also acquire the synchronization position and synchronization command of the information processing device described in the scenario. .

Second Embodiment FIG. 7 is a block diagram showing a configuration of the information processing apparatus according to the present embodiment. The information processing apparatus 11 includes three processing apparatuses from a first processing apparatus 13 to a third processing apparatus 15, and each processing apparatus 11 to 13 receives a scenario input unit 16 from a scenario storage unit 12. Scenario A for the first processing device 13, scenario B for the second processing device 14, and scenario C for the third processing device 15 are read and sent individually. And each processing apparatus 13-15 has the function similar to the operation execution part 3 in the information processing apparatus of 1st Embodiment fundamentally, and operate | moves similarly to the operation execution part 3. FIG. In the present embodiment, another scenario is stored in the scenario storage unit 12 for each of the processing devices 13 to 15 and an individual scenario is given to each of the processing devices 13 to 15. However, the present invention is not limited to this. The scenario storage unit 12 stores a scenario for each of the processing devices 13 to 15 as one data, and gives the same scenario to each of the processing devices 13 to 15. It is also possible to select and execute only the scenario for the own processing apparatus within 13 to 15.

  The synchronization transmission unit 17 receives the synchronization information of the processing devices 13 to 15 and transmits them to the processing devices 13 to 15 that are required in response to the request. Here, the synchronization transmission unit 17 does not necessarily have an active function, and is a memory that is shared among the processing devices 13 to 15 and temporarily stores synchronization information of the processing devices 13 to 15. There is no problem. In this case, each of the processing devices 13 to 15 accesses the synchronization transmission unit (memory) 17 as necessary, and acquires synchronization information of other processing devices. In addition, when it is necessary to synchronize with other processing devices distributed to physically different devices in addition to the processing devices 13 to 15, the synchronization transmission unit 17 is connected via the communication unit 20. It communicates with the other processing apparatus and acquires / transmits the operating status. As the communication unit 20, a conventional communication module such as wired, wireless, or infrared may be used.

  The information output from each of the processing devices 13 to 15 is integrated by the first and second integrated output units 18 and 19 for each type of information such as image information and audio information and presented to the user. In the case of FIG. 7, the output from the second processing device 14 and the output from the third processing device 15 are integrated and output by the second integrated output unit 19.

  In the present embodiment, there are three processing devices from the first processing device 13 to the third processing device 15, of which the outputs from the second and third processing devices 14 and 15 are the second integrated output. The unit 19 integrates them. However, the number of processing devices, the output from which processing devices are integrated, the number of processing devices whose outputs are integrated, etc., are appropriately changed according to the structure of the multimedia user interface to be constructed. There is no problem.

  Hereinafter, the operation of the information processing apparatus having the above configuration will be specifically described according to the following embodiments.

Example 4
The present embodiment shows an example of an information processing apparatus that explains a computer operation using an image and a voice of an agent using an image processing apparatus and a speech synthesis processing apparatus.

  In the present embodiment, it is assumed that the first processing device 13 in FIG. 7 is an image processing device and the second processing device 14 is a speech synthesis processing device. FIG. 8A shows a display screen of the first integrated output unit 18 that outputs image information from the image processing device 13. FIG. 8B shows an image scenario related to the agent 21, and FIG. 8B shows a speech synthesis scenario related to the agent 21, which is individual data in this embodiment. The synchronization command in this embodiment is described at the beginning of each scenario such as “T0”.

  In “” in each scenario, the operation content of the corresponding processing device, the operation content of the waiting operation when the operation ends, and the operation of the urgent operation when the other processing device ends the operation first The contents are described by separating them with “/”. That is, each scenario is described as “normal operation / waiting operation / rush operation”. Of course, the scenario description method is not limited to this. Hereinafter, the operation in the present embodiment will be described along the scenarios of FIG. 8B and FIG. 8B.

  First, the agent 21 utters “first” while “instructing position A” with the synchronization command “T0”. If the action “specify position A” does not end even when the utterance “first” ends, the speech synthesis side shifts to “waiting operation” and adds “er” as a voice.

  Next, with the synchronization command “T1”, the agent 21 utters “Press the button with this icon A” while stopping the action. In this case, since the stop of the action ends first, the image side shifts to the “waiting operation” and performs the “shake a stick” action.

  Next, the agent 21 utters “drag as it is” while “moving from the position of the icon A to the position of the icon B” by the synchronization command “T2”. Here, when the action “movement from A to B” is finished first, the image side shifts to “waiting operation” and the agent 21 performs the action “mouth speech” until the end of the utterance. Express something. On the other hand, if it takes time to move from A to B and the utterance ends first, the image side shifts to “rush action” and switches to the action “run”. Say “Good, good” until the movement is finished.

  Next, in response to the synchronization command “T3”, the agent 21 silently “instructs position B”.

  Next, with the synchronization command “T4”, the agent 21 utters “Please release with this B” while stopping the action. In this case, since the stop of the action ends first, the image side shifts to the “waiting operation” and performs the action “shake a stick”.

  Thus, since the synchronization command “Tn: n is 0 and a positive integer” is added to the head of each scenario and each scenario is described as “normal operation / waiting operation / hurry operation”, it actually takes. The operation contents between the image processing apparatus and the speech synthesis processing apparatus can be automatically switched according to the processing time, and natural image information and voice information can be presented. In that case, it is not necessary to individually set the position and time of the pose of the scenario, and it is not difficult to create the scenario.

Third Embodiment FIG. 9 is a block diagram showing a configuration when the information processing apparatus according to the present embodiment is realized using a computer. The information processing device 31 is configured by connecting a CPU (Central Processing Unit) 32, a storage unit 33, an input unit 34, an output unit 35 and a program reading unit 36 to a bus 37. The program reading unit 36 reads an information processing program from an external program recording medium 38. Here, an optical disk, a magnetic disk, a flexible disk, a read-only memory, a rewritable memory, or the like is used as the program recording medium 38, and this computer functions as the information processing apparatus of the first and second embodiments. An information processing program is recorded.

  The information processing program read by the program reading unit 36 is stored in the storage unit 33. The storage unit 33 is generally composed of a magnetic disk or a rewritable memory. The CPU 32 operates as an information processing apparatus by controlling each unit in accordance with the information processing program stored in the storage unit 33. In addition, the storage unit 33 is partially used as a work area in the information processing operation performed by the CPU 32. The input unit 34 receives input from the user and instructs execution of an information processing program, switching of processing, and the like. The input unit 34 corresponds to a keyboard, a microphone, a tablet, a pointing device, and the like. The output unit 35 presents the result of information processing by the CPU 32 to the user. The output unit 35 corresponds to a display for outputting characters and images, a speaker or headphones for outputting sound, music, and the like.

  Further, the output unit 35 may output control signals for industrial and household electric devices and robots. Alternatively, the output unit 35 may output a control signal for operating another device.

  Furthermore, when information processing is performed by combining a plurality of information processing apparatuses 31 having the above configuration, it goes without saying that a communication unit for synchronization is required for each information processing apparatus.

It is a block diagram which shows the structure in the information processing apparatus of this invention. It is a flowchart of the information processing operation performed by the operation | movement execution part in FIG. 3 is a flowchart of information processing operation different from FIG. 2. It is a figure which shows the scenario memorize | stored in the scenario memory | storage part in FIG. 1, and the output synthetic speech based on it. It is a figure which shows the scenario different from FIG. 4, and the output synthetic speech based on it. It is a figure which shows the scenario different from FIG. 4 and FIG. 5, and the output synthetic speech based on it. It is a block diagram which shows the structure in the information processing apparatus different from FIG. It is a figure which shows the scenario memorize | stored in the scenario memory | storage part in FIG. 7, the output image based on it, and an output synthetic | combination audio | voice. It is a block diagram which shows the structure at the time of implement | achieving the information processing apparatus of FIG. 1 and FIG. 7 using the computer. It is a figure which shows the scenario at the time of reading a scenario using a general speech synthesis apparatus, and the time flow of synthetic speech. It is a figure which shows the scenario in the case of operating the speech synthesis module according to a speaker in parallel, and the time flow of synthetic speech. It is a figure which shows the scenario and the time flow of a synthetic | combination speech in the case of making it operate | move by designating a pause in a scenario and operating the speech synthesis module classified by speaker simultaneously.

Explanation of symbols

1, 11, 31 ... Information processing device,
2,12 ... scenario storage unit,
3 ... operation execution part,
4 ... output part,
5 ... Synchronization detector,
13 ... 1st processing apparatus,
14 ... 2nd processing apparatus,
15 ... 3rd processing apparatus,
16 ... Scenario input part,
17 ... synchronous transmission part,
18 ... 1st integrated output part,
19 ... 2nd integrated output part,
20 ... communication section,
21 ... Agent,
32 ... CPU,
33 ... storage unit,
34 ... input section,
35 ... output section,
36 ... Program reading part,
37 ... Bus,
38 ... Program recording medium.

Claims (12)

Text information storage means for storing text information for instructing the operation content of the information processing operation;
Operation execution means for reading out the text information from the text information storage means, acquiring the operation content instructed by the read text information, and executing an information processing operation according to the acquired operation content;
Synchronization detection means for detecting synchronization information sent from another device in accordance with an instruction from the operation execution means,
The operation execution means is synchronized with the information processing operation executed by the other device based on the synchronization information acquired through the synchronization detection means at a preset position on the text information. An information processing apparatus that performs the information processing operation. The information processing apparatus according to claim 1,
In the text information, a synchronization command for specifying a position to be synchronized with the information processing operation executed by the other device is described.
When the position of the currently executed information processing operation on the text information reaches the synchronization command, the operation execution unit acquires the synchronization information via the synchronization detection unit and executes the other device. An information processing apparatus characterized by being synchronized with an information processing operation. The information processing apparatus according to claim 2,
The operation execution means compares the synchronization information acquired through the synchronization detection means with the synchronization instruction, and the currently executed information processing operation is compared with the information processing operation executed by the other device. Information processing operation performed by the other device by determining whether it is ahead, and performing a waiting operation to wait for the end of the information processing operation performed by the other device if it is ahead An information processing apparatus that is adapted to be synchronized with the information processing apparatus. The information processing apparatus according to claim 2,
The operation execution means compares the synchronization information acquired through the synchronization detection means with the synchronization instruction, and the currently executed information processing operation is compared with the information processing operation executed by the other device. It is determined whether or not it is delayed, and if it is delayed, the information processing operation currently being executed is performed in a hurry to synchronize with the information processing operation executed by the other device. An information processing apparatus characterized by being configured. The information processing apparatus according to claim 2,
The operation execution means compares the synchronization information acquired through the synchronization detection means with the synchronization instruction, and the currently executed information processing operation is compared with the information processing operation executed by the other device. Judge whether it is ahead or behind, and if it is ahead, waits for the end of the information processing operation being executed by the other device, and if it is behind, it is currently running An information processing apparatus characterized in that the information processing operation is performed in a hurry to synchronize with an information processing operation performed by the other apparatus. The information processing apparatus according to claim 5,
The synchronization information obtained through the synchronization detection means includes information representing the predicted time to the next synchronization position,
The operation execution means determines whether the currently executed information processing operation is ahead of or behind the information processing operation executed by the other device. The degree of advance or delay is obtained based on information representing the predicted time to the synchronization position, and the operation speed of the currently executed information processing operation is continuously switched according to this degree. A characteristic information processing apparatus. The information processing apparatus according to claim 1,
The text information includes the position to be synchronized, and the information processing operation and delay to be executed when the information processing operation currently being executed precedes the information processing operation executed by the other device. A synchronization instruction that specifies at least one of the information processing operations to be executed is described.
When the position of the currently executed information processing operation on the text information reaches the synchronization command, the operation execution unit acquires the synchronization information via the synchronization detection unit, and the acquired synchronization information From the comparison with the synchronization command, it is determined whether the information processing operation currently being executed is ahead or behind the information processing operation executed by the other device, and the synchronization is performed according to the determination result. An information processing apparatus, wherein an information processing operation specified by an instruction is performed to synchronize with an information processing operation executed by the other apparatus. The information processing apparatus according to claim 1,
The text information includes a part that indicates an operation content of an information processing operation to be executed when the information processing operation currently being executed is synchronized with the information processing operation executed by the other device, and the currently executed information. At least one of the operation content of the information processing operation executed when the information processing operation precedes the information processing operation executed by the other device and the operation content of the information processing operation executed when the information processing operation is delayed. And a synchronization command that specifies the position to be synchronized is added.
When the position of the currently executed information processing operation on the text information reaches the synchronization command, the operation execution unit acquires the synchronization information via the synchronization detection unit, and the acquired synchronization information From the comparison with the synchronization command, it is determined whether the information processing operation currently being executed is ahead or behind the information processing operation executed by the other device. An information processing apparatus characterized in that an information processing operation designated by a corresponding part is performed to synchronize with an information processing operation executed by the other apparatus. The information processing apparatus according to claim 1,
There are a plurality of the operation execution means, and each operation execution means sends the synchronization information to the synchronization detection means.
The synchronization detection means receives an instruction from each of the operation execution means and detects synchronization information transmitted from at least one of the operation execution means. Processing equipment. The information processing apparatus according to claim 1,
The information processed by the operation execution means is any one of image information, audio information, music information, device control information, and robot control information. Computer
An information processing program that functions as a text information storage unit, an operation execution unit, and a synchronization detection unit according to claim 1.   A computer-readable program recording medium on which the information processing program according to claim 11 is recorded.

Images