JP2015108705A - Voice reproduction system, voice reproduction method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for allowing audience to feel as if a user speaks naturally by oneself even when the user is not fluent in the used language.SOLUTION: The voice reproduction system comprises a storage unit, an imaging unit, and a control unit. The storage unit previously stores voice data including a plurality of portion waveform data. The imaging unit is designed to be able to image the mouth of a user. The control unit determines that the mouth of the user is opened on the basis of an image imaged by the imaging unit and controls the timing of reproduction of the voice data so as to sequentially reproduce the portion waveform data in response to the fact that the mouth of the user is opened.

Description

  The present technology relates to an audio reproduction system, an audio reproduction method, and a program for reproducing audio.

  In recent years, opportunities to present speeches and presentations in languages other than their native language are increasing at international conferences and events. For example, when a person who makes a presentation is a Japanese person, the person may make a presentation in English or in a language of a visited country.

  In order to make presentations fluently in a language other than your native language, a great deal of effort is required to learn that language. In the case of a presenter who cannot make a fluent presentation, a method is used in which the presenter's presentation is simultaneously interpreted by the interpreter, and the interpreter's voice is played through the earphones and the voice is presented to the audience. There are many (for example, refer the following patent document 1).

JP 2007-306420 A

  However, when simultaneous interpretation is used, there is a problem that the naturalness of the presentation is lost. Therefore, there is a demand for a technology that can make the audience feel as if the user is speaking naturally even if the user is not necessarily fluent in the language used.

  In view of the circumstances as described above, the purpose of the present technology is to provide a technology that can make the audience feel as if the user is speaking naturally even if the user is not necessarily proficient in the language used. It is in.

The audio reproduction system according to the present technology includes a storage unit, an imaging unit, and a control unit.
The storage unit stores in advance audio data including a plurality of partial waveform data.
The imaging unit can image a user's mouth.
The control unit determines that a user's mouth has been opened based on an image captured by the imaging unit, and the audio is sequentially played back in response to the user's mouth being opened. Control the timing of data playback.

  In the present technology, for example, audio data used for presentation or the like is stored in advance in the storage unit. When the user's mouth opens when an announcement is actually made, the partial waveform data in the audio data is sequentially reproduced, so that the reproduction timing of the audio data coincides with the timing at which the mouth is opened. As a result, even if the user is not necessarily proficient in the language used, the audience can feel as if the user is speaking naturally.

In the audio reproduction system, the imaging unit may be capable of imaging the user's face.
In this case, the control unit may extract face information based on the captured image, and may control speech inflection in the currently reproduced audio data based on the face information.

  As described above, by controlling the inflection of the sound in the currently reproduced sound data based on the face information, it becomes possible to add an emotional expression to the currently reproduced sound data.

  In the audio reproduction system, the control unit may extract a distance between the eyes and the eyebrows as the face information, and control the size of the audio based on the distance between the eyes and the eyebrows.

  This makes it possible to add emotional expressions to the currently reproduced audio data.

  In the audio reproduction system, the control unit may extract a face orientation in the vertical direction as the face information, and control the voice height based on the face vertical orientation.

  This makes it possible to add emotional expressions to the currently reproduced audio data.

The audio reproduction system may further include a prompter unit.
In this case, the storage unit may further store text data corresponding to the voice data.
In this case, the control unit may control display of the prompter unit so that the text data is displayed on the prompter unit.

  In this audio reproduction system, when the user reads out the text data displayed on the prompter section (or may be a mouthpiece), the partial waveform data in the audio data is sequentially reproduced in accordance with the timing when the user's mouth is opened.

In the audio reproduction system, the text data may include a plurality of partial text data.
In this case, the storage unit may store the partial waveform data and the partial text data corresponding to the partial waveform data in association with each other.
In this case, the control unit controls the display of the prompter unit so that the partial text data corresponding to the currently reproduced partial waveform data is highlighted in accordance with the reproduction timing of the partial waveform data. Also good.

  In this audio reproduction system, the user can easily recognize which part of the text data is currently being reproduced by visually recognizing the highlighted partial text data.

  In the audio reproduction system, the control unit associates the partial waveform data with the partial text data corresponding to the partial waveform data, and stores the partial waveform data, the partial text data, You may perform the process which matches.

  Thereby, partial waveform data and partial text data can be matched appropriately.

  In the audio reproduction system, the storage unit may store data acquired by a narrator reading out text data as the audio data.

  In the voice reproduction system, the storage unit may store synthesized voice by voice synthesis as the voice data.

The audio reproduction method according to the present technology includes storing audio data including a plurality of partial waveform data in advance.
The user's mouth is imaged.
It is determined that the user's mouth is open based on the captured image.
The reproduction timing of the audio data is controlled so that the partial waveform data is sequentially reproduced according to the opening of the user's mouth.

A program according to the present technology is stored in an audio playback system.
Pre-storing audio data including a plurality of partial waveform data;
Imaging a user's mouth;
Determining that the user's mouth has been opened based on the captured image;
Controlling the reproduction timing of the audio data so as to sequentially reproduce the partial waveform data in response to the opening of the user's mouth.

  As described above, according to the present technology, even if the user is not necessarily fluent in the language to be used, it is possible to provide a technology that can make the audience feel as if the user is speaking naturally.

1 is a block diagram showing an electrical configuration of a sound reproduction system according to a first embodiment of the present technology. It is a figure for demonstrating operation | movement of the audio | voice reproduction | regeneration system in a preparation stage. It is a figure for demonstrating the process which calculates | requires the matching from which cost becomes the minimum by dynamic programming. It is a flowchart which shows operation | movement of the audio | voice reproduction system when a user gives a presentation. It is a schematic diagram for demonstrating operation | movement of the audio | voice reproduction system when a user makes a presentation. It is a figure showing the change of the distance between eyes and eyebrows. It is a figure showing the change of the direction in the up-down direction of a face.

  Hereinafter, embodiments according to the present technology will be described with reference to the drawings.

<First Embodiment>
[Overall Configuration of Audio Reproduction System 10 and Configuration of Each Unit]
FIG. 1 is a block diagram showing an electrical configuration of an audio reproduction system 10 according to the first embodiment of the present technology. In the description of the present embodiment, the audio reproduction system 10 will be described as a system used when, for example, a presentation such as speech or presentation is performed. In addition, the audio reproduction system 10 according to the present embodiment can be used for other uses such as theater besides the announcement.

  In the description of the first embodiment, a case will be described in which the audio reproduction system 10 is used when the user cannot speak the language (English) used for the presentation fluently.

  As shown in FIG. 1, the sound reproduction system 10 includes a control unit 1, a storage unit 2, an input unit 3, a display unit 4, an imaging unit 5, a prompter unit 6, a microphone 7, and a speaker 8. including.

  The storage unit 2 includes a non-volatile memory (for example, ROM (Read Only Memory)) in which various programs necessary for the processing of the control unit 1 and a volatile memory (as a work area of the control unit 1) For example, RAM (Random Access Memory). The various programs may be read from a portable recording medium such as an optical disk or a semiconductor memory.

  In particular, in the present embodiment, the storage unit 2 includes text data (speech manuscript) prepared by a user and created in a language (English) used for presentation, and voice data corresponding to the text data. They are stored in association with each other.

  The display unit 4 is configured by, for example, a liquid crystal display, an EL display (EL: Electro Luminescence), or the like, and displays various images on the screen based on the control of the control unit 1. For example, the same data as the text data (for one page) displayed on the prompter unit 6 is displayed on the screen of the display unit 4.

  The input unit 3 includes a keyboard, a mouse, and the like. The input unit 3 inputs an instruction from the user and outputs a signal corresponding to the user's instruction to the control unit 1.

  The control unit 1 is configured by a CPU (Central Processing Unit) or the like. The control unit 1 executes various calculations based on various programs stored in the storage unit 2 and controls each unit of the audio reproduction system 10 in an integrated manner. For example, the control unit 1 determines that the user's mouth has been opened based on the image captured by the imaging unit 5 or sequentially reproduces partial waveform data (described later) in response to the user's mouth being opened. In this way, the playback timing of audio data is controlled. Details of the processing of the control unit 1 will be described later.

  The control unit 1, the storage unit 2, the input unit 3, and the display unit 4 may be configured by, for example, a PC (Personal Computer). Further, the storage unit 2 may be the storage unit 2 in the server device. In this case, the control unit 1 may acquire various data stored in the storage unit 2 via the network.

  The imaging unit 5 includes an imaging device such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and an optical system such as an imaging lens that forms an image on the imaging surface of the imaging device. The imaging unit 5 is configured to capture at least the user's mouth. In the present embodiment, the imaging unit 5 is configured to be able to image not only the mouth but the entire user's face.

  The prompter unit 6 displays text data for each page in accordance with the control of the control unit 1. The prompter unit 6 includes a plate-like half mirror arranged to be inclined with respect to a vertical plane, and a liquid crystal display device arranged upward to project text data (for one page) onto the half mirror. including. The prompter unit 6 is configured so that a character corresponding to the text data can be seen on the half mirror from the user (presenter), but the character is not seen from the audience and the half mirror looks transparent. For example, the prompter unit 6 is arranged on a platform where a user makes a presentation.

  Note that the prompter unit 6 is not limited to a half mirror and a liquid crystal display device. For example, the prompter unit 6 may be configured only by a liquid crystal display device. In this case, the liquid crystal display device is disposed at a position where the user (presenter) can easily see the characters displayed on the liquid crystal display device.

  The microphone 7 converts the input sound into an electrical signal and outputs it to the control unit 1. The microphone 7 is used when the narrator reads out text data (speech manuscript) created in a language (English) used for the presentation in a preparatory stage before the presentation. The narrator is typically a person who can read the text data (English) fluently.

  The speaker 8 outputs sound according to the control of the control unit 1. The speaker 8 is arranged at a venue where the presentation is performed. The number of speakers 8 arranged in the venue may be one or two or more.

[Description of operation]
Next, the operation of the audio reproduction system 10 will be described. In the description of the operation, first, an operation of the audio reproduction system 10 at a preparation stage performed before the presentation is performed will be described, and then an operation of the audio reproduction system 10 when the user makes an announcement will be described.

“Operation of the audio playback system 10 in the preparation stage”
FIG. 2 is a diagram for explaining the operation of the audio reproduction system 10 at the preparation stage, and the correspondence between the text data created in English and the audio data acquired when the text data is read out. FIG.

  First, referring to the upper side of FIG. 2, the user prepares text data (speech manuscript) created in English, which is used when making a presentation. In the upper diagram of FIG. 2, the head portion of the text data “I am a cat. I don't have a name. I am a cat” is shown as an example. Note that the text data is actually longer than the text data shown in FIG.

  Next, the user asks the narrator who can read the prepared text data fluently to read the text data. The voice read out by the narrator is converted into an electric signal by the microphone 7. Thereby, the voice data read out fluently is acquired (refer to the lower diagram in FIG. 2). The acquired voice data is stored in the storage unit 2.

  Next, the control unit 1 detects a silent part (refer to a part surrounded by a square in the lower diagram in FIG. 2) from the acquired voice data, and divides the voice data by the detected silent part. As a result, the audio data is divided into a plurality of partial waveform data. This partial waveform data corresponds to one unit (one segment) reproduced when the user makes a presentation. The partial waveform data is data from the time when the silent portion is started to the time when the next silent portion is started through a group of waveforms.

  Next, the control unit 1 divides the text data into partial text data. Typically, the control unit 1 divides the text data into partial text data by dividing the text data based on delimiters (period, comma) included in the text data.

  Note that, before the partial text data is divided, a comma as a mark for division is added to the text data by the user. This comma is input via the input unit 3. In the example shown in FIG. 2, a comma is added by the user between “I am” and “a cat” (two places). In the example illustrated in FIG. 2, a comma is added by the user between “I don't have” and “a name”.

  The user adds a comma to the text data while predicting the length of the partial text data corresponding to the partial waveform data. The control unit 1 may execute a process of predicting the length of the partial text data corresponding to the partial waveform data and automatically dividing the text data into the partial text data.

  When the voice data is divided into partial waveform data and the text data is divided into partial text data, the control unit 1 next executes a process of associating the partial waveform data with the partial text data. In the example shown in FIG. 2, the partial waveform data of the first, second,... Are in order of “I am”, “a cat”,. Each is associated. As a result, the partial waveform data and the partial text data are associated with each other and stored in the storage unit 2.

  Here, it may be assumed that the voice data delimiter and the text data delimiter do not completely match. For example, the case where the number of partial waveform data differs from the number of partial text data is also assumed. In such a case, when the partial waveform data and the partial text data are associated with each other in a one-to-one correspondence, the association process is inaccurate.

  Therefore, when the voice data delimiter and the text data delimiter do not completely match, the control unit 1 executes a process for obtaining an association that minimizes the cost by using dynamic programming.

FIG. 3 is a diagram for explaining processing for obtaining an association that minimizes the cost by dynamic programming. Referring to FIG. 3, the reproduction times of the partial waveform data in the audio data are now a ₁ , a ₂ , a ₃ ,. Similarly, let t ₁ , t ₂ , t ₃ ,... Be the reproduction times of the partial text data in the text data. The reproduction time of the partial text data can be estimated from the reproduction time when the text data is input to the voice synthesizer and converted into speech.

At this time, the association between the voice data and the text data can be expressed by a single broken line as shown in FIG. These broken lines are designated as S ₁ , S ₂ , S ₃ ,. _{Further, {t k, t k +} 1, ··} text data corresponding to _{S i = t {S i}} , and _{{a l, a l + 1} , ··} = a {S i}. In the example illustrated in FIG. 3, t {S ₂ } = {t ₂ , t ₃ } and a {S ₂ } = {a ₂ }. In this case, the cost of this mapping _{is, Σ ((Σt {S i} }) - (Σa {S i})) becomes 2, i = 1, n. The control unit 1 obtains an association that minimizes the cost by dynamic programming.

  As a result, even when the voice data delimiter and the text data delimiter do not completely match, the partial waveform data and the partial text data can be associated with each other appropriately.

  “Operation of the audio playback system 10 when the user makes a presentation”

  FIG. 4 is a flowchart showing the operation of the audio reproduction system 10 when the user makes a presentation. FIG. 5 is a schematic diagram for explaining the operation of the audio reproduction system 10 when the user makes a presentation.

  First, the control unit 1 causes the prompter unit 6 to display text data corresponding to the first page of the text data (step 101). The control unit 1 may display the same data as the text data displayed on the prompter unit 6 on the screen of the display unit 4.

  Next, the control unit 1 determines whether or not the user's mouth has been opened based on the image captured by the imaging unit 5 (by image analysis) (step 102). When the user's mouth is in a closed state (NO in Step 102), the control unit 1 returns to Step 102 again and determines whether or not the user's mouth has been opened. Therefore, when the user's mouth is closed, the reproduction of the partial waveform data is not started.

  For example, when the user reads out the text data (for one page) displayed on the prompter unit 6 (it is not always necessary to read aloud and may be spoken), the control unit 1 determines that the user's mouth has been opened ( The reproduction of the partial waveform data is started (step 103). When the reproduction of the partial waveform data is started, fluent English read out by the narrator is output from the speaker 8.

  Next, the control unit 1 controls the display of the prompter unit 6 so that the partial text data corresponding to the currently reproduced partial waveform data is highlighted in accordance with the reproduction timing of the partial waveform data (Step S1). 104).

  Thus, the user can easily recognize which part of the text data is currently being reproduced by visually recognizing the highlighted partial text data.

  Examples of the highlighting method include a method of blinking all or part of the partial text data, a method of changing the color of all or part of the partial text data to another color, and the like. Note that any method may be used for highlighting.

  When the highlighting process is executed, the control unit 1 then extracts face information based on the image captured by the image capturing unit 5 (by image analysis), and is currently being reproduced based on the face information. The voice inflection in the voice data is controlled (step 105).

  In step 105, the control unit 1 extracts the distance between the eyes and the eyebrows as the face information, and executes a process for controlling the volume of the sound based on the distance between the eyes and the eyebrows.

  FIG. 6 is a diagram illustrating a change in the distance between the eyes and the eyebrows. The control unit 1 controls the sound volume so that the sound volume increases as the distance between the eyes and the eyebrows increases.

  Further, in step 105, the control unit 1 extracts the face vertical direction as face information, and controls the voice height based on the face vertical direction.

  FIG. 7 is a diagram illustrating a change in the orientation of the face in the vertical direction. The control unit 1 controls the voice level so that the voice level is lower when the face orientation is lower than the reference. On the other hand, when the face orientation is higher than the reference, the voice level is controlled so that the voice level becomes higher.

  When the reproduction time of the partial waveform data (different for each partial waveform data) has elapsed after the reproduction of the partial waveform data is started in step 103, the reproduction of the partial waveform data is finished (step 106).

  When the reproduction of the partial waveform data is completed, the control unit 1 determines whether or not the next partial waveform data exists (step 107). When the next partial waveform data exists (YES in Step 107), the control unit 1 determines whether or not the reproduction of the audio data corresponding to the text data in the page currently displayed on the prompter unit 6 is completed. (Step 108).

  When the reproduction of the audio data corresponding to the text data in the page currently displayed on the prompter unit 6 is completed (YES in step 108), the control unit 1 prompts the text data corresponding to the next page. It is displayed on the part 6 (step 109). Then, the control unit 1 returns to Step 102 and determines whether or not the user's mouth has been opened.

  On the other hand, when the reproduction of the voice data corresponding to the text data in the page currently displayed on the prompter unit 6 is not completed (NO in step 108), the control unit 1 does not execute the process of switching pages. Returning to step 102, it is determined whether or not the user's mouth has been opened.

  That is, the control unit 1 determines again whether or not the user's mouth has been opened when the reproduction of the partial waveform data is completed. When it is determined that the user's mouth has been opened, the control unit 1 starts reproduction of the next partial waveform data (step 103). Thereby, the partial waveform data in the audio data is sequentially reproduced in accordance with the timing when the user's mouth is opened.

  If it is determined in step 107 that the next partial waveform data does not exist (NO in step 107), the control unit 1 ends the process.

[Action etc.]
In the present embodiment, voice data used for presentation is stored in advance in the storage unit. When the user's mouth opens when an announcement is actually made, the partial waveform data in the audio data is sequentially reproduced, so that the reproduction timing of the audio data coincides with the timing at which the mouth is opened. As a result, even if the user is not necessarily proficient in the language used, the audience can feel as if the user is speaking naturally.

  Here, as a comparative example, it is assumed that the audio data is not divided into partial waveform data and the audio data is reproduced without interruption from the beginning to the end. In such a case, it is not impossible for the user to perform a speech according to the audio data to be reproduced, but the tempo of the presentation is controlled by the tempo of the audio data recorded in advance. In this case, the user cannot make a presentation at the user's own tempo while watching the reaction of the audience.

  On the other hand, in the audio reproduction system 10 according to the present embodiment, if the user closes his / her mouth, the next partial waveform data is not reproduced. Therefore, the user can also make a presentation at the user's own tempo while watching the reaction of the audience.

  Furthermore, in this embodiment, since the inflection of the voice data currently being reproduced is controlled based on the face information, it is possible to add emotional expressions to the currently reproduced voice data.

<Various modifications>
In the above description, the audio data has been described as data obtained by the narrator reading the text data. On the other hand, the voice data may be synthesized voice by voice synthesis. This synthesized speech is also divided into partial waveform data and stored in advance in the storage unit 2 in a preparatory stage before the presentation is performed. When synthesized speech is used as speech data, it is not necessary to perform a process of associating partial waveform data and partial text data by dynamic programming.

  Here, when synthesized speech is used as speech data, it tends to be monotonous compared to speech data read by a narrator, and the fluency may be somewhat reduced. However, by executing the above-described inflection control (see step 105), monotony can be eliminated and fluency can be improved.

  In the above description, the case where the audio reproduction system 10 is used in the presentation when the user cannot speak English fluently has been described. On the other hand, even when the user cannot speak for some reason (such as hearing impairment or vocal cord disturbance), the audio reproduction system 10 according to the present technology can be used in the presentation.

  The audio reproduction system may have a pause function or a skip function. For example, the user pauses the reproduction of audio data when a question is asked during the presentation. In addition, when the presentation time runs out, the user skips the reproduction position of the audio data to an arbitrary position. In this case, when a pause button or a skip button provided on a remote controller (not shown) is operated, the control unit may execute a pause or skip based on this operation signal.

  In the above description, the case where the audio reproduction system 10 is used in the presentation has been described. However, the audio reproduction system 10 can also be used for other uses such as theater. Furthermore, for example, the sound reproduction system 10 can be provided with entertainment by using sound of characters such as animations, movies, and actors as sound data. In this case, when the user moves his / her mouth, voices of characters, actors, etc. are output from the speaker 8. You can also pretend that a woman is speaking in a male voice (and vice versa), and pretend that a child is speaking in an adult voice (or vice versa).

  Furthermore, the audio reproduction system 10 can be used for learning purposes. For example, text data in an English textbook is displayed on the prompter unit 6 page by page. When the user reads out the English displayed on the prompter unit 6, the partial waveform data in the audio data is sequentially reproduced in accordance with the opening timing of the mouth. As a result, fluent English as an example is output from the speaker 8. The user can correct the mistake based on the difference between the English pronunciation read out by the user and the fluent English pronunciation as an example, and can learn the correct English pronunciation.

  In the above description, the case where there is one kind of language to be reproduced has been described. However, two or more languages may be played back. In this case, for example, audio data in different languages is output to two or more types of channels and received by the earphone. The earphone is configured to be able to adjust a channel to be received. By selecting a channel corresponding to the desired language, the audience can listen to the presentation or theatrical performance in the desired language via the earphone.

  In addition to the text data, voice data may be displayed on the prompter unit 6. For example, when a user makes a presentation or a play in a language with little knowledge, the user can grasp a foothold of how to move his / her mouth by visually recognizing the waveform of audio data.

The present technology can also employ the following configurations.
(1) a storage unit that stores in advance audio data including a plurality of partial waveform data;
An imaging unit capable of imaging a user's mouth;
Timing of reproducing the audio data so as to determine that the user's mouth has been opened based on the image captured by the imaging unit and to sequentially reproduce the partial waveform data in accordance with the opening of the user's mouth A sound reproduction system comprising a control unit for controlling the sound.
(2) The audio reproduction system according to (1) above,
The imaging unit is capable of imaging the user's face,
The said control part extracts face information based on the imaged image, and controls sound inflection in the audio | voice data currently reproduced | regenerated based on the said face information.
(3) The audio reproduction system according to (2) above,
The said control part extracts the distance between eyes and eyebrows as said face information, and controls the magnitude | size of an audio | voice based on the distance between eyes and eyebrows. The audio | voice reproduction system.
(4) The audio reproduction system according to (2) or (3) above,
The control unit extracts an orientation of the face in the vertical direction as the face information, and controls a voice height based on the orientation of the face in the vertical direction.
(5) The audio reproduction system according to any one of (1) to (4) above,
Further comprising a prompter,
The storage unit further stores text data corresponding to the voice data,
The audio control system, wherein the control unit controls display of the prompter unit so that the text data is displayed on the prompter unit.
(6) The audio reproduction system according to (5) above,
The text data includes a plurality of partial text data,
The storage unit stores the partial waveform data in association with partial text data corresponding to the partial waveform data,
The control unit controls display of the prompter unit so that partial text data corresponding to the currently reproduced partial waveform data is highlighted in accordance with the reproduction timing of the partial waveform data.
(7) The audio reproduction system according to (6) above,
The control unit executes a process of associating the partial waveform data and the partial text data in order to associate and store the partial waveform data and the partial text data corresponding to the partial waveform data in the storage unit. Audio playback system.
(8) The audio reproduction system according to any one of (1) to (7) above,
The said memory | storage part memorize | stores the data acquired when the narrator reads the said text data as said audio | voice data. Voice reproduction | regeneration system.
(9) The audio reproduction system according to any one of (1) to (7) above,
The said memory | storage part memorize | stores the synthetic | combination audio | voice by audio | voice synthesis | combination as said audio | voice data Voice reproduction | regeneration system.

DESCRIPTION OF SYMBOLS 1 ... Control part 2 ... Memory | storage part 3 ... Input part 4 ... Display part 5 ... Imaging part 6 ... Prompter part 7 ... Microphone 8 ... Speaker 10 ... Sound reproduction system

Claims (11)

A storage unit for storing in advance audio data including a plurality of partial waveform data;
An imaging unit capable of imaging a user's mouth;
Timing of reproducing the audio data so as to determine that the user's mouth has been opened based on the image captured by the imaging unit and to sequentially reproduce the partial waveform data in accordance with the opening of the user's mouth A sound reproduction system comprising a control unit for controlling the sound. The audio playback system according to claim 1,
The imaging unit is capable of imaging the user's face,
The said control part extracts face information based on the imaged image, and controls sound inflection in the audio | voice data currently reproduced | regenerated based on the said face information. The audio playback system according to claim 2,
The said control part extracts the distance between eyes and eyebrows as said face information, and controls the magnitude | size of an audio | voice based on the distance between eyes and eyebrows. The audio | voice reproduction system. The audio playback system according to claim 2,
The control unit extracts an orientation of the face in the vertical direction as the face information, and controls a voice height based on the orientation of the face in the vertical direction. The audio playback system according to claim 1,
Further comprising a prompter,
The storage unit further stores text data corresponding to the voice data,
The audio control system, wherein the control unit controls display of the prompter unit so that the text data is displayed on the prompter unit. The audio reproduction system according to claim 5,
The text data includes a plurality of partial text data,
The storage unit stores the partial waveform data in association with partial text data corresponding to the partial waveform data,
The control unit controls display of the prompter unit so that partial text data corresponding to the currently reproduced partial waveform data is highlighted in accordance with the reproduction timing of the partial waveform data. The audio reproduction system according to claim 6,
The control unit executes a process of associating the partial waveform data and the partial text data in order to associate and store the partial waveform data and the partial text data corresponding to the partial waveform data in the storage unit. Audio playback system. The audio playback system according to claim 1,
The said memory | storage part memorize | stores the data acquired when the narrator reads text data as said audio | voice data. The audio playback system according to claim 1,
The said memory | storage part memorize | stores the synthetic | combination audio | voice by audio | voice synthesis | combination as said audio | voice data Voice reproduction | regeneration system. Pre-store voice data including multiple partial waveform data,
Image the mouth of the user,
Based on the captured image, it is determined that the user's mouth has been opened,
An audio reproduction method for controlling the reproduction timing of the audio data so as to sequentially reproduce the partial waveform data in response to the opening of the user's mouth. In the audio playback system,
Pre-storing audio data including a plurality of partial waveform data;
Imaging a user's mouth;
Determining that the user's mouth has been opened based on the captured image;
Controlling the reproduction timing of the audio data so as to sequentially reproduce the partial waveform data in response to the opening of the user's mouth.