JP6485214B2 - Electronic library system - Google Patents

Description

  The present invention is an invention in which an electronic library that lends electronic materials can be operated by voice using the Internet.

  For those who are blind (hereinafter referred to as “visually impaired”), the enjoyment gained from reading is greater than that of healthy people, so as a library for the visually impaired, books for the visually impaired (Braille books) Braille libraries have already been established to lend out books to the visually impaired. However, because the books for the visually impaired, which are collected by the Braille Library, are produced with the cooperation of volunteers, the number of books for the visually impaired is much smaller than the regular library. There is also a problem that books for the visually impaired cannot be borrowed in real time because books for the visually impaired are lent to the visually impaired by mail.

  On the other hand, with regard to libraries for healthy people, electronic libraries that lend electronic materials using the Internet are increasing in order to improve convenience for users. Inventions related to electronic libraries using the Internet have already been disclosed. For example, in Patent Document 1, a user accesses an electronic library via the Internet, searches for electronic materials, rents electronic materials, and returns electronic materials. A system that can do this is disclosed.

  If visually impaired people can use electronic libraries, visually impaired people can borrow electronic materials without going to the library building, and the number of electronic materials that visually impaired people can borrow increases. Electronic libraries that lend electronic materials are not designed to be easily accessible by the visually impaired.

  In order to make it easy for visually impaired people to use the electronic library by themselves, it is only necessary to be able to use the electronic library without looking at the screen, and if the electronic library can be operated using voice, it will be easier for visually impaired people to use the library independently. Conceivable.

  As an invention relating to a book system using voice, Patent Document 2 discloses an information access system that uses voice to improve convenience for visually impaired persons. Patent Document 2 describes, for example, generating speech synthesis of text of electronic materials by recognizing user's speech and searching electronic materials (publications) based on speech recognition results. However, the invention according to Patent Document 2 does not focus on the fact that the electronic library can be operated using voice.

  The electronic library has a plurality of Web pages and can be switched arbitrarily. However, for visually impaired people who cannot see the screen, the Web page of the electronic library can be switched arbitrarily by voice. Even if it is possible, it is difficult to use the electronic library by arbitrarily switching the Web page of the electronic library.

JP 2008-262518 A JP-A-7-152787

  Therefore, an object of the present invention is to provide an electronic library system in which a visually impaired person can easily use an electronic library by voice.

The first invention that solves the above-mentioned problems provides various electronic library services that can be used via a network. When a call request for the electronic library service is received, the electronic library service that receives the call request is executed, An electronic library site that transmits the execution result of the electronic library service and an application program that uses the electronic library service provided by the electronic library site are installed on the device that has requested the electronic library service to be called. A voice recognition server that performs voice recognition using a voice feature amount and transmits a voice recognition result in a text format obtained by voice recognition to the portable terminal when a voice feature amount is received from the portable terminal and the portable terminal When text is received from the mobile terminal, a synthesized speech of the text is generated. Serial at least configured digital library system from a voice synthesizing server for transmitting to the portable terminal.
Even if it is a visually impaired person, when the application program is started so that the electronic library can be easily used by voice, the portable terminal extracts the feature amount of the voice input by the user to the microphone, and A feature amount extraction unit that transmits a feature recognition amount corresponding to a speech input by a user to a microphone from the speech recognition server, a procedure related to operation of the electronic library site, and When an electronic library operation flow describing the execution order is registered and when the procedure described in the electronic library operation flow is executed, when the operation of starting speech input is detected, the feature amount extraction unit is operated, After the speech recognition result of the voice input by the user to the microphone is acquired from the feature amount extraction unit, the speech recognition result acquired from the feature amount extraction unit is used. Then, when the electronic library service corresponding to the procedure at this time is called and the execution result of the called electronic library service is received from the electronic library site, the electronic library site is utilized using the speech synthesis server. A procedure execution unit for generating a synthesized speech corresponding to the execution result received from and outputting the synthesized speech.

  According to a second invention for solving the above-described problem, in the electronic library system according to the first invention, when the procedure execution unit detects an operation of starting voice input, the voice guidance related to the procedure at this time is displayed. The feature extraction unit is activated after outputting the voice. When the procedure execution unit detects an operation to start voice input, the voice guidance related to the procedure at this time is configured to be output by voice so that the user can Can be grasped by voice guidance.

  According to a third invention for solving the above-described problem, in the electronic library system according to the first invention or the second invention, the mobile terminal includes a motion sensor, and the procedure execution unit is configured to shake the mobile terminal. Is detected as an operation of starting voice input. The action of starting voice input can be an action of selecting a button object for starting voice input, but assuming that the user is a visually impaired person, It is preferable to detect as an operation of starting voice input.

  According to a fourth invention for solving the above-mentioned problem, in the electronic library system according to any one of the first to third inventions, the electronic library service for retrieving materials is called up, and the feature quantity extracting unit A document search procedure for searching for a document that matches the speech recognition result obtained from the above, a synthesized speech of text included in the search result of the document search procedure is output as voice, and the electronic library service for selecting the document is called, A material selection procedure for selecting a material corresponding to the speech recognition result acquired from the amount extraction unit, the electronic library service for logging in to the electronic library site is called, and the speech recognition result acquired from the feature amount extraction unit is used. Login procedure for logging in to the electronic library site, calling the electronic library service for renting materials, Lending procedures for borrowing materials selected in the fee selection procedure from the electronic library site, calling the electronic library service providing the content of the materials, and text included in the contents of the materials borrowed from the electronic library site in the lending procedure Is described in the electronic library operation flow as the procedure relating to the operation of borrowing and reading materials from the electronic library site and the execution order thereof. According to the fourth invention, a series of operations for borrowing and reading materials from the electronic library site can be performed by voice.

  A fifth invention that solves the above-described problem is the electronic library system according to the fourth invention, wherein the electronic document system returns a material when a material borrowed by a user is searched in the material selection procedure. Calling the library service and executing the return procedure for returning the material specified by the voice recognition result acquired from the feature amount extraction unit to the electronic library site is performed as the procedure related to the material return operation and the execution order thereof. It is described in the electronic library operation flow. According to the fifth aspect, the operation of returning the materials borrowed from the electronic library site can be performed by voice.

  A sixth invention for solving the above-described problem is the electronic library system described in the fourth invention or the fifth invention, and reserves a material when the material selected in the material selection procedure cannot be lent. Calling the electronic library service and executing the reservation procedure for reserving the material selected in the material selection procedure in the electronic library site is the electronic library operation flow as the procedure relating to the material reservation operation and its execution order. It is characterized by having described it. According to the sixth invention, the operation of reserving materials at the electronic library site can be performed by voice.

  According to a seventh invention for solving the above-described problem, in the electronic library system according to any one of the fourth to sixth inventions, when the application program is activated, the portable terminal is connected to a microphone by a user. A voice print authentication unit for authenticating a voice print of a voice input to the electronic library, and the procedure executing unit is configured to perform the electronic library only when the voice print authentication unit succeeds in the voice print authentication of a voice input by a user to a microphone in the login procedure. Calling the electronic library service to log in to the site. By authenticating the voice print of the voice input to the microphone by the user, security related to login to the electronic library site can be enhanced.

  According to the electronic library system according to the present invention described above, the procedure using voice is automatically executed according to the operation flow of the electronic library, so that even the visually impaired can easily use the electronic library by voice. It becomes like this.

The figure explaining the structure of the electronic library system which concerns on this embodiment. The block diagram of the portable terminal which concerns on this embodiment. The figure explaining the electronic library operation flow which concerns on this embodiment. The figure explaining the process which a procedure execution part performs in a document search procedure. The figure explaining the top page of an electronic library site. The figure explaining the process which a procedure execution part performs in a data selection procedure. The figure explaining the page of the electronic library site which displays the search result of a document. The figure explaining the process which a procedure execution part performs in a loan procedure. The figure explaining the process which a procedure execution part performs in a login procedure. The figure explaining the process which a procedure execution part performs in a reproduction | regeneration procedure. The figure explaining the process which a procedure execution part performs in a material return procedure.

  From here, preferred embodiments of the present invention will be described. The following description is not intended to limit the technical scope of the present invention, but is provided to aid understanding.

  From here, the electronic library system 1 which concerns on this embodiment is demonstrated. FIG. 1 is a diagram for explaining the configuration of an electronic library system 1 according to the present embodiment. An electronic library system 1 according to the present embodiment is a system designed so that a visually impaired person can use an electronic library by himself. As shown in FIG. 1, a portable terminal 2, an electronic library site 3, a voice recognition server 4 and the speech synthesis server 5, each of which can perform data communication via the network 1a.

  The electronic library site 3 constituting the electronic library system 1 searches various electronic materials accumulated in the electronic library site 3 (hereinafter simply referred to as “materials”), lends materials, reserves materials, An electronic library service is provided in order to make it possible for an application program operating on the mobile terminal 2 to use various procedures related to electronic library operations such as return, and in this embodiment, an API (Application Programming) for calling the electronic library service is provided. Interface) is open to the public. The electronic library site 3 performs data communication with the mobile terminal 2 using HTTP (HyperText Transfer Protocol), executes processing related to the electronic library service corresponding to the API called from the mobile terminal 2, and carries the execution result on the mobile phone. A process of returning to the terminal 2 is performed. The electronic library site 3 is preferably composed of a cloud type system.

  The speech recognition server 4 that constitutes the electronic library system 1 is a server that recognizes speech features received from the mobile terminal 2 and transmits a text-format speech recognition result to the mobile terminal 2. Although there are various methods of speech recognition, the speech recognition server 4 according to the present embodiment holds a speech feature amount for each word in the recognition unit, and identifies a word corresponding to the speech feature amount received from the mobile terminal 2 Then, a speech recognition result in a text format is generated by arranging in order.

  The speech synthesis server 5 constituting the electronic library system 1 analyzes the text received from the mobile terminal 2 and synthesizes speech corresponding to characters included in the text, thereby synthesizing speech in a predetermined format (for example, MP3 format). Is generated and transmitted to the mobile terminal 2. There are various speech synthesis methods, but the morphological analysis of the target text is performed, the text is converted into a phonetic symbol string, parameters necessary for speech synthesis are generated from the phonetic symbol string, and the speech waveform is generated using these parameters. A technique for generating synthesized speech by synthesis is already known.

  A portable terminal 2 constituting the electronic library system 1 is a terminal used when a user (here, a visually impaired person) uses the electronic library site 3, and specifically, is a smartphone or a tablet.

  FIG. 2 is a block diagram of the mobile terminal 2 according to the present embodiment. As illustrated in FIG. 2, the mobile terminal 2 includes a microphone 24, a motion sensor 25, a touch panel 26 and a speaker 27. The microphone 24 included in the mobile terminal 2 is a device that converts sound into an electrical signal. The motion sensor 25 included in the mobile terminal 2 is a sensor that detects a change in the movement of the mobile terminal 2, and specifically, a 3-axis gyro sensor or a 3-axis acceleration sensor. The touch panel 26 included in the mobile terminal 2 is an electronic device in which a display such as a liquid crystal display and a device that performs an operation by directly touching the display are combined. The speaker 27 provided in the portable terminal 2 is a device used for outputting sound, and is a device that converts an electrical signal into sound. Note that an audio output terminal may be used instead of the speaker 27.

  In addition, the mobile terminal 2 uses the API published by the electronic library site 3 as a computer program for operating a processor not shown in FIG. 2, and provides an electronic library service provided by the electronic library site 3. The electronic library application 20 configured to call and operate the electronic library site 3 by voice is installed. As illustrated in FIG. 2, the electronic library application 20 includes a procedure execution unit 21 and a feature amount extraction unit. 22 and a voiceprint authentication unit 23.

  The feature amount extraction unit 22 included in the electronic library application 20 is a function provided for recognizing a voice of a user who uses the electronic library site 3. When the voice recognition is specified and the procedure execution unit 21 is called, the feature quantity extraction unit 22 of the electronic library application 20 extracts and extracts the voice feature quantity (for example, frequency spectrum) input to the microphone 24 by the user. The received feature amount is transmitted to the speech recognition server 4, and the speech input by the user to the microphone 24 is caused to be recognized by the speech recognition server 4, and then the speech recognition result received from the speech recognition server 4 is delivered to the procedure execution unit 21. Execute the process. When the feature recognition unit 22 of the electronic library application 20 is called from the procedure execution unit 21 with voice recognition and voiceprint authentication specified, in addition to the above-described content, the voiceprint input by the user to the microphone 24 will be described later. Deliver it to the authentication unit 23, receive the voice print authentication result from the voice print authentication unit 23, and execute a process of transferring the voice recognition result received from the voice recognition server 4 and the voice print authentication result received from the voice print authentication unit 23 to the procedure execution unit 21. .

  The voiceprint authentication unit 23 included in the electronic library application 20 is a function provided so that a user who uses the electronic library site 3 can recognize a voiceprint. The voiceprint passed from the feature amount extraction unit 22 and a voiceprint authentication unit in advance. The voice print registered in 23 is collated, and the voice print authentication result is transferred to the feature amount extraction unit 22.

  The procedure execution unit 21 included in the electronic library application 20 is a function provided for controlling the operation of the electronic library site 3 by voice. The electronic library site 3 having a plurality of Web pages is configured so that the Web pages can be arbitrarily switched. However, it is difficult for the visually impaired who cannot see the Web pages to arbitrarily switch the Web pages. . Therefore, the procedure execution unit 21 according to the present embodiment is configured to sequentially execute the procedures included in the electronic library operation flow according to the electronic library operation flow describing the procedure related to the operation of the electronic library site 3 and the execution order thereof. When the operation for starting the voice input is detected when the procedure is executed, the feature amount extraction unit 22 is operated, and the voice recognition result of the voice input by the user to the microphone 24 is acquired from the feature amount extraction unit 22. , Using the speech recognition result acquired from the feature quantity extraction unit 22, call the electronic library service corresponding to the procedure at this time, that is, the time when the operation of starting the voice input is detected, and execute the called electronic library service When the result is received from the electronic library site 3, the speech synthesis server 5 is used to correspond to the execution result received from the electronic library site 3. It generates a synthesized voice that, the synthesized speech executes processing for audio output.

  FIG. 3 is a diagram for explaining an electronic library operation flow according to the present embodiment. In the electronic library operation flow shown in FIG. 3, the electronic library service for retrieving materials is called as a procedure for operating the electronic library site 3 by using voice, and is adapted to the voice recognition result obtained from the feature amount extraction unit 22 The material selection which selects the material corresponding to the speech recognition result acquired from the feature amount extraction unit 22 after outputting the synthesized speech of the text included in the retrieval result of the material retrieval procedure T1 and the material retrieval procedure T1 The procedure T2, the speech recognition result acquired from the feature quantity extraction unit 22 as a login keyword, the electronic library service for logging in to the electronic library site 3 and the login procedure T4 for logging in to the electronic library site 3 And borrow the material selected in the document selection procedure T2 from the electronic library site 3 Lending procedure T3, calling the electronic library service that provides the contents of the material, calling the electronic library service for returning the material borrowed from the electronic library site 3 in the lending procedure T3, calling the electronic library service for returning the material, and selecting the material T2 A return procedure T6 for returning the material selected in step 1 to the electronic library site 3 is included.

  According to the electronic library operation flow shown in FIG. 3, after the electronic library application 20 is started, the procedure executed by the procedure execution unit 21 is the material search procedure T1, and when the material is searched by the material search procedure T1, The procedure executed by the execution unit 21 transitions to a material selection procedure T2. When a material is selected in the material selection procedure T2, the procedure executed by the procedure execution unit 21 transitions to a lending procedure T3. In the lending procedure T3, it is confirmed whether or not the electronic library site 3 is logged in. When the electronic library site 3 is not logged in, the procedure executed by the procedure execution unit 21 transitions to the login procedure T4. In addition, when logging in to the electronic library site 3 in the lending procedure T3 or after logging in to the electronic library site 3 in the login procedure T4, the materials selected in the document selecting procedure T2 are borrowed from the electronic library site 3. Then, the procedure executed by the procedure execution unit 21 transitions to the reproduction procedure T5. When the reproduction procedure T5 ends, the procedure executed by the procedure execution unit 21 returns to the material search procedure T1. In addition, when a material to be returned to the electronic library site 3 is searched in the material search procedure T1, the procedure executed by the procedure execution unit 21 shifts to the return procedure T6, and the selected material is returned to the electronic library site 3. The procedure executed by the procedure execution unit 21 returns to the material retrieval procedure T1.

  Note that the operation of starting the electronic library application 20 is an operation of selecting (double-tapping) the icon of the electronic library application 20, and the portable terminal 2 generates a voice so that a visually impaired person can also start the electronic library application 20. It is desirable to support a screen reader that reads the contents displayed on the touch panel 26 and activates the used application.

  From here, the processing executed by the procedure execution unit 21 of the electronic library application 20 in each procedure included in the electronic library operation flow illustrated in FIG. 3 will be described in detail. FIG. 4 is a diagram illustrating processing executed by the procedure execution unit 21 in the material search procedure T1. In the document retrieval procedure T1, the procedure execution unit 21 of the electronic library application 20 first enters a state of accepting an operation for starting voice input, and when an operation for starting voice input is detected (S1), at this time, that is, a voice. After voice guidance corresponding to the time point at which an input start operation is detected (for example, “Speak a keyword”) is output from the speaker 27 (S2), voice recognition is designated and the feature amount extraction unit 22 is set. Operate (S3).

  The operation for starting voice input can be an operation in which a button object is displayed at the center of the touch panel 26 of the portable terminal 2 and this button object is selected (tapped). It is preferable that the terminal 2 be shaken. The operation for starting the voice input is the operation for shaking the mobile terminal 2, and when the operation for starting the voice input is detected, the voice guidance is output by voice. The current procedure being executed by the procedure execution unit 21 of the library application 20 can be grasped. Note that the motion of shaking the mobile terminal 2 can be detected using the motion sensor 25 of the mobile terminal 2.

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 of the electronic library application 20 extracts the voice feature quantity input to the microphone 24 by the user and transmits the extracted feature quantity to the voice recognition server 4. The voice recognition result received from the voice recognition server 4 is transferred to the procedure execution unit 21, and the procedure execution unit 21 acquires the voice recognition result of the voice input by the user to the microphone 24 (S4).

  When the procedure execution unit 21 of the electronic library application 20 receives the speech recognition result from the feature amount extraction unit 22, the procedure execution unit 21 branches the process according to the speech recognition result (S5). When the speech recognition result delivered from the feature quantity extraction unit 22 is a category registered in advance in the procedure execution unit 21, the procedure execution unit 21 sets the speech recognition result delivered from the feature quantity extraction unit 22 as a category, An electronic library service that searches for materials suitable for this category is called (S6), and a search result for materials suitable for this category is acquired from the electronic library site 3 as an execution result of this electronic library service (S7). Then, the procedure execution unit 21 of the electronic library application 20 uses the speech synthesis server 5 as the synthesized speech corresponding to the execution result of the electronic library service to notify the number of materials included in the material search result. And the synthesized speech is output from the speaker 27 (S10), and the procedure executed by the procedure execution unit 21 is shifted to the material selection procedure T2.

  If the speech recognition result delivered from the feature quantity extraction unit 22 is not a category registered in advance in the procedure execution unit 21, the procedure execution unit 21 of the electronic library application 20 is delivered from the feature quantity extraction unit 22. The electronic library service for retrieving the material that matches the search keyword is called as a search keyword (S8), and the search result of the material that matches the search keyword is used as the execution result of the electronic library service. Obtained from the site 3 (S9). Then, the procedure execution unit 21 of the electronic library application 20 uses the speech synthesis server 5 as the synthesized speech corresponding to the execution result of the electronic library service to notify the number of materials included in the material search result. Is generated, the synthesized speech is output from the speaker 27 (S10), and the procedure executed by the procedure execution unit 21 is shifted to the material selection procedure T2.

  FIG. 5 is a diagram illustrating the top page 6 of the electronic library site 3. The top page 6 illustrated in FIG. 5 includes three tabs, “New Arrival”, “Ranking”, and “My Page”, as tabs for changing the display contents without moving the page. The “New Arrival” tab 6a is a tab for displaying new arrivals published within a predetermined period, and the “Ranking” tab 6b is a tab for displaying materials with a large number of loans in a predetermined period in a ranking format. The “page” tab 6c is a tab for displaying at least materials lent to the user. For example, when the category whose voice recognition result matches is “new arrival”, the procedure execution unit 21 searches the display contents (list of new arrivals published within a predetermined period) when the “new arrival” tab 6a is selected. As a result, it is obtained from the electronic library site 3. The top page 6 shown in FIG. 5 has an input form 6d for inputting a search keyword and a button 6e for executing a keyword search. For example, when the speech recognition result is “history”, the procedure execution unit 21 The search result of the keyword search when “history” is input to the input form 6 d is acquired from the electronic library site 3.

  In FIG. 5, the “My Page” tab 6 c is a tab that is displayed only when the user is logged in to the electronic library site 3. If the category with the same voice recognition result is “My Page” while logged in to the electronic library site 3, the procedure execution unit 21 searches the display contents when the “My Page” tab 6c is selected. As a result, it is obtained from the electronic library site 3.

  FIG. 6 is a diagram for explaining processing executed by the procedure execution unit 21 in the material selection procedure T2. In the document selection procedure T2, the procedure execution unit 21 of the electronic library application 20 transmits the text included in the search result acquired from the electronic library site 3 in the document search procedure T1 to the speech synthesis server 5, and is included in the search result. After the synthesized speech of the text to be obtained is acquired from the speech synthesis server 5, the synthesized speech acquired from the speech synthesis server 5 is output from the speaker 27 (S20).

  When the procedure execution unit 21 of the electronic library application 20 outputs the search result acquired from the electronic library site 3 by voice, the procedure execution unit 21 enters a state of accepting an operation for starting voice input, and detects the operation for starting voice input (S21). A voice guidance corresponding to this time point (for example, “speak the title of the material to be selected or the number of the material to be selected”) is output from the speaker 27 (S22), then voice recognition is designated and the feature amount extraction unit 22 is specified. Is operated (S23).

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 of the electronic library application 20 extracts the voice feature quantity input to the microphone 24 by the user and transmits the extracted feature quantity to the voice recognition server 4. The voice recognition result received from the voice recognition server 4 is delivered to the procedure execution unit 21, and the procedure execution unit 21 acquires the voice recognition result of the voice input by the user to the microphone 24 (S24).

  When the procedure execution unit 21 of the electronic library application 20 receives the speech recognition result from the feature amount extraction unit 22, the procedure execution unit 21 branches the process according to the speech recognition result (S25). When the speech recognition result delivered from the feature quantity extraction unit 22 is a document title or document number, the procedure execution unit 21 selects a document identified by the speech recognition result (in this case, the title or number). (S28), the procedure executed by the procedure execution unit 21 is shifted to the lending procedure T3.

  Further, the procedure execution unit 21 of the electronic library application 20 recognizes the speech recognition delivered from the feature quantity extraction unit 22 when the speech recognition result delivered from the feature quantity extraction unit 22 is other than the material title or the material number. It is confirmed whether the result is a word related to rearrangement registered in advance in the procedure execution unit 21. If the speech recognition result delivered from the feature amount extraction unit 22 is a word related to rearrangement, The electronic library service that executes the corresponding rearrangement is called (S26), the search result after the rearrangement is acquired from the electronic library site 3 (S27), and the process returns to the top of FIG. If the speech recognition result delivered from the feature quantity extraction unit 22 is other than the above word, the procedure executed by the procedure execution unit 21 is shifted to the material search procedure T1.

  FIG. 7 is a diagram for explaining the page 7 of the electronic library site 3 that displays the search result of the material. In the page 6 illustrated in FIG. 7, “history” is input as a search keyword, and a list 7 a of materials matching “history” is displayed as a search result. The search result includes, for each material included in the search result, Includes the cover image of the material, the title of the material, the name of the author of the material, and the date of publication of the material. Further, the page 7 illustrated in FIG. 7 includes a button object 7b to be selected when the search results are rearranged. By selecting this button object 7b, the search results are rearranged in the new arrival order, ascending order. , You can choose either descending order.

  The procedure execution unit 21 of the electronic library application 20 acquires the contents of the material list 7a as a result of the material search, and reads out the text included in the material list 7a. When the speech recognition result is the title of the history book 2, the procedure execution unit 21 puts the history book 2 in a selected state. If the speech recognition results are in the order of arrival, the procedure execution unit 21 calls an electronic library service that rearranges the search results of the materials in the order of arrival.

  FIG. 8 is a diagram for explaining processing executed by the procedure execution unit 21 in the lending procedure T3. In the lending procedure T3, the procedure execution unit 21 of the electronic library application 20 first branches the process depending on whether or not the electronic library site 3 is logged in (S30). The procedure execution unit 21 of the electronic library application 20 manages the login status to the electronic library site 3 as internal information, and refers to the internal information to determine whether or not the electronic library site 3 is logged in.

  If the procedure execution unit 21 of the electronic library application 20 is not logged in to the electronic library site 3, the procedure executed by the procedure execution unit 21 is shifted to the login procedure T4, and the process related to the login procedure T4 is executed. Also, if the procedure execution unit 21 is logged in to the electronic library site 3, the procedure execution unit 21 calls an electronic library service that provides the lending status of the material (S31), and displays the lending status of the material selected in the material selection procedure T2. The processing branches depending on whether the user has already borrowed the material acquired from the electronic library site 3 (S32) and selected (S33).

  If the selected material is lent out according to the lending status of the selected material, and the person who borrows this material is the user himself, the procedure execution unit 21 of the electronic library application 20 is the procedure execution unit. The procedure executed by 21 is shifted to the reproduction procedure T5.

  If the user has not borrowed the selected material, the procedure execution unit 21 of the electronic library application 20 branches the process depending on whether the selected material can be lent (S34). When the selected material can be lent, the procedure execution unit 21 of the electronic library application 20 enters a state of accepting an operation for starting voice input, and when detecting the operation for starting voice input (S42), at this time point After corresponding voice guidance (for example, “Do you borrow material?”) Is output from the speaker 27 (S43), voice recognition is designated and the feature amount extraction unit 22 is activated (S44).

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 extracts the feature quantity of the voice input to the microphone 24 by the user, transmits the extracted feature quantity to the voice recognition server 4, and then the voice recognition server 4. The procedure execution unit 21 acquires the speech recognition result of the speech input by the user to the microphone 24 (S45).

  The procedure execution unit 21 of the electronic library application 20 branches the process according to the voice recognition result of the voice input by the user to the microphone 24 (S46). If the speech recognition result is a positive word (for example, “Yes”), the procedure execution unit 21 of the electronic library application 20 calls the electronic library service that lends the material (S47), and the execution result of the electronic library service that lends the material. Is received from the electronic library site 3, the title of the material borrowed from the electronic library site 3, that is, the title of the material selected in the material selection procedure T 2 is transmitted to the speech synthesis server 5, and the synthesized speech of the material title is voiced. After obtaining from the synthesis server 5, the synthesized voice obtained from the voice synthesis server 5 is output from the speaker 27 (S48), and the procedure executed by the procedure execution unit 21 is shifted to the reproduction procedure T5. If the speech recognition result is not a word indicating affirmation, the procedure execution unit 21 transitions the procedure executed by the procedure execution unit 21 to the material search procedure T1.

  When the selected material cannot be lent out, the procedure execution unit 21 of the electronic library application 20 enters a state of accepting an operation for starting voice input, and when an operation for starting voice input is detected (S35), at this time point The corresponding voice guidance (for example, “Do you reserve material?”) Is output from the speaker 27 (S36), and then the feature recognition unit 22 is activated by specifying voice recognition (S37).

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 of the electronic library application 20 extracts the voice feature quantity input to the microphone 24 by the user and transmits the extracted feature quantity to the voice recognition server 4. The voice recognition result received from the voice recognition server 4 is transferred to the procedure execution unit 21, and the procedure execution unit 21 acquires the voice recognition result of the voice input by the user to the microphone 24 (S38).

  The procedure execution unit 21 of the electronic library application 20 branches the process according to the voice recognition result of the voice input to the microphone 24 by the user (S39). When the speech recognition result is a word indicating affirmation, the procedure execution unit 21 of the electronic library application 20 calls the electronic library service for reserving the material (S40), and the execution result of the electronic library service for reserving the material is stored in the electronic library site 3 Is received from the speech synthesis server 5 by transmitting to the speech synthesis server 5 the title of the material reserved in the electronic library site 3, that is, the title of the material selected in the material selection procedure T2. After that, the synthesized speech acquired from the speech synthesis server 5 is output from the speaker 27 (S41), and the procedure executed by the procedure execution unit 21 is shifted to the material retrieval procedure T1. If the speech recognition result is not a word indicating affirmation, the procedure execution unit 21 transitions the procedure executed by the procedure execution unit 21 to the material search procedure T1.

  FIG. 9 is a diagram for explaining processing executed by the procedure execution unit 21 in the login procedure T4. In the login procedure T4, the procedure execution unit 21 of the electronic library application 20 is in a state of accepting an operation for starting voice input. When the operation for starting voice input is detected (S50), voice guidance corresponding to this time (for example, , "Please tell me your login password") from the speaker 27 (S51), and then designates voice recognition and voiceprint authentication to activate the feature quantity extraction unit 22 (S52).

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 of the electronic library application 20 extracts the voice feature quantity input to the microphone 24 by the user and transmits the extracted feature quantity to the voice recognition server 4. The voice recognition result is received from the voice recognition server 4. The feature amount extraction unit 22 passes the voice input by the user to the microphone 24 to the voiceprint recognition unit, and receives the voiceprint authentication result from the voiceprint authentication unit 23. Then, the feature amount extraction unit 22 delivers the voice authentication result and the voice print authentication result to the procedure execution unit 21, and the procedure execution unit 21 acquires the voice recognition result and the voice print authentication result (S53).

  When the speech recognition result and the voiceprint authentication result are delivered from the feature amount extraction unit 22, the procedure execution unit 21 of the electronic library application 20 first branches the process according to the speech recognition result (S54). If the voice recognition result delivered from the feature quantity extraction unit 22 does not match the login password registered in advance in the procedure execution unit 21, the procedure execution unit 21 provides voice guidance for notifying a login password error from the speaker 27. After outputting (S55), the process returns to the top of FIG.

  If the speech recognition result delivered from the feature quantity extraction unit 22 matches the login password registered in advance in the procedure execution unit 21, the procedure execution unit 21 of the electronic library application 20 receives the feature quantity extraction unit 22 from the feature quantity extraction unit 22. The process branches in accordance with the voiceprint authentication result delivered (S56). When the voiceprint authentication result delivered from the feature quantity extraction unit 22 indicates a voiceprint authentication failure, the procedure execution unit 21 outputs a voice guidance for notifying a voiceprint authentication error from the speaker 27 (S57), and FIG. Return to the top of.

  The procedure execution unit 21 of the electronic library application 20, when the voice print authentication success is indicated by the voice print authentication result delivered from the feature amount extraction unit 22, the voice that becomes the user ID and login password registered in the procedure execution unit 21. When the authentication result is used as a parameter, an electronic library service for logging in to the electronic library site 3 is called (S58), and the execution result of the electronic library service for logging in to the electronic library site 3 is acquired from the electronic library site 3. 3 is transmitted to the speech synthesis server 5 and the synthesized speech of this text is acquired from the speech synthesis server 5, and then the synthesized speech acquired from the speech synthesis server 5 is output from the speaker 27. (S59), the procedure executed by the procedure execution unit 21 To transition to the lending procedures T3.

  FIG. 10 is a diagram for explaining processing executed by the procedure execution unit 21 in the reproduction procedure T5. In the playback procedure T5, the procedure execution unit 21 of the electronic library application 20 is in a state of accepting an operation for starting voice input. When the operation for starting voice input is detected (S60), voice guidance corresponding to this time (for example, , “Do you want to read the borrowed material?” Is output from the speaker 27 (S61), the voice recognition is designated and the feature amount extraction unit 22 is activated (S62).

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 of the electronic library application 20 extracts the voice feature quantity input to the microphone 24 by the user and transmits the extracted feature quantity to the voice recognition server 4. The voice recognition result received from the voice recognition server 4 is delivered to the procedure execution unit 21, and the procedure execution unit 21 acquires the voice recognition result of the voice input by the user to the microphone 24 (S63).

  The procedure execution unit 21 of the electronic library application 20 branches the process according to the voice recognition result of the voice input to the microphone 24 by the user (S64). If the speech recognition result is not a word indicating affirmation, the procedure execution unit 21 of the electronic library application 20 changes the state of the electronic library operation flow to the material search procedure T1. If the speech recognition result is a word indicating affirmation, the procedure execution unit 21 calls the electronic library service that provides the content of the material (S65), and the electronic library service is executed as the execution result of the electronic library service that provides the content of the material. The contents of the materials borrowed from the site 3 are acquired from the electronic library site 3 (S66).

  When the procedure execution unit 21 of the electronic library application 20 acquires the content of the material from the electronic library site 3, the procedure execution unit 21 transmits the text included in the content of the material acquired from the electronic library site 3 to the speech synthesis server 5, The corresponding synthesized voice is acquired, and the synthesized voice acquired from the voice synthesis server 5 is output from the speaker 27 as a voice to reproduce the material borrowed from the electronic library site 3 (S67). The procedure execution unit 21 reproduces the material acquired from the electronic library site 3 in units of a predetermined number of pages. The procedure execution unit 21 executes the procedure executed by the procedure execution unit 21 when the reproduction of the material is completed. Is shifted to the document retrieval procedure T1.

  The procedure execution unit 21 of the electronic library application 20 preferably corresponds to a function capable of operating the reproduction of the material by voice. When the procedure execution unit 21 of the electronic library application 20 supports a function capable of operating the reproduction of the material by voice, the procedure execution unit 21 of the electronic library application 20 receives the user's voice while reproducing the material. It will be in the state which receives and performs reproduction | regeneration operation corresponding to the word which the user input into the microphone 24. FIG. It should be noted that the playback operation of the material can be the end of playback, pause and restart of playback.

  FIG. 11 is a diagram illustrating processing executed by the procedure execution unit 21 in the material return procedure. In FIG. 4, the tab “My Page” is valid only when the user is logged in to the electronic library site 3. In the state where the electronic library site 3 is logged in, the above-described material search procedure T1 is executed, and when the category whose voice recognition result matches is “My Page”, the procedure execution unit 21 of the electronic library application 20 reads “My page”. The display content when the “page” tab is selected is acquired from the electronic library site 3 as a search result, and the processing related to the return procedure T6 shown in FIG. 11 can be executed.

  In the document return procedure, the procedure execution unit 21 of the electronic library application 20 enters a state of accepting an operation for starting voice input, and when detecting the operation for starting voice input (S70), the voice guidance corresponding to the document return procedure ( For example, after “speak the material to be returned” is output from the speaker 27 (S71), the voice recognition is designated and the feature amount extraction unit 22 is operated (S72).

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 of the electronic library application 20 extracts the voice feature quantity input to the microphone 24 by the user and transmits the extracted feature quantity to the voice recognition server 4. The voice recognition result received from the voice recognition server 4 is transferred to the procedure execution unit 21, and the procedure execution unit 21 acquires the voice recognition result of the voice input by the user to the microphone 24 (S73).

  The procedure execution unit 21 of the electronic library application 20 branches the process according to the voice recognition result of the voice input to the microphone 24 by the user (S74). If the speech recognition result delivered from the feature quantity extraction unit 22 is not a material title or material number, the procedure execution unit 21 transitions the procedure executed by the procedure execution unit 21 to the material search procedure T1. If the speech recognition result delivered from the feature quantity extraction unit 22 is either a material title or a material number, the procedure execution unit 21 is identified by the speech recognition result (here, the title or number). After selecting the material to be selected (S75), it is in a state of accepting the operation for starting voice input, and when the operation for starting voice input is detected (S76), the voice guidance corresponding to the material return procedure (for example, “Select” Is output from the speaker 27 (S77), voice recognition is designated and the feature amount extraction unit 22 is activated (S78).

  When the user inputs voice to the microphone 24, the feature quantity extraction unit 22 of the electronic library application 20 extracts the voice feature quantity input to the microphone 24 by the user and transmits the extracted feature quantity to the voice recognition server 4. The voice recognition result received from the voice recognition server 4 is transferred to the procedure execution unit 21, and the procedure execution unit 21 acquires the voice recognition result of the voice input by the user to the microphone 24 (S79).

  The procedure execution unit 21 of the electronic library application 20 branches the process according to the voice recognition result of the voice input to the microphone 24 by the user (S80). When the speech recognition result is an affirmative word, the procedure execution unit 21 of the electronic library application 20 calls the electronic library service that returns the material when the speech recognition result is delivered from the feature amount extraction unit 22 (S81). When the material specified by the recognition result is returned to the electronic library site 3 and the execution result of the electronic library service for returning the material is received from the electronic library site 3, the title of the material returned to the electronic library site 3 is converted into the speech synthesis server 5 And the synthesized speech of the title of the material is acquired from the speech synthesis server 5, and then the synthesized speech acquired from the speech synthesis server 5 is output from the speaker 27 (S82), and the procedure executed by the procedure execution unit 21 Is shifted to the document retrieval procedure T1. If the speech recognition result is not a word indicating affirmation, the procedure execution unit 21 transitions the procedure executed by the procedure execution unit 21 to the material search procedure T1.

DESCRIPTION OF SYMBOLS 1 Electronic library system 2 Portable terminal 20 Electronic library application 21 Procedure execution part 22 Feature-value extraction part 23 Voiceprint authentication part 24 Microphone 25 Motion sensor 26 Touch panel 27 Speaker 3 Electronic library site 4 Speech recognition server 5 Speech synthesis server

Claims (7)

Provide various electronic library services that can be used via the network, and upon receiving a call request for an electronic library service, execute the electronic library service that has received the call request, and provide a device that has requested the electronic library service to be called An electronic library site for transmitting the execution result of the electronic library service;
A mobile terminal in which an application program using the electronic library service provided by the electronic library site is installed;
A voice recognition server that receives voice feature values from the portable terminal, performs voice recognition using the voice feature values, and transmits a voice recognition result in a text format obtained by voice recognition to the portable terminal;
An electronic library system comprising at least a voice synthesis server that receives a text from the portable terminal and generates a synthesized voice of the text and transmits the synthesized voice to the portable terminal,
When the application program starts, the portable terminal
A feature that extracts a feature amount of voice input by a user to a microphone, transmits the feature amount of voice to the voice recognition server, and acquires a voice recognition result corresponding to the voice input by the user to the microphone from the voice recognition server. A quantity extraction unit;
When an electronic library operation flow describing the procedure related to the operation of the electronic library site and its execution order is registered, and when the procedure described in the electronic library operation flow is executed, an operation to start voice input is detected , By operating the feature amount extraction unit to acquire the speech recognition result of the voice input by the user to the microphone from the feature amount extraction unit, and then using the speech recognition result acquired from the feature amount extraction unit, The electronic library service corresponding to the procedure at the time is called, and when the execution result of the called electronic library service is received from the electronic library site, it is received from the electronic library site using the speech synthesis server. It has a procedure execution unit that generates synthesized speech corresponding to the execution result and outputs it.
An electronic library system characterized by this.   When the procedure execution unit provided in the portable terminal by starting the application program detects an operation to start voice input, the procedure guidance unit outputs voice guidance related to the procedure at this time, and then extracts the feature amount. The electronic library system according to claim 1, wherein the electronic library system operates.   The portable terminal includes a motion sensor, and the procedure execution unit included in the portable terminal detects that the portable terminal is shaken as an operation of starting voice input when the application program is activated. The electronic library system according to claim 1, wherein the electronic library system is characterized.   Calls the electronic library service to retrieve materials, retrieves materials that match the speech recognition results obtained from the feature extraction unit, and outputs the synthesized speech of the text included in the retrieval results of the material retrieval procedures Then, the electronic library service for selecting the material is called, the material selection procedure for selecting the material corresponding to the speech recognition result acquired from the feature quantity extraction unit, the electronic library service for logging in to the electronic library site, Using the speech recognition result acquired from the feature extraction unit, the login procedure for logging in to the electronic library site, the electronic library service for renting materials are called, and the material selected by the material selection procedure is selected from the electronic library. Lending procedure to borrow from the site, the electronic book providing the contents of the material Calling a service and executing the reproduction procedure of reproducing the text included in the content of the material borrowed from the electronic library site in the lending procedure in order by synthesizing the sound is an operation of borrowing the material from the electronic library site and reading it. The electronic library system according to any one of claims 1 to 3, wherein the procedure and the execution order thereof are described in the electronic library operation flow.   When the material borrowed by the user is searched in the material selection procedure, the electronic library service that returns the material is called, and the material specified by the voice recognition result acquired from the feature amount extraction unit is stored in the electronic library site. 5. The electronic library system according to claim 4, wherein execution of a return procedure to return to the electronic library is described in the electronic library operation flow as the procedure relating to the return operation of the material and the execution order thereof.   In the lending procedure, if the material selected in the material selection procedure cannot be lent, the electronic library service that reserves the material is called, and the reservation procedure for reserving the material selected in the material selection procedure to the electronic library site 6. The electronic library system according to claim 4, wherein execution of the electronic library is described in the electronic library operation flow as the procedure relating to the reservation operation of the material and the execution order thereof. When the application program is activated, the portable terminal includes a voice print authentication unit that authenticates a voice print of a voice input by a user to the microphone, and the procedure execution unit is configured to perform a voice test of the voice input by the user to the microphone in the login procedure. Calling the electronic library service for logging into the electronic library site only when the voiceprint authentication unit succeeds in voiceprint authentication,
The electronic library system according to any one of claims 4 to 6.

Images