JP2020201435A - Information processing terminal and information processing method - Google Patents

Abstract

To realize an information processing terminal which can more preferably grasp a document composed of multiple pages through reading aloud and provide an experience of enjoying sentences for a visually impaired person.SOLUTION: An information processing terminal (1) includes a camera (11) which captures each page of a document composed of a plurality of pages, a controller (12) which executes text data generation processing and audio signal generation processing, an output unit which outputs the audio signal, and one or both of speakers which converts and outputs the audio signal into a sound wave.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing terminal. The present invention also relates to a support for mounting an information processing terminal. The present invention also relates to an information processing method.

Documents used by the visually impaired include Braille books, which are converted from printed books into Braille, transliterated books, which are read aloud from printed books, enlarged books, which are published by enlarging the characters and figures of the printed books. Electronic books are known (Patent Document 1).

"About reading for the visually impaired", [online], Social Welfare Corporation Japan Blind Association, [Search on May 27, 1st year of Reiwa], Internet <URL: http://nichimou.org/morebooks/reading/>

However, Braille books, transliterated books, and enlarged books require time to be created, and there is a problem that these books are available to the visually impaired at a late time. Furthermore, there is a problem that Braille books and transliterated books are operated by relying on volunteers from the viewpoint of cost. In addition, there is a problem that enlarged books and electronic books cannot be used by blind people. Furthermore, although these are intended for commercially available books, they cover a wide range of printed matter such as academic treatises, public relations magazines published by local governments, newspapers, newspaper inserts, and pamphlets. It is not the situation that has been done. Therefore, it is desired to develop a more suitable tool for reading aloud sentences for the visually impaired.

One aspect of the present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a visually impaired person with an experience of grasping a document composed of a plurality of pages through reading aloud and enjoying a sentence. The purpose is to realize an information processing terminal that can be provided.

In order to solve the above-mentioned problems, the information processing terminal according to one aspect of the present invention refers to a camera that captures each page of a document composed of a plurality of pages and image data obtained by the camera. Then, a text data generation process for generating text data representing the text described on the page and a voice signal generation process for generating a voice signal representing the voice read out from the text are executed with reference to the text data. The controller includes an output unit that outputs the audio signal, and one or both of a speaker that converts the audio signal into sound waves and outputs the data.

In order to solve the above-mentioned problems, the support according to one aspect of the present invention is the support that supports the information processing terminal so that the pages of the book and the camera lens of the information processing terminal face each other. A top plate on which a terminal is placed, the top plate having an open portion overlapping with the camera lens, and a side plate whose upper end is connected to the outer edge of the top plate and whose lower end is pressed against the outer edge of the page. And have.

In order to solve the above-mentioned problems, the information processing method according to one aspect of the present invention uses a plurality of information processing terminals including a camera, a controller, an output unit, and one or both of speakers. An information processing method for outputting a document composed of pages as audio, wherein the camera obtains an imaging process for capturing each page of a document composed of a plurality of pages, and the controller is obtained by the camera. A text data generation process that generates text data representing the text described on the page by referring to the image data, and a voice signal representing a voice that the controller reads the text by referring to the text data. Includes an audio signal generation process for generating the audio signal, and an audio output process for the output unit to output the audio signal and / or the speaker to convert the audio signal into sound waves and output the data.

According to one aspect of the present invention, it is possible to realize an information processing terminal capable of more preferably grasping a document composed of a plurality of pages through reading aloud and providing an experience of enjoying a sentence to a visually impaired person. it can.

It is a figure which shows the outline of the usage method of the information processing terminal which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the information processing terminal which concerns on one Embodiment of this invention. It is a flowchart which shows the flow of the information processing method implemented using the information processing terminal shown in FIG. It is a perspective view which shows the structure of the support of the information processing terminal which concerns on one Embodiment of this invention. It is a development view which shows the structure of the support of the information processing terminal which concerns on one Embodiment of this invention.

[Basic concept]
The basic concept of this embodiment will be described with reference to FIG. FIG. 1 is a perspective view showing an outline of how to use the information processing terminal 1 and the support 2 according to the present embodiment. Examples of the information processing terminal 1 include a smartphone with a camera.

First, as shown in 1001 of FIG. 1, a user opens a page of a document composed of a plurality of pages including a book. Examples of documents composed of a plurality of pages include printed books such as new books and paperback books, academic treatises, public relations magazines issued by local governments, newspapers, newspaper insert advertisements, pamphlets, and the like. The user then places the support 2 on the opened page, as shown in 1002 of FIG. As a result, the lower end of the side plate 22 of the support 2 is pressed against the outer edge of the opened page, and as a result, the opened page becomes a flat state suitable for imaging. After that, as shown in 1003 of FIG. 1, the user places the information processing terminal 1 on the top plate 21 of the support 2. As a result, the camera lens of the information terminal device 1 faces the page opened through the opening 21a provided in the top plate 21, and the distance from the camera lens of the information terminal device 1 to the opened page is the camera lens. Will be included in the depth of field of. The information processing terminal 1 captures an image of a page opened in this state, and reads out the text described on the page with reference to the obtained image data. As a result, the user can listen to the text written on the opened page as a voice.

[Information processing terminal]
The configuration of the information processing terminal 1 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the information processing terminal 1. Hereinafter, a case where a document composed of a plurality of pages is a book will be described as an example.

An application program (hereinafter, referred to as a "reading application") for reading aloud the text written on a page of a book is installed in the information processing terminal 1. The information processing terminal 1 has a camera 11, a controller 12, an output terminal 13 (output unit), a speaker 14, a memory 15, and a communication interface as hardware resources used to execute a reading process according to a reading application. 16 and. The information processing terminal 1 can be realized by, for example, a smartphone. The reading application may be stored in the memory 15 or may be stored in another recording medium.

The camera 11 is configured to capture a page of a book. As the camera 11, for example, a digital camera composed of a camera lens, an image sensor, and an image signal processing circuit is used. As the memory 15, for example, one or a plurality of semiconductor RAMs (random access memory) are used.

The controller 12 is configured to execute the reading process according to the reading application. As the controller 12, for example, one or a plurality of CPUs (Central Processing Units) are used. The reading process includes at least a text data generation process and an audio signal generation process. Here, the text data generation process refers to a process of generating text data representing the text described on a page of a book by referring to the image data obtained by the camera 11. Further, the audio signal generation process refers to a process of referring to the text data generated by the text data generation process and generating an audio signal representing the voice obtained by reading the text described on the page of the book. The details of the reading process will be described later instead of the reference drawings.

The output terminal 13 is configured to output the voice signal generated by the voice signal generation process to the outside as an electric signal. Examples of the output destination of the audio signal include the earphone 3. The speaker 14 is configured to output the audio signal generated by the audio signal generation process to the outside as a sound wave. The information processing terminal 1 may include only the output terminal 13, only the speaker 14, or both the output terminal 13 and the speaker 14. Further, the information processing terminal 1 may be provided with a short-range wireless communication interface instead of the output terminal 13 which is a wired interface, or may be provided with a wireless interface in addition to the output terminal 13. Examples of the short-range wireless communication interface include a Bluetooth (registered trademark) interface. When the information processing terminal 1 is provided with a short-range wireless interface, audio can be output to earphones, headphones, speakers, or the like that support short-range wireless communication instead of the earphone 3 used by connecting to the output terminal 13. ..

The communication interface 16 is an interface for communicating with the server. The controller 12 can execute a part of the reading process in cooperation with the server by communicating with the server via the communication interface 16. A specific example of the process executed by the controller 12 in cooperation with the server will be described later instead of the reference drawing.

[Reading process]
The flow of the reading process S1 executed by the information processing terminal 1 according to the reading application will be described with reference to FIG. FIG. 3 is a flowchart showing the flow of reading processing S1 (information processing method for outputting a document composed of a plurality of pages as voice).

As shown in FIG. 3, the reading process S1 includes an image pickup process S10, an inverted image determination process S11, an inverted image notification process S12, a cover surface determination process S13, a text data generation process S14, and an audio signal generation process. S15, audio output processing S16, vertical / horizontal determination processing S17, text data generation processing S18, character underpage determination processing S19, character underpage notification processing S20, audio signal generation processing S21, and audio output processing S22. , Including.

The imaging process S10 is a process of imaging a page of a book. The image pickup process S10 is executed by the camera 11 controlled by the controller 12. The timing of executing the imaging process S10 may be the timing when the user instructs the imaging by a voice command, or the timing when the camera 11 succeeds in focusing on the page of the book.

The upright image determination process S11 is a process of determining whether or not the image represented by the image data obtained in the imaging process S10 is an upright image (whether it is an upright image). Here, the inverted image refers to an image in which the upper part of the page is arranged at the lower part of the screen and the lower part of the page is arranged at the upper part of the screen. The upright image refers to an image in which the upper part of the page is arranged at the upper part of the screen and the lower part of the page is arranged at the lower part of the screen. The inverted image determination process S11 is executed by the controller 12.

When the inverted image determination process S11 determines that the image is an inverted image, the inverted image notification process S12 is executed. The inverted image notification process S12 is a process of notifying the user that the image data obtained in the imaging process S10 represents an inverted image. In order to execute the inverted image notification process S12, the controller 12 outputs, for example, an audio signal "the book is upside down" to the output terminal 13 or the speaker 14. When the controller 12 outputs the audio signal to the output terminal 13, the output terminal 13 outputs the audio signal to the earphone 3. The earphone 3 converts the audio signal into sound waves and outputs the signal. When the controller 12 outputs the audio signal to the speaker 14, the speaker 14 converts the audio signal into sound waves and outputs the signal. This can prompt the user to rotate the book 180 ° to the correct orientation.

When the controller 12 determines that the image is an inverted image in the inverted image determination process S11, the controller 12 rotates the inverted image by 180 ° to obtain an upright image, and proceeds to the cover determination process S13 described later. It may be.

When the upright image determination process S11 determines that the image is upright, the cover determination process S13 is executed. The cover page determination process S13 is a process of determining whether or not the page included as a subject in the image represented by the image data obtained in the image pickup process S10 is the cover page (whether it is other than the cover page). The cover determination process S13 is executed by the controller 12 of the information processing terminal 1.

When it is determined in the cover page determination process S13 that the page is the cover page, the text data generation process S14 is executed. The text data generation process S14 is a process of generating text data representing the text described on the page by referring to the image data obtained in the imaging process S10. The text data generation process S14 is executed by the controller 12 of the information processing terminal 1. In particular, in the text data generation process S14 executed when the page is the cover of a book, the controller 12 uses the title data representing the title described on the cover and the author name described on the cover as text data. Generate author name data to represent. The generation of the title data and the author name data is realized by, for example, the following method.

Method 1: The character string described in the largest font among the character strings written on the cover is specified as the title, and is described in the second largest font among the character strings written on the cover. The character string is specified as the author name. Then, the text data representing the specified title and author name is used as the title data and the author name data.

Method 2: A machine-learned classifier (AI) is used to input image data representing the cover of a book and output the title and author name representing the title and author name of the book. In this case, when the image data obtained in the imaging process S10 is input to this classifier, the text data representing the title and author name output from this classifier is used as the title data and the author name data. The classifier may be mounted on the information processing terminal 1 or may be mounted on a server capable of communicating with the information processing terminal 1 via the communication interface 16. In the latter case, the information processing terminal 1 uses a classifier mounted on the server to specify the title and the author name described on the cover page.

Method 3: Use a title database in which titles of various books are registered and an author name database in which author names of various books are registered. In this case, the character string that matches the title registered in the title database is specified as the title in the character string written on the cover, and is registered in the author name database in the character string written on the cover. The character string that matches the author name is specified as the author name. Then, the text data representing the specified title and author name is used as the title data and the author name data. These databases may be implemented in the information processing terminal 1 or may be implemented in a server capable of communicating with the information processing terminal 1 via the communication interface 16. In the latter case, the information processing terminal 1 uses the database implemented in the server to specify the title and the author name described on the cover page.

It is preferable that the author name in hiragana notation or katakana notation is registered in the author name database together with the author name in kanji notation. As an example, it is preferable that the author name "Kosawa Ryota" written in hiragana is registered together with the author name "Ryota Kosawa" written in kanji. In this case, if the author name data is generated based on the author name in hiragana notation, the voice data in which the author name is correctly read aloud can be generated in the voice signal generation process S15 described later. The same is true for the title database.

Note that the controller 12 may execute the text data generation process S14 after the page is out of the image pickup target area of the camera 11 with reference to the image data obtained in the image pickup process S10.

The audio signal generation process S15 is a process of referring to the text data generated in the text data generation process S14 to generate an audio signal representing the voice of reading the text described on the page. The audio signal generation process S15 is executed by the controller 12 of the information processing terminal 1 after the text data generation process S14 is executed. In particular, in the audio signal generation process S15 executed when the page is the front page, the controller 12 uses the controller 12 as an audio signal to represent the audio signal reading the author's name before or after the audio signal representing the audio reading the title. Is added to generate an audio signal.

In the audio signal generation process S15, the controller 12 may execute the audio signal generation process after the page is out of the image pickup target area of the camera 11 with reference to the image data obtained in the image pickup process S10. Good. Further, in the audio signal generation process S15, the controller 12 may generate an audio signal representing the audio that reads the text written on the page at the reading speed specified by the user as the audio signal.

The audio output processing S16 is a process of outputting the audio signal generated by the audio signal generation processing S15 as an electric signal from the output terminal 13, and the audio signal generated by the audio signal generation processing S15 as a sound source from the speaker 14. One or both of the output processes. The audio output process S16 is realized by the controller 12.

When it is determined in the cover page determination process S13 that the page is other than the cover page, the vertical / horizontal determination process S17 is executed. The vertical / horizontal determination process S17 is a process for determining whether the text described on the page is vertical text or horizontal text. The vertical / horizontal determination process S17 is executed by the controller 12.

The text data generation process S18 is a process of generating the text data described on the page by referring to the result of the vertical / horizontal determination process S17 and the image data obtained in the image pickup process S10. The text data generation process S18 is executed by the controller 12 of the information processing terminal 1. In particular, in the text data generation process S18 executed when the page is other than the cover page, the controller 12 uses the page number data representing the page number described on the page and the text described on the page as text data. Generate text data to represent.

In the text data generation process S18, the controller 12 adds the unfinished word to the beginning of the text of the current page as the text data (1) when the end of the text of the previous page is an unfinished word, and (2) presents it. When the end of the page body is an unfinished word, the body data representing the unfinished word deleted from the end of the current page body may be generated. For example, when "Ra" is written at the end of the text of the previous page and "Men" is written at the beginning of the text of the current page, that is, when the word "Ramen" is written over two pages. The controller 12 generates text data representing a text in which "Ra" is added to the beginning of the text of the current page. In addition, when "Cha" is written at the end of the text of the current page and "Han" is written at the beginning of the text of the next page, that is, the word "fried rice" is written over two pages. If so, the controller 12 generates text data representing the one in which the "char" is deleted from the end of the text of the current page. In the text data generation process S18, the algorithm for determining whether or not the end of the text is an unfinished word is not particularly limited, and a known algorithm can be used.

Further, in the text data generation process S18, the controller 12 may specify a heading included in the text data. As a method of specifying the heading, for example, among the character strings included in the text data, a method of specifying a character string in bold or a character string having a space around it as a heading can be mentioned. ..

The character underpage determination process S19 is a process of determining whether or not the number of characters of the text written on the page is less than a predetermined threshold value. The character underpage determination process S19 is executed by the controller 12.

When it is determined in the character underpage determination process that the number of characters is less than the threshold value, the character underpage notification process S20 is executed. The character shortage page notification process S20 is a process of notifying the user that the page has a small number of characters in the described text. In order to execute the character under-page notification process S20, the controller 12 outputs, for example, an audio signal that "the entire page is shown in the figure" to the output terminal 13 or the speaker 14. When the controller 12 outputs the audio signal to the output terminal 13, the output terminal 13 outputs the audio signal to the earphone 3. The earphone 3 converts the audio signal into sound waves and outputs the signal. When the controller 12 outputs the audio signal to the speaker 14, the speaker 14 converts the audio signal into sound waves and outputs the signal. As a result, the user can recognize that the figure is described on the page, and thus can prompt the user to move to the next page of shooting.

The audio signal generation process S21 is a process of referring to the text data generated in the text data generation process S18 to generate an audio signal representing a voice reading the text described on the page. The audio signal generation process S21 is executed by the controller 12. In particular, in the audio signal generation process S21 executed when the page is other than the front page, the controller 12 uses the audio signal representing the audio signal reading the page number followed by the audio signal representing the audio signal reading the text as the audio signal. Generate the added audio signal.

In the voice signal generation process S21, the controller 12 may generate a voice signal representing a voice in which the text is read aloud so that the heading can be discerned. For example, the voice that reads out the text "... <Line break> (3) Departure <Line break> I finally decided. Today is ..." is "(pause)." ) Ms. Midashi (pause) (pause) Every time (pause) I'm finally sick (pause) Today ... ". As another example, voice data obtained by reading the headline and other parts with different voices (for example, the headline is a male voice and the other parts are female voices) may be used.

The audio output processing S22 is a process of outputting the audio signal generated by the audio signal generation processing S21 as an electric signal from the output terminal 13, and the audio signal generated by the audio signal generation processing S21 as a sound source from the speaker 14. One or both of the output processes. The audio output process S22 is executed by the controller 12.

In the voice output processing S22, when the controller 12 acquires a voice command instructing the reading stop from the user, the controller 12 stops the output of the voice signal, and when the voice command instructing the resumption of reading is obtained from the user, the controller 12 outputs the voice signal. To resume. For example, when the controller 12 acquires the voice command "stop" from the user, the controller 12 stops the output of the voice signal. Further, when the voice command "restart" is acquired from the user, the output of the voice signal is restarted. Alternatively, the controller 12 stops the output of the audio signal by the operation of tapping the screen of the information processing terminal 1 by the user, and further restarts the output of the audio signal by the operation of tapping the screen of the information processing terminal 1 by the user. May be good. The operation of tapping the screen may be an operation of tapping a specific GUI element (button or the like) provided on the screen, or an operation of tapping an arbitrary place on the screen.

As described above, the reading process S1 according to the present embodiment refers to the image data obtained by the camera 11 and generates text data representing the text described on the page. Text data generation processes S14, S18, In addition, the voice signal generation processes S15 and S21 for generating a voice signal representing the voice read aloud the text with reference to the text data are included. Therefore, according to the reading process S1 according to the present embodiment, it is possible to more preferably provide a visually impaired person with an experience of grasping a document composed of a plurality of pages through reading aloud and enjoying the text. Specifically, regardless of the degree of visual impairment, a person with severe visual impairment, a person with mild visual impairment, an elderly person with impaired vision, or a healthy person. It is also possible to read. In addition, the cost is lower than that of Braille books, transliteration books, and audio books. In addition, since it is possible to read using printed books, it is possible to read a wide variety of books immediately. As described above, the information processing terminal 1 according to the present embodiment has specifications that are thoroughly easy for users such as visually impaired people to use.

The reading process S1 may include a maximum page number notification process in addition to the above-mentioned processes. The maximum page number notification process is a process of notifying the maximum page number among the page numbers represented by the page number data stored in the memory 15. The maximum page number notification process is executed by the controller 12 of the information processing terminal 1.

In order to execute the maximum page number notification process, the controller 12 outputs, for example, an audio signal "read up to 30 pages" to the output terminal 13 or the speaker 14. When the controller 12 outputs the audio signal to the output terminal 13, the output terminal 13 outputs the audio signal to the earphone 3. The earphone 3 converts the audio signal into sound waves and outputs the signal. When the controller 12 outputs the audio signal to the speaker 14, the speaker 14 converts the audio signal into sound waves and outputs the signal. This makes it possible for the user to recognize how many pages have been read.

[Structure of support]
The configuration of the support 2 according to the present embodiment will be described with reference to FIGS. 4 and 5. FIG. 4 is a perspective view showing the configuration of the support 2. FIG. 5 is a developed view showing the configuration of the support 2. The method of using the support 2 is as described above.

The support 2 is a stand for supporting the information processing terminal 1 so that the pages of the book and the camera lens of the information processing terminal 1 face each other. The support 2 includes one top plate 21 and four side plates 22.

The top plate 21 is a plate-shaped member that constitutes the upper surface of the support 2. In the present embodiment, the top plate 21 has a rectangular shape in a plan view. The information processing terminal 1 is placed on the top plate 21. The top plate 21 is formed with an opening 21a for opening a portion overlapping the camera lens of the information processing terminal 1. The shapes and sizes of the top plate 21 and the opening 21a are optimized according to the type of the target information processing terminal 1. For example, the shape and size of the top plate 21 are set according to the shape and size of the back surface of the information processing terminal 1. The shape and size of the opening 21a are set according to the position of the camera lens on the back surface of the information processing terminal 1.

The side plate 22 is a plate-shaped member that constitutes the side surface of the support 2, and the upper end of the side plate 22 is connected to the outer edge of the top plate 21. In the present embodiment, the side plate 22 has a trapezoidal shape in a plan view. The lower end of the side plate 22 is pressed against the outer edge of the book page. The shape and size of the side plate 22 are optimized according to the type of the target information processing terminal 1 and the type of the target book. For example, the distance from the top plate 21 to the lower end of the side plate 22 measured along the direction orthogonal to the top plate 21 is included in the depth of field of the camera lens of the target information processing terminal 1, and the side plate. The lower end of 22 is set to be pressed against the outer edge of the target book page.

In the present embodiment, each side plate 22 is deployably connected to the outer edge of the top plate 21. When each side plate 22 is expanded and the angle formed by each side plate 22 and the top plate 21 is 180 °, the support 2 is flattened as shown in FIG. On the contrary, when each side plate 22 is folded and the angle formed by each side plate 22 and the top plate 21 is a predetermined angle (less than 180 °), the support 2 is three-dimensionalized as shown in FIG. When selling the support 2 and when storing the support 2, it is preferable to expand and flatten the support 2 in order to improve portability.

It is preferable that one or both of the top plate 21 and the side plate 22 is made of a translucent material. As a result, the illuminance of the page to be imaged can be improved by utilizing the external light transmitted through the top plate 21 or the side plate 22. When the user uses the information processing terminal 1 having a light source on the same surface as the camera lens, the information processing terminal 1 may be used to secure the illuminance of the page on which the light source is to be imaged.

Further, one or both of the top plate 21 and the side plate 22 have a structure 22a that palpably indicates the type of book suitable for use of the support 2, and a type of information processing terminal suitable for use of the support 2. It is preferable that one or both of the structures 22b are provided so as to be palpable. As a result, even a visually impaired user can feel whether or not the support 2 is suitable for the target information processing terminal 1 and / or a book. Note that these structures 22a and 22b can be realized, for example, by the unevenness formed on the surface of the top plate 21 or the side plate 22.

[Example of realization by software]
The controller 12 of the information processing terminal 1 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the information processing terminal 1 includes a computer that executes instructions of a program that is software that realizes each function. The computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, in addition to a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the above program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. It should be noted that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

[Summary]
The information processing terminal according to the first aspect of the present invention refers to a camera that captures each page of a document composed of a plurality of pages and image data obtained by the camera, and the text described on the page. A controller that executes a text data generation process that generates text data representing the text, and a voice signal generation process that generates a voice signal representing the voice that reads the text by referring to the text data, and outputs the voice signal. It is configured to include an output unit for information processing and one or both of a speaker that converts the audio signal into sound waves and outputs the data.

In the information processing terminal according to the second aspect of the present invention, in the first aspect, the controller refers to the image data and performs the text data generation process after the page is out of the imaging target area of the camera. It may be executed, or the controller may execute the voice signal generation process after the page is out of the image pickup target area of the camera with reference to the image data.

The information processing terminal according to the third aspect of the present invention is described on the page in the above aspect 1 or 2 in the audio signal generation process, in which the controller describes the audio signal as the audio signal at a reading speed specified by the user. It may be configured to generate an audio signal representing the audio read out from the text.

In the information processing terminal according to the fourth aspect of the present invention, in any one of the above aspects 1 to 3, in the text data generation process, the controller uses the page number described on the page as the text data. The page number data to be represented and the text data representing the text described on the page are generated, and in the voice signal generation process, the controller uses the voice signal as the voice signal to represent the voice reading the page number. Later, a voice signal to which a voice signal representing the voice reading the text is added may be generated.

In the information processing terminal according to the fifth aspect of the present invention, in any one of the above aspects 1 to 4, in the text data generation process, the controller uses the text data as (1) the end of the text on the previous page. If it is an unfinished word, the unfinished word is added to the beginning of the current page text, and (2) if the end of the current page text is an unfinished word, the unfinished word is deleted from the end of the current page text. It may be configured to generate text data.

In the information processing terminal according to the sixth aspect of the present invention, in any one of the above aspects 1 to 5, in the text data generation process, the controller identifies a heading included in the text data and the voice signal. In the generation process, the controller may be configured to generate, as the voice signal, a voice signal representing the voice in which the text is read aloud so that the heading can be discerned.

In the information processing terminal according to the seventh aspect of the present invention, in any one of the above aspects 1 to 6, when the page is the cover of a document composed of the plurality of pages, in the text data generation process, The controller generates title data representing the title described on the cover page and author name data representing the author name described on the cover page as the text data, and in the voice signal generation process, the controller generates the title data. , The audio signal may be configured to generate an audio signal to which an audio signal representing the audio reading the author's name is added before or after the audio signal representing the audio reading the title.

The information processing terminal according to the eighth aspect of the present invention is the character string described in the largest font among the character strings described on the cover page in the text data generation process in the above aspect 7. May be specified as the title, and the character string described in the second largest font among the character strings described on the cover may be specified as the author name.

The information processing terminal according to the ninth aspect of the present invention receives image data representing the cover of a document composed of a plurality of pages as input in the above aspect 7, and the title and author name of the document composed of the plurality of pages. A classifier that outputs text data representing the above, or a communication interface for communicating with a server equipped with the classifier is further provided, and in the text data generation process, the controller is an image obtained by the camera. The title data and the author name data may be generated from the text data output from the classifier while inputting the data.

In the above aspect 7, the information processing terminal according to the tenth aspect of the present invention further includes a database in which a plurality of author names are registered in advance, or a communication interface for communicating with a server provided with the database, and the text. In the data generation process, the controller may have a configuration in which the character string registered as the author name in the database is used as the author name data among the character strings described on the cover page.

In the information processing terminal according to the eleventh aspect of the present invention, in any one of the above aspects 1 to 10, the controller has the inverted image of the page represented by the image data obtained by the camera. Inverted image determination processing for determining whether or not, and inverted image notification processing for notifying the user when the image is determined to be the inverted image in the inverted image determination process. It may be configured to be further executed.

In the information processing terminal according to the twelfth aspect of the present invention, in any one of the above aspects 1 to 11, the controller determines whether or not the number of characters of the text described on the page is less than a predetermined threshold value. If it is determined by the character under-page determination process and the character under-page determination process that the number of characters is less than the threshold value, the character underpage notification process for notifying the user to that effect is further executed. Good.

The information processing terminal according to the thirteenth aspect of the present invention further includes a memory for storing page number data representing the page number described on the page in any one of the above aspects 1 to 12, and the controller includes a memory. Among the page numbers represented by the page number data stored in the memory, the maximum page number notification process for notifying the maximum page number may be further executed.

In the information processing terminal according to the fourteenth aspect of the present invention, in any one of the above aspects 1 to 13, the controller determines whether the text described on the page is vertical text or horizontal text. The vertical / horizontal determination process for determination may be further executed, and the controller may execute the text data generation process with reference to the result of the vertical / horizontal determination process.

In any one of the above aspects 1 to 14, the information processing terminal according to the fifteenth aspect of the present invention stops the output of the audio signal when the controller acquires a voice command instructing to stop reading from the user. , When a voice command instructing the resumption of reading is obtained from the user, the output of the voice signal may be restarted.

The support according to aspect 16 of the present invention is a support that supports the information processing terminal so that each page of a document composed of a plurality of pages faces the camera lens of the information processing terminal. A top plate on which a top plate is placed, the portion of which overlaps with the camera lens is open, and a side plate whose upper end is connected to the outer edge of the top plate and whose lower end is pressed against the outer edge of the page. , Is a configuration.

The support according to the 17th aspect of the present invention may be configured such that one or both of the top plate and the side plate has translucency in the 16th aspect.

The support according to the aspect 18 of the present invention may have a configuration in which the side plate is deployably connected to the top plate in the above aspect 16 or 17.

In the support according to the 19th aspect of the present invention, the distance from the top plate to the lower end of the side plate measured along the direction orthogonal to the top plate in any one of the above aspects 16 to 18 is determined. The configuration may be included in the depth of field of the camera lens.

In any one of the above aspects 16 to 19, the information processing terminal support according to the 20th aspect of the present invention has a plurality of supports suitable for use on one or both of the top plate and the side plates. One or both of a structure that palpably indicates the type of the document composed of the above pages and a structure that palpably indicates the type of the information processing terminal suitable for the use of the support are provided. May be.

The information processing method according to aspect 21 of the present invention uses an information processing terminal including a camera, a controller, an output unit, and one or both of speakers to use a document composed of a plurality of pages as audio. An information processing method for outputting, wherein the camera takes an image of each page of a document composed of a plurality of pages, and the controller refers to the image data obtained by the camera. A text data generation process that generates text data representing the text described on the page, and a voice signal generation process in which the controller refers to the text data and generates a voice signal representing the voice that reads out the text. The output unit may include an audio output process in which the audio signal is output and / or the speaker converts the audio signal into sound waves and outputs the audio signal.

[Additional notes]
The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

1 Information processing terminal 2 Support 3 Earphone 11 Camera 12 Controller 13 Output terminal (output unit)
14 Speaker 15 Memory 21 Top plate 22 Side plate

Claims (21)

A camera that captures each page of a document consisting of multiple pages,
A text data generation process for generating text data representing the text described on the page with reference to the image data obtained by the camera, and a voice reading the text with reference to the text data. A controller that executes audio signal generation processing that generates the represented audio signal,
It includes an output unit that outputs the audio signal, and one or both of a speaker that converts the audio signal into sound waves and outputs the sound wave.
An information processing terminal characterized by this. The controller refers to the image data and executes the text data generation process after the page is out of the imaging target area of the camera, or
The controller refers to the image data and executes the audio signal generation process after the page is out of the imaging target area of the camera.
The information processing terminal according to claim 1, wherein the information processing terminal is characterized by the above. In the voice signal generation process, the controller generates, as the voice signal, a voice signal representing the voice reading the text described on the page at a reading speed specified by the user.
The information processing terminal according to claim 1 or 2. In the text data generation process, the controller generates page number data representing the page number described on the page and text data representing the text described on the page as the text data.
In the audio signal generation process, the controller generates an audio signal as the audio signal by adding an audio signal representing the audio that reads the text to the audio signal that represents the audio that reads the page number.
The information processing terminal according to any one of claims 1 to 3, wherein the information processing terminal is characterized by the above. In the text data generation process, the controller adds the unfinished word to the beginning of the text of the current page as the text data (1) when the end of the text of the previous page is an unfinished word, and (2) the current page. If the end of the text is an unfinished word, the text data representing the unfinished word deleted from the end of the text of the current page is generated.
The information processing terminal according to any one of claims 1 to 4, wherein the information processing terminal is characterized by the above. In the text data generation process, the controller identifies a heading included in the text data.
In the audio signal generation process, the controller generates, as the audio signal, an audio signal representing the voice in which the text is read aloud so that the heading can be discerned.
The information processing terminal according to any one of claims 1 to 5, wherein the information processing terminal is characterized. When the page is the cover of a document composed of the plurality of pages
In the text data generation process, the controller generates title data representing the title described on the cover page and author name data representing the author name described on the cover page as the text data.
In the audio signal generation process, the controller generates an audio signal to which an audio signal representing the voice reading the author's name is added before or after the audio signal representing the voice reading the title as the audio signal. ,
The information processing terminal according to any one of claims 1 to 6, characterized in that. In the text data generation process, the controller specifies the character string described in the largest font among the character strings described on the cover page as the title, and among the character strings described on the cover page. Then, specify the character string written in the second largest font as the author name,
The information processing terminal according to claim 7, wherein the information processing terminal is characterized by the above. A classifier that inputs image data representing the cover of a document composed of a plurality of pages and outputs text data representing the title and author name of the document composed of the plurality of pages, or a classifier. It also has a communication interface for communicating with the server.
In the text data generation process, the controller inputs the image data obtained by the camera and generates the title data and the author name data from the text data output from the classifier.
The information processing terminal according to claim 7, wherein the information processing terminal is characterized by the above. A database in which multiple author names are registered in advance, or a communication interface for communicating with a server equipped with the database is further provided.
In the text data generation process, the controller uses the character string registered as the author name in the database as the author name data among the character strings described on the cover page.
The information processing terminal according to claim 7, wherein the information processing terminal is characterized by the above. The controller has an inverted image determination process for determining whether or not the image represented by the image data obtained by the camera is an inverted image of the page, and the inverted image determination process for the image. When it is determined that the image is an inverted image, the inverted image notification process for notifying the user to that effect is further executed.
The information processing terminal according to any one of claims 1 to 10. When the controller determines whether or not the number of characters of the text described on the page falls below a predetermined threshold value, the character under-page determination process and the character under-page determination process show that the number of characters falls below the threshold value. If it is determined, the character underpage notification process for notifying the user to that effect is further executed.
The information processing terminal according to any one of claims 1 to 11, characterized in that. A memory for storing page number data representing the page number described on the page is further provided.
The controller further executes a maximum page number notification process for notifying the maximum page number among the page numbers represented by the page number data stored in the memory.
The information processing terminal according to any one of claims 1 to 12, characterized in that. The controller further executes a vertical / horizontal determination process for determining whether the text described on the page is vertical text or horizontal text.
The controller executes the text data generation process with reference to the result of the vertical / horizontal determination process.
The information processing terminal according to any one of claims 1 to 13, wherein the information processing terminal is characterized. When the controller acquires a voice command instructing to stop reading from the user, the output of the voice signal is stopped, and when a voice command instructing to resume reading is obtained from the user, the controller resumes the output of the voice signal.
The information processing terminal according to any one of claims 1 to 14, wherein the information processing terminal is characterized. In a support that supports the information processing terminal so that each page of a document composed of a plurality of pages and the camera lens of the information processing terminal face each other.
A top plate on which the information processing terminal is placed, the top plate having an open portion overlapping with the camera lens, and
It is provided with a side plate whose upper end is connected to the outer edge of the top plate and whose lower end is pressed against the outer edge of the page.
A support characterized by that. One or both of the top plate and the side plate is translucent.
16. The support according to claim 16. The side plate is deployably connected to the top plate,
The support according to claim 16 or 17. The distance from the top plate to the lower end of the side plate measured along the direction orthogonal to the top plate is included in the depth of field of the camera lens.
The support according to any one of claims 16 to 18, characterized in that. One or both of the top plate and the side plate has a structure that palpably indicates the type of document composed of a plurality of pages suitable for the use of the support, and information suitable for the use of the support. One or both of structures that palpate the type of processing terminal are provided.
The support for an information processing terminal according to any one of claims 16 to 19, wherein the information processing terminal supports. An information processing method for outputting a document composed of a plurality of pages as voice using an information processing terminal provided with a camera, a controller, an output unit, and one or both of speakers.
An imaging process in which the camera captures each page of a document composed of a plurality of pages.
A text data generation process in which the controller refers to the image data obtained by the camera to generate text data representing the text described on the page.
An audio signal generation process in which the controller refers to the text data to generate an audio signal representing the audio that reads the text.
The output unit includes an audio output process in which the audio signal is output and / or the speaker converts the audio signal into sound waves and outputs the signal.
An information processing method characterized by this.

Images