JP2019040005A - Reading aloud system and reading aloud method - Google Patents

Abstract

To provide a reading aloud system which assists a weak-sighted person.SOLUTION: A reading aloud system comprises: an image pickup unit which is included in a wearing tool that a user puts on and uses and picks up the front direction of the user; an extraction unit which extracts characters from an image that the image pickup unit picks up; a conversion unit which convers the characters that the extraction unit extracts into a speech; an output unit which is included in the wearing tool, and outputs a speech; an input unit which is included in the wearing tool, and receives input from the user; and a control unit which is included in the wearing tool, and controls a playback speed of the speech output from the output unit based upon the input received from the user through the input unit.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a reading system and a reading method for converting a sentence into speech and reading it out.

  In recent years, development of devices that support the visual recognition of people with low vision and people with reading disabilities who are difficult to read characters has been carried out. For example, Patent Document 1 discloses a wearable display that can capture and display a forward field of view so that a low vision person can walk outdoors at night. According to the low vision person wearable display of Patent Document 1, the contrast and brightness of the captured image are converted and displayed. It also discloses that when a character is present in a captured image, character recognition processing is performed to inform the user of the character by voice.

JP 2014-165616 A

  By the way, in the wearable display for the low vision person described in the above-mentioned Patent Document 1, it is only described that the character is transmitted to the low vision person through the speaker by the character recognition process, and specifically how to transmit the voice. Is not disclosed. In addition, since the way the sound is heard varies depending on the user, the low-vision wearable display described in Patent Document 1 has a problem of lack of usability.

  Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide a reading system that is more convenient for a user who uses the wearable display for low vision users described in Patent Document 1 described above. To do.

  In order to solve the above-described problem, a reading system according to an aspect of the present invention is provided in a wearing tool worn by a user and includes an imaging unit that images the front direction of the user, and an image captured by the imaging unit. An extraction unit that extracts characters from the input unit, a conversion unit that converts the characters extracted by the extraction unit into speech, an output unit that is provided in the wearing tool and outputs sound, and is provided in the wearing tool and receives input from the user An input unit, and a control unit that is provided in the wearing tool and controls a reproduction speed of the sound output from the output unit based on an input from a user received through the input unit.

  In order to solve the above-described problem, a reading method according to an aspect of the present invention includes an imaging step of imaging a front direction of a user by an imaging unit provided in a wearing tool worn by the user, and an imaging step. An extraction step of extracting characters from the image captured in step, a conversion step of converting the characters extracted in the extraction step into speech, an output step of outputting sound from an output unit included in the mounting device, and a mounting device. An input step for receiving an input from the user from the input unit, and a control step for controlling the reproduction speed of the sound output from the output unit based on the input from the user received through the input unit.

  In the reading system, the control unit may temporarily stop the sound output from the output unit based on the input from the user received through the input unit.

  In the reading system, the control unit may repeatedly reproduce the sound output from the output unit based on the input from the user received through the input unit.

  In the reading system, the output unit may output a sound indicating that the character conversion process is being performed while the conversion unit converts the character into a voice.

  In the reading system, the reading system includes a server including an extraction unit and a conversion unit, and the wearing tool receives a transmission unit that transmits an image captured by the imaging unit to the server, and a sound converted by the conversion unit. The output unit may output a sound indicating that character conversion processing is in progress from when the transmission unit transmits an image until the reception unit receives voice.

  In the reading system, the wearing tool may include an acquisition unit that acquires environment information about the surrounding environment, and the imaging unit may change the imaging condition based on the environment information.

  In the reading system, the wearing tool includes a determination unit that determines whether there is a character in the captured image captured by the imaging unit, and the output unit includes the character in the captured image. When it is determined that a character is present, a sound indicating that a character is present in the imaging direction by the imaging unit may be output.

  In the reading system, the reading system associates the captured image captured by the imaging unit with the sound obtained by the conversion unit converting based on the captured image, and transmits the associated image to the user information processing terminal. It is good also as providing a log transmission part.

  Further, in the reading system, the wearing tool includes a position information acquisition unit that acquires position information indicating the position of the terminal, and the imaging unit acquires the position information acquired by the position information when the captured image is captured. The log transmission unit may transmit the positional information together with the captured image and the sound to the information processing terminal of the user.

  The reading system according to one embodiment of the present invention can freely adjust the speed of reading a voice converted from a character. Therefore, it is possible to provide a reading system with excellent usability.

(A) is a figure which shows the example of an external appearance of the user with which the mounting tool is mounted | worn, (b) is a figure which shows the example of external appearance which images and reads out using a mounting tool. It is a figure which shows the system configuration example of a reading-out system. (A) is a figure which shows the structural example of the data which a mounting tool transmits to a server, (b) is a figure which shows the structural example of the reading voice information which a server memorize | stores for every user. It is a sequence diagram which shows the exchange between a mounting tool and a server. It is a flowchart which shows operation | movement of a mounting tool. It is a flowchart which shows operation | movement of a server. It is a figure which shows the example of a range which extracts a character preferentially from an image. It is a figure which shows the example of a screen for reproducing the reading sound using a map. It is a figure which shows another example of the system configuration | structure of a reading-out system.

  Hereinafter, a reading system according to an embodiment of the present invention will be described in detail with reference to the drawings.

<Embodiment>
<Configuration>
FIG. 1A is a diagram illustrating an appearance example of a user wearing the wearing tool 100 according to the reading system 1. FIG. 1B is a diagram illustrating an example of an external appearance showing a state where imaging is performed using the wearing tool 100 and reading is performed. FIG. 2 is a diagram illustrating a system configuration example of the reading system 1.

  As shown in FIGS. 1 and 2, the reading system 1 is provided in a wearing tool 100 worn by a user and includes an imaging unit 111 that captures the front direction of the user, and an image captured by the imaging unit 111. An extraction unit 231 that extracts characters, a conversion unit 232 that converts the characters extracted by the extraction unit 231 into speech, an output unit 156 that outputs sound, an output unit 156 that outputs sound, and a user An input unit 154 that receives an input from the control unit 155 and a control unit 155 that is provided in the wearing tool 100 and controls the reproduction speed of the sound output from the output unit 156 based on the input from the user received through the input unit 154. With. Such a reading system 1 will be described in detail below.

  As shown in FIGS. 1A and 1B, the user 10 wears and uses a wearable glass 110. The wearable glass 110 is provided with an imaging unit 111 at a position where the front direction of the user can be imaged in accordance with an instruction from the user. The imaging unit 111 is a so-called camera. The wearable glass 110 is connected to the controller 150. In FIG. 1A, the wearable glass 110 is shown to be connected via the cord 120 via the earphone 130 and via the cord 140, but the wearable glass 110 and the controller 150 are connected to the earphone. 130 and the controller 150 may be directly connected. In addition, the user 10 can wear the earphone 130 on the ear and listen to the read-out voice transmitted from the controller 150. As shown in FIG. 1A, the user 10 holds the controller 150 and can use the controller 150 to give an instruction for imaging and an instruction related to reproduction of a reading voice. As illustrated in FIG. 1B, when the user gives an imaging instruction, the imaging unit 111 captures an imaging range 160. Then, a character included in the imaging range 160 is recognized, and the character is converted into machine synthesized speech and read out. Therefore, the reading-out system 1 can provide information on characters that are difficult to read to a low-sighted person or the like.

  FIG. 2 is a system configuration example of the reading system 1, and the reading system 1 includes a wearing tool 100 and a server 200. The wearing tool 100 and the server 200 are configured to be able to communicate with each other via the network 300. The wearing tool 100 and the network 300 communicate by wireless communication. Note that any communication protocol may be used as long as wireless communication can be performed. The server 200 also communicates with the network, and this may take any form of communication, such as wireless communication or wired communication, and any communication protocol may be used as long as communication can be performed.

  As shown in FIG. 2, the wearing tool 100 includes a wearable glass 110, an earphone 130, and a controller 150. In other words, in the present embodiment, as shown in FIG. 2, wearable glass 110, earphone 130, and controller 150 are collectively referred to as wearing device 100. Here, although wearable glass 110 is used, it is only necessary to be able to image the front direction (viewing direction) of user 10, and needless to say, it is not limited to glasses.

  The wearable glass 110 includes an imaging unit 111 and a communication I / F 112.

  The imaging unit 111 is a camera that can capture the front direction of the user. The imaging unit 111 receives an imaging signal instructed from the communication I / F 112 and performs imaging. The imaging unit 111 may be provided anywhere in the wearable glass 110 as long as the imaging unit 111 is provided so as to be able to capture the front direction of the user. Although FIG. 1 shows an example in which the wearable glass is provided on the left hinge portion, the imaging unit 111 may be provided on the right hinge portion or on the bridge portion. The imaging unit 111 transmits a captured image obtained by imaging to the communication I / F 112. In addition, the imaging unit 111 may have a detection function of analyzing the captured image and detecting the presence or absence of characters in the captured image while sequentially capturing images. At this time, the captured image includes characters. If it is determined, the presence signal indicating that there is a character in the front direction of the user is transmitted to the communication I / F 112.

  The communication I / F 112 is a communication interface having a function of communicating with the controller 150. The communication I / F 112 is communicably connected to the communication I / F 151 of the controller 150. Here, as shown in FIG. 1, it is assumed that they are connected by wire, but this may be a wireless connection. The communication I / F 112 transmits the imaging signal transmitted from the communication I / F 151 of the controller 150 to the imaging unit 111. In addition, the communication I / F 112 transmits a captured image transmitted from the imaging unit 111 and a presence signal indicating that a character exists in the front direction of the user to the communication I / F 151.

  The earphone 130 is connected to the output unit 156 of the controller 150 and has a function of outputting the audio signal transmitted from the output unit 156 as audio. Here, as shown in FIG. 1, the earphone 130 is connected to the controller 150 by wire, but this may be a wireless connection. The earphone 130 outputs a reading voice that reads a character detected based on the captured image, a sound indicating that the character is being analyzed, and a sound indicating that there is a character in the front direction of the imaging unit 111.

  The controller 150 includes a communication I / F 151, a communication unit 152, a storage unit 153, an input unit 154, a control unit 155, and an output unit 156. As shown in FIG. 1, each part of the controller 150 is connected to each other by a bus.

  The communication I / F 151 is a communication interface having a function of performing communication with the communication I / F 112 of the wearable glass 110. When the communication I / F 151 receives an imaging signal from the control unit 155, the communication I / F 151 transmits the imaging signal to the communication I / F 112. In addition, when the communication I / F 151 receives a captured image or a presence signal from the communication I / F 112, the communication I / F 151 transmits the captured image or presence signal to the control unit 155.

  The communication unit 152 is a communication interface having a function of performing communication with the server 200 via the network 300. The communication unit 152 functions as a transmission unit that transmits a captured image to the server 200 in accordance with an instruction from the control unit 155 and also functions as a reception unit that receives from the server 200 read-out sound obtained by converting characters included in the captured image into speech. . When the communication unit 152 receives the reading voice from the server 200, the communication unit 152 transmits the reading voice to the control unit 155.

  The storage unit 153 has a function of storing various programs and data necessary for the controller 150 to function. The storage unit 153 can be realized by, for example, an HDD (Hard Disc Drive), an SSD (Solid State Drive), a flash memory, or the like, but is not limited thereto. The storage unit 153 stores a reading program executed by the control unit 155, a captured image captured by the imaging unit 111, information of a reading voice received by the communication unit 152, and the like. In addition, the storage unit 153 is a sound output from the output unit 156 between the time when the communication unit 152 transmits the captured image to the server 200 and the time when the read-out sound is received, and indicates that the character is being converted into a sound. The audio information shown is stored. Furthermore, the storage unit 153 also stores audio information for notifying the user 10 when there is a character in the imaging direction of the imaging unit 111.

  The input unit 154 has a function of accepting an input from the user 10. The input unit 154 can be realized by, for example, a hard key provided in the controller 150, but this may be realized by a touch panel or the like. The input unit 154 includes an imaging button 154a for the user 10 to instruct imaging, a playback button 154b for instructing playback, pause, and replay, and an adjustment button 154c for adjusting the audio playback speed. Good. The input unit 154 transmits a signal indicating the pressed content to the control unit 155 in response to pressing of each button.

  The control unit 155 is a processor having a function of controlling each unit of the controller 150. The control unit 155 fulfills a function to be executed as the controller 150 by executing various programs stored in the storage unit 153.

  When receiving an imaging instruction from the input unit 154, the control unit 155 instructs the communication I / F 151 to transmit an imaging signal to the wearable glass 110.

  In addition, when the captured image is transmitted from the communication I / F 151, the control unit 155 instructs the communication unit 152 to transmit the captured image to the server 200. Further, after the instruction, the control unit 155 reads from the storage unit 153 a voice indicating that the character included in the captured image is being converted into a voice, and instructs the output unit 156 to output the voice.

  When the reading voice is transmitted from the communication unit 152, the control unit 155 instructs the output unit 156 to stop outputting the voice indicating that conversion is in progress. Then, the control unit 155 instructs the output unit 156 to output a reading voice.

  In addition, when the presence signal is transmitted from the communication I / F 151, the control unit 155 reads a voice indicating that a character exists in the front direction of the user 10 from the storage unit 153 and outputs the voice to the output unit 156. To instruct.

  In addition, the control unit 155 executes a reproduction control process for the read-out voice according to the instruction from the user 10 transmitted from the input unit 154. For example, when a pause instruction is received, the output unit 156 is instructed to pause the reproduction of the reading voice.

  Further, for example, when a slow reproduction instruction is received, the control unit 155 instructs the output unit 156 to perform slow reproduction of the reading voice. The slow reproduction instruction can be replaced by an instruction to adjust the reproduction speed, and the control unit 155 can increase or decrease the reproduction speed of the reading voice. Further, when the control unit 155 receives a replay instruction, the control unit 155 instructs the output unit 156 to reproduce the read-out voice that has been output so far.

  The output unit 156 has a function of outputting an audio signal instructed from the control unit 155 to the earphone 130. The output unit 156 outputs to the earphone 130 a reading voice, a voice indicating that a character is being converted into a voice, or a voice indicating that a character is present in the front direction of the user 10.

  The above is the description of the configuration of the wearing tool 100.

  Next, the server 200 will be described. As shown in FIG. 2, the server 200 includes a communication unit 210, a storage unit 220, and a control unit 230. The communication unit 210, the storage unit 220, and the control unit 230 are connected to each other via a bus.

  The communication unit 210 is a communication interface having a function of executing communication with the wearing tool 100 (controller 150) via the network 300. The communication unit 210 functions as a transmission unit that transmits a read-out voice to the wearing device 100 according to an instruction from the control unit 230 and also functions as a reception unit that receives a captured image. When the communication unit 210 receives a captured image from the wearing tool 100, the communication unit 210 transmits the captured image to the control unit 230.

  The storage unit 220 stores various programs and data necessary for the operation of the server 200. The storage unit 220 can be realized by, for example, an HDD (Hard Disc Drive), an SSD (Solid State Drive), a flash memory, or the like, but is not limited thereto. The storage unit 220 stores a character recognition program for extracting characters from the image, a voice conversion program for converting the recognized characters into voice, and read-out voice information. Details of the reading voice information will be described later.

  The control unit 230 is a processor having a function of controlling each unit of the server 200. The control unit 230 fulfills functions to be executed as the server 200 by executing various programs stored in the storage unit 220. The control unit 230 functions as the extraction unit 231 by executing the character recognition program, and functions as the conversion unit 232 by executing the voice conversion program.

  The extraction unit 231 has a function of analyzing a captured image and extracting characters included in the captured image. An existing character recognition process can be used for the analysis technique.

  The conversion unit 232 has a function of converting the character extracted by the extraction unit 231 into a voice (reading voice). An existing conversion process can be used for the conversion technique.

  The above is the description of the configuration of the server 200.

<Data>
FIG. 3 is a diagram illustrating a data configuration example of data related to the reading system 1.

  FIG. 3A is a diagram illustrating a data configuration example (format example) of transmission data 310 (captured image) transmitted from the wearing tool 100 (controller 150) to the server 200. FIG.

  As illustrated in FIG. 3A, the transmission data 310 is information in which a user ID 311, captured image information 312, and imaging time information 313 are associated with each other.

  The user ID 311 is identification information that can uniquely identify the user 10 who uses the wearing tool 100. Thereby, the server 200 can specify which user the captured image is from, and can manage the captured image and the generated reading voice for each user.

  The captured image information 312 is information indicating actual data of a captured image captured by the imaging unit 111.

  The imaging time information 313 is information indicating the date and time when the captured image indicated by the captured image information 312 is captured. Although not shown, the information can be acquired from the internal clock of the imaging unit 111.

  FIG. 3B is a diagram illustrating a data configuration example of the read voice information stored in the storage unit 220 of the server 200 and managed for each user who uses the read system 1. The data is information for managing the read-out voice obtained by the server 200 converting in the past.

  As illustrated in FIG. 3B, the reading voice information 320 is information in which the imaging time information 321, the captured image information 322, and the reading voice 323 are associated with each other.

  The imaging time information 321 is information indicating the date and time when the corresponding captured image was captured, and is the same information as the imaging time information 313.

  The captured image information 322 is information indicating actual data of the captured image, and is the same information as the captured image information 312.

  The reading voice 323 is actual data indicating the reading voice obtained by the extraction unit 231 extracting characters from the corresponding captured image information 322 and converting the characters by the conversion unit 232.

  The presence of the reading voice information 320 allows the server 200 to manage past reading voice information.

  The above is the description of the information mainly related to the reading system 1.

<Operation>
From here, the operation of the reading system 1 will be described. First, the overall operation of the reading system 1 will be described using the sequence diagram shown in FIG. 4, and then the detailed operation of the wearing tool 100 and the server 200 will be described using the flowcharts of FIGS. 5 and 6, respectively. .

  FIG. 4 is a sequence diagram showing the exchange between the wearing tool 100 and the server 200. As illustrated in FIG. 4, the wearing tool 100 performs imaging in the front direction of the user 10 (step S401). Then, the wearing tool 100 transmits the obtained captured image to the server 200 (step S402).

  The server 200 receives the captured image transmitted from the wearing tool 100 (step S403). Then, the server 200 extracts characters from the received captured image (step S404). Then, the server 200 converts the extracted character into a voice and generates a reading voice (step S405). When the reading voice is generated, the server 200 transmits it to the wearing tool 100 (step S406).

  The wearing tool 100 receives the read-out voice transmitted from the server 200 (step S407). Then, the wearing tool 100 outputs the received reading voice (step S408). Thereby, the reading system 1 can recognize the character which exists in the front direction (viewing direction) of the user 10, and can tell the user 10 with a sound.

  FIG. 5 is a flowchart showing the operation of the wearing tool 100.

  First, the input unit 154 of the wearing tool 100 determines whether or not there is an input from the user based on whether or not various buttons are pressed (step S501). If there is an input from the user (YES in step S501), the process proceeds to step S502. If not (NO in step S501), the process proceeds to step S512.

  In step S502, the control unit 155 determines whether or not the input received by the input unit 154 is an imaging instruction (step S502). If the input is an imaging instruction (YES in step S502), the process proceeds to step S503. If the input is not an imaging instruction (NO in step S502), the process proceeds to step S506.

  In step S503, when the input unit 154 receives an imaging instruction from the user, the imaging instruction is transmitted to the control unit 155. In response to this, the control unit 155 instructs the communication I / F 151 to transmit the imaging signal to the wearable glass 110. The communication I / F 151 transmits the imaging signal to the communication I / F 112 according to the instruction. Then, the communication I / F 112 transmits an imaging signal to the imaging unit 111, and the imaging unit 111 executes imaging (step S503).

  The imaging unit 111 transmits the obtained captured image to the communication I / F 112, and the communication I / F 112 transmits the captured image to the communication I / F 151. The communication I / F 151 transmits the transmitted captured image to the control unit 155, and the control unit 155 instructs the communication unit 152 to transmit this to the server 200. In response to the instruction, the communication unit 152 transmits the captured image to the server 200 via the network 300 (step S504).

  Then, after transmitting the captured image, the control unit 155 reads from the storage unit 153 a voice indicating that characters in the captured image are being converted into a voice, and instructs the output unit 156 to output the voice. In response to this, the output unit 156 outputs the sound to the earphone 130, and the earphone 130 notifies the sound and returns to the process of step S501. By notifying the voice indicating that the character included in the captured image is being converted into speech, the user 10 may recognize that the character is being converted into speech now. As compared with the case where no sound is generated (no notification is made to the user 10), the user can wait without annoyance.

  On the other hand, if it is determined in step S502 that the input instruction is not an imaging instruction (NO in step S502), it is determined whether the input is a voice pause instruction (step S506). If the input is a voice pause instruction (YES in step S506), the control unit 155 instructs the output unit 156 to pause the voice being output. Receiving the instruction, the output unit 156 stops outputting the sound (step S507), and returns to the process of step S501. The pause is performed until a new reproduction instruction is input or a complete stop instruction is input.

  If it is determined in step S506 that the input instruction is not a pause instruction (NO in step S506), it is determined whether the input is a slow reproduction instruction (step S508). If the input is a slow playback instruction (YES in step S508), the control unit 155 instructs the output unit 156 to perform slow playback of the output audio. In response to the instruction, the output unit 156 starts slow playback of the output audio (step S509), and returns to the process of step S501. As a result, even a user who finds difficulty in listening to a quick mouth can recognize voice correctly. Although an example of slow playback is shown here, this may be achieved by increasing the playback speed as described above. Increasing the playback speed can be used to shorten the time for grasping the outline of the characters included in the captured content.

  If it is determined in step S508 that the input instruction is not slow playback (NO in step S508), it is determined whether the input is playback or replay input (step S510). If the input is not a reproduction or replay input (NO in step S510), the process returns to step S501. If the input is a playback or replay input (YES in step S510), the control unit 155 resumes the output of the audio that has been temporarily stopped in the output unit 156 or executes the replay of the audio that has been output once. To instruct. In response to this, the output unit 156 resumes or replays the sound and returns to the process of step S501. Thereby, even if the user 10 misses the reading voice, he / she can listen again.

  When there is no input from the user in step S501 (NO in step S501), the control unit 155 determines whether or not a read-out voice is received from the server 200 (step S512). If no reading voice has been received (NO in step S512), the process returns to step S501.

  If the read-out voice has been received (YES in step S512), the control unit 155 first outputs to the output unit 156 a voice indicating that the output character is being converted into a voice. Instruct to stop. In response to the instruction, the output unit 156 stops outputting the sound (step S513).

  Then, the control unit 155 instructs the output unit 156 to output the reading voice transmitted from the communication unit 132. The output unit 156 starts outputting the reading voice transmitted from the control unit 155 (step S514), and returns to step S501.

  The above is description of operation | movement of the mounting tool 100 (controller 150).

  FIG. 6 is a flowchart illustrating an operation when the server 200 receives a captured image from the wearing tool 100.

  First, the communication unit 210 of the server 200 receives a captured image from the wearing tool 100 via the network 300 (step S601). The communication unit 210 transmits the received captured image to the control unit 230.

  As the extraction unit 231, the control unit 230 analyzes the transmitted captured image and extracts characters (step S602). The extraction unit 231 transmits the extracted character string to the conversion unit 232.

  The conversion unit 232 converts the extracted character string into speech (step S603), and generates read-out speech that is synthesized speech of opportunity speech. The conversion unit 232 transmits the generated reading voice to the communication unit 210.

  The communication unit 210 transmits the converted synthesized speech to the wearing tool 100 via the network 300 as a read-out speech (step S604).

  Thereafter, the control unit 230 sets the received captured image, the captured date and time of the captured image, and the reading voice obtained from the captured image as the captured image information 322, the imaging time information 321, and the reading voice 323, respectively. It registers in the reading voice information (step S605), and the process is terminated.

  The above is the operation of the server 200. By performing the operation described above, the reading system 1 can reproduce the voice so as to be easily heard by the user, not simply reading the recognized character.

<Summary>
The reading system 1 can recognize characters included in the captured image and output them as speech. At this time, in the reading system 1, since the user can perform operations such as slow playback, primary stop, and replay for the reading voice, the user can reproduce the voice so that it is easy to hear depending on each preference. it can. Therefore, it is possible to provide a reading system that is convenient for the user. Moreover, in the reading system 1, while performing the process which produces | generates the reading sound from a captured image, the user 10 can be made to recognize a condition by alert | reporting the sound which shows that it is processing.

<Supplement>
It goes without saying that the reading system according to the above embodiment is not limited to the above embodiment, and may be realized by other methods. Hereinafter, various modifications will be described.

  (1) In the above embodiment, the sound is output using the earphone 130. However, the wearable glass 110 or the controller 150 includes a speaker, and the output unit 156 outputs the reading sound from the speaker. It is good. By configuring in this way, even a user who suffers from wearing earphones 130 can listen to the reading voice. In this case, there is also an advantage that a plurality of users can listen to the sound at the same time.

  (2) In the said embodiment, the wearable glass 110, the earphone 130, and the controller 150 were included as the mounting tool 100, and the example comprised as a respectively separate apparatus was shown. However, this is not limited thereto, and the wearable glass 110, the earphone 130, and the controller 150 may be integrally formed. That is, wearable glass 110 may include a speaker as an alternative to the function of outputting sound from earphone 130 and may retain the function of controller 150. For example, the temple portion of the wearable glass 110 may have a hollow structure, and a processor, a memory, a communication module, and the like of the controller 150 may be mounted therein. And various buttons for voice reproduction control and imaging instructions may be arranged on the exterior of the temple or rim of the wearable glass 110.

  (3) In the above embodiment, the wearing tool 100 and the server 200 are described as separate devices. However, the wearing tool 100 may include the functions (extraction unit and conversion unit functions) of the server 200. Good. For example, the controller 150 may include a chip that realizes the function of the server 200. If comprised in this way, the mounting tool 100 can construct | assemble a reading-out system by stand-alone. In addition, it is possible to suppress latency associated with transmission of captured images and reception of read-out sound.

  (4) In the above embodiment, the range for extracting characters from the captured image is determined in advance, but this is not limited thereto. For example, the wearable glass 110 is provided with a camera that captures the eyes of the user, detects the line-of-sight direction, applies a predetermined range centered on the line-of-sight direction to the captured image, and detects characters within the predetermined range May be. For example, the wearable glass 110 transmits a first captured image captured by the imaging unit 111 and a second captured image captured by the user's eyes to the controller 150, and the controller 150 transmits the first captured image and the second captured image. Is transmitted to the server 200. The extraction unit 231 of the server 200 specifies the line-of-sight direction of the user 10 from the second captured image, specifies a predetermined range including the specified line-of-sight direction, and starts from a location corresponding to the predetermined range in the first captured image. You may comprise so that a character may be extracted.

  (5) In the above embodiment, the imaging unit 111 receives an input of an imaging instruction to the controller 150 and performs imaging, but the imaging trigger is not limited to this. For example, the wearable glass 110 or the controller 150 is provided with a microphone, and the voice emitted by the user is acquired with the microphone. And you may image based on the specific word which the user uttered. That is, imaging by voice input may be performed.

  Alternatively, a camera that captures the user's eyes may be provided in the wearable glass 110, and a blink of the user's eyes may be used as an imaging trigger.

  (6) In the above embodiment, the input unit 154 is configured to be provided in the controller 150. However, this is not limited thereto, and the input unit 154 may be provided in the middle of the cable 140.

  (7) Although not specifically described in the above embodiment, the reading system 1 may include a setting unit that can set the language of the reading voice. And the translation part which translates the character which the extraction part 231 extracted into the language set to the setting part is provided, and the conversion part 232 is good also as converting the character translated by the translation part into an audio | voice. With this configuration, the reading system 1 can function as an interpreter system for written characters, and can be a system useful not only for low-sighted people but also for users in foreign countries.

  (8) Although not particularly described in the above embodiment, the extraction unit 231 may limit the range in which characters are extracted from the captured image to a predetermined range instead of the entire captured image. FIG. 7 shows an example of the captured image 700, and the extraction unit 231 may use only a predetermined range 710 in the captured image 700 as a range for extracting characters. Alternatively, the predetermined range 710 may be a range in which characters are extracted with priority. The range in which characters are preferentially extracted is defined as a range in which characters are first extracted, and a process of extracting characters from outside the predetermined range 710 when characters cannot be extracted from the predetermined range 710. To do.

  Here, the predetermined range 710 may be set by a user who uses the reading system 1. In general, the user tends to see a slightly lower direction than the front direction. Therefore, it is effective to set the predetermined range 710 near the lower part of the captured image 700.

  Further, the predetermined range 710 may be set by the control unit 230. Specifically, a range in which characters can be extracted from a large number of captured images received by the server 200 is specified. Then, the average range may be a predetermined range 710 for extracting characters.

  Furthermore, the wearable glass 110 may be provided with various sensors, and the predetermined range 710 may be determined based on sensing data obtained from the sensors. For example, a gyro sensor is mounted on the wearable glass 110 and the wearing tool 100 transmits sensing data of the gyro sensor to the server 200 together with the captured image. And the extraction part 231 may determine the predetermined range 710 based on the sensing data of the said gyro sensor. For example, when it is estimated from the sensing data that the user 10 is inadvertently adjusted, the predetermined range 710 may be set at a position below the entire captured image 700.

  By not using the entire captured image 700 as an analysis target, it is possible to reduce the time required for the conversion process to sound.

  (9) Although not particularly described in the above embodiment, the server 200 has a configuration for transmitting the corresponding read-out voice information 320 as a past log to an information processing apparatus such as a PC held by the user 10. May be. With this configuration, the user 10 can listen to the past reading voice at any time.

  Furthermore, the wearing tool 100 may further include a position information acquisition unit for acquiring position information indicating a location where the own apparatus exists. A position information acquisition part is realizable by utilizing GPS, GNSS, etc., for example.

  Each time the imaging unit 111 obtains a captured image, the position information acquisition unit acquires position information and associates the acquired position information with the captured image. The wearing tool 100 transmits the captured image associated with the position information to the server 200. The server 200 may further manage the image pickup position information indicating the image pickup position in association with the reading voice information 320.

  Then, since information including position information is transmitted from the server 200 to the information processing apparatus of the user 10 as the read-out sound information 320, the information processing apparatus of the user 10 further reads out the sound as shown in FIG. Can be presented with a map application. That is, the user 10 can recognize when and where the read-out voice is acquired on the map. Then, by positioning the cursor 803 on the log information 801 or 802 of the map information and clicking it, the information processing apparatus may reproduce the read-out voice by voice reproduction software or the like. For example, as shown in the map 800 of FIG. 8, the presence of the log information 801 and the log information 802 makes it possible to recognize where the voice is read out based on the captured image captured.

  (10) Although the above embodiment has not been described in detail as the operation of the wearing tool 100, the imaging unit 111 performs sequential imaging and determines whether or not characters are included in the obtained captured image. It may be detected. When it is detected that a character is included, the fact is transmitted to the controller 150, and the control unit 155 generates a voice for causing the user 10 to recognize that the character is present in the front direction at that time. You may inform. Then, the user 10 can input an imaging instruction to the timing and input unit 154. With this configuration, the user 10 can recognize the presence of the character when the user 10 cannot recognize the presence of the character, such as when the user 10 is a low vision person, particularly when the user 10 is blind. Therefore, it is possible to provide a convenient reading system 1.

  (11) Although not particularly described in the above embodiment, the imaging unit 111 may change the imaging condition according to the environment where the user (wearable glass 110) is placed. For example, the wearable glass 110 may include various sensors (for example, an illuminance sensor) and change the exposure time and the angle of view.

  (12) In the above embodiment, although not specifically described, the server 200 cannot extract characters from the image, the extracted characters cannot be converted into sound, or the image does not contain characters. Alternatively, the error signal may be transmitted to the wearing tool 100, and the wearing tool 100 may receive the error signal and output a sound indicating the error from the output unit 156. In addition to the error sound and the sound indicating that the conversion is being performed as described in the above embodiment, for example, the start sound when the wearing tool 100 is started and the image pickup sound when the imaging unit 111 performs imaging ( Various sounds such as a shutter sound), a sound indicating standby, and a cancel sound when the user inputs a cancellation of the process are stored in the storage unit 153, and the control unit 155 corresponds to the state of the wearing tool 100. The corresponding sound may be output from the output unit 156. In addition, when the communication unit 152 cannot communicate (cannot connect to the network), a sound indicating that may be output from the output unit 156. By adopting a configuration that outputs sounds according to various states as described above, the wearing tool 100 can notify the user of the state of the device with only sounds.

  (13) Although not particularly described in the above embodiment, the server 200 generates a read-out voice that is generated in accordance with the portion where the character is extracted from the captured image and the ratio of the character extraction range to the captured image. It is good also as changing.

  Changing the sound mode according to the location where the character is extracted means changing the direction in which the user hears the sound according to the location in the captured image where the character is extracted from the captured image. For example, when the part from which the character is extracted is extracted from the right side of the captured image, it may be configured to output from the output unit 156 so that the reading voice can be heard from the right side of the user. By adopting this configuration, it is possible to make the user sensuously recognize which direction the character is read out as viewed from the user.

  In addition, changing the aspect of the read-out sound to be generated according to the ratio of the character extraction range to the captured image changes the volume of the read-out sound according to the ratio of the ratio of the character extraction range to the captured image. You may comprise. That is, the percentage of the ratio and the volume for outputting the reading voice are stored in association with each other, and the volume for outputting the reading voice is determined by comparing with the percentage of the range in which characters are extracted from the captured image. It is also possible to output a read-out voice at the volume set.

  (14) In the above embodiment, the transmission data 310 is associated with the user ID 311, the captured image information 312, and the imaging time information 313. It may be attached. For example, as shown in the supplement, position information indicating the location where the wearing tool 100 exists and information on sensing data such as a gyro sensor or an acceleration sensor that can specify the posture of the wearing tool 100 may be associated with each other. Good.

  In addition, the reading voice information is also information in which the imaging time information 321, the captured image information 322, and the reading voice 323 are associated with each other. Character text data, position information included in the transmission data 310, sensing data, and the like may also be associated.

  The reading voice information can be used as a life log of each user by accumulating and accumulating various kinds of information. And according to the request from a user, the server 200 may be provided with the provision part which provides the designated information among the accumulated information. For example, by accumulating position information, it provides information on the amount of movement per unit time of the user (for example, a day), provides information on where the user has gone, and uses gyro sensor information By specifying the user's posture, posture information can be provided (for example, whether the posture is good or bad).

  (15) In the above embodiment, as a method for the reading system 1 to read out the voice, the processor (the control unit 155 and the control unit 230) functioning as each functional unit constituting the reading system 1 reads the reading program and the like. It is supposed to execute a reading process by executing this, which is a logic circuit (hardware) or a dedicated circuit formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)) or the like in the device. It may be realized by incorporating. These circuits may be realized by one or a plurality of integrated circuits, and the functions of the plurality of functional units described in the above embodiments may be realized by a single integrated circuit. An LSI may be called a VLSI, a super LSI, an ultra LSI, or the like depending on the degree of integration. That is, as shown in FIG. 9, each functional unit in the wearing tool 100 and the server 200 that constitute the reading system 1 may be realized by a physical circuit. That is, as shown in FIG. 9, the wearing tool 100 includes a wearable glass 110 including an imaging circuit 111a and a communication I / F circuit 112a, an earphone 130, a communication I / F circuit 151a, a communication unit 152a, and a storage. The circuit 153a, the input circuit 154a, the control circuit 155a, and the output circuit 156a may be included, and may have the same functions as the corresponding functional units in the above embodiment. Similarly, the server 200 may include a communication circuit 210a, a storage circuit 220a, and a control circuit 230a including an extraction circuit 231a and a conversion circuit 232a.

  The reading program may be recorded on a recording medium that can be read by the processor. As the recording medium, a “non-temporary tangible medium”, for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like. Etc. can be used. The reading program may be supplied to the processor via any transmission medium (such as a communication network or a broadcast wave) that can transmit the reading program. The present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the reading program is embodied by electronic transmission.

  The reading-out program can be implemented using, for example, a script language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5.

  (16) The configurations described in the above embodiment and each supplement may be combined as appropriate.

1 Reading System 100 Wearing Tool 110 Wearable Glass 111 Imaging Unit 112 Communication I / F
130 Earphone 150 Controller 151 Communication I / F
152 communication unit 153 storage unit 154 input unit 155 control unit 156 output unit 200 server 210 communication unit 220 storage unit 230 control unit 231 extraction unit 232 conversion unit

Claims (10)

An imaging unit that is provided in a wearing tool that a user wears and uses, and that images the front direction of the user;
An extraction unit that extracts characters from an image captured by the imaging unit;
A conversion unit that converts the characters extracted by the extraction unit into speech;
An output unit provided in the wearing tool and outputting the sound;
An input unit provided in the wearing tool for receiving input from a user;
A reading system comprising: a control unit that is provided in the wearing tool and that controls a reproduction speed of a sound output from the output unit based on an input from the user received through the input unit.   The reading system according to claim 1, wherein the control unit temporarily stops the sound output from the output unit based on an input from the user received through the input unit.   The reading system according to claim 1 or 2, wherein the control unit repeatedly reproduces the sound output from the output unit based on the input from the user received through the input unit.   4. The output unit according to claim 1, wherein the output unit outputs a sound indicating that a character conversion process is being performed while the conversion unit converts the character into a voice. 5. Reading system described. The reading system includes a server including the extraction unit and the conversion unit,
The wearing tool is
A transmission unit that transmits an image captured by the imaging unit to the server;
A receiving unit that receives the voice converted by the converting unit;
The output unit outputs a sound indicating that the character conversion process is in progress from when the transmitting unit transmits an image until the receiving unit receives the voice. 4. A reading system according to 4. The wearing device includes an acquisition unit that acquires environmental information about the surrounding environment,
The reading system according to claim 1, wherein the imaging unit changes an imaging condition based on the environment information. The wearing tool includes a determination unit that determines whether there is a character in a captured image captured by the imaging unit,
The output unit outputs sound indicating that a character is present in an imaging direction of the imaging unit when the determination unit determines that a character is included in the captured image. The reading system according to any one of 1 to 6. The reading system is
A log transmission unit configured to associate the captured image captured by the imaging unit with the sound obtained by the conversion unit based on the captured image and transmit the associated image to the information processing terminal of the user; The reading system according to any one of claims 1 to 7. The wearing tool includes a position information acquisition unit that acquires position information indicating the position of the terminal.
The imaging unit associates the captured image with the positional information acquired by the positional information when the captured image is captured,
The reading system according to claim 8, wherein the log transmission unit transmits the position information together with the captured image and the sound to the information processing terminal of the user. An imaging step of imaging the front direction of the user by an imaging unit provided in a wearing tool worn by the user; and
An extraction step of extracting characters from the image captured in the imaging step;
A conversion step of converting the characters extracted in the extraction step into speech;
An output step of outputting the sound from an output unit provided in the wearing tool;
An input step for receiving an input from a user from an input unit provided in the wearing tool;
And a control step of controlling a playback speed of the sound output from the output unit based on an input from the user received via the input unit.

Images