JP2020036123A - Wearable terminal and display system - Google Patents

Abstract

To utilize a wearable terminal in telephone conversation through mobile communication.SOLUTION: A screen display control unit 201 displays, on a display surface, a screen of a processor operating according to a user's operation of an operation device superimposed on the background. A reception time vibration control unit 203 controls a vibration device to vibrate when there is an incoming call to communication means. A received speech voice data output unit 204 acquires received speech voice data received by the communication means and outputs, to sound emission means, a received speech voice shown by the received speech voice data. An utterance text data generation unit 104 generates utterance text data indicating a sentence input with the use of the operation device. An utterance voice data generation unit 205 generates utterance voice data showing a voice of the sentence input with the use of the operation device. An utterance voice data output unit 206 outputs the generated utterance voice data as utterance data of mobile communication.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a wearable terminal.

  Patent Literature 1 discloses that a head-mounted display including a half mirror that reflects image light and projects the image onto an observer's eye can reduce visibility from the outside when image light transmitted through the half mirror is transmitted. A technique of providing a visibility inhibiting member is disclosed.

JP 2012-159680 A

By displaying a screen of a personal computer or the like on a display surface of a head-mounted type wearable terminal in a superimposed manner in a real space, it is possible to easily work in an open space where neither a display nor a partition is installed. In such an environment, it is common that no telephone is installed, so communication using a mobile phone is also important.
Therefore, an object of the present invention is to utilize a wearable terminal for a call by mobile communication.

  In order to achieve the above object, the present invention relates to a head-mounted wearable terminal, which includes a transmissive display surface located in a field of view of a user wearing the terminal, A display control unit that superimposes a screen of a processing device operating on a background and displays the screen on the display surface, and acquires audio data received by a communication unit that performs mobile communication, and emits a reception audio indicated by the audio data. A wearable terminal including a reception output unit that outputs to a sound unit.

  A generating unit configured to generate voice data indicating voice of a sentence input using the operation device and displayed on the display surface; and an utterance output configured to output the generated voice data as utterance data of mobile communication. Unit may be provided.

  Further, text data indicating a text input using the operation device and displayed on the display surface is generated, and voice data indicating a voice of the text is generated from the text data, and the voice data is transmitted to a mobile communication utterance. A text output unit that outputs the data to a voice processing device that transmits the data may be provided.

  A specifying unit for specifying a user who has transmitted the incoming call received by the communication unit; and a requesting unit for requesting a connection to a destination of the user registered in a real-time communication service used by the specified user. May be provided.

  Also, a vibration unit that vibrates when an incoming call to the communication unit is received may be provided.

  The display control unit may display an image indicating the incoming call on the display surface when the incoming call to the communication unit is received.

  Further, the display control unit does not display the image if the importance of the screen displayed on the display surface is equal to or higher than a predetermined reference when there is an incoming call to the communication unit, and The unit may vibrate when the importance is at least equal to or higher than the reference.

  In addition, when the operation frequency to the operation device when there is an incoming call to the communication unit is equal to or higher than a threshold, the image is not displayed, and the vibrating unit determines that the operation frequency is at least equal to or higher than the threshold. May be vibrated.

  The present invention also provides the wearable terminal described above, an operation device operated by a user wearing the wearable terminal, and an operation performed according to an operation on the operation device, and a screen of the own device is displayed on the wearable terminal. And a display system comprising the processing device.

  ADVANTAGE OF THE INVENTION According to this invention, a wearable terminal can be utilized by the telephone call by mobile communication.

FIG. 1 is a diagram illustrating an entire configuration of an ARPC system according to an embodiment. Diagram showing hardware configuration of personal computer Diagram showing hardware configuration of wearable glass Diagram showing the configuration of the projection device Diagram showing functional configuration realized by ARPC system Diagram showing an example of the displayed incoming image Diagram showing an example of the displayed input image The figure showing an example of the operation procedure in a telephone call process Diagram showing the functional configuration realized in the modified example Diagram showing the functional configuration realized in the modified example Diagram showing an example of the displayed RTCS screen Diagram showing an example of the importance table

[1] Embodiment FIG. 1 shows an overall configuration of an ARPC system 1 according to an embodiment. The ARPC system 1 is a system for using a PC (Personal Computer) using an AR (Augmented Reality) technique.

  The ARPC system 1 includes a personal computer 10 and a wearable glass 20. The personal computer 10 is a so-called personal computer, and includes an operation device such as a keyboard. The personal computer 10 of the present embodiment is integrated with a keyboard, and a component of the personal computer main body such as a processor is stored in a housing of the keyboard. The personal computer 10 performs operations such as execution of processing in accordance with operations on those operation devices. The personal computer 10 is an example of the “processing device” of the present invention.

  The wearable glass 20 is a wearable terminal that is used by being worn on the user's head (that is, a head-mounted type), and is specifically a glasses-type wearable terminal. The wearable glass 20 is an example of the “wearable terminal” of the present invention. In this embodiment, the personal computer 10 and the wearable glass 20 are connected by wire as shown in FIG. The connection between the two devices may be wireless.

  The wearable glass 20 has a display surface on which the background is seen through, and displays the screen of the personal computer 10 on the display surface. The user viewing the displayed screen, that is, the user wearing the wearable glass 20 operates the operation device included in the personal computer 10. As described above, the personal computer 10 operates according to the operation on the operation device, and the screen of the own device is displayed on the wearable glass 20.

  FIG. 2 shows a hardware configuration of the personal computer 10. The personal computer 10 is a computer including a processor 11, a memory 12, a storage 13, a communication device 14, an input device 15, a bus 16, and a keyboard housing 17 for storing the devices. Note that the term “apparatus” can be read as a circuit, a device, a unit, or the like. In addition, one or more devices may be included, or some devices may not be included.

  The processor 11 controls an entire computer by operating an operating system, for example. The processor 11 may be configured by a central processing unit (CPU) including an interface with a peripheral device, a control device, an arithmetic device, a register, and the like. Further, the processor 11 reads out a program (program code), a software module, data, and the like from the storage 13 and / or the communication device 14 to the memory 12, and executes various processes according to these.

  The number of processors 11 that execute various processes may be one, or two or more, and two or more processors 11 may execute various processes simultaneously or sequentially. Further, the processor 11 may be implemented by one or more chips. The program may be transmitted from a network via a telecommunication line.

  The memory 12 is a computer-readable recording medium, and includes at least one of a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a RAM (Random Access Memory), and the like. May be done. The memory 12 may be called a register, a cache, a main memory (main storage device), or the like. The memory 12 can store the above-described programs (program codes), software modules, data, and the like.

  The storage 13 is a computer-readable recording medium, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (for example, a compact disk, a digital versatile disk, a Blu- (registered trademark), a smart card, a flash memory (for example, a card, a stick, a key drive), a floppy (registered trademark) disk, and a magnetic strip. The storage 13 may be called an auxiliary storage device.

  The storage medium described above may be, for example, a database including the memory 12 and / or the storage 13, a server, or any other suitable medium. The communication device 14 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like. In the present embodiment, the communication device 14 includes a mobile communication device 141. The mobile communication device 141 is hardware for performing mobile communication via a base station, and is an example of the “communication unit” of the present invention.

  The input device 15 is an operation device that receives an operation for inputting information. In this embodiment, the input device 15 includes a keyboard 151 and a mouse 152 (may be a touch pad) as operation devices. Each device such as the processor 11 and the memory 12 can access each other via a bus 16 for communicating information. The bus 16 may be constituted by a single bus, or may be constituted by different buses between the devices.

  The personal computer 10 and the like include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The hardware may implement some or all of the functional blocks. For example, the processor 11 may be implemented by at least one of these hardware.

  FIG. 3 shows a hardware configuration of the wearable glass 20. The wearable glass 20 is a computer including a processor 21, a memory 22, a storage 23, a communication device 24, an input device 25, a bus 26, a projection device 30, and a frame 40. The processor 21 to the communication device 24 are devices of the same type, although there are differences in performance, specifications, and the like from the device of the same name shown in FIG.

  The input device 25 includes, for example, a touch pad as an operation device that can be operated by a user wearing the wearable glass 20 without looking at the user. The projection device 30 projects the image indicated by the image data supplied from the external device to the eye of the user wearing the wearable glass 20. The projection device 30 includes a projection unit 31 and an optical path forming unit 32. The projection unit 31 projects light indicating the image described above. The optical path forming unit 32 forms an optical path from the projection unit 31 to the user's eyes, and guides the light projected by the projection unit 31 to the user's eyes.

  FIG. 4 shows a configuration of the projection device 30. FIG. 4 illustrates a projection unit 31L that projects light to the left eye and a projection unit 31R that projects light to the right eye. Further, as members provided in the optical path forming unit 32, the light guide plate 321, mirrors 322 L and 322 R (hereinafter, referred to as “mirror 322”), and beam splitters 323 L and 323 R (if not distinguished, “beam splitter”) 323 ”).

  The light guide plate 321 is an elongated plate-shaped member. When the wearable glass 20 is worn by the user, the light guide plate 321 is positioned in front of the user's eyes, and the longitudinal direction of the light guide plate 321 is in the direction in which the user's eyes are lined up, that is, in the left-right direction for the user. Has become. In the description below, the arrangement of each unit is represented by the position on the left or right as viewed from the user wearing the wearable glass 20. For example, the projection unit 31L is arranged on the left side of the user's eye, and the projection unit 31R is arranged on the right side of the user's eye.

  The mirror 322L is provided at the left end inside the light guide plate 321, and the mirror 322R is provided at the right end inside the light guide plate 321. The mirror 322L reflects the projection light projected from the projection unit 31L and incident on the light guide plate 321 toward the right side (the side where the beam splitter 323L is located). The mirror 322R reflects the projection light projected from the projection unit 31R and incident on the light guide plate 321 toward the left side (the side where the beam splitter 323R is located).

  In FIG. 4 and subsequent figures, the optical path of the projection light is represented by lines and arrows (indicating the traveling direction of the projection light). Although the projection light reflected by both mirrors travels while reflecting inside the light guide plate 321, the optical path is represented by a simple straight line for easy viewing of the drawing. The reflected projection light reaches the beam splitter 323 on both the left and right sides. The beam splitter 323 is a plate-like member (also referred to as a half mirror) that transmits part of incident light and reflects part of the light.

  The beam splitter 323L is arranged in the field of view of one eye (left eye) of the user wearing the terminal. The beam splitter 323L has a reflection surface 324L that reflects the projection light that has been reflected by the mirror 322L and has traveled inside the light guide plate 321. On the opposite side of the reflection surface 324L, a transmission surface 325L that transmits light from the outside world is provided. Have. The projection light reflected by the reflection surface 324L reaches the left eye of the user, and the image indicated by the projection light is reflected on the left eye of the user.

  The beam splitter 323R is arranged in the field of view of one eye (right eye) of the user wearing the terminal. The beam splitter 323R has a reflection surface 324R that reflects the projection light that has been reflected by the mirror 322R and has traveled inside the light guide plate 321. The beam splitter 323R has a transmission surface 325R that transmits light from the outside on the opposite side of the reflection surface 324R. Have. The projection light reflected by the reflection surface 324R reaches the right eye of the user, and the image indicated by the projection light is reflected on the right eye of the user.

  Thus, the reflection surfaces 324L and 324R are surfaces on which an image indicated by the projection light is displayed. Further, when viewed from the user, the background ahead of the reflection surfaces 324L and 324R can be seen through. That is, the reflective surfaces 324L and 324R are transmissive display surfaces located in the field of view of the user wearing the terminal (wearable glass 20), and are examples of the “display surface” of the present invention. Hereinafter, when these are not distinguished, they are referred to as “display surface 324”.

  The frame 40 is a component that supports the projection device 30 and fixes (attaches) the own terminal to the user's head. The frame 40 is, for example, a metal or plastic part having a shape similar to a frame of glasses. The frame 40 is provided with a vibration device 41 and a connector 42. The vibration device 41 is, for example, a device (also referred to as a “vibrator”) that includes a motor and a weight, and generates vibration when the motor rotates the weight. The vibration device 41 is an example of the “vibration unit” of the present invention.

  The connector 42 is a component for connecting sound emitting means for outputting sound (for example, the earphone 50 in the figure). The earphone 50 converts an analog signal indicating a sound supplied via the connector 42 into a sound and outputs the sound, that is, emits a sound. The earphone 50 has a cord long enough to reach the ear of a user who wears the wearable glass 20 on the head while connected to the connector 42.

Each device included in the ARPC system 1 stores a program provided by the system, and a function group described below is realized by each processor executing the program and controlling each unit.
FIG. 5 shows a functional configuration realized by the ARPC system 1. The personal computer 10 includes a screen data generation unit 101, a mobile communication reception processing unit 102, an utterance input reception unit 103, an utterance text data generation unit 104, and a mobile communication transmission processing unit 105. The wearable glass 20 includes a screen display control unit 201, a data acquisition unit 202, a reception vibration control unit 203, a received voice data output unit 204, a voice voice data generation unit 205, and a voice voice data output unit 206. Prepare.

  The screen display control unit 201 superimposes the screen of the processing device that operates in response to the operation of the operation device by the user on the display surface 324 (reflection surfaces 324L and 324R illustrated in FIG. 4). The screen display control unit 201 is an example of the “display control unit” of the present invention. In this embodiment, the screen display control unit 201 displays a screen (hereinafter, referred to as a “PC screen”) of the personal computer 10 (a processing device that operates in response to operations of the keyboard 151 and the mouse 152) on the display surface 324.

  For example, when the user performs an operation for displaying a screen, the screen display control unit 201 transmits request data for requesting screen data indicating the personal computer screen to the personal computer 10. The personal computer 10 supplies the received request data to the screen data generation unit 101. The screen data generation unit 101 generates screen data requested by the supplied request data, that is, screen data indicating a personal computer screen of the own apparatus.

  The screen data generation unit 101 transmits the generated screen data to the wearable glass 20. The screen display control unit 201 causes the display screen 324 to display the personal computer screen indicated by the transmitted screen data. So far, the functions related to the display of the personal computer screen have been described. From now on, the functions related to the call by the mobile communication will be described.

  The mobile communication reception processing unit 102 performs processing (reception processing) related to reception in voice communication in mobile communication performed by the mobile communication device 141 illustrated in FIG. For example, when there is an incoming call to the mobile communication device 141, the mobile communication reception processing unit 102 generates the caller's telephone number and incoming data indicating that there is an incoming call, and transmits it to the wearable glass 20. . When a call is started, the mobile communication reception processing unit 102 generates received voice data indicating the received voice (hereinafter, referred to as “received voice”), and transmits the generated voice data to the wearable glass 20.

  The data acquisition unit 202 of the wearable glass 20 acquires data transmitted from the personal computer 10. Upon acquiring the incoming data, the data acquiring unit 202 supplies the acquired incoming data to the screen display control unit 201 and the reception vibration control unit 203. When the incoming data is supplied, that is, when there is an incoming call to mobile communication device 141 (communication means), screen display control section 201 causes display surface 324 to display an image indicating the incoming call (an incoming image). .

  FIG. 6 shows an example of the displayed incoming image. In FIG. 6A, the screen display control unit 201 displays the incoming image C1 as a part of the personal computer screen A1 on the display surface 324 (the end of the display surface 324 is transparent and cannot be clearly seen by the user, so it is represented by a two-dot chain line). )). In FIG. 6B, the screen display control unit 201 displays the incoming call image C2 at the end of the display surface 324 so as to protrude from the personal computer screen A1.

  If the display surface 324 is sufficiently wide, the screen display control unit 201 may display the incoming image C2 so as not to overlap the personal computer screen A1 at all. In the case of the incoming image C1, it is displayed on the personal computer screen A1 that the user is paying attention to, so it is easy to notice the incoming call, and in the case of the incoming image C2, it is hard to obstruct the viewing of the personal computer screen A1. Also, in any of the incoming images, the telephone number of the caller can be transmitted.

  The reception-time vibration control unit 203 controls the vibration device 41 shown in FIG. 4 to vibrate when incoming data is supplied, that is, when there is an incoming call to the mobile communication device 141 (communication means). When the vibration device 41 vibrates, the frame 40 provided with the vibration device also vibrates, and the vibration is transmitted to the user's head. This makes it easier for the user to notice the incoming call than when only the incoming image is displayed.

  The screen display control unit 201 displays a response button B1 and a rejection button B2 together with the incoming image as shown in FIG. 6 (these buttons are also shown as a part of the personal computer screen A1 in FIG. 6A). 6 (b), it is displayed irrespective of the personal computer screen A1). When the reject button B2 is operated, the mobile communication reception processing unit 102 ends the reception processing, and when the response button B1 is operated, starts the telephone call and starts generation and transmission of the reception voice data. . Upon acquiring the received voice data, the data acquisition unit 202 supplies the received voice data to the received voice data output unit 204.

  The received voice data output unit 204 acquires the supplied received voice data, that is, the received voice data received by the mobile communication device 141 (communication means), and outputs the received voice indicated by the received voice data to the sound emitting means ( In this embodiment, the signal is output to the earphone 50 shown in FIG. The received voice data output unit 204 is an example of the “received voice output unit” of the present invention. The receiving voice data output unit 204 converts the receiving voice data, which is digital data, into an analog signal and supplies the analog signal to the earphone 50 via the connector 42.

  The earphone 50 emits a received voice indicated by the supplied analog signal. This allows the user to hear the incoming voice of the incoming mobile communication. On the other hand, the reply from the user is made not by his own voice but by text input. When the response button B1 is operated, the screen display control unit 201 displays an input screen for inputting the utterance content (answer in this case) from the user.

  FIG. 7 shows an example of the displayed input image. The screen display control unit 201 displays a character string “Please input the utterance content”, an input field D1, and an enter button B3 as the utterance content input screen A2. Upon hearing the received voice, the user performs an operation of inputting a reply in the input field D1. The utterance input receiving unit 103 receives an input of the utterance content on the utterance content input screen.

  The utterance input receiving unit 103 receives an input using an operation device (a keyboard 151 and a mouse 152) included in the own device (the personal computer 10). Note that the utterance input receiving unit 103 does not receive a voice input regardless of whether or not the own device has a microphone. This is because, when the utterance content is uttered by voice, the content is known to surrounding people.

  The utterance input reception unit 103 supplies the utterance content whose input has been received to the utterance text data generation unit 104. The utterance text data generation unit 104 generates the supplied utterance content, that is, text data (hereinafter, referred to as “utterance text data”) indicating a sentence input using the operation device. The utterance text data generation unit 104 transmits the generated utterance text data to the wearable glass 20.

  The screen display control unit 201 of the wearable glass 20 causes the utterance content indicated by the transmitted utterance text data to be displayed in the input field D1. In the example of FIG. 7 is displayed utterance content input of "Hello. It's hot.". When the user performs an operation of pressing the confirmation button B3 in this state, the screen display control unit 201 supplies the utterance text data generation unit 205 with utterance text data indicating the content of the utterance being displayed.

  The utterance voice data generation unit 205 generates utterance content indicated by the supplied utterance text data, that is, utterance voice data indicating the voice of a sentence input using the operation device. The utterance voice data generation unit 205 is an example of the “generation unit” of the present invention. The utterance voice data generation unit 205 generates utterance voice data using a well-known voice synthesis technique, for example, generating voice data by combining phoneme data stored in advance.

  The utterance voice data generation unit 205 supplies the generated utterance voice data to the utterance voice data output unit 206. The utterance voice data output unit 206 outputs the supplied utterance voice data, that is, the utterance voice data generated by the utterance voice data generation unit 205, as the utterance data of the mobile communication. The utterance voice data output unit 206 is an example of the “utterance output unit” of the present invention.

  The utterance data of the mobile communication referred to here is data indicating the utterance content to be transmitted to the other party in the mobile communication. In this embodiment, the input is performed by a character string (text), but utterance voice data indicating voice is output as utterance data. The utterance voice data output unit 206 outputs the utterance voice data to the personal computer 10. The personal computer 10 supplies the output utterance voice data to the mobile communication transmission processing unit 105.

  The mobile communication transmission processing unit 105 performs processing (transmission processing) related to transmission in a voice call in mobile communication performed by the mobile communication device 141 illustrated in FIG. In this embodiment, the mobile communication transmission processing unit 105 performs a process of transmitting the supplied utterance voice data as it is to the mobile communication terminal of the call destination as a transmission process. In this case, the called mobile communication terminal emits the uttered voice indicated by the uttered voice data transmitted in the same manner as a normal voice call.

The personal computer 10 and the wearable glass 20 perform call processing for making a call by mobile communication based on the above configuration.
FIG. 8 shows an example of an operation procedure in the call processing. This operation procedure is started, for example, when a user receives a call while the wearable glass 20 is being worn and the screen of the personal computer 10 is being displayed.

  First, the personal computer 10 (mobile communication reception processing unit 102) generates incoming telephone data and incoming data indicating that there is an incoming call (step S11), and transmits the generated incoming data to the wearable glass 20 (step S11). Step S12). When the incoming data is transmitted, the wearable glass 20 (the reception-time vibration control unit 203) controls the vibration device 41 to vibrate (step S13).

  Next, wearable glass 20 (screen display control unit 201) displays an incoming call image indicating the incoming call on display surface 324 (step S14). The operations of steps S13 and S14 may be performed in the opposite order or may be performed in parallel. Subsequently, when a response operation is performed on the screen on which the incoming image is displayed, the personal computer 10 receives the operation (step S21). Then, the personal computer 10 (mobile communication reception processing unit 102) starts a telephone call to generate received voice data (step S22), and transmits the generated received voice data to the wearable glass 20 (step S23).

  Wearable glass 20 (received voice data output unit 204) outputs the received voice indicated by the transmitted received voice data to sound emitting means (earphone 50) (step S24). Next, the wearable glass 20 (screen display control unit 201) displays an utterance content input screen on the display surface 324 (step S31). The personal computer 10 (the utterance input receiving unit 103) receives an input of the utterance content on the utterance content input screen (step S32).

  Next, the personal computer 10 (the utterance text data generation unit 104) generates utterance text data indicating the input sentence (Step S33) and transmits it to the wearable glass 20 (Step S34). The wearable glass 20 (screen display control unit 201) displays the utterance content indicated by the transmitted utterance text data on the display surface 324 (step S35). The wearable glass 20 receives the confirmation operation when performed on the utterance content input screen (step S36).

  Next, the wearable glass 20 (the utterance voice data generation unit 205) generates the utterance voice data indicating the determined utterance content, that is, the voice of the sentence input using the operation device (step S37). Subsequently, the wearable glass 20 (the utterance voice data output unit 206) outputs the generated utterance voice data to the personal computer 10 as the utterance data of the mobile communication (Step S38).

  The personal computer 10 (the mobile communication transmission processing unit 105) performs a process of transmitting the transmitted uttered voice data to the mobile communication terminal of the communication destination as it is (a process related to the transmission in the voice communication in the mobile communication). ) (Step S39). Thereafter, the operations of steps S22 to S39 are repeated, and when the user performs an operation to end the call, the operation procedure ends.

  In the present embodiment, the voice of the call by the communication means (the mobile communication device 141 of the personal computer 10) for performing the mobile communication is output from the wearable glass 20. The operation of rejecting or answering the call is also performed through a screen displayed on the wearable glass 20. As described above, according to the present embodiment, the wearable terminal (wearable glass 20) can be used for a call by mobile communication.

  Further, in the present embodiment, the user who has started the mobile communication can make a response by inputting the utterance content using the operation device (the keyboard 151 and the mouse 152) of the personal computer 10 without uttering a voice. As a result, it is possible to make a call by mobile communication without the contents being known to the surrounding third parties.

[2] Modifications The embodiment described above is merely an example of the embodiment of the present invention, and may be modified as follows.

[2-1] Wearable terminal The wearable terminal of the present invention is not limited to the wearable glass 20. For example, a well-known method (for example, a method using polarized light) other than the method illustrated in FIG. 4 may be used as a method of forming an optical path. Further, the wearable glass 20 is of a glasses type, but may be of a goggles type or a single glasses type (in that case, one display surface is provided). In short, any type of wearable terminal may be used as long as it is a head-mounted type and has a transmissive display surface located in the field of view of the user.

[2-2] Processing Device The processing device of the present invention is not limited to the personal computer 10. For example, an ordinary desktop personal computer may be used instead of the integrated keyboard. Further, a processing device having a display surface such as a notebook computer, a smartphone, or a tablet terminal may be used.

  In that case, the display on the display surface of the processing device main body may be turned off, or the processing device main body may be put in a bag or the like, so that the display surface of the processing device main body may not be seen by a third party. . The screen displayed on the wearable terminal may be a screen of the processing device, and need not be a personal computer screen. Any of the processing devices may be any device that operates in response to an operation of an operation device (an external keyboard or the like in the case of a smartphone or a tablet terminal) by a user.

[2-3] Communication Means In the embodiment, the personal computer 10 as the processing device is provided with the communication means for mobile communication. However, the present invention is not limited to this. For example, the other processing device (such as a smartphone) described above may include the communication unit, or the wearable glass 20 that is a wearable terminal may include the communication unit. Regardless of which device is provided with the communication means, it is sufficient that data relating to mobile communication can be exchanged between the device and the wearable terminal.

[2-4] Sound emitting means The sound emitting means by which the wearable terminal outputs the received voice is not limited to the one described in the embodiment (an external earphone). For example, a headphone or a speaker (such as a bone conduction speaker) may be used, or they may be built in the wearable terminal. In short, any sound emitting means may be used as long as the user wearing the wearable terminal on the head can hear the emitted sound. However, in order to prevent information leakage, it is desirable that the emitted call voice is not heard by surrounding third parties.

[2-5] Utterance Voice Data Generation Device In the embodiment, the wearable glass 20 generates the utterance voice data, but another device may generate the utterance voice data.
FIG. 9 shows a functional configuration realized in this modification. In FIG. 9, a personal computer 10a including an utterance voice data generation unit 106 in addition to the units shown in FIG. And a wearable glass 20a having the same.

  When the confirm button B3 is operated in the state shown in FIG. 7, the screen display control unit 201 of this modification supplies utterance text data indicating the content of the utterance being displayed to the utterance text data output unit 207. The utterance text data output unit 207 outputs the supplied utterance text data, that is, the sentence data indicating the text input using the operation device and displayed on the display surface 324 to the personal computer 10a. The utterance text data output unit 207 is an example of the “sentence output unit” of the present invention.

  The personal computer 10a supplies the output utterance text data (sentence data) to the utterance voice data generation unit 106. Speech data generating unit 106, the supplied speech text data, that is, speech data indicating the voice of the sentence from the text data described above (in the example of Figure 7, "Hello. It's hot." Sentence) Generate For generating the utterance voice data, a well-known voice synthesis technique is used similarly to the utterance voice data generation unit 205.

  The utterance voice data generation unit 106 supplies the generated utterance voice data to the mobile communication transmission processing unit 105. As in the embodiment, the mobile communication transmission processing unit 105 performs processing of transmitting the supplied utterance voice data as it is to the mobile communication terminal of the call destination as transmission processing. The personal computer 10a that performs the processing on the uttered voice (the processing of generating and transmitting the uttered voice data) is an example of the “voice processing device” of the present invention.

  In this modification, as described above, the generation processing of the uttered voice data is performed not by the wearable terminal but by the processing device (the personal computer 10a). In general, since a wearable terminal is made light and thin, the processing capability as an information processing device is suppressed lower than that of a normal personal computer or the like. In this modification, the processing device having a higher processing capability than the wearable terminal performs the generation processing of the utterance voice data, thereby improving the quality of the utterance voice as compared with the case where this processing is performed in the wearable terminal. In addition, the time lag due to the processing on the uttered voice can be reduced.

  Note that the above-described audio processing device need not be a processing device that displays a screen on a wearable terminal, may be an external server device, or may be a device other than a processing device such as a smartphone owned by a user. You may. Even in those cases, if the voice processing device has a higher processing capability than the wearable terminal, the effect of the above-described modification can be obtained.

[2-6] Calls other than voice In each of the above examples, although the user himself does not make a voice, a call was made using uttered voice data generated by a wearable terminal or the like, but the present invention is not limited to this.
FIG. 10 shows a functional configuration realized in the present modification. FIG. 10 illustrates a personal computer 10b including each unit except for the mobile communication transmission processing unit 105 illustrated in FIG.

  Also, FIG. 10 illustrates a wearable glass 20b including a transmission source user specifying unit 208 and a service processing unit 209 instead of the utterance voice data generation unit 205 and the utterance voice data output unit 206 illustrated in FIG. In the present modification, the data acquisition unit 202 supplies the acquired incoming data to the transmission source user identification unit 208. The origin user identification unit 208 identifies the origin user of the incoming call received by the mobile communication device 141 (communication means). The source user identification unit 208 is an example of the “identification unit” of the present invention.

  The caller user specifying unit 208 specifies the caller user using, for example, a user table in which the caller's telephone number registered in advance by the user of the own device (the wearable glass 20b) is associated with the user. . The source user specifying unit 208 may specify the source user by inquiring of an external device having information such as a user table. The source user specifying unit 208 supplies the service processing unit 209 with identification information for identifying the specified user.

  The service processing unit 209 performs a process (hereinafter, “service”) related to a real-time communication service (RTCS) used by the user indicated by the supplied identification information, that is, the user specified by the source user specifying unit 208. Processing). The real-time communication service (RTCS) is a service that enables real-time communication using a data communication line, and is sometimes called “chat” or the like.

  The information exchanged by the RTCS is mainly text data, but other images and moving images may be exchanged. However, voice exchange is excluded. The service processing unit 209 performs, for example, a process of requesting a connection to a destination of the user (user specified by the transmission source user specifying unit 208) registered in the RTCS as a service process. The service processing unit 209 is an example of the “request unit” of the present invention.

  The service processing unit 209 is supplied using, for example, a service table in which an RTCS used by another user registered in advance by the user of the own device and a user ID used in the RTCS are associated with each other. The RTCS and the user ID associated with the user indicated by the identified identification information are specified. This user ID is an example of the “address of the user registered in the RTCS” described above.

  The wearable glass 20b stores a program (browser program or application program) for using the main RTCS. The service processing unit 209 starts a program for using the specified RTCS, and performs a process of requesting a connection to the specified user ID. Specifically, the service processing unit 209 performs, for example, a search process for searching for the user ID on the activated RTCS screen as a process for requesting a connection.

  Further, the service processing unit 209 performs a request process of requesting a chat start with a user of a user ID found by the search as a process of requesting a connection. Note that these processes (search process and request process) are merely examples, and in addition to this, a process that is normally performed when starting exchange with a user in the RTCS to be used may be performed as a process for requesting a connection. . When the RTCS program is started, the screen display control unit 201 causes the display screen 324 to display the corresponding RTCS screen.

  FIG. 11 shows an example of the displayed RTCS screen. FIG. 11A shows a chat partner search screen A3 displayed on the display surface 324 of the wearable glass 20b. The chat partner search screen A3 displays a search key input field D2 and a search result E1 (that is, a search processing result). When the service processing unit 209 performs the process of selecting the search result E1 as the above-described request process, the screen display control unit 201 displays the chat screen A4 with the searched user (“User A” in this example) in FIG. It is displayed as shown in (b).

  On the chat screen A4, an input field D3 of the utterance content is displayed. When the user inputs the utterance content in the input field D3, the utterance input receiving unit 103 receives the input of the utterance content in the input field D2. Then, when the user performs an operation of pressing the confirm button, the utterance text data generation unit 104 generates utterance text data indicating the input text, and supplies the generated utterance text data to the screen display control unit 201.

  Screen display control unit 201 displays a speech content indicated by the supplied speech text data "Hello. It's hot!" The balloon image E2 contains the string as represented in FIG. 11 (c). The utterance text data generation unit 104 also supplies the generated utterance text data to the service processing unit 209. The service processing unit 209 transmits the supplied utterance text data to the destination of the user A specified as described above (the destination registered in the RTCS).

  The utterance content indicated by the transmitted utterance text data is displayed on the display means of the terminal used by the user A. FIG. 11D shows the chat screen A5 thus displayed. The chat screen A5, (in this example "user B"), a user who is a chat partner and the avatar E3 that shows, is the contents of the user's utterance B, "Hello. It's hot!" And balloon image E4 contain the string Is displayed.

  Further, since the chat screen A5 is a screen for chatting, an input field D4 of the utterance content is displayed like the chat screen A4. The user A may, for example, enter the utterance content in the input field D4 and continue the conversation by chat. In this case, the connection in the mobile communication may be terminated. Alternatively, the user A may continue the voice conversation in the connected mobile communication without performing the input in the input field D4. In this case, while listening to the voice of the user A, the user B has a conversation while inputting the utterance content in the input field D3.

  In any case, the user B does not state his or her utterance content by voice, and the displayed balloon image E2 is also displayed on the display surface 324 of the wearable glass 20b that can be viewed only by the user B. It is possible to eliminate the possibility that the utterance content is known to surrounding third parties. In addition, since the voice of the user A can be heard only from the earphone 50 to the user B, there is no possibility that the utterance content of the user A is known to a third party. In addition, since the chat connection with the user A is automatically performed as described above, the conversation using the RTCS can be started smoothly.

  In the above example, the wearable glass 20b executes the program for using the RTCS. However, the present invention is not limited to this, and the personal computer 10b may execute this program. In this case, a function corresponding to the service processing unit 209 is also realized in the personal computer 10b, and the service processing unit 209 may perform a process of requesting a connection via the function of the personal computer 10b.

[2-7] Notification Method of Incoming Call In the embodiment, when there is an incoming call, the user is notified of both the vibration and the incoming call image. . In this case, which notification is to be performed may be made different depending on the situation.

  In this modification, when the incoming data is supplied, that is, when there is an incoming call to the mobile communication device 141 (communication means), the screen display control unit 201 displays the screen displayed on the display surface 324. Determine importance. The screen display control unit 201 determines the degree of importance in accordance with, for example, the type of the program activated on the displayed personal computer screen. The screen display control unit 201 makes a determination using, for example, an importance table in which types of programs are associated with importance.

  FIG. 12 shows an example of the importance table. In the example of FIG. 12, the importance of “for file browsing and mail browsing” is “low”, the importance of “document creation” is “medium”, and the importance of “business and video playback” is “high”. ]. The screen display control unit 201 determines the importance associated with the type of the program activated at the time of the incoming call in the importance table as the importance of the screen.

  The screen display control unit 201 displays, for example, the incoming image described in the embodiment when the importance of “low” is determined, and displays the incoming image when the importance of “medium” and “high” is determined. Is displayed, and the determination result is notified to the reception-time vibration control unit 203. Upon receiving this notification, the reception-time vibration control unit 203 controls the vibration device 41 to vibrate, as in the embodiment. As described above, in the present modification, when the determined importance is equal to or higher than the predetermined reference (in this case, “medium”), the vibration device 41 vibrates, and the screen display control unit 201 does not display the incoming image. Is working like that.

  When the incoming image is displayed, as shown in FIG. 6, the incoming image may overlap and part of the information displayed on the personal computer screen may not be visible. In the present modified example, by controlling the display of the incoming image as described above, while the screen with high importance is displayed, the incoming image is not disturbed, and the vibration even if the incoming image is not displayed. The caller is made aware of the incoming call.

  The importance shown in FIG. 12 is merely an example, and the present invention is not limited to this. For example, a different importance may be determined based on a user's request, or the user may be able to set the importance. When the importance is determined to be less than the predetermined reference, the incoming image is displayed as described above, but the vibration device 41 may be vibrated together or not. In the present modification, the vibration device 41 may vibrate when the determined importance is at least equal to or higher than a predetermined reference.

  The method of notifying the incoming call is not limited to this. For example, the notification method may differ depending on the operation frequency to the operation device when there is an incoming call. In this case, for example, the screen display control unit 201 operates the operation device (keyboard 151 and mouse 152) when incoming data is supplied, that is, when there is an incoming call to the mobile communication device 141 (communication means). Calculate the frequency.

  The screen display control unit 201 stores, for example, an operation time when an operation is performed on the operation device, and when incoming data is supplied, the operation time is set to a unit period (for example, 10 seconds) before the time. The number of stored operation times is calculated as the operation frequency. The screen display control unit 201 displays the incoming image when the operation frequency less than the predetermined threshold is calculated, and receives the calculation result without displaying the incoming image when the operation frequency is higher than the threshold. The time vibration control unit 203 is notified.

  Upon receiving this notification, the reception-time vibration control unit 203 controls the vibration device 41 to vibrate, as in the embodiment. As described above, in the present modification, when the calculated operation frequency is equal to or greater than the threshold, the vibration device 41 vibrates, and the screen display control unit 201 operates so as not to display the incoming image.

  In a situation where the user is busy performing an operation, the user often concentrates on looking at the screen as compared with a situation where the operation is few. In the present modification, the incoming image is not disturbed in such a situation where the user is busy operating, and the incoming call is noticed by vibration even if the incoming image is not displayed.

[2-8] Apparatus for Realizing Each Unit The apparatus for realizing each function illustrated in FIG. 5 and the like is not limited to the apparatuses illustrated in those drawings. For example, the utterance text data generation unit 104 included in the personal computer 10 may be realized by the wearable glass 20 or another external device. Further, the operation performed by one function may be shared by two or more functions. For example, the display control of the personal computer screen and the display control of the incoming image performed by the screen display control unit 201 may be shared by different functions. In short, it is sufficient that these functions are realized as the entire ARPC system.

[2-9] Category of the Invention The present invention can be regarded as a processing system such as a personal computer and a wearable terminal such as a wearable glass, as well as a display system such as an ARPC system including such a device. Further, the present invention can be regarded as an information processing method for realizing the processing performed by each device, and also as a program for causing a computer that controls each device to function. This program may be provided in the form of a recording medium such as an optical disk in which the program is stored, or provided in a form in which the program is downloaded to a computer via a network such as the Internet and installed and made available. May be done.

[2-10] Processing Procedures, Etc. The processing procedures, sequences, flowcharts, and the like of the embodiments described in this specification may be interchanged as long as there is no inconsistency. For example, the methods described herein present elements of various steps in a sample order, and are not limited to the specific order presented.

[2-11] Handling of Input / Output Information, etc. Input / output information, etc., may be stored in a specific location (for example, a memory) or may be managed in a management table. Information that is input and output can be overwritten, updated, or added. The output information or the like may be deleted. The input information or the like may be transmitted to another device.

[2-12] Software Software is called an instruction, an instruction set, a code, a code segment, a program code, or a program, regardless of whether it is called software, firmware, middleware, microcode, a hardware description language, or another name. , Subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.

  In addition, software, instructions, and the like may be transmitted and received via a transmission medium. For example, if the software uses a wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or a wireless technology such as infrared, wireless and microwave, the website, server, or other When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

[2-13] Information, Signals The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., that can be referred to throughout the above description are not limited to voltages, currents, electromagnetic waves, magnetic or magnetic particles, optical or photons, or any of these. May be represented by a combination of

[2-14] System, Network As used herein, the terms “system” and “network” are used interchangeably.

[2-15] Meaning of “based on” As used herein, the term “based on” does not mean “based on” unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[2-16] "and", "or"
In this specification, as for a configuration that can be implemented by “A and B” or “A or B”, the configuration described in one expression may be used as the configuration described in the other expression. For example, when "A and B" are described, they may be used as "A or B" if they can be implemented without inconsistency with other descriptions.

[2-17] Variation of Aspects Each of the embodiments described in this specification may be used alone, in combination, or may be switched with execution. Further, the notification of the predetermined information (for example, the notification of “X”) is not limited to being explicitly performed, and is performed implicitly (for example, not performing the notification of the predetermined information). Is also good.

  Although the present invention has been described in detail, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described in this specification. The present invention can be embodied as modified and changed aspects without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description in this specification is for the purpose of illustration and description, and does not have any restrictive meaning to the present invention.

DESCRIPTION OF SYMBOLS 1 ... ARPC system, 10 ... Personal computer, 20 ... Wearable glass, 30 ... Projection device, 40 ... Frame, 41 ... Vibration device, 42 ... Connector, 50 ... Earphone, 101 ... Screen data generation part, 102 ... Mobile communication reception processing Unit, 103: utterance input reception unit, 104: utterance text data generation unit, 105: mobile communication transmission processing unit, 106: utterance voice data generation unit, 141: mobile communication device, 151: keyboard, 152: mouse, 201 ... Screen display control unit, 202 ... Data acquisition unit, 203 ... Reception vibration control unit, 204 ... Reception voice data output unit, 205 ... Utterance voice data generation unit, 206 ... Utterance voice data output unit, 207 ... Utterance text data output Unit, 208: source user identification unit, 209, service processing unit, 324, display surface.

Claims (9)

A head-mounted wearable terminal,
A transmissive display surface located in the field of view of the user wearing the terminal,
A display control unit that superimposes a screen of a processing device that operates in accordance with an operation of the operation device by the user and displays the screen on the display surface,
A receiving unit that acquires voice data received by a communication unit that performs mobile communication and outputs a received voice indicated by the voice data to a sound emitting unit. A generation unit that generates voice data indicating voice of a sentence displayed on the display surface that is input using the operation device;
The wearable terminal according to claim 1, further comprising: an utterance output unit configured to output the generated voice data as utterance data of mobile communication. Sentence data indicating a sentence displayed on the display surface as input using the operation device, generating audio data indicating the voice of the sentence from the sentence data, and using the audio data as speech data for mobile communication. The wearable terminal according to claim 1, further comprising a text output unit that outputs the sentence to a voice processing device that transmits the sentence. A specifying unit that specifies a user of a source of an incoming call received by the communication unit,
The wearable terminal according to any one of claims 1 to 3, further comprising: a request unit that requests connection to a destination of the user registered in the real-time communication service used by the specified user. The wearable terminal according to any one of claims 1 to 4, further comprising a vibration unit that vibrates when there is an incoming call to the communication unit. The wearable terminal according to claim 5, wherein the display control unit causes an image indicating the incoming call to be displayed on the display surface when the incoming call to the communication unit is received. The display control unit does not display the image when the importance of the screen displayed on the display surface is equal to or more than a predetermined reference when there is an incoming call to the communication unit,
The wearable terminal according to claim 6, wherein the vibration unit vibrates when the importance is at least equal to or higher than the reference. Do not display the image when the operation frequency to the operation device when there is an incoming call to the communication means is a threshold or more,
The wearable terminal according to claim 6, wherein the vibration unit vibrates when the operation frequency is at least equal to or greater than the threshold. A wearable terminal according to any one of claims 1 to 8,
An operation device operated by a user wearing the wearable terminal;
A processing device that operates in response to an operation on the operation device and displays a screen of the device on the wearable terminal.

Images