JP2016035632A - Menu selection system and menu selection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to easily select menu items according to the situation of the user.SOLUTION: A voice reproduction terminal arranges virtual icons I1 to I8 showing first to eighth menu items in a virtual space corresponding to a real space that includes a user. The voice reproduction terminal performs signal processing according to transfer characteristics of a sound corresponding to the positions where the virtual icons I1 to I8 are arranged and the position of the user, on voice signals indicating the first to eighth menu items, and outputs sounds according to the voice signals subjected to the signal processing from a speaker. A display terminal 20 displays the virtual icons I1 to I8 arranged in the virtual space. The voice reproduction terminal detects the movement of the body of the user and the amount of the movement. For example, when the movement of moving the body in the direction where the virtual icon I1 is arranged and the amount of the detected movement exceeds a threshold, the voice reproduction terminal selects the first menu item shown by the virtual icon I1.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a technique for selecting a menu item.

  A technique for realizing selection of a menu item using a virtual surround technique is known. For example, in Patent Document 1, a plurality of menu sound images corresponding to a plurality of menu items are localized in a virtual three-dimensional sound field, and an arbitrary menu sound image is selected by the wearer moving his / her head and selected. It describes that a menu item corresponding to a menu sound image is executed. Further, as disclosed in Patent Document 2, a technique for displaying a virtual object using AR (Augmented Reality) is known.

JP 2000-194460 A Japanese Patent Application Laid-Open No. 2014-2151569

When the user selects a menu item by moving his / her body as in the invention described in Patent Document 1, it may be difficult to select the menu item depending on the situation of the user. For example, when the user is on a crowded train, it is difficult to select a menu item because the body cannot move much.
An object of this invention is to make it easy to select a menu item according to a user's condition.

  The present invention provides an arrangement unit that arranges a virtual object indicating a menu item in a virtual space corresponding to a real space where a user exists, a position in the virtual space where the virtual object is arranged, and the virtual space on the virtual space A signal processing unit that applies signal processing corresponding to a sound transfer characteristic corresponding to a user's position to an audio signal indicating the menu item; and a speaker that outputs sound corresponding to the audio signal subjected to the signal processing; A display unit that displays the virtual object arranged in the virtual space, a detection unit that detects the movement of the user's body and the amount of movement of the movement, and moves the body in the direction in which the virtual object is arranged When a motion is detected and the motion amount of the detected motion exceeds a threshold value, a selection unit that selects the menu item and a speaker having the speaker A terminal, a measuring unit for measuring a distance between the second terminal having the display unit, according to the distance that is the measurement, which provides a menu selection system and a changing unit for changing the threshold value.

  The speaker outputs a first notification sound until the operation amount reaches the threshold after the operation is started, and outputs a second notification sound when the operation amount exceeds the threshold. Good.

  The speaker may output a third notification sound when the threshold is changed.

  The changing unit may change the threshold value according to a situation or action of the user.

  The present invention also includes a step of arranging a virtual object indicating a menu item in a virtual space corresponding to a real space in which a user exists, a position in the virtual space where the virtual object is arranged, and a position on the virtual space Applying signal processing corresponding to the sound transfer characteristic corresponding to the user's position to the audio signal indicating the menu item, and outputting sound corresponding to the audio signal subjected to the signal processing from a speaker Displaying the virtual object arranged in the virtual space on a display unit, detecting the movement of the user's body and the amount of movement of the movement, and the body in the direction in which the virtual object is arranged Is detected, and the amount of motion of the detected motion exceeds a threshold value, the step of selecting the menu item is detected. And a step of measuring a distance between the first terminal having the speaker and the second terminal having the display unit, and changing the threshold according to the measured distance. Provide a selection method.

  ADVANTAGE OF THE INVENTION According to this invention, it can make it easy to select a menu item according to a user's condition.

The figure which shows the structure of a menu selection system. The figure which shows the hardware constitutions of an audio | voice reproduction terminal. The figure which shows the function structure of the control part of an audio | voice reproduction terminal. The figure which shows the hardware constitutions of a display terminal. The figure which shows the function structure of the control part of a display terminal. The sequence chart which shows operation | movement of a menu selection system. The figure which shows an example of arrangement | positioning of a virtual icon. The figure which shows the example of a display of a virtual icon. The figure which shows an example of a movement of a display terminal. The figure which shows the example of a display of a virtual icon. The figure explaining an example of the operation | movement which selects a menu item.

1. Configuration (1) Overall Configuration of System FIG. 1 is a diagram showing a configuration of a menu selection system 1. The menu selection system 1 includes an audio playback terminal 10 and a display terminal 20. The audio reproduction terminal 10 is an earphone type wearable computer. The audio reproduction terminal 10 is worn on the user's ear. The display terminal 20 is a wristwatch-type wearable computer. The display terminal 20 is worn on the user's arm. The audio reproduction terminal 10 and the display terminal 20 are paired in advance. The audio reproduction terminal 10 and the display terminal 20 cooperate to realize menu item selection by the user.

(2) Configuration of Audio Playback Terminal 10 The audio playback terminal 10 includes a right ear casing 101, a left ear casing 102, a cable 103, and a main body 104. The housing 101 is attached to the right ear of the user. The casing 101 accommodates a speaker 16-1. The housing 102 is attached to the user's left ear. The casing 102 accommodates a speaker 16-2. The cable 103 connects the housing 101 and the main body 104, and the housing 102 and the main body 104. The main body 104 reproduces sound in a stereo manner by supplying two-channel audio signals to the speakers 16-1 and 16-2.

  FIG. 2 is a diagram illustrating a hardware configuration of the audio reproduction terminal 10. The audio reproduction terminal 10 has a function of arranging virtual icons corresponding to menu items in a virtual space corresponding to a real space where the user exists, and allowing the user to recognize the arrangement of the virtual icons using virtual surround. The user can select a menu item indicated by the virtual icon by performing an operation of moving the body in the direction in which the virtual icon is arranged. The audio reproduction terminal 10 includes a control unit 11, a storage unit 12, a communication unit 13, an operation unit 14, a microphone 15, speakers 16-1 and 16-2, a positioning unit 17, a sensor unit 18, And a power source 19. These components are connected via a bus or cable 103. Note that components other than the speakers 16-1 and 16-2 are basically housed in the main body 104. However, the sensor unit 18 is accommodated in the housing 101 or 102.

  The control unit 11 executes a program stored in the storage unit 12 to perform calculation of data and control of other components. For example, a CPU is used as the control unit 11. The storage unit 12 stores programs executed by the control unit 11 and various data. The storage unit 12 includes a main storage device and an auxiliary storage device. For example, a RAM is used as the main storage device. For example, a flash memory is used as the auxiliary storage device.

  The communication unit 13 is a communication interface for performing data communication. The communication unit 13 communicates with the display terminal 20 in accordance with a near field communication standard such as Bluetooth (registered trademark). The operation unit 14 inputs a signal corresponding to a user operation to the control unit 11. For example, a touch sensor is used for the operation unit 14. The microphone 15 receives sound and inputs a signal corresponding to the received sound to the control unit 11. The speakers 16-1 and 16-2 output sound according to the audio signal.

  The positioning unit 17 measures the current position of the audio playback terminal 10. For the positioning unit 17, for example, a GPS (Global Positioning System) receiver is used. The power source 19 supplies power to each component of the audio playback terminal 10. For example, a battery is used as the power source 19.

  The sensor unit 18 is used to detect a user operation. The sensor unit 18 includes, for example, an acceleration sensor 181, an angular velocity sensor 182, and a geomagnetic sensor 183. The acceleration sensor 181 measures triaxial acceleration. The angular velocity sensor 182 measures the angular velocity in the triaxial direction. The geomagnetic sensor 183 measures the triaxial geomagnetism.

  FIG. 3 is a diagram illustrating a functional configuration of the control unit 11. The control unit 11 executes one or more programs stored in the storage unit 12, thereby arranging the arrangement unit 111, signal processing unit 112, output control unit 113, transmission unit 114, detection unit 115, selection unit 116, execution The functions of the unit 117, the measuring unit 118, and the changing unit 119 are realized. Further, these functions may be realized by cooperation between the control unit 11 and other components.

  The arrangement unit 111 arranges a virtual object indicating a menu item in a virtual space corresponding to the real space where the user exists. The signal processing unit 112 performs signal processing corresponding to the sound transfer characteristics corresponding to the position in the virtual space where the virtual object is placed and the position of the user in the virtual space on the audio signal indicating the menu item. The output control unit 113 outputs sound corresponding to the audio signal subjected to signal processing from the speakers 16-1 and 16-2. The transmission unit 114 transmits arrangement information indicating the arrangement of the virtual object to the display terminal 20. The detection unit 115 detects the movement of the user's body and the amount of movement. The selection unit 116 selects a menu item when a motion of moving the body in the direction in which the virtual object is arranged is detected and the motion amount of the motion exceeds a threshold value. The execution unit 117 executes a process corresponding to the menu item selected by the selection unit 116. The measurement unit 118 measures the distance between the audio reproduction terminal 10 and the display terminal 20. In the present embodiment, the audio playback terminal 10 is used as the first terminal, and the display terminal 20 is used as the second terminal. The changing unit 119 changes the threshold according to the measured distance.

(3) Configuration of Display Terminal 20 FIG. 4 is a diagram illustrating a hardware configuration of the display terminal 20. The display terminal 20 has a function of displaying a virtual icon. The display terminal 20 includes a control unit 21, a storage unit 22, a communication unit 23, an operation unit 24, a display unit 25, a sensor unit 26, and a power source 27. These components are connected via a bus.

  The control unit 21 executes a program stored in the storage unit 22 to perform calculation of data and control of other components. For example, a CPU is used as the control unit 21. The storage unit 22 stores programs executed by the control unit 21 and various data. The storage unit 22 includes a main storage device and an auxiliary storage device. For example, a RAM is used as the main storage device. For example, a flash memory is used as the auxiliary storage device.

  The communication unit 23 is a communication interface for performing data communication. The communication unit 23 communicates with the audio reproduction terminal 10 in accordance with a near field communication standard such as Bluetooth (registered trademark). The operation unit 24 inputs a signal corresponding to a user operation to the control unit 21. For the operation unit 24, for example, a touch sensor is used. The display unit 25 displays information. As the display unit 25, for example, a liquid crystal display is used. The power supply 27 supplies power to each component of the display terminal 20. For example, a battery is used as the power source 27.

  The sensor unit 26 is used to detect a user's operation. The sensor unit 26 includes, for example, an acceleration sensor 261, an angular velocity sensor 262, and a geomagnetic sensor 263. The acceleration sensor 261 measures acceleration in three axis directions. The angular velocity sensor 262 measures the angular velocity in the triaxial direction. The geomagnetic sensor 263 measures the geomagnetism in the triaxial direction.

  FIG. 5 is a diagram illustrating a functional configuration of the control unit 21. The control unit 21 implements the functions of the display control unit 211 and the transmission unit 212 by executing one or more programs stored in the storage unit 22. Further, these functions may be realized by cooperation between the control unit 21 and other components.

  The display control unit 211 displays virtual objects arranged in the virtual space. The transmission unit 212 transmits information for identifying the virtual object displayed on the display unit 25 to the audio reproduction terminal 10.

2. Operation FIG. 6 is a sequence chart showing the operation of the menu selection system 1. This operation is started, for example, when the user performs an operation for starting the menu item using the audio reproduction terminal 10. This operation is, for example, a voice operation using voice recognition technology. For example, this operation is started when the word “menu” is issued by the user.

  In step S101, the arrangement unit 111 arranges a virtual icon (an example of a virtual object) indicating a menu item in a virtual space corresponding to a real space where the user exists. “Arranging virtual icons” specifically refers to determining the position of a virtual icon in the virtual space. The placement of the virtual icons is performed based on, for example, a terminal coordinate system indicating a position on the xy plane with the position of the user's head as the origin. FIG. 7 is a diagram illustrating an example of the arrangement of virtual icons. FIG. 7 is a view of the placement of virtual icons as seen from above. In FIG. 7, the user's front direction is indicated by the y-axis direction, the user's rear direction is indicated by the -y-axis direction, the user's right direction is indicated by the x-axis direction, and the user's left direction is indicated by the -x-axis direction. It is shown in

  In the example illustrated in FIG. 7, virtual icons I1 to I8 are arranged at an interval of 45 degrees on a virtual circle C1 centered on the position of the user's head on the xy plane. Virtual icons I1 to I8 indicate first to eighth menu items, respectively. The virtual icon I1 is arranged in front of the user. The virtual icon I2 is arranged on the right front side of the user. The virtual icon I3 is arranged on the right side of the user. The virtual icon I4 is arranged on the right rear side of the user. The virtual icon I5 is arranged behind the user. The virtual icon I6 is arranged on the left rear side of the user. The virtual icon I7 is arranged on the left side of the user. The virtual icon I8 is arranged on the left front side of the user.

  In step S <b> 102, the transmission unit 114 transmits arrangement information indicating the arrangement of virtual icons to the display terminal 20. This arrangement information includes position coordinates indicating the position where the virtual icon is arranged in the virtual space. In the example shown in FIG. 7, the arrangement information includes the position coordinates of the virtual icons I1 to I8 in the virtual space. Further, the arrangement information includes position coordinates indicating the position of the user's head in the virtual space and position coordinates indicating the position of the display terminal 20. The position of the display terminal 20 is, for example, the position of the display terminal 20 when the user takes a basic attitude of looking at the display unit 25 of the display terminal 20 and is registered in the audio reproduction terminal 10 in advance. When receiving the arrangement information from the audio reproduction terminal 10, the display terminal 20 proceeds to step S103.

  In step S103, the display control unit 211 displays virtual icons on the display unit 25 in accordance with the arrangement indicated by the received arrangement information. At this time, all the virtual icons may be displayed, or only a part of the virtual icons may be displayed. In the latter case, the display control unit 211 sets a selection range in the virtual space and displays only the virtual icons in the selection range on the display unit 25. The display of the virtual icon is performed based on a screen coordinate system that indicates a position on the XY plane with the upper left of the screen of the display unit 25 as the origin.

  FIG. 8 is a diagram illustrating a display example of virtual icons. In this example, the selection range R1 is first set so as to include the display terminal 20 and the user's head on the xy plane shown in FIG. The selection range R1 has a size and shape corresponding to the display unit 25. As shown in FIG. 7, the selection range R1 includes virtual icons I1 and I8. In this case, as shown in FIG. 8, virtual icons I1 and I8 are displayed on the display unit 25 based on the positions of the virtual icons I1 and I8 in the selection range R1.

  Specifically, the virtual icon I1 is displayed at a position on the screen corresponding to the position in the selection range R1. The virtual icon I8 is displayed at a position on the screen corresponding to the position in the selection range R1. The positions of the virtual icons I1 and I8 on the screen are obtained by converting the position coordinates of the terminal coordinates of the virtual icons I1 and I8 into the position coordinates of the screen coordinate system.

  Further, the display unit 25 displays marks W1 and H1 indicating the position of the display terminal 20 and the position of the head in the selection range R1. Specifically, the mark W1 is displayed at a position on the screen corresponding to the position of the display terminal 20 in the selection range R1. The mark H1 is displayed at a position on the screen corresponding to the position of the head in the selection range R1. The positions of the marks W1 and H1 on the screen are obtained by converting the position coordinates in the terminal coordinate system of the display terminal 20 and the position coordinates in the head terminal coordinate system into position coordinates in the screen coordinate system.

  When the user moves the display terminal 20, the selection range R1 moves according to the movement amount, and the display on the display unit 25 is updated. For example, as shown in FIG. 9, when the user moves the display terminal 20 to the right side, the selection range R1 also moves to the right side. As shown in FIG. 9, the selection range R1 after movement includes virtual icons I1 and I2. In this case, as shown in FIG. 10, virtual icons I1 and I2 are displayed on the display unit 25 based on the positions of the virtual icons I1 and I2 in the selected range R1 after movement. The display unit 25 displays marks W1 and H1 indicating the position of the display terminal 20 and the position of the head in the selected range R1 after movement.

  In step S <b> 104, the measurement unit 118 measures the distance between the audio reproduction terminal 10 and the display terminal 20. For example, the measurement unit 118 transmits a signal to the display terminal 20. When receiving a signal from the audio reproduction terminal 10, the transmission unit 212 transmits a response to the audio reproduction terminal 10. The measurement unit 118 measures a round trip delay time from when a signal is transmitted until a response is returned, and measures a distance between the audio reproduction terminal 10 and the display terminal 20 according to the measured round trip delay time. . It shows that the distance between the audio | voice reproduction terminal 10 and the display terminal 20 is so large that a round trip delay time is large.

  In step S105, the changing unit 119 changes the threshold used for selecting the menu item according to the distance measured in step S104. When the distance between the audio reproduction terminal 10 and the display terminal 20 is small, it is considered that the user is in a place where the user cannot move much, such as a crowded train. Therefore, the changing unit 119 decreases the threshold when the measured distance is smaller than the predetermined distance. For example, when the threshold is the head tilt angle and the initial value is 55 degrees, the threshold is changed to 50 degrees.

  When the threshold value is changed, the output control unit 113 outputs a notification sound (an example of a third notification sound) indicating that the threshold value has been changed from the speaker 16-1 and the speaker 16-2. This notification sound is, for example, a drop sound falling on the water surface. This notification sound changes according to the changed threshold value. For example, when the threshold value decreases, the reverberation time of the notification sound decreases.

  In step S <b> 106, the signal processing unit 112 responds to the sound transmission characteristics from the position of the virtual icon to the user based on the position in the virtual space where the virtual icon is arranged and the position of the user in the virtual space. Signal processing is performed on the audio signal indicating the menu item. Humans recognize the direction and distance of sound arrival based on the difference in the volume of sound transmitted to the left and right ears, the difference in time, and the difference in frequency characteristics. Therefore, by adding a sound pressure level difference, a time difference, and a frequency characteristic difference to the audio signals supplied to the left and right speakers 16-1 and 16-2 according to the position and direction in which the virtual icon is arranged, Sound that can be heard from the position where the virtual icon is pseudo-positioned can be created. In this case, there is a virtual sound source at the position of the virtual icon. Specifically, in the signal processing, a process of convolving the head-related transfer function from the position where the virtual icon is arranged to the user's position into an audio signal indicating the menu item is performed.

  In step S107, the output control unit 113 performs a menu item reading process. In this reading process, sounds indicating menu items are output one by one from the speakers 16-1 and 16-2. The sound indicating the menu item is, for example, a sound that reads out the name of the menu item.

  Specifically, the output control unit 113 first determines the reading order of menu items. In the example shown in FIG. 7, when viewed clockwise from the virtual icon I1 arranged in front of the user, the virtual icons I1 to I8 are the virtual icons I1, I2, I3, I4, I5, I6, I7, I8. It is arranged in the order. In this case, the reading order of the first to eighth menu items is as follows: first menu item, second menu item, third menu item, fourth menu item, fifth menu item, sixth menu item, seventh menu item, The order is the eighth menu item.

  Subsequently, the output control unit 113 supplies audio signals indicating the menu items subjected to signal processing to the speakers 16-1 and 16-2 one by one in accordance with the reading order. At this time, the audio signal for the right ear is supplied to the speaker 16-1, and the audio signal for the left ear is supplied to the speaker 16-2. Thus, sounds indicating the first to eighth menu items are output one by one from the speakers 16-1 and 16-2.

  In the example shown in FIG. 7, the virtual icon I1 indicating the first menu item is arranged in front of the user. Therefore, the sound indicating the first menu item is heard from the front of the user in a pseudo manner. The virtual icon I2 indicating the second menu item is arranged on the right front side of the user. Therefore, the sound indicating the second menu item is heard from the front right of the user in a pseudo manner. The same applies to the sounds indicating the third to eighth menu items. Thereby, the user can recognize the arrangement of the virtual icons I1 to I8.

  As another example, when only a part of the virtual icon is displayed on the display unit 25, a sound indicating the menu item is output only for the menu item indicated by the virtual icon displayed on the display unit 25. Also good. In the example illustrated in FIG. 8, virtual icons I <b> 1 and I <b> 8 are displayed on the display unit 25. In this case, the transmission unit 212 transmits IDs for identifying the first and eighth menu items indicating the virtual icons I1 and I8 displayed on the display unit 25 to the audio reproduction terminal 10. When receiving the IDs of the first and eighth menu items from the display terminal 20, the output control unit 113 sequentially outputs the sound indicating the first menu item and the sound indicating the eighth menu item from the speakers 16-1 and 16-2. Output.

  The same processing may be performed when the display of the display unit 25 is updated by moving the display terminal 20 by the user. For example, when the display of the display unit 25 is updated as illustrated in FIG. 10, virtual icons I1 and I2 are displayed on the display unit 25. In this case, the transmission unit 212 transmits IDs for identifying the first and second menu items indicated by the virtual icons I1 and I2 displayed on the display unit 25 to the audio reproduction terminal 10. When receiving the IDs of the first and second menus from the display terminal 20, the output control unit 113 sequentially outputs sounds indicating the first menu and sounds indicating the second menu item from the speakers 16-1 and 16-2. .

  In step S <b> 108, each time a sound indicating each menu item is output from the speakers 16-1 and 16-2, the transmission unit 114 transmits an ID for identifying the menu item to the display terminal 20. For example, when a sound indicating the first menu item is output from the speakers 16-1 and 16-2, an ID for identifying the first menu item is transmitted to the display terminal 20. When a sound indicating the second menu item is output from the speakers 16-1 and 16-2, an ID for identifying the second menu item is transmitted to the display terminal 20. When the display terminal 20 receives the menu item ID from the audio reproduction terminal 10, the process proceeds to step S109.

  In step S109, the display control unit 211 changes the display of a virtual icon indicating a menu item identified by the received ID. This change in display includes, for example, changing the color, shape, or size of the virtual icon. In the example illustrated in FIG. 8, virtual icons I1 and I2 indicating the first and second menu items are displayed on the display unit 25. For example, when the ID of the first menu item is received, the virtual icon I1 indicating the first menu item is illuminated.

  The user recognizes the arrangement of the virtual icons based on the sound indicating the menu items output from the speakers 16-1 and 16-2 and the position of the virtual icon displayed on the display unit 25. In the example shown in FIG. 7, since the sound of the first menu item can be heard from the front of the user, it can be seen that the virtual icon I1 indicating the first menu item is arranged in front of the user. In the example shown in FIG. 8, it can be seen that the virtual icon I1 is arranged in front of the user based on the relative positional relationship between the virtual icon I1 and the mark H1. For example, when selecting the first menu item, the user performs an operation of tilting the head forward or an operation of moving the arm on which the display terminal 20 is worn forward.

  In step S110, the detection unit 115 detects the movement of the user's body and the amount of movement. For example, when the user performs an operation of tilting his / her head, the operation and the operation amount are detected based on the angular velocity measured by the angular velocity sensor 182. On the other hand, when the user performs an operation of moving the arm on which the display terminal 20 is worn, the operation and the operation amount are detected based on the acceleration measured by the acceleration sensor 261. Specifically, the transmission unit 212 transmits the acceleration measured by the acceleration sensor 261 to the voice reproduction terminal 10. When the acceleration is received from the display terminal 20, the detection unit 115 detects the movement of the arm on which the display terminal 20 is worn and the amount of movement based on the position of the display terminal 20 registered in advance and the acceleration. .

  In step S <b> 111, the selection unit 116 selects a menu item indicated by the virtual icon when the movement of moving the body in the direction in which the virtual icon is arranged is detected and the movement amount exceeds the threshold value. Note that the direction in which the virtual icon is arranged does not necessarily need to be a direction passing through the position in which the virtual icon is arranged. For example, the direction passing through a predetermined range including the position where the virtual icon is arranged may also be included in the direction where the virtual icon is arranged. For example, when a motion of tilting the head forward or a motion of moving the arm on which the display terminal 20 is worn forward is detected and the motion amount exceeds a threshold value, a virtual icon arranged in front of the user The first menu item indicated by I1 is selected. In step S111, the threshold value changed in step S105 is used.

  FIG. 11 is a diagram illustrating an example of an operation for selecting a menu item. For example, when the user performs an operation of tilting his / her head forward, when the tilt angle reaches a predetermined angle θ1, it is determined that the first menu item selection operation has started. At this time, a notification sound indicating that the selection operation has been started is output from the speakers 16-1 and 16-2. A notification sound (an example of a first notification sound) indicating that the selection operation is being performed until the tilt angle of the user's head reaches the threshold value θ2 from the predetermined angle θ1 is output from the speakers 16-1 and 16-2. Is output. The sound level of the notification sound gradually decreases as the tilt angle of the user's head increases. When the tilt angle of the user's head exceeds the threshold θ2, the first menu item indicated by the virtual icon I1 disposed in front of the user is selected. At this time, a notification sound (an example of a second notification sound) indicating that the selection of the first menu item is completed is output from the speakers 16-1 and 16-2.

  As another example, the selection unit 116 may select only the menu item indicated by the virtual icon displayed on the display unit 25. In the example illustrated in FIG. 8, virtual icons I <b> 1 and I <b> 8 are displayed on the display unit 25. In this case, as described above, the transmission unit 212 transmits the ID for identifying the first and eighth menu items indicated by the virtual icons I1 and I8 displayed on the display unit 25 to the audio reproduction terminal 10. When receiving the IDs of the first and eighth menu items from the display terminal 20, the selection unit 116 can select only the first and eighth menu items and cannot select the other second to seventh menu items. To be limited. In this case, when the movement of moving the body in the direction in which the virtual icon I1 or I8 is arranged and the movement amount exceeds the threshold, the first or eighth menu item is selected. On the other hand, even if an operation of moving the body in the direction in which the virtual icons I2 to I7 are arranged, the second to seventh menu items are not selected.

  In step S112, the execution unit 117 executes processing corresponding to the menu item selected by the user. For example, when the first menu item is related to an application, when the first menu item is selected, the application corresponding to the first menu item is activated.

  In the present embodiment, when the user cannot move the body much, the threshold used for selecting the menu item becomes small, so the user selects the menu item without moving the body greatly. be able to. Thus, according to this embodiment, it is possible to easily select a menu item according to the user's situation.

  Further, according to the present embodiment, the notification sound indicating that the selection operation is being performed is output from the speakers 16-1 and 16-2 while the user performs the operation of selecting the menu item. The user can recognize that the selection operation is accepted. When the selection of the menu item is completed, a notification sound indicating that the selection of the menu item is completed is output from the speakers 16-1 and 16-2, so that the user recognizes that the selection of the menu item is completed. be able to.

  Furthermore, according to this embodiment, when the threshold value is changed, a notification sound indicating that the threshold value has been changed is output from the speakers 16-1 and 16-2, so that the user has changed the threshold value. Can be recognized.

3. The present invention is not limited to the above-described embodiment. For example, the embodiment may be modified as follows. Further, the following modifications may be combined.

(1) The threshold value may be changed according to the user's situation and behavior. For example, the change unit 119 may decrease the threshold when the user is in a crowded place such as when the user is on a crowded train. In this case, the changing unit 119 calculates the number of other communication terminals existing in the communication range by communicating with other communication terminals existing in the communication range by the communication unit 13. When the number of other communication terminals is larger than the predetermined number, the changing unit 119 determines that the user is in a crowded place and decreases the threshold value.

  The changing unit 119 may increase the threshold when the user is walking. In this case, the changing unit 119 determines whether the user is walking based on the acceleration measured by the acceleration sensor 181. When the user is walking, the changing unit 119 increases the threshold value. When the user is walking, the user's body is constantly moving. Therefore, if the threshold is small, an unintended menu item may be selected by mistake. According to this modification, such erroneous selection can be suppressed.

(2) The operation of selecting the menu item may be changed according to the user's posture. For example, when the user is sleeping, the menu item may be selected by rotating the head. In this case, the selection unit 116 determines whether the user is in a sleeping posture based on the angular velocity measured by the angular velocity sensor 182. For example, when the user is lying down for a certain time, it is determined that the user is in a sleeping posture. In this case, when the operation of rotating the head is detected by the detection unit 115, the selection unit 116 selects a menu item corresponding to the rotation angle.

(3) The method for measuring the distance between the audio reproduction terminal 10 and the display terminal 20 is not limited to the method described in the embodiment. As described above, the position of the display terminal 20 when the user takes a basic attitude of looking at the display unit 25 of the display terminal 20 is registered in the audio reproduction terminal 10 in advance. The distance between the audio reproduction terminal 10 and the display terminal 20 may be measured in accordance with the movement amount of the display terminal 20 from the registered position. In this case, the transmission unit 212 transmits the acceleration measured by the acceleration sensor 261 to the voice reproduction terminal 10. When the acceleration is received from the display terminal 20, the measurement unit 118 measures the current position of the display terminal 20 based on the position of the display terminal 20 registered in advance and the acceleration. Then, the measurement unit 118 measures the distance between the audio reproduction terminal 10 and the display terminal 20 based on the current position of the display terminal 20.

(4) The audio reproduction terminal 10 and the display terminal 20 are not limited to the example described in the embodiment. For example, the audio reproduction terminal 10 may be an ear hook type or overhead type wearable computer. The display terminal 20 may be a smartphone. Also, some of the functions of the audio playback terminal 10 may be realized by the display terminal 20 or an external server device. For example, the functions of the placement unit 111, the signal processing unit 112, the detection unit 115, the selection unit 116, the execution unit 117, the measurement unit 118, or the change unit 119 illustrated in FIG. 3 may be realized by the display terminal 20.

(5) Menu items are not limited to the examples described in the embodiment. The menu item may relate to music or images downloaded to the audio playback terminal 10.

(6) In the embodiment, the example in which the function of the control unit 11 and the function of the control unit 21 are realized by software has been described. At least a part of the function of the control unit 11 and the function of the control unit 21 may be realized by hardware. For example, the signal processing unit 112 may be realized by hardware.

(7) The program executed by the control unit 11 of the audio reproduction terminal 10 and the control unit 21 of the display terminal 20 may be downloaded via a communication line such as the Internet. The program is provided in a state of being recorded on a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or a semiconductor memory. May be.

10: Audio playback terminal, 16-1, 16-2: Speaker, 20: Display terminal, 25: Display unit, 111: Arrangement unit, 112: Signal processing unit, 113: Output control unit, 114: Transmission unit, 115: Detection unit, 116: selection unit, 117: execution unit, 118: measurement unit, 119: change unit, 211: display control unit, 212: transmission unit

Claims (5)

A placement unit for placing a virtual object indicating a menu item in a virtual space corresponding to a real space where the user exists;
A signal processing unit that applies signal processing corresponding to a sound transfer characteristic corresponding to a position in the virtual space where the virtual object is arranged and a position of the user in the virtual space to an audio signal indicating the menu item; ,
A speaker that outputs a sound corresponding to the audio signal subjected to the signal processing;
A display unit for displaying the virtual object arranged in the virtual space;
A detection unit for detecting the movement of the user's body and the movement amount of the movement;
A selection unit that selects the menu item when a motion of moving the body in the direction in which the virtual object is arranged is detected and the motion amount of the detected motion exceeds a threshold;
A measuring unit for measuring a distance between the first terminal having the speaker and the second terminal having the display unit;
A menu selection system comprising: a changing unit that changes the threshold according to the measured distance. The speaker outputs a first notification sound until the operation amount reaches the threshold value after the operation is started, and outputs a second notification sound when the operation amount exceeds the threshold value. 1. The menu selection system according to 1. The menu selection system according to claim 1, wherein the speaker outputs a third notification sound when the threshold is changed. The menu selection system according to any one of claims 1 to 3, wherein the changing unit changes the threshold value according to a situation or action of the user. Placing a virtual object indicating a menu item in a virtual space corresponding to a real space where the user exists;
Applying signal processing corresponding to a sound transfer characteristic corresponding to a position in the virtual space where the virtual object is arranged and a position of the user in the virtual space to the audio signal indicating the menu item;
Outputting a sound according to the audio signal subjected to the signal processing from a speaker;
Displaying the virtual object arranged in the virtual space on a display unit;
Detecting the movement of the user's body and the amount of movement of the movement;
When a motion of moving the body in the direction in which the virtual object is arranged is detected, and when the motion amount of the detected motion exceeds a threshold, selecting the menu item;
Measuring a distance between a first terminal having the speaker and a second terminal having the display;
A menu selection method comprising: changing the threshold value according to the measured distance.

Images