JP6582024B2 - Information support system for shows - Google Patents

Description

  The present invention relates to a system for providing information support in a show or the like.

  In a show held at a theme park or stage, the audience may not understand the lines of the performers. For example, when the audience is deaf and you cannot hear the dialogue itself, or when you hear the dialogue but cannot understand it because the dialogue is not the native language of the audience.

  In order to cope with such a problem, a system for displaying subtitles on a display device provided in a seat of a spectator has been proposed (Patent Document 1). In this system, the caption information transmitting device transmits the dialogue as character information to the display device of each seat and displays it.

  According to this system, dialogue can be displayed with character information for a deaf audience, and dialogue can be displayed with translated character information for a foreign audience.

JP2007-61412

  However, in the above prior art, it is necessary to provide a caption information transmitting device and to provide each display device with a receiving device for receiving caption character information. In addition, it is necessary for the operator to operate the timing for transmitting dialogue so that the display timing of dialogue characters matches the progress of the show.

  Further, such a problem has occurred not only when displaying the text information of dialogue but also when displaying auxiliary text information accompanying the progress of the show.

  Further, even when explanations of exhibits and tourism resources (historical buildings, historic sites, etc.) are played through speakers or the like, it is desired to convey the explanations in text information.

  An object of the present invention is to provide an information support system that solves any of the above-described problems and does not require a device for transmitting character information.

  Several independently applicable features of the invention are listed.

(1) (2) (3) An information support system according to the present invention includes a speaker that outputs at least content sound toward a user, a signal output device that outputs a signal to the speaker, and a portable device that the user has An information support system comprising a terminal device, wherein the signal output device is configured to start outputting content audio signals for the content audio, selection sound signals for selecting the content audio, and output of the content audio. The portable terminal device is configured to output a signal including at least an associated start sound signal to the speaker, and the portable terminal device is configured to output a sound output from the speaker over time and a reproduction timing by reproducing the content audio signal. And a recording unit that records multiple units of support information including character text created so that the A selection sound detecting means for detecting whether or not the selection sound output from the speaker is included in the sound; and when the selection sound is detected by the selection sound detection means, the plurality of units of characters based on the selection sound Support information selecting means for selecting one support information from support information including text, start sound detecting means for detecting whether or not the start sound output from the speaker is included in the ambient sound acquired by the microphone, When the start sound is detected by the start sound detection means, the information support system includes a display control means for starting display on the display of the support information including the character text selected by the support information selection means.

  Therefore, it is possible to select the support information with the selected sound, reproduce it by matching the timing with the start sound, and display the support information appropriately.

(4) In the information support system according to the present invention, the support information of a plurality of units recorded in the recording unit corresponds to content audio that is reproduced in time order on a speaker installed in the same place. In addition, the selection sound includes a sound having a different frequency for each support information.

  Therefore, it is possible to provide appropriate support information for a plurality of contents reproduced from speakers installed at the same place.

(5) In the information support system according to the present invention, the support information of a plurality of units recorded in the recording unit corresponds to content sound reproduced by speakers installed in different places, and the selected sound is Each support information includes a sound having a different frequency.

  Therefore, it is possible to provide appropriate support information for a plurality of contents reproduced from speakers installed in different places.

(6) In the information support system according to the present invention, the selection sound is a sound of a predetermined frequency, and the selection sound detection means includes whether the sound of the predetermined frequency is included in the ambient sound acquired by the microphone above a predetermined level. It is characterized by determining whether or not a selection sound is included depending on whether or not it is included.

  Therefore, it is possible to provide a system that reduces the user's discomfort due to the selected sound.

(7) In the information support system according to the present invention, the selection sound is obtained by synthesizing sounds of a plurality of frequencies, and the selection sound detecting means adds the sounds of the plurality of predetermined frequencies to the ambient sound acquired by the microphone. It is characterized in that it is determined whether or not a selection sound is included depending on whether or not all of the above are included at a predetermined level or more.

Therefore, it is easy to select many types of support information with high accuracy.
(8) The information support system according to the present invention is characterized in that the support information selection means reads the selected support information from the recording unit and expands it in a memory that can be accessed at high speed for display processing on the display. Yes.

  Therefore, the rise of the support information reproduction process is quick, and the timing can be accurately taken.

(9) In the information support system according to the present invention, the start sound is a sound of a predetermined frequency, and the start sound detection means includes a sound having the predetermined frequency included in the ambient sound acquired by the microphone. It is characterized by determining whether or not the start sound is included depending on whether or not.

  Therefore, it is possible to provide a system that reduces the discomfort to the user due to the start sound.

(10) In the information support system according to the present invention, the start sound is obtained by synthesizing sounds having a plurality of frequencies, and the start sound detecting means adds the sounds having the plurality of predetermined frequencies to the ambient sound acquired by the microphone. It is characterized in that it is determined whether or not the start sound is included depending on whether or not all of the above are included at a predetermined level or more.

Therefore, it is easy to prepare corresponding start sounds for many types of support information.
(11) (12) (13) The information support system according to the present invention includes at least one speaker that sequentially outputs at least the first content sound and the second content sound toward the user, and a signal that outputs a signal to the speaker. An information support system including an output device and a mobile terminal device possessed by a user, wherein the signal output device includes a first content audio signal for the first content audio and the second content audio, and Responding to the start of output of the second content audio signal, the first selection audio signal and the second selection audio signal for selecting the first content audio and the second content audio, and the first content audio and the second content audio. A signal including at least the first start sound signal and the second start signal attached is output to the speaker. The portable terminal device includes character text created so that the reproduction timing coincides with the sound output from the speaker over time due to reproduction of the first content audio signal and the second content audio signal. A recording unit for recording the first support information and the second support information, and a selection sound for detecting whether the ambient sound acquired by the microphone includes the first selection sound or the second selection sound output from the speaker. When the first selection sound is detected by the detection means and the selection sound detection means, the first support information recorded in the recording unit is selected to be reproduced, and the second selection sound is detected by the selection sound reproduction means. Then, the support information selection means for selecting the second support information recorded in the recording unit as to be reproduced and the microphone are used. When the first sound is detected by the start sound detection means and the start sound detection means for detecting whether or not the acquired ambient sound includes the first start sound or the second start sound output from the speaker. The display of the first support information including the character text selected by the support information selecting means is started on the display, and when the second start sound is detected by the start sound detecting means, it is selected by the support information selecting means. Display control means for starting display of the second support information including the character text on the display.

  Therefore, it is possible to select the support information with the selected sound, reproduce it by matching the timing with the start sound, and display the support information appropriately.

(14) In the information support system according to the present invention, the first selected sound includes at least a first frequency sound, and the first start sound includes at least a first frequency sound and a third frequency sound. The second selection sound includes at least a fourth frequency sound, and the second start sound includes at least a fourth frequency sound and a third frequency sound.

  Therefore, it is possible to select the support response more accurately and perform reproduction at an accurate timing.

(15) In the information support system according to the present invention, the first start sound further includes a mixed sound including a sound of the second frequency, and the second start sound further includes a mixed sound including a sound of the fifth frequency. It is characterized by including.

  Therefore, it is possible to select the support response more accurately and perform reproduction at an accurate timing.

(16) In the information support system according to the present invention, the display is a head mounted display.

  Therefore, the user can easily refer to the support information.

(17) In the information support system according to the present invention, the speaker is arranged in an area where the show is performed, and at least outputs the speech of the performer of the show as content audio, and the mobile terminal device is at least a character corresponding to the speech It is characterized by playing text.

  Therefore, even when the dialogue cannot be heard or understood, the dialogue can be known from the text.

(18) In the information support system according to the present invention, the speaker is disposed in an area where a moving image is displayed, and outputs at least explanatory sound relating to the moving image as content audio, and the mobile terminal device includes at least characters corresponding to the explanatory sound It is characterized by playing text.

  Therefore, even when the explanation about the moving image cannot be heard or understood, the explanation can be known in text.

(19) In the information support system according to the present invention, the speaker is disposed in the vicinity of the explanation object, and at least outputs explanation sound related to the explanation object as content sound, and the mobile terminal device corresponds to at least the explanation sound. It is characterized by reproducing textual text.

  Therefore, even when the explanation about the exhibit cannot be heard or understood, the explanation can be known in the text.

(20) (21) (22) An information support system according to the present invention includes a speaker that outputs at least content sound toward a user, a signal output device that outputs a signal to the speaker, and a portable device that the user has An information support system comprising a terminal device, wherein the signal output device includes a signal including at least a content audio signal for the content audio and a start sound signal associated with the start of output of the content audio. The mobile terminal device is configured to output to the speaker, and the portable terminal device includes a text that is created so that the playback timing coincides with the sound output from the speaker over time due to the playback of the content audio signal. The recording unit for recording information and the ambient sound acquired by the microphone include the start sound output from the speaker. Start sound detecting means for detecting whether or not, and when the start sound is detected by the start sound detecting means, display control means for starting display of support information including character text recorded in the recording unit on the display. Therefore, it is possible to reproduce by matching the timing with the start sound and appropriately display the support information.

  In the present invention, “show” refers to something that is presented to the audience, such as a stage, a movie, or a musical.

  The “portable terminal device” includes a smartphone, a portable information terminal, a mobile phone, a tablet, a wearable computer, and the like.

  In the embodiment, “selected sound detection means” corresponds to steps S55 and S56.

  The “support information selection means” refers to at least processing for selecting a support information file for reproduction. In the embodiment, step S57 corresponds to this. In the embodiment, the selected support information file is expanded in the memory. However, those that do not perform such processing also correspond to the support information selection means.

  In the embodiment, “starting sound detection means” corresponds to step S59.

  In the embodiment, “display control means” corresponds to step S61.

  The “program” is a concept that includes not only a program that can be directly executed by the CPU, but also a source format program, a compressed program, an encrypted program, and the like.

It is a functional block diagram of the information support system by one Embodiment of this invention. It is a block diagram of an information support system. It is a figure which shows performance of the room 65 of the pre-show in the area which performs a show, and the room 53 of the main show. It is an example of arrangement of areas when applied to a theme park. It is a figure which shows the structure of control apparatus CTL for sound reproduction. 1 is a diagram illustrating an appearance of a wearable computer 10. FIG. This is a hardware configuration of the wearable computer 10. 4 is a flowchart of a portable terminal program 76 recorded in a nonvolatile memory 66. 4 is a flowchart of a portable terminal program 76 recorded in a nonvolatile memory 66. 4 is a flowchart of a portable terminal program 76 recorded in a nonvolatile memory 66. 4 is a flowchart of a portable terminal program 76 recorded in a nonvolatile memory 66. It is an example of a show selection screen. It is a table which shows a response | compatibility with a show and support information. It is an example of a display of assistance information. It is an example of a display of assistance information. It is a figure which shows the timing of a selection sound and a start sound. It is a figure which shows the timing of the selection sound and start sound at the time of applying to the movie etc. with a curtain. It is a figure which shows the selection sound and start sound at the time of applying to guidance, such as a historical site in a sightseeing spot. It is a functional block diagram of the information support system by 2nd Embodiment. It is a system configuration figure of an information support system. It is an example of arrangement | positioning of the access point 2 and the speaker 6 in the area which performs a show. This is an example of the reproduction circuit 46 and the speaker 6. 2 is a hardware configuration of the mobile terminal device 10. It is a flowchart of a portable terminal program. It is a flowchart of a portable terminal program. It is an example of an SSID table. It is a display example of dialogue. It is a display example of dialogue. It is a display example of dialogue.

1. First embodiment
1.1 System Configuration FIG. 1 is a functional block diagram of an information support system for a show according to an embodiment of the present invention. A signal output device 4 and a speaker 6 are provided in the show area.

  The signal output device 4 gives sound signals such as lines and music to the speaker 6 in accordance with the progress of the show. The speaker 6 receives this sound signal and outputs speech and music as sound.

  In this embodiment, the line of the performer of the show is not given by the performer, but is given to the audience by the sound from the speaker 6. Therefore, in the signal output device 4, sound signals such as lines and music in accordance with the progress of the show are recorded in advance.

  The signal output device 4 gives the above sound signal to the speaker 6 at the start of the show. In this embodiment, a selection signal and a start signal are inserted at the beginning of the sound signal.

  A user who is a spectator possesses the mobile terminal device 10. In the recording unit 22 of the portable terminal device 10, support information is recorded with text information such as lines of shows performed in each area.

  Before the start of the show (or before the start of each film), the speaker 6 outputs a selection sound based on the selection signal. The selection sound detection unit 11 of the mobile terminal device 10 determines whether or not the selection sound is included in the ambient sound acquired by the microphone 14. The support information selection unit 13 of the mobile terminal device 10 selects any one of the support information A11, A12,... Recorded in the recording unit 22 as a display target based on the acquired selection sound. The selected support information is assumed to be As.

  Along with the start of the show (or the start of each curtain), the speaker 6 outputs a start sound based on the start signal. The start sound detecting means 16 of the mobile terminal device 10 determines whether or not the start sound is included in the ambient sound acquired by the microphone 14. When the start sound is detected, the display control unit 20 of the mobile terminal device 10 displays the support information As selected by the support information selection unit 13 on the display 18.

  Therefore, support information such as lines is displayed as character text on the mobile terminal device 10 of the user as the show progresses.

  FIG. 2 shows a system configuration when an information support system for a show is realized in a theme park. Areas A1 to An for performing shows are provided in the theme park. A user who desires to use this information support system wears a glasses-type wearable computer 10, which is a mobile terminal device 10.

  As shown in FIG. 3, a pre-show room 65 and a main show room 53 are provided in each of the show areas A1 to An. The pre-show room 65 is provided with an entrance door 63 and a door 61 to the main show room 53. The pre-show room 65 is provided with a screen 59, a projector 57, and a speaker 7 for performing the pre-show. The user can know prior knowledge and background at the pre-show before viewing the main show. The room 53 of the main show is provided with a stage 51 and a spectator seat (audience area) 49. A speaker 6 is also provided. As shown in FIG. 4, for example, the areas A1 to A6 where the show is performed are scattered in the theme park.

  FIG. 5 shows the configuration of the signal output device 4. In the recording units 43 and 42, pre-show data 45 such as music and lines used in a show in the area, and main show data 44 are recorded. The reproduction circuit 47 reads the preshow data 45 and reproduces it from the speaker 7. Further, the selection sound and the start sound are reproduced from the speaker 7 under the control of the control device CTL. The reproduction circuit 46 reads the main show data 44 and reproduces it from the speaker 6. Further, the selection sound and the start sound are reproduced from the speaker 7 under the control of the control device CTL.

  In the pre-show, there are no performers, the video from the projector 57 is displayed on the screen 59, and sound is output from the speaker 7.

  In the main show, the performer does not speak the dialogue, and the dialogue included in the main show data is output from the speaker 6 and the performer performs correspondingly. That is, the progress of the show is performed based on the main show data 44.

  FIG. 6 shows an appearance of a glasses-type wearable computer 10 that is a portable terminal device. A battery, a CPU, and the like are built in the side surface of the glasses-type main body 90. A transmissive binocular display 92 is provided on the front surface. The display 92 projects an image from the side surface so that the image is shown to the user's eyes by a prism action. In addition, since it is a transmission type, the user can try to superimpose the image on the display on the real scenery.

  In this embodiment, a touch pad 94 is provided. By operating the touch pad 94, the user can move a cursor on the screen of the display, or click to perform a selection operation. Furthermore, a microphone 70 and a camera 72 are provided.

  FIG. 7 shows a hardware configuration of the wearable computer 10. A memory 62, a head mounted display 92, a touch pad 94, a nonvolatile memory 66, a wireless circuit 68, a microphone 70, and a camera 72 are connected to the CPU 60.

  The memory 62 is constituted by a high-speed but volatile semiconductor memory or the like, and functions as a work area for the CPU 60. The head mounted display 92 displays support information and the like. The wireless circuit 68 is a circuit for connecting to the Internet.

  In the nonvolatile memory 66, an operating system 74, a portable terminal program 76, support information A11, A12, A21. The portable terminal program 76 exhibits its functions in cooperation with the operating system 74. These programs are downloaded from the server device via the wireless circuit 68, for example.

  Further, support information A11, A12, A21... Is also recorded in the nonvolatile memory 66. The support information 78 is data to be displayed on the touch screen display 64 in correspondence with the show data that is flowed at the show in each of the areas A1 to A6.

  Support information A11 is for a pre-show in area A1, and support information A12 is for a main show in area A1. Similarly, support information A21 is for a pre-show in area A2, and support information A22 is for a main show in area A2.

In this embodiment, support information A11, A12, A21... For all areas A1 to A6 is downloaded and recorded in advance from a server device or the like.

1.2 Support Information Providing Processing FIGS. 8 to 11 show flowcharts of support information providing processing by the portable terminal program 76. FIG. When the portable terminal program 76 of the wearable computer 10 worn by the user is activated, the CPU 60 of the wearable computer 10 (hereinafter sometimes abbreviated as the wearable computer 10) displays a show selection screen as a head mounted display (HMD). ) 92 (step S51).

  An example of the show selection screen is shown in FIG. Since this screen is displayed on the transmissive head-mounted display 92, the user sees the screen of FIG. Show titles SH1, SH2, SH3... Are displayed. Furthermore, a cursor 98 is displayed. In this embodiment, one show is performed in one area (therefore, the area may be selected). However, a plurality of shows may be performed in one area.

  The user wearing the wearable computer 10 operates the touch pad 94 with a finger or the like to move the cursor 94 over a desired show title and clicks to select it. When a show is selected (step S52), wearable computer 10 determines whether pre-show support information and main show support information corresponding to the selected show are recorded (step S53). Here, the description will proceed assuming that “Sesame Street (trademark) 4D movie magic” of the show ID “0001” is selected. When the show is selected, the wearable computer 10 displays “Selecting file” on the HMD 92.

  As shown in FIG. 13, the nonvolatile memory 66 stores a table indicating the file name of the support information used in the pre-show and the file name of the support information used in the main show in association with each show. Therefore, the wearable computer 10 can determine whether or not support information corresponding to the selected show is recorded with reference to this table. Here, it is determined whether the pre-show support information A11 and the main show support information A12 are recorded in the nonvolatile memory 66.

  If the support information A11, A12 is not recorded, the wearable computer 10 accesses the server device on the Internet via the wireless circuit 68 and downloads the support information (step S58).

  If there is the necessary support information A11, A12, the wearable computer 10 proceeds to the selection process of the support information file A11 used in the pre-show. In this embodiment, in the room of the pre-show room 65 before the pre-show, the sound of the first frequency f1 (for example, 18.2 KHz) and the second frequency f2 are output from the speaker 7 under the control of the control device CTL of FIG. A mixed sound with a sound (for example, 19.2 kHz) (hereinafter referred to as f1f2 mixed sound) is output as a selected sound.

  The wearable computer 10 detects the f1f2 mixed sound and selects the support information file A11 for pre-show. As will be described below, when the user wearing the wearable computer 10 enters the pre-show room 65, the f1f2 mixed sound is detected.

  The wearable computer 10 first acquires ambient sound by the microphone 70, A / D converts it into a digital signal, and performs FFT processing (Fourier transform processing) (step S54). The FFT process can be executed by a program for FFT, for example.

  Next, the wearable computer 10 determines whether or not the component of the first frequency f1 exceeds the predetermined threshold in the ambient sound subjected to the FFT (step S55). Further, it is determined whether or not the component of the second frequency f2 exceeds a predetermined threshold value (step S56). If the threshold values are exceeded at both frequencies f1 and f2, the selected sound is being output from the speaker 6. Therefore, the pre-show support information file A11 corresponding to the selected sound based on the combination of f1 and f2 is selected.

  The wearable computer 10 reads the selected pre-show support information file A11 from the non-volatile memory 66, and expands it to the memory 62 (decompresses if compressed) (step S57). Thereby, the start of the process for the support information file A11 can be speeded up. Then, “Pre-show start waiting” is displayed on the head mounted display 92.

  When it is confirmed that a predetermined number of users have entered the pre-show room 65 of FIG. 3, the operation staff presses the pre-show start switch 41 of the control device CTL of FIG. As a result, the reproduction circuit 47 that has so far output the f1f2 mixed sound starts reproducing the pre-show data 45. That is, the control circuit CTL controls the reproduction circuit 47 to reproduce the pre-show data 45 instead of the f1f2 mixed sound (at the same time, reproduction of video is also started). At this time, the control circuit CTL reproduces the sound of the third frequency f3 (for example, 18.6 KHz) as the start sound for only a predetermined second (for example, 3 seconds) superimposed on the reproduction of the pre-show data 45.

  Since the beginning portion of the pre-show data 45 is silent, substantially only the f3 sound is reproduced.

  In this embodiment, the control device CTL reproduces the f1f2 mixed sound superimposed for a predetermined time (for example, 3 seconds) after the reproduction of the pre-show data 45 is started. This is to ensure that the wearable computer 10 can receive the pre-show start sound (f3 sound) while the f1 f2 mixed sound is being produced.

  The wearable computer 10 acquires ambient sounds from the microphone 70 and performs FFT processing (step S59). Further, in the surrounding sound, it is determined whether or not the frequency f3 component exceeds a threshold value (step S60). That is, it is determined whether the start sound (f3 sound) has been acquired.

  If the start sound (f3 sound) can be acquired, the pre-show support information A11 developed in the memory 62 is reproduced by the HMD 92. In this embodiment, the support information A11 is configured as a moving image file. Character text corresponding to the pre-show voice is displayed on the HMD 92 so as to coincide with the progress of the pre-show in time. Therefore, the user can see the text on the text while viewing the pre-show video.

  If the user stops the support display or if the support information file is defective and cannot be displayed, the support display process is stopped and the process proceeds to step S64 (step S62).

  When the support information file is reproduced and displayed to the end (the support information file is created so that this timing coincides with the end of the pre-show), the wearable computer 10 deletes the pre-show support information A11 from the memory 62 ( Step S64).

  On the other hand, the control device CTL controls to automatically open the door 61 in FIG. 3 when the reproduction of the pre-show data 43 ends (that is, when the pre-show ends), and further selects from the speaker 6 in the room 53 of the main show. A synthesized sound of a sound having a frequency f4 (for example, 18.0 KHZ) and a sound having a frequency f5 (for example, 19.0 KHz) is reproduced.

  The user who was in the pre-show room 65 moves from the open door 61 to the main show room 53. The wearable computer 10 selects support information A12 for the main show as will be described below.

  The wearable computer 10 acquires ambient sounds and performs FFT processing (step S65). Then, it is determined whether or not both the frequency f4 component and the f5 component exceed the threshold in the FFT-processed ambient sound (steps S66 and S67). That is, it is determined whether the f4f5 mixed sound (selected sound of the main show support information A12) is detected.

  When the f4f5 mixed sound is detected, the wearable computer 10 selects the main show support information A12 as a reproduction target. And this is read from the non-volatile memory 66, and expand | deployed to the memory 62 (step S68).

  On the other hand, when the operation staff confirms that all the users have moved to the main show room, the operation staff presses the main show start switch 40 of FIG. In response to this, the control device CTL controls the reproduction circuit 46 so as to output the main show data 44 from the speaker 6 instead of the f4f5 mixed sound. At this time, the control circuit CTL reproduces the sound of the third frequency f3 (for example, 18.6 KHz) as a start sound for a predetermined second (for example, 3 seconds) while being superimposed on the reproduction of the main show data 44.

  Since the beginning portion of the main show data 44 is silent, substantially only the f3 sound is reproduced.

  In this embodiment, the control device CTL reproduces the mixed f4f5 sound for a predetermined second (for example, 3 seconds) after the reproduction of the main show data 44 is started. This is to ensure that the wearable computer 10 can receive the start sound (f3 sound) of the main show while the f4f5 mixed sound is being produced.

  The wearable computer 10 acquires ambient sound from the microphone 70 and performs FFT processing (step S69). Further, in the surrounding sound, it is determined whether or not the frequency f3 component exceeds a threshold value (step S70). That is, it is determined whether the start sound (f3 sound) has been acquired.

  If the start sound (f3 sound) can be acquired, the main show support information A12 developed in the memory 62 is reproduced on the HMD 92 (step S71). In this embodiment, the support information A12 is configured as a moving image file. Character text corresponding to the voice of the main show is displayed on the HMD 92 so as to coincide with the progress of the main show. Therefore, the user can see the text on the text while superimposing the video of the main show.

  FIG. 14 shows a display example of the HMD 92. The user can see the support information displayed on the stage where the user is actually viewing. A balloon 100 shows a portrait 100 of the character that the performer is jealous of and the text of the performer's dialogue. Therefore, even a deaf user can know the dialogue. A character name is displayed under the portrait 100. In the example of FIG. 14, portraits of the same character are displayed on the same side (left side or right side) so that they can be easily identified.

  The wearable computer 10 displays support information on the HMD 92 one after another according to the support information file A12. Further, since the support information is generated so as to coincide with the progress of the show, the dialogue of each performer accompanying the progress of the show can be obtained.

  Further, as shown in FIG. 15, points that should be noted on the stage other than the dialogue and explanation 108 can be included in the support information. In this embodiment, the attention point and the explanation 108 are displayed in a format that is not a balloon.

  If the user stops the support display or if the support information file is defective and cannot be displayed, the support display process is stopped and the process proceeds to step S74 (step S72).

  When the support information file is reproduced and displayed to the end (the support information file A12 is created so that this timing coincides with the end of the main show), the wearable computer 10 stores the support information A12 for the main show from the memory 62. Delete (step S74).

  On the other hand, when the reproduction of the main show data 42 ends (that is, when the main show ends), the control device CTL controls to output the selected sound f1f2 synthesized sound from the speaker 7 in the pre-show room 65.

The wearable computer 10 returns to the show selection screen in step S51 again. Therefore, when the user goes to an area where another show is being performed, the same processing as described above is repeated.

1.3 Other
(1) In the above embodiment, the case where the user holds the wearable computer 10 and performs the user's operation has been described. However, the staff may lend the wearable computer 10 to the user in each show area, and the wearable computer 10 may be collected when the show ends. In this case, the staff may operate the wearable computer 10 and give it to the user.

  FIG. 16 shows a processing flow when the staff operates the wearable computer 10. In this example, before the user enters the pre-show room 65, the staff selects the show on the show selection screen. Therefore, the user receives the wearable computer 10 with the show selected.

  When the main show ends, the staff collects the wearable computer 10 from the user. The staff lends the collected wearable computer 10 to the next user.

  In this way, the user does not need to operate the wearable computer 10.

(2) In the above embodiment, the wearable computer 10 is used as the mobile terminal device. However, a smartphone, a tablet computer, a portable information terminal, or the like can be used.

(3) In the above embodiment, the user selects a show. However, this selection may also be performed automatically by making a selection sound from a speaker used in the show.

(4) In the above embodiment, the case where different speakers are used for the pre-show and the main show has been described. However, the present invention can also be applied to the case where a plurality of curtains (one curtain, two curtains ...) are used with the same speaker. In this case, a selection sound corresponding to the start of each curtain may be output. As described above, the present invention can be applied to all cases where, when different contents are output from the same speaker, support information corresponding to the sound is selected and displayed.

  For example, when the present invention is applied to a movie with an interval, the selection sound and the start sound may be output at the timing shown in FIG. Since the content of the support information varies depending on the movie title, the user may select the title on the screen (the same as the selection of the show). Note that the selection of a title may also be performed using a selection sound.

  Furthermore, the present invention can also be applied to a case where different contents are output from different speakers in different places. For example, when there are a plurality of historical facilities, buildings, and the like, the support information can be provided by applying the description to a speaker from a speaker. In this case, a support information file may be prepared for each location where the speaker is located, and a selection sound may be output.

  FIG. 18 shows an example in which a description corresponding to each of a plurality of historical sites is output from a speaker at each location in a sightseeing spot. The selection sound and the start sound are output at each historical place. Since the content of the support information differs depending on the sightseeing spot (Kyoto or Osaka), the user may select the sightseeing spot on the screen (the same as the selection of the show). It should be noted that selection of a sightseeing spot may also be performed using a selection sound.

(5) In the above embodiment, the selection sound and the start sound are frequency sounds. However, other sounds that are difficult for the user to hear (for example, high-frequency sounds) may be used. In addition, since the frequency sound has low directivity, it can be transmitted to a wide range of users regardless of the direction of the speaker. On the other hand, since high frequency sound has high directivity, it can be sent to a user in a predetermined direction.

(6) In the above embodiment, a selection sound and a start sound are prepared separately from the show content sound. However, the selection sound and the start sound may be recorded in advance in the show content sound file.

(7) In the above embodiment, dialogues are displayed in Japanese for Japanese users. However, dialogue may be displayed in an understandable language such as a native language for a foreign user.

(8) In the above embodiment, the selected sound is constituted by a combination of sounds of two frequencies. However, a combination of sounds having three or more frequencies may be used. Moreover, one sh is good also as only the sound of a number.

(9) In the above embodiment, the start sound is a sound of one frequency. Therefore, the detection process is quick and it is easy to adjust the timing. Note that the start sound may be configured by a combination of sounds having a plurality of frequencies.

(10) In the above embodiment, the plurality of speakers 6 and 7 arranged close to each other are set as target speakers. However, only one target speaker may be used.

(11) In the above embodiment, a high frequency sound is used as the sound. However, low frequency sounds may be used. Since low-frequency sound has no directivity, it can be distributed evenly to users in the area regardless of the direction of the speaker.

(12) The above embodiments and modifications can be implemented in combination with other embodiments and modifications as long as they do not contradict the essence.

2. Second embodiment
2.1 System Configuration FIG. 19 shows a functional block diagram of an information support system for a show according to the second embodiment. An identifier transmission device 2, a signal output device 4, and a speaker 6 are provided in the area where the show is performed. The identifier transmission device 2 transmits its own ID. In this embodiment, an identifier transmission device 2 having a different ID for each area is provided.

  The signal output device 4 gives sound signals such as lines and music to the speaker 6 in accordance with the progress of the show. The speaker 6 receives this sound signal and outputs speech and music as sound.

  In this embodiment, the line of the performer of the show is not given by the performer, but is given to the audience by the sound from the speaker 6. Therefore, in the signal output device 4, sound signals such as lines and music in accordance with the progress of the show are recorded in advance.

  The signal output device 4 gives the above sound signal to the speaker 6 at the start of the show. In this embodiment, a start signal is inserted at the beginning of the sound signal.

  A user who is a spectator possesses the mobile terminal device 10. In the recording unit 22 of the portable terminal device 10, support information is recorded with text information such as lines of shows performed in each area.

  The area specifying unit 12 of the mobile terminal device 10 receives the ID from the identifier transmitting device 2 and specifies which area it is in.

  Along with the start of the show, a start sound based on a start signal is output from the speaker 6. The start sound detecting means 16 of the mobile terminal device 10 determines whether or not the start sound is included in the ambient sound acquired by the microphone 14. The display control device 20 of the mobile terminal device 10 selects the support information 24 corresponding to the area specified by the area specifying unit 12 from the support information 24 recorded in the recording unit 22. Further, when the start sound is detected, the selected support information 24 is read and displayed on the display 18.

  Therefore, support information such as lines is displayed as character text on the mobile terminal device 10 of the user as the show progresses.

  FIG. 20 shows a system configuration when an information support system for a show is realized in a theme park. Areas A1 to An for performing shows are provided in the theme park. As shown in FIG. 21, a stage 51 and a spectator seat (audience area) 49 are provided in each of the show areas A1 to An. Further, in each of the areas A1 to An, a wireless access point 2 that is an identifier transmission device, an acoustic device 4 that is a signal output device, and a speaker 6 that is connected to the acoustic device 4 are provided. As shown in FIG. 4, for example, the areas A1 to A6 where the show is performed are scattered in the theme park.

  The wireless access point 2 is a device for relaying a user's smartphone or the like to the Internet. The wireless access point 2 has an identifier called SSID, receives an inquiry from a terminal device such as a smartphone, and returns an SSID. This SSID is different for each wireless access point. In addition, the range of radio waves from the wireless access point is narrow (about 100 m). Therefore, the terminal device can know which area A1 to A6 is near the access point 2 by receiving this SSID.

  In this embodiment, an identifier transmission device that transmits an identifier (SSID) to the mobile terminal device 10 using the function of the access point 2 for connecting a terminal device such as a smartphone to the Internet.

  FIG. 22 shows the configuration of the acoustic device 4. The recording unit 42 is given show data such as music and dialogue used in the show in the area to the speaker 6. In the show, the performer does not speak the dialogue, and the dialogue included in the show data is output from the speaker 6 and the performer performs correspondingly. In other words, the progress of the show is performed based on the show data.

  FIG. 23 shows a hardware configuration of the mobile terminal device 10. In this embodiment, the tablet computer is the mobile terminal device 10. A memory 62, a touch screen display 64, a nonvolatile memory 66, a wireless LAN circuit 68, a microphone 70, and a camera 72 are connected to the CPU 60.

  The memory 62 is constituted by a high-speed but volatile semiconductor memory or the like, and functions as a work area for the CPU 60. The touch screen display 64 performs display and accepts input by a user operation. The wireless LAN circuit 68 is a circuit for communicating with the access point 2.

  In the nonvolatile memory 66, an operating system 74, a portable terminal program 76, and support information 78 are recorded. The portable terminal program 76 exhibits its functions in cooperation with the operating system 74. These programs are downloaded from the server device via the wireless LAN circuit 68, for example.

In addition, support information 78 is also recorded in the nonvolatile memory 66. The support information 78 is data to be displayed on the touch screen display 64 in correspondence with the show data that is flowed at the show in each of the areas A1 to A6. In this embodiment, support information for all areas A1 to A6 is downloaded from a server device or the like and recorded in advance.

2.2 Support Information Providing Process FIGS. 24 and 25 show a flowchart of the support information providing process by the portable terminal program 76. When the mobile terminal program 76 of the mobile terminal device 10 possessed by the user is started, the CPU 60 of the mobile terminal device 10 (hereinafter, may be abbreviated as the mobile terminal device 10) causes the SSID recorded in the non-volatile memory 66. Is read (step S1). FIG. 9 shows an example of the SSID list. The SSID is recorded in association with each of the access points 2 provided in the areas A1 to A6 of each show shown in FIG. Further, the file name of the show support information file executed at the access point and the start time of the show are also recorded.

  Next, the mobile terminal device 10 acquires a signal from the access point received by the wireless LAN circuit 68 and analyzes the SSID. Since the signal from the access point 2 in each of the areas 1 to A6 is attenuated according to the distance, only the signal of the access point in the vicinity of the mobile terminal device 10 is received.

  Further, it is determined whether or not the received SSID is a predetermined SSID registered in the above SSID list (step S2). This is because some theme parks have access points that are not related to providing information for the show.

  If the received SSID is not a predetermined SSID, signal reception and SSID confirmation processing are repeated. When receiving the predetermined SSID recorded in the list, the mobile terminal device 10 determines whether or not the intensity of the radio wave exceeds a threshold value (step S3). As described above, the radio field intensity attenuates according to the distance from the access point. Therefore, by confirming whether the radio field intensity is equal to or higher than a predetermined value, it is confirmed whether the mobile terminal device 10 (that is, the user having it) has entered the show area (whether it has approached a predetermined distance). be able to. The radio wave intensity can be acquired from the wireless LAN circuit 68.

  If the radio wave intensity does not exceed the predetermined value, the mobile terminal device 10 returns to step S2 and performs radio wave reception and SSID confirmation. When the radio field intensity exceeds the predetermined value, the support information file name of the access point (area) corresponding to the received SSID is specified with reference to the SSID list in FIG. 26 and searched from the nonvolatile memory 66 (step S5). ). Here, in the support information file, the lines of the performers, explanations of scenes, and the like are recorded so as to match the progress of the show.

  When the mobile terminal device 10 finds the support information file, the mobile terminal device 10 reads it from the nonvolatile memory 66 and develops it in the memory 62, which is a work area (step S7). If the support information file is not found even after a predetermined time elapses due to a broken file or the like (step S10), the server device is accessed via the access point and the support file is downloaded again. Then, it returns to step S1 and repeats a process.

  When the support information file is expanded in the memory 62, the mobile terminal apparatus 10 acquires the start time of the nearest show ahead of the current time from the SSID list (step S8). For example, when the SSID is “32149373” and the current time is 12:00, 13:30 of the start time 2 is acquired.

  The mobile terminal device 10 determines whether or not the current time is less than 5 minutes before the start time (step S9). When 5 minutes have passed, the mobile terminal device 10 acquires ambient sound through the microphone 70. In this embodiment, the synthesized sound of the first frequency sound (100 Hz in this embodiment) and the second frequency sound (50 Hz in this embodiment) is included as the start sound at the beginning of the show data 44 ( (See FIG. 22).

  Therefore, a sound obtained by synthesizing the first frequency sound and the second frequency sound is output from the speaker 6 arranged in the show area a predetermined time before the show start (in this embodiment, three minutes before). . Since these sounds have directivity (high) frequencies, they reach the area where the user is located depending on the direction and arrangement of the speakers. Further, the apparatus shown in FIG. 22 is a device that is originally necessary for a show, and only a part of the show data is changed. Therefore, existing facilities for the show can be used as they are.

  The mobile terminal device 10 determines whether or not the acquired ambient sound includes the first frequency component (100 Hz in this embodiment) at a level equal to or higher than a predetermined value (step S12). This determination can be realized by analyzing data obtained by binarizing the acquired ambient sound using a bandpass filter program.

  If the first frequency component equal to or greater than the predetermined value is not included, the process returns to step S11 and the acquisition of the ambient sound is repeated. If the first frequency component equal to or higher than the predetermined value is included, the mobile terminal device 10 includes the second frequency component (50 Hz in this embodiment) in the acquired ambient sound at a level equal to or higher than the predetermined value. It is determined whether or not (step S13).

  If the second frequency component equal to or greater than the predetermined value is not included, the process returns to step S11 and the acquisition of the ambient sound is repeated. If it is determined in step S13 that the second frequency component equal to or greater than the predetermined value is included, a start sound is generated from the speaker 6 in the area as a synthesized sound of the first frequency sound and the second frequency sound. Therefore, it can be determined that it is a predetermined time before the show start (in this embodiment, 3 minutes before).

  The portable terminal device 10 reproduces the support information file developed in the memory 62 and displays it on the touch screen display 64 together with the detection of the start sound (step S14).

  FIG. 27 shows a display example of the touch screen display 64. A balloon 100 shows a portrait 100 of the character that the performer is jealous of and the text of the performer's dialogue. Therefore, even a deaf user can know the dialogue. A character name is displayed under the portrait 100. In the example of FIG. 27, a portrait of the same character is displayed on the same side (left side or right side) so that it can be easily discriminated.

  The mobile terminal device 10 displays the support information on the touch screen display 64 one after another according to the support information file. Note that when the screen cannot be displayed, the screen is scrolled upward. In this embodiment, the support information file is not a moving image but is composed of a still image in which display timing is recorded for each line. Further, since the support information is generated so as to coincide with the progress of the show, the dialogue of each performer accompanying the progress of the show can be obtained.

  Further, as shown in FIG. 28, points that should be noted on the stage other than the dialogue and explanation 108 can be included in the support information. In this embodiment, the attention point and the explanation 108 are displayed in a format that is not a balloon.

  When the end of the support information file is displayed, the mobile terminal device 10 deletes the support file expanded in the memory 62 from the memory 62 (step S17). When the stop button 106 is pressed during the display of the support information file, the mobile terminal device 10 stops displaying the support information file (step S15).

As described above, the support information is displayed on the mobile terminal device 10 at hand of the user as the show progresses.

2.3 Other
(1) In the above embodiment, a synthesized sound having two frequencies is used as the start sound. However, a single frequency may be used as the start sound. Moreover, you may make it use the synthetic | combination sound of three or more frequencies.

(2) In the above embodiment, a high sound is used as the start sound. However, sounds having other frequencies may be used. For example, ultrasonic waves or low sounds can be used.

(3) In the above embodiment, an access point is used as the identifier transmission device. However, you may use the apparatus which transmits identifiers, such as a beacon.

(4) In the above embodiment, when the start sound is not captured, the support information about the show is not displayed. However, in correspondence with the time during the show (for example, 3 minutes after the start of the show), the show data 44 is set to have a different frequency from the other start sounds (for example, 40 Hz and 110 Hz). May be included. When the mobile terminal device 10 receives the other start sound, the display is started on the touch screen display 64 from the support information corresponding to the time during the show.

  In this way, even when the original start sound cannot be captured, it is possible to display support information that matches the progress of the show.

(5) In the above embodiment, the theme park show has been described. However, it can be applied to general shows such as movies in a movie theater where the lines of the performers progress on time.

(6) In the above embodiment, the portraits are displayed in FIGS. However, you may make it display a photograph.

(7) In the above embodiment, dialogue is displayed in a balloon from a portrait. However, as shown in FIG. 12, the performer imaged by the user with the camera 72 of the mobile terminal device 10 is displayed on the touch screen display 64 (see 110 in the figure), and the dialogue is displayed on top of this. It may be. In this case, the mobile terminal device 10 determines who the character that the imaged performer plays is based on the combination of the clothes color of the performer (such as the presence or absence of a hat, glasses), and so on. This can be realized by displaying. In this case, the support information may include dialogue text data, display start time based on the time from the start of the show, display end time, and information indicating the character.

(8) In the above embodiment, dialogues are displayed in Japanese for Japanese users. However, dialogue may be displayed in an understandable language such as a native language for a foreign user.

(9) In the above embodiment, sound is used as the start signal. However, this may be used as a start signal, for example, by turning on / off the light emitted from the illumination used for the show. In this case, a sensor that receives the illumination light is provided in the mobile terminal device 10 and it may be determined whether the signal is a start signal based on whether the illumination blinks in a pattern predetermined as a start signal. Moreover, it is good also as a start signal combining a sound and light.

(10) In the above embodiment, a common start sound is used for the pre-show and the main show. However, the start sound for the pre-show and the start sound for the main show may have different frequencies (a combination of different frequencies).

(11) The above embodiments and modifications can be implemented in combination with other embodiments and modifications as long as they do not contradict its essence.

Claims (28)

An information support system comprising a speaker that outputs at least content sound toward a user, a signal output device that outputs a signal to the speaker, and a mobile terminal device that the user has,
The signal output device includes a signal including at least a content audio signal for the content audio, a selection sound signal for selecting the content audio, and a start sound signal associated with the start of output of the content audio. Configured to output to the speaker;
The portable terminal device
A recording unit that records a plurality of units of support information including character text created so that the reproduction timing coincides with the sound output from the speaker over time due to the reproduction of the content audio signal;
Selection sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the selection sound output from the speaker;
When a selection sound is detected by the selection sound detection means, support information selection means for selecting one support information from the support information including the plurality of units of text text based on the selection sound;
Start sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the start sound output from the speaker;
Display control means for starting display on the display of the support information including the character text selected by the support information selection means when the start sound is detected by the start sound detection means;
Information support system with A portable terminal device for providing information support to a user,
A recording unit that records a plurality of units of support information including character text created so that the reproduction timing coincides with the sound output from the speaker over time due to the reproduction of the content audio signal;
Selection sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the selection sound output from the speaker;
When a selection sound is detected by the selection sound detection means, support information selection means for selecting one support information from the support information including the plurality of units of text text based on the selection sound;
Start sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the start sound output from the speaker;
Display control means for starting display on the display of the support information including the character text selected by the support information selection means when the start sound is detected by the start sound detection means;
A mobile terminal device. A portable terminal program for realizing a portable terminal device for providing information support to a user by a computer, the computer comprising:
Selection sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the selection sound output from the speaker;
When a selection sound is detected by the selection sound detection means, based on the selection sound, support information selection means for selecting one support information from support information including a plurality of units of text text recorded in the recording unit ;
Start sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the start sound output from the speaker;
When the start sound is detected by the start sound detection means, a portable terminal program for functioning as a display control means for starting display on the display of the support information including the character text selected by the support information selection means. ,
The support information includes character text created so that the playback timing coincides with the sound output from the speaker as time elapses due to the playback of the content audio signal, and is recorded in a plurality of units. A portable terminal program. The system of claim 1, wherein
The plurality of units of support information recorded in the recording unit corresponds to content audio that is reproduced in time order on a speaker installed in the same place,
The selected sound includes a sound having a different frequency for each support information . The system of claim 1, wherein
The plurality of units of support information recorded in the recording unit corresponds to content audio played back by speakers installed in different places,
The selected sound includes a sound having a different frequency for each support information . The system of claim 4 or 5,
The selection sound is a sound of a predetermined frequency,
The selection sound detection means determines whether or not the selection sound is included in the ambient sound acquired by the microphone depending on whether or not the sound of the predetermined frequency is included in a predetermined level or more. System. The system of claim 6, wherein
The sound is a combination of sounds of a plurality of frequencies,
The selection sound detecting means determines whether or not the selection sound is included in the ambient sound acquired by the microphone based on whether or not all of the plurality of sounds having the predetermined frequency are included in a predetermined level or more. A system characterized by that . The system according to any one of claims 1, 4 to 7,
The support information selection means reads the selected support information from the recording unit and develops it in a memory that can be accessed at high speed for display processing on the display . The system according to any one of claims 1, 4 to 8,
The start sound is a sound of a predetermined frequency,
The start sound detection means is configured to determine whether or not the ambient sound acquired by the microphone includes a start sound depending on whether or not the sound of the predetermined frequency is included in a predetermined level or more. System. The system of claim 9, wherein
The sound is a combination of sounds of a plurality of frequencies,
The start sound detection means determines whether or not the ambient sound acquired by the microphone includes a start sound based on whether or not all of the plurality of sounds having the predetermined frequency are included in a predetermined level or more. A system characterized by that . In the program of Claim 3,
The plurality of units of support information recorded in the recording unit corresponds to content audio that is reproduced in time order on a speaker installed in the same place,
The selection sound, the program characterized by comprising a sound of different frequencies for each support information. In the program of Claim 3,
The plurality of units of support information recorded in the recording unit corresponds to content audio played back by speakers installed in different places,
The selection sound, the program characterized by comprising a sound of different frequencies for each support information. The program according to claim 11 or 12,
The selection sound is a sound of a predetermined frequency,
The selection sound detection means determines whether or not the selection sound is included in the ambient sound acquired by the microphone depending on whether or not the sound of the predetermined frequency is included in a predetermined level or more. Program to do . The program of claim 13,
The sound is a combination of sounds of a plurality of frequencies,
The selection sound detecting means determines whether or not the selection sound is included in the ambient sound acquired by the microphone based on whether or not all of the plurality of sounds having the predetermined frequency are included in a predetermined level or more. A program characterized by that . The program according to any one of claims 3, 11 to 14,
The support information selecting means reads out the selected support information from the recording unit, for display processing of the display, the program characterized by deploying high-speed accessible memory. In the program in any one of Claim 3, 11-15,
The start sound is a sound of a predetermined frequency,
The start sound detection means is configured to determine whether or not the ambient sound acquired by the microphone includes a start sound depending on whether or not the sound of the predetermined frequency is included in a predetermined level or more. Program to do. The program of claim 16,
The sound is a combination of sounds of a plurality of frequencies,
The start sound detection means determines whether or not the ambient sound acquired by the microphone includes a start sound based on whether or not all of the plurality of sounds having the predetermined frequency are included in a predetermined level or more. A program characterized by that . Information including one or more speakers that sequentially output at least the first content sound and the second content sound toward the user, a signal output device that outputs a signal to the speaker, and a mobile terminal device that the user has A support system,
The signal output device is configured to select a first content audio signal and a second content audio signal for the first content audio and the second content audio, and a first for selecting the first content audio and the second content audio. and selection tone signals and second selection signal, the first content audio and first start sound signal and including at least signal a second start sound signal is Installing corresponding to the output start of the second content audio to the speaker Configured to output to
The portable terminal device
The first support information and the second support information including character text created so that the playback timing coincides with the sound output from the speaker over time due to the playback of the first content audio signal and the second content audio signal. A recording section for recording
Selection sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the first selection sound or the second selection sound output from the speaker;
When the first selection sound is detected by the selection sound detection means, the first support information recorded in the recording unit is selected to be reproduced, and when the second selection sound is detected by the selection sound reproduction means, Support information selection means for selecting the second support information recorded in the recording unit as to be reproduced;
Start sound detection means for detecting whether the ambient sound acquired by the microphone includes the first start sound or the second start sound output from the speaker;
When the first start sound is detected by the start sound detection means, the display of the first support information including the character text selected by the support information selection means is started on the display, and the second start sound is detected by the start sound detection means. Is detected, the display control means for starting the display on the display of the second support information including the character text selected by the support information selection means,
Information support system with A portable terminal device for providing information support to a user,
The first support information and the second support information including character text created so that the playback timing coincides with the sound output from one or more speakers as time passes by reproducing the first content audio signal and the second content audio signal. A recording section for recording
Selection sound detecting means for detecting whether or not the ambient sound acquired by the microphone includes the first selection sound or the second selection sound output from the speaker;
When the first selection sound is detected by the selection sound detection means, the first support information recorded in the recording unit is selected to be reproduced, and when the second selection sound is detected by the selection sound reproduction means, Support information selection means for selecting the second support information recorded in the recording unit as to be reproduced;
Start sound detection means for detecting whether the ambient sound acquired by the microphone includes the first start sound or the second start sound output from the speaker;
When the first start sound is detected by the start sound detection means, the display of the first support information including the character text selected by the support information selection means is started on the display, and the second start sound is detected by the start sound detection means. Is detected, the display control means for starting the display on the display of the second support information including the character text selected by the support information selection means,
A mobile terminal device. A portable terminal program for realizing a portable terminal device for providing information support to a user by a computer, the computer comprising:
A selection sound detecting means for detecting whether the ambient sound acquired by the microphone includes the first selection sound or the second selection sound output from one or more speakers;
When the first selection sound is detected by the selection sound detection means, the first support information recorded in the recording unit is selected to be reproduced, and when the second selection sound is detected by the selection sound reproduction means, Support information selection means for selecting the second support information recorded in the recording unit as to be reproduced;
Start sound detection means for detecting whether the ambient sound acquired by the microphone includes the first start sound or the second start sound output from the speaker;
When the first start sound is detected by the start sound detection means, the display of the first support information including the character text selected by the support information selection means is started on the display, and the second start sound is detected by the start sound detection means. In the portable terminal program for functioning as display control means for starting display on the display of the second support information including the character text selected by the support information selection means,
The first support information and the second support information are character texts created so that the playback timing coincides with the sound output from the speaker over time due to the playback of the first content audio signal and the second content audio signal. A portable terminal program characterized by including. The system of claim 18, wherein
The first selection sound includes at least a sound of a first frequency;
The first start sound includes at least a mixed sound of a sound of a first frequency and a sound of a third frequency;
The second selected sound includes at least a sound of a fourth frequency;
The second start sound includes a mixed sound of at least a fourth frequency sound and a third frequency sound . The system of claim 21, wherein
The previous first start sound further includes a mixed sound including the sound of the second frequency,
The second start sound, furthermore, the system characterized by comprising a mixed sound including a sound of a fifth frequency. The system according to any one of claims 1, 4 to 10, 18, 21, and 22 .
The system, wherein the display is a head mounted display . The system according to any one of claims 1, 4 to 10, 18, 21 to 23 ,
The speaker is arranged in the vicinity of an area where a show is performed, an area where a video is displayed, or an explanation object, and at least outputs a speech of the performer of the show, an explanation voice regarding the movie, or an explanation voice regarding the explanation object as content audio And
The mobile terminal device, and wherein the in correspondence with the content audio and reproduces at least character text corresponding to the words, a character text corresponding to the character text or a description voice corresponding to the description speech System. The program of claim 20,
The first selection sound includes at least a sound of a first frequency;
The first start sound includes at least a mixed sound of a sound of a first frequency and a sound of a third frequency;
The second selected sound includes at least a sound of a fourth frequency;
The second start sound includes a mixed sound of at least a fourth frequency sound and a third frequency sound . The program of claim 25.
The previous first start sound further includes a mixed sound including the sound of the second frequency,
The second start sound further program characterized by comprising a mixed sound including a sound of a fifth frequency. In the program according to any one of claims 3, 11 to 17, 20, 25, and 26,
The program , wherein the display is a head mounted display . In the program according to any one of claims 3, 11 to 17, 20, 25 to 27,
The speaker is arranged in the vicinity of an area where a show is performed, an area where a video is displayed, or an explanation object, and at least outputs a speech of the performer of the show, an explanation voice regarding the movie, or an explanation voice regarding the explanation object as content audio And
The mobile terminal device, and wherein the in correspondence with the content audio and reproduces at least character text corresponding to the words, a character text corresponding to the character text or a description voice corresponding to the description speech Program to do.