JP6995224B1 - How to provide virtual travel - Google Patents

Abstract

A virtual travel providing device 10, a guide terminal 20 and a guide data acquisition device 30 used by a guide to guide a virtual trip, an interpreter terminal 40 and an interpreter data acquisition device 50 used by an interpreter who interprets the guide, and a virtual A virtual travel providing system is provided that includes a user terminal 60 and a playback device 70 used by a user who travels, and can virtually experience a trip without going to the site.

Description

This disclosure relates to a virtual travel provision method that allows a virtual experience of travel without going to the site.

Traditionally, travel (especially overseas travel) has been a big event that is complicated, expensive and time consuming to prepare. In particular, it is expensive and takes a long time to reach the site by airplane.

Japanese Unexamined Patent Publication No. 2018-155887

An object of the present disclosure is to provide a virtual travel providing method, a virtual travel providing device, a virtual travel providing program, and a virtual travel providing system that enable a virtual experience of travel without going to the site. Further, the subject of the present invention is a user terminal, a virtual travel reproduction device, a virtual travel reproduction method, a guide terminal, a guide data acquisition device, a guide method, an interpreter terminal, which can be used for such a virtual travel providing method and the like. It is to provide an interpretation data acquisition device and an interpretation method.

According to one embodiment, a guide information registration step for registering guide information for each of a plurality of guides, a guide information receiving step for receiving a guide condition from a user, the guide condition, and the registered guide information. A guide specifying step for identifying any of the plurality of guides, first video data from the camera of the specified guide, and first audio data from the microphone of the specified guide, based on the above. The local data reception step to be received and the second video data for reproducing the video corresponding to the first video data on the display in the glasses-type wearable device specified by the user are generated based on the first video data. , A reproduction data generation step of generating a second audio data for reproducing the audio corresponding to the first audio data by the speaker in the glasses-type wearable device based on the first audio data, the second video data, and the second video data. The reproduction data transmission step of transmitting the second audio data to the glasses-type wearable device, the image based on the second video data is reproduced on the display of the glasses-type wearable device, and the audio based on the second audio data is reproduced. Provided is a virtual travel providing method comprising a reproduction step of reproduction on a speaker of the glasses-type wearable device.

It is desirable that the user not accompany the identified guide.

Based on the interpreter information registration step for registering the interpreter information for each of the plurality of interpreters, the interpreter condition receiving step for receiving the interpreter condition from the user, the interpreter condition, and the registered interpreter information. , The interpreter identification step for identifying any of the plurality of interpreters, and the interpreter voice data for reproducing the voice corresponding to the first voice data on the speaker of the specified interpreter. From the interpreter's voice data generation step to generate based on the voice data, the interpreter's voice reproduction step to reproduce the voice based on the interpreter's voice data on the specified interpreter's speaker, and the interpreter's speaker. The reproduction data generation is provided with an interpreted voice data receiving step of receiving the interpreted voice data picked up by the interpreter's microphone from the voice translated by the specified interpreter based on the reproduced voice. In the step, it is desirable to generate the first voice data and the second voice data for reproducing the voice corresponding to the interpreted voice data based on the first voice data and the translated voice data.

It is desirable that the identified interpreter not accompany the user or the identified guide.

According to one embodiment, one of the plurality of guides is based on a guide information registration unit that registers guide information for each of the plurality of guides, a guide condition from a user, and the registered guide information. The local data receiving unit that receives the guide specifying unit, the first video data from the camera of the specified guide, and the first audio data from the microphone of the specified guide, and the first. The second video data for playing back the video corresponding to the video data on the user's display is generated based on the first video data, and the voice corresponding to the first voice data is played back on the user's speaker. A virtual travel providing device including a reproduction data generation unit for generating a second voice data for the purpose based on the first voice data is provided.

It is desirable that the reproduction data generation unit generate the reproduction data including translation data in which at least a part of the utterance of the guide included in the first voice data is translated into another language.

It is desirable that the translation data is video data for displaying the translation result in subtitles on the user's display.

One of the plurality of interpreters is selected based on the interpreter information registration unit that registers the interpreter information for each of the plurality of interpreters, the interpretation conditions from the user, and the registered interpreter information. The reproduction data generation unit includes an interpreter specifying unit for specifying, and the reproduction data generation unit uses the first voice for interpreting to reproduce the voice corresponding to the first voice data on the speaker of the specified interpreter. An interpreter data receiving unit that is generated based on the data and receives the interpreted voice data from the specified interpreter's microphone is provided, and the reproduced data generation unit is used for the first voice data and the interpreted voice data. It is desirable to generate the second voice data for reproducing the corresponding voice based on the first voice data and the interpreted voice data.

It is desirable that the translated voice data includes a voice picked up by the interpreter's microphone from the voice translated by the specified interpreter based on the voice reproduced from the interpreter's speaker.

It is desirable that the reproduction data generation unit generate the second video data including the interpreter's avatar.

The guide information includes a first language in which the guide can be used, the interpreter information includes a second language in which the interpreter can be used, and the guide condition includes a third language desired by the user. It is desirable that the interpreter identification unit identifies the interpreter based on the first language, the second language, and the third language in which the specified guide can be used.

It is desirable to include a billing unit that performs processing for passing the interpreter fee from the user to the specified interpreter.

It is desirable that the reproduction data generation unit generate the second video data including the guide avatar.

It is desirable to include a billing unit that performs processing for passing the guide fee from the user to the specified guide.

The guide information includes information indicating a guideable first time and a guideable first area, and the guide condition includes information indicating a second time for which a guide is desired and a second area for which a guide is desired. It is desirable that the guide specifying unit identifies the guide by matching the first time with the second time and matching the first area with the second area.

It is desirable that the user's display and speaker be provided in a spectacle-type wearable device.

It is desirable that the camera and microphone of the identified guide be provided on a drone, robot or eyeglass-type wearable device.

The user's display is provided on the spectacle-type wearable device, and the spectacle-type wearable device has means for acquiring movement of the user's face wearing the display and transmitting the movement to the virtual travel providing device, and is specified. The guide camera is a 360-degree camera, and the reproduction data generation unit generates the second video data synchronized with the movement of the user's face based on the first video data from the 360-degree camera. Is desirable.

It is desirable that the two or more playback devices specified by the user be provided with a playback data transmission unit for transmitting the second video data and the second audio data.

According to one embodiment, the computer is subjected to the plurality of guides based on a guide information registration unit for registering guide information for each of the plurality of guides, guide conditions from the user, and the registered guide information. A guide specifying unit that specifies any of the above, a local data receiving unit that receives the first video data from the camera of the specified guide, and the first audio data from the microphone of the specified guide. The second video data for reproducing the video corresponding to the first video data on the user's display is generated based on the first video data, and the voice corresponding to the first voice data is generated by the user's speaker. Provided is a virtual travel providing program that functions as a reproduction data generation unit that generates second audio data for reproduction in the first audio data based on the first audio data.

According to one embodiment, one of the plurality of guides is based on a guide information registration step for registering guide information for each of the plurality of guides, a guide condition from the user, and the registered guide information. The local data receiving step for receiving the guide specifying step, the first video data from the camera of the specified guide, and the first audio data from the microphone of the specified guide, and the first. The second video data for playing back the video corresponding to the video data on the user's display is generated based on the first video data, and the voice corresponding to the first voice data is played back on the user's speaker. Provided is a virtual travel providing method including a reproduction data generation step for generating a second voice data for the purpose based on the first voice data.

According to one embodiment, it is a user terminal that can communicate with the virtual travel providing device and is used by the virtual travel user, receives a guide condition for specifying a guide from the user, and receives the guide condition from the user, and the virtual travel providing device. A user terminal provided with a guide condition setting unit for transmitting to is provided.

It is desirable that the guide condition includes information on a time when a guide is desired, an area where the guide is desired, and a language desired by the user.

It is desirable to have an interpreter condition setting unit that receives the interpreter condition for identifying the interpreter from the user and transmits it to the virtual travel providing device.

It is desirable that the interpreting condition includes information in the language desired by the user.

According to one embodiment, the user terminal that can communicate with the virtual travel providing device and is used by the virtual travel user receives the guide condition for specifying the guide from the user and transmits the guide condition to the virtual travel providing device. A program that functions as a guide condition setting unit is provided.

According to one embodiment, the above-described display that reproduces an image based on image data from a camera of a guide specified based on a guide condition from a user, guide information for each of a plurality of guides, and the like. A virtual travel playback device is provided that includes a speaker that reproduces audio based on audio data from a guide microphone.

It is desirable that the virtual travel reproduction device is a glasses-type wearable device.

The camera of the identified guide is a 360-degree camera, the spectacle-type wearable device has means for acquiring movement of the user's face wearing the camera, and the display is the display from the 360-degree camera. It is desirable to reproduce an image synchronized with the movement of the user's face based on the image data.

It is desirable that the display display subtitles in which at least a part of the utterance of the guide contained in the voice data is translated into another language.

It is desirable that the display reproduces the video including the guide avatar.

It is desirable that the speaker reproduce the voice based on the translated voice data from the microphone of the interpreter specified based on the guide condition and the interpreter information for each of the plurality of interpreters.

It is desirable that the display reproduces a video including the interpreter's avatar.

According to one embodiment, a camera of a guide specified based on a guide condition reception step for receiving a guide condition for specifying a guide from a user, the guide condition, and guide information for each of a plurality of guides. It is desirable to include a reproduction step of reproducing the image based on the video data from the user on the display of the user and reproducing the sound based on the audio data from the microphone of the specified guide on the speaker of the user.

According to one embodiment, it is a guide terminal that can communicate with the virtual travel providing device and is used by the guide, and is a guide information setting unit that receives guide information from the guide and transmits it to the virtual travel providing device. A guide terminal is provided.

It is desirable to include a guide request processing unit that receives a guide request based on the guide conditions from the user and the guide information.

It is desirable that the guide information includes information on guideable time, guideable area and available language.

According to one embodiment, the guide terminal that can communicate with the virtual travel providing device and is used by the guide functions as a guide information setting unit that receives guide information from the guide and transmits the guide information to the virtual travel providing device. A program to let you do it is provided.

According to one embodiment, it is a guide data acquisition device that can communicate with the virtual travel providing device and is used by a guide, and is a camera, a microphone, video data from the camera, and audio data from the microphone. A guide data acquisition device is provided, which comprises a local data transmission unit for transmitting the first video data and the first audio data to the virtual travel providing device, respectively.

It is desirable that the guide data acquisition device be provided in a drone, a robot, or a glasses-type wearable device worn by the guide.

The camera is preferably a 360 degree camera.

According to one embodiment, when a guide request is received based on the step of receiving the guide information from the guide, the guide condition from the user, and the guide information, the video data from the camera of the guide is used. A guide method is provided comprising a step of transmitting audio data from the guide microphone to the virtual tour providing device.

According to one embodiment, it is a terminal for an interpreter that can communicate with the virtual travel providing device and is used by an interpreter, receives interpreter information from the interpreter, and transmits the information to the virtual travel providing device. An interpreter terminal provided with an interpreter information setting unit is provided.

It is desirable to have an interpreter request processing unit that receives an interpreter request based on the interpreter conditions from the user and the interpreter information.

The interpreter information preferably includes interpretable time and available languages.

According to one embodiment, the interpreter who can communicate with the virtual travel providing device and receives the interpreter information from the interpreter and transmits the interpreter terminal used by the interpreter to the virtual travel providing device. A program that functions as an information setting unit is provided.

According to one embodiment, the interpreter data acquisition device that can communicate with the virtual travel providing device and is used by the interpreter is provided with an interpreter data receiving unit that receives the voice data for interpretation from the virtual travel providing device. A speaker that reproduces voice corresponding to the voice data for interpretation, a microphone that picks up the speech of the interpreter, and an interpretation data transmission unit that transmits voice data from the microphone to the virtual travel providing device as the translated voice data. And, an interpreter data acquisition device is provided.

According to one embodiment, voice data from the guide's microphone when an interpreter request is received based on the step of receiving the interpreter information from the interpreter, the interpreter conditions from the user, and the interpreter information. The step of receiving the voice data for interpretation based on the above, the step of reproducing the voice corresponding to the voice data for interpretation on the speaker, and the voice that the interpreter interprets based on the reproduced voice is the interpreter. An interpreting method is provided that includes a step of transmitting the translated voice data picked up by the microphone of.

According to one embodiment, the virtual travel providing device, the user terminal, the playback device, the guide terminal, the guide data acquisition device, the interpreter terminal, and the interpreter data acquisition device. , A virtual travel provision system is provided.

The block diagram which shows the schematic structure of the virtual travel provision system which concerns on one Embodiment. The figure which shows an example of the structure of a guide database schematically. The figure which shows an example of the structure of an interpreter database schematically. A sequence diagram showing the processing operation of the virtual travel providing system until the guide is specified. A sequence diagram showing the processing operation of the virtual travel providing system for sharing the local video and audio with the user after the guide is specified. A sequence diagram showing the processing operation of the virtual travel providing system for identifying the interpreter. A sequence diagram showing the processing operation of a virtual travel providing system for sharing local video and audio with a user when there is an interpreter.

FIG. 1 is a block diagram showing a schematic configuration of a virtual travel providing system according to an embodiment. The virtual travel providing system includes a virtual travel providing device 10, a guide terminal 20 and a guide data acquisition device 30 used by a guide who guides a virtual trip, an interpreter terminal 40 and an interpreter data used by an interpreter who interprets the guide. It includes an acquisition device 50, a user terminal 60 used by a user who makes a virtual trip, and a playback device 70.

The virtual travel providing device 10 is, for example, a server, and can communicate with the guide terminal 20, the guide data acquisition device 30, the interpreter terminal 40, the interpreter data acquisition device 50, the user terminal 60, and the reproduction device 70. There are no particular restrictions on the type of communication, but wireless communication is desirable.

The virtual travel providing device 10 includes a guide information registration unit 11, a guide information storage unit 12, a guide identification unit 13, an interpreter information registration unit 14, an interpreter information storage unit 15, an interpreter identification unit 16, and the like. It has a local data receiving unit 17, an interpreter data receiving unit 18, a reproduction data generation unit 19, a reproduction data transmission unit 1A, and a charging unit 1B. Each of these parts may be contained in one device or may be distributed in a plurality of devices (in the latter case, the plurality of devices are collectively referred to as "virtual travel providing device 10". ). Further, at least a part of the functions of each of these parts may be realized by the processor executing a predetermined program.

The guide information registration unit 11 receives the guide information from the guide terminal 20 and registers it in the guide information storage unit 12 as a guide database.

FIG. 2 is a diagram schematically showing an example of the structure of the guide database. The guide information is information about each guide and may include information such as guideable time, guideable area, available language, etc. associated with the guide ID that identifies the guide, as shown.

The format of the guideable time is arbitrary, and it may be specified by the day of the week and the time zone such as "weekdays 8 to 12 o'clock", or a specific date and time such as "August 10 9 to 17 o'clock". And the time zone may be specified. In addition, the format of the guideable area is arbitrary, and it may be specified in a wide area such as a prefecture or a state, or it may be specified in a narrow area such as Asakusa or Las Vegas. In addition, the guide information may include information such as a strong area, a strong theme, a fee, a gender, an age, self-introduction, and SNS profile information.

Returning to FIG. 1, the guide specifying unit 13 specifies a guide according to the user's wishes. More specifically, the guide specifying unit 13 identifies the guide by comparing the guide condition transmitted from the user terminal 60 with the guide information stored in the guide information storage unit 12. The guide condition is also a condition for searching the guide, and may include information such as the date and time when the guide is desired, the area where the guide is desired, and the language available to the user. In addition, the guide conditions may include information such as a fee, a guide gender, age, and user evaluation.

The interpreter information registration unit 14 receives the interpreter information from the interpreter terminal 40 and registers it in the interpreter information storage unit 15 as an interpreter database.

FIG. 3 is a diagram schematically showing an example of the structure of the interpreter database. The interpreter information is "information about the interpreter, and may include information such as the interpretable time and available language associated with the interpreter ID that identifies the interpreter, as shown in the figure. Other interpreter information. May include information such as a favorite language, fee, gender, age, self-introduction, SNS profile information, etc. In this embodiment, the interpreter does not need to accompany the guide. Therefore, the interpreter information Does not have to include the interpretable area or the interpreter's activity area.

Returning to FIG. 1, the interpreter identification unit 16 identifies an interpreter according to the user's wishes. More specifically, the interpreter identification unit 16 identifies the interpreter by comparing the interpreter conditions transmitted from the user terminal 60 with the interpreter information stored in the interpreter information storage unit 15. The interpreter condition is also a condition for searching for an interpreter, and may include a date and time when a person wants to travel, a language desired by the user, and the like. In addition, the interpreting conditions may include information such as fees, gender, age, and user evaluation of the interpreter.

The local data receiving unit 17 receives the video and audio actually taken by the guide at the travel destination (local). More specifically, the local data receiving unit 17 receives the video data from the camera 31 of the guide data acquisition device 30 and the audio data from the microphone 32.

The interpreter data receiving unit 18 receives the voice obtained by the interpreter interpreting the utterance of the guide (hereinafter referred to as “interpreter voice”). More specifically, the interpreter data receiving unit 18 receives voice data from the microphone 32 of the interpreter data acquisition device 50.

The reproduction data generation unit 19 generates reproduction data for reproducing video and audio on the user's reproduction device 70.

More specifically, the reproduction data generation unit 19 displays the image on the display 72 of the user's reproduction device 70 based on the image data from the guide data acquisition device 30 in order to share the image taken in the field with the user. Generate video data for playback. Further, the reproduction data generation unit 19 is for reproducing the sound on the speaker 73 in the user's reproduction device 70 based on the voice data from the guide data acquisition device 30 in order to share the locally acquired voice with the user. Generate audio data.

Further, the reproduction data generation unit 19 generates voice data for reproducing the voice by the speaker 73 in the user's reproduction device 70 based on the voice data from the interpretation data acquisition device 50 in order to reproduce the interpreter voice.

The reproduction data generation unit 19 has a function of automatically performing translation (or a function of acquiring the result of automatic translation from the outside), and the reproduction data is at least one of the utterances of the guide in the voice data from the guide acquisition device. It may include translation data obtained by translating the part into another language. The translation data may be video data for displaying the translation result in subtitles, or may be audio data for outputting the translation result by voice.

The reproduction data transmission unit 1A transmits the reproduction data generated by the reproduction data generation unit 19 to the reproduction apparatus 70 of the user.

The billing unit 1B uses, for example, the credit card information of the pre-registered user to perform processing for giving and receiving a guide fee and an interpreter fee. More specifically, the billing unit 1B performs a process for receiving a guide fee and an interpreter fee from the user. Then, the billing unit 1B performs a process for passing the fee obtained by subtracting the fee from the guide fee received from the user to the guide. In addition, the billing unit 1B performs a process for passing the fee, which is obtained by subtracting the fee from the interpreter fee received from the user, to the interpreter.

The guide terminal 20 is used by a local guide, for example, a smartphone. In FIG. 1, only one guide terminal 20 is drawn, but these may be as many as the number of guides.

The guide terminal 20 has a guide information setting unit 21 and a guide request processing unit 22. At least some of the functions of each of these parts may be realized by the processor executing a predetermined program.

The guide information setting unit 21 receives the input of the above-mentioned guide conditions from the guide via the input interface (not shown) of the guide terminal 20. Then, the guide information setting unit 21 transmits the guide condition to the virtual travel providing device 10, so that the guide information is stored in the guide information storage unit 12 of the virtual travel providing device 10. Further, the guide information setting unit 21 may receive information from the guide that identifies the model of the guide self-introduction and the available guide data acquisition device 30, and may transmit the information to the virtual travel providing device 10.

The guide request processing unit 22 receives a notification from the virtual travel providing device 10 that the guide request has been made by the user. The guide request may include user information (gender, age, etc.). Then, in response to the input from the guide, the guide request processing unit 22 sends an answer for accepting or rejecting the request to the virtual travel providing device 10.

The guide data acquisition device 30 is used by a local guide to acquire local video and audio, and includes a camera 31, a microphone 32, and a local data transmission unit 33. The guide data acquisition device 30 is typically a smart glass (glasses-type wearable device) worn by a guide, but may be a camera 31 and a microphone 32 mounted on a machine such as a drone or a robot. Further, the camera 31 may be a 360-degree camera. Further, the camera 31 and the microphone 32 may be integrated or may be separate devices.

The camera 31 photographs the site. The microphone 32 acquires the local voice. Then, the local data transmission unit 33 transmits the video data obtained by the camera 31 and the audio data obtained by the microphone 32 to the virtual travel providing device 10.

The interpreter terminal 40 is used by the interpreter, and is, for example, a smartphone. In FIG. 1, only one interpreter terminal 40 is drawn, but these may be as many as the number of interpreters.

The interpreter terminal 40 has an interpreter information setting unit 41 and an interpreter request processing unit 42. At least some of the functions of each of these parts may be realized by the processor executing a predetermined program.

The interpreter information setting unit 41 receives the input of the above-mentioned interpreting conditions from the interpreter via the input interface (not shown) of the interpreter terminal 40. Then, the interpreter information setting unit 41 transmits the interpreter conditions to the virtual tour providing device 10, so that the interpreter information is stored in the interpreter information storage unit 15 of the virtual travel providing device 10. Further, the interpreter information setting unit 41 may receive information from the interpreter that identifies the model of the interpreter's self-introduction and the available interpreter data acquisition device 50 and transmit it to the virtual travel providing device 10.

The interpreter request processing unit 42 receives a notification from the virtual travel providing device 10 that the user has requested an interpreter. The interpreter request may include user information (gender, age, etc.). Then, in response to the input from the interpreter, the interpreter request processing unit 42 transmits an answer for accepting or rejecting the request to the virtual travel providing device 10.

The interpreter data acquisition device 50 is used by the interpreter to perform interpretation, and has a local data receiving unit 51, a speaker 52, a microphone 53, and an interpreting data transmitting unit 54. The interpreter data acquisition device 50 may be integrated with the interpreter terminal 40 or may be a separate device.

The local data receiving unit 51 receives voice data including voice to be interpreted from the virtual travel providing device 10. This voice data is based on the voice data from the microphone 32 of the guide data acquisition device 30. The voice corresponding to this voice data is reproduced from the speaker 52. The interpreter interprets the reproduced voice and speaks. Mike 53 picks up the utterance of this interpreter. Then, the interpreter data transmission unit 54 transmits the voice data (interpreted voice data) obtained by the microphone 53 to the virtual travel providing device 10.

The user terminal 60 is used by a virtual travel user, for example, a smartphone. In FIG. 1, only one user terminal 60 is drawn, but these may be as many as the number of users.

The user terminal 60 has a guide condition setting unit 61 and an interpreter condition setting unit 62. At least some of the functions of each of these parts may be realized by the processor executing a predetermined program.

The guide condition setting unit 61 receives the above-mentioned input of the guide condition from the user via the input interface (not shown) of the user terminal 60. Then, the guide condition setting unit 61 transmits the guide condition to the virtual travel providing device 10. Further, the guide condition setting unit 61 may receive information specifying the model of the reproduction device 70 for reproducing the video and audio from the guide and the reproduction device 70, and transmit the information to the virtual travel providing device 10. The reproduction device 70 specified by the user may be simply referred to as "user's reproduction device 70". The video and audio from the guide may be reproduced by a plurality of reproduction devices 70, and in that case, it is necessary to specify the plurality of reproduction devices 70.

The interpretation condition setting unit 62 receives the above-mentioned input of the interpretation condition from the user via the input interface (not shown) of the user terminal 60. Then, the interpreter condition setting unit 62 transmits the interpreter condition to the virtual travel providing device 10.

The reproduction device 70 is used by the user to reproduce local video and audio, and has a reproduction data receiving unit 71, a display 72, and a speaker 73. The user data acquisition device is typically a smart glass or VR goggles (glasses-type wearable device) worn by the user. The VR goggles may have a means (not shown) for acquiring the movement of the face of the wearing user and transmitting the movement to the virtual travel providing device 10.

The reproduction data receiving unit 71 receives reproduction data from the virtual travel providing device 10. Then, the display 72 and the speaker 73 reproduce the video and audio according to the reproduction data, respectively.

Next, the processing operation of the virtual travel providing system will be described. In the present embodiment, an interpreter is not essential, and the user can request an interpreter at any time. Therefore, a case where there is no interpreter will be described first.

FIG. 4 is a sequence diagram showing the processing operation of the virtual travel providing system until the guide is specified.

The guide information setting unit 21 of the guide terminal 20 receives input of guide information from the guide and transmits the input to the virtual travel providing device 10 (step S111). In response to this, the virtual travel providing device 10 receives the guide information and is registered in the guide information storage unit 12 by the guide information registration unit 11 (step S121). It is desirable that the guide information (particularly, the guideable time and the guideable area) can be updated at an arbitrary timing or at a timing designated by the virtual travel providing device 10 according to the convenience of the guide. Further, it is desirable that the guide information storage unit 12 stores the guide information of a large number of guides.

The guide condition setting unit 61 of the user terminal 60 receives a guide condition from a user who desires a virtual trip and transmits the guide condition to the virtual travel providing device 10 (step S131). In response to this, the virtual travel providing device 10 receives the guide condition (step S122).

Subsequently, the guide specifying unit 13 of the virtual travel providing device 10 identifies a guide candidate based on the guide information stored in the guide information storage unit 12 and the guide conditions from the user terminal 60 (step S123). .. As a specific example, the guide specifying unit 13 can use a guide whose guideable time and guideable area in the guide information match the desired date and time of travel and the area desired to travel under the guide conditions as guide candidates. ..

Subsequently, the guide specifying unit 13 transmits a guide request to the specified guide candidate guide terminal 20 (step S124). In response to this, the guide request processing unit 22 of the guide terminal 20 receives the guide request (step S112). Then, the guide request processing unit 22 transmits the response of acceptance or refusal of the guide request received from the guide to the virtual travel providing device 10 (step S113).

If the guide's answer is acceptable, the guide identification unit 13 identifies the guide as a user's guide. Then, the guide specifying unit 13 transmits a notification that the guide has been specified to the user terminal 60 (step S125). The guide information (gender, age, etc.) may be transmitted to the user terminal 60, and the user may decide whether or not the guide is acceptable.

In step S123, there is no guide having guide information satisfying the guide condition, and the guide candidate may not be specified. In addition, the answer in step S133 may be rejected. In these cases, the guide specifying unit 13 sends a notification to the user terminal 60 that the guide has not been found. Then, the user may be urged to change the guide condition.

Further, in step S123, when there are a plurality of guides having guide information satisfying the guide conditions, the guide specifying unit 13 may set priorities for the plurality of guides based on user evaluation or the like and specify a plurality of guide candidates. .. Then, the guide specifying unit 13 may request a guide in order from the guide having the highest priority.

FIG. 5 is a sequence diagram showing the processing operation of the virtual travel providing system for sharing the local video and audio with the user after the guide is specified. The process shown in the figure is performed by connecting the guide data acquisition device 30 and the reproduction device 70 to the virtual travel providing device 10 by, for example, instructing the virtual travel providing device 10 to start a trip via the user terminal 60. ..

The guide identified by the process of FIG. 4 acquires local video and audio by using the camera 31 and the microphone 32 of the guide data acquisition device 30 according to the wish transmitted from the user. Then, the local data transmission unit 33 transmits the video data from the camera 31 and the audio data from the microphone 32 to the virtual travel providing device 10 (step S211).

The user's wishes may be communicated to the guide in advance, or may be communicated from the user during the virtual trip. In the latter case, if the user's voice is picked up by a microphone (not shown) provided in the reproduction device 70 and reproduced by the speaker (not shown) of the guide data acquisition device 30 or the guide terminal 20 via the virtual travel providing device 10. good.

The local data receiving unit 17 of the virtual travel providing device 10 receives video data and audio data from the guide camera 31 and the microphone 32, respectively (step S221). Then, the reproduction data generation unit 19 processes the received video data and audio data as necessary, generates reproduction data for reproducing the video and audio in the user's reproduction device 70, and the reproduction data transmission unit 1A It is transmitted to the playback device 70 of the user (step S222).

The reproduction data generation unit 19 may translate the language included in the audio from the guide data acquisition device 30, and in that case, the reproduction data includes video data for displaying the result in subtitles. Further, when the camera 31 of the guide data acquisition device 30 is a 360-degree camera and the movement of the face of the user wearing the reproduction device 70 is received from the reproduction device 70, the reproduction data generation unit 19 synchronizes with the movement of the face. Playback data including the video data may be generated. For example, when the user turns to the back, video data including the rear portion of the 360-degree video may be generated. Further, the reproduced data may include video data for displaying the guide as an avatar on the display 72.

Then, the reproduction device 70 receives the reproduction data. Then, the display 72 reproduces the video based on the video data in the reproduced data, and the speaker 73 reproduces the audio based on the audio data in the reproduced data (step S231). The reproduced video corresponds to the video data from the guide camera 31, and the reproduced audio corresponds to the audio data from the guide microphone 32.
As described above, the user can share the local video and audio with the guide without going to the site and while staying at a place different from the guide (for example, at home).

You may be able to shop during the virtual trip. As a specific example, the user may make a payment with the playback device 70 or the user terminal 60, and the guide may mail the product to the user.

Next, a virtual trip through an interpreter will be described. The user can request an interpreter at any time. For example, when deciding on a guide, an interpreter may also be decided. Alternatively, an interpreter may be requested by transmitting the interpreter conditions during the virtual tour (that is, when the voice from the guide is received and played back).

FIG. 6 is a sequence diagram showing the processing operation of the virtual travel providing system for identifying the interpreter.

The interpreter information setting unit 41 of the interpreter terminal 40 receives the input of the interpreter information from the interpreter and transmits the input to the virtual travel providing device 10 (step S311). In response to this, the virtual tour providing device 10 receives the interpreter information and is registered in the interpreter information storage unit 15 by the interpreter information registration unit 14 (step S321). It is desirable that the interpreter information (particularly, the interpreter available time) can be updated at an arbitrary timing or at a timing designated by the virtual tour providing device 10 according to the convenience of the interpreter. Further, it is desirable that the interpreter information storage unit 15 stores the interpreter information of a large number of interpreters.

The interpreter condition setting unit 62 of the user terminal 60 receives the interpreter condition from the user who desires the virtual trip and transmits it to the virtual travel providing device 10 (step S331). In response to this, the virtual travel providing device 10 receives the interpreting condition (step S322).

Subsequently, the interpreter identification unit 16 of the virtual travel providing device 10 identifies an interpreter candidate based on the interpreter information stored in the interpreter information storage unit 15 and the interpreter conditions from the user terminal 60. (Step S323). As a specific example, the interpreter identification unit 16 can select an interpreter whose interpreter available time in the interpreter information matches the desired date and time in the interpreter conditions as an interpreter candidate. Further, the interpreter identification unit 16 may specify an interpreter candidate based on the languages that can be used by the specified guide, the languages that can be used in the interpreter information, and the desired language in the interpreter conditions. For example, if the language in which the guide can be used is Spanish and the language desired by the user is Japanese, an interpreter who can use Spanish and Japanese is a candidate.

Subsequently, the interpreter specifying unit 16 transmits an interpreter request to the identified interpreter candidate terminal 40 (step S324). In response to this, the interpreter request processing unit 42 of the interpreter terminal 40 receives the interpreter request (step S312). The interpreter request may include information indicating how many words (language used by the guide) should be translated into what language (language desired by the user). Then, the interpreter request processing unit 42 sends an answer to the acceptance or refusal of the interpreter request received from the interpreter to the virtual travel providing device 10 (step S313).

If the interpreter's answer is acceptable, the interpreter identification unit 16 identifies the interpreter as the user's interpreter. Then, the interpreter identification unit 16 transmits a notification that the interpreter has been identified to the user terminal 60 (step S325). Information on the interpreter (gender, age, etc.) may be transmitted to the user terminal 60, and the user may decide whether or not the interpreter is acceptable.

In addition, there may be no interpreter having interpreter information satisfying the interpreting condition in step S323, and the interpreter candidate may not be specified. In addition, the answer in step S313 may be rejected. In these cases, the interpreter identification unit 16 sends a notification to the user terminal 60 that the interpreter has not been found. Then, the user may be urged to change the interpretation conditions.

Further, in step S323, when there are a plurality of interpreters having interpreter information satisfying the interpreting conditions, the interpreter identification unit 16 sets priorities for the plurality of interpreters based on user evaluation and the like, and a plurality of interpreter candidates. May be specified. Then, the interpreter specifying unit 16 may request interpretation in order from the interpreter having the highest priority.

FIG. 7 is a sequence diagram showing a processing operation of a virtual travel providing system for sharing local video and audio with a user when there is an interpreter. Hereinafter, the common points with FIG. 5 will be omitted.

The guide identified by the process of FIG. 4 acquires local video and audio by using the camera 31 and the microphone 32 of the guide data acquisition device 30 according to the wish transmitted from the user. The voice may include the utterance of the guide. Then, the local data transmission unit 33 transmits the video data from the camera 31 and the audio data from the microphone 32 to the virtual travel providing device 10 (step S411).

In response to this, the local data receiving unit 17 of the virtual travel providing device 10 receives the video data and the audio data from the guide data acquisition device 30 (step S421).

Here, unlike FIG. 7, it is necessary to transmit the voice of the guide to the interpreter as well. Therefore, the reproduction data generation unit 19 generates audio data to be transmitted to the interpreter based on the audio data from the guide data acquisition device 30, and the reproduction data transmission unit 1A transmits this to the interpreter terminal 40 (the reproduction data transmission unit 1A). Step S422). This voice data may include the utterance of the guide.

In response to this, the interpreter data receiving unit 18 of the interpreter data acquisition device 50 receives the voice data, and the speaker 52 reproduces the voice data (step S431). While listening to the voice reproduced by the speaker 52, the interpreter interprets the utterance of the guide included in the voice. The microphone 53 of the interpreter data acquisition device 50 picks up the speech of the interpreter. Then, the voice data (interpreted voice data) from the microphone 53 is transmitted to the virtual travel providing device 10 by the interpreter data transmission unit 54 (step S432).

This voice data is received by the virtual travel providing device 10 (step S423). Then, the reproduction data generation unit 19 processes the video data and the audio data received from the guide data acquisition device 30 and the audio data received from the interpretation data acquisition device 50 as necessary, and the reproduction device 70 of the user. Generates reproduction data for reproducing video and audio, and transmits the reproduction data to the user's reproduction device 70 (step S424). The reproduced data may include video data for displaying the interpreter as an avatar on the display 72.

Then, the reproduction device 70 receives the reproduction data, and the display 72 and the speaker 73 reproduce it (step S441). As a result, the local video and audio are shared with the user. Moreover, the reproduced audio includes audio translated into the desired language by the interpreter.

According to the above, the interpreter receives the voice of the guide and transmits the result of the interpretation to the user via the virtual travel providing device 10. Therefore, the interpreter does not need to accompany the guide and does not need to accompany the user.

As described above, in the virtual trip in the present embodiment, since the user does not need to go to the site, it is easier and easier than the actual trip, and the trip can be experienced at a low cost and time. In addition, with virtual travel, there are few concerns about accidents, troubles, and security in real travel (especially overseas travel). In addition, if there is an interpreter, the guide does not need language skills, so it is possible to create employment for the guide. Furthermore, since the interpreter does not need to accompany the user or guide, the interpreter can perform the interpretation at any place such as at home, and the employment of the interpreter can be created.

The above-described embodiments have been described for the purpose of allowing a person having ordinary knowledge in the technical field to which the present invention belongs to carry out the present invention. Various modifications of the above embodiment can be naturally made by those skilled in the art, and the technical idea of the present invention can be applied to other embodiments. Accordingly, the invention is not limited to the described embodiments and should be the broadest scope according to the technical ideas defined by the claims.

10 Virtual travel provider 11 Guide information registration unit 12 Guide information storage unit 13 Guide identification unit 14 Interpreter information registration unit 15 Interpreter information storage unit 16 Interpreter identification unit 17 Local data reception unit 18 Interpretation data reception unit 19 Playback data generation Unit 1A Playback data transmission unit 1B Billing unit 20 Guide terminal 21 Guide information setting unit 22 Guide request processing unit 30 Guide data acquisition device 31 Camera 32 Mike 33 Local data transmission unit 40 Interpreter terminal 41 Interpreter information setting unit 42 Interpreter Request processing unit 50 Interpretation data acquisition device 51 Local data reception unit 52 Speaker 53 Microphone 54 Interpretation data transmission unit 60 User terminal 61 Guide condition setting unit 62 Interpretation condition setting unit 70 Playback device 71 Playback data reception unit 72 Display 73 Speaker

Claims (13)

Guide information registration step to register guide information for each of multiple guides,
A guide information receiving step that receives a guide condition from the user,
A guide specifying step for specifying one of the plurality of guides based on the guide conditions and the registered guide information.
A local data receiving step for receiving the first video data from the camera of the identified guide and the first audio data from the microphone of the identified guide.
The second video data for reproducing the video corresponding to the first video data on the display in the glasses-type wearable device specified by the user is generated based on the first video data, and the second video data is generated according to the first audio data. A reproduction data generation step for generating a second audio data for reproducing the audio on the speaker of the glasses-type wearable device based on the first audio data, and a reproduction data generation step.
A reproduction data transmission step of transmitting the second video data and the second audio data to the glasses-type wearable device, and
A virtual travel providing method comprising a reproduction step of reproducing an image based on the second video data on the display of the glasses-type wearable device and reproducing the sound based on the second audio data on the speaker of the glasses-type wearable device. .. A guide information registration unit that registers guide information for each of multiple guides,
A guide specifying unit that identifies one of the plurality of guides based on the guide conditions from the user and the registered guide information.
A local data receiving unit that receives the first video data from the camera of the specified guide and the first audio data from the microphone of the specified guide.
The second video data for reproducing the video corresponding to the first video data on the user's display is generated based on the first video data, and the voice corresponding to the first voice data is generated by the user's speaker. A virtual travel providing device including a reproduction data generation unit that generates second audio data for reproduction in the first audio data based on the first audio data. The virtual data according to claim 2, wherein the reproduction data generation unit generates the second voice data including translation data in which at least a part of the utterance of the guide included in the first voice data is translated into another language. Travel provider. The interpreter information registration department, which registers interpreter information for each of multiple interpreters,
An interpreter identification unit that identifies any of the plurality of interpreters based on the interpreter conditions from the user and the registered interpreter information is provided.
The reproduction data generation unit generates interpreter voice data for reproducing the voice corresponding to the first voice data on the speaker of the specified interpreter based on the first voice data.
The interpreter data receiving unit for receiving the translated voice data from the microphone of the specified interpreter is provided.
The reproduction data generation unit generates the first voice data and the second voice data for reproducing the voice corresponding to the translated voice data based on the first voice data and the translated voice data. The virtual travel providing device according to claim 2 or 3. The guide information includes information indicating a guideable first time and a guideable first area.
The guide condition includes information indicating a second time for which a guide is desired and a second area for which a guide is desired.
The virtual guide according to any one of claims 2 to 4, wherein the guide specifying unit specifies a guide by matching the first time with the second time and matching the first area with the second area. Travel provider. The virtual travel providing device according to any one of claims 2 to 5, wherein the user's display and speaker are provided in a glasses-type wearable device. Computer,
A guide information registration unit that registers guide information for each of multiple guides,
A guide specifying unit that identifies one of the plurality of guides based on the guide conditions from the user and the registered guide information.
A local data receiving unit that receives the first video data from the camera of the specified guide and the first audio data from the microphone of the specified guide.
The second video data for reproducing the video corresponding to the first video data on the user's display is generated based on the first video data, and the voice corresponding to the first voice data is generated by the user's speaker. A virtual travel providing program that functions as a reproduction data generation unit that generates second audio data for reproduction in the first audio data based on the first audio data. Guide information registration step to register guide information for each of multiple guides,
A guide specifying step for specifying one of the plurality of guides based on the guide conditions from the user and the registered guide information.
A local data receiving step for receiving the first video data from the camera of the identified guide and the first audio data from the microphone of the identified guide.
The second video data for reproducing the video corresponding to the first video data on the user's display is generated based on the first video data, and the voice corresponding to the first voice data is generated by the user's speaker. A virtual travel providing method comprising a reproduction data generation step of generating a second audio data for reproduction in the above-mentioned first audio data based on the first audio data. A display that reproduces video based on video data from the camera of the guide specified based on the guide conditions from the user, guide information for each of the multiple guides, and
A virtual travel playback device comprising a speaker that reproduces audio based on audio data from the microphone of the identified guide. A guide condition reception step that accepts a guide condition for specifying a guide from the user,
The video based on the video data from the camera of the guide specified based on the guide condition and the guide information for each of the plurality of guides is played back on the user's display and from the microphone of the specified guide. A virtual travel playback method comprising a playback step of playing back a voice based on the voice data of the user on the speaker of the user. A guide data acquisition device capable of communicating with the virtual travel providing device according to any one of claims 2 to 6 and used by a guide.
With the camera
With Mike
A local data transmission unit that transmits video data from the camera and audio data from the microphone to the virtual tour providing device according to any one of claims 2 to 6 as the first video data and the first audio data, respectively. , A guide data acquisition device. Steps to receive interpreter information from the interpreter,
A step of receiving interpretation voice data based on the voice data from the guide's microphone when an interpretation request is received based on the interpretation conditions from the user and the interpreter information.
The step of playing back the voice corresponding to the voice data for interpretation on the speaker,
An interpreting method comprising a step of transmitting translated voice data picked up by the interpreter's microphone from the voice interpreted by the interpreter based on the reproduced voice. The virtual travel providing device according to claim 4 and
User terminals used by virtual travel users and
The virtual travel reproduction device according to claim 9 and
The guide terminal used by the guide and
The guide data acquisition device according to claim 11 and
Interpreter terminals used by interpreters and
Equipped with an interpreter data acquisition device used by the interpreter,
The user terminal has a guide condition setting unit that receives a guide condition for specifying a guide from the user and transmits the guide condition to the virtual travel providing device.
The guide terminal has a guide information setting unit that receives guide information from the guide and transmits it to the virtual travel providing device.
The interpreter terminal has an interpreter information setting unit that receives interpreter information from the interpreter and transmits it to the virtual travel providing device.
The interpreter data acquisition device is
An interpreter data receiving unit that receives the interpreter voice data from the virtual travel provider, and an interpreter data receiver.
A speaker that reproduces voice according to the voice data for interpretation,
Mike picking up the interpreter's utterance,
A virtual tour providing system including an interpreter data transmitting unit that transmits voice data from the microphone to the virtual travel providing device as the interpreted voice data.

Images