JP2023092929A - Communication system, communication method, and program - Google Patents

Abstract

To convey to a customer an atmosphere of a place where a trip or the like is planned, with an enhanced sense of realism, in order to solve the problem in which: compared to face-to-face customer services, remote customer services lack in excitement of customers when making travel plans; in a case of a conventional face-to-face customer service, a customer and an explainer can plan a trip while sharing materials of travel pamphlets and exchanging opinions; during this step, listening directly to experience of the explainer livens up the conversation and increases the customer's expectations for the trip; on the other hand, even the remote customer service involves sharing electronic data of travel pamphlets through screen sharing while planning a trip, however, because it is not the face-to-face customer service, it is difficult to raise the customer's expectations for the trip.SOLUTION: According to an instruction from an explainer terminal side, a personal image of the explainer is displayed in a main image column 140, a personal image of a customer is displayed in a sub image column 150, and images of landscapes and the like of a travel destination are displayed in each column 140 and 150 as a background image.SELECTED DRAWING: Figure 33

Description

The present disclosure relates to communication systems, communication methods, and programs.

In recent years, remote concierge (remote customer service) services have been provided from the viewpoint of streamlining and efficiency of store operations. As a result, for example, even if the customer (guest) has to wait in line for customer service because an employee in the store is serving customers, the customer (guest) can access the store terminal via the Internet, etc. By accessing the explainer terminal of the explainer (concierge) who has specialized skills in a remote service center and making a video call with the explainer, it is possible to receive customer service (see Patent Document 1).

In recent years, remote customer service has been introduced to travel agencies. With the introduction of this service, customers (guests) can receive customer service without having to wait at the store, and it has begun to spread as a service that prevents loss of customer acquisition opportunities for travel agencies.

JP 2008-154059 A

However, some problems have also emerged. Compared to face-to-face customer service, remote customer service has the problem that it lacks excitement for customers (guests) when planning travel plans. In the case of the conventional face-to-face service, the customer and the presenter can plan the trip while sharing the materials of the travel pamphlet and exchanging opinions. At that time, listening directly to the experience of the explainer will make the conversation lively and increase the customer's expectations for the trip. On the other hand, even remote customer service involves sharing the electronic data of travel brochures by sharing screens while formulating travel plans, but because it is not a face-to-face format, it is difficult to raise customer expectations for travel. Therefore, when performing a video call, there arises a problem that it is difficult to liven up the conversation compared to the face-to-face format.

In the invention according to claim 1, communication mediating the image data of the subject at the first base so that the second communication terminal receives the image data of the subject at the first base transmitted by the first communication terminal. a management device comprising a receiving means for receiving a transmission request for a specific background image transmitted by the first communication terminal; display the specific background image as a background image for the subject at the base, and display video data of the subject at the second base photographed by the second communication terminal while the second communication terminal is displaying the second transmitting means for transmitting data of the specific background image to the second communication terminal based on the transmission request in order to display the specific background image as a background image for a subject at the base of It is a communication management device that

As described above, according to the present invention, even in a video call, each user can share the same background image to make a video call, so that the conversation can be lively.

1 is an overall configuration diagram of a communication system according to an embodiment; FIG. FIG. 10 is an image diagram showing a situation in which a customer in a store and an explainer in a service center are having a remote video call; 3 is a hardware configuration diagram of each terminal and each device that constructs the communication system of the present embodiment; FIG. It is each functional block diagram of the communication system concerning this embodiment. It is each functional block diagram of the communication system concerning this embodiment. (A) is a conceptual diagram showing an explainer management table, and (B) is a conceptual diagram showing a skill group management table. (A) is a conceptual diagram showing an authentication management table, and (B) is a conceptual diagram showing a terminal state management table. (A) is a conceptual diagram showing a destination list management table, (B) is a conceptual diagram showing a session management table, and (C) is a conceptual diagram showing a communication information management table. (A) is a conceptual diagram of a system update information management table, (B) is a conceptual diagram of a provider management table, and (C) is a conceptual diagram of an area management table. (A) is a conceptual diagram of a terminal management table, and (B) is a conceptual diagram of a skill management table. (A) is a conceptual diagram of a login information management table, (B) is a conceptual diagram of a subgroup management table, and (C) is a conceptual diagram of a user update information management table. FIG. 10 is a sequence diagram showing service information registration processing; It is a figure which shows the login screen displayed on an administrator terminal. It is a figure which shows the store list screen displayed on an administrator terminal. It is a figure which shows the presenter list screen displayed on an administrator terminal. It is a figure which shows the group list screen displayed on an administrator terminal. FIG. 10 is a diagram showing an explainer information edit screen displayed on an administrator terminal; FIG. 10 is a diagram showing a group information edit screen displayed on the administrator terminal; It is a figure which shows the shop information edit screen displayed on an administrator terminal. FIG. 10 is a sequence diagram showing preparation processing for a video call of a store terminal, and particularly showing processing for acquiring service information; FIG. 10 is a sequence diagram showing preparation processing of a store terminal for a video call, and particularly showing processing for acquiring an operating state of each terminal; FIG. 10 is a flow chart showing preparation processing of a store terminal for a video call, and particularly showing processing for determining the operating state of a group of explainer terminals for each skill; FIG. It is a figure which shows the screen displayed on a shop terminal. FIG. 10 is a sequence diagram showing processing for starting a video call; FIG. 10 is a sequence diagram showing detailed processing of communication establishment control; 4 is a flow chart showing various processes for starting a call; It is an example of a call screen displayed on the explainer terminal. It is an example of the call screen displayed on the store terminal. FIG. 10 is a sequence diagram showing processing in which the explainer terminal and the store terminal activate a video call from a dedicated application and create a travel plan; It is an example of a call screen (including an input window and a map window) displayed on the explainer terminal. It is an example of a call screen (input window, map window) displayed on the explainer terminal. It is an example of a call screen (including an input window, a map window, and an image of a tourist spot) displayed on the explainer terminal. It is an example of a call screen (including an image of a tourist spot) displayed on the store terminal. It is an example of a call screen (including an input window, a map window, an image of the appearance of an accommodation facility, and a price list) displayed on the explainer terminal. It is an example of a call screen (including an image of the appearance of an accommodation facility and a price list) displayed on a store terminal. It is an example of a call screen (including an input window, a map window, an image of a room, and a price list) displayed on the explainer terminal. It is an example of the call screen (including the image of the room and the price list) displayed on the store terminal.

[Overall Configuration of Communication System]
The overall configuration of the communication system will be described with reference to FIG. FIG. 1 is an overall configuration diagram of a communication system according to an embodiment of the present invention. Here, a customer, a travel agency that sells travel products to the customer, and a system provider that provides the travel agency with a system for remote concierge (remote customer service) appear.

A travel agency operates manned shops, unmanned shops, and service centers for each of a plurality of areas such as Kanto and Kansai. FIG. 1 shows that the travel agency operates a manned store MS, an unmanned store US, and a service center SC1 in area E1. A manned store MS is a store where employees who serve customers are registered. An unmanned store is a store that does not have employees who serve customers. Although only one manned store MS and one unmanned store US are shown in FIG. 1, there may be a plurality of each.

At manned stores MS, employees usually guide customers about travel products, but there are cases where the employees are busy serving customers and cannot serve new customers. Moreover, in the unmanned store US, there is no employee who responds to customers in the first place. In such a case, the customer uses the store terminal 1c1 in the manned store MS or the store terminal 1c2 in the unmanned store US, and uses the communication network 100 such as the Internet to communicate with any explainer in the service center SC1. By accessing the terminal, it is possible to receive remote customer service regarding travel products from the presenter. The explainer is, for example, the remote concierge described above. Since the explainer basically uses his or her own explainer terminal, the specific explainer and the specific explainer terminal are linked.

Each of the store terminals 1c1 and 1c2 is configured by a PC (personal computer). A plurality of store terminals may be installed in the manned store MS and the unmanned store US. Hereinafter, a generic term for a plurality of store terminals such as store terminals 1c1 and 1c2 will be referred to as "store terminal 1".

The service center SC1 includes a manned store priority support organization MG1 for preferentially receiving a call start request for starting remote customer service from the store terminal 1c1 of the manned store MS, and a service center MG1 for preferentially receiving calls from the store terminal 1c2 of the unmanned store US. There is an unmanned shop priority support organization UG1 for preferentially receiving a call start request for starting remote customer service. Note that there may be three or more priority response organizations.

Further, the manned store priority response organization MG1 includes an organization SR11 in charge of skill (skill) SK1 and an organization SR12 in charge of skill (skill) SK2. For example, skill SK1 is an explainer's ability to make proposals regarding domestic travel, and skill SK2 is an explainer's ability to make proposals regarding overseas travel. Similarly, the unmanned store priority support organization UG1 includes an organization SR21 in charge of skill SK1 and an organization SR22 in charge of skill SK2. That is, organizations SR11 and SR21 (or organizations SR12 and SR22) in which explainers having the same skill are enrolled exist in the manned store priority organization MG1 and the unmanned store priority organization UG1. It should be noted that there may be three or more skills and organizations in charge of skills.

Furthermore, in the manned store priority support organization MG1, explainers mg11 and mg12 who have the skill SK1 are enrolled in the organization SR11 in charge of the skill SK1. The explainer mg11 uses the explainer terminal 9mg11, and the explainer mg12 uses the explainer terminal 9mg12.

Also, explainers mg21 and mg22 who have skill SK2 are enrolled in organization SR12 in charge of skill SK2. The explainer mg21 uses the explainer terminal 9mg21, and the explainer mg22 uses the explainer terminal 9mg22.

Similarly, in the unmanned store priority support organization UG1, explainers ug11 and ug12 who have the skill SK1 are enrolled in the organization SR21 in charge of the skill SK1. The explainer ug11 uses the explainer terminal 9ug11, and the explainer ug12 uses the explainer terminal 9ug12. Also, explainers ug21 and ug22 who have skill SK2 are enrolled in organization SR22 in charge of skill SK2. The explainer ug21 uses the explainer terminal 9ug21, and the explainer ug22 uses the explainer terminal 9ug22.

Three or more presenters may be enrolled in each responsible organization (SR11, SR12, SR21, SR22). In this case, an explainer terminal is assigned to each of three or more explainers. Further, hereinafter, when a plurality of explainer terminals such as explainer terminals 9mg11, 9mg12, 9mg21, 9mg22, 9ug12, 9ug12, 9ug21, and 9ug22 are collectively referred to as "explainer terminal 9".

In addition, the travel agency has an administrator ad1 who manages terminals in each service area. The administrator ad1 uses the administrator terminal 4 to access the service information management device 7 via the communication network 100, and sets various service information. This service information is information on the processing and settings of each terminal of the travel agency.
The administrator terminal 4 is configured by a PC.

A customer c1 and a customer c2 are those who visited the manned store MS and the unmanned store US, respectively.

The system provider manages and operates the relay device 3 , the communication management device 5 and the service information management device 7 . The relay device 3, the communication management device 5, and the service information management device 7 are computers such as servers. The relay device 3 relays image data and sound data when a video call is made between one of the store terminals 1 and one of the explainer terminals 9 . The communication management device 5 centrally manages login authentication from the store terminal 1 and the explainer terminal 9, management of the communication status of the store terminal 1 and the explainer terminal 9, management of the destination list, communication status of the relay device 30, and the like. to manage. The service information management device 7 manages various set service information. The relay device 3, the communication management device 5, and the service information management device 7 will be described later in detail.

Also, the store terminal 1, manager terminal 4, explainer terminal 9, relay device 3, communication management device 5, and service information management device 7 can be connected to a communication network 100 such as the Internet.

The relay device 3, the communication management device 5, and the service information management device 7 may be constructed by a single computer, or constructed by a plurality of computers in which each part (function or means) is divided and arbitrarily assigned. may have been

FIG. 2 is an image diagram showing a situation in which a customer in a store and an explainer in a service center are having a remote video call. FIG. 2 shows a state in which a customer c2 visiting an unmanned store US uses a store terminal 1c2, and an explainer ug11 of an unmanned store priority support organization UG1 uses an explainer terminal 9ug11. In this way, the customer c2 can receive an explanation of travel products through remote customer service.

[Each hardware configuration of the communication system]
3, each terminal (store terminal 1, administrator terminal 4, explainer terminal 9 and each device (relay device 3, communication management device 5, The hardware configuration of the service information management device 7) will be described.In this embodiment, the hardware configuration of the store terminal 1 will be described on the assumption that each terminal and each device have the same configuration, and other terminals and devices will be described. description of the hardware configuration of is omitted.

As shown in FIG. 3, the store terminal 1 is composed of a computer, and includes a CPU 101, a ROM 102, a RAM 103, an HD 104, a HDD (Hard Disk Drive) controller 105, a display 106, an external device connection I/F (Interface ) 108 , a network I/F 109 , a data bus 110 , a keyboard 111 , a pointing device 112 , a DVD-RW (Digital Versatile Disk Rewritable) drive 114 , and a media I/F 116 .

Among these, the CPU 101 controls the operation of the entire computer. The ROM 102 stores programs used to drive the CPU 101 such as IPL. A RAM 103 is used as a work area for the CPU 101 . The HD 104 stores various data such as programs. The HDD controller 105 controls reading or writing of various data to/from the HD 104 under the control of the CPU 101 . The display 106 displays various information such as cursors, menus, windows, characters, and images. The external device connection I/F 108 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like. A network I/F 109 is an interface for data communication using the communication network 100 . A bus line 110 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 101 shown in FIG.

Also, the keyboard 111 is a kind of input means having a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 112 is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. A DVD-RW drive 114 controls reading or writing of various data to a DVD-RW 113 as an example of a removable recording medium. It should be noted that not only DVD-RW but also DVD-R, Blu-ray Disc (Blu-ray Disc), etc. may be used. A media I/F 116 controls reading or writing (storage) of data to a recording medium 115 such as a flash memory.

A microphone 121 as an example of a sound collector, a speaker 122 as an example of a sound output device, and a camera 123 as an example of an imaging device are connected to the external device connection I/F. In addition, although the microphone 121, the speaker 122, and the camera 123 are shown as external devices in FIG.

[Each function configuration of the communication system]
Next, each functional configuration of the communication system according to this embodiment will be described with reference to FIGS. 3 to 11. FIG. 4 and 5 are functional block diagrams of the communication system according to this embodiment.

<Functional configuration of store terminal>
First, the functional configuration of the store terminal 1 will be described with reference to FIGS. 3 and 4. FIG. As shown in FIG. 4 , the store terminal 1 has a transmission/reception section 11 , reception section 12 , communication control section 13 , display control section 14 , determination section 15 , and storage/read processing section 19 . Each of these units is a function realized by any of the components shown in FIG. be. The store terminal 1 also has a storage unit 1000 constructed by the RAM 103 and HD 104 shown in FIG. In the storage unit 1000, an explainer management DB 1001 and a skill group management DB 1002 are constructed. Each piece of information managed by the presenter management DB 1001 and the skill group management DB 1002 is included in the client information sent from the service information management device 7 in step S19, which will be described later. The shop terminal 1 uses each information managed by the explainer management DB 1001 and the skill group management DB 1002 when displaying a screen shown in FIG. 23, which will be described later.

(explainer management table)
FIG. 6A is a conceptual diagram of the explainer management table. In the storage unit 1000, an explainer management DB 1001 configured by an explainer management table as shown in FIG. 6A is constructed. In this explainer management table, various types of information (communication ID, explainer name, operation status (presence) of the destination candidate, and description of the exclusive use of the explainer terminal) of the destination candidate of the store terminal 1 (in this case, the explainer terminal 9) A skill name indicating a person's skill is associated and managed.

A communication ID is identification information for specifying a communication destination in the communication system 1 . The communication ID is not particularly limited, but includes, for example, a user ID for identifying the explainer who uses the explainer terminal 9, a communication ID for identifying the store terminal, and the like. The explainer's name in FIG. 6(A) is the explainer's name shown in FIG. 23(C), which will be described later. The operating state indicates the operating state of the terminal of the other party of the video call. There are three types of operating states: Online (can talk), Chat (during a call), and Offline (cannot talk). The operating state is reflected in the displays of FIGS. 23(A) and 23(C). The skill indicates the name of the skill possessed by the presenter, as described above. As shown in FIG. 6A, one presenter may have multiple skills (Specialty 1, Specialty 2).

(Skill group management table)
FIG. 6B is a conceptual diagram showing a skill group management table. A skill group management DB 1002 is constructed in the storage unit 1000 using a skill group management table as shown in FIG. 6B. In this skill group management table, skill names, operating states (presence) of skill groups, communication IDs of explainer terminals that can be communicated with store terminals, and communication IDs of explainer terminals that are in communication are associated and managed.

The operating state of a skill group indicates the overall operating state of each explainer terminal of a group of explainers having the same skill such as specialty 1. FIG. For example, if there are multiple explainer terminals of a group of explainers who have the same skill, and even one of these explainer terminals is online, the operating status of the specialty 1 skill group is managed as online. be done. In addition, even if none of the terminals of a plurality of explainers are online, if the operating status of even one of these terminals is Chat, the operating status of the specialty 1 skill group is Chat. managed as Furthermore, if the operating statuses of all of the plurality of presenter terminals are neither Online nor Chat, that is, if all the operating statuses are Offline, the operating status of the specialty 1 skill group is managed as Offline.

(Each functional configuration of the store terminal)
Next, each component of the store terminal 1 will be described. The transmission/reception unit 11 is realized by commands from the CPU 101 shown in FIG. or information).

The reception unit 12 is mainly implemented by commands from the CPU 101, the keyboard 111 and the pointing device 112 shown in FIG. 3, and receives various inputs from the user.

The communication control unit 13 is realized by commands from the CPU 101 shown in FIG. input. Further, the communication control unit 13 inputs sound data obtained by collecting sounds from an external microphone connected to the external device connection I/F 108 . Furthermore, the communication control unit 13 outputs the image data received from the other party of the video call and the image data obtained by the own terminal to the display control unit 14 . Further, the communication control unit 13 outputs sound data received from the other party of the video call and sound data obtained by its own terminal to an external speaker connected to the external device connection I/F 108 .

The display control unit 14 is mainly implemented by commands from the CPU 101 shown in FIG. 3, and outputs image data to the display 106 or an external display connected to the external device connection I/F. to display the image.

The determination unit 15 is mainly implemented by commands from the CPU 101 shown in FIG. 3, and performs various determinations. Details of the determination will be described later.

The storage/read processing unit 19 is mainly executed by commands from the CPU 101 and the HDD controller 105 shown in FIG. and perform processing such as reading

Further, in the storage unit 1000, image data and sound data received during communication with another terminal are overwritten and stored each time they are received. Among them, an image is displayed on the display 106 or an external display by the image data before being overwritten, and sound is output from the external speaker by the sound data before being overwritten.

<Administrator terminal>
Next, the functional configuration of the administrator terminal 4 will be described with reference to FIGS. 3 and 4. FIG. As shown in FIG. 4 , the administrator terminal 4 has a transmission/reception section 41 , a reception section 42 , a display control section 44 and a storage/read processing section 49 . The administrator terminal 9 also has a storage unit 4000 constructed by the RAM 103 and HD 104 shown in FIG.

The transmission/reception unit 41, the reception unit 42, the display control unit 44, and the storage/read processing unit 49 of the administrator terminal 4 correspond to the transmission/reception unit 11, the reception unit 12, the display control unit 14, and the storage unit 11 of the store terminal 1, respectively.・Since it has the same function as the reading processing unit 49, the description thereof will be omitted.

<Functional configuration of explainer terminal>
Next, the functional configuration of the explainer terminal 9 will be described with reference to FIGS. 3 and 4. FIG. As shown in FIG. 4, the explainer terminal 9 has a transmission/reception section 91, a reception section 92, a communication control section 93, a display control section 94, a determination section 95, and a storage/read processing section 99. . The presenter terminal 9 also has a storage unit 9000 constructed by the RAM 103 and HD 104 shown in FIG. A explainer management DB 9001 and a skill group management DB 9002 are constructed in the storage unit 9000 .

The transmitting/receiving unit 91, receiving unit 92, communication control unit 93, display control unit 94, determining unit 95, and storage/reading processing unit 99 of the presenter terminal 9 correspond to the transmitting/receiving unit 11 and receiving unit of the shop terminal 1, respectively. 12, the communication control unit 13, the display control unit 14, the determination unit 15, and the storage/reading processing unit 19, and therefore the description thereof will be omitted. Also, the explainer management DB 9001 and the skill group management DB 9002 of the explainer terminal 9 have the same data structure as the explainer management DB 1001 and the skill group management DB 1002 of the store terminal 1, and thus description thereof will be omitted.

<Functional configuration of relay device>
Next, the functional configuration of the relay device 3 will be described with reference to FIGS. 3 and 5. FIG. As shown in FIG. 5, the relay device 3 has a transmitting/receiving section 31 that also functions as a transfer section, a determining section 32, and a storage/read processing section 39. FIG. Each of these units is a function or a means of functioning realized by any one of the components shown in FIG. . The relay device 3 also has a storage unit 3000 constructed by the RAM 103 and HD 104 shown in FIG.

(Each functional configuration of the relay device)
Next, each functional configuration of the relay device 3 will be described in detail. The transmitting/receiving unit 31 of the relay device 3 shown in FIG. 5 is realized by commands from the CPU 101 shown in FIG. 3 and the network I/F 109 shown in FIG. It transmits and receives various data (or information) to and from other terminals, devices, or systems. The transmitting/receiving section 31 also serves as a transfer section, and transfers image data and sound data transmitted from a predetermined terminal to another terminal.

Judgment unit 32 is realized by a command from CPU 101 shown in FIG. 3, and judges the delay state of data.

The storage/read processing unit 39 is implemented by commands from the CPU 101 shown in FIG. 3 and the HDD controller 105 shown in FIG. It performs processing such as reading out various stored data.

<Functional configuration of communication management device>
Next, the functional configuration of the communication management device 5 will be described with reference to FIGS. 3 and 5. FIG. As shown in FIG. 5 , the communication management device 5 has a transmission/reception section 51 , a selection section 53 , a creation section 54 , a determination section 55 and a storage/read processing section 59 . Each of these units is a function or a means of functioning realized by any one of the components shown in FIG. . The communication management device 5 also has a storage unit 5000 constructed by the RAM 103 and HD 104 shown in FIG.

(Authentication management table)
FIG. 6A is a conceptual diagram showing an authentication management table. An authentication management DB 5001 configured by an authentication management table as shown in FIG. 6A is built in the storage unit 5000 . In this authentication management table, each communication ID of all the terminals 1 and 9 managed by the communication management device 5 is managed in association with each password. A password is an example of authentication information, and an access token is also included in the authentication information.

(Terminal state management table)
FIG. 7B is a conceptual diagram showing a terminal state management table. In the storage unit 5000, a terminal state management DB 5002 configured by a terminal state management table as shown in FIG. 7B is constructed. In this terminal status management table, for each communication ID of each terminal 1, 9, the destination name when each terminal 1, 9 is the destination, the operation status of each terminal 1, 9, and login request information described later are stored in the communication management apparatus. 5 and the IP addresses of the store terminals 1 and 9 are associated and managed. The communication ID, the destination name, and the IP address of the terminal are stored when the terminals 1 and 9 are pre-registered to receive services provided by the communication management device 5 .

(Address list management table)
FIG. 8A is a conceptual diagram showing a destination list management table. A destination list management DB 5003 configured by a destination list management table as shown in FIG. 8A is constructed in the storage unit 5000 . In this destination list management table, all the communication IDs of the destination terminals registered as destination terminal candidates are associated with the communication ID of the originating terminal requesting the start of a call and managed. This destination terminal candidate is updated by being added or deleted in response to an addition or deletion request from the administrator terminal 4 to the communication management device 5 .

Note that the destination list is an example of destination information, and the destination information may not be in a list format and may list information about destinations such as communication IDs.

(session management table)
FIG. 8B is a conceptual diagram showing a session management table. A session management DB 5004 configured by a session management table as shown in FIG. 8B is constructed in the storage unit 5000 . In this session management table, for each communication session ID for identifying a session for mutual communication between each terminal and the relay device 3, the relay device ID of the relay device 3 used, the communication ID of the starting terminal , the communication ID of the destination terminal, the reception delay time (ms) when the image data is received at the destination terminal, and the delay information indicating the delay time are sent from the destination terminal to the communication management device 5. The reception date and time received in is associated and managed.

(Communication information management table)
FIG. 8C is a conceptual diagram showing a communication information management table. A communication information management DB 5005 configured by a communication information management table as shown in FIG. 8C is constructed in the storage unit 5000 . In this communication information management table, for each communication ID for identifying mutual communication performed between each terminal and the relay device 3, the IP address of the relay device of the relay device 3 to be used and the terminal for mutual communication communication IDs are associated with each other and managed.

(Image information management table)
FIG. 8D is a conceptual diagram showing an image information management table. An image information management DB 50006 configured by an image information management table as shown in FIG. 8D is constructed in the storage unit 5000 . In this image information management table, the landmark ID, the file name of the image of the tourist spot, the accommodation facility information, the file name of the image of the exterior of the accommodation facility, the grade (rank) and fee of the room, the room ID, and the image of the room. File names are associated and managed.

Among these, the landmark ID is sightseeing spot identification information for identifying a landmark, which will be described later. The accommodation facility information is accommodation facility identification information for identifying an accommodation facility such as a hotel name. The room ID is room identification information for identifying the grade of the room. Room grade and rate, and room ID are part of the room information. The types of each image data are still image data, moving image data, and omnidirectional image data. In the present embodiment, it is assumed that omnidirectional image data (including still images and moving images) is used as a background image in order to provide a sense of realism as if you were actually there when creating a travel plan.

Here, each data of the image of the sightseeing spot, the image of the appearance of the accommodation facility, and the image of the room is stored in the storage unit 5000 . Note that each data of the image of the tourist spot, the image of the appearance of the lodging facility, and the image of the room are stored in another device (server or the like) instead of the communication management device 5, and the communication management device 5 Each data may be acquired from the device.

(Each functional configuration of the communication management device)
Next, each functional configuration of the communication management device 5 will be described in detail. In describing each functional configuration of the communication management device 5, the main components for realizing each functional configuration of the communication management device 5 among the components shown in FIG. Also explain the relationship.

The transmission/reception unit 51 of the communication management device 5 shown in FIG. 5 is realized by commands from the CPU 101 and the network I/F 109 shown in FIG. Send and receive various data (or information) with the system.

The selection unit 53 is implemented by a command from the CPU 101 shown in FIG. 3, and selects the optimum relay device for mutual communication based on the IP address of each terminal.

The creating unit 54 is implemented by a command from the CPU 101 shown in FIG. 3, and creates communication information in response to a mutual communication start request from each terminal. This communication information includes the IP address of the relay device selected by the selection unit 53, the communication ID described above, and the like.

Determination unit 55 is realized by commands from CPU 101 shown in FIG. 3, and performs various determinations.

The storage/read processing unit 59 is implemented by commands from the CPU 101 and the HDD controller 105 shown in FIG. process.

<Service information management device>
Next, the functional configuration of the service information management device 7 will be described with reference to FIGS. 3 and 5. FIG. As shown in FIG. 5 , the service information management device 7 has a transmission/reception section 71 , a determination section 75 and a storage/read processing section 79 . Each of these units is a function or a means of functioning realized by any one of the components shown in FIG. . The service information management device 7 also has a storage unit 7000 constructed by the RAM 103 and HD 104 shown in FIG. A service information management DB 7001 is constructed in this storage unit 7000 . Each table shown in FIGS. 9 to 11 is constructed in the service information management DB 7001. FIG.

(System update information management table)
FIG. 9A is a conceptual diagram showing a system update information management table. In this system update information management table, a terminal name, the latest version, update application date and time, and a download URL (Uniform Resource Locator) are associated and managed for each terminal ID. In the client name column, the store terminal application is an application for the store terminal 1 and the explainer terminal application is an application for the explainer terminal 9 . Each application is managed by each download URL.

(Business operator management table)
FIG. 9B is a conceptual diagram showing the provider management table. In this operator management table, operator IDs and operator names are associated and managed. Business operators are, for example, travel agencies, leasing companies, and the like.

(group management table)
FIG. 9C is a conceptual diagram showing a group management table. In this group management table, a group name (area name) and a company ID are associated with each group ID (area ID) and managed. A group indicates an area E1 or the like in FIG.

(terminal management table)
FIG. 10A is a conceptual diagram showing a terminal management table. In this terminal management table, destination names, roles, login IDs, passwords, provider IDs, group IDs, and communication IDs are associated and managed for each account ID. Account ID is identification information for identifying a store, an explainer, and an administrator. In the case of a store, the account of the store itself is managed instead of the account of the customer. The destination name is also managed in the terminal state management table of FIG. 7(B), but is the display name shown on the destination selection screen displayed on the store terminal 1 as shown in FIG. 23(C). The role indicates the role of the user of the terminal. The login ID and password are used for authentication when the administrator ad1 accesses the service information management device 7 from the administrator terminal 4. FIG. The "password" of this terminal management table is the same as the "password" of the authentication management table of FIG. 7A, but may be a different password. The communication ID is also managed in the terminal state management table (see FIG. 7A) in the communication management device 5. FIG.

(Skill management table)
FIG. 10B is a conceptual diagram showing a skill management table. In this skill management table, for each skill ID, a skill name and an account ID of the explainer who possesses this skill are associated and managed. As shown in FIG. 10(B), the same presenter (001) can also manage multiple skills such as specialty 1 and specialty 2.

(Login information management table)
FIG. 11A is a conceptual diagram showing a login information management table. In this login information management table, version, account ID, and last login date and time are associated with each reference number ID and managed. The version is the version of the client application (each application of the store terminal 1 and the explainer terminal 9) at the time of login. This allows you to keep track of which account is using which version when an application is updated.

(Subgroup management table)
FIG. 11B is a conceptual diagram showing a subgroup management table. In this subgroup management table, a subgroup name, a group ID, and an account ID of an explainer belonging to the subgroup are associated with each subgroup ID and managed. As shown in FIG. 10B, it is also possible to manage that the same explainer (account ID "001") possesses multiple skills such as specialty 1 and specialty 2. FIG. In the case of the account ID of the shop terminal, as shown in FIG. 1, the subgroups indicate manned shops and unmanned shops. In the case of the explainer's account ID, the subgroup indicates a manned store priority organization or an unmanned store priority organization.

According to the subgroup management table, for example, the store terminal (Tokyo head office No. 1 machine) with an account ID of "003" preferentially As a step prior to the video call start request (see S101 in FIG. 25), the connection request information indicating the connection request for the communication session for the video call is transmitted all at once (first transmission) (described later). (See S84-1 and S85-1 in FIG. 24). Then, if there is no response from the explainer terminals of the explainers with account IDs "001" and "002" within a certain period of time (see S204 in FIG. 25 described later), the store terminal with account ID "003" (Tokyo head office No. 1 machine ) sends a call start request all at once ( second transmission) (see S206 in FIG. 25 to be described later).

(User update information management table)
FIG. 11C is a conceptual diagram showing a user update information management table. In this user update information management table, an account ID to be updated, an update scheduled version, an update application date and time, and a download URL are associated and managed for each update ID.

(Each functional configuration of the service information management device)
Next, each functional configuration of the service information management device 7 will be described in detail. In describing each functional configuration of the service information management device 7, main components for realizing each functional configuration of the service information management device 7 among the components shown in FIG. Also explain the relationship between

The transmission/reception unit 11 of the service information management device 7 shown in FIG. 5 is realized by commands from the CPU 101 shown in FIG. Alternatively, it transmits and receives various data (or information) to and from the system.

Determination unit 75 is realized by commands from CPU 101 shown in FIG. 3, and performs various determinations.

The storage/read processing unit 79 is realized by commands from the CPU 101 and the HDD controller 105 shown in FIG. process.

[Processing or operation of the embodiment]
Next, processing or operations in the communication system according to this embodiment will be described using FIGS. 12 to 26. FIG. In the following, the store terminal 1 as a request source requesting the start of a video call will be referred to as a "start terminal", and the explainer terminal 9 as a destination (relay destination) of the request will be referred to as a "destination terminal". sometimes

<Service information registration process>
First, the process of registering various service information from the administrator terminal 4 to the service information management device 7 will be described with reference to FIGS. 12 to 19. FIG. The service information is managed by each table shown in FIGS. 9 to 11. FIG.

FIG. 12 is a sequence diagram showing service information registration processing. FIG. 13 is a diagram showing a login screen displayed on the administrator terminal. FIG. 14 is a diagram showing a store list screen displayed on the administrator terminal. FIG. 15 is a diagram showing an explainer list screen displayed on the administrator terminal. FIG. 16 is a diagram showing a group list screen displayed on the administrator terminal. FIG. 17 is a diagram showing an explainer information editing screen displayed on the administrator terminal. FIG. 18 is a diagram showing a group information edit screen displayed on the administrator terminal. FIG. 19 is a diagram showing a store information edit screen displayed on the administrator terminal.

As shown in FIG. 12, the administrator ad1 uses the administrator terminal 4 and accesses the service information management device 7 via the communication network 100 to display the web browser (display control unit) of the administrator terminal 4. 44) displays the screens of FIGS. 13 to 19 on the display 106, and the transmitter/receiver 41 transmits each service information input or selected on each screen by the administrator ad1 to the service information management device 7. (S1). In the service information management device 7 , the transmission/reception section 71 receives each service information, and the storage/read processing section 79 stores each service information in the service information management DB 7001 . Thus, each information is managed in each table shown in FIGS. 9 to 11. FIG.

The screens are displayed in the order of FIGS. 13 to 19. FIG. For example, on the group information edit screen shown in FIG. 18, the administrator ad1 assigns the account ID of the explainer of the explainer terminal 9 to whom the call start request is preferentially transmitted to the "explainer ” field. Further, on the store information edit screen shown in FIG. 19, a subgroup name (here, for unmanned store) is entered in the "Priority Called Subgroup" field. As a result, the store terminal 1 obtains the service information from the service information management device 7 (see S19 in FIG. 20 described later), so that the store terminal 1 preferentially selects the explainer terminal 9 to which the call start request should be transmitted. can be managed (see S20).

<Preparation processing for video call>
Next, processing for preparing for a video call for remote customer service will be described with reference to FIGS. 20 to 23. FIG. FIG. 20 is a sequence diagram showing processing for preparing for a call at the store terminal.

First, when a customer visiting the store turns on the power switch of the store terminal 1, the reception unit 12 receives the power ON and activates the store terminal 1 (S11). When the store terminal 1 is activated, the communication application installed in the store terminal 1 is activated (S12). Hereinafter, the processing in the store terminal 1 is executed according to the command of the communication application. Then, the storage/read processing unit 19 reads the local setting information from the storage unit 1000 (S13). As a result, the shop terminal 1 is ready for setting the background image of the display screen, login ID, password, and the like.

Next, the transmitter/receiver 11 of the store terminal 1 transmits a login request to the service information management device 7 via the communication network 100 (S14). This login request includes the login ID and password read in step S13. Thereby, the transmission/reception unit 71 of the service information management device 7 receives the login request. By transmitting a login request from the store terminal 1 to the service information management device 7, the service information management device 7 on the receiving side can acquire the IP address of the store terminal 1 on the transmitting side.

Next, the determination unit 75 of the service information management device 7 searches the terminal management table (FIG. 10A) of the service information management DB 7001 using the login ID and password included in the login request as search keys. Authentication is performed by determining whether the same login ID and password are managed in the terminal management table (S15).

Then, the transmission/reception unit 71 of the service information management device 7 transmits authentication result information indicating the authentication result to the shop terminal 1 (S16). Thereby, the transmitter/receiver 11 of the store terminal 1 receives the authentication result information.

Next, the transmitter/receiver 11 of the store terminal 1 transmits a service information request to the service information management device 7 (S17). As a result, the transmission/reception unit 71 of the service information management device 7 receives the service information request.

Next, in the service information management device 7, the storage/read processing unit 79 reads service information from the service information management DB 7001 (S18). Then, the transmitting/receiving section 71 transmits the service information to the store terminal 1 (S19). Thereby, the transmitting/receiving section 11 of the store terminal 1 receives the service information. Next, in the shop terminal 1, the storage/read processing unit 19 sets service information (S20).

<Start processing of video call>
First, the creating unit 54 of the store terminal 1 creates a session ID for identifying a communication session by itself (S22). Next, the transmitter/receiver 11 transmits login request information indicating a login request to the communication management device 5 via the communication network 100 (S23). This login request information includes the session ID created in step S22 and the communication ID and password of the own terminal in the service information acquired in step S19. Here, the communication ID is the start terminal ID for identifying the store terminal 1 that has transmitted the login request in step S23.

Next, the storage/readout processing unit 59 of the communication management device 5 uses the communication ID and password included in the login request information received via the transmission/reception unit 51 as search keys to search the authentication management table (FIG. 7A). reference) and determines whether the same communication ID and the same password are managed, thereby performing terminal authentication (S24). Here, it is assumed that the same communication ID and the same password are managed by the storage/reading processing unit 59, and the explanation will be continued.

If the storage/readout processing unit 59 determines that the login request is from a terminal that has valid usage authority assuming that the same communication ID and the same password are managed, the storage/readout processing unit In the terminal state management table (see FIG. 7B), 59 changes the operating state field of the record of the communication ID received in step S23 to "Online (available for communication)" and changes the date and time of reception. The date and time when the login request was received in step S23 is stored in the field (S25). As a result, for example, in the terminal status management table, the communication ID "01aa", the operating status "Online (callable)", the received date and time "2019.4.10.13:40", and the IP address "1.2. 1.4” will be associated and managed. Note that the IP address of the terminal may be transmitted from the shop terminal 1 in step S23 instead of being registered in advance.

Next, the storage/read processing unit 59 adds and manages a new record including the communication ID and password of the shop terminal 1 received in step S23 to the session management table (see FIG. 8(B)) ( S26). Then, the transmitting/receiving unit 51 of the communication management device 5 transmits the authentication result information indicating the authentication result obtained by the processing of step 24 to the shop terminal 1 requesting the login via the communication network 100. (S27).

When the transmitting/receiving unit 11 of the login requesting terminal (shop terminal 1) receives the authentication result information showing the result of the determination that the terminal has valid usage authority, the transmitting/receiving unit 11 performs communication via the communication network 100. Destination list request information indicating that a destination list is requested is transmitted to the management device 5 (S28). As a result, the transmission/reception unit 51 of the communication management device 5 receives the destination list request information.

Next, the storage/read processing unit 59 searches the destination list management table (see FIG. 8A) using the communication ID “01ab” of the login request terminal (shop terminal 1) as a search key, and searches the login request terminal (store terminal 1). The communication ID of a destination candidate that can communicate with the store terminal 1) is read, and the destination name corresponding to this communication ID is read from the terminal state management table (see FIG. 7B) (S29). Here, the communication IDs of the destination candidates corresponding to the communication ID "01ab" of the login requesting terminal (shop terminal 1) and the destination names corresponding to these are extracted.

Next, the storage/read processing unit 59 of the communication management device 5 reads the data of the destination list frame from the storage unit 5000 (S30). Then, the transmitting/receiving unit 51 sends "destination list information (destination list frame, communication ID, destination name)" including this destination list frame and the communication ID and destination name read by the storage/read processing unit 59, It is transmitted to the login request terminal (store terminal 1) (S31). As a result, in the login request terminal (shop terminal 1), the transmission/reception section 11 receives the destination list information, and the storage/read processing section 19 stores the destination list information in the storage section 1000 (S32).

As described above, in this embodiment, the communication management device 5 centrally manages the destination list information of all the terminals instead of managing the destination list information in each terminal. Therefore, if a new terminal is included in the communication system, a new model terminal is included in place of the already included terminal, or the appearance of the destination list frame is changed. However, since the communication management device 5 side handles all of them collectively, it is possible to save the trouble of changing the destination list information on each terminal side.

In addition, the storage/read processing unit 59 of the communication management device 5 searches the terminal status management table (see FIG. 7B) using the communication ID of the read destination candidate as a search key, and searches for each communication ID. , and obtains each operating state of the communication terminal as a destination candidate by reading the corresponding operating state (S33).

Next, the transmitting/receiving unit 51 transmits, via the communication network 100, the "terminal status information" including each communication ID as the search key used in step S26 and the operating status of each corresponding destination terminal. to the start terminal (shop terminal 1) (S34).

Next, the storage/read processing unit 19 of the start terminal (shop terminal 1) sequentially stores the terminal state information received from the communication management device 5 in the storage unit 1000 (S35). Therefore, the starting terminal (shop terminal 1), by receiving the state information of each of the above-mentioned terminals, is able to communicate with the starting terminal (shop terminal 1) at the present time of each of the explainer terminals 9, etc., which are destination candidates. Operation status can be acquired.

Next, the display control unit 14 of the login request terminal (shop terminal 1) reflects the state of the terminal as a destination candidate based on the destination list information stored in the storage unit 1000 and the state information of the terminal. A destination list is created (S36). This destination list is displayed on the shop terminal 1 as shown in FIG. 23(C) later.

On the other hand, the storage/read processing unit 59 of the communication management device 5 searches the destination list management table (see FIG. 8(A)) based on the communication ID of the login request terminal (shop terminal 1) to obtain the login request terminal. The communication IDs of other terminals in which the communication ID of (shop terminal 1) is registered as destination candidates are extracted (S37).

Next, the storage/read processing unit 59 of the communication management device 5 searches the terminal state management table (see FIG. 7B) based on the communication ID of the login request terminal (shop terminal 1), and The operating state of the store terminal 1) is acquired (S38).

Then, the transmitting/receiving unit 51 sends the terminal whose operation state is "Online" in the terminal state management table (see FIG. 7B) among the terminals related to the communication ID extracted in step S37 to "Terminal status information" including the communication ID and operating status of the login requesting terminal (shop terminal 1) acquired in S38 is transmitted (S39). When the transmitting/receiving unit 51 transmits terminal status information to the explainer terminal 9, the IP address of the communication terminal managed in the terminal status management table (see FIG. 7B) is based on each communication ID. See As a result, the communication ID and operating status of the login requesting terminal (shop terminal 1) can be transmitted to each of the other destination terminals that can communicate with the login requesting terminal (shop terminal 1) as a destination candidate. As a result, it is possible to display the state of destination candidates as shown in FIG.

<Processing for Determining Operating State of Presenter Terminal Group>
Next, with reference to FIG. 22, the processing performed by the store terminal 1 to determine the operating state of the explainer terminal group for each skill will be described. FIG. 22 is a flow chart showing processing for preparing video calls of store terminals, particularly processing for determining the operating state of a group of explainer terminals for each skill. Note that the operating state "Online" of the skill group indicates that at least one destination candidate among the destination candidates of this skill group has a presence [Online] that allows communication. Skill group availability "Chat" indicates that none of this skill group's potential destinations has a status of "Online" and contains at least one potential destination with an active presence of "Chat". show. Furthermore, the operating status "Offline" of the skill group indicates that all of the destination candidates of this skill group are in the operating status "Offline". An example of processing for determining the operating state of a skill group will be described below.

The storage/read processing unit 19 of the store terminal 1 acquires information recorded in the updated record each time the operation status of the destination management table is updated in step S34 of FIG. 21 (S51). The acquired information includes the communication ID of the explainer terminal 9ug11, the operating state indicating the latest state of the explainer terminal 9ug11 that is the destination candidate, and the skill of the explainer using the explainer terminal 9ug11 that is the destination candidate. Contains the skill name.

The storage/read processing unit 19 of the shop terminal 1 updates each record containing the skill name acquired in step S51 in the skill group management DB 1002 . That is, when the acquired skill names are "Specialty 1, Specialty 2", the storage/read processing unit 19 updates each record including the skill name "Specialty 1, Specialty 2" in the skill group management DB 1002. .

Hereinafter, as an example, operating state information (operating state "Online", skill name (Specialty 1, Specialty 2), communication ID "01ba") is transmitted from the explainer terminal 9ug11 to the store terminal 1, and in step S51 these Processing when information is acquired will be described (S52; Online).

First, a process of updating a record in which the skill name "specialty 1" is recorded in the table of the skill group management DB 1002 by loop processing will be described. The storage/reading processing unit 19 of the store terminal 1 refers to the communication ID field of the callable communication ID in the record in which the skill name "specialty 1" is recorded in the skill group management DB 1002 (see (B) of FIG. 6). . Accordingly, the determination unit 15 determines whether the communication ID "01ba" acquired in step S51 is included in the field for communication available (S53).

If it is determined that the communication ID “01ba” acquired in step S51 is not included in the callable field in skill group management DB 1002 (S53; NO), storage/read processing unit 19 sets the callable field (S54).

When the process of step S54 is completed, or when YES in step S53, the storage/read processing unit 19 of the store terminal 1 stores the skill name "specialized" in the skill group management DB 1002 (see (B) of FIG. 6). 1” refers to the busy field in the recorded record. Accordingly, the determination unit 15 determines whether the communication ID “01ba” acquired in step S51 is included in the “busy” field (S55).

If it is determined that the communication ID “01ba” acquired in step S51 is included in the busy field in skill group management DB 1002 (S55; YES), storage/read processing unit 19 sets the busy field , the acquired communication ID is deleted (S56). By the processing up to this point, the operating state of the explainer terminal 9ug11 changes from "Chat" to "Online" and from "Offline" to "Online", and the skill name " In the record in which "Specialty 1" is recorded, the communication ID of the explainer terminal 9ug11 is recorded only in the field for communication.

If the process of step S56 is completed, or if NO in step S55, the storage/read processing unit 19 of the shop terminal 1 stores the skill name "specialized" in the skill group management DB 1002 (see (B) of FIG. 6). 1” refers to the operating status of the group in the record recorded. Accordingly, the determination unit 15 determines whether the operating state of the group is "Online" (S57).

When it is determined that the operating status of the group in the record in which the skill name "Specialty 1" is recorded is not "Online" (S57; NO), the storage/reading processing unit 19 updates this operating status to "Online". (S58). As a result, when displaying the screen shown in FIG. 23A, the display control unit 14 performs a normal display such as "domestic travel" that looks clear, for example. On the other hand, if it is determined that the operating status of the group in the record in which the skill name "specialty 1" is recorded is "Online" (S57; YES), the storage/read processing unit 19 determines that the skill name "specialty 1" is Do not update the group's health status in the logged record. By the processing up to this point, the operating status of the group in the record in which the skill name "specialty 1" of the skill group management DB 1002 is recorded becomes "Online".

Since the skill names "Specialty 1, Specialty 2" have been acquired in step S51, the storage/read processing unit 19 also performs loop processing for each record in which the skill name "Specialty 2" is recorded in the skill group management DB 1002. , to update as well.

Subsequently, the operating state information (operating state “Chat”, skill name (specialty 1, specialty 2), communication ID “01ba”) is transmitted from the explainer terminal 9ug11 to the shop terminal 1, and in step S51 these operating states Processing when information is acquired will be described (S52; Chat).

First, a process of updating a record in which the skill name "specialty 1" is recorded in the skill group management DB 1002 by loop processing will be described. The storage/read processing unit 19 of the store terminal 1 refers to the callable field in the record in which the skill name “specialty 1” is recorded in the table of the skill group management DB 1002 . Accordingly, the determination unit 15 determines whether or not the communication ID "01ba" acquired in step S51 is included in the callable field (S63).
If it is determined that the communication ID “01ba” acquired in step S51 is included in the callable field in skill group management DB 1002 (S63; YES), storage/read processing unit 19 sets the callable field , the acquired communication ID "01ba" is deleted (S64).

If the process of step S64 is completed, or if NO in step S63, the storage/read processing unit 19 of the store terminal 1 stores the call in the record in which the skill name "specialty 1" is recorded in the skill group management DB 1002. Refer to the fields inside. Accordingly, the determination unit 15 determines whether the communication ID “01ba” acquired in step S51 is included in the “busy” field (S65).

When it is determined that the communication ID “01ba” acquired in step S51 is not included in the busy field in skill group management DB 1002 (S65; NO), storage/read processing unit 19 sets the busy field (S66). By the processing up to this point, the communication ID of the explainer terminal 9ug11 is recorded only in the "in-call" field in the record in which the skill name "specialty 1" of the skill group management DB 1002 is recorded.

If the process of step S66 is completed, or if YES in step S65, the storage/read processing unit 19 of the store terminal 1 stores the call in the record in which the skill name "specialty 1" is recorded in the skill group management DB 1002. See possible fields. Accordingly, the storage/read processing unit 19 determines whether or not the number of communication IDs recorded in the callable field in the record in which the skill name "Specialty 1" is recorded is 0 (zero) (S67).

When the number of communication IDs recorded in the callable field in the record in which the skill name "Specialty 1" is recorded is 0 (zero) (S67; YES), the storage/read processing unit 19 stores the skill name " The operating status of the group in the record in which "Specialty 1" is recorded is updated to "Chat" (S68). Accordingly, when displaying the screen shown in FIG. 23A, the display control unit 14 performs translucent display such as "wedding", for example. On the other hand, if the number of communication IDs recorded in the callable field in the record in which the skill name "Specialty 1" is recorded is not 0 (zero) (S67; NO), the storage/read processing unit 19 stores the skill Do not update the active state of the group in the record where the name "professional 1" is recorded. That is, when a communication ID is recorded in the callable field in the record in which the skill name "Specialty 1" is recorded, the corresponding operating status remains "Online" and is not updated.

Since the skill names "Specialty 1, Specialty 2" have been acquired in step S51, the storage/read processing unit 19 also performs loop processing for each record in which the skill name "Specialty 2" is recorded in the skill group management DB 1002. , to update as well.

Subsequently, the operating state information (operating state "Offline", skill name "Specialty 3, Specialty 4", communication ID "01ca") is transmitted from the explainer terminal 9ug21 to the store terminal 1, and in step S51, these operating state information is acquired (S52; Offline).

First, a process of updating a record in which the skill name "specialty 3" is recorded in the skill group management DB 1002 by loop processing will be described. The storage/read processing unit 19 of the shop terminal 1 refers to the callable field in the record in which the skill name "Specialty 3" is recorded in the table of the skill group management DB 1002. FIG. Accordingly, the determination unit 15 determines whether the communication ID "01ca" acquired in step S51 is included in the callable field (S73).

If it is determined that the communication ID “01ca” acquired in step S51 is included in the callable field in skill group management DB 1002 (S63; YES), storage/read processing unit 19 sets the callable field , delete the acquired communication ID "01ca" (S74).

When the process of step S74 is completed, or when NO in step S73, the storage/read processing unit 19 of the shop terminal 1 selects the record in which the skill name "Specialty 3" is recorded in the table of the skill group management DB 1002. See the In-Call field in . Accordingly, the determination unit 15 determines whether the communication ID "01ca" acquired in step S51 is included in the "busy" field (S75).

If it is determined that the communication ID "01ca" acquired in step S51 is included in the busy field in the skill group management DB 1002 (S75; YES), the storage/read processing unit 19 reads from the busy field. , the acquired communication ID "01ca" is deleted (S76). Through the processing up to this point, the communication ID of the explainer terminal 9ug21 is deleted from the record in which the skill name "specialty 3" is recorded in the skill group management DB 1002. FIG.

If the process of step S76 is completed, or if NO in step S75, the storage/read processing unit 19 of the store terminal 1 stores the call in the skill group management DB 1002 in the record in which the skill name "specialty 3" is recorded. See the Available and Busy fields. Thereby, the storage/read processing unit 19 determines whether the conditions IF1 and IF2 are satisfied (S77). Condition IF1 is that the number of communication IDs recorded in the field of communication available in the record in which the skill name "professional 1" is recorded is 0 (zero), and the number of communication IDs recorded in the field of communication is not 0 (zero). Condition IF2 is that the number of communication IDs recorded in the available field in the record in which the skill name "specialty 3" is recorded is 0 (zero) and the number of communication IDs recorded in the busy field. is 0 (zero).

The number of communication IDs recorded in the Callable field in the record in which the skill name "Specialty 3" is recorded is 0 (zero), and the number of communication IDs recorded in the Talking field is 0 (zero) ) (S77; IF1), the storage/read processing unit 19 updates the operating status of the group in the record in which the skill name "specialty 3" is recorded in the skill group management DB 1002 to "Chat" (S78). The number of communication IDs recorded in the Callable field in the record in which the skill name "Specialty 3" is recorded is 0 (zero), and the number of communication IDs recorded in the Talking field is 0 (zero) ) (S77; IF2), the storage/read processing unit 19 updates the operating status of the group in the record in which the skill name "Specialty 3" is recorded in the skill group management DB 1002 to "Offline" (S79). Accordingly, when displaying the screen shown in FIG. 23A, the display control unit 14 performs translucent display such as "wedding", for example. On the other hand, if neither IF1 nor IF2 is applicable in step S77 (S77; NO), the storage/read processing unit 19 leaves the operating status of the group in the record in which the skill name "Specialty 3" is recorded as "Online". Do not update.

Since the skill names "Specialty 3, Specialty 4" have been acquired in step S51, the storage/read processing unit 19 performs loop processing for each record in which the skill name "Specialty 4" is recorded in the skill group management DB 1002. are also updated in the same way.

<Start processing of video communication>
Next, processing for starting video communication for remote customer service will be described with reference to FIGS. 23 and 24. FIG. FIG. 23 is a diagram showing a screen displayed on the store terminal. FIG. 24 is a sequence diagram showing processing for starting video communication.

In the store terminal 1, the display control unit 14 displays one of the screens shown in FIGS. 23A to 23D on the display 106 of the store terminal 1, and the reception unit 12 receives the customer's selection of a skill group. Alternatively, the nomination of an explainer is accepted (S82).

Specifically, the storage/read processing unit 19 stores each skill name stored in the table of the skill group management DB 1002 (see FIG. 6B) and the operation status of the skill group corresponding to this skill name. to get As a result, the display control unit 14 causes the display 106 of the shop terminal 1 to display the skill selection screen shown in FIG.

On the right side of the skill selection screen, a skill display button (domestic travel, overseas travel, etc.) for selecting a skill and an explainer list display button for displaying a list of all destinations of the explainers are displayed. Here, the skill display button for "domestic travel" is normally displayed and is in a state that the customer can select it. On the other hand, the skill display button for "overseas travel" is translucent and cannot be selected by the customer. Then, for example, when the customer selects the "domestic travel" skill display button, the reception unit 12 receives the selection, and the display control unit 14 displays the confirmation screen shown in FIG. 23(B) on the display 106. . When the customer presses the "OK" button, the reception unit 12 accepts the push, and the transmission/reception unit 11 connects the communication session for the video call to the desired explainer terminal 9 via the communication management device 5. It transmits connection request information indicating the request (see S84-1 and S85-1 to be described later).

On the other hand, when the customer presses the explainer list display button on the skill selection screen shown in FIG. 6(A)), the operating status of the explainer terminal, and the skill name, the presenter's address list shown in FIG. 23(C) is displayed on the display 106 display. Here, the customer selects the name of the explainer who uses the desired explainer terminal from among the explainer terminals that can be communicated with, and presses the "OK" button. As a result, the display control unit 14 displays the confirmation screen shown in FIG. 23(D) on the display 106 . Then, when the customer presses the "OK" button, the reception unit 12 accepts the depression, and the transmission/reception unit 11 transmits a video call start request to the designated explainer terminal 9 via the communication management device 5 (described later). S90, see the processing of FIG. 25).

Subsequently, in the store terminal 1, when the customer selects a desired skill group (for example, an organization in charge that can explain domestic travel) on the skill selection screen of FIG. Send a request to start a video call to the online (talkable) explainer terminal in a specific skill group (for example, domestic travel) of the support organization UG1, It is determined whether to transmit a request to start a video call to the explainer terminal that is online (callable) during travel (S83). In this case, the store terminal 1 determines based on the subgroup name to which the account ID of the own terminal belongs in the subgroup management table (see FIG. 11B) in the service information received in step S19. If the own terminal (store terminal 1) belongs to an unmanned store, the determination unit 15 determines the destination of the start request as a specific skill group of the unmanned store priority support organization UG1. On the other hand, if the own terminal (shop terminal 1) belongs to a manned shop, the determination unit 15 determines the destination of the start request as a specific skill group of the manned shop priority support organization MG1.

Subsequently, in FIG. 1, when the store terminal 1 is the store terminal 1c2 installed in the unmanned store US, the organization SR21 in charge of the skill SK1 of the unmanned store priority support organization UG1 goes Online (call available). ), a case (first time) in which a video call start request is transmitted to the explainer terminals 9ug11 and 9ug12.

First, the transmitting/receiving unit 11 of the store terminal 1c2 sends connection request information indicating a connection request for a communication session for a video call all at once as a first step before issuing a video call start request (see S101 in FIG. 25). Send (S84-1). This connection request information includes, among the communication IDs read out in step S83, the communication ID whose corresponding operation status is "Online", the communication ID of the store terminal 1c2 that requested the start, and the skill selected in step S82. Contains a skill name that indicates

Next, when receiving the connection request information transmitted by the store terminal 1c2, the transmission/reception unit 51 of the communication management device 5 sends this connection request to the explainer terminal 9ug11 specified by the communication ID included in the received connection request information. Information is transmitted (S84-2).

If the operating state is updated in step S34 (see FIG. 21) after the store terminal 1c2 transmits the connection request information and before it receives a response, the store terminal 1c2 repeats the processes of steps S82 and S83. . As a result, when the communication ID is extracted as a communication ID whose corresponding operating state is "Online" by updating the operating state on the explainer terminal 9 side to "Online", the transmitting/receiving unit of the shop terminal 1c2 11 transmits connection request information including the communication ID to the communication management device 5 (S85-1). The connection request information transmitted here also includes the skill name indicating the skill selected in step S82 and the communication ID of the own terminal.

Upon receiving the connection request information transmitted by the store terminal 1, the transmission/reception unit 51 of the communication management device 5 transmits this connection request information to the explainer terminal 9ug12 specified by the communication ID included in the received connection request information. (S85-2).

Each transmission/reception unit 91 of the explainer terminals 9ug11 and 9ug12 receives the connection request information transmitted via the communication management device 5 from the store terminal 1c2. When this connection request information is received, the respective display control units 94 of the explainer terminals 9ug11 and 9ug12 display the information of the requester corresponding to the communication ID included in the connection request information and the skill name included in the call start request. A message containing a skill name (skill information) corresponding to is displayed on the display 106 of each explainer terminal 9ug11, 9ug12. By indicating the skill in the message, the explainers of the explainer terminals 9ug11 and 9ug12 can grasp in advance what kind of travel products the customer desires.

Next, when each explainer ug11, ug12 performs an operation input for response permission at each explainer terminal 9ug11, 9ug12, the reception unit 92 of each explainer terminal 9ug11, 9ug12 receives an operation input for response permission ( S86-1, S86-2). When the operation input for permitting the response is accepted, the transmitter/receiver 91 of each explainer terminal 9ug11, 9ug12 transmits the response information indicating that the connection is permitted to the communication management device 5 (S87-1, S87-2). This response information includes the communication ID of the transmission source (shop terminal 1c2) and the communication ID of its own terminals (9ug11, 9ug12). As a result, the transmission/reception unit 51 of the communication management device 5 receives the response information transmitted by each of the explainer terminals 9ug11 and 9ug12. The above process is only an example, and when receiving the connection request information from the store terminal 1c2, the transmitting/receiving units 91 of the explainer terminals 9ug11 and 9ug12 automatically send response information to the store terminal 1c2 to the connection request. may be sent. Further, when receiving a plurality of pieces of connection request information from a plurality of store terminals 1, the transmitting/receiving unit 11 of each of the explainer terminals 9ug11 and 9ug12 sends response information to the store terminal 1 that transmitted the connection request information received first. may be sent.

Next, the transmission/reception unit 51 of the communication management device 5 transmits the received response information to the shop terminal 1c2 specified by the communication ID included in this response information (S87-2, S88-2).

When the transmitting/receiving unit 11 of the store terminal 1c2 receives a plurality of pieces of response information, the determination unit 15 determines the presenter terminal 9 that has transmitted the earliest response information (S89). Hereinafter, in this embodiment, a case where the explainer terminal 9ug11 responds first will be described.

The transmitting/receiving unit 51 of the communication management device 5 transmits the relay device 30 between the explainer terminal 9ug11 and the shop terminal 1 specified by the communication ID "the communication ID of the shop terminal, the communication ID of the explainer" included in the communication establishment request. Control is performed to establish a communication session for transmitting content data such as image data and sound data via a video call (S90). This step S90 will be described later in detail with reference to FIG. The communication session is not limited to the above, and may be a session for directly transmitting content data via the communication network 100 between the explainer terminal 9ug11 and the shop terminal 1c2. When communication is established between the store terminal 1c2 and the explainer terminal 9ug11, each store terminal 1c2 and the explainer terminal 9ug11 transmit image data of images captured on the own terminal side and sound data collected on the own terminal side. is transmitted to the terminal of the other party via the relay device 3. This allows a video call to be initiated between the customer and the presenter, as shown in FIG.

When communication is established in step S90, the transmitting/receiving unit 91 of the explainer terminal 9ug11 sends the state information including the operating state "Chat", the skill name "professional 1", and the communication ID of the explainer terminal 9ug11 to the communication management device. 5.

Further, when communication is established in step S90, the transmitting/receiving unit 11 of the store terminal 1c2 transmits cancellation information in order to cancel the call start request to the sender of the response other than the sender of the response received first. It is transmitted to the management device 5 (S91-1). The cancellation information includes the communication ID of the own terminal (shop terminal 1c2) and the communication ID of the explainer terminal 9 (in this case, explainer terminal 9ug12) that is the sender of the response other than the sender of the response received first. is

When the transmission/reception unit 51 of the communication management device 5 receives the cancellation information, it transmits the cancellation information to the explainer terminal 9ug12 specified by the communication ID included in the cancellation information (S91-2). The transmitting/receiving unit 11 of the explainer terminal 9ug12 receives the cancellation information transmitted by the communication management device 5. FIG. As a result, the explainer terminal 9ug12 side can grasp that the response to the call start request from the store terminal 1 side is no longer necessary. <Communication Establishment Control Processing>
Next, the communication establishment control process (see S90) in FIG. 24 will be described in detail with reference to FIG. FIG. 25 is a sequence diagram showing detailed processing of communication establishment control.

First, the store terminal 1c2 transmits start request information indicating a request to start video communication to the communication management device 5 (S101). This start request information includes the communication ID of the start terminal (here, store terminal 1c2) and the communication ID of the explainer ug11 of the destination terminal (here, explainer terminal 9ug11). Thereby, the communication management device 5 receives the start request information.

Next, in the communication management device 5, the selection unit 53 selects the nearest relay device based on the IP addresses of the start terminal and the destination terminal (S102). The IP address of each terminal is managed in a terminal management table, and the IP address of each relay device is pre-stored in the storage unit 5000 . Next, the creation unit 54 creates communication information including the IP address of the relay device and the communication ID described above (S103). Then, the storage/read processing unit 59 stores the communication ID and the IP address of the relay device, which are the contents of the communication information, and the starting terminal and Management is performed by associating and storing each communication ID of the explainer of the destination terminal. After that, the transmitting/receiving unit 51 transmits the start request information to the destination terminal (in this case, the explainer terminal 9ug11) (S105). This start request information includes communication information in addition to each communication ID sent from the start terminal in step S101. As a result, the transmission/reception unit 91a of the destination terminal receives the start request information.

Next, in the destination terminal, the determination unit 95 of the explainer terminal 9ug11 has already received the communication ID of the start terminal (shop terminal 1c2) received in step S105 in the process of step S84-2 in FIG. It is determined whether the communication ID of the shop terminal 1c2 is the same (S106). In this case, since they are basically the same, the transmitting/receiving unit 91a transmits the responsive information to the communication management device 5 (S107). In this case, the responsive information indicates that the response is permitted, and includes the communication ID of the starting terminal. As a result, the transmission/reception unit 51 of the communication management device 5 receives the responsive information.

Next, the storage/read processing unit 59 of the communication management device 5 searches the communication information management DB 5005 using the communication ID of the starting terminal received in step S107 as a search key, thereby obtaining the corresponding communication information (communication ID, relay device IP address) is read (S108). Then, the transmitting/receiving unit 51 transmits responsive information to the starting terminal. The responsive information in this case includes the communication information read in step S108. As a result, the transmitting/receiving unit 11 of the starting terminal receives the acknowledgment information.

Next, the transmission/reception unit 91 of the destination terminal transmits establishment request information indicating a request to establish a mutual communication session to the IP address of the relay device included in the communication information received in step S105 (S110). This establishment request information includes the communication ID of the originating terminal and the communication ID of the presenter of the destination terminal. In addition, the IP address of the destination terminal, which is the source of transmission, is also transmitted. On the other hand, the transmitting/receiving unit 11 of the starting terminal transmits establishment request information indicating a request to establish a mutual communication session to the IP address of the relay device included in the communication information received in step S109 (S111). This establishment request information includes the communication ID of the originating terminal and the communication ID of the presenter of the destination terminal. In addition, the IP address of the initiating terminal that is the source is also transmitted. As a result, a communication session for mutual communication of images and sounds is established between the relay device 3 and the start terminal (S112-1), and images and sounds are exchanged between the relay device 3 and the destination terminal. A communication session is established for mutual communication (S112-2).

Next, the transmitting/receiving unit 11 of the starting terminal transmits the status information of its own terminal (starting terminal) to the communication management device 5 (S113-1). This status information includes the communication ID of the initiating terminal and operating status information (here, "communicating"). As a result, the transmission/reception unit 51 of the communication management device 5 receives the state information of the starting terminal. On the other hand, the transmitting/receiving unit 91 of the destination terminal transmits the status information of its own terminal (destination terminal) to the communication management device 5 (S113-2). This status information includes the communication ID of the destination terminal and operation status information (here, "Chat (in communication)"). Thereby, the transmission/reception unit 51 of the communication management device 5 receives the status information of the destination terminal. In this way, each terminal itself notifies the communication management device 5 of its operating state. Then, in the communication management device 5, the storage/read processing unit 59 changes the operating state associated with the communication ID of the starting terminal to "Chat (in communication)" in the terminal management table (see FIG. 10). , the operating state associated with the communication ID of the destination terminal is changed to "Chat (in communication)" (S114). As described above, the start terminal (shop terminal 1c2) and the destination terminal (instructor terminal 9ug11) can make a video call by mutual communication, as shown in FIG. The above is the processing when a skill group such as "domestic travel" is selected in step S82 in FIG.

<Request to start video call when presenter is nominated>
Next, in step S82, processing for starting a video call when an explainer is designated in FIG. 23(C) will be described. In this case, in FIG. 24, only the process of step S90 is executed among the steps after step S84.

Also, if the presenter is designated, the processing of steps S84-2 and S85-2 of FIG. 24 is not performed, so in step S106 of FIG. allows the designated presenter to manually select whether or not to respond. Then, when the designated explainer selects whether or not to respond, the reception unit 92 receives the selection of whether or not to respond.

Then, in steps S107 and S109, response availability information is transmitted instead of response availability information. Again, the communication ID of the originating terminal is included. Processing other than these is the same as when a skill group is selected, so description thereof will be omitted.

<Processing for changing the transmission destination of the video call start request>
Next, referring to FIG. 26, in steps S86-1 and S86-2 of FIG. 24, when all explainer terminals do not respond to the start request, processing for changing the transmission destination of the video call start request will be explained. FIG. 26 is a flow chart showing various processes for starting a call. Specifically, FIG. 26 is a flow chart showing three patterns of a start request after skill group selection, a change in the transmission destination of the start request, and a start request when an explainer is appointed.

The determination by the determination unit 15 described below is made based on each information managed in the explainer management DB 1001 and the skill group management DB 1002 of the storage unit 1000 .

First, at the store terminal 1c2 of the unmanned store US, as a result of the process of step S82 shown in FIG. 24, the determination unit 15 determines whether to select a skill group or designate an explainer (S201). Then, in the case of selecting a skill group (S201; Skill), the determination unit 15 explains an unmanned store priority support organization that is Online (callable) for the accepted skill group within the same area E1. It is determined whether there is a user terminal (S202). Then, if there is an explainer terminal of an online (callable) unmanned store priority organization (S202; YES), the transmitting/receiving unit 11 of the store terminal 1c2 sends an online (callable) unmanned store priority organization. (S203) to each explainer terminal, as a first step before requesting the start of a video call (see S101 in FIG. 25), connection request information indicating a connection request for a communication session for a video call (S203 ). The processing of step S203 corresponds to the processing of steps S84-1 and S85-1 in FIG.

Next, after the processing of steps S84-1 and S85-1, the determination unit 15 receives the response information from the explainer terminals 9ug11 and 9ug12 within a certain period of time (for example, 30 seconds) (S87-2, S88- 2) It is determined whether or not (S204). Then, if response information is received within a certain period of time (S204; YES), the store terminal 1c2 performs communication establishment control with respect to the explainer terminal that responded first (in this case, explainer terminal 9ug11) (S205). ). The processing of step S205 corresponds to the processing of steps S89 and S90 in FIG. In this case, the processing of FIG. 26 ends.

On the other hand, if the response information indicating that the connection is not possible is received in step S202 (NO), or if the connection request information is transmitted in step S204, the connection is confirmed within a predetermined time (for example, 30 seconds). (NO), the transmitting/receiving unit 11 of the store terminal 1c2 changes the transmission destination to each of the presenter terminals 9mg11 and 9mg12 of the online (callable) manned store priority organization. , the connection request information is transmitted all at once as the second time (S206). Then, after the second processing of steps S84-1 and S85-1, the determination unit 15 determines whether the response information (S87-2 and S88-2) has been received from the explainer terminals 9mg11 and 9mg12 within a predetermined time. It is determined whether or not (S207). Then, if response information is received within a predetermined period of time (S207; YES), in step S205, the store terminal 1c2 performs communication establishment control with respect to the explainer terminal that responded first (in this case, explainer terminal 9mg11). conduct. If no response information is received within the predetermined time (S207; NO), the display control unit 14 displays an error dialog indicating that there is no response on the display 106 of its own terminal (S208). After that, the process returns to step S201.

By the way, in step S201, if the explainer is designated (S201: Nomination), the display control unit 14 causes the display 106 of the own terminal to display the explainer's destination list shown in FIG. (S209). Then, the reception unit 12 receives the nomination (selection) of the explainer from the customer c2 (S210). The processing of steps S209 and S210 corresponds to the processing of step S82 in FIG.

Further, as a result, the store terminal 1c2 executes communication establishment control processing only for the designated explainer terminal (in this case, explainer terminal 9ug11) (S211). The process of step S211 corresponds to the process of step S90 in FIG.

<Screen example during video call between explainer terminal and store terminal>
FIG. 27 is an example of a call screen displayed on the explainer terminal. FIG. 27 is a diagram showing a call screen of the presenter's terminal. A call screen 930 of the explainer terminal includes a main image field 940 , a sub-image field 950 and a button display field 960 . A hold button 961 , a travel plan button 962 , a display switching button 963 , a camera button 964 , a microphone button 965 and a disconnect button 966 are displayed in the button display field 960 . Note that the main image column 940 is displayed over the entire call screen 930 of the explainer terminal 9, and the sub image column 950 is displayed in a portion of the call screen 930 (here, the lower right) and is smaller than the main image column 940. is displayed.

The hold button 961 is a button for the terminal device to stop transmitting images and sounds while maintaining communication. Used when the presenter is away for a short period of time. The fact that the reserve button 961 has been pressed is transmitted to the store terminal 1 . When the reception unit 92 of the explainer terminal 9 detects that the hold button has been pressed, it reads out a predetermined image from the storage unit and displays it on the display of the explainer terminal 9 . For example, by displaying a message such as "I'm out of my seat", the customer's feeling of discomfort is suppressed. The image to be displayed can be set by the store staff, so advertisements and videos can also be displayed.

A travel plan button 962 is a function for activating a mode for creating a travel plan while listening to the customer's request.

A display switching button 963 can be used to hide the image of the customer displayed in the main image field 940 and switch the display to only the background image. After pressing the travel plan button 962, the explainer creates a travel plan with the customer, and when displaying the local image (omnidirectional image) 941 as shown in FIG. It is a function that temporarily hides the image of the customer in order to show the customer.

A camera button 964 and a microphone button 965 are buttons for the presenter to switch the camera of his/her presenter terminal 9 and adjust the mute of the microphone.

A disconnect button 966 is a button for disconnecting communication between terminals (presence changes from Chat to Online). Only the presenter can disconnect the communication.

Next, a call screen displayed by the store terminal will be described with reference to FIG. 28 . FIG. 28 is an example of a call screen displayed on the store terminal. As shown in FIG. 28, the call screen 130 displayed on the store terminal 1 includes a main image field 140, a sub-image field 150, and a button display field 160. FIG. A display switching button 163 , a camera button 164 , a microphone button 165 and a disconnect button 166 are displayed in the button display field 160 . The display switching button 163, camera button 164, microphone button 165, and disconnect button 166 play the same roles as the display switching button 963, camera button 964, microphone button 965, and disconnect button 966 in FIG. The customer can only display the sub-image column (in which the customer's own face is displayed) 150, display/hide the explainer displayed in the main image column 140, and operate the microphone and camera. The main image column 140 is displayed over the entire call screen 180 of the store terminal 1, and the sub-image column 150 is displayed in a portion (here, the lower right) of the call screen 180 smaller than the main image column 140. It is

<Suggestions for travel plans>
Next, a description will be given of a process in which the explainer proposes a travel plan to a customer by remote call. When serving customers, the presenter inputs a country or region from a world map or a sightseeing spot specified by the customer (guest) while listening to the customer (guest) about their desired travel destination. After inputting, a map will be displayed on the screen. When you select a sightseeing value within the area of the map, an image (omnidirectional image) will be displayed in the background image, allowing you to plan your trip while experiencing the local atmosphere. A detailed description will be given below.

FIG. 29 is a sequence diagram showing processing in which the explainer terminal and the store terminal activate a video call from a dedicated application and create a travel plan. FIG. 30 is an example of a call screen (including an input window and a map window) displayed on the explainer terminal. FIG. 31 is an example of a call screen (input window, map window) displayed on the explainer terminal.

First, as shown in FIGS. 27 and 28, the store terminal 1 and the explainer terminal 9 are making a video call. When the explainer presses the travel plan button 962 displayed on the call screen of the explainer terminal shown in FIG. Invoke creation mode. When the "travel plan creation mode" is activated, the display control unit 94 displays the sightseeing spots, lodging facilities, lodging dates, and number of people desired by the customer in the main image field 940 of the call screen 930, as shown in FIG. , the number of rooms, etc., is displayed on the upper left, and a map window 980 including the landmarks of tourist spots input to the input window 970 is displayed on the lower left. Here, the presenter hears the desired sightseeing spot from the customer by remote call and inputs the conditions according to the customer's request in the items displayed in the input window 970, and the reception unit 92 accepts the input of For example, the explainer inputs Okinawa, Okinawa prefecture, and remote island 1 as tourist spots where the customer stays.

With this input, the display control unit 94 displays a map corresponding to the remote island 1 in the map window 980 as shown in FIG. This map displays a plurality of landmarks that indicate areas that can be introduced by the presenter. In this case, each hotel on remote islands centering on Miyako Island in Okinawa Prefecture is displayed as a landmark.

Next, when the explainer selects a specific landmark 981 on the map window 980 while listening to the customer's request, the reception unit 92 receives selection of a tourist spot related to the specific landmark 981 (S302). When a specific landmark 981 is selected, the transmission/reception unit 91 of the explainer terminal 9 transmits a request for an image of the tourist spot to the communication management device 5 via the communication network 100 (S303). At this time, the request transmitted from the explainer terminal 9 includes a landmark ID for identifying the specific landmark 981 . As a result, the transmission/reception unit 51 of the communication management device 5 receives the request sent in step S317.

Next, in the communication management device 5, based on the landmark ID included in the request received from the explainer terminal 9 by the transmission/reception unit 51, the storage/read processing unit 59 stores the image information management DB 5006 (see FIG. 8D). The file name of the tourist spot image corresponding to the landmark ID stored in is read out, and the image data of the tourist spot with this file name is read out from the storage unit 5000 (S304). Then, the transmission/reception unit 51 of the communication management device 5 transmits the image data of the sightseeing spot to the explainer terminal 9 (S305) and to the store terminal 1 (S306). As a result, the transmitting/receiving unit 91 of the explainer terminal 9 and the transmitting/receiving unit 11 of the store terminal 1 receive the image data of the sightseeing spot, respectively.

FIG. 32 is an example of a call screen (including an input window, a map window, and an image of a tourist spot) displayed on the explainer terminal. FIG. 33 is an example of a call screen (including an image of a tourist spot) displayed on the store terminal.

As a result of step S305, the display control unit 94 of the explainer terminal 9 causes the display 106 of the explainer terminal 9 to display the image of the tourist spot as a background image as shown in FIG. 32 (S307). In this case, the display control unit 94 displays not only the background image of the main image column 940 in which the image of the customer is displayed, but also the background image of the sub-image column 950 in which the explainer himself/herself is displayed. On the other hand, in step S306, as shown in FIG. 33, in the store terminal 1, the display control unit 14 displays the image of the tourist spot as a background image on the display 106 of the store terminal 1 (S308). In this case, the display control unit 14 displays not only the background image of the main image field 140 in which the image of the explainer is displayed, but also the background image of the sub-image field 150 in which the customer himself is displayed.

As a result, each user as an explainer and a customer can share the same background image and make a video call, so that they can have a conversation about the tourist spot while enlivening the conversation.

Furthermore, the presenter hears the desired accommodation facility (here, "○ × △ hotel") from the customer by remote call, and inputs the conditions according to the customer's request in the items displayed in the input window 970. The unit 92 receives the input of the conditions (accommodation facility in this case) according to the customer's request (S309).

Next, the transmitting/receiving unit 91 of the explainer terminal 9 transmits a request for an exterior image of the accommodation facility to the communication management device 5 via the communication network 100 (S310). At this time, the request transmitted from the explainer terminal 9 includes the accommodation facility information (here, "0×Δ hotel") as the condition accepted in step S309. As a result, the transmission/reception unit 51 of the communication management device 5 receives the request sent in step S310.

Next, in the communication management device 5, based on the accommodation facility information included in the request received from the explainer terminal 9 by the transmission/reception unit 51, the storage/read processing unit 59 stores the image information management DB 5006 (see FIG. 8D). The file name of the image of the appearance of the accommodation facility corresponding to the accommodation facility information stored in the storage unit 5000 is read from the storage unit 5000, and the accommodation facility received in step S310. Room information corresponding to the information is read from the image information management DB 5006 (S311). Then, the transmission/reception unit 51 of the communication management device 5 transmits the image data of the appearance of the accommodation facility and the room information read out in step S311 to the explainer terminal 9 (S312), and also transmits it to the shop terminal 1 (S313). ). Thereby, the transmitting/receiving unit 91 of the explainer terminal 9 and the transmitting/receiving unit 11 of the store terminal 1 receive the image data of the appearance of the accommodation facility and the room information, respectively.

FIG. 34 is an example of a call screen (including an input window, a map window, an image of the appearance of an accommodation facility, and a price list) displayed on the explainer terminal. FIG. 35 is an example of a call screen (including an image of the appearance of an accommodation facility and a price list) displayed on a store terminal.

By the above step S312, as shown in FIG. 34, in the explainer terminal 9, the display control unit 94 causes the display 106 of the explainer terminal 9 to display an image of the exterior of the accommodation facility as a background image, A price list 990 based on the room information is displayed (S314). In this case, the display control unit 94 displays not only the background image of the main image column 940 in which the image of the customer is displayed, but also the background image of the sub-image column 950 in which the explainer himself/herself is displayed. On the other hand, in step S313, as shown in FIG. 35, in the store terminal 1, the display control unit 14 causes the display 106 of the store terminal 1 to display an image of the exterior of the accommodation facility as a background image, A price list 190 based on the room information is displayed (S315). In this case, the display control unit 14 displays not only the background image of the main image field 140 in which the image of the explainer is displayed, but also the background image of the sub-image field 150 in which the customer himself is displayed.

As a result, each user as an explainer and a customer can share the same background image and have a video call, so that they can have a conversation about the accommodation facility while enlivening the conversation.

Furthermore, the explainer hears the grade (rank) of the desired room (here, "deluxe") from the customer by remote call, and selects the room grade (here, "deluxe") displayed on the price list 990 is accepted (S316).

Next, the transmitting/receiving unit 91 of the explainer terminal 9 transmits the request for the image of the room accepted in step S316 to the communication management device 5 via the communication network 100 (S317). At this time, the request transmitted from the explainer terminal 9 includes the room ID indicating the grade of the room accepted in step S316. As a result, the transmission/reception unit 51 of the communication management device 5 receives the request sent in step S317.

Next, in the communication management device 5, based on the room ID included in the request received from the explainer terminal 9 by the transmission/reception unit 51, the storage/read processing unit 59 stores the image information management DB 5006 (see FIG. 8D). The file name of the image of the room corresponding to the stored image ID is read, and the image data of the room with this file name is read from the storage unit 5000 (S318). Then, the transmitting/receiving unit 51 of the communication management device 5 transmits the image data of the room read out in step S318 to the explainer terminal 9 (S318) and to the store terminal 1 (S320). Thereby, the transmitting/receiving unit 91 of the explainer terminal 9 and the transmitting/receiving unit 11 of the store terminal 1 receive the image data of the room.

FIG. 36 is an example of a call screen (including an input window, a map window, an image of a room, and a price list) displayed on the explainer terminal. FIG. 37 is an example of a call screen (including an image of a room and a price list) displayed on a shop terminal.

As shown in FIG. 36, in the explainer terminal 9, the display control unit 94 causes the display 106 of the explainer terminal 9 to display the image of the room (here, "deluxe") as the background image in step S319. (S321). In this case, the display control unit 94 displays not only the background image of the main image column 940 in which the image of the customer is displayed, but also the background image of the sub-image column 950 in which the explainer himself/herself is displayed. On the other hand, in step S320, as shown in FIG. 36, in the store terminal 1, the display control unit 14 displays the image of the room as the background image on the display 106 of the store terminal 1 (S322). In this case, the display control unit 14 displays not only the background image of the main image field 140 in which the image of the explainer is displayed, but also the background image of the sub-image field 150 in which the customer himself is displayed.

As a result, each user as an explainer and a customer can share the same background image and have a video call, so that they can have a conversation about the room while enlivening the conversation.

[Main effects of the present embodiment]
As described above, according to the present embodiment, the explainer allows the customer (guest) to virtually experience the atmosphere of the place where the travel is planned. In addition, since it is possible to see images of the site from various viewpoints with spherical images, it is possible to make plans while feeling the site more realistically than with photographs, which gives a sense of security. As a result, each user as an explainer and a customer can share the same background image displayed over the entire main image column 140, 940 to make a video call with each other. be able to have a conversation.

In addition, even though the store terminal US of the unmanned store made a call start request (first time) to each of the explainer terminals 9ug11 and 9ug12 of the organization SK12 in charge of the desired skill SK1 of the unmanned store priority support organization UG1. , Even if there is no response from any of the explainer terminals, a call start request (second time) is made to each explainer terminal 9mg11, 9mg12 of the responsible organization SR11 with the same skill SK1 of the manned store priority support organization MG1. It can be performed. As a result, while taking advantage of the unmanned store's advantages of streamlining and streamlining store operations, the explainer terminals 9ug11 and 9ug12 that are prioritized for unmanned stores are installed to reduce the number of customers leaving the unmanned store. Furthermore, even if the explainer terminals 9ug11 and 9ug12 that are prioritized for unmanned stores cannot respond, the store terminals can make a call start request to the explainer terminals 9mg11 and 9mg12 that are prioritized for manned stores. By doing so, there is an effect that the loss of customers can be further suppressed.

〔supplement〕
In the above embodiment, steps S302, S309, and S316 are performed, but at least two of them may be performed in the same step. Also, any one of steps S302, S309, and S316 may be omitted.

Also, the store terminal 1 is an example of a communication terminal. The store terminal 1 includes not only a PC but also a smart watch, a game machine, a dedicated video call machine, and the like.

Furthermore, in communication between the communication management device 5 and the store terminal 1 and explainer terminal 9, another server, router, or the like may relay data.

1 store terminal (an example of a second communication terminal)
3 relay device 4 administrator terminal 5 communication management device 7 service information management device 9 explainer terminal (an example of a first communication terminal)
11 transmission/reception unit 12 reception unit 13 communication control unit 14 display control unit 15 determination unit 51 transmission/reception unit (an example of transmission means, an example of reception means)
5000 storage unit (an example of storage means)
100 communication network

Claims (9)

A communication management device that mediates the video data of the subject at the first base so that the second communication terminal receives the video data of the subject at the first base transmitted by the first communication terminal,
receiving means for receiving a transmission request for a specific background image transmitted by the first communication terminal;
While the second communication terminal displays the subject at the first base, the specific background image is displayed as a background image for the subject at the first base, and a second image captured by the second communication terminal is displayed. in order to display the specific background image as a background image for the subject at the second base while the second communication terminal is displaying the video data of the subject at the base, the second communication based on the transmission request transmission means for transmitting data of the specific background image to the terminal;
and a communication management device. the video data transmitted by the first communication terminal is video data of a first subject photographed by the first communication terminal;
The second communication terminal transmits the image data of the subject at the second base photographed by the second communication terminal, whereby the receiving means receives the image data of the subject at the second base,
the transmitting means transmits the image data of the subject at the second base so that the first communication terminal receives the image data of the subject at the second base;
The transmission means sends the transmission request so that the first communication terminal receives the video data of the subject at the second location and displays the specific background image as a background image for the second subject being displayed. 2. The communication management device according to claim 1, wherein the data of the specific background image is transmitted based on. To display the specific background image as a background image for the subject at the first base while the first communication terminal is displaying video data of the subject at the first base photographed by the first communication terminal, 3. The communication management apparatus according to claim 2, wherein said transmission means transmits data of said specific background image based on said transmission request. The communication management device according to any one of claims 1 to 3,
having management means for associating and managing predetermined background image data and predetermined identification information for identifying the predetermined background image;
The receiving means receives specific identification information as a transmission request for the specific background image,
The communication management apparatus, wherein the transmission means transmits data of the specific background image corresponding to the specific identification information. 5. The communication management device according to any one of claims 1 to 4, wherein the background image is an omnidirectional image. 6. The communication device according to claim 5, wherein the omnidirectional image is a still image or a moving image. A communication management method executed by a communication management device that mediates video data of a subject at a first base so that a second communication terminal receives video data of a subject at the first base transmitted by a first communication terminal. and
a receiving step of receiving a transmission request for a specific background image transmitted by the first communication terminal;
While the second communication terminal displays the subject at the first base, the specific background image is displayed as a background image for the subject at the first base, and a second image captured by the second communication terminal is displayed. in order to display the specific background image as a background image for the subject at the second base while the second communication terminal is displaying the video data of the subject at the base, the second communication based on the transmission request a transmission step of transmitting the data of the specific background image to the terminal;
A communication management method characterized by executing: A program that causes a computer to execute the method according to claim 7. A second communication terminal receives the image data of the subject at the first location transmitted by the first communication terminal, and transmits the image data of the subject at the second location transmitted by the second communication terminal to the first communication terminal. A communication management device for managing a video call between the first communication terminal and the second communication terminal to be received by
receiving means for receiving a transmission request for a specific background image transmitted by the first communication terminal;
In order to display the specific background image as a background image for the subject at the second base while the second communication terminal is displaying video data of the subject at the second base photographed by the second communication terminal, transmitting means for transmitting data of the specific background image to the second communication terminal based on the transmission request;
has
The transmission means sends the transmission request so that the first communication terminal receives the video data of the subject at the second location and displays the specific background image as a background image for the second subject being displayed. and transmitting the data of the specific background image based on.

Images