JP7809976B2 - Communication system, communication method, and program - Google Patents

Description

This disclosure relates to a communication system, a communication method, and a program.

In recent years, remote concierge (remote customer service) services have become available in an effort to streamline and streamline store operations. For example, even if a customer (guest) has to wait their turn to be served because an in-store employee is busy serving customers, the customer (guest) can access a concierge terminal with specialized skills at a remote service center from a store terminal via the Internet or other means, and have a video call with the concierge to receive customer service (see Patent Document 1).

Remote customer service has been increasingly adopted by travel agencies in recent years. The introduction of this service allows customers (guests) to be served without having to wait in a store, and it is beginning to become popular as a service that helps travel agencies prevent lost opportunities to acquire new customers.

Japanese Patent Application Laid-Open No. 2008-154059

However, several issues have become apparent. Compared to face-to-face customer service, remote customer service poses the challenge of not getting the customer (guest) excited when planning their trip. In traditional face-to-face customer service, the customer and the presenter can share travel brochure materials and exchange opinions while planning their trip. Hearing the presenter's personal experiences directly creates a lively conversation, and the customer's expectations for the trip grow. On the other hand, even in remote customer service, travel plans are made while sharing electronic travel brochures via screen sharing, but because it is not a face-to-face format, it is difficult to get the customer excited about the trip. As a result, when using video calls, it becomes more difficult to get the conversation going compared to face-to-face formats, which is an issue.

The invention of claim 1 is a communication management device that mediates video data of a subject at a first location transmitted by a first communication terminal so that a second communication terminal can receive the video data of the subject at the first location, comprising: a receiving means for receiving a request to send a specific background image transmitted by the first communication terminal; and a transmitting means for transmitting data of the specific background image to the second communication terminal based on the transmission request so that the second communication terminal displays the specific background image as a background image for the subject at the first location while the second communication terminal is displaying video data of the subject at the second location captured by the second communication terminal.

As explained above, according to the present invention, even during video calls, each user can share the same background image with each other, which has the effect of livening up the conversation.

1 is a diagram illustrating the overall configuration of a communication system according to an embodiment of the present invention. FIG. 10 is an image diagram showing a situation in which a customer at a store and a presenter at a service center are remotely making a video call. FIG. 2 is a hardware configuration diagram of each terminal and each device that constitutes the communication system of the present embodiment. FIG. 2 is a functional block diagram of the communication system according to the embodiment. FIG. 2 is a functional block diagram of the communication system according to the embodiment. 1A is a conceptual diagram showing a presenter management table, and FIG. 1B is a conceptual diagram showing a skill group management table. 1A is a conceptual diagram showing an authentication management table, and FIG. 1B is a conceptual diagram showing a terminal status management table. 1A is a conceptual diagram showing a destination list management table, FIG. 1B is a conceptual diagram showing a session management table, and FIG. 1C is a conceptual diagram showing a communication information management table. 1A is a conceptual diagram of a system update information management table, FIG. 1B is a conceptual diagram of a business operator management table, and FIG. 1C is a conceptual diagram of an area management table. 1A is a conceptual diagram of a terminal management table, and FIG. 1B is a conceptual diagram of a skill management table. 1A is a conceptual diagram of a login information management table, FIG. 1B is a conceptual diagram of a subgroup management table, and FIG. 1C is a conceptual diagram of a user update information management table. FIG. 10 is a sequence diagram showing a registration process of service information. FIG. 10 is a diagram showing a login screen displayed on the administrator terminal. FIG. 10 is a diagram showing a store list screen displayed on the administrator terminal. FIG. 10 is a diagram showing a presenter list screen displayed on the administrator terminal. FIG. 10 is a diagram showing a group list screen displayed on the administrator terminal. FIG. 10 is a diagram showing an explainer information editing screen displayed on the manager terminal. FIG. 10 is a diagram showing a group information editing screen displayed on the administrator terminal. FIG. 10 is a diagram showing a store information editing screen displayed on the manager terminal. FIG. 10 is a sequence diagram showing a preparation process for a video call by a store terminal, and in particular showing a process for acquiring service information. FIG. 10 is a sequence diagram showing a preparation process for a video call at a store terminal, and in particular showing a process for acquiring the operating status of each terminal. 10 is a flowchart showing a preparation process for a video call at a store terminal, and in particular a process for determining the operating status of a group of presenter terminals for each skill. FIG. 10 is a diagram showing a screen displayed on a store terminal. FIG. 10 is a sequence diagram showing a process for starting a video call. FIG. 10 is a sequence diagram showing detailed processing of communication establishment control. 10 is a flowchart showing various processes for starting a call. 10 is an example of a call screen displayed on a presenter terminal. 10 is an example of a call screen displayed on a store terminal. FIG. 10 is a sequence diagram showing a process in which a presenter terminal and a store terminal start a video call from a dedicated application and create a travel plan. 10 is an example of a call screen (including an input window and a map window) displayed on the presenter terminal. 10 is an example of a call screen (input window, map window) displayed on the presenter terminal. 10 is an example of a call screen (including an input window, a map window, and an image of a tourist spot) displayed on the presenter terminal. 10 is an example of a call screen (including an image of a tourist spot) displayed on a store terminal. 10 is an example of a call screen (including an input window, a map window, an image of the exterior of an accommodation facility, and a price list) displayed on the presenter terminal. 10 is an example of a call screen (including an image of the exterior of an accommodation facility and a price list) displayed on a store terminal. 10 is an example of a call screen (including an input window, a map window, an image of a room, and a price list) displayed on the presenter terminal. 10 is an example of a call screen (including an image of a room and a price list) displayed on a store terminal.

[Overall configuration of communication system]
The overall configuration of a communication system will be described using Figure 1. Figure 1 is an overall configuration diagram of a communication system according to an embodiment of the present invention. Here, a customer, a travel agency that sells travel products to the customer, and a system provider that provides a system for remote concierge (remote customer service) to the travel agency appear.

Travel agencies operate staffed stores, unmanned stores, and service centers in multiple areas, such as Kanto and Kansai. Figure 1 shows that a travel agency operates a staffed store MS, an unmanned store US, and a service center SC1 in area E1. A staffed store MS is a store that has employees who handle customer service. An unmanned store is a store that does not have employees who handle customer service. Note that while Figure 1 shows only one staffed store MS and one unmanned store US, there may be multiple of each.

At manned stores MS, employees typically introduce travel products to customers who visit, but there are times when an employee is busy serving a customer and is unable to assist a new customer. Furthermore, at unmanned stores US, there are no employees to assist customers in the first place. In such cases, customers can use store terminal 1c1 in the manned store MS or store terminal 1c2 in the unmanned store US to access the presenter terminal of any presenter at service center SC1 via communication network 100 such as the Internet, and receive remote assistance regarding travel products from the presenter. The presenter is, for example, the remote concierge mentioned above. Since presenters basically use their own dedicated presenter terminal, specific presenters and specific presenter terminals are linked.

Store terminals 1c1 and 1c2 are each configured as a PC (personal computer). Multiple store terminals may be installed in a staffed store MS and an unstaffed store US. Hereinafter, multiple store terminals, such as store terminals 1c1 and 1c2, will be collectively referred to as "store terminal 1."

The service center SC1 includes a staffed store priority response organization MG1 that prioritizes receiving call start requests to start remote customer service from the store terminal 1c1 of the staffed store MS, and an unmanned store priority response organization UG1 that prioritizes receiving call start requests to start remote customer service from the store terminal 1c2 of the unmanned store US. Note that there may be three or more priority response organizations.

Furthermore, the manned store priority response organization MG1 includes an organization SR11 responsible for skill SK1 and an organization SR12 responsible for skill SK2. For example, skill SK1 is the ability of a presenter to make suggestions regarding domestic travel, and skill SK2 is the ability of a presenter to make suggestions regarding overseas travel. Similarly, the unmanned store priority response organization UG1 includes an organization SR21 responsible for skill SK1 and an organization SR22 responsible for skill SK2. In other words, the manned store priority response organization MG1 and the unmanned store priority response organization UG1 have organizations SR11 and SR21 (or organizations SR12 and SR22) that employ presenters with the same skills. Note that there may be three or more skills and skill-responsible organizations.

Furthermore, in the manned store priority response organization MG1, the organization SR11 responsible for skill SK1 includes presenters mg11 and mg12 who possess skill SK1. Presenter mg11 uses presenter terminal 9mg11, and presenter mg12 uses presenter terminal 9mg12.

Furthermore, the organization SR12 responsible for skill SK2 includes presenters mg21 and mg22 who possess skill SK2. Presenter mg21 uses presenter terminal 9mg21, and presenter mg22 uses presenter terminal 9mg22.

Similarly, in unmanned store priority response organization UG1, organization SR21 responsible for skill SK1 includes presenters ug11 and ug12 who possess skill SK1. Presenter ug11 uses presenter terminal 9ug11, and presenter ug12 uses presenter terminal 9ug12. Furthermore, organization SR22 responsible for skill SK2 includes presenters ug21 and ug22 who possess skill SK2. Presenter ug21 uses presenter terminal 9ug21, and presenter ug22 uses presenter terminal 9ug22.

Note that there may be three or more presenters in each responsible organization (SR11, SR12, SR21, SR22). In this case, a presenter terminal is assigned to each of the three or more presenters. Furthermore, hereinafter, when referring to multiple presenter terminals such as presenter terminals 9mg11, 9mg12, 9mg21, 9mg22, 9ug12, 9ug12, 9ug21, 9ug22, etc., they will be referred to collectively as "presenter terminal 9."

The travel agency also has an administrator ad1 who manages the terminals in each service area. The administrator ad1 uses the administrator terminal 4 to access the service information management device 7 via the communication network 100 and sets various types of service information. This service information is information related to the processing and settings of each terminal at the travel agency,
The administrator terminal 4 is configured as a PC.

Customers c1 and c2 visited staffed store MS and unstaffed store US, respectively.

The system provider manages and operates the relay device 3, communication management device 5, and service information management device 7. The relay device 3, communication management device 5, and service information management device 7 are computers such as servers. The relay device 3 relays image data and audio data when a video call is made between one of the store terminals 1 and one of the presenter terminals 9. The communication management device 5 centrally manages login authentication from the store terminal 1 and the presenter terminal 9, management of the communication status of the store terminal 1 and the presenter terminal 9, management of the destination list, and communication status of the relay device 30. The service information management device 7 manages various set service information. The relay device 3, communication management device 5, and service information management device 7 will be explained in more detail later.

In addition, the store terminal 1, manager terminal 4, presenter terminal 9, relay device 3, communication management device 5, and service information management device 7 can be connected to a communication network 100 such as the Internet.

The relay device 3, communication management device 5, and service information management device 7 may be constructed using a single computer, or may be constructed using multiple computers to which each section (function or means) is divided and assigned as desired.

Figure 2 is an illustration showing a situation in which a store customer and a presenter at the service center are remotely engaged in a video call. Figure 2 shows customer c2, who has visited unmanned store US, using store terminal 1c2, while presenter ug11 from unmanned store priority response organization UG1 is using presenter terminal 9ug11. In this way, customer c2 can receive an explanation of a travel product through remote customer service.

[Hardware configuration of communication system]
Next, using FIG. 3, the hardware configuration of each terminal (store terminal 1, manager terminal 4, presenter terminal 9) and each device (relay device 3, communication management device 5, service information management device 7) that constitute the communication system shown in FIG. 1 will be described. In this embodiment, it is assumed that each terminal and each device have the same configuration, so the hardware configuration of the store terminal 1 will be described and descriptions of the hardware configurations of the other terminals and devices will be omitted.

As shown in Figure 3, the store terminal 1 is configured as a computer and includes a CPU 101, ROM 102, RAM 103, HD 104, HDD (Hard Disk Drive) controller 105, display 106, external device connection I/F (Interface) 108, network I/F 109, data bus 110, keyboard 111, pointing device 112, DVD-RW (Digital Versatile Disk Rewritable) drive 114, and media I/F 116.

Of these, CPU 101 controls the operation of the entire computer. ROM 102 stores programs used to drive CPU 101, such as IPL. RAM 103 is used as a work area for CPU 101. HD 104 stores various data, such as programs. HDD controller 105 controls the reading and writing of various data from HD 104 under the control of CPU 101. Display 106 displays various information, such as a cursor, menu, window, text, or image. External device connection I/F 108 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. Network I/F 109 is an interface for data communication using communication network 100. Bus line 110 is an address bus, data bus, etc. for electrically connecting each component, such as CPU 101, shown in Figure 3.

The keyboard 111 is a type of input device equipped with multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 112 is a type of input device for selecting and executing various instructions, selecting processing targets, moving the cursor, etc. The DVD-RW drive 114 controls the reading and writing of various data from a DVD-RW 113, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may also be a DVD-R or Blu-ray Disc. The media I/F 116 controls the reading and writing (storing) of data from a recording medium 115, such as a flash memory.

Also, a microphone 121 as an example of a sound collection device, a speaker 122 as an example of a sound output device, and a camera 123 as an example of an imaging device are connected to the external device connection I/F. Note that while the microphone 121, speaker 122, and camera 123 are shown as external devices in Figure 3, they may also be built-in devices without going through the external device connection I/F.

[Functional configuration of the communication system]
Next, the functional configuration of the communication system according to this embodiment will be described with reference to Figures 3 to 11. Figures 4 and 5 are block diagrams of the functions of the communication system according to this embodiment.

<Functional configuration of store terminal>
First, the functional configuration of the store terminal 1 will be described using Figures 3 and 4. As shown in Figure 4, the store terminal 1 includes a transmission/reception unit 11, a reception unit 12, a communication control unit 13, a display control unit 14, a determination unit 15, and a storage/readout processing unit 19. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Figure 3 operates in response to commands from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The store terminal 1 also includes a memory unit 1000 constructed by the RAM 103 and HD 104 shown in Figure 3. A presenter management DB 1001 and a skill group management DB 1002 are constructed in the memory unit 1000. The information managed in the presenter management DB 1001 and the skill group management DB 1002 is included in the client information sent from the service information management device 7 in step S19, which will be described later. The store terminal 1 uses the information managed in the presenter management DB 1001 and the skill group management DB 1002 when displaying a screen shown in FIG. 23, which will be described later.

(Explainer Management Table)
Fig. 6A is a conceptual diagram of a presenter management table. A presenter management DB 1001 configured by the presenter management table shown in Fig. 6A is constructed in the storage unit 1000. In this presenter management table, various information (communication ID, presenter name, operating status (presence) of the destination candidate of the store terminal 1, and skill name indicating the skill of the presenter who exclusively uses the presenter terminal) is associated and managed.

The communication ID is identification information used to identify the communication destination in the communication system 1. The communication ID is not particularly limited, but may be, for example, a user ID for identifying the presenter using the presenter terminal 9, or a communication ID for identifying a store terminal. The presenter name in FIG. 6(A) is the presenter name shown in FIG. 23(C) described below. The operating status indicates the operating status of the terminal on the other side of the video call. There are three operating statuses: Online (available for communication), Chat (in communication), and Offline (unable to communicate). The operating status is reflected in the displays in FIGS. 23(A) and (C). As mentioned above, the skill indicates the name of the skill possessed by the presenter. As shown in FIG. 6(A), one presenter may have multiple skills (Specialty 1, Specialty 2).

(Skill group management table)
Fig. 6B is a conceptual diagram showing a skill group management table. A skill group management DB 1002 is constructed in the storage unit 1000 using the skill group management table shown in Fig. 6B. In this skill group management table, skill names, operating status (presence) of skill groups, communication IDs of presenter terminals that can be contacted by a store terminal, and communication IDs of presenter terminals that are currently in communication are associated and managed.

The operating status of a skill group indicates the overall operating status of each presenter terminal of a group of presenters who have the same skill, such as Specialty 1. For example, if there are multiple presenter terminals in a group of presenters who have the same skill, and the operating status of even one of these presenter terminals is Online, the operating status of the Specialty 1 skill group will be managed as Online. Also, even if the operating status of all multiple presenter terminals is not Online, if the operating status of even one of these presenter terminals is Chat, the operating status of the Specialty 1 skill group will be managed as Chat. Furthermore, if the operating status of all multiple presenter terminals is neither Online nor Chat, i.e., if all operating statuses are Offline, the operating status of the Specialty 1 skill group will be managed as Offline.

(Functional configuration of store terminal)
Next, we will explain each component of the store terminal 1. The transmitting/receiving unit 11 is realized by commands from the CPU 101 shown in Fig. 3, the external device connection I/F 108, and the network I/F 109, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100.

The reception unit 12 is realized mainly by commands from the CPU 101 shown in Figure 3, as well as the keyboard 111 and pointing device 112, and receives various inputs from the user.

The communication control unit 13 is realized by commands from the CPU 101 shown in FIG. 3, and inputs image data obtained by an external camera connected to the external device connection I/F 108 by capturing an image of a subject. The communication control unit 13 also inputs sound data obtained by an external microphone connected to the external device connection I/F 108 by collecting sound. The communication control unit 13 also outputs image data received from the other end of the video call and image data obtained by its own terminal to the display control unit 14. The communication control unit 13 also outputs sound data received from the other end of the video call and sound data obtained by its own terminal to an external speaker connected to the external device connection I/F 108.

The display control unit 14 is mainly realized by commands from the CPU 101 shown in Figure 3, and displays images by outputting image data to the display 106 or an external display connected to the external device connection I/F.

The judgment unit 15 is mainly realized by commands from the CPU 101 shown in Figure 3, and makes various judgments. The contents of the judgments will be described later.

The storage/read processing unit 19 is mainly executed by commands from the CPU 101 and the HDD controller 105 shown in Figure 3, and performs processes such as storing various data in the storage unit 1000 and reading various data stored in the storage unit 1000.

Furthermore, image data and sound data received when communicating with other terminals are overwritten and stored in the memory unit 1000 each time they are received. Of these, the image data before being overwritten displays an image on the display 106 or an external display, and the sound data before being overwritten outputs sound from an external speaker.

<Administrator terminal>
Next, the functional configuration of the administrator terminal 4 will be described with reference to Figures 3 and 4. As shown in Figure 4, the administrator terminal 4 has a transmission/reception unit 41, a reception unit 42, a display control unit 44, and a storage/read processing unit 49. The administrator terminal 4 also has a storage unit 4000 constructed by the RAM 103 and HD 104 shown in Figure 3.

Note that the transmitter/receiver unit 41, reception unit 42, display control unit 44, and storage/readout processing unit 49 of the manager terminal 4 have the same functions as the transmitter/receiver unit 11, reception unit 12, display control unit 14, and storage/readout processing unit 49 of the store terminal 1, respectively, and therefore their description will be omitted.

<Functional configuration of the presenter terminal>
Next, the functional configuration of the presenter terminal 9 will be described with reference to Figures 3 and 4. As shown in Figure 4, the presenter terminal 9 has a transmitting/receiving unit 91, a receiving unit 92, a communication control unit 93, a display control unit 94, a determination unit 95, and a storage/readout processing unit 99. The presenter terminal 9 also has a memory unit 9000 constructed by the RAM 103 and HD 104 shown in Figure 3. A presenter management DB 9001 and a skill group management DB 9002 are constructed in the memory unit 9000.

Note that the transmitter/receiver unit 91, reception unit 92, communication control unit 93, display control unit 94, judgment unit 95, and storage/readout processing unit 99 of the presenter terminal 9 have the same functions as the transmitter/receiver unit 11, reception unit 12, communication control unit 13, display control unit 14, judgment unit 15, and storage/readout processing unit 19 of the store terminal 1, respectively, and therefore their descriptions will be omitted. Furthermore, the presenter management DB 9001 and skill group management DB 9002 of the presenter terminal 9 have the same data structure as the presenter management DB 1001 and skill group management DB 1002 of the store terminal 1, and therefore their descriptions will be omitted.

<Functional configuration of relay device>
Next, the functional configuration of the relay device 3 will be described using Figures 3 and 5. As shown in Figure 5, the relay device 3 has a transmitting/receiving unit 31 that also serves as a forwarding unit, a determining unit 32, and a storage/reading processing unit 39. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Figure 3 operates in response to an instruction from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The relay device 3 also has a storage unit 3000 constructed by the RAM 103 and HD 104 shown in Figure 3.

(Functional configuration of relay device)
Next, a detailed description will be given of each functional configuration of the relay device 3. The transmitting/receiving unit 31 of the relay device 3 shown in Fig. 5 is realized by commands from the CPU 101 shown in Fig. 3 and the network I/F 109 shown in Fig. 3, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100. The transmitting/receiving unit 31 also serves as a transfer unit, and transfers image data and sound data transmitted from a predetermined terminal to other terminals.

The judgment unit 32 is implemented by commands from the CPU 101 shown in Figure 3, and performs tasks such as judging the data delay status.

The storage/read processing unit 39 is realized by commands from the CPU 101 shown in FIG. 3 and the HDD controller 105 shown in FIG. 3, and performs processes such as storing various data in the storage unit 3000 and reading various data stored in the storage unit 3000.

<Functional configuration of the communication management device>
Next, the functional configuration of the communication management device 5 will be described using Figures 3 and 5. As shown in Figure 5, the communication management device 5 has a transmitting/receiving unit 51, a selecting unit 53, a creating unit 54, a determining unit 55, and a storing/reading processing unit 59. Each of these units is a function or a means of functioning that is realized when any of the components shown in Figure 3 operates in response to an instruction from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The communication management device 5 also has a memory unit 5000 constructed by the RAM 103 and HD 104 shown in Figure 3.

(Authentication Management Table)
Fig. 6A is a conceptual diagram showing an authentication management table. An authentication management DB 5001 configured with an authentication management table such as that shown in Fig. 6A is constructed in the storage unit 5000. In this authentication management table, each password is associated with each communication ID of all terminals 1 and 9 managed by the communication management device 5 and managed. Note that a password is an example of authentication information, and authentication information also includes an access token.

(Terminal Status Management Table)
7B is a conceptual diagram showing a terminal status management table. A terminal status management DB 5002 configured by the terminal status management table shown in FIG. 7B is constructed in the storage unit 5000. In this terminal status management table, for each communication ID of each terminal 1, 9, the destination name when the terminal 1, 9 is the destination, the operating status of each terminal 1, 9, the date and time when login request information (described below) was received by the communication management device 5, and the IP address of the store terminal 1, 9 are associated and managed. The communication ID, destination name, and terminal IP address are stored when each terminal 1, 9 pre-registers to receive services provided by the communication management device 5.

(Destination list management table)
8A is a conceptual diagram showing a destination list management table. A destination list management DB 5003 configured with the destination list management table shown in FIG. 8A is constructed in the storage unit 5000. In this destination list management table, the communication IDs of all destination terminals registered as candidate destination terminals are associated with the communication ID of the initiating terminal requesting the start of a call and are managed. These destination terminal candidates are updated by being added or deleted in response to an addition or deletion request from the administrator terminal 4 to the communication management device 5.

Note that the destination list is an example of destination information, and the destination information may not be in list format, but may instead be a list of information about the destinations, such as communication IDs.

(Session management table)
8B is a conceptual diagram showing a session management table. A session management DB 5004 configured by the session management table shown in FIG. 8B is constructed in the storage unit 5000. In this session management table, for each communication session ID for identifying a session for mutual communication between each terminal and the relay device 3, the relay device ID of the relay device 3 used, the communication ID of the initiating terminal, the communication ID of the destination terminal, the reception delay time (ms) when the image data is received at the destination terminal, and the reception date and time when the delay information indicating this delay time is sent from the destination terminal and received by the communication management device 5 are associated and managed.

(Communication Information Management Table)
8C is a conceptual diagram showing a communication information management table. A communication information management DB 5005 configured by a communication information management table such as that shown in FIG. 8C is constructed in the storage unit 5000. In this communication information management table, for each communication ID for identifying mutual communication performed between each terminal and the relay device 3, the IP address of the relay device 3 used and the communication ID of the terminal with which the mutual communication is performed are associated and managed.

(Image information management table)
Fig. 8(D) is a conceptual diagram showing an image information management table. An image information management DB 50006 configured with an image information management table such as that shown in Fig. 8(D) is constructed in the storage unit 5000. In this image information management table, landmark IDs, file names of images of tourist spots, accommodation information, file names of images of the exteriors of accommodation facilities, room grades (ranks) and prices, room IDs, and file names of images of rooms are associated and managed.

Of these, the landmark ID is tourist attraction identification information used to identify landmarks, which will be described later. The accommodation information is accommodation identification information used to identify accommodations, such as the hotel name. The room ID is room identification information used to identify the room grade. The room grade and price, as well as the room ID, are part of the room information. The types of image data are still image data, video image data, and spherical image data. In this embodiment, when supporting the creation of travel plans, it is assumed that spherical image data (including still images and videos) will be used as background images to create a sense of realism, as if the user were actually at the actual location.

Here, the data for images of tourist spots, images of the exteriors of accommodation facilities, and images of rooms are stored in the memory unit 5000. Note that the data for images of tourist spots, images of the exteriors of accommodation facilities, and images of rooms may be stored in another device (such as a server) rather than in the communication management device 5, and the communication management device 5 may acquire the data from the other device.

(Functional configuration of the communication management device)
Next, we will explain in detail each functional configuration of the communication management device 5. In explaining each functional configuration of the communication management device 5, we will also explain the relationship between each component shown in Figure 3 and the main components that realize each functional configuration of the communication management device 5.

The transmission/reception unit 51 of the communication management device 5 shown in Figure 5 is realized by commands from the CPU 101 shown in Figure 3 and the network I/F 109, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100.

The selection unit 53 is implemented by commands from the CPU 101 shown in Figure 3, and selects the optimal relay device for mutual communication based on the IP address of each terminal.

The creation unit 54 is implemented by commands from the CPU 101 shown in FIG. 3, and creates communication information in response to requests from each terminal to start mutual communication. This communication information includes the IP address of the relay device selected by the selection unit 53, the communication ID described above, etc.

The judgment unit 55 is implemented by commands from the CPU 101 shown in Figure 3 and makes various judgments.

The storage/readout processing unit 59 is realized by commands from the CPU 101 and the HDD controller 105 shown in Figure 3, and performs processes such as storing various data in the storage unit 5000 and reading various data stored in the storage unit 5000.

<Service Information Management Device>
Next, the functional configuration of the service information management device 7 will be described using Figures 3 and 5. As shown in Figure 5, the service information management device 7 has a transmission/reception unit 71, a determination unit 75, and a storage/readout processing unit 79. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Figure 3 operates in response to commands from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The service information management device 7 also has a storage unit 7000 constructed by the RAM 103 and HD 104 shown in Figure 3. A service information management DB 7001 is constructed in this storage unit 7000. The service information management DB 7001 contains the tables shown in Figures 9 to 11.

(System update information management table)
9A is a conceptual diagram showing a system update information management table. In this system update information management table, the terminal name, latest version, update application date and time, and download URL (Uniform Resource Locator) are associated and managed for each terminal ID. In the client name column, the store terminal app is an application for the store terminal 1, and the presenter terminal app is an application for the presenter terminal 9. Each application is managed by its download URL.

(Business operator management table)
9B is a conceptual diagram showing a business operator management table. In this business operator management table, business operator IDs and business operator names are associated and managed. Business operators are, for example, travel agencies, leasing companies, etc.

(Group management table)
9C is a conceptual diagram showing a group management table. In this group management table, a group name (area name) and a carrier ID are associated with each group ID (area ID) and managed. The group indicates area E1 in FIG. 1, etc.

(Terminal management table)
FIG. 10A is a conceptual diagram illustrating a terminal management table. In this terminal management table, a destination name, role, login ID, password, business ID, group ID, and communication ID are associated with and managed for each account ID. The account ID is identification information for identifying the store, the presenter, and the administrator. In the case of a store, the account of the store itself, not the customer, is managed. The destination name is also managed in the terminal status management table of FIG. 7B, but is the display name shown on the destination selection screen displayed on the store terminal 1 as shown in FIG. 23C. The role indicates the role of the terminal user, etc. The login ID and password are used for authentication when the administrator ad1 accesses the service information management device 7 from the administrator terminal 4. The "password" in this terminal management table is the same as the "password" in the authentication management table of FIG. 7A, but may be a different password. The communication ID is also managed in the terminal status management table (see FIG. 7A) in the communication management device 5.

(Skill management table)
10B is a conceptual diagram showing a skill management table. In this skill management table, a skill name and the account ID of the presenter who possesses the skill are associated with each skill ID and managed. As shown in FIG. 10B, it is also possible to manage the fact that the same presenter (001) possesses multiple skills, such as Specialty 1 and Specialty 2.

(Login information management table)
11A is a conceptual diagram showing a login information management table. In this login information management table, the version, account ID, and last login date and time are associated and managed for each serial number ID. The version is the version of the client application (each application of the store terminal 1 and the presenter terminal 9) at the time of login. This makes it possible to know which account is using which version when an application is updated.

(Subgroup management table)
FIG. 11B is a conceptual diagram showing a subgroup management table. In this subgroup management table, the subgroup name, group ID, and account ID of the presenter belonging to the subgroup are associated and managed for each subgroup ID. As shown in FIG. 10B, it is also possible to manage the fact that the same presenter (account ID "001") has multiple skills, such as specialty 1 and specialty 2. In the case of the account ID of a store terminal, as shown in FIG. 1, the subgroup indicates a staffed store or an unstaffed store. Furthermore, in the case of the presenter's account ID, the subgroup indicates a staffed store priority organization or an unstaffed store priority organization.

For example, according to the subgroup management table, the store terminal (Tokyo main store No. 1) with account ID "003" will prioritize sending connection request information indicating a connection request for a communication session for a video call to the presenter terminals used by the presenters with account IDs "001" and "002" simultaneously (first transmission) as a preliminary step to a video call start request (see S101 in FIG. 25) (see S84-1 and S85-1 in FIG. 24 described below). If there is no response from the presenter terminals of the presenters with account IDs "001" and "002" within a certain period of time (see S204 in FIG. 25 described below), the store terminal (Tokyo main store No. 1) with account ID "003" will send a call start request simultaneously (second transmission) to specific presenter terminals that are online (available for calls) among the presenter terminals of presenters with the same skills other than those with account IDs "001" and "002" (see S206 in FIG. 25 described below).

(User update information management table)
11C is a conceptual diagram showing a user update information management table, which manages, for each update ID, the account ID of the update target, the scheduled update version, the update application date and time, and the download URL in association with each other.

(Functional configuration of the service information management device)
Next, we will explain in detail each functional component of the service information management device 7. In explaining each functional component of the service information management device 7, we will also explain the relationship between each component shown in Figure 3 and the main components that realize each functional component of the service information management device 7.

The transmission/reception unit 11 of the service information management device 7 shown in Figure 5 is realized by commands from the CPU 101 shown in Figure 3 and the network I/F 109, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100.

The judgment unit 75 is implemented by commands from the CPU 101 shown in Figure 3 and makes various judgments.

The storage/read processing unit 79 is realized by commands from the CPU 101 and the HDD controller 105 shown in Figure 3, and performs processes such as storing various data in the storage unit 7000 and reading various data stored in the storage unit 7000.

[Processing or Operation of the Embodiment]
Next, the processing or operation of the communication system according to this embodiment will be described with reference to Figures 12 to 26. In the following, the store terminal 1 as the request source requesting the start of a video call may be referred to as the "start terminal," and the presenter terminal 9 as the destination (relay destination) to which the request is made may be referred to as the "destination terminal."

<Service Information Registration Process>
12 to 19, a process for registering various service information from the administrator terminal 4 to the service information management device 7 will be described. The service information is managed in the tables shown in FIGS.

Figure 12 is a sequence diagram showing the service information registration process. Figure 13 is a diagram showing a login screen displayed on an administrator terminal. Figure 14 is a diagram showing a store list screen displayed on an administrator terminal. Figure 15 is a diagram showing a presenter list screen displayed on an administrator terminal. Figure 16 is a diagram showing a group list screen displayed on an administrator terminal. Figure 17 is a diagram showing a presenter information editing screen displayed on an administrator terminal. Figure 18 is a diagram showing a group information editing screen displayed on an administrator terminal. Figure 19 is a diagram showing a store information editing screen displayed on an administrator terminal.

As shown in FIG. 12, administrator ad1 uses administrator terminal 4 to access the service information management device 7 via communication network 100, causing the web browser (display control unit 44) of administrator terminal 4 to display each of the screens shown in FIGS. 13 to 19 on the display 106, and the transmission/reception unit 41 to transmit each piece of service information entered or selected by administrator ad1 on each screen to the service information management device 7 (S1). Then, in the service information management device 7, the transmission/reception unit 71 receives each piece of service information, and the storage/read processing unit 79 stores each piece of service information in the service information management DB 7001. As a result, each piece of information is managed in each table shown in FIGS. 9 to 11.

The screens are displayed in the order shown in Figures 13 to 19. For example, on the group information editing screen shown in Figure 18, administrator ad1 enters the account ID of the presenter of the presenter terminal 9 to which the store terminal 1 wishes to give priority in sending a call start request in the "Presenter" field for the subgroup (for unmanned stores). Furthermore, on the store information editing screen shown in Figure 19, the administrator enters the subgroup name (here, for unmanned stores) in the "Subgroup to call with priority" field. This allows the store terminal 1 to obtain service information from the service information management device 7 (see S19 in Figure 20 described below), thereby managing the presenter terminal 9 to which the store terminal 1 should give priority in sending a call start request (see S20).

<Video call preparation process>
Next, the process of preparing for a video call for remote customer service will be described with reference to Fig. 20 to Fig. 23. Fig. 20 is a sequence diagram showing the process of preparing for a call by the store terminal.

First, when a customer visiting a store turns on the power switch of the store terminal 1, the reception unit 12 receives the power-on signal and starts up the store terminal 1 (S11). When the store terminal 1 starts up, the communication app installed on the store terminal 1 starts up (S12). Thereafter, processing in the store terminal 1 is carried out according to commands from the communication app. The storage/read processing unit 19 then reads local setting information from the storage unit 1000 (S13). This completes the store terminal 1's preparation for setting the background image for the display screen, the login ID and password, etc.

Next, the transmitter/receiver 11 of the store terminal 1 sends a login request to the service information management device 7 via the communication network 100 (S14). This login request includes the login ID and password read in step S13. The transmitter/receiver 71 of the service information management device 7 then receives the login request. By sending the login request from the store terminal 1 to the service information management device 7, the service information management device 7, which is the receiving side, can obtain the IP address of the store terminal 1, which is the sending side.

Next, the determination unit 75 of the service information management device 7 searches the terminal management table (Figure 10(A)) of the service information management DB 7001 using the login ID and password included in the login request as search keys, and performs authentication by determining whether the same login ID and password are managed in this terminal management table (S15).

Then, the transmitter/receiver unit 71 of the service information management device 7 transmits authentication result information indicating the authentication result to the store terminal 1 (S16). As a result, the transmitter/receiver unit 11 of the store terminal 1 receives the authentication result information.

Next, the transmitter/receiver 11 of the store terminal 1 transmits a request for service information to the service information management device 7 (S17). As a result, the transmitter/receiver 71 of the service information management device 7 receives the request for service information.

Next, in the service information management device 7, the storage/read processing unit 79 reads the service information from the service information management DB 7001 (S18). Then, the transmission/reception unit 71 transmits the service information to the store terminal 1 (S19). As a result, the transmission/reception unit 11 of the store terminal 1 receives the service information. Next, in the store terminal 1, the storage/read processing unit 19 sets the service information (S20).

<Video call start process>
First, the creation unit 54 of the store terminal 1 creates a session ID for identifying the communication session (S22). Next, the transmission/reception unit 11 transmits login request information indicating a login request to the communication management device 5 via the communication network 100 (S23). This login request information includes the session ID created in step S22 and the communication ID and password of the store terminal 1 itself in the service information acquired in step S19. Note that the communication ID here is the initiating terminal ID for identifying the store terminal 1 that transmitted the login request in step S23.

Next, the storage/readout processing unit 59 of the communication management device 5 searches the authentication management table (see FIG. 7(A)) using the communication ID and password included in the login request information received via the transmission/reception unit 51 as search keys, and performs terminal authentication by determining whether the same communication ID and the same password are managed (S24). Here, the following explanation will continue assuming that the same communication ID and the same password are managed by the storage/readout processing unit 59.

If the storage/readout processing unit 59 determines that the login request is from a terminal with valid usage rights, assuming that the same communication ID and password are managed, the storage/readout processing unit 59 changes the operation status field of the record for the communication ID received in step S23 above to "Online (available for calls)" in the terminal status management table (see FIG. 7(B)), and stores the reception date and time when the login request was received in step S23 above in the reception date and time field (S25). As a result, for example, the terminal status management table will manage the communication ID "01aa" in association with the operation status "Online (available for calls)", the reception date and time "2019.4.10.13:40", and the IP address "1.2.1.4". Note that the terminal's IP address does not need to be registered in advance, but may be sent from the store terminal 1 in step S23 above.

Next, the storage/readout processing unit 59 adds a new record including the communication ID and password of the store terminal 1 received in step S23 to the session management table (see Figure 8 (B)) and manages it (S26). Then, the transmission/reception unit 51 of the communication management device 5 transmits authentication result information indicating the authentication result obtained by the processing of step 24 above via the communication network 100 to the store terminal 1 that made the login request (S27).

When the transmitter/receiver unit 11 of the login request terminal (store terminal 1) receives authentication result information indicating that the terminal has been determined to be authorized for use, the transmitter/receiver unit 11 transmits destination list request information indicating a request for a destination list to the communication management device 5 via the communication network 100 (S28). As a result, the transmitter/receiver unit 51 of the communication management device 5 receives the destination list request information.

Next, the storage/read processing unit 59 searches the destination list management table (see FIG. 8(A)) using the communication ID "01ab" of the login request terminal (store terminal 1) as a search key, reads out the communication IDs of destination candidates that can communicate with the login request terminal (store terminal 1), and reads out the destination names corresponding to these communication IDs from the terminal status management table (see FIG. 7(B)) (S29). Here, the communication IDs of the destination candidates that correspond to the communication ID "01ab" of the login request terminal (store terminal 1) and the corresponding destination names are extracted.

Next, the storage/readout processing unit 59 of the communication management device 5 reads the data in the destination list frame from the memory unit 5000 (S30). Then, the transmission/reception unit 51 transmits this destination list frame and the "destination list information (destination list frame, communication ID, destination name)" including the communication ID and destination name read by the storage/readout processing unit 59 to the login request terminal (store terminal 1) (S31). As a result, in the login request terminal (store terminal 1), the transmission/reception unit 11 receives the destination list information, and the storage/readout processing unit 19 stores the destination list information in the memory unit 1000 (S32).

In this way, in this embodiment, rather than managing the destination list information on each terminal, the communication management device 5 centrally manages the destination list information for all terminals. Therefore, even if a new terminal is added to the communication system, a new model of terminal is added to replace an existing terminal, or the appearance of the destination list frame needs to be changed, this can be handled collectively on the communication management device 5 side, eliminating the need to change the destination list information on each terminal side.

In addition, the storage/readout processing unit 59 of the communication management device 5 searches the terminal status management table (see Figure 7 (B)) using the communication IDs of the destination candidates read out above as a search key, and obtains the operating status of each communication terminal as a destination candidate by reading out the corresponding operating status for each communication ID (S33).

Next, the transmitter/receiver 51 transmits "terminal status information" containing each communication ID used as a search key in step S26 above and the operating status of each corresponding destination terminal to the initiating terminal (store terminal 1) via the communication network 100 (S34).

Next, the storage/readout processing unit 19 of the initiating terminal (store terminal 1) sequentially stores the terminal status information received from the communication management device 5 in the storage unit 1000 (S35). Therefore, by receiving the status information of each of the above terminals, the initiating terminal (store terminal 1) can obtain the current operating status of each of the presenter terminals 9 and other terminals that are potential destinations with which the initiating terminal (store terminal 1) can communicate.

Next, the display control unit 14 of the login request terminal (store terminal 1) creates a destination list that reflects the status of the terminal as a destination candidate based on the destination list information and terminal status information stored in the storage unit 1000 (S36). This destination list is later displayed on the store terminal 1 as shown in Figure 23 (C).

Meanwhile, the storage/readout processing unit 59 of the communication management device 5 searches the destination list management table (see Figure 8 (A)) based on the communication ID of the login request terminal (store terminal 1) to extract the communication IDs of other terminals that have the communication ID of the login request terminal (store terminal 1) registered as a destination candidate (S37).

Next, the storage/readout processing unit 59 of the communication management device 5 searches the terminal status management table (see Figure 7 (B)) based on the communication ID of the login request terminal (store terminal 1) and obtains the operating status of the login request terminal (store terminal 1) (S38).

Then, the transmitter/receiver unit 51 transmits "terminal status information" including the communication ID and operating status of the login request terminal (store terminal 1) acquired in step S38 to terminals associated with the communication IDs extracted in step S37 and whose operating status is "Online" in the terminal status management table (see FIG. 7(B)) (S39). When transmitting the terminal status information to the presenter terminal 9, the transmitter/receiver unit 51 references the IP addresses of the communication terminals managed in the terminal status management table (see FIG. 7(B)) based on each communication ID. This allows the communication ID and operating status of the login request terminal (store terminal 1) to be communicated to each of the other destination terminals that can communicate with the login request terminal (store terminal 1) as a destination candidate. This allows the destination candidate (presenter terminal 9) to display the status of the destination candidate as shown in FIG. 23(C) (S40).

<Process for determining the operating status of presenter terminals>
Next, a process in which the store terminal 1 determines the operating status of the explainer terminal group for each skill will be described with reference to FIG. 22 . FIG. 22 is a flowchart showing the preparation process for a video call by the store terminal, and in particular, the process of determining the operating status of the explainer terminal group for each skill. The operating status of a skill group "Online" indicates that at least one of the destination candidates in this skill group has a presence of "Online" that allows communication. The operating status of a skill group "Chat" indicates that none of the destination candidates in this skill group have an operating status of "Online," and that at least one destination candidate has a presence of "Chat" that is currently in communication. Furthermore, the operating status of a skill group "Offline" indicates that all of the destination candidates in this skill group have an operating status of "Offline." An example of the process for determining the operating status of a skill group will be described below.

In step S34 of FIG. 21, the storage/readout processing unit 19 of the store terminal 1 acquires the information recorded in the updated record each time the operating status of the destination management table is updated (S51). The acquired information includes the communication ID of the presenter terminal 9ug11, the operating status indicating the latest status of the presenter terminal 9ug11 that is a destination candidate, and the skill name indicating the skill of the presenter who uses the presenter terminal 9ug11 that is a destination candidate.

The storage/readout processing unit 19 of the store terminal 1 updates each record in the skill group management DB 1002 that includes the skill name acquired in step S51. That is, if the acquired skill name is "Specialty 1, Specialty 2," the storage/readout processing unit 19 updates each record in the skill group management DB 1002 that includes the skill name "Specialty 1, Specialty 2."

As an example, the following describes the processing when operating status information (operating status "Online", skill name (Specialty 1, Specialty 2), communication ID "01ba") is sent from the presenter terminal 9ug11 to the store terminal 1 and this information is acquired in step S51 (S52: Online).

First, we will explain the process of updating records in which the skill name "Specialty 1" is recorded in a table in the skill group management DB 1002 during loop processing. The storage/read processing unit 19 of the store terminal 1 references the field for communication IDs that can be used for calls in the record in the skill group management DB 1002 (see (B) in Figure 6) in which the skill name "Specialty 1" is recorded. As a result, the judgment unit 15 determines whether the communication ID "01ba" obtained in step S51 is included in the communication ID field (S53).

If it is determined that the communication ID "01ba" acquired in step S51 is not included in the call-enabled field in the skill group management DB 1002 (S53; NO), the storage/read processing unit 19 adds the acquired communication ID "01ba" to the call-enabled field (S54).

When the processing of step S54 is completed, or if the answer is YES in step S53, the storage/read processing unit 19 of the store terminal 1 references the "on call" field of the record in the skill group management DB 1002 (see (B) of Figure 6) in which the skill name "Specialty 1" is recorded. This causes the judgment unit 15 to determine whether the "on call" field contains the communication ID "01ba" obtained in step S51 (S55).

If it is determined that the communication ID "01ba" acquired in step S51 is included in the in-call field in the skill group management DB 1002 (S55; YES), the storage/read processing unit 19 deletes the acquired communication ID from the in-call field (S56). Through the processing up to this point, whether the operating state of the presenter terminal 9ug11 transitions from "Chat" to "Online" or from "Offline" to "Online," the communication ID of the presenter terminal 9ug11 is recorded only in the available-for-call field in the record in the skill group management DB 1002 in which the skill name "Specialty 1" is recorded.

When the processing of step S56 is completed, or if the answer is NO in step S55, the storage/readout processing unit 19 of the store terminal 1 references the operating status of the group in the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002 (see (B) of Figure 6). As a result, the judgment unit 15 determines whether the operating status of the group is "Online" (S57).

If it is determined that the operating status of the group in the record in which the skill name "Specialty 1" is recorded is not "Online" (S57; NO), the storage and retrieval processing unit 19 updates this operating status to "Online" (S58). As a result, when displaying the screen shown in FIG. 23 (A), the display control unit 14 displays a normal display that clearly looks like "Domestic Travel," for example. On the other hand, if it is determined that the operating status of the group in the record in which the skill name "Specialty 1" is recorded is "Online" (S57; YES), the storage and retrieval processing unit 19 does not update the operating status of the group in the record in which the skill name "Specialty 1" is recorded. As a result of the processing up to this point, the operating status of the group in the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002 becomes "Online."

In step S51, the skill names "Specialty 1, Specialty 2" are obtained, so the storage/read processing unit 19 similarly updates each record in the skill group management DB 1002 in which the skill name "Specialty 2" is recorded through loop processing.

Next, the operating status information (operating status "Chat", skill name (Specialty 1, Specialty 2), communication ID "01ba") is sent from the presenter terminal 9ug11 to the store terminal 1, and the processing when this operating status information is acquired in step S51 will be described (S52; Chat).

First, the loop process for updating the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002 will be described. The storage/read processing unit 19 of the store terminal 1 references the "Call Allowed" field in the record in which the skill name "Specialty 1" is recorded in the table of the skill group management DB 1002. As a result, the determination unit 15 determines whether the "Call Allowed" field contains the communication ID "01ba" acquired in step S51 (S63).
If it is determined that the communication ID "01ba" acquired in step S51 is included in the call-enabled field in the skill group management DB 1002 (S63; YES), the storage/readout processing unit 19 deletes the acquired communication ID "01ba" from the call-enabled field (S64).

If the processing of step S64 is completed, or if the answer is NO in step S63, the storage/read processing unit 19 of the store terminal 1 references the "on call" field of the record in the skill group management DB 1002 in which the skill name "Specialty 1" is recorded. This causes the judgment unit 15 to determine whether the "on call" field contains the communication ID "01ba" obtained in step S51 (S65).

If it is determined that the communication ID "01ba" acquired in step S51 is not included in the in-call field in the skill group management DB 1002 (S65; NO), the storage/read processing unit 19 adds the acquired communication ID "01ba" to the in-call field (S66). Through the processing up to this point, the communication ID of the presenter terminal 9ug11 is recorded only in the in-call field of the record in the skill group management DB 1002 in which the skill name "Specialty 1" is recorded.

When the processing of step S66 is completed, or if the answer is YES in step S65, the storage/read processing unit 19 of the store terminal 1 references the "call available" field in the record in the skill group management DB 1002 in which the skill name "Specialty 1" is recorded. As a result, the storage/read processing unit 19 determines whether the number of communication IDs recorded in the "call available" field in the record in which the skill name "Specialty 1" is recorded is 0 (zero) (S67).

If the number of communication IDs recorded in the "Calls Allowed" field in the record in which the skill name "Specialty 1" is recorded is 0 (zero) (S67; YES), the storage and retrieval processing unit 19 updates the operating status of the group in the record in which the skill name "Specialty 1" is recorded to "Chat" (S68). As a result, when displaying the screen shown in FIG. 23(A), the display control unit 14 displays the screen in a semi-transparent manner, such as "Wedding." On the other hand, if the number of communication IDs recorded in the "Calls Allowed" field in the record in which the skill name "Specialty 1" is recorded is not 0 (zero) (S67; NO), the storage and retrieval processing unit 19 does not update the operating status of the group in the record in which the skill name "Specialty 1" is recorded. In other words, if a communication ID is recorded in the "Calls Allowed" field in the record in which the skill name "Specialty 1" is recorded, the corresponding operating status remains "Online" and is not updated.

In step S51, the skill names "Specialty 1, Specialty 2" are obtained, so the storage/read processing unit 19 similarly updates each record in the skill group management DB 1002 in which the skill name "Specialty 2" is recorded through loop processing.

Next, we will explain the case where operating status information (operating status "Offline", skill names "Specialty 3, Specialty 4", communication ID "01ca") is sent from the presenter terminal 9ug21 to the store terminal 1, and this operating status information is acquired in step S51 (S52; Offline).

First, we will explain the process of updating the record in the skill group management DB 1002 in which the skill name "Specialty 3" is recorded in the loop processing. The storage/read processing unit 19 of the store terminal 1 references the "Calls Allowed" field in the record in the skill group management DB 1002 table in which the skill name "Specialty 3" is recorded. As a result, the judgment unit 15 determines whether the "Calls Allowed" field contains the communication ID "01ca" obtained in step S51 (S73).

If it is determined that the communication ID "01ca" acquired in step S51 is included in the call-enabled field in the skill group management DB 1002 (S63; YES), the storage/read processing unit 19 deletes the acquired communication ID "01ca" from the call-enabled field (S74).

If the processing of step S74 is completed, or if the answer is NO in step S73, the storage/read processing unit 19 of the store terminal 1 references the "on call" field of the record in the skill group management DB 1002 table in which the skill name "Specialty 3" is recorded. As a result, the judgment unit 15 determines whether the "on call" field contains the communication ID "01ca" obtained in step S51 (S75).

If it is determined that the communication ID "01ca" acquired in step S51 is included in the in-call field in the skill group management DB 1002 (S75; YES), the storage/read processing unit 19 deletes the acquired communication ID "01ca" from the in-call field (S76). Through the processing up to this point, the communication ID of the presenter terminal 9ug21 is deleted from the record in the skill group management DB 1002 in which the skill name "Specialty 3" is recorded.

If the processing of step S76 is completed, or if the answer is NO in step S75, the storage/readout processing unit 19 of the store terminal 1 references the Available for Calls and In-Call fields in the record in the skill group management DB 1002 in which the skill name "Specialty 3" is recorded. As a result, the storage/readout processing unit 19 determines whether conditions IF1 and IF2 are met (S77). Condition IF1 is when the number of communication IDs recorded in the Available for Calls field in the record in which the skill name "Specialty 1" is recorded is 0 (zero), and the number of communication IDs recorded in the In-Call field is not 0 (zero). Condition IF2 is when the number of communication IDs recorded in the Available for Calls field in the record in which the skill name "Specialty 3" is recorded is 0 (zero), and the number of communication IDs recorded in the In-Call field is 0 (zero).

If the number of communication IDs recorded in the "Available for Calls" field in the record containing the skill name "Specialty 3" is 0 (zero) and the number of communication IDs recorded in the "In Call" field is not 0 (zero) (S77; IF1), the storage/read processing unit 19 updates the operating status of the group in the record containing the skill name "Specialty 3" in the skill group management DB 1002 to "Chat" (S78). If the number of communication IDs recorded in the "Available for Calls" field in the record containing the skill name "Specialty 3" is 0 (zero) and the number of communication IDs recorded in the "In Call" field is 0 (zero) (S77; IF2), the storage/read processing unit 19 updates the operating status of the group in the record containing the skill name "Specialty 3" in the skill group management DB 1002 to "Offline" (S79). As a result, when displaying the screen shown in FIG. 23(A), the display control unit 14 displays the screen in a semi-transparent manner, such as "Wedding." On the other hand, if neither IF1 nor IF2 applies in step S77 (S77; NO), the storage/readout processing unit 19 does not update the operating status of the group in the record in which the skill name "Specialty 3" is recorded, leaving it as "Online."

In step S51, the skill names "Specialty 3, Specialty 4" are obtained, so the storage/read processing unit 19 performs a loop process to similarly update each record in the skill group management DB 1002 in which the skill name "Specialty 4" is recorded.

<Video communication start process>
Next, the process of starting video communication for remote customer service will be described with reference to Fig. 23 and Fig. 24. Fig. 23 is a diagram showing a screen displayed on a store terminal. Fig. 24 is a sequence diagram showing the process of starting video communication.

At the store terminal 1, the display control unit 14 displays one of the screens shown in Figures 23(A) to 23(D) on the display 106 of the store terminal 1, and the reception unit 12 receives the customer's selection of a skill group or the designation of an instructor (S82).

Specifically, the storage/readout processing unit 19 acquires the combination of each skill name stored in the table of the skill group management DB 1002 (see FIG. 6(B)) and the operating status of the skill group corresponding to this skill name. As a result, the display control unit 14 displays the skill selection screen shown in FIG. 23(A) on the display 106 of the store terminal 1.

On the right side of the skill selection screen, skill display buttons (domestic travel, international travel, etc.) for selecting a skill and a presenter list display button for displaying a list of all presenter destinations are displayed. Here, the "domestic travel" skill display button is displayed normally and is selectable by the customer. In contrast, the "international travel" skill display button is displayed translucently and is not selectable by the customer. For example, when a customer selects the "domestic travel" skill display button, the reception unit 12 accepts the selection, and the display control unit 14 displays the confirmation screen shown in FIG. 23(B) on the display 106. When the customer presses the "OK" button, the reception unit 12 accepts the press, and the transmission/reception unit 11 sends connection request information indicating a connection request for a communication session for a video call to the desired presenter terminal 9 via the communication management device 5 (see S84-1 and S85-1 described below).

On the other hand, when a customer presses the presenter list display button on the skill selection screen shown in FIG. 23(A), the reception unit 12 accepts the press, and the display control unit 14 displays a destination list of presenters on the display 106, as shown in FIG. 23(C), based on the presenter names, the operating status of the presenter terminals, and the skill names stored in the table of the presenter management DB 1001 (see FIG. 6(A)). Here, the customer selects the name of the presenter who uses the desired presenter terminal from among the presenter terminals available for communication, and presses the "OK" button. The reception unit 12 accepts the presenter's selection, and the display control unit 14 displays the confirmation screen shown in FIG. 23(D) on the display 106. Then, when the customer presses the "OK" button, the reception unit 12 accepts the press, and the transmission/reception unit 11 sends a request to start a video call to the designated presenter terminal 9 via the communication management device 5 (see the processing of S90, described below, in FIG. 25).

Next, in the store terminal 1, when the customer selects a desired skill group (e.g., an organization that can explain domestic travel) on the skill selection screen of FIG. 23(A), the judgment unit 15 determines whether to send a request to start a video call to an online (call-enabled) presenter terminal in a specific skill group (e.g., domestic travel) in the unmanned store priority response organization UG1, or to send a request to start a video call to an online (call-enabled) presenter terminal in the same specific skill group (e.g., domestic travel) in the manned store priority response organization MG1 (S83). In this case, the store terminal 1 makes a judgment based on the subgroup name to which its own terminal's account ID belongs in the subgroup management table (see FIG. 11(B)) in the service information received in step S19 above. If its own terminal (store terminal 1) belongs to an unmanned store, the judgment unit 15 determines the destination of the start request to be a specific skill group in the unmanned store priority response organization UG1. On the other hand, if the terminal (store terminal 1) belongs to a staffed store, the determination unit 15 determines the destination of the start request to be a specific skill group in the staffed store priority response organization MG1.

Next, in Figure 1, we will explain the case where, when the store terminal 1 is the store terminal 1c2 installed in the unmanned store US, a request to start a video call is sent (first time) to the online (call-enabled) presenter terminals 9ug11 and 9ug12 of the organization SR21 responsible for skill SK1 of the unmanned store priority response organization UG1.

First, as a preliminary step to a request to start a video call (see S101 in FIG. 25), the transmitter/receiver 11 of the store terminal 1c2 simultaneously transmits connection request information indicating a connection request for a communication session for the video call for the first time (S84-1). This connection request information includes the communication IDs read in step S83 that have a corresponding operating status of "Online," the communication ID of the store terminal 1c2 that originated the start request, and the skill name indicating the skill selected in step S82.

Next, when the transmitter/receiver unit 51 of the communication management device 5 receives the connection request information transmitted by the store terminal 1c2, it transmits this connection request information to the presenter terminal 9ug11 identified by the communication ID contained in the received connection request information (S84-2).

If the operating status is updated in step S34 (see FIG. 21) between when the store terminal 1c2 sends the connection request information and when it receives a response, the store terminal 1c2 repeats the processes of steps S82 and S83. As a result, the operating status of the presenter terminal 9 is updated to "Online," and if a communication ID is extracted as a communication ID whose corresponding operating status is "Online," the transmitter/receiver 11 of the store terminal 1c2 sends connection request information including the communication ID to the communication management device 5 (S85-1). Note that the connection request information sent here also includes the skill name indicating the skill selected in step S82 and the communication ID of its own terminal.

When the transmitter/receiver unit 51 of the communication management device 5 receives the connection request information transmitted by the store terminal 1, it transmits this connection request information to the presenter terminal 9ug12 identified by the communication ID contained in the received connection request information (S85-2).

The transmitter/receiver 91 of each presenter terminal 9ug11, 9ug12 receives connection request information transmitted from the store terminal 1c2 via the communication management device 5. When this connection request information is received, the display control unit 94 of each presenter terminal 9ug11, 9ug12 displays on the display 106 of each presenter terminal 9ug11, 9ug12 a message including request source information corresponding to the communication ID included in the connection request information and a skill name (skill information) corresponding to the skill name included in the call start request. By indicating the skill in the message, the presenter at each presenter terminal 9ug11, 9ug12 can determine in advance what kind of travel product the customer is looking for.

Next, when each presenter ug11, ug12 performs an operation input to allow a response at each presenter terminal 9ug11, 9ug12, the reception unit 92 of each presenter terminal 9ug11, 9ug12 accepts the operation input to allow a response (S86-1, S86-2). When the operation input to allow a response is accepted, the transmission/reception unit 91 of each presenter terminal 9ug11, 9ug12 transmits response information indicating that connection is permitted to the communication management device 5 (S87-1, S87-2). This response information includes the communication ID of the sender (store terminal 1c2) and the communication ID of the presenter's own terminal (9ug11, 9ug12). As a result, the transmission/reception unit 51 of the communication management device 5 receives the response information transmitted by each presenter terminal 9ug11, 9ug12. Note that the above process is an example, and the transmitter/receiver unit 91 of each presenter terminal 9ug11, 9ug12 may automatically send response information to the store terminal 1c2 in response to the connection request when it receives connection request information from the store terminal 1c2. Furthermore, when the transmitter/receiver unit 11 of each presenter terminal 9ug11, 9ug12 receives multiple pieces of connection request information from multiple store terminals 1, it may send the response information to the store terminal 1 that sent the connection request information that was received first.

Next, the transmitter/receiver 51 of the communication management device 5 transmits the received response information to the store terminal 1c2 identified by the communication ID included in the response information (S87-2, S88-2).

If the transmitter/receiver unit 11 of the store terminal 1c2 receives multiple pieces of response information, the judgment unit 15 judges the presenter terminal 9 that sent the earliest response information (S89). Hereinafter, in this embodiment, we will explain the case where the presenter terminal 9ug11 responds first.

The transmitter/receiver unit 51 of the communication management device 5 controls the establishment of a communication session for transmitting image data and audio content data via the relay device 30 between the presenter terminal 9ug11 and the store terminal 1, identified by the communication IDs "store terminal communication ID, presenter communication ID" included in the communication establishment request, to conduct a video call (S90). This step S90 will be described in detail later using FIG. 25. Note that the communication session is not limited to the above and may be a session for directly transmitting content data between the presenter terminal 9ug11 and the store terminal 1c2 via the communication network 100. Once communication is established between the store terminal 1c2 and the presenter terminal 9ug11, each store terminal 1c2 and the presenter terminal 9ug11 transmits image data of images captured at the respective terminals and audio data collected at the respective terminals to the other terminal via the relay device 3. This allows a video call to be initiated between the customer and the presenter, as shown in FIG. 2.

When communication is established in step S90, the transmitter/receiver 91 of the presenter terminal 9ug11 transmits status information including the operating status "Chat", the skill name "Specialty 1", and the communication ID of the presenter terminal 9ug11 to the communication management device 5.

Furthermore, once communication is established in step S90, the transceiver unit 11 of the store terminal 1c2 sends cancellation information to the communication management device 5 to cancel the call start request for the sender of the response other than the sender of the first-received response (S91-1). The cancellation information includes the communication ID of the store terminal 1c2 itself and the communication ID of the presenter terminal 9 (here, the presenter terminal 9ug12) that sent the response other than the sender of the first-received response.

When the transmitting/receiving unit 51 of the communication management device 5 receives the cancellation information, it transmits the cancellation information to the presenter terminal 9ug12 identified by the communication ID included in the cancellation information (S91-2). The transmitting/receiving unit 11 of the presenter terminal 9ug12 receives the cancellation information transmitted by the communication management device 5. This allows the presenter terminal 9ug12 to understand that it is no longer necessary to respond to the call start request from the store terminal 1. <Communication Establishment Control Process>
Next, the communication establishment control process (see S90) in Fig. 24 will be described in detail with reference to Fig. 25. Fig. 25 is a sequence diagram showing the detailed communication establishment control process.

First, the store terminal 1c2 sends start request information indicating a request to start video communication to the communication management device 5 (S101). This start request information includes the communication ID of the start terminal (here, the store terminal 1c2) and the communication ID of the presenter ug11 of the destination terminal (here, the presenter terminal 9ug11). As a result, the communication management device 5 receives the start request information.

Next, in the communication management device 5, the selection unit 53 selects the nearest relay device based on the IP addresses of the initiating terminal and the destination terminal (S102). The IP addresses of each terminal are managed in a terminal management table, and the IP addresses of each relay device are pre-stored in the storage unit 5000. Next, the creation unit 54 creates communication information including the IP address of the relay device and the above-mentioned communication ID (S103). The storage/readout processing unit 59 then manages the communication information by storing the communication ID and IP address of the relay device, which are the contents of the communication information, as well as the communication IDs of the presenters of the initiating terminal and the destination terminal received in step S101 in the communication information management DB 5005 (see Figure 8(C)), in association with each other. Thereafter, the transmission/reception unit 51 transmits start request information to the destination terminal (here, the presenter terminal 9ug11) (S105). This start request information includes the communication information in addition to the communication IDs sent from the initiating terminal in step S101. As a result, the transmission/reception unit 91a of the destination terminal receives the start request information.

Next, at the destination terminal, the determination unit 95 of the presenter terminal 9ug11 determines whether the communication ID of the initiating terminal (store terminal 1c2) received in step S105 is the same as the communication ID of the store terminal 1c2 already received in the processing of step S84-2 in FIG. 24 (S106). In this case, since they are essentially the same, the transmission/reception unit 91a transmits response possible information to the communication management device 5 (S107). In this case, the response possible information indicates that a response is possible and includes the communication ID of the initiating terminal. As a result, the transmission/reception unit 51 of the communication management device 5 receives the response possible information.

Next, the storage/read processing unit 59 of the communication management device 5 searches the communication information management DB 5005 using the communication ID of the initiating terminal received in step S107 as a search key to read out the corresponding communication information (communication ID, relay device IP address) (S108). The transmission/reception unit 51 then transmits response possible information to the initiating terminal. In this case, the response possible information includes the communication information read out in step S108. As a result, the transmission/reception unit 11 of the initiating terminal receives the response possible information.

Next, the transmitter/receiver 91 of the destination terminal transmits establishment request information indicating a request to establish a mutual communication session to the IP address of the relay device included in the communication information received in step S105 (S110). This establishment request information includes the communication ID of the initiating terminal and the communication ID of the presenter of the destination terminal. The IP address of the destination terminal, which is the sender, is also transmitted. Meanwhile, the transmitter/receiver 11 of the initiating terminal transmits establishment request information indicating a request to establish a mutual communication session to the IP address of the relay device included in the communication information received in step S109 (S111). This establishment request information includes the communication ID of the initiating terminal and the communication ID of the presenter of the destination terminal. The IP address of the initiating terminal, which is the sender, is also transmitted. As a result, a communication session for mutual communication of images and sound is established between the relay device 3 and the initiating terminal (S112-1), and a communication session for mutual communication of images and sound is also established between the relay device 3 and the destination terminal (S112-2).

Next, the transmitter/receiver 11 of the initiating terminal transmits status information of its own terminal (initiating terminal) to the communication management device 5 (S113-1). This status information includes the communication ID of the initiating terminal and operating status information (here, "in communication"). As a result, the transmitter/receiver 51 of the communication management device 5 receives the status information of the initiating terminal. Meanwhile, the transmitter/receiver 91 of the destination terminal transmits status information of its own terminal (destination terminal) to the communication management device 5 (S113-2). This status information includes the communication ID of the destination terminal and operating status information (here, "Chat (in communication)"). As a result, the transmitter/receiver 51 of the communication management device 5 receives the status information of the destination terminal. In this way, each terminal notifies the communication management device 5 of its own operating status. Then, in the communication management device 5, the storage/read processing unit 59 changes the operating status associated with the communication ID of the initiating terminal to "Chat (in communication)" in the terminal management table (see FIG. 10), and also changes the operating status associated with the communication ID of the destination terminal to "Chat (in communication)" (S114). As a result, the initiating terminal (store terminal 1c2) and the destination terminal (presenter terminal 9ug11) can conduct a video call via mutual communication, as shown in FIG. 2. This is the processing performed in step S82 when a skill group such as "Domestic Travel" is selected in FIG. 23 (A).

<Request to start a video call when a presenter is designated>
Next, a process of starting a video call when a presenter is designated in step S82 in Fig. 23C will be described. In this case, only the process of step S90 is executed among the processes after step S84 in Fig. 24.

Furthermore, when a presenter is designated, the processes of steps S84-2 and S85-2 in FIG. 24 are not performed, and therefore in step S106 in FIG. 25, the display control unit 94 displays the communication ID of the starting terminal, allowing the designated presenter to manually select whether or not to respond. Then, when the designated presenter selects whether or not to respond, the reception unit 92 accepts the selection.

Then, in steps S107 and S109, response availability information is sent instead of response availability information. In this case, the communication ID of the initiating terminal is also included. The rest of the processing is the same as when a skill group is selected, so a description is omitted.

<Processing for changing the destination of a video call start request>
Next, referring to Fig. 26, a process for changing the destination of a video call start request when none of the presenter terminals responds to the start request in steps S86-1 and S86-2 of Fig. 24 is described. Fig. 26 is a flowchart showing various processes for starting a call. Specifically, Fig. 26 is a flowchart showing three patterns: a start request after a skill group is selected, a change in the destination of the start request, and a start request when a presenter is designated.

The judgment made by the judgment unit 15 below is based on the information managed in the presenter management DB 1001 and skill group management DB 1002 of the memory unit 1000.

First, in the store terminal 1c2 of the unmanned store US, as a result of the processing of step S82 shown in FIG. 24, the judgment unit 15 determines whether a skill group is selected or a presenter is designated (S201). If a skill group is selected (S201; Skill), the judgment unit 15 determines whether there is an online (call-enabled) presenter terminal of an unmanned store priority organization in the same area E1 for the accepted skill group (S202). If there is an online (call-enabled) presenter terminal of an unmanned store priority organization (S202; YES), the transceiver unit 11 of the store terminal 1c2 transmits connection request information indicating a connection request for a communication session for a video call to each online (call-enabled) presenter terminal of the unmanned store priority organization simultaneously for the first time (S203), as a preliminary step to a request to start a video call (see S101 in FIG. 25). The processing of step S203 corresponds to the processing of steps S84-1 and S85-1 in FIG. 24.

Next, after steps S84-1 and S85-1 are processed, the judgment unit 15 determines whether response information is received from each of the presenter terminals 9ug11 and 9ug12 within a certain period of time (e.g., 30 seconds) (S87-2, S88-2) (S204). If response information is received within the certain period of time (S204; YES), the store terminal 1c2 performs communication establishment control with the presenter terminal that responded first (here, presenter terminal 9ug11) (S205). The processing of step S205 corresponds to the processing of steps S89 and S90 in Figure 24. In this case, the processing of Figure 26 ends.

On the other hand, if response information indicating a connection failure is received in step S202 (NO), or if response information indicating a connection failure is not received within a predetermined time (e.g., 30 seconds) after sending the connection request information in step S204 (NO), the transceiver unit 11 of the store terminal 1c2 simultaneously sends the connection request information for the second time to each presenter terminal 9mg11, 9mg12 of the online (call-enabled) staffed store priority organization (S206). After processing steps S84-1 and S85-1 for the second time, the judgment unit 15 judges whether response information (S87-2, S88-2) has been received from each presenter terminal 9mg11, 9mg12 within the predetermined time (S207; YES). If response information has been received within the predetermined time (S207; YES), in step S205, the store terminal 1c2 performs communication establishment control with the presenter terminal that responded first (here, presenter terminal 9mg11). If no response information is received within the specified time (S207; NO), the display control unit 14 displays an error dialog on the display 106 of the terminal itself indicating that there is no response (S208). Then, the process returns to step S201.

In the meantime, in step S201 above, if a presenter is designated (S201; Designation), the display control unit 14 displays the destination list of presenters shown in FIG. 23(C) on the display 106 of the terminal (S209). Then, the reception unit 12 receives the designation (selection) of a presenter from customer c2 (S210). The processing of steps S209 and S210 corresponds to the processing of step S82 in FIG. 24.

As a result, the store terminal 1c2 executes communication establishment control processing only for the designated presenter terminal (here, presenter terminal 9ug11) (S211). The processing of this step S211 corresponds to the processing of step S90 in Figure 24.

<Example of a screen during a video call between the presenter's device and the store's device>
27 is an example of a call screen displayed on the presenter terminal. FIG. 27 is a diagram showing the call screen of the presenter terminal. The call screen 930 of the presenter terminal includes a main image field 940, a sub-image field 950, and a button display field 960. The button display field 960 displays a hold button 961, a travel plan button 962, a display switch button 963, a camera button 964, a microphone button 965, and a hang up button 966. The main image field 940 is displayed across the entire call screen 930 of the presenter terminal 9, and the sub-image field 950 is displayed in a portion of the call screen 930 (here, the lower right) smaller than the main image field 940.

The hold button 961 is a button that causes the terminal device to stop transmitting images and audio while maintaining communication. It is used when the presenter will be away from their seat for a short period of time. A message that the hold button 961 has been pressed is sent to the store terminal 1. When the reception unit 92 of the presenter terminal 9 detects that the hold button has been pressed, it reads a predetermined image from the memory unit and displays it on the display of the presenter terminal 9. For example, by displaying a message such as "The presenter has left their seat," customers are prevented from feeling uncomfortable. The image to be displayed can be set by store staff, so advertisements, videos, etc. can also be displayed.

The Travel Plan button 962 is a function that activates a mode for creating a travel plan while listening to the customer's requests.

The display switch button 963 can be used to hide the image of the customer displayed in the main image field 940 and switch to displaying only the background image. When the presenter creates a travel plan with the customer after pressing the travel plan button 962, this function temporarily hides the image of the customer so that the customer can see the image of the appropriate location when displaying an image of the local area (spherical image) 941, as shown in Figure 30, for example.

The camera button 964 and microphone button 965 are buttons that allow the presenter to switch the camera and mute the microphone on their presenter terminal 9.

The disconnect button 966 is a button for disconnecting communication between terminals (presence changes from Chat to Online). Only the presenter can disconnect communication.

Next, we will explain the call screen displayed by the store terminal using Figure 28. Figure 28 is an example of a call screen displayed on a store terminal. As shown in Figure 28, the call screen 130 displayed on the store terminal 1 includes a main image field 140, a sub-image field 150, and a button display field 160. The button display field 160 displays a display switch button 163, a camera button 164, a microphone button 165, and a hang-up button 166. The display switch button 163, the camera button 164, the microphone button 165, and the hang-up button 166 perform the same functions as the display switch button 963, the camera button 964, the microphone button 965, and the hang-up button 966 in Figure 27. The customer can only display the sub-image field 150 (where the customer's own face is displayed), switch between displaying and hiding the presenter displayed in the main image field 140, and operate the microphone and camera. The main image field 140 is displayed across the entire call screen 180 of the store terminal 1, and the sub-image field 150 is displayed in a portion of the call screen 180 (here, the bottom right) and is smaller than the main image field 140.

<Travel plan suggestions>
Next, we will explain the process in which the presenter proposes a travel plan to a customer via remote communication. While serving the customer (guest), the presenter asks the customer (guest) about their desired travel destination, and then inputs the country or region from a world map, or the tourist spot specified by the customer (guest). After inputting, the map is displayed on the screen. When a tourist spot within the corresponding map area is selected, an image (spherical video) is displayed in the background image, allowing the visitor to create a travel plan while experiencing the local atmosphere. This will be explained in detail below.

Figure 29 is a sequence diagram showing the process of creating a travel plan when a presenter terminal and a store terminal start a video call from a dedicated application. Figure 30 is an example of a call screen (including an input window and map window) displayed on the presenter terminal. Figure 31 is an example of a call screen (including an input window and map window) displayed on the presenter terminal.

First, as shown in Figures 27 and 28, a video call is being conducted between the store terminal 1 and the presenter terminal 9. When the presenter creates a travel plan with a customer, the presenter presses the Travel Plan button 962 displayed on the call screen of the presenter terminal shown in Figure 27. The reception unit 92 accepts the opening of the travel plan mode and activates the travel plan creation mode. When this "travel plan creation mode" is activated, as shown in Figure 30, the display control unit 94 displays an input window 970 in the upper left of the main image field 940 of the call screen 930, allowing the customer to input their desired tourist destinations, accommodations, stay dates, number of people, number of rooms, etc., and a map window 980 in the lower left, showing landmarks of the tourist destinations entered in the input window 970. Here, the presenter asks the customer about their desired tourist destinations via remote call and enters the conditions according to the customer's requests in the fields displayed in the input window 970. The reception unit 92 then accepts the input of the conditions according to the customer's requests. For example, the presenter enters "Okinawa," "Okinawa Prefecture," and "Remote Island 1" as the tourist destinations where the customer will stay.

This input causes the display control unit 94 to display a map corresponding to remote island 1 in the map window 980, as shown in Figure 31. This map displays multiple landmarks indicating areas that the presenter can introduce. In this case, each hotel on the remote islands, centered around Miyako Island in Okinawa Prefecture, is displayed as a landmark.

Next, the presenter selects a specific landmark 981 in the map window 980 while listening to the customer's requests, and the reception unit 92 accepts the selection of the tourist attraction associated with the specific landmark 981 (S302). Once the specific landmark 981 is selected, the transmission/reception unit 91 of the presenter terminal 9 transmits a request for an image of the tourist attraction to the communication management device 5 via the communication network 100 (S303). At this time, the request transmitted from the presenter terminal 9 includes a landmark ID for identifying the specific landmark 981. As a result, the transmission/reception unit 51 of the communication management device 5 receives the request transmitted in step S317.

Next, in the communication management device 5, the storage/read processing unit 59 reads the file name of the tourist spot image corresponding to the landmark ID stored in the image information management DB 5006 (see Figure 8 (D)) based on the landmark ID included in the request received from the presenter terminal 9 by the transmission/reception unit 51, and reads the image data of the tourist spot with this file name from the storage unit 5000 (S304). The transmission/reception unit 51 of the communication management device 5 then transmits the image data of the tourist spot to the presenter terminal 9 (S305) and also to the store terminal 1 (S306). As a result, the transmission/reception unit 91 of the presenter terminal 9 and the transmission/reception unit 11 of the store terminal 1 each receive the image data of the tourist spot.

Figure 32 shows an example of a call screen (including an input window, map window, and an image of a tourist spot) displayed on a presenter terminal. Figure 33 shows an example of a call screen (including an image of a tourist spot) displayed on a store terminal.

As a result of step S305 above, as shown in FIG. 32, the display control unit 94 of the presenter terminal 9 displays an image of the tourist destination as a background image on the display 106 of the presenter terminal 9 (S307). In this case, the display control unit 94 displays not only the background image of the main image field 940 in which the image of the customer is displayed, but also the background image of the sub-image field 950 in which the presenter himself is displayed. Meanwhile, as a result of step S306 above, as shown in FIG. 33, the display control unit 14 of the store terminal 1 displays an image of the tourist destination as a background image on the display 106 of the store terminal 1 (S308). In this case, the display control unit 14 displays not only the background image of the main image field 140 in which the image of the presenter is displayed, but also the background image of the sub-image field 150 in which the customer himself is displayed.

This allows users, both presenters and customers, to share the same background image during a video call, allowing them to have lively conversations about tourist spots.

Furthermore, the presenter asks the customer via remote call about their desired accommodation (here, "XXX Hotel") and inputs the conditions according to the customer's requests into the fields displayed in the input window 970. The reception unit 92 then accepts the input of the conditions according to the customer's requests (here, accommodation) (S309).

Next, the transmitter/receiver unit 91 of the presenter terminal 9 sends a request for an image of the exterior of the accommodation facility to the communication management device 5 via the communication network 100 (S310). At this time, the request sent from the presenter terminal 9 includes the accommodation facility information (here, "XXX Hotel") as the condition accepted in step S309. As a result, the transmitter/receiver unit 51 of the communication management device 5 receives the request sent in step S310.

Next, in the communication management device 5, based on the accommodation facility information included in the request received from the presenter terminal 9 by the transmission/reception unit 51, the storage/readout processing unit 59 reads the file name of the image of the accommodation facility's exterior that corresponds to the accommodation facility information stored in the image information management DB 5006 (see FIG. 8(D)), reads image data of the accommodation facility's exterior with this file name from the storage unit 5000, and reads room information corresponding to the accommodation facility information received in step S310 from the image information management DB 5006 (S311). The transmission/reception unit 51 of the communication management device 5 then transmits the image data of the accommodation facility's exterior and the room information read in step S311 to the presenter terminal 9 (S312), and also transmits it to the store terminal 1 (S313). As a result, the transmission/reception unit 91 of the presenter terminal 9 and the transmission/reception unit 11 of the store terminal 1 receive the image data of the accommodation facility's exterior and the room information, respectively.

Figure 34 is an example of a call screen (including an input window, map window, an image of the accommodation facility's exterior, and a price list) displayed on the presenter terminal. Figure 35 is an example of a call screen (including an image of the accommodation facility's exterior, and a price list) displayed on a store terminal.

As a result of step S312 above, as shown in FIG. 34, the display control unit 94 of the presenter terminal 9 displays an image of the exterior of the accommodation facility as a background image on the display 106 of the presenter terminal 9, and also displays a price list 990 based on room information (S314). In this case, the display control unit 94 displays not only the background image of the main image field 940 in which an image of the customer is displayed, but also the background image of the sub-image field 950 in which the presenter himself is displayed. Meanwhile, as a result of step S313 above, as shown in FIG. 35, the display control unit 14 of the store terminal 1 displays an image of the exterior of the accommodation facility as a background image on the display 106 of the store terminal 1, and also displays a price list 190 based on room information (S315). In this case, the display control unit 14 displays not only the background image of the main image field 140 in which an image of the presenter is displayed, but also the background image of the sub-image field 150 in which the customer himself is displayed.

This allows users, both presenters and customers, to share the same background image while making video calls, allowing them to have lively conversations about accommodations.

Furthermore, the presenter asks the customer via remote communication for the desired room grade (here, "Deluxe") and accepts the selection of the room grade (here, "Deluxe") to be displayed on the price list 990 (S316).

Next, the transmitter/receiver unit 91 of the presenter terminal 9 transmits a request for an image of the room accepted in step S316 to the communication management device 5 via the communication network 100 (S317). At this time, the request transmitted from the presenter terminal 9 includes a room ID indicating the grade of the room accepted in step S316. As a result, the transmitter/receiver unit 51 of the communication management device 5 receives the request transmitted in step S317.

Next, in the communication management device 5, the storage/read processing unit 59 reads the file name of the room image corresponding to the image ID stored in the image information management DB 5006 (see Figure 8 (D)) based on the room ID included in the request received from the presenter terminal 9 by the transmission/reception unit 51, and reads the image data of the room with this file name from the storage unit 5000 (S318). Then, the transmission/reception unit 51 of the communication management device 5 transmits the room image data read in step S318 to the presenter terminal 9 (S318) and also to the store terminal 1 (S320). As a result, the transmission/reception unit 91 of the presenter terminal 9 and the transmission/reception unit 11 of the store terminal 1 each receive the room image data.

Figure 36 is an example of a call screen (including an input window, map window, room image, and price list) displayed on the presenter terminal. Figure 37 is an example of a call screen (including a room image and price list) displayed on the store terminal.

As a result of step S319 above, as shown in FIG. 36, the display control unit 94 of the presenter terminal 9 displays an image of the room (here, "Deluxe") as a background image on the display 106 of the presenter terminal 9 (S321). In this case, the display control unit 94 displays the image not only as a background image in the main image field 940 where the image of the customer is displayed, but also as a background image in the sub-image field 950 where the presenter himself is displayed. Meanwhile, as a result of step S320 above, as shown in FIG. 36, the display control unit 14 of the store terminal 1 displays the image of the room as a background image on the display 106 of the store terminal 1 (S322). In this case, the display control unit 14 displays the image of the room as a background image not only as a background image in the main image field 140 where the image of the presenter is displayed, but also as a background image in the sub-image field 150 where the customer himself is displayed.

This allows each user, acting as the presenter and the customer, to share the same background image during a video call, allowing them to have lively conversations about the room.

[Major Effects of the Present Embodiment]
As described above, according to this embodiment, the presenter allows the customer (guest) to virtually experience the atmosphere of the local area where the trip is planned. Furthermore, because the local area images can be viewed from various viewpoints using spherical video, the customer can feel the local area more realistically than through photographs while making plans, providing a sense of security. This allows the presenter and the customer users to share the same background image displayed across the entire main image field 140, 940 during a video call, allowing them to have lively conversations about the trip.

Furthermore, even if a first call start request is made from the store terminal US of an unmanned store to each presenter terminal 9ug11, 9ug12 of the organization SK12 responsible for the desired skill SK1 in the unmanned store priority organization UG1 and no response is received from any of the presenter terminals, a second call start request can be made to each presenter terminal 9mg11, 9mg12 of the organization SR11 responsible for the same skill SK1 in the manned store priority organization MG1. This allows for the benefit of unmanned stores, such as streamlined and efficient store operations, while reducing customer exits from unmanned stores by installing unmanned store priority presenter terminals 9ug11, 9ug12. Furthermore, even if the unmanned store priority presenter terminals 9ug11, 9ug12 cannot respond, the ability to make a call start request from the store terminal to the manned store priority presenter terminals 9mg11, 9mg12 further reduces customer loss.

〔supplement〕
In the above embodiment, the processes of steps S302, S309, and S316 are performed, but at least two of these processes may be performed in the same step. Also, any one of steps S302, S309, and S316 may be omitted.

The store terminal 1 is also an example of a communications terminal. Store terminals 1 include not only PCs, but also smart watches, game consoles, and devices dedicated to video calls.

Furthermore, other servers, routers, etc. may relay data in communications between the communication management device 5 and the store terminal 1 and presenter terminal 9.

1. Store terminal (an example of a second communication terminal)
3 Relay device 4 Manager terminal 5 Communication management device 7 Service information management device 9 Explainer terminal (an example of a first communication terminal)
11 Transmitting/receiving unit 12 Receiving unit 13 Communication control unit 14 Display control unit 15 Determination unit 51 Transmitting/receiving unit (an example of a transmitting means, an example of a receiving means)
5000 Storage unit (an example of storage means)
100 Communication Network

Claims (9)

A communication management device that mediates video data of a subject at a first location transmitted from a first communication terminal so that the video data of the subject at a first location can be received by a second communication terminal,
a management means for managing data of a predetermined background image and predetermined identification information for identifying the predetermined background image in association with each other;
a receiving means for receiving a transmission request for the predetermined background image transmitted by the first communication terminal, the transmission request including the predetermined identification information;
a readout means for reading out data of the predetermined background image corresponding to the predetermined identification information included in the transmission request from the management means;
transmitting the data of the predetermined background image read by the reading means to the second communication terminal so that the second communication terminal displays the predetermined background image as a background image for the subject at the first location while the second communication terminal is displaying the subject at the first location, and the predetermined background image as a background image for the subject at the second location while the second communication terminal is displaying video data of the subject at the second location photographed by the second communication terminal;
a transmitting means for transmitting the data of the predetermined background image read by the reading means to the first communication terminal so that the first communication terminal displays the predetermined background image as a background image for the subject at the second location while the first communication terminal is displaying the subject at the second location, and the predetermined background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location photographed by the first communication terminal;
A communication management device comprising: the video data transmitted by the first communication terminal is video data of a first subject photographed by the first communication terminal,
the receiving means receives the video data of the subject at the second location, the video data being captured by the second communication terminal;
the transmitting means transmits the video data of the subject at the second location so that the first communication terminal receives the video data of the subject at the second location;
The communication management device described in claim 1, characterized in that the transmitting means transmits data of the specified background image based on the transmission request so that the first communication terminal receives video data of the subject at the second location and displays the specified background image as a background image for the subject at the second location that is being displayed. The communication management device described in claim 2, characterized in that the transmission means transmits the data of the specified background image based on the transmission request, so that the specified background image is displayed as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location captured by the first communication terminal. 4. The communication management device according to claim 1,
The receiving means receives a request to send a predetermined background image based on information input or selected from a call screen displayed on the first communication terminal. The communication management device described in any one of claims 1 to 4, characterized in that the background image is a spherical image. The communication device described in claim 5, wherein the spherical image is a still image or a video. A communication management method executed by a communication management device including a management means for mediating video data of a subject at a first location transmitted by a first communication terminal so that the video data of the subject at the first location can be received by a second communication terminal, and for managing data of a predetermined background image and predetermined identification information for identifying the predetermined background image in association with each other,
a receiving step of receiving a transmission request for the predetermined background image transmitted by the first communication terminal, the transmission request including the predetermined identification information;
a reading step of reading, from the management means, data of the predetermined background image corresponding to the predetermined identification information included in the transmission request;
transmitting the data of the predetermined background image read in the reading step to the second communication terminal so that the second communication terminal displays the predetermined background image as a background image for the subject at the first location while the second communication terminal is displaying the subject at the first location, and the predetermined background image as a background image for the subject at the second location while the second communication terminal is displaying video data of the subject at the second location photographed by the second communication terminal;
a transmitting step of transmitting the data of the predetermined background image read in the reading step to the first communication terminal so that the first communication terminal displays the predetermined background image as a background image for the subject at the second location while the first communication terminal is displaying the subject at the second location, and the predetermined background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location photographed by the first communication terminal;
A communication management method comprising: A program causing a computer to execute the method described in claim 7. A communication management device that manages a video call between a first communication terminal and a second communication terminal so that video data of a subject at a first location transmitted by the first communication terminal is received by the second communication terminal, and video data of a subject at a second location transmitted by the second communication terminal is received by the first communication terminal,
a management means for managing data of a predetermined background image and predetermined identification information for identifying the predetermined background image in association with each other;
a receiving means for receiving a transmission request for the predetermined background image transmitted by the first communication terminal, the transmission request including the predetermined identification information;
a readout means for reading out data of the predetermined background image corresponding to the predetermined identification information included in the transmission request from the management means;
displaying the predetermined background image as a background image for the subject at the first location while the second communication terminal is displaying the subject at the first location, and transmitting the data of the predetermined background image read by the read means to the second communication terminal so that the second communication terminal can display the predetermined background image as a background image for the subject at the second location while the second communication terminal is displaying video data of the subject at the second location photographed by the second communication terminal;
a transmitting means for transmitting the data of the predetermined background image read by the reading means to the first communication terminal so that the first communication terminal displays the predetermined background image as a background image for the subject at the second location while the first communication terminal is displaying the subject at the second location, and the predetermined background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location taken by the first communication terminal;
and
A communication management device characterized in that the transmitting means transmits data of the specified background image based on the transmission request so that the first communication terminal receives video data of a subject at the second location and displays the specified background image as a background image for the subject at the second location that is being displayed.