JP7560166B1 - Virtual space exhibition system - Google Patents

Abstract

[Problem] To facilitate communication between guides and users in a virtual exhibition in a virtual space. [Solution] A virtual exhibition system is constructed in which an exhibitor can exhibit a virtual booth IM1 in a virtual space IM0. The user can view the exhibits while moving around each booth, but the positions to which the user can move are limited to specified positions within the booth, such as viewpoint positions VP1 and VP2. By perspectively projecting the virtual space at each viewpoint position, a user image in which the exhibits at the booth are three-dimensionally displayed is displayed on the user terminal. The user image is also displayed as is on the exhibitor terminal. In this way, the exhibitor's operator can converse with the user based on what the user is looking at, and can provide guidance that is in line with the user's interests. [Selected Figure] Figure 4

Description

The present invention relates to a virtual space exhibition system that displays a virtual space to a user.

Recently, technology for holding exhibitions and other events in virtual spaces has been attracting attention.
Patent Literature 1 discloses a system that presents data in a virtual space to a viewing user in a state that allows the data to be viewed by a three-dimensional viewer such as a head-mounted display. The viewing user can view virtual exhibits while moving around in the virtual space. If the viewing user makes a request, an explanation of the virtual exhibit is displayed in text or output as audio.
Patent Literature 2 discloses a technology that allows staff at an online exhibition to guide visitors via video calls. Staff can also share a screen with visitors and give explanations.
Patent Literature 3 discloses a virtual tourism system that uses a virtual space for sightseeing. A user can move an avatar around the virtual space to enjoy sightseeing and can also converse with other users. A guide can also guide multiple users on a virtual sightseeing tour.

JP 2022-133109 A JP 2022-164988 A JP 2023-48271 A

In the prior art, in a virtual exhibition in a virtual space, explanations of exhibits are provided to visitors using pre-prepared text and audio, which causes a lack of realism. In Patent Documents 2 and 3, it is possible to converse with visitors, but even in this case, the staff cannot grasp which part of the exhibit the visitor is looking at when asking a question, so communication is not as smooth as in a real exhibition. In addition, if the presenter wants to show the visitors an exhibit, he or she must verbally explain where the exhibit is located, making it difficult to guide the visitors.
This problem is not limited to cases where an exhibition is held in a virtual space, but is a common problem when a virtual space is exhibited to users for various purposes such as sightseeing or other nature observation, visiting art galleries, museums, factories and other facilities, etc. The present invention aims to solve this problem and facilitate communication between guides and users when exhibiting a virtual space.

The present invention provides, as a first aspect,
A virtual exhibition system that performs a virtual exhibition electronically constructed in a virtual space, comprising:
a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition;
a visibility information input unit for inputting visibility information that specifies a visibility in the virtual space of a user viewing the virtual exhibit;
a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user;
a guide display control unit that displays, on a guide output device used by a guide of the virtual exhibition, an image identical to an image to be displayed on the user image display device;
a guide information input unit that receives, from the guide, guide information to be provided to the user;
The virtual exhibition system may further include an output control unit that causes the user output device to output the guidance information.

In a first aspect of the present invention, images of the virtual space and the virtual exhibit are displayed, so that the user can perceive that he or she is viewing the virtual exhibit within the virtual space. The same images displayed to the user can also be displayed to the guide of the virtual exhibit.
In the prior art, it was possible for the guide to share images prepared by the guide with the user, but this method did not allow the guide to know what the user was interested in. In contrast, in the present invention, the guide can guide the user while checking what scene the user is looking at in the virtual space, making it possible to provide guidance that is in line with the user's interests.
In the present invention, the field of view information may be the position of the viewpoint, the line of sight, the angle of view, etc. in the virtual space. Only a part of these may be input. In addition, a part of the information may be a fixed value set in advance.
In the present invention, the virtual space and virtual exhibit may be two-dimensional, but by making them three-dimensional, an exhibit with a sense of realism can be created. The virtual space and virtual exhibit may be provided using CG (computer graphics), or may be provided by attaching textures of photographs, videos, or still images or videos created using CG of real exhibits. The virtual space and virtual exhibit may be provided in various other ways.
If real-life video or CG video (animation) is used as the texture, it is possible to reproduce the movement in the virtual space, further improving the sense of realism. In particular, when the present invention is used for tourist information, the use of video has the advantage of being able to express the flow of a waterfall or a river, or the way trees flutter in the wind.
Hereinafter, in this specification, when photographs and CG are mentioned, actual videos and animations may be used instead of or in addition to them.
The virtual exhibit may be an exhibition that explains the contents of a product or service, or may be a variety of contents such as scenery of a tourist spot, an art museum, or an exhibition.
The user output device may be a display of a personal computer, tablet, smartphone, or the like, or may utilize goggles or glasses worn on the head. In particular, when goggles or glasses are used, there is an advantage that the sense of realism is further improved. The same applies to the guide output device. Furthermore, the user output device and the guide output device may be capable of outputting not only images but also sounds.
The guidance information provided to the user can take various forms, such as text, images, and audio.

In the present invention,
The user display control unit generates a two-dimensional image of the virtual exhibit based on the visibility information,
The guide display control unit may perform the display by utilizing the two-dimensional image generated by the user display control unit, without generating an image based on the visibility information.

There are two possible methods for sharing the image viewed by the user with the guide. The first method is to generate images to be displayed on the user output device and the guidance output device separately, and to share parameters for generating the images between the two. The second method is to use the image data generated for the user output device as is on the guidance output device as well.
The above embodiment corresponds to the second method. This has the disadvantage of transferring relatively large amounts of image data, but has the advantage of reducing the load of generating images to be displayed on the guidance output device and making it relatively easy and reliable to synchronize the two.

In the present invention,
the user display control unit is capable of displaying a moving image on the user output device using moving image data designated by the guide,
The guide display control unit may intermittently acquire playback position information indicating at what time in the video data the user display control unit is playing, and at the time of acquisition, display a video on the guide output device using the video data specified by the guide, while synchronizing with the playback position information.

When playing back moving images, since there is no need for perspective projection or the like within the virtual space, it is preferable to play them separately on the user output device and the guide output device. However, in this case, since they are played separately, the synchronization between the two is not necessarily maintained, and there may be a discrepancy in the playback position. On the other hand, if one tries to always maintain the synchronization between the two, it is necessary to constantly monitor the playback position of the moving image on the user output device, which imposes a large load on the playback.
According to the above aspect, by intermittently acquiring the playback position information, it is possible to realize synchronization between the user output device and the guidance output device to a degree that does not cause discomfort while suppressing the load.
In the above aspect, the timing for intermittently acquiring the playback position information may be determined arbitrarily within a range that does not cause discomfort, depending on the occurrence of a discrepancy when the video is played on the user output device and the guidance output device.

In a second aspect, the present invention provides a method for producing a liquid crystal display comprising:
A virtual exhibition system that performs a virtual exhibition electronically constructed in a virtual space, comprising:
a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition;
a position setting unit that sets a position of a user in the virtual space in response to an instruction from the user viewing the virtual exhibition;
a visibility information input unit for inputting visibility information that specifies the visibility of a user viewing the virtual exhibit at the position in the virtual space;
a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user;
The virtual exhibition system may be provided with a push-type guidance unit that controls the position setting by the position setting unit and/or the field of view information based on instructions from a guide of the virtual exhibition or on data prepared in advance.

In a second aspect of the present invention, the user can set a position in the virtual space by himself/herself, that is, the user can move in the virtual space at his/her own will. The user may be allowed to move freely in the virtual space, just as one would look around various booths at a real exhibition. Also, the user may be allowed to move to the virtual exhibit corresponding to a desired exhibitor by selecting the exhibitor from a list or the like.
In the second aspect, the position of the user and/or the visibility information can be controlled based on the guide's instructions or on data prepared in advance. In other words, the guide's instructions can be prioritized over the user's instructions. This makes it possible to guide the user while moving around in the virtual space, in the same way that a guide guides the user in the real space. Furthermore, by controlling the visibility information, the user can be guided in a desired direction. If these types of guidance can be performed based on data prepared in advance, automatic guidance can also be realized.
In the above aspect, the operating state in which the guide's instructions are prioritized may be always valid, or may be valid only when consent is obtained from the user.
Also, the guide may be configured to freely move the user's position, or may be limited to a predetermined guide point prepared in advance in the virtual space. The latter method makes it possible to easily move the user to a desired position. Also, the user may feel some discomfort if he or she is allowed to move freely against his or her will, even in a virtual space, but the latter method makes it possible to reduce such discomfort.

In either the first or second aspect of the present invention,
the virtual data storage unit stores guide image data for displaying an image representing the guide,
The user display control unit is capable of displaying an image of the guide based on the guide image data,
The display position of the image of the guide may be restricted to a predetermined position set in advance within the virtual space.

By displaying the image of the guide in this way, it is possible to give the user a sense of realism. It is preferable that the image of the guide is the same as that of the virtual exhibition. In other words, if the virtual exhibition is performed using two-dimensional images, it is preferable that the image of the guide is also two-dimensional, and if the virtual exhibition is performed using three-dimensional images, it is preferable that the image of the guide is also three-dimensional. The image of the guide can be of various types, such as a human appearance, an animal, or a fictional creature.
The guide image data may be created using CG, or may be a photograph, a video, or the like.
Although the position of the guide can be moved according to the position of the user, in the above embodiment, the position of the guide in the virtual space is restricted to a predetermined position. This has the advantage that it is easy to avoid the inconvenience of the guide's position interfering with the virtual exhibit. The position of the guide in the virtual space may be defined by one or more points, or may be defined by an area.
The image of the guide may be displayed at all times, or may be displayed in response to a request from the user or an operation by the guide.

In the above aspect,
the predetermined position is set so that the image of the guide can be displayed within the field of view of the user regardless of the position of the user,
Only one image of the guide may be displayed in the field of view.

By doing so, the image of the guide is displayed to the user no matter where the user is, so the sense of realism can be maintained. The guide only needs to be within the user's field of vision, and does not necessarily need to be displayed in front of the user. Also, the guide does not need to be in the user's field of vision at all times, and it is sufficient that the guide can be seen by looking in any direction from each position.
Although multiple guides may be displayed, displaying only one will allow the user to feel as if he or she is communicating with the displayed guide, further enhancing the sense of realism.
There are, for example, two possible methods for displaying only one guide. The first method is to determine the position of the guide in advance so that multiple guides do not come within the user's field of view. The second method is to select and display only one guide when multiple guides come within the user's field of view. Either method may be adopted.
In the above embodiment, the user's "field of view" may be limited to a predetermined distance range from the user's position. In this way, guides can be displayed in the distance, while only one guide can be displayed nearby.

In either the first or second aspect of the present invention,
The position of the user in the virtual space may be restricted to a preset default position.

In this way, a virtual exhibition can be provided efficiently, and the user can view it efficiently.
In the above aspect, the visibility information at the specified position may be arbitrary, or the visibility information may be restricted.

In either the first or second aspect of the present invention,
The virtual space may be individually allocated to one or a particular group of users.

In the above embodiment, the data constituting the virtual space may be used by multiple users. "Individually" means that no users other than the user himself or the group to which the user belongs exist in the same virtual space.
This makes it possible to prevent communication between the user and the guide from being leaked to other users.
A technology known as the Metaverse allows an unspecified number of users to gather and communicate within a virtual space, but the above embodiment does not use this method; instead, a unique virtual space is created for each user or specific group.
It should be noted that such a unique virtual space and a virtual space in which unspecified users coexist may be selected at will.

In either the first or second aspect of the present invention,
a material distribution unit that, in response to an instruction from the user, causes the user to acquire material data prepared by the guide as a distribution material in a state in which the user can freely use the material;
The information providing unit may provide the guide with information about which of the users has acquired the material data.

This makes it possible to analyze which handouts users showed interest in. The results of this analysis can be used, for example, to select users who are more likely to close a deal. They can also be used to improve the content of handouts and for other purposes.

In either the first or second aspect of the present invention,
The virtual space may further include an information providing unit that provides the guide with information about the time that the user has stayed in the virtual space.

This makes it possible to analyze the extent to which the virtual space has attracted users' interest. This analysis can also be used to select users who are likely to close business deals, improve the content of virtual exhibits, and for other purposes.

In either the first or second aspect of the present invention,
the virtual data storage unit stores a plurality of virtual data;
The user display control unit may perform display using virtual data selected by the user from the plurality of virtual data.

This makes it possible to browse several virtual exhibits in a virtual space, just like browsing several booths at an exhibition. The multiple virtual exhibits may be provided by different entities or by the same entity.

The present invention does not necessarily have to have all of the above-mentioned features, and some of them may be omitted or combined as appropriate. The above-mentioned virtual exhibition may be configured as a virtual exhibition method executed by a computer, or as a computer program for causing a computer to execute such a method. Furthermore, it may be configured as a computer-readable recording medium on which such a computer program is recorded.

FIG. 1 is an explanatory diagram showing a configuration of a virtual exhibition system in an embodiment. FIG. 2 is an explanatory diagram showing a data structure. FIG. 13 is an explanatory diagram showing a method of generating a displayed image. FIG. 11 is an explanatory diagram showing an example image (1) displayed to the user. FIG. 13 is an explanatory diagram showing an example image (2) displayed to the user. FIG. 11 is an explanatory diagram showing how to set the position of a guide. 13 is a flowchart of a virtual space selection process. 13 is a flowchart of a user image generation process. 13 is a flowchart of a viewpoint change process. 13 is a flowchart of a video playback process. 13 is a flowchart of an information providing process.

Hereinafter, an embodiment of the present invention will be described in the following sections, taking as an example a case where an exhibition is held in a virtual space.
A. System Configuration:
B. How to display the image:
C. Guide Position:
D. Virtual Space Selection Process:
E. User Image Generation Process:
F. Video playback processing:
G. Information Processing:
H. Effects and Variations:

A. System Configuration:
1 is an explanatory diagram showing the configuration of a virtual exhibition system according to an embodiment of the present invention, in which a plurality of exhibitors set up virtual booths in a virtual space and provide users with virtual exhibitions of products or services.
The virtual exhibition system is composed of a server 100 that provides its main functions, exhibitor terminals 200 used by exhibitors at each booth, and user terminals 300 used by users visiting the booths, all connected via a network NE such as the Internet. The exhibitor terminals 200 and the user terminals 300 may be various devices that can be connected to the network NE, such as personal computers, tablets, and smartphones. The functions of the server 100 may be provided by a plurality of devices. A plurality of exhibitor terminals 200 and user terminals 300 may also be connected.

The exhibitor terminal 200 and the user terminal are provided with the functions shown in the figure.
The transmitting/receiving unit 211 and the transmitting/receiving unit 311 have a function of transmitting and receiving information via the network NE. The information includes text, images, videos, sounds, etc. Also included are various instructions from exhibitors and users to the virtual exhibition system.
The input/output unit 212 and the input/output unit 312 control devices that input and output information. In this embodiment, the exhibitor's operator wears goggles 201, and the user also wears goggles 301. These goggles 201, 301 display various images provided by the virtual exhibition system and output audio. They also have built-in microphones, allowing audio input. When the operator makes an input to the virtual exhibition system, they use devices such as a keyboard and mouse of the exhibitor terminal 200. When the user makes an input, they use the touch panel of the user terminal 300. Input and output are not limited to the above example, and can take various forms.
The application 213 and the application 313 provide functions such as performing the above-mentioned input and output based on information transmitted from the virtual exhibition system. They may be configured as software specific to the virtual exhibition system, or may use general-purpose software such as a browser.

The server 100 has the functions shown in the figure.
The virtual data storage unit 101 stores various data (hereinafter, sometimes collectively referred to as "virtual data") for realizing a virtual space and a virtual exhibition. In this embodiment, the virtual data representing the entire venue where the virtual exhibition is held is prepared by the organizer of the virtual exhibition or the operator of the virtual exhibition system. The virtual data representing each booth is prepared by the exhibitor.
The user data storage unit 102 stores information about users who visit each booth.
The exhibitor data storage unit 103 stores information about the exhibitors who will be exhibiting at each booth.
The material data storage unit 104 stores data such as materials and videos to be distributed to users at each booth.
The specific contents and structures of these data will be described later.

The virtual space allocation unit 111 allocates a virtual space to a user. In this embodiment, when a user visits a booth, the virtual space of that booth is set as a space unique to that user. This allows the user to communicate with the exhibitor's operator as if that user were the only person present at that booth. The virtual space allocation unit 111 realizes this function.
The data management unit 112 manages the input and output of data in the virtual data storage unit 101 , the user data storage unit 102 , the exhibitor data storage unit 103 , and the material data storage unit 104 .
The transmission/reception unit 113 transmits and receives information to and from the exhibitor terminal 200 and the user terminal 300 via the network NE.

The exhibitor display control section 121 generates and transmits image data to be displayed on the exhibitor terminal 200. Information for generating an image may be transmitted to the exhibitor terminal 200 to cause the exhibitor terminal 200 to generate the image data.
The guide information input unit 122 inputs guide information for introducing products or services from the exhibitor terminal 200. The operator can input the guide information in the form of characters, voice, etc. at the exhibitor terminal 200. The input guide information is transmitted to the user terminal 300, thereby realizing communication between the operator and the user.
The information providing unit 123 provides various information regarding the performance of the exhibit to the exhibitor. For example, the information includes the time that the user stayed at the booth, the number of users who visited the booth, the type of materials that the user acquired at the booth, etc. The information may also include the results of an analysis based on this information.
The push-type guidance unit 124 has a function whereby the operator can set the user's position, line of sight, etc. This allows the operator to guide the user around the booth as if he or she were taking the user with them. Also, if the operator is unable to respond, the server 100 may automatically guide the user according to pre-prepared data.

The user display control unit 131 generates and transmits image data (hereinafter referred to as "user image") to be displayed on the user terminal 300. Information for generating an image may be transmitted to the user terminal 300 to cause the user terminal 300 to generate the image data.
The position setting unit 132 sets the position of the user in the virtual space. In this embodiment, the user can move within the virtual space by setting the position.
The visibility information input unit 133 inputs visibility information from the user terminal 300. The visibility information is information such as the viewpoint position, line of sight direction, and angle of view for generating a user image.
The material distribution unit 134 transmits data of materials distributed in the booth in accordance with instructions from the user. The data of the materials is downloaded to the user terminal 300 or the like.

The virtual exhibition system is not limited to the above configuration and may have various configurations. It is not necessary to have all the functions shown in the figure, and some of them may be omitted. In addition, the virtual exhibition system may have functions other than those shown in the figure.
In this embodiment, each of the functions described above is implemented as software, but it is also possible to implement a part or all of them as hardware.

FIG. 2 is an explanatory diagram showing a data structure.
In the exhibitor data storage unit 103, an exhibitor ID, which is an identifier, is assigned to each exhibitor, and exhibitor information such as exhibitor name, contact information, virtual data, invitees, and visitors is stored.
The virtual data information indicates the location of the three-dimensional model, photos, videos, textures, and other data that constitute the exhibitor's booth. In this embodiment, the virtual data is stored in the virtual data storage unit 101 with a virtual data ID, which is an identifier, so that the virtual data ID can be stored as the virtual data information.
The guest information is information about users invited by the exhibitor to their booth. As will be described later, each user is assigned a user ID as an identifier, so the user IDs can be stored as the guest information.
The visitor information is information about the user who visited the booth. As with the invitee, the user ID is stored.

The user data storage unit 102 stores user information such as name, contact information, visit log, material log, and group for each user ID.
The visit log is a history of the booths visited. The time when the booth was visited and the time when it was left can be recorded. Also, the booth can be specified by the exhibitor ID.
The material log is a history of materials acquired by downloading. It can be recorded using a unique ID attached to the material.
The group information stores the user IDs of the user and the other users who make up the group. In this embodiment, when a booth is visited, the booth is assigned as a virtual space specific to the user, but it is also possible to assign a virtual space specific to a group by specifying the group. It is preferable that the group information be editable by the user as appropriate.

The virtual data storage unit 101 stores virtual data that constitutes a virtual space and a virtual exhibition.
The "venue data" is data prepared by the organizer of the virtual exhibition or the operator of the virtual exhibition system. It stores a three-dimensional model, photos, videos, textures, etc., that represent the virtual space of the entire venue where the virtual exhibition will be held.
The virtual data for the booth prepared by each exhibitor is stored with a virtual data ID for each booth. When an exhibitor exhibits multiple booths, multiple virtual data IDs are assigned to each exhibitor.
The virtual data information includes exhibitor IDs, booth data, exhibit data, viewpoint data, guide data, viewing logs, and the like.
The booth data is data on a three-dimensional model constituting the booth, photographs, videos, textures, etc. The booth may be represented using photographs and videos of an actual booth.
The exhibit data may be three-dimensional data, two-dimensional image data, video data, etc. of the exhibits in the booth. In addition to providing the exhibits in CG, the exhibits may also be provided by attaching photographs and videos of the exhibits.
The viewpoint data stores the positions to which the user can move within the booth. In this embodiment, as described later, the user is not allowed to move completely freely within the booth, but can selectively move to pre-set positions. The viewpoint data is data that represents the positions to which the user can move.
The guide data is three-dimensional image data, so-called avatar data, that represents a person who will guide the user in the booth. In this embodiment, the guide does not move freely in the booth, but is fixed and displayed at a specified position. The guide data also stores information on the position at which the guide is displayed.
The viewing log is a history of users who visited the booth. The visit time and the departure time are recorded for each user. The content of the communication between the user and the operator may also be recorded.
In addition, data may be prepared to automatically guide users around the booth when an operator is not present, such as voice guidance data, data for controlling the user's viewpoint and line of sight, data for instructing the user to enlarge panels in the booth, play videos, etc.

The virtual space allocation data is data generated by the virtual space allocation unit 111 when a user visits a booth. As shown in the figure, a virtual data ID that identifies the booth is associated with an exhibitor ID that identifies the exhibitor, and a user ID that identifies the user. When users are visiting in a group, multiple user IDs in the group are associated. Questions from the user are conveyed only to the operator of the associated exhibitor ID, and conversely, the operator's responses are conveyed only to the associated user. This allows users to communicate privately with the operator without worrying about information being leaked to other users.
The virtual space allocation data is generated for each user or group. The virtual data ID may be duplicated. In this way, the booth exhibit can be provided to multiple users, while providing each user with a unique virtual space.
When multiple users are visiting a booth, multiple virtual space allocation data are set for one exhibitor ID. In such a case, the operator will select one of the visiting users and communicate with that user. The conversation flag is a flag that indicates whether or not a conversation with the operator is possible. When the operator selects one of the visiting users, the conversation flag in the virtual space allocation data for that user is turned on, and the conversation flags in the other virtual space allocation data are turned off. In this way, the above-mentioned closed communication can be realized even when multiple users are visiting a booth.
When a plurality of exhibitor terminals 200 are prepared, the users may be handled by a plurality of operators. In such a case, the flag may be omitted.

B. How to display the image:
3 is an explanatory diagram showing a method of generating a displayed image. In this embodiment, the virtual space representing the venue is constructed by pasting textures of photos and videos representing the venue onto the inner surface of a spherical virtual space S. Also, as shown in the figure, the exhibitor has constructed virtual data for booth B using a three-dimensional model or the like, and the booth B is placed in the virtual space. Booth B may also be constructed by pasting photos and videos taken of an actual booth.
The user image can be generated by perspective projection in a virtual space according to visibility information such as a viewpoint VP, a line of sight direction VL, and an angle of view VA. The viewpoint VP and the line of sight direction VL can be determined using three-dimensional coordinates in the virtual space. The angle of view VA may be fixed or may be adjustable by the user.
For ease of illustration and explanation, in FIG. 3 the viewpoint VP is shown as being located outside the virtual space S of the venue, but in reality the viewpoint VP is located at the center of the virtual space S.

4 is an explanatory diagram showing an example image (1) displayed to the user. An image IM0 of the venue and an image IM1 depicting a booth in three dimensions are displayed. Also displayed is a guide G guiding the user around the booth. A person PM in the booth is a fixed human-shaped model drawn to give a sense of realism. Exhibits hung on the wall are panels shown as still images, videos, etc. VP1 drawn on the floor represents a viewpoint position to which the user can move.
A floor plan W1 of the booth is displayed in the upper left of the user image. An enlarged view of the floor plan W1 is shown at the top of the figure. In this embodiment, the user cannot move freely in the booth, but selectively moves to a preset viewpoint position. The floor plan W1 illustrates the viewpoint positions and field of view to which the user can move. The viewpoint positions to which the user can move are indicated by a camera icon. The user image shown in FIG. 4 represents a state seen from the viewpoint position VP0 as in the field of view VA. The circular viewpoint position VP1 shown on the floor in the user image corresponds to the viewpoint position VP1 shown in the lower right of the floor plan W1.
When the user selects one of the camera icons by clicking on it, the viewpoint position moves. The user may select a circular viewpoint position VP1 drawn on the floor in the user image. The image when viewpoint position VP2 is selected is shown in FIG. 5.

5 is an explanatory diagram showing an example of an image (2) displayed to the user. From the plan view W1 shown in the upper left, it can be seen that the image is of the state in which the viewpoint position VP2 in FIG. 4 is selected and the user is looking toward the wall.
A display is drawn on the wall, and a video MV is automatically played silently, creating a sense of realism as an exhibition. When a user clicks on icon IC1, the video MV is enlarged and played with sound. The video may be played over the entire user image.
A panel is displayed next to it. When the user clicks on the icon IC2 below the panel, the panel is enlarged. When the user clicks on the icon IC3, the document can be downloaded.

An icon may be prepared for users to apply for guidance from exhibitors. When the user clicks on this icon, the user can receive guidance in text, audio, etc. through closed communication with the operator. When the user applies for guidance and the operator responds, an image of the guide G shown in FIG. 4 may be displayed.
Conversely, the operator may offer guidance to the user. An icon for calling the user may be provided on the screen of the exhibitor terminal 200. When the operator operates the icon to call the user, an icon indicating that the user is being called may be displayed on the user screen, and the user may respond to this icon to be connected to the operator. When the user is connected to the operator, an image of the guide G shown in FIG. 4 may be displayed.

C. Guide Position:
6 is an explanatory diagram showing how to set the position of the guide. In this embodiment, the position where the guide is displayed is preset within the booth. This has the advantage that it is easy to avoid problems such as the guide interfering with the exhibits in the booth.
The position of the guide can be determined arbitrarily, but in this embodiment, it is set so that one guide always appears on the user image.
As shown in the figure, the user can move between viewpoint positions P1 to P5. Also, the user can view fields V1 to V5 at viewpoint positions P1 to P5, respectively. The guide positions G1 to G5 may be set at any positions within each of the fields of view V1 to V5. Since the fields of view V1 and V2 partially overlap, the guide may be placed at position G12, and positions G1 and G2 may be omitted. With the above settings, at viewpoints P1 to P4, only one guide is displayed in the fields of view V1 to V4. When the user communicates with the operator, displaying the guide in this way can give the user a sense of realism. Even if the guide is always displayed at positions G1 to G4, the user can only view one guide, so no sense of incongruity is felt. It is acceptable if the guide temporarily disappears from the user image as the user moves his or her gaze within the fields of view V1 to V4.

For viewpoint V5, only one guide at position G5 is displayed within field of view V5, but guides at positions G1 to G4 may be displayed in the distance. Since such distant guides do not cause discomfort to the user at viewpoint position P5, there is no problem in displaying them as is. Furthermore, if there is a possibility that multiple guides may be displayed within the field of view, a guide may be displayed only at position G5 within field of view V5, and guides at the other positions G1 to G4 may not be displayed.

D. Virtual Space Selection Process:
A flowchart for realizing a virtual exhibition will be described below.
7 is a flowchart of the virtual space selection process. This process generates virtual space allocation data (see FIG. 2) when a user visits a booth. This process is mainly executed by the virtual space allocation unit 111 (see FIG. 1), and in terms of hardware, it is mainly executed by the server 100.
In this embodiment, the user can visit a booth by selecting an exhibitor or booth from the exhibitor list or booth list. The user can also visit a booth by moving around various booths in the virtual space and moving into the desired booth.
In the virtual space selection process, the server 100 checks whether or not an exhibitor or booth has been designated by the user (step S10). If so, the server 100 generates virtual space allocation data accordingly (step S13).
If not, the server 100 inputs the user's position information in the virtual space (step S11) and determines whether or not the user is in any of the exhibitor's booths (step S12).
If the user is within an exhibitor's booth, it is determined that the user is visiting that booth, and virtual space allocation data is generated (step S13).If the user is not within an exhibitor's booth, the process from step S10 onwards is repeated.
By the above process, a unique virtual space can be assigned to each user. If the users have set up groups, when an instruction is given by any user (step S10) or when any user is in an exhibitor booth (step S12), virtual space assignment data can be generated assuming that all users in the group will visit the booth.

In the above explanation, it is assumed that the user moves freely through the virtual space and visits booths, just as if the user were to visit booths while touring the venue at a real exhibition. This embodiment is not necessarily limited to a situation in which the user can move freely through the virtual space. In such a case, step S11 (input of user location information) and step S12 (determination of whether or not the user is within an exhibitor's booth) in FIG. 7 may be omitted. When the user has not specified a booth (step S10), the server 100 may generate virtual space allocation data for a predetermined location.

E. User Image Generation Process:
Fig. 8 is a flowchart of the user image generation process. This process generates a user image by performing the perspective projection shown in Fig. 3. This process is mainly executed by the user display control unit 131 (see Fig. 1), and in terms of hardware, this process is mainly executed by the server 100.
When the process starts, the server 100 inputs the user's position information and visibility information (step S21).
Then, the position of the guide is set (step S22). The position of the guide is set in advance in the booth as explained in FIG. 6. In step S22, a position to be displayed is selected from these positions. If the guide is to be displayed at all of the set positions, the process of step S22 may be omitted.
The server 100 generates a user image by perspective projection (step S23). The generated user image is transmitted to the user terminal.
Moreover, when the operator is giving guidance to the user (step S24), the generated user image is also transmitted to the exhibitor terminal (step S25).
In this way, the operator can see the same image as the user is viewing. In a real space, the guide can guide the user according to the user's interests depending on where the user is looking, but this is difficult in a virtual space. However, according to this embodiment, the operator can check the user's image, so the operator can infer the user's interests from the image and provide guidance.
It should be noted that the process of determining whether or not a guide is being given (step S24) may be omitted, and the user image may be transmitted to the exhibitor terminal at all times.
The above process is repeatedly executed while the virtual exhibition system is being used.

The user image displayed on the exhibitor terminal may be generated in the same manner as the user image, based on the user's position information and field of view information. However, with this method, due to differences in calculation timing, etc., synchronization between the image displayed on the exhibitor terminal and the user image may not be achieved. In this embodiment, as described above, the generated user image is also used on the exhibitor terminal, thereby easily and reliably achieving synchronization of the user image displayed on the user terminal and the exhibitor terminal.

9 is a flowchart of the viewpoint change process. This process is performed when the user moves within the booth or moves his/her line of sight. This process is mainly executed by the push-type information unit 124 (see FIG. 1), and in terms of hardware, this process is mainly executed by the server 100.
When the process starts, the server 100 checks whether the user agrees to the operator guiding the user (step S31). For example, a screen may be displayed to the user asking whether the user agrees, and the user may respond. If the operator verbally confirms with the user and obtains consent, the operator may instruct the server 100 to that effect. Furthermore, if the user is at a specified viewpoint position in the booth, the user may be treated as unconditionally agreeing.
If the user does not consent (step S31), the server 100 inputs an instruction for position information and visibility information from the user terminal (step S33). This instruction can be input, for example, by using a touch panel of the user terminal. Alternatively, the instruction can be input by the movement of goggles worn by the user.
On the other hand, if the user agrees to the guidance (step S31), the server 100 inputs instructions for location information and visibility information from the exhibitor terminal (step S32). These instructions can be input using, for example, the keyboard or mouse of the exhibitor terminal. Alternatively, they may be input by the movement of goggles worn by the operator. In this case, the operator will decide the user's location information and visibility information regardless of the user's wishes. The operator may be able to specify only one of the location information or visibility information.
The server sets the user's position information and field of view information depending on whether the user has consented, and then ends this process. These pieces of information are used to generate a user image (FIG. 4).

According to the viewpoint change process described above, if the user does not consent to being guided by the operator, the user can freely browse inside the booth by selecting the viewpoint position (see Figure 4) and field of view information of his/her own volition. If the user consents to being guided by the operator, the operator can move around the booth and guide the user as if he/she were taking the user around. For example, when the operator guides the user to the exhibition location of the information he/she is looking for, this has the advantage that the guidance can be given more simply and quickly than by giving verbal guidance to the location in the virtual space. It is also useful when the operator proactively guides the user about the exhibition contents based on conversation with the user.

In the above process, when the operator is unable to respond, the server 100 may automatically provide guidance based on instruction data prepared in advance. For example, in the above process, instead of the process of inputting the position information and visibility information based on the operator's instruction (step S32), the user's position information and visibility information may be input based on instruction data prepared in advance in the virtual data storage unit 101 or the like. For example, by preparing instruction data for the position information and visibility information in chronological order, the user can be moved sequentially and guidance can be provided while changing the viewing direction.
When providing guidance, audio data prepared in advance may be played back. Also, the display of the panel may be enlarged, a video may be played back, and documents may be downloaded automatically.

F. Video playback processing:
Fig. 10 is a flowchart of the video playback process. This process is executed when the user clicks on the video playback icon IC1 in Fig. 5. This process is executed mainly by the exhibitor display control unit 121 (see Fig. 1), and in terms of hardware, it is executed mainly by the server 100.
When video playback is instructed, the server 100 plays the instructed video data (step S41). The played video data can be viewed on the user terminal. In this embodiment, the video data is played by streaming, and when the server 100 sequentially transmits video data to the user terminal 300, the user terminal 300 receives the data and sequentially plays the data while storing it in a buffer. The video data may be played on the display MV in the user image shown in FIG. 5, or a window for video playback may be displayed on the user terminal 300.
In step S41, the server 100 also plays the video on the exhibitor terminal 200 in conjunction with the playback of the video on the user terminal 300. The video is also played on the exhibitor terminal 200 in streaming mode, just like the user terminal 300. In order to synchronize the image displayed on the exhibitor terminal 200 with the image displayed on the user terminal 300, a method of transmitting the image data being played on the user terminal 300 to the exhibitor terminal 200 via the network NE is also conceivable. However, this method has the disadvantage that complete synchronization is difficult to achieve due to the large amount of traffic and delays associated with communication. Therefore, this embodiment does not adopt this method, and each is played in streaming mode.

However, when playing back by streaming, there is a risk of the timing of playback being out of sync between the exhibitor terminal 200 and the user terminal 300. Therefore, in this embodiment, synchronization between the two is attempted intermittently at a predetermined timing. The synchronization timing is set to one second in this embodiment, but it may be determined arbitrarily within a range that realizes synchronization without discomfort, taking into consideration the magnitude of the delay that occurs in the playback of the video.
When it is determined that it is time for synchronization (step S42), the server 100 performs a process of synchronizing the exhibitor's video playback position with the user's video playback position (step S43). The exhibitor side is synchronized with the user side in order to prevent the user from feeling uncomfortable about the video being viewed.
The server 100 repeats the above process until the playback of the video ends or until the user instructs the server 100 to end the process (step S44).

An overview of the synchronization process (step S43) is shown in Fig. 10(b). At the top, each frame of the video being played on the user terminal is shown in chronological order. As shown in the figure, frames F1 to F11 are played in sequence. Frames F5 and F10 represent frames being played at the synchronization timing.
The lower part shows the frames of the video being played on the exhibitor's terminal in chronological order. In the example shown in the figure, there is a lag between the playback timing of the user terminal and the video, and while frame F5 is being played on the user terminal, frame F5 has not yet been played on the exhibitor's terminal. Therefore, as a synchronization process, the next frame F6 is played in sync. In this case, the playback of frame F5 on the exhibitor's terminal is omitted.
After that, a lag occurs between the playback timings of the two, and this time, frame F10 is being played back on the user terminal, while frame F11 is already being played back on the exhibitor terminal. Therefore, as a synchronization process, the next frame F11 is played back in synchronization. In this case, frame F11 is played back again on the exhibitor terminal.
In this way, the playback position of the video on the user terminal is acquired at the synchronization timing, and this is reflected in the playback position of the video on the exhibitor terminal. By doing this, it is possible to synchronize the playback to a degree that does not cause discomfort, while allowing for a slight delay in the playback of the videos on both terminals.

G. Information Processing:
11 is a flowchart of the information provision process. This process provides exhibitors with information such as the history of visits to their booths. This process is mainly executed by the information provision unit 123 (see FIG. 1), and in terms of hardware, it is mainly executed by the server 100.
When the process starts, the server 100 creates a list of visitors who visited the exhibitor's booth (step S51). Then, information on all visitors at the time of their visit is compiled (step S52). For example, the time spent at the booth, whether or not materials were downloaded and the type of materials downloaded, the time an operator guided the visitors, etc. can be compiled. Other information may also be compiled.

The server calculates interest points for each user based on this information (step S53). Interest points are an index that represents the user's level of interest in the exhibitor's products or services. For example, each of the various pieces of information collected in step S52 may be converted into points using a predetermined calculation formula, and the interest points may be calculated by adding them up. In addition, interest points may be calculated using a learning model obtained by performing machine learning using the various pieces of information collected in step S52 and the results of successful business negotiations with users as learning data. The calculation of interest points may be omitted, or other analyses may be performed.

The server outputs the tabulated results thus obtained (step S54). An output example is shown in the figure. In this example, the tabulated results are output in the form of a list, but the output can be in various other formats. The tabulated results may also be sorted by some key before being output.
The collected data can be used by exhibitors for their marketing and sales activities, and can also be used to improve booth configurations and materials.

H. Effects and Variations:
According to the virtual exhibition system of the embodiment described above, it is possible to guide the user while checking the user's image, and it is possible to realize guidance that is in line with the user's interests. In addition, it is also possible to realize effective guidance by having the operator control the user's position and line of sight.
It is not necessary to have all of the various features described above, and some may be omitted or combined. Furthermore, the present invention can be configured in various modified forms other than the system for holding an exhibition in a virtual space.
For example, if the virtual exhibit is a landscape of a tourist spot instead of a booth, it can be configured as a system for sightseeing in a virtual space. In addition, it is possible to provide various kinds of virtual exhibits such as art galleries and museums.

The present invention can be used to present a virtual space to a user.

100 Server 101 Virtual data storage unit 102 User data storage unit 103 Exhibitor data storage unit 104 Document data storage unit 111 Virtual space allocation unit 112 Data management unit 113 Transmission/reception unit 121 Exhibitor display control unit 122 Guide information input unit 123 Information provision unit 124 Push-type guide unit 131 User display control unit 132 Position setting unit 133 Visibility information input unit 134 Document distribution unit 200 Exhibitor terminal 201 Goggles 211 Transmission/reception unit 212 Input/output unit 213 Application 300 User terminal 301 Goggles 311 Transmission/reception unit 312 Input/output unit 313 Application

Claims (13)

A virtual exhibition system that performs a virtual exhibition electronically constructed in a virtual space, comprising:
a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition;
a visibility information input unit for inputting visibility information that specifies a visibility in the virtual space of a user viewing the virtual exhibit;
a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user;
a guide display control unit that causes a guide output device used by a guide of the virtual exhibition to display the same image as the image to be displayed on the user output device;
a guide information input unit that receives, from the guide, guide information to be provided to the user;
An output control unit that outputs the guidance information to the user output device. 2. The virtual exhibition system according to claim 1,
The user display control unit generates a two-dimensional image of the virtual exhibit based on the visibility information,
A virtual exhibition system in which the guide display control unit performs the display using the two-dimensional image generated by the user display control unit, without generating an image based on the field of view information. 2. The virtual exhibition system according to claim 1,
the user display control unit is capable of displaying a moving image on the user output device using moving image data designated by the guide,
The guide display control unit intermittently acquires playback position information indicating what time of the video data is being played by the user display control unit, and displays a video on the guide output device using the video data specified by the guide, while synchronizing with the playback position information at the time of acquisition.This is a virtual exhibition system. A virtual exhibition system that performs a virtual exhibition electronically constructed in a virtual space, comprising:
a virtual data storage unit that stores virtual data representing the virtual space and the virtual exhibition;
a position setting unit that sets a position of a user in the virtual space in response to an instruction from the user viewing the virtual exhibition;
a visibility information input unit for inputting visibility information that specifies the visibility of a user viewing the virtual exhibit at the position in the virtual space;
a user display control unit that uses the virtual data based on the visibility information to display an image of the virtual exhibition on a user output device used by the user;
A virtual exhibition system comprising a push-type guidance unit that controls the position setting by the position setting unit and/or the visibility information based on instructions from a guide of the virtual exhibition or on data prepared in advance. A virtual exhibition system according to claim 1 or 4,
the virtual data storage unit stores guide image data for displaying an image representing the guide,
The user display control unit is capable of displaying an image of the guide based on the guide image data,
A virtual exhibition system in which the display position of the image of the guide is restricted to a predetermined position set in advance within the virtual space. 6. The virtual exhibition system according to claim 5,
the predetermined position is set so that the image of the guide can be displayed within the field of view of the user regardless of the position of the user,
A virtual exhibition system in which only one image of the guide is displayed in the field of view. A virtual exhibition system according to claim 1 or 4,
A virtual exhibition system in which the user's position within the virtual space is restricted to a preset position. A virtual exhibition system according to claim 1 or 4,
A virtual exhibition system in which the virtual space is individually assigned to one user or to a specific group of users. A virtual exhibition system according to claim 1 or 4,
a material distribution unit that, in response to an instruction from the user, causes the user to acquire material data prepared by the guide as a distribution material in a state in which the user can freely use the material;
and an information providing unit that provides the guide with information about which of the users has acquired the material data. A virtual exhibition system according to claim 1 or 4,
and an information providing unit that provides the guide with the time that the user stayed in the virtual space. A virtual exhibition system according to claim 1 or 4,
the virtual data storage unit stores a plurality of virtual data;
The user display control unit is a virtual exhibition system that performs display using virtual data selected by the user from the plurality of virtual data. A virtual exhibition method for performing an electronically constructed virtual exhibition in a virtual space, comprising the steps of:
The steps that the computer implements are:
storing virtual data representing the virtual space and the virtual exhibit;
inputting visibility information that identifies a visibility in the virtual space of a user viewing the virtual exhibit;
displaying an image of the virtual exhibition on a user output device used by the user using the virtual data based on the visibility information;
displaying the same image as the image to be displayed on the user output device on a guide output device used by a guide of the virtual exhibition;
receiving guidance information to be provided to the user from the guide;
and outputting the guidance information to the user output device. A virtual exhibition method for performing an electronically constructed virtual exhibition in a virtual space, comprising the steps of:
The steps that the computer implements are:
storing virtual data representing the virtual space and the virtual exhibit;
setting a position of a user within the virtual space in response to an instruction from the user viewing the virtual exhibit;
inputting visibility information that identifies a visibility of a user viewing the virtual exhibit at the location in the virtual space;
displaying an image of the virtual exhibition on a user output device used by the user using the virtual data based on the visibility information;
and when an instruction is given by a guide of the virtual exhibition, setting the position according to the instruction from the guide regardless of an instruction from the user.