JP7650309B2 - Program and system - Google Patents

Description

The present invention relates to a program and a system.

In recent years, services that provide a virtual space (metaverse space) in which multiple users communicate through avatars are becoming popular. In addition, Patent Documents 1 and 2 disclose technology for changing the appearance of other users' avatars (hereinafter referred to as "other users' avatars") with the aim of diversifying ways to enjoy the virtual space.

More specifically, in Patent Documents 1 and 2, the user selects a body part (e.g., the upper body) of the other person's avatar within the user's field of vision and the outfit that the user wants to put on that body part. This causes the outfit for the selected body part of the other person's avatar to be changed only on the screen displayed to the user.

Patent No. 6818106 JP 2022-147265 A

However, in the method of changing clothes for each selected body part, as in Patent Documents 1 and 2, there is a possibility that the image of the changed body part may not match the other body parts. On the other hand, if you try to unify the overall image, the number of body parts to select increases, making the operation cumbersome. Furthermore, the above-mentioned problem becomes more pronounced the more other avatars there are in the field of view, and when you try to unify the images of items in the field of view.

The present invention was made in consideration of the above circumstances, and its purpose is to provide a program that allows users to change the presentation of a virtual space to a desired image with simple operations.

In order to solve the above problem, the program of the present invention causes a computer to function as a selection means for having a first user of a plurality of users select one of a plurality of image candidates representing a presentation image of a virtual space, a change means for collectively changing the presentation of each of a plurality of change targets included in the virtual space from a common appearance perceived by a plurality of users other than the first user to a special appearance that matches the selected image candidate, and a perception means for causing the first user to perceive the virtual space including the plurality of change targets whose presentation has been changed to the special appearance.

According to the present invention, the presentation of a virtual space can be changed to a desired image with simple operations.

FIG. 1 is a diagram showing an overview of a system according to an embodiment of the present invention. FIG. 2 is a hardware configuration diagram of a server. FIG. 2 is a hardware configuration diagram of an HMD set which is an example of a user terminal. FIG. 11 is a hardware configuration diagram of a tablet terminal, which is another example of a user terminal. FIG. 1 is a diagram conceptually illustrating one aspect of a virtual space. 1 is a diagram showing a YZ cross section of a field of view in a virtual space as viewed from an X direction. 13 is a diagram showing an XZ cross section of a field of view in a virtual space as viewed from a Y direction. FIG. 2 is a functional block diagram of a server and a user terminal according to the first embodiment. 5 is a flowchart showing a process of a server according to the first embodiment. 6 is a flowchart showing processing of a user terminal according to the first embodiment. 13 is a flowchart of a presentation change process. 13 is an example of a virtual space image displayed on a monitor of a first user. 13 is an example of a virtual space image displayed on the monitor of a second user. 11 is an example of an interface that allows a first user to select candidate images. 1 is an example of a performance database. 13 is an example of the result of changing the virtual space image displayed on the monitor of the first user to a special mode. 13 is an example of an interface that allows a first user to customize a presentation. FIG. 11 is a functional block diagram of a server and a user terminal according to the second embodiment. 13 is a flowchart showing processing of a user terminal according to the second embodiment. 13 is a flowchart of a duplication control process. This is an example of friend list data. 1 is an example of a virtual space image displayed on a monitor of a user terminal.

Below, the system 1 according to the embodiment will be described with reference to the drawings. Note that the embodiment of the present invention described below is an example of how the present invention can be realized, and does not limit the scope of the present invention to the scope of the described embodiment. Therefore, the present invention can be implemented by making various modifications to the embodiment.

[Overview of System 1]
FIG. 1 is a diagram showing an overview of a system 1 according to this embodiment. As shown in FIG. 1, the system 1 mainly includes a server 10 and user terminals 20A, 20B, and 20C (hereinafter, these may be collectively referred to as "user terminals 20"). Although three user terminals 20 are illustrated in FIG. 1, examples of the user terminals 20 included in the system 1 are not limited to these. The server 10 and the user terminals 20 are connected to each other so as to be able to communicate with each other via a communication network 2. Specific examples of the communication network 2 are not particularly limited, and may be, for example, the Internet, a mobile communication system (e.g., 4G, 5G, etc.), a wireless network such as Wi-Fi (registered trademark), or a combination thereof.

The system 1 according to this embodiment is a system that allows each user of multiple user terminals 20A to 20C to experience a common virtual space provided by a server 10. More specifically, the user terminal 20 places a character (hereinafter, sometimes referred to as an "avatar") associated with the user in the virtual space, and causes the user terminal 20 to display an image of the virtual space viewed from a preset viewpoint.

The user terminal 20 also operates the avatar in the virtual space in conjunction with the user's operation of the operating device. The avatar's movements include, for example, moving within the virtual space, moving various parts of the body, changing posture, changing facial expressions, speaking, and moving objects placed in the virtual space.

Furthermore, the user terminal 20 transmits avatar data indicating the avatar's movements and status changes to the server 10 via the communication network 2. The server 10 transmits the avatar data received from the user terminal 20 to other user terminals 20 via the communication network 2. The user terminal 20 then updates the movements and status of the corresponding avatar in the virtual space based on the avatar data received from the server 10.

[Configuration of Server 10]
Fig. 2 is a hardware configuration diagram of the server 10. The server 10 is realized by, for example, a general-purpose computer such as a workstation or a personal computer. As shown in Fig. 2, the server 10 mainly includes a processor 11, a memory 12, a storage 13, an input/output interface 14, and a communication interface 15. Each component of the server 10 is connected to a communication bus 19.

The processor 11 performs the processes described below by executing a series of instructions contained in the server program 13P stored in the memory 12 or the storage 13. The processor 11 is realized, for example, as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an MPU (Micro Processor Unit), an FPGA (Field-Programmable Gate Array), or other device.

The memory 12 temporarily holds the server program 13P and data. The server program 13P is loaded, for example, from the storage 13. The data includes data input to the server 10 and data generated by the processor 11. For example, the memory 12 is realized as a RAM (Random Access Memory) or other volatile memory.

The storage 13 permanently holds the server program 13P and data. The storage 13 is realized, for example, as a ROM (Read-Only Memory), a hard disk drive, a flash memory, or other non-volatile storage device. The storage 13 may also be realized as a removable storage device such as a memory card. As yet another example, instead of being built into the server 10, the storage 13 may be connected to the server 10 as an external storage device. With this configuration, for example, in a situation where multiple user terminals 20 are used, such as an amusement facility, it becomes possible to collectively update the server program 13P and data.

The input/output interface 14 is an interface for connecting external devices such as a monitor, an input device (e.g., a keyboard, a pointing device), an external storage device, a speaker, a camera, a microphone, and a sensor to the server 10. The processor 11 communicates with the external devices through the input/output interface 14. The input/output interface 14 is realized using, for example, a Universal Serial Bus (USB), a Digital Visual Interface (DVI), a High-Definition Multimedia Interface (HDMI, registered trademark), or other terminals.

The communication interface 15 communicates with other devices (e.g., user terminal 20) connected to the communication network 2. The communication interface 15 is realized, for example, as a wired communication interface such as a LAN (Local Area Network), or a wireless communication interface such as Wi-Fi (Wireless Fidelity), Bluetooth (registered trademark), or NFC (Near Field Communication).

[Configuration of user terminal 20]
Fig. 3 is a hardware configuration diagram of an HMD (Head Mounted Display) set, which is an example of the user terminal 20. Fig. 4 is a hardware configuration diagram of a tablet terminal, which is another example of the user terminal 20. The user terminal 20 is realized as, for example, the HMD set shown in Fig. 3, the tablet terminal shown in Fig. 4, a smartphone, a feature phone, a laptop computer, a desktop computer, or the like.

As shown in FIG. 3, the user terminal 20 realized as an HMD set includes a computer 26 having a processor 21, memory 22, storage 23, input/output interface 24, and communication interface 25. Each component of the computer 26 is connected to a communication bus 29. The basic configuration of the processor 21, memory 22, storage 23, input/output interface 24, communication interface 25, and communication bus 29 is the same as that of the processor 11, memory 12, storage 13, input/output interface 14, communication interface 15, and communication bus 19 shown in FIG. 2. In addition, the storage 23 holds a terminal program 23P.

The user terminal 20 realized as an HMD set also includes an HMD 30, a motion sensor 41, an operation device 42, a tactile device 43, an olfactory device 44, and a taste device 45 as external devices of the computer 26. The HMD 30, the motion sensor 41, the operation device 42, the tactile device 43, the olfactory device 44, and the taste device 45 are connected to the processor 21 via the input/output interface 24.

The HMD 30 is worn on the user's head to provide the user with a virtual space. More specifically, the HMD 30 may include both a so-called head-mounted display equipped with a monitor and a head-mounted device to which a smartphone or other terminal equipped with a monitor can be attached. The HMD 30 mainly includes a monitor 31 (display screen), a gaze sensor 32, cameras 33 and 34, a microphone 35, and a speaker 36.

As an example, the monitor 31 is realized as a non-transmissive display device. The monitor 31 is disposed on the main body of the HMD 30 so as to be located, for example, in front of both eyes of the user. The non-transmissive monitor 31 is realized, for example, as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor.

As another example, the monitor 31 is realized as a transmissive display device. In this case, the HMD 30 is not a closed type that covers the user's eyes, but an open type such as glasses. The monitor 31 may include a configuration that simultaneously displays a part of an image that constitutes a virtual space and the real space. As one example, the transmissive monitor 31 may display an image of the real space captured by a camera mounted on the HMD 30. As another example, the transmissive monitor 31 may be configured to have an adjustable transmittance. The transmissive monitor 31 may also set the transmittance of a part of the display area high so that the real space can be directly viewed.

The monitor 31 may also have the following configuration to allow the user to view a three-dimensional image. As one example, the monitor 31 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. As another example, the monitor 31 may be configured to display an image for the right eye and an image for the left eye as a single unit. In this case, the monitor 31 includes a high-speed shutter. The high-speed shutter operates to alternately display an image for the right eye and an image for the left eye so that the image is recognized by only one of the eyes.

The gaze sensor 32 detects the direction in which the user's right and left eyes are looking. In other words, the gaze sensor 32 detects the user's gaze. The gaze sensor 32 is realized, for example, by a sensor with an eye tracking function. It is preferable that the gaze sensor 32 includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 32 irradiates the user's right and left eyes with infrared light, and detects the rotation angle of each eyeball by receiving reflected light from the cornea and iris in response to the irradiated light. The gaze sensor 32 then identifies the user's gaze based on each detected rotation angle.

Camera 33 captures the upper part of the face of the user wearing HMD 30 (more specifically, the user's eyes, eyebrows, etc.). Camera 34 captures the lower part of the face of the user wearing HMD 30 (more specifically, the user's nose, mouth, etc.). For example, camera 33 is attached to the side of the housing of HMD 30 that faces the user, and camera 34 is attached to the side opposite the side facing the user. Note that HMD 30 may be provided with one camera that captures the entire face of the user instead of the two cameras 33 and 34.

The microphone 35 converts the user's speech into an audio signal (electrical signal) and outputs it to the computer 26. The speaker 36 converts the audio signal output from the computer 26 into sound and outputs it to the user. Note that the HMD 30 may include earphones instead of the speaker 36.

The motion sensor 41 has a position tracking function for detecting the motion of the HMD 30. As one example, the motion sensor 41 may read multiple infrared rays emitted by the HMD 30 to detect the position and inclination of the HMD 30 in real space. As another example, the motion sensor 41 may be realized by a camera. In this case, the motion sensor 41 analyzes the image information of the HMD 30 output from the camera to detect the position and inclination of the HMD 30. As yet another example, the motion sensor 41 may be realized by an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor.

The operation device 42 is connected to the computer 26 by wire or wirelessly. The operation device 42 accepts input (operation) of commands to the computer 26 by the user. As one example, the operation device 42 may be a so-called controller that is held and operated by the user. As another example, the operation device 42 may be configured to be wearable on the user's body or part of his/her clothing, and may detect the user's movements by a motion sensor. However, specific examples of the operation device 42 are not limited to these, and may be a keyboard, a pointing device, a touch panel, etc.

The haptic device 43 is, for example, a glove-like device worn by the user of the user terminal 20. The haptic device 43 generates vibrations, pressure, and the like, allowing the user to feel the texture (e.g., texture, hardness, weight, etc.) of an object touched by the avatar in the virtual space.

The olfactory device 44 is, for example, a mask-like device that covers the nose of the user of the user terminal 20. The olfactory device 44 combines multiple odor components and releases them into the mask, allowing the user to smell the odor smelled by the avatar in the virtual space.

The taste device 45 is, for example, a gel-like device that the user of the user terminal 20 places in the mouth. The taste device 45 outputs an electrical signal to the gel, allowing the user to feel the taste of the food that the avatar has eaten (drank) in the virtual space.

However, the specific method of stimulating the user's senses of touch, smell, and taste is not limited to the above-mentioned examples, and may be a method of directly outputting an electrical signal to the brain. In addition, some or all of the tactile device 43, the olfactory device 44, and the taste device 45 may be omitted.

As shown in FIG. 4, the user terminal 20 realized as a tablet terminal mainly includes a processor 21, a memory 22, a storage 23, a communication interface 25, a monitor 31, cameras 33 and 34, a microphone 35, a speaker 36, a motion sensor 41, and an operation device 42. Each component of the tablet terminal is connected to a communication bus 29. The basic configuration of the processor 21, memory 22, storage 23, communication interface 25, monitor 31, cameras 33 and 34, microphone 35, speaker 36, motion sensor 41, and operation device 42 is the same as that of an HMD set, so the configuration specific to the tablet terminal will be described below. In addition, the user terminal 20 shown in FIG. 4 does not necessarily include the tactile device 43, the olfactory device 44, and the taste device 45.

The monitor 31 is provided on the surface of a flat housing. The camera 33 is a so-called in-camera that is attached to the surface of the flat housing and captures an image of the face of a user viewing the monitor 31. The camera 34 is a so-called out-camera that is attached to the back surface of the flat housing (the surface opposite the monitor 31) and captures an image of the surroundings. The motion sensor 41 detects the motion of the housing (e.g., rotation around three mutually orthogonal axes). An example of an operation device 42 suitable for a tablet terminal is a touch panel that is superimposed on the monitor 31 and accepts various touch operations by the user (e.g., tapping, sliding, flicking, pinching in, pinching out, etc.).

[Outline of Virtual Space 90]
Fig. 5 is a diagram conceptually illustrating one aspect of the virtual space 90. Fig. 6 is a diagram illustrating a YZ cross section of the field of view 94 in the virtual space 90 as viewed from the X direction. Fig. 7 is a diagram illustrating an XZ cross section of the field of view 94 in the virtual space 90 as viewed from the Y direction.

As shown in FIG. 5, the virtual space 90 has a spherical structure that covers the entire 360-degree area from the center C. In FIG. 5, the celestial sphere in the upper half of the virtual space 90 is illustrated to avoid complicating the explanation. Meshes are defined in the virtual space 90. The position of each mesh is defined in advance as a coordinate value in the XYZ coordinate system, which is a global coordinate system defined in the virtual space 90. Each partial image that constitutes a panoramic image 91 (still image, video, etc.) that can be deployed in the virtual space 90 is associated with a corresponding mesh in the virtual space 90.

For example, in virtual space 90, an XYZ coordinate system is defined with center C as the origin. The XYZ coordinate system is, for example, parallel to the real coordinate system. The horizontal direction, vertical direction (up-down direction), and front-to-back direction in the XYZ coordinate system are defined as the X-axis, Y-axis, and Z-axis, respectively. Therefore, the X-axis (horizontal direction) of the XYZ coordinate system is parallel to the x-axis of the real coordinate system, the Y-axis (vertical direction) of the XYZ coordinate system is parallel to the y-axis of the real coordinate system, and the Z-axis (front-to-back direction) of the XYZ coordinate system is parallel to the z-axis of the real coordinate system.

A virtual camera 92 associated with the user terminal 20 is placed in the virtual space 90. The position of the virtual camera 92 in the virtual space 90 corresponds to the user's viewpoint in the virtual space 90. The orientation of the virtual camera 92 corresponds to the user's line of sight (reference line of sight 93) in the virtual space 90. The processor 21 then determines a viewing area 94 (the angle of view of the virtual camera 92) in the virtual space 90 based on the position and orientation of the virtual camera 92.

As shown in FIG. 6, the field of view 94 includes an area 95 in the YZ cross section. Area 95 is a range of polar angle α centered on the reference line of sight 93 in a vertical cross section (YZ cross section) including the reference line of sight 93 in the virtual space 90. As shown in FIG. 7, the field of view 94 includes an area 96 in the XZ cross section. Area 96 is a range of azimuth angle β centered on the reference line of sight 93 in a horizontal cross section (XZ cross section) including the reference line of sight 93 in the virtual space 90.

The processor 21 generates (extracts) a partial image included in the field of view 94 of the panoramic image 91 deployed in the virtual space 90 as a virtual space image 97 captured by the virtual camera 92. The processor 21 then displays the generated virtual space image 97 on the monitor 31. That is, the field of view 94 corresponds to the user's field of view in the virtual space 90. Furthermore, the field of view 94 moves in response to changes in the position and orientation of the virtual camera 92 in the virtual space 90, and the virtual space image 97 displayed on the monitor 31 is updated. That is, the user's field of view moves.

For example, the processor 21 moves the virtual camera 92 in the virtual space 90 in conjunction with the user's operation received by the operation device 42. The processor 21 also changes the orientation of the virtual camera 92 (i.e., the reference line of sight 93) in conjunction with the movement of the user terminal 20 detected by the motion sensor 41 (e.g., rotation around three mutually orthogonal axes). Furthermore, the processor 21 displays on the monitor 31 a virtual space image 97 captured by the virtual camera 92 after the position and orientation have been changed.

[First embodiment]
[Functional block diagram of server 10 and user terminal 20]
Fig. 8 is a functional block diagram of the server 10 and the user terminal 20 according to the first embodiment. As shown in Fig. 8, the server program 13P loaded in the memory 12 causes the server 10 to function as a synchronization means 110 and a relay means 120. In addition, the terminal program 23P loaded in the memory 22 causes the user terminal 20 (computer 26) to function as a definition means 210, a placement means 220, a generation means 230, a perception means 240, a selection means 250, a change means 260, a customization means 270, a notification means 280, and an assignment means 290.

The synchronization means 110 transmits synchronization data received from one of the user terminals 20 via the communication network 2 to all other user terminals 20. The synchronization data is data that should be synchronized and held by all user terminals 20. The synchronization data includes avatar data and item data. As a result, all user terminals 20A to 20C hold the same synchronization data. As a result, the same avatars and items are placed in the same positions in the virtual spaces 90 defined by each of the user terminals 20A to 20C. In other words, the avatars and items are synchronized in all user terminals 20A to 20C.

The relay means 120 relays the relay data received from the first user terminal 20 to the second user terminal 20. The relay data is data transmitted and received between specific user terminals 20. In other words, the relay means 120 transmits the relay data only to the user terminal 20 indicated by the destination information included in the relay data. However, the destination of the relay data is not limited to one. The relay data includes, for example, notification data, assignment data, and inquiry data, which will be described later.

The definition means 210 defines a virtual space 90 corresponding to the real space. More specifically, the definition means 210 expands virtual space data indicating the virtual space 90 in the memory 22. The virtual space data indicates, for example, a panoramic image 91 and the shapes and positions of basic objects (e.g., buildings, plants) to be placed in the virtual space 90. The virtual space data may also include BGM data indicating the BGM of the virtual space 90. The virtual space data may be downloaded in advance from the server 10 and stored in the storage 23, or may be downloaded from the server 10 when the virtual space 90 is defined. The definition means 210 may also define a virtual space 90 selected from among a plurality of virtual spaces 90 through the operation device 42. The specific process of defining the virtual space 90 is already known, so a detailed description will be omitted.

The placement means 220 places arbitrary objects in the virtual space 90 defined by the definition means 210. Here, a case where the terminal program 23P is executed on the user terminal 20A will be described. The arbitrary objects that the placement means 220 places in the virtual space 90 include, for example, the avatar of the user of the user terminal 20A (hereinafter referred to as the "user's avatar"), the avatars of the users of the other user terminals 20B and 20C (hereinafter referred to as the "other users' avatars"), and items.

An avatar refers to a character that moves within the virtual space 90 in response to user operations via the user terminal 20. An avatar (user's avatar, other person's avatar) is defined by avatar data. Avatar data indicates, for example, the position of the avatar within the virtual space 90 (XYZ coordinates) and the shape of the avatar (for example, height, skin color, facial expression, gestures). Avatar data also includes item data that indicates items worn by the avatar (for example, clothing, accessories, etc.).

An item refers to any object (e.g., furniture, food, drink, decorations, vehicles) placed within the virtual space 90. Items are distinguished from basic objects such as roads, buildings, and natural objects (sea, rivers, mountains) within the virtual space 90. Item data may be included in the virtual space data beforehand, or may be newly generated when the avatar takes out an item. The item data indicates the type of item, the shape of the item, and the position of the item within the virtual space 90.

First, the placement means 220 places the principal avatar in the virtual space 90 based on the avatar data pre-stored in the storage 23. Then, the placement means 220 transmits the avatar data of the principal avatar to the server 10 via the communication network 2. The placement means 220 also moves the principal avatar in the virtual space 90 based on an instruction for avatar movement via the motion sensor 41 or the operation device 42. Furthermore, the placement means 220 moves the virtual camera 92 in the virtual space 90 in accordance with the movement of the principal avatar. Then, the placement means 220 updates the avatar data to indicate the state of the principal avatar after the movement, and transmits the updated avatar data to the server 10 via the communication network 2.

The placement means 220 also receives avatar data of the other person's avatar from the server 10 via the communication network 2. The placement means 220 then places the other person's avatar in the virtual space 90 based on the avatar data received from the server 10. Furthermore, the placement means 220 rearranges the other person's avatar every time the avatar data is updated, thereby causing the other person's avatar to operate within the virtual space 90.

Furthermore, the placement means 220 places the item indicated by the item data in the virtual space 90. When the user's avatar moves an item, the placement means 220 updates the item data to indicate the state of the item after it has been moved, and transmits the updated item data to the server 10 via the communication network 2. When the user's avatar moves an item, the placement means 220 receives the updated item data from the server 10 via the communication network 2, and updates the position of the item in the virtual space 90.

The virtual space data, avatar data, and item data are data that define (specify) the virtual space 90. In other words, the virtual space data, avatar data, and item data are data that determine the manner in which the virtual space 90 is rendered. The rendering of the virtual space 90 refers to expressing the state of the virtual space 90 through at least one of the senses of sight, hearing, touch, smell, and taste.

The generating means 230 generates sensory data according to the positional relationship between the avatars and items arranged by the arranging means 220. The sensory data includes at least one of the visual data output by the monitor 31, the auditory data output by the speaker 36, the tactile data output by the haptic device 43, the olfactory data output by the olfactory device 44, and the gustatory data output by the gustatory device 45. In this specification, the visual data and the auditory data are collectively referred to as "auditory-visual data".

First, the generating means 230 generates a virtual space image 97 by capturing an image of the virtual space 90 with the virtual camera 92. More specifically, the generating means 230 extracts an image corresponding to the field of view 94 from the panoramic image 91 as the virtual space image 97, and stores virtual space image data representing the virtual space image 97 in the memory 22. The generating means 230 also arranges the virtual space images 97 generated at predetermined time intervals (e.g., 1/60th of a second) in chronological order to generate virtual space video data. The virtual space image data and the virtual space video data are examples of visual data.

The generating means 230 also extracts (generates) speech data of avatars in the virtual space 90 that are included in the field of view 94. The speech data is voice data input by the user of the user terminal 20 via the microphone 35, and is included in the avatar data. The generating means 230 also extracts (generates) preset background music data for the virtual space 90. The speech data and background music data are examples of auditory data.

The generating means 230 also extracts (generates) tactile data indicating the tactile sensation of an item touched by the personal avatar. The generating means 230 also extracts (generates) olfactory data indicating the smell of an item near the personal avatar. The generating means 230 also extracts (generates) taste data indicating the taste of an item placed in the mouth of the personal avatar. The tactile data, olfactory data, and taste data may be included in the item data, for example.

The perception means 240 allows the user of the user terminal 20 to perceive the perception data (i.e., the rendering of the virtual space 90) generated by the generation means 230. More specifically, the perception means 240 displays (outputs) an image or video represented by the visual data on the monitor 31, outputs a sound represented by the auditory data from the speaker 36, outputs a tactile sensation represented by the tactile data through the tactile device 43, outputs an odor represented by the olfactory data through the olfactory device 44, and outputs a taste represented by the taste data through the taste device 45.

The selection means 250 presents the user with a number of image candidates that represent the rendering image of the virtual space 90, and allows the user to select one of the presented image candidates. The image candidates are parameters that serve as indices for determining (unifying) the rendering image of the virtual space 90. The image candidates belong to one of a number of groups. A group is a name that groups together common image candidates.

Candidate images for the group "event" include, for example, "Christmas," "New Year's," and "Halloween." Candidate images for the group "weather" include, for example, "sunny," "rain," and "snow." Candidate images for the group "season" include, for example, "spring," "summer," "autumn," and "winter." Candidate images for the group "era" include, for example, "19th century," "20th century," and "21st century." Candidate images for the group "time" include, for example, "morning," "afternoon," and "night." Candidate images for the group "religion" include, for example, "Christianity," "Islam," and "Buddhism."

The number of image candidates belonging to one group is not particularly limited as long as it is two or more. The selection means 250 may change the number of image candidates presented to the user (for example, displayed on the selection screen 61 in FIG. 14(B)) according to the activity record of the principal avatar in the virtual space 90. As one example, the selection means 250 may increase the number of image candidates the longer the principal avatar's activity time in the virtual space 90. As another example, the selection means 250 may add a new image candidate in response to the principal avatar completing an event held in the virtual space 90.

Furthermore, the selection means 250 may allow the user to select whether or not to have the modification means 260 modify a plurality of modification targets for which the performance can be changed. As an example, the modification targets may be "sight," "hearing," "touch," "smell," "taste," etc., as shown in FIG. 14(B). As another example, the modification targets may be "appearance of other person's avatar," "voice of other person's avatar," "appearance of item," "smell of item," "taste of item," "BGM," etc. Note that the selection means 250 may allow the user to select whether or not to have the modification means 260 modify each of the plurality of modification targets individually, or may allow the user to select multiple modification targets collectively.

The modification means 260 modifies the presentation of the virtual space 90 from a common mode to a special mode in accordance with the image candidate selected by the user. The modification means 260 modifies the presentation of the virtual space 90, for example, based on a presentation database described later with reference to FIG. 15. The common mode is a presentation mode set in common to all user terminals 20A, 20B, and 20C. On the other hand, the special mode is a presentation mode set specifically for the user terminal 20 from which the image candidate was selected. The modification means 260 also modifies at least one of the presentations of the virtual space 90, including visual, auditory, taste, smell, and touch.

The customization means 270 allows the user to customize the special mode of presentation. For example, the customization means 270 allows the user to select the target of the presentation change (for example, the upper body of the avatar) and the mode of change (for example, the outfit). The customization means 270 then changes the target of the presentation selected by the user to the mode selected by the user (i.e., changing the outfit of the upper body of the avatar). The specific method by which the customization means 270 customizes the presentation is already well known, as described in, for example, Patent Documents 1 and 2, and therefore a detailed description will be omitted.

The notification means 280 notifies the user terminal 20 operating the avatar of the appearance of the avatar in the special mode in response to the change of the appearance of the avatar to the special mode by the change means 260 or the customization means 270. More specifically, the notification means 280 transmits notification data including an image showing the appearance of the avatar in the special mode and the terminal ID of the destination user terminal 20 to the server 10 via the communication network 2.

The assigning means 290 assigns item data (i.e., item data indicating clothing and accessories) for realizing a special appearance of the avatar to the user terminal 20 that operates the avatar. More specifically, the assigning means 290 transmits assignment data including the item data to be assigned and the terminal ID of the destination user terminal 20 to the server 10 via the communication network 2.

The user of the user terminal 20 that has received the assigned data can use the assigned item data to execute a switching operation to switch the appearance of the avatar. The appearance of the avatar switched by the switching operation is reflected on the monitors 31 of all user terminals 20A, 20B, and 20C. In other words, the appearance of the avatar changed by the switching operation is presented in a common manner.

[Operation of System 1 According to the First Embodiment]
The operation of the system 1 according to the first embodiment will be described with reference to Figs. 9 to 17. In the first embodiment, it is assumed that avatars 50A, 50B, and 50C operated by users A, B, and C of user terminals 20A, 20B, and 20C are present in a virtual space 90 simulating a room. The following description focuses on the processing of the user terminal 20A. That is, user A of the user terminal 20A is an example of a "first user," and users B and C of the user terminals 20B and 20C are examples of a "second user." In addition, the virtual cameras 92 of the user terminals 20A, 20B, and 20C are denoted as virtual cameras 92A, 92B, and 92C, respectively.

Fig. 9 is a flowchart showing the processing of the server 10 according to the first embodiment. Fig. 10 is a flowchart showing the processing of the user terminal 20A according to the first embodiment. Fig. 11 is a flowchart showing the processing of changing the presentation. Fig. 12 is an example of a virtual space image 97 displayed on the monitor 31 of the first user. Fig. 13 is an example of a virtual space image 97 displayed on the monitor 31 of the second user. Fig. 14 is an example of an interface that allows the first user to select an image candidate. Fig. 15 is an example of a presentation database. Fig. 16 is an example of the result of changing the virtual space image 97 displayed on the monitor 31 of the first user to a special mode. Fig. 17 is an example of an interface that allows the first user to customize the presentation. The processing of Fig. 9 by the server 10 and the processing of Fig. 10 by the user terminal 20A are executed in parallel.

The server 10 repeatedly executes the process shown in FIG. 9 at a predetermined time interval. First, the server 10 waits to execute the process from step S13 onwards until it receives synchronous data (S11) or relay data (S12) from the user terminals 20A to 20C. Note that the synchronous data and relay data are transmitted irregularly from each of the user terminals 20A to 20C.

When the server 10 (synchronization means 110) receives synchronization data from one of the user terminals 20A to 20C via the communication network 2 (S11: Yes), it transmits (relays) the synchronization data received via the communication network 2 to all other user terminals 20A to 20C (S13). This allows all user terminals 20A to 20C to hold the same synchronization data. Also, when the server 10 (relay means 120) receives relay data from one of the user terminals 20A to 20C via the communication network 2 (S12: Yes), it transmits (relays) the relay data via the communication network 2 only to the user terminal 20 indicated in the destination information (S14).

As shown in FIG. 10, the user terminal 20A (definition means 210) defines a virtual space 90 by expanding the virtual space data stored in the storage 23 in the memory 12 (S21). The virtual space data according to this embodiment includes item data indicating a table 51, a painting 52, a cake 53, and teacups 54, 55, and 56, and BGM data "default BGM." The user terminal 20A (placement means 220) then places the table 51, the painting 52, the cake 53, and the teacups 54, 55, and 56 in the virtual space 90 in accordance with the item data, as shown in FIG. 12(A).

Next, as shown in FIG. 12(A), the user terminal 20A (placement means 220) places the person's avatar 50A and the virtual camera 92A in the virtual space 90 defined in step S21, and transmits the avatar data to the server 10 via the communication network 2 (S22). The initial position of the person's avatar 50 is, for example, the center C of the virtual space 90. The position of the virtual camera 92 may be, for example, the head position of the person's avatar 50, or may be behind the person's avatar 50.

Next, when the user terminal 20A (placement means 220) receives avatar data from the server 10 via the communication network 2 (S23: Yes), it places other people's avatars 50B, 50C in the virtual space 90 according to the received avatar data (S24), as shown in FIG. 12(A). Similarly, when the user terminal 20A (placement means 220) receives item data from the server 10 via the communication network 2 (S23: Yes), it places items in the virtual space 90 according to the received item data (S24).

Next, the user terminal 20A (the generating means 230 and the perceiving means 240) generates and outputs perceptual data for allowing the user to perceive the rendering of the virtual space 90 (S29). First, the user terminal 20A (the generating means 230) identifies the field of view 94 of the virtual camera 92A, and generates a virtual space image 97 including the other person's avatars 50B, 50C, the table 51, the painting 52, the cake 53, and the teacups 54, 55, and 56 included in the identified field of view 94 as visual data. The user terminal 20A (the generating means 230) also extracts (generates) the BGM data "default BGM" included in the virtual space data as auditory data. Then, the user terminal 20A (the perceiving means 240) displays the generated virtual space image 97 on the monitor 31, as shown in FIG. 12(B), and outputs the music indicated by the BGM data "default BGM" from the speaker 36.

The same virtual space 90 is also generated in the user terminals 20A and 20B. However, since the viewpoints (virtual cameras 92B and 92C) of the avatars 50B and 50C are different, the virtual space image 97 displayed on the monitor 31 is different from that shown in FIG. 12(B). For example, the virtual camera 92B of the avatar 50B faces the avatars 50A and 50B from the position shown in FIG. 13(A). Therefore, the virtual space image 97 shown in FIG. 13(B) is displayed on the monitor 31 of the user terminal 20B. Although not shown, the virtual space image 97 including the avatars 50A and 50B is displayed on the monitor 31 of the user terminal 20C. The avatars 50B displayed in FIG. 12(B) and FIG. 13(B) should be facing in different directions, but in order to simplify the drawings, the avatars 50A, 50B, and 50C are all shown facing forward.

Next, when the user terminal 20A (placement means 220) receives an operation through the operation device 42 instructing the principal avatar 50 to move (S25: Yes), it moves the principal avatar 50 in accordance with the user's operation on the operation device 42, moves the virtual camera 92 following the principal avatar 50, and transmits avatar data showing the principal avatar 50 after the operation to the server 10 through the communication network 2 (S26). Then, the user terminal 20A (generation means 230 and perception means 240) generates and outputs perception data corresponding to the principal avatar 50A after the movement (S29).

As one example, when the user terminal 20A (arrangement means 220) moves the personal avatar 50A in the virtual space 90, the user terminal 20A (arrangement means 220) generates a virtual space image 97 included in the field of view 94 of the virtual camera 92 after the movement and displays it on the monitor 31. As another example, the user terminal 20A (arrangement means 220) outputs a voice input through the microphone 35 as the voice of the personal avatar 50A through the speaker 36. As another example, when the user terminal 20A (arrangement means 220) makes the personal avatar 50A hold a teacup 54, the user terminal 20A (arrangement means 220) generates tactile data indicating the touch of the teacup 54 and outputs it through the tactile device 43, and generates olfactory data indicating the smell of black tea and outputs it through the olfactory device 44. As yet another example, when the user terminal 20A (arrangement means 220) makes the personal avatar 50A drink the black tea in the teacup 54, the user terminal 20A (arrangement means 220) generates taste data indicating the taste of the black tea and outputs it through the taste device 45.

In this way, user terminals 20A, 20B, and 20C generate virtual space images 97 (visual data) captured from different viewpoints of the common virtual space 90, and display them on monitor 31. Similarly, user terminals 20A, 20B, and 20C output different auditory data, tactile data, olfactory data, and taste data generated (extracted) from the common virtual space 90 in accordance with the movements of avatars 50A, 50B, and 50C. In other words, the specific information perceived by the users of user terminals 20A, 20B, and 20C is different.

However, user terminals 20A, 20B, and 20C execute the processes of steps S21 to S26 and S29 using the same virtual space data, avatar data, and item data. That is, the processes of steps S21 to S26 and S29 executed by user terminals 20A, 20B, and 20C, respectively, are an example of allowing users to perceive the presentation of a common aspect of virtual space 90. In other words, users A, B, and C perceive the presentation of a common aspect through user terminals 20A, 20B, and 20C.

Also, as shown in Figs. 12 to 16, a menu icon 60 is displayed on the monitor 31. In response to receiving an operation to select the menu icon 60 through the operation device 42 (S27: Yes), the user terminal 20A (selection means 250) displays an "image change" icon and a "customize" icon on the monitor 31 as shown in Fig. 14(A), and executes the presentation change process shown in Fig. 11 (S28). The presentation change process is a process of changing the presentation of the virtual space 90 from a common mode to a special mode. In other words, the presentation change process is a process of changing the presentation of the virtual space 90 in the user terminal 20A to a special presentation, independent of the presentation of the virtual space 90 in the user terminals 20B and 20C.

In response to receiving an operation to select the [Image change] icon through the operation device 42 (S41: Image change), the user terminal 20A (selection means 250) displays a selection screen 61 on the monitor 31 as shown in FIG. 14(B) (S42). The selection screen 61 is a screen that allows the user to select the group "Event", "Weather", "Seasons", a number of image candidates belonging to the "Event" group, "Christmas", "New Year", and "Halloween", and the change targets "Vision", "Hearing", "Touch", "Smell", and "Taste" to be changed by the change means 260. The user terminal 20A (selection means 250) then receives the user's operation to select the group, image candidates, and change target through the operation device 42 (S43).

Next, in response to the selection of a group, an image candidate, and a change target via the selection screen 61 (S43: Yes), the user terminal 20A (change means 260) changes the presentation of the virtual space 90 in the user terminal 20A from a common mode to a special mode based on the presentation database shown in FIG. 15 (S44). Here, the following process will be explained assuming that the group "event", the image candidate "Christmas", and the change targets "sight", "hearing", and "taste" have been selected as shown in FIG. 14 (B).

Figure 15 shows a production database corresponding to the group "Event". The production database is a database that stores special modes of production methods (more specifically, synchronization data) corresponding to image candidates. For example, the production database stores, in association with the image candidate "Christmas", men's costume item data "Santa Costume 1", women's costume item data "Santa Costume 2", decoration item data "Christmas Wreath", food item data "Turkey", drink item data "Wine", and background music data "Jingle Bells".

As one example, the performance database is stored in the storage 23 of each of the user terminals 20A, 20B, and 20C. The performance database may be customized by the user of the user terminal 20. As another example, the performance database is stored in the storage 13 of the server 10. The server 10 may transmit the performance database to the user terminal 20 based on a request from the user terminal 20. The performance database may also be customized by an administrator of the server 10.

As an example, when the image candidate "Christmas" is selected, the user terminal 20A (modification means 260) changes the costumes of other avatars 50B and 50C to Santa costumes, the painting 52 to a Christmas wreath 52A, the cake 53 to a turkey 53A, the teacups 54, 55, and 56 to wine glasses 54A, 55A, and 56A, and changes the background music from "default background music" to "jingle bells" in the virtual space 90 defined on the user terminal 20A, as shown in FIG. 16(A).

As another example, when the image candidate "New Year" is selected, the user terminal 20A (change means 260) changes the clothes of other avatars 50B and 50C to Japanese clothing, changes the painting 52 to New Year decorations 52B, changes the cake 53 to osechi 53B, changes the teacups 54, 55, and 56 to sake cups 54B, 55B, and 56B, and changes the background music from "default background music" to "Happy New Year" in the virtual space 90 defined on the user terminal 20A, as shown in FIG. 16 (B).

In the above example, because "sight" and "taste" are selected as the changes to be made, the appearance of the teacup 54 is changed to a wine glass 54A or a sake cup 54B, and when the user's avatar 50A drinks from the contents, the taste of wine or sake is perceived. On the other hand, because "touch" and "smell" are not selected as changes to be made, the texture or smell of the changed item is not changed.

That is, the user terminal 20A (modification means 260) replaces the items placed in the virtual space 90 by the placement means 220 in steps S24 and S26 with items indicated by the item data extracted from the performance database. The user terminal 20A (modification means 260) also replaces the BGM data included in the virtual space data with the BGM data extracted from the performance database. In other words, the user terminal 20A (modification means 260) overwrites the synchronization data held by all users only on the user terminal 20A with the data extracted from the performance database. The performance of the virtual space 90 changed in this way is a special mode.

Note that the special modes that can be changed by the change means 260 are not limited to the above examples. As another example, the skin color or voice of the other person's avatars 50B and 50C may be changed. As another example, clouds, rain, snow, etc. may be added to match the candidate images of the group "weather." As another example, the sun, moon, and stars may be switched to match the candidate images of the group "time." As yet another example, a car or a carriage may be switched to match the candidate images of the group "era."

Next, the user terminal 20A (the generation means 230 and the perception means 240) generates perception data from the virtual space 90 that has been changed to a special mode presentation, and allows the user to perceive it (S45). The process of step S45 is the same as step S29 in FIG. 10. That is, the monitor 31 of the user terminal 20A displays the virtual space image 97 shown in FIG. 16(A) or FIG. 16(B). The other perception data is updated in a similar manner. As a result, the user A of the user terminal 20A perceives the special mode presentation of the virtual space 90. That is, the perception means 240 of the user terminal 20A is an example of a first perception means.

On the other hand, the data extracted from the rendering database is not transmitted to user terminals 20B and 20C. In other words, changes to the rendering of the virtual space 90 on user terminal 20A are not reflected in the other user terminals 20B and 20C. Therefore, the virtual space image 97 shown in FIG. 13(B) continues to be displayed on the monitor 31 of user terminal 20B. In other words, user terminals 20B and 20C continue to allow the user to perceive the rendering of the common aspect of the virtual space 90. In other words, the perception means 240 of user terminals 20B and 20C is an example of a second perception means.

The presentation of the special aspect is not limited to being perceived only by the user of the user terminal 20A (i.e., the first user who selected the image candidate), but may also be perceived by users who have a predetermined relationship with the first user. A user who has a predetermined relationship with the first user refers to a user who is closer physically or psychologically to the first user than other users, such as a user who is close to the first user in real space, a user who is registered in the first user's friend list (described below), or a user of an avatar who acts together with the first user's avatar in virtual space.

The user terminal 20A also determines whether or not the appearance of the other person's avatar has been changed in steps S44-S45 (S46). Then, in response to determining that the appearance of the other person's avatar has been changed (S46: Yes), the user terminal 20A displays on the monitor 31 the message "Would you like to give this outfit as a gift?" along with a [Yes] icon and a [No] icon, as shown in FIG. 16(A). The user terminal 20A then accepts the user's operation of selecting the [Yes] icon or the [No] icon through the operation device 42 (S47).

Next, in response to the selection of the [Yes] icon (S47: Yes), the user terminal 20A (granting means 290) transmits the grant data including the costume item data extracted from the performance database to the user terminals 20B and 20C via the server 10 (S48). The user terminals 20B and 20C receive the grant data via the server 10. Then, the user terminals 20B and 20C store the item data included in the grant data in the storage 23. Well-known billing processing and NFT (Non-Fungible Token) technology can be used to send and receive the grant data.

For example, user B of user terminal 20B can change the appearance of avatar 50B using item data (an example of appearance data) included in the assigned data. User terminal 20B then transmits avatar data indicating avatar 50B with the changed appearance to server 10 via communication network 2. This causes the change in the appearance of avatar 50B to be reflected in all user terminals 20A and 20C. In other words, user B of user terminal 20B can use the assigned data received from user terminal 20A to adopt the appearance of avatar 50B in a special mode on user terminal 20A as a common mode. The same applies to user terminal 20C.

Meanwhile, in response to the selection of the [No] icon (S47: No), the user terminal 20A (notification means 280) transmits notification data including an image showing the appearance of the avatars 50B, 50C changed to a special mode to the user terminals 20B, 20C via the server 10 (S49). The user terminals 20B, 20C receive the notification data via the server 10. Then, the user terminals 20B, 20C display the image included in the notification data on the monitor 31, thereby notifying the user that the appearance of the avatars 50B, 50C has been changed on the user terminal 20A.

In response to receiving an operation to select the [Customize] icon through the operation device 42 (S41: Customize), the user terminal 20A (customization means 270) displays a pointer 62 for selecting a body part for customizing the effects on the monitor 31, as shown in FIG. 17(A). Next, when a body part to be customized (e.g., the lower body of the avatar 50B) is selected by the pointer 62, the user terminal 20A (customization means 270) displays an item list of items 63, 64 that can customize the effects of that body part (in this case, dressing up) on the monitor 31 (S50). Then, the user terminal 20A (customization means 270) receives a user operation to select one of the items 63, 64 in the item list through the operation device 42 (S51).

Next, in response to the selection of item 64 (S51: Yes), for example, the user terminal 20A (customization means 270) replaces (customizes) the costume (effect) of the body part selected through the pointer 62 with the item 64 selected from the item list (S52), as shown in FIG. 17(B). Next, the user terminal 20A executes the processes of steps S45 to S49 to reflect the effect customized in step S52. Note that the processes of steps S50 to S52 are not limited to visual customization, but can also be applied to auditory, tactile, olfactory, and gustatory customization. Also, the processes of steps S50 to S52 may be executed only for the body part changed to a special mode effect in step S44, or may be executed for the body part that still has the common mode effect.

The user terminal 20A then repeatedly executes the process of FIG. 10 until an operation (e.g., logout) instructing the user to terminate the terminal program 23P is received through the operation device 42. Note that the user terminal 20A irregularly receives synchronization data and irregularly accepts user operations. In other words, the processes of steps S24, S26, and S28 are executed at any timing and frequency.

[Effects of the First Embodiment]
According to the first embodiment, the rendering of the virtual space 90 is changed in accordance with the image candidate selected by the user, so that the rendering of the virtual space 90 can be unified to the image represented by the selected image candidate. This allows the rendering of the virtual space 90 to be changed to a desired image with a simple operation, compared to changing the rendering by individually specifying a plurality of body parts.

More specifically, by collectively changing the effects of multiple items (avatars, items, background music, etc.) registered in the effect database, the effects of the entire virtual space 90 can be changed in a unified manner. In addition, by updating the effect database, it is possible to incorporate trendy effects. However, the change means 260 is not limited to changing the effects based on the effect database, and may use AI technology to change the effects to match candidate images.

Furthermore, according to the first embodiment, by allowing the user to select the target for which the performance is to be changed, the performance can be changed to the extent desired by the user. This makes it possible to prevent excessive changes to the performance.

Furthermore, according to the first embodiment, by making it possible to customize the presentation that has been changed to a special mode, the presentation of the virtual space 90 can be made to match the user's preferences.

In addition, according to the first embodiment, changes to the appearance of other users' avatars 50B, 50C are notified to the users who operate those avatars 50B, 50C, which is expected to have the effect of deterring other users from making changes to their appearance that go against their will.

Furthermore, according to the first embodiment, by providing the user of the avatar 50B, 50C with item data that changes the appearance of the other person's avatar 50B, 50C, not only the first user who wishes to change the presentation of the virtual space 90 can enjoy the benefits, but also the second user of the other person's avatar 50B, 50C.

Furthermore, according to the first embodiment, the number of image candidates displayed on the selection screen 61 can be changed according to the activity record of the person's avatar 50A within the virtual space 90, thereby encouraging activity within the virtual space 90. This also contributes to revitalizing the virtual space 90.

[Second embodiment]
A system 1 according to the second embodiment will be described with reference to Figures 18 to 22. Note that a detailed description of the points in common with the first embodiment will be omitted, and the differences will be mainly described.

[Functional block diagram of server 10 and user terminal 20]
Fig. 18 is a functional block diagram of the server 10 and the user terminal 20 according to the second embodiment. The functional blocks of the server 10 according to the second embodiment are common to those of the first embodiment. Meanwhile, in the second embodiment, the terminal program 23P loaded into the memory 22 causes the user terminal 20 (computer 26) to function as a definition means 210, a placement means 220, a generation means 230, a perception means 240, a change means 300, a duplication means 310, and a diffusion control means 320, as shown in Fig. 18. The roles of the definition means 210, the placement means 220, the generation means 230, and the perception means 240 according to the second embodiment are common to those of the first embodiment.

The modification means 300 changes the presentation of the virtual space 90 from a common appearance to a special appearance in accordance with the user's operation via the operation device 42. In particular, the modification means 300 according to the second embodiment changes the appearance of the other person's avatars 50B, 50C from a common appearance to a special appearance (i.e., replaces the clothes and accessories). As one example, the modification means 300 may execute a process common to the modification means 260 and the customization means 270 according to the first embodiment. As another example, the modification means 300 may change the presentation of the virtual space 90 by the method described in Patent Documents 1 and 2.

The object that the modification means 300 modifies in the manner of presentation is not limited to an avatar, but may be an item, so long as it is an object that the user places in the virtual space 90 through the user terminal 20. Also, the object that the modification means 300 modifies in the manner of presentation is not limited to an object that the user places in the virtual space 90, so long as it is an object that is associated with the user and placed in the virtual space 90. Furthermore, the object that the modification means 300 modifies is not limited to appearance (i.e., visual data), but may be auditory data, tactile data, olfactory data, or taste data.

The duplication means 310 duplicates the sensory data generated by the generation means 230 (in other words, the sensory data that the perception means 240 causes the user to perceive). As one example, duplication may refer to capturing sensory data and storing it in the storage 23. As another example, duplication may refer to uploading sensory data to a shared server (e.g., an SNS server, a file exchange server, a streaming server) in order to share the sensory data with other user terminals 20. Furthermore, the sensory data after duplication is not limited to being completely identical to the sensory data before duplication, and may be partially edited (modified). In other words, editing sensory data and then duplication is also included in the category of "duplication" in this specification.

The diffusion control means 320 controls to restrict diffusion of sensory data including objects of a special aspect (hereinafter, sometimes referred to as "sensory data of a special aspect") by the user of the user terminal 20. "Diffusion of sensory data by a user" refers to an action (operation) of the user that may cause unspecified people to perceive the sensory data. Diffusion may include not only a direct action that causes unspecified people to perceive the sensory data, but also an action that is highly likely to cause such diffusion, or an action that precedes such an action (e.g., capture). Diffusion may also be intended not to cause a large number of people to perceive the data, but to cause one unspecified person to perceive it.

The diffusion control means 320, for example, controls the copying means 310 to copy sensory data in a special mode. As one example, the control of copying may refer to prohibiting the copying of sensory data in a special mode. As another example, the control of copying may refer to controlling sensory data that has been returned from a special mode to a common mode so that it can be spread (i.e., having the copying means 310 copy it). As yet another example, the control of copying may refer to controlling sensory data in which the part changed to the special mode is masked so that it can be spread (i.e., having the copying means 310 copy it). However, the object of control by the diffusion control means 320 is not limited to copying by the copying means 310, but refers to any process in which sensory data in a special mode can be perceived by unspecified people.

[Operation of System 1 According to the Second Embodiment]
The operation of the system 1 according to the second embodiment will be described with reference to Figures 19 to 22. Figure 19 is a flowchart showing the processing of the user terminal 20A according to the second embodiment. Figure 20 is a flowchart of the copy control processing. Figure 21 is an example of friend list data. Figure 22 is an example of a virtual space image 97 displayed on the monitor 31 of the user terminals 20A and 20B. The premise of the virtual space 90 in the second embodiment is the same as in the first embodiment.

The processing of steps S61 to S66 and S70 in FIG. 19 is common to steps S21 to S26 and S29 in FIG. 10. As a result, the virtual space image 97 shown in FIG. 12(B) is displayed on the monitor 31 of the user terminal 20A, the virtual space image 97 shown in FIG. 13(B) is displayed on the monitor 31 of the user terminal 20B, and a virtual space image (not shown) including avatars 50A and 50B is displayed on the monitor 31 of the user terminal 20C. These are examples of perceiving sensory data including avatars 50A, 50B, and 50C of a common aspect.

In the examples of FIG. 12(B) and FIG. 13(B), avatar 50B and the items worn by avatar 50B are an example of an object placed in virtual space 90 by user B (an object associated with user B and placed in virtual space 90). Furthermore, if user B takes cake 53 from his/her item list and places it in virtual space 90, cake 53 is another example of an object placed in virtual space 90 by user B (an object associated with user B and placed in virtual space 90). In other words, an object associated with a second user and placed in virtual space 90 refers to an object for which the second user has the power to decide the presentation style.

When the menu icon 60 according to the second embodiment is selected, a "Change" icon and a "Copy" icon are displayed on the monitor 31 as shown in FIG. 22(A). Then, in response to receiving an operation to select the "Change" icon via the operation device 42 (S67: Change), the user terminal 20A (change means 300) changes the presentation of the virtual space 90 from a common mode to a special mode in accordance with the user operation (S68). In step S68, for example, the user terminal 20A (change means 300) executes the process of steps S42 to S44 in FIG. 11, the process of steps S50 to S52 in FIG. 11, or the process of changing the costume of the avatar described in Patent Documents 1 and 2.

Next, the user terminal 20A (the generation means 230 and the perception means 240) generates and outputs perception data from the virtual space 90 in which the presentation has been changed to a special mode (S70). As a result, for example, a virtual space image 97 shown in FIG. 16(A) is displayed on the monitor 31 of the user terminal 20A. This process is an example of perceiving perception data including an avatar in a special mode. Meanwhile, a virtual space image 97 including avatars 50A, 50B, and 50C in a common mode is displayed on the monitors 31 of the user terminals 20B and 20C. This process is an example of perceiving perception data including an avatar in a common mode.

In addition, in response to receiving an operation to select the [Copy] icon through the operation device 42 (S67: Copy), the user terminal 20A (diffusion control means 320) executes the copy control process shown in FIG. 20 (S69). The copy control process is a process for controlling (permitting, restricting, prohibiting) copying of the special mode sensory data generated in the immediately preceding step S70.

First, the user terminal 20A (diffusion control means 320) determines whether or not the sensory data includes an avatar of another person in a special mode (S81). Then, in response to determining that the sensory data does not include an avatar of another person in a special mode (S81: Yes), the user terminal 20A (diffusion control means 320) permits the duplication means 310 to duplicate the sensory data of the special mode (S82). That is, in step S82, the user terminal 20A (duplication means 310) duplicates the sensory data without editing it. The same applies to steps S84 and S87 described below.

In addition, in response to determining that the sensory data includes other person avatars 50B, 50C in a special mode (S81: No), the user terminal 20A (diffusion control means 320) determines whether the users who operate the other person avatars 50B, 50C are registered in the friend list shown in FIG. 21(A) (S83). In response to determining that all users B, C of the other person avatars 50B, 50C in a special mode are registered in the friend list shown in FIG. 21(A) (S83: Yes), the user terminal 20A (diffusion control means 320) permits the duplication means 310 to duplicate the sensory data in the special mode (S84).

A friend list is provided for each of users A, B, and C who share virtual space 90. When the "friend request" and "acceptance" procedures are carried out between users A and C, "user C" is registered in user A's friend list, and "user A" is registered in user C's friend list. When the "friend request" and "acceptance" procedures are carried out between users B and C, "user C" is registered in user B's friend list, and "user B" is registered in user C's friend list.

The friend request and acceptance procedure is an example of a mutual agreement between two users. In other words, when two users reach a mutual agreement, one user is registered in the friend list of the other user. On the other hand, the "list" in this specification is different from a list registered solely by the will of one user, such as a favorites list or follow list. The friend list may be stored in the storage 23 of the user terminals 20A, 20B, and 20C, or may be stored in the storage 13 of the server 10. A user registered in the friend list is an example of a user who has a predetermined relationship with the first user.

Meanwhile, in response to determining that user B of the special mode other person avatar 50B is not registered in the friend list shown in FIG. 21 (A) (S83: No), the user terminal 20A (diffusion control means 320) transmits inquiry data including an image showing the appearance of the avatar 50B changed to the special mode to the user terminal 20B via the server 10 (S85). The user terminal 20B receives the inquiry data via the server 10. Then, as shown in FIG. 22 (B), the user terminal 20B displays on the monitor 31 the message "Do you allow copying of this avatar?", the image included in the inquiry data, a [Yes] icon, and a [No] icon.

Then, in response to receiving an operation to select the [Yes] icon through the operation device 42, the user terminal 20B transmits permission data to the user terminal 20A via the server 10. The selection of the [Yes] icon or the transmission of permission data is an example of permission by the user of the user terminal 20B (i.e., the second user). On the other hand, in response to receiving an operation to select the [No] icon through the operation device 42, the user terminal 20B transmits refusal data to the user terminal 20A via the server 10.

Next, in response to receiving permission data from the user terminal 20B via the server 10 (S86: Yes), the user terminal 20A (diffusion control means 320) permits the duplication means 310 to duplicate the sensory data of the special mode (S87). On the other hand, in response to receiving rejection data from the user terminal 20B via the server 10 (S86: No), the user terminal 20A (diffusion control means 320) restricts the duplication means 310 from duplication of the sensory data of the special mode (S88). For example, in step S88, the user terminal 20A (diffusion control means 320) may prohibit the duplication means 310 from duplication of the sensory data of the special mode by the duplication means 310, may permit the duplication means 310 to duplicate the sensory data that has been restored from the special mode to the common mode by the duplication means 310, or may permit the duplication means 310 to duplicate the sensory data (in this case, visual data) in which the portion changed to the special mode is masked (e.g., mosaic).

In steps S82, S84, S87, and S88, the copying means 310 copies the sensory data according to the control of the diffusion control means 320 (in other words, within the scope permitted by the diffusion control means 320). Allowing, restricting, or prohibiting the copying of sensory data by the copying means 310 is an example of control that restricts the diffusion of special-mode sensory data by the user of the user terminal 20A. However, the content and order of control that restricts diffusion are not limited to the example in FIG. 20.

[Effects of the Second Embodiment]
According to the second embodiment, by controlling the duplication of sensory data including the other person's avatars 50B, 50C in a special mode, it is possible to prevent the spread of changes to the other person's avatars 50B, 50C that are contrary to the will of the users B, C. Meanwhile, on the user terminal 20A, the other person's avatars 50B, 50C can be changed to a special mode, so that the diversity of ways to enjoy the virtual space 90 can be maintained.

In addition, according to the second embodiment, rather than completely prohibiting the duplication of sensory data, by returning the other person's avatars 50B, 50C to a common aspect or masking and duplicating the special aspect parts of the other person's avatars 50B, 50C, it is possible to both diversify ways of enjoying the virtual space 90 and prevent the spread of unwanted changes to the other person's avatars 50B, 50C.

Furthermore, according to the second embodiment, when user B permits duplication of the other person's avatar 50B in a special mode, the perceptual data including the other person's avatar 50B in a special mode is duplicated. This makes it possible to spread the changes to the other person's avatar 50B permitted by user B. However, the inquiry and response regarding whether or not duplication of the other person's avatar 50B in a special mode is not limited to those made through the virtual space 90 as in steps S85 and S86. As another example, the user terminal 20A may transmit and receive the inquiry data, permission data, and refusal data directly to and from the user terminal 20B without going through the server 10.

Furthermore, according to the second embodiment, when user C is registered on the friend list, the sensory data including the special mode other person's avatar 50C is replicated. This is because user C, who is registered on the friend list, is considered to have a close relationship with user A, and therefore user C can be considered to have given tacit consent.

However, the control to limit the spread is not limited to sensory data including other person's avatars 50B, 50C in a special aspect, but may be performed on sensory data including items in a special aspect (e.g., turkey 53A, wine glasses 54A, 55A, 56A, etc.). Furthermore, the special aspect is not limited to visual data, but may be auditory data, tactile data, olfactory data, or taste data. As another example, the spread control means 320 may control to limit the spread of avatar data showing other person's avatars 50B, 50C whose voice tone has been changed. As yet another example, the spread control means 320 may control to limit the spread of item data in which the appearance, taste, and smell of cake 53 have been changed to turkey 35A.

[Other embodiments]
Furthermore, the program according to the present invention is not limited to a single program, but may be a collection of multiple programs. Furthermore, the program according to the present invention is not limited to being executed by a single device, but may be executed by multiple devices in a shared manner. Furthermore, the division of roles between the server 10 and the user terminal 20 is not limited to the above example. That is, part of the processing of the server 10 may be executed by the user terminal 20, and part of the processing of the user terminal 20 may be executed by the server 10.

Furthermore, some or all of the means realized by the program can be realized by hardware such as an integrated circuit. Furthermore, the program may be provided in a form recorded on a non-transitory recording medium that can be read by a computer. Examples of recording media include hard disks, SD cards, DVDs, and servers on the Internet.

1...system, 2...communication network, 3...detection device, 10...server, 11, 21...processor, 12, 22...memory, 13, 23...storage, 13P...server program, 14, 24...input/output interface, 15, 25...communication interface, 19, 29...communication bus, 20...user terminal, 23P...terminal program, 26...computer, 30...HMD, 31...monitor, 32...gaze sensor, 33, 34...camera, 35...microphone, 36...speaker, 41...motion sensor, 42...operation device, 43...tactile device, 44...olfactory device, 45...taste device, 50A...user's avatar, 50B, 50C...other person's avatar, 51...table, 52...painting, 52A...Christmas wreath, 52B...New Year's decoration, 53...cake, 53A...turkey, 5 3B...New Year's food, 54, 55, 56...Teacup, 54A, 55A, 55B...Wine glass, 54B, 55B, 55B...Sake cup, 60...Menu icon, 61...Selection screen, 62...Pointer, 63, 64...Item, 90...Virtual space, 91...Panoramic image, 92...Virtual camera, 92A...Self-viewpoint camera, 92B, 92B, 92C, 92D...Set viewpoint camera, 93...reference line of sight, 94...visual field of view, 95, 96...area, 97...virtual space image, 110...synchronization means, 120...relay means, 210...definition means, 220...placement means, 230...generation means, 240...perception means, 250...selection means, 260, 300...change means, 270...customization means, 280...notification means, 290...addition means, 310...copying means, 320...diffusion control means

Claims (7)

Computer,
A selection means for allowing a first user of the multiple users to select one of a plurality of image candidates representing a rendering image of the virtual space;
A change means for collectively changing the performance of each of a plurality of change targets included in the virtual space from a common appearance perceived by a plurality of users other than the first user to a special appearance corresponding to the selected image candidate;
A program that functions as a perception means for allowing the first user to perceive the virtual space including a plurality of change objects whose presentation has been changed to the special mode. The program according to claim 1,
The selection means is a program that allows the first user to select whether or not to cause the change means to make a change to a plurality of change targets that can change the presentation. The program according to claim 1,
A program that causes the computer to function as a customization means that allows the first user to customize the presentation of the special mode. The program according to claim 1,
A program that causes the computer to function as a notification means that notifies the second user of the appearance of the avatar in the special mode through the virtual space in response to the appearance of an avatar operated by a second user different from the first user being changed to the special mode by the change means. The program according to claim 4,
A program that causes the computer to function as an assigning means that assigns appearance data indicating the special appearance of the avatar to the second user through the virtual space so that the second user can adopt the appearance of the avatar in the special appearance as the common appearance. The program according to claim 1,
The selection means is a program that changes the number of image candidates presented to the first user depending on the amount of time the avatar operated by the first user is active in the virtual space or the number of events held in the virtual space that are completed. A selection means for allowing a first user of the multiple users to select one of a plurality of image candidates representing a rendering image of the virtual space;
A change means for collectively changing the performance of each of a plurality of change targets included in the virtual space from a common appearance perceived by all users other than the first user to a special appearance corresponding to the selected image candidate;
A first perception means for allowing the first user to perceive the virtual space including the plurality of change targets whose performance has been changed to the special mode;
The system further comprises a second perception means for allowing a second user different from the first user to perceive the virtual space, the performance of which includes a plurality of the change targets having the common aspect.

Images