JP7762385B2 - Game system, wireless capture device, and game control method - Google Patents

Description

This disclosure relates to technology for controlling a game console by detecting the actions of a user or the movement of a controller within a communication area from propagation path information of wireless signals communicated between wireless devices.

Known methods for controlling game consoles include using a controller connected to the game console via a wired connection, or using a controller connected to the game console wirelessly. Furthermore, Non-Patent Document 1 discloses a method for controlling a game console by connecting a camera to the game console and having the user make gestures in front of the camera.

Kinect Mechanism and Natural User Interface, Journal of the Institute of Image Information and Television Engineers, Vol. 66, No. 9, pp. 755-759 (2012).

However, with methods that use a controller connected to the game console via a wired connection, or methods in which the user makes gestures in front of the camera, the user must be within a certain range of the game console, which limits the degree of freedom in positioning.

Furthermore, while using a controller wirelessly connected to a game console gives the user greater freedom in positioning, there is the issue that the controller requires a battery.

This disclosure has been made to solve the above-mentioned problems, and its primary objective is to provide a game system that allows the user greater freedom in positioning while playing the game and eliminates the need for batteries in the controller.

Furthermore, a second objective of this disclosure is to provide a wireless capture device that allows the user greater freedom in positioning while playing the game and eliminates the need for a battery in the controller.

Furthermore, a third objective of this disclosure is to provide a game control method that allows the user playing the game greater freedom in their position and eliminates the need for a battery in the controller.

A first aspect of the present disclosure is a game system that detects a user's action or a controller's movement using wireless communication, comprising:
a first wireless device having two or more antennas;
a second wireless device that wirelessly communicates with the first wireless device;
a wireless capture device that captures the wireless communication signal and detects the user's actions or the movement of the controller within the area of the wireless communication;
a game machine that controls a game based on label data indicating game operations transmitted from the wireless capture device;
Equipped with
the first wireless device is configured to perform a process of transmitting a wireless signal including a known wireless frame from the antenna;
The second wireless device
a process of calculating information of a wireless propagation path between the first wireless device and the second wireless device from the known wireless frame;
a process of returning a wireless frame including information on the wireless propagation path to the first wireless device;
configured to run
The wireless capture device
capturing a wireless frame including information about the wireless propagation path;
A process of extracting information about the wireless propagation path;
an operation determination process for determining a game operation or a controller operation performed by a user within the wireless communication area based on the information on the wireless propagation path;
a process of transmitting the game operation or the controller operation performed by the user to the game machine as label data;
configured to run
The game machine is preferably configured to execute a process for controlling a game based on the label data transmitted from the wireless capture device.

A second aspect of the present invention is a method for capturing a wireless frame including information about a wireless propagation path transmitted from a second wireless device to a first wireless device ;
A process of extracting information about the wireless propagation path;
determining, from the information on the wireless propagation path, a game operation or a controller operation performed by a user within an area of wireless communication between the first wireless device and the second wireless device ;
a process of transmitting the game operation or the controller operation performed by the user to the game machine as label data;
Preferably, the wireless capture device is configured to perform the following:

A third aspect is a game control method that detects a user's action or a controller movement using wireless communication, comprising:
the first wireless device has two or more antennas and performs a process of transmitting a wireless signal including a known wireless frame from the antennas;
The second wireless device
a process of calculating information on a wireless propagation path between the first wireless device and the second wireless device from the known wireless frame transmitted from the first wireless device;
a process of returning a wireless frame including information on the wireless propagation path to the first wireless device;
Run
The wireless capture device
capturing a wireless frame including information about the wireless propagation path;
A process of extracting information about the wireless propagation path;
determining, from the information on the wireless propagation path, a game operation or a controller operation performed by a user within a wireless communication area between the first wireless device and the second wireless device;
a process of transmitting the game operation or the controller operation performed by the user to the game machine as label data;
Run
It is desirable that the game machine executes a process for controlling the game based on the label data transmitted from the wireless capture device.

The first to third aspects of the present disclosure provide a game system, a wireless capture device, and a game control method that allow a user playing a game with greater freedom in their position and eliminates the need for a battery in the controller.

1 illustrates an example configuration of a game system according to a first embodiment of the present disclosure. 1 illustrates an example configuration of a wireless sensing device and a game machine according to a first embodiment of the present disclosure. 10 is a diagram illustrating an example sequence of wireless communication between an AP and a STA according to the first embodiment of the present disclosure. 1 is a flowchart for creating training data used in machine learning by a wireless sensing device according to a first embodiment of the present disclosure. 10 is a flowchart showing how the game machine controls a game based on a user's action or a controller movement according to the first embodiment of the present disclosure. 10 is a configuration example of a game system according to a second embodiment of the present disclosure. 10 illustrates an example configuration of a game system according to a third embodiment of the present disclosure.

First Embodiment
FIG. 1 shows an example of the configuration of a game system according to the first embodiment of the present disclosure.

The wireless LAN AP (hereinafter referred to as AP) 101 performs wireless LAN communication based on the wireless LAN standard IEEE802.11ac/ax. The AP 101 is equipped with two or more antennas 105(1), 105(2), ... 105(n). The AP 101 uses these antennas simultaneously to perform MIMO (Multi-Input Multi-Output) communication with the wireless LAN terminal (hereinafter referred to as STA) 102.

AP101 transmits a wireless signal containing a known frame to STA102. AP101 also receives the wireless signal returned from STA102 via antenna 105.

STA102 receives a wireless signal from AP101. Based on the received wireless signal, STA102 calculates the difference from a known frame and derives compressed channel state information. Note that compressed channel state information (CSI) refers to information about the wireless propagation path. Furthermore, STA102 stores the compressed CSI information in a Compressed Beamforming Report (CBFR) frame and feeds it back to antenna 105 of AP101.

The wireless sensing device 103 captures wireless packets fed back from the STA 102 to the AP 101. Furthermore, the wireless sensing device 103 extracts compressed CSI from the captured packets based on the IEEE 802.11ac/ax standard. Furthermore, it analyzes the game operation content based on the compressed CSI and outputs label data indicating the operation content to the game console 104.

The game console 104 controls the game based on the label data output from the wireless sensing device 103.

In the example shown in Figure 1, the wireless sensing device 103 is placed midway between the AP 101 and the STA 102, but it may be placed anywhere as long as it can capture wireless packets transmitted from the STA 102.

Also, in the example of Figure 1, the game console 104 may be equipped with a display or the like.

Within the range where AP101 and STA102 communicate wirelessly, user 106 operates the game using gestures, changes in position, changes in posture, etc., or by pressing buttons on the controller 107 of the game console 104. However, if user 106 operates the game using gestures, etc., the controller 107 does not have to be used.

The actions and other information performed by user 106 within the wireless communication area between AP 101 and STA 102 is measured using compressed CSI transmitted and received between AP 101 and STA 102. The wireless sensing device 103 captures wireless packets containing this compressed CSI information, enabling the actions and other information of user 106 to be detected.

Figure 2 shows an example configuration of a wireless sensing device and a game console according to embodiment 1 of the present disclosure.

The wireless interface unit 110 sends and receives wireless frames according to the IEEE 802.11ac/ax (Wi-Fi 5/Wi-Fi 6) standard specifications. The capture unit 111 acquires frames received by the wireless interface unit 110 and outputs them to the filtering unit 112.

The filtering unit 112 outputs to the extraction unit 113 only those wireless frames acquired by the capture unit 111 that meet predetermined conditions. The predetermined conditions include, for example, that the source wireless terminal station is the target STA 102 and that the destination wireless terminal station is the target AP 101. The predetermined conditions also include that the type of wireless frame is a frame that includes the target CSI, i.e., a CBFR frame.

The extraction unit 113 extracts the compressed CSI from the CBFR frame output from the filtering unit 112. The pre-processing unit 114 calculates the compressed CSI and detects its changes over time. If a change over time is detected in the compressed CSI, the pre-processing unit 114 outputs the compressed CSI data to the allocation unit 115.

The allocating unit 115 outputs the compressed CSI data to either the teacher data creation unit 117 or the determination unit 118 within the machine learning unit 116, depending on the machine learning phase. The machine learning phase is determined by whether or not label data has been input to the allocating unit 115 from the game console 104. If label data has been input, the allocating unit 115 allocates the compressed CSI information to the teacher data creation unit 117. On the other hand, if label data has not been input, the information is allocated to the determination unit 118.

The machine learning unit 116 performs machine learning using training data. When the machine learning unit 116 is in the learning phase, the training data creation unit 117 creates training data to be used in machine learning. Specifically, it creates training data by matching the compressed CSI data output from the sorting unit 115 with the label data output from the game console 104.

On the other hand, when the machine learning unit 116 is in the judgment phase, the teacher data creation unit 117 provides the judgment unit 118 with teacher data to be used in machine learning.

When the machine learning unit 116 is in the judgment phase, the judgment unit 118 performs machine learning judgment on the compressed CSI data output from the allocation unit 115. Specifically, it uses the compressed CSI training data provided by the training data creation unit 117 to judge the game operation or operation of the controller 107 performed by the user 106. Furthermore, the judgment unit 118 transmits label data indicating the determined operation content to the game console 104.

The sensor result input unit 119 in the game console 104 is an interface that receives label data output from the determination unit 118 of the wireless sensing device 103. Label data corresponding to, for example, the position, gesture, and posture of the user 106, the state of buttons pressed on the controller 107, and the state of the user 106's fingers are input to the sensor result input unit 119.

The controller communication unit 120 converts the label data output from the sensor result input unit 119 into movements of the controller 107 of the game console 104 and outputs it to the game control unit 121. The movements of the controller 107 are, for example, movements such as pressing a specific button. The game control unit 121 is a block within the game console 104 that controls the input signals of the controller 107. The game console 122 is a typical game console. The label creation unit 123 outputs, as label data, to the wireless sensing device 103 the actions that the user 106 should perform in order to create training data in the training data creation unit 117.

With the above configuration, the wireless sensing device 103 can extract compressed CSI information from captured wireless packets and detect changes over time in the user's 106 movements, etc. Furthermore, machine learning can be used to determine the game operations or controller 107 operations performed by the user 106. Furthermore, the operation details determined by the wireless sensing device 103 can be transmitted to the game console 104 as label data, allowing the game console 104 to control the game.

Figure 3 shows an example sequence of wireless communication between an AP and a STA according to the first embodiment of the present disclosure. AP 101 transmits known NDP (Null Data Packet) frames 130(1), 130(2), ... 130(n) from antennas 105(1), 105(2), ... 105(n), respectively.

Here, it is assumed that the NDP frame 130 will be transmitted periodically, but because the data volume of the NDP frame 130 is not large, it is unlikely to interfere with the original communication between the AP 101 and the STA 102. Furthermore, for example, Wi-Fi supports multiple frequencies, so by transmitting the NDP frame 130 using a frequency band or frequency channel different from the original communication, it is possible to avoid affecting the original communication. Wi-Fi supports three frequencies: the 2.4 GHz band, the 5 GHz band, and the 6 GHz band, which is expected to be used in the future. Furthermore, a maximum of 3 channels can be used in the 2.4 GHz band, a maximum of 20 channels in the 5 GHz band, and a maximum of 25 channels in the 6 GHz band.

When STA 102 receives NDP frames 130(1), 130(2), ... 130(n) from all antennas 105, it calculates compressed CSI from the NDP frames 130 according to the method specified in IEEE 802.11ac/ax.

STA102 stores the calculated compressed CSI information in a CBFR frame 131 and feeds it back to AP101.

Based on the compressed CSI information fed back from STA102, AP101 obtains information about the wireless propagation paths between each of antennas 105(1), 105(2), ... 105(n) and STA102. AP101 controls antenna 105 based on this information, adjusting the amplitude and phase of the wireless signal transmitted by antenna 105 to further improve the efficiency of wireless communication.

In this way, by utilizing the wireless communication sequence between AP101 and STA102 based on the IEEE 802.11ac/ax standard, it is possible to efficiently transmit the wireless signals required to measure the movements of a user or controller within the communication area.

As explained above using Figures 2 and 3, actions performed by user 106 in the wireless communication area covered by AP 101 and STA 102 are measured using compressed CSI. By capturing CBFR frames 131 within the wireless communication area and extracting the compressed CSI, the game operations performed by user 106 can be analyzed.

Figure 4 is a flowchart for creating training data used by a wireless sensing device in machine learning according to embodiment 1 of the present disclosure.

First, the user 106 starts the in-game tutorial (step 140). The tutorial is a mode or the like that is conducted in the game to help the user 106 learn how to operate the game before the main game begins.

Next, the game console 104 outputs information about the action that the user 106 is to perform as label data to the wireless sensing device 103 (step 141). The game console 104 then instructs the user 106 to perform that action (step 142). The user 106 then performs the action instructed in step 142 (step 143).

Furthermore, the behavior of the user 106 is measured using the compressed CSI contained in the wireless signal between the AP 101 and the STA 102 (step 144). Furthermore, the wireless sensing device 103 captures a packet including a CBFR frame 131 in which the compressed CSI is stored (step 145).

The wireless sensing device 103 extracts the compressed CSI data (step 146). In addition, it determines whether label data has been input from the game console 104 (step 147). If label data has been input, training data is created using the extracted CSI information and label data (step 148). Finally, the user 106 finishes the in-game tutorial (step 149).

In this way, by having the user 106 perform predetermined actions as a tutorial, training data can be created that matches compressed CSI information corresponding to those actions with label data indicating the game operations.

Figure 5 is a flowchart showing how a game console controls a game based on a user's actions or controller movements according to embodiment 1 of the present disclosure.

First, the user 106 starts the main part of the game (step 150). Next, the user 106 performs an arbitrary action (step 151).

Next, the behavior of the user 106 is measured using the compressed CSI contained in the wireless signal between the AP 101 and the STA 102 (step 152). Furthermore, the wireless sensing device 103 captures a packet containing a CBFR frame 131 in which the compressed CSI is stored (step 153).

The wireless sensing device 103 extracts the compressed CSI data (step 154). Furthermore, the game operations performed by the user 106 are determined from the compressed CSI data using machine learning (step 155). Furthermore, the determined game operations are transmitted to the game console 104 as label data (step 156). The game console 104 controls the game based on the label data (step 157). As a result, the user 106 can further perform step 151 and progress through the game.

In this way, the game console 104 can control the game in response to any action performed by the user 106.

As explained above, in the game system 100 of this embodiment, gestures and other actions performed by the user 106, or button presses on the controller 107, are measured by wireless communication between the AP 101 and the STA 102 and analyzed by the wireless sensing device 103. Therefore, the user 106 can play the game anywhere within the range covered by the wireless communication between the AP 101 and the STA 102.

In addition, in the game system 100 according to this embodiment, there is no need to connect the controller 107 to the game console 104 by wire or wirelessly. Therefore, there is no need to install a battery in the controller 107.

In this disclosure, the processing performed by the wireless sensing device 103 may be executed by a program using a computer equipped with a CPU and memory and storing a program in the memory. Alternatively, the processing may be executed by a program using an integrated circuit such as an FPGA (Field Programmable Gate Array). The program may be provided by being recorded on a storage medium, or provided via a network. This point is common to all of the following embodiments.

[Explanation of the correspondence with terms used in claims]
In this embodiment, the wireless LAN AP 101 is referred to as a first wireless device. Similarly, the wireless LAN terminal 102 is referred to as a second wireless device. Furthermore, the wireless sensing device 103 is referred to as a wireless capture device.

Embodiment 2
6 shows an example of the configuration of a game system according to embodiment 2 of the present disclosure. A game system 200 has a configuration in common with the game system according to embodiment 1, but includes a plurality of STAs.

In the example of Figure 6, multiple STAs 202(1) and 202(2) are installed so that the behavior of multiple users 106(1), 106(2), and 106(3) can be measured via wireless communication. Note that while there are two STAs in the example of Figure 6, the more STAs there are, the better.

By deploying multiple STAs 202 in this way, the wireless coverage area can be expanded. This allows multiple users 106 to play the game simultaneously, or allows for more dynamic game play that incorporates users 106 walking and stopping.

Embodiment 3
7 shows an example configuration of a game system according to a third embodiment of the present disclosure. The game system 300 has a configuration in which an obstacle 308 is added to the example configuration of the second embodiment. The obstacle 308 is positioned so that the actions of multiple users 106(1) and 106(2) are not visible to each other.

By placing multiple wireless LAN terminals 202(1) and 202(2) on either side of an obstacle 308, it is possible to measure the movements of users 106(1) and 106(2) on either side of the obstacle 308. This allows users 106 to operate the game without disclosing each other's movements.

100: Game system; 101: Wireless LAN AP; 102: Wireless LAN terminal; 103: Wireless sensing device; 104: Game console; 105: Antenna; 106: User; 107: Controller; 110: Wireless interface unit; 111: Capture unit; 112: Filtering unit; 113: Extraction unit; 114: Preprocessing unit; 115: Allocation unit; 116: Machine learning unit; 117: Training data creation unit; 118: Determination unit; 119: Sensor result input unit; 120: Controller communication unit; 121: Game control unit; 122: Game console; 123: Label creation unit; 130: NDP frame; 131: CBFR frame; 200: Game system; 202: Wireless LAN terminal; 300: Game system; 308: Obstacle

Claims (7)

A game system that detects a user's actions or a controller's movements using wireless communication,
a first wireless device having two or more antennas;
a second wireless device that wirelessly communicates with the first wireless device;
a wireless capture device that captures the wireless communication signal and detects the user's actions or the movement of the controller within the area of the wireless communication;
a game machine that controls a game based on label data indicating game operations transmitted from the wireless capture device;
Equipped with
the first wireless device is configured to perform a process of transmitting a wireless signal including a known wireless frame from the antenna;
The second wireless device
a process of calculating information of a wireless propagation path between the first wireless device and the second wireless device from the known wireless frame;
a process of returning a wireless frame including information on the wireless propagation path to the first wireless device;
configured to run
The wireless capture device
capturing a wireless frame including information about the wireless propagation path;
A process of extracting information about the wireless propagation path;
an operation determination process for determining a game operation or a controller operation performed by a user within the wireless communication area based on the information on the wireless propagation path;
a process of transmitting the game operation or the controller operation performed by the user to the game machine as label data;
configured to run
The gaming system is configured such that the gaming machine executes a process for controlling a game based on the label data transmitted from the wireless capturing device. The game machine includes:
A process of instructing a user to perform an action;
a process of transmitting label data indicating the operation instructed by the user to the wireless capture device;
configured to run
2. The game system according to claim 1, wherein the wireless capturing device is configured to, when the label data is input from the game machine, execute a process of creating training data that corresponds information about the wireless propagation path with the label data. The game system described in claim 2, wherein the operation determination process of the wireless capture device includes a process of making a determination based on machine learning using the training data. The game system described in claim 1, comprising a plurality of second wireless devices. The game system described in claim 4, further comprising an obstacle within the wireless communication area covered by the first wireless device and the second wireless device. capturing a wireless frame including information about a wireless propagation path transmitted from a second wireless device to a first wireless device ;
A process of extracting information about the wireless propagation path;
determining, from the information on the wireless propagation path, a game operation or a controller operation performed by a user within an area of wireless communication between the first wireless device and the second wireless device ;
a process of transmitting the game operation or the controller operation performed by the user to the game machine as label data;
2. A wireless capture device configured to perform the steps of: A game control method for detecting a user's action or a controller's movement using wireless communication, comprising:
the first wireless device has two or more antennas and performs a process of transmitting a wireless signal including a known wireless frame from the antennas;
The second wireless device
a process of calculating information on a wireless propagation path between the first wireless device and the second wireless device from the known wireless frame transmitted from the first wireless device;
a process of returning a wireless frame including information on the wireless propagation path to the first wireless device;
Run
The wireless capture device
capturing a wireless frame including information about the wireless propagation path;
A process of extracting information about the wireless propagation path;
determining, from the information on the wireless propagation path, a game operation or a controller operation performed by a user within a wireless communication area between the first wireless device and the second wireless device;
a process of transmitting the game operation or the controller operation performed by the user to the game machine as label data;
Run
The game machine executes a process for controlling the game based on the label data transmitted from the wireless capture device.