JP7598959B2 - Virtual space display device, program - Google Patents

Description

Related to virtual reality, hyper-reality, and mixed reality.

In recent years, games that utilize augmented reality (AR), virtual reality (VR), and mixed reality (MR) have appeared (Patent Document 1). In these VR games, objects within the game are represented virtually, and the user progresses through the game by touching the virtual objects using a handheld pointing device or the like.

In addition, games that were previously played with real cards and pieces are now implemented as computer games, making game progression and card management easier for users than when dealing with real cards and pieces.

Patent Publication No. 2020-064652

By playing card games or board games as computer games, users can automate tedious game progress processing and check various information such as scores in an easy-to-understand format. On the other hand, in addition to the gameplay aspects of card games and board games, users also enjoy touching and handling the cards or game pieces, such as holding them in their hands or lining them up. These advantages are lost when playing them as computer games or VR games.

The present invention provides a system that maintains the advantages of turning games that involve cards and pieces, such as card games and board games, into computer games, while also providing the fun of dealing with real cards and pieces.

A virtual object is assigned to a real object whose identification ID, position, and orientation can be detected, and the virtual object assigned to the real object's identification ID is displayed in a virtual space at a position in the virtual space that corresponds to the position of the real object, with an orientation that corresponds to the orientation of the real object.

The allocation of virtual objects to real objects changes as the game progresses.

The present invention makes it possible to provide a system that maintains the advantages of turning card games, board games, and other games that involve playing cards and pieces into computer games, while also providing the fun of playing with real cards and pieces.

In the present invention, since the progress of the game can be managed by a computer, cheating can be prevented. Also, random elements such as shuffling of cards can be managed by a computer. For example, in the case of cards, it is possible to prevent the order of cards from being sufficiently random depending on the shuffling method, and certain cards from remaining together. Also, it is possible to intentionally control elements that are originally random according to the rules of the game. While having the above features, the present invention allows users to play the game using real cards and pieces. It is possible to obtain the convenience that can be obtained by making it a computer game, such as shuffling, card selection, automating game processing, and displaying various information such as the number of cards, while retaining the fun of users handling cards and pieces that is lost in computer games.

Actual card example Example of recognition of actual card Modification of the card face of the virtual card System configuration example An example of the board

Real cards and pieces are referred to as real objects. Cards and pieces that are virtually managed and represented on a computer, such as in games, are referred to as virtual objects.

Below, we will mainly explain the application of the present invention to card games, but the present invention can also be applied to things other than cards, such as figures and pieces used in board games.

In this embodiment, we will explain the application of augmented reality (AR) to a card game. In addition to augmented reality, it can also be applied to virtual reality (VR), mixed reality (MR), and projection mapping. It can also be applied to technology that overlays images onto real-world objects, or displays videos or images on the surface of real-world objects.

The present invention assigns virtual cards to multiple real cards with different identifiers. Also, the assignment of real cards and virtual cards is changed as the game progresses. This makes it possible to automate operations that are cumbersome for users, such as shuffling and searching for specific cards, in a card game system using augmented reality, for example. For example, by shuffling the assignment of virtual cards to a set of real cards in a deck, the same effect as actually shuffling the deck can be achieved. In another example, when a user wants to select a specific card from the deck and add it to their hand, the top card of the deck can be assigned to that card, eliminating the need for the user to search for the desired card among multiple cards as would be the case when using real cards.

The game system can carry out processing on the assigned virtual card depending on the position, orientation, etc. of the real card. In addition to using a camera to detect real objects such as cards and pieces, the game system can also incorporate a mechanism for detecting the position and orientation of the real objects. For example, barcodes or markers may be provided. In this case, the identification mechanism and the mechanism for detecting position and orientation may be separated. It is possible to combine an identification means that can identify real objects but has difficulty detecting their exact position and orientation with a mechanism that can detect the exact position and orientation of real objects but has difficulty identifying them.

A virtual card assigned to a real card is drawn in the virtual space at a position in the virtual space that corresponds to the position of the real card, captured by a camera or other device. In an example application to AR, the face of the virtual card assigned to the real card is drawn on the face of the real card in the image captured by the camera, allowing the user to see an image or video in which the real card has been replaced by the virtual card.

The allocation of real objects and virtual objects can be changed as the game progresses. Below we explain how to change the allocation of real cards in a card game.

Explain how to shuffle cards.
Changing the order of cards can be achieved by changing the allocation of actual cards, randomizing the order of some or all of the cards to be shuffled, or rearranging some or all of the cards into a predetermined order.

This section describes an example of shuffling a deck in a card game in which cards are drawn from an unknown deck. This section describes shuffling processes in two cases: when virtual cards are assigned to real cards in the deck in advance, and when virtual cards are assigned to cards drawn from the deck.

Here is an example of how cards are assigned in advance. The order of the cards can be changed by rearranging the assignments of the physical cards in the deck. In this case, it is necessary to manage for each physical card whether that card is included in the deck, i.e., the set of cards to be shuffled.

An example of assigning a virtual card to a card to be drawn will be described below. When a card is taken from a collection whose order is not disclosed, such as a deck, and the card information is made public, that is, when the top card of an unknown deck is drawn and added to one's hand, a virtual card may be assigned to the real card drawn. If the deck has been randomly shuffled, a real card may be randomly assigned from the collection of unassigned virtual cards. If the deck order is manipulated during the game and the next card to be drawn is already decided, then the virtual card may be assigned to the real card drawn.

Explain card selection.
A specific virtual object is selected from a collection of virtual objects, such as a deck of cards, and assigned to a real object. In a card game, for example, this is applied to an operation of selecting a specific card from the deck of cards and adding it to one's hand.

We will now explain the case where a user selects a card from the deck and adds it to their hand. When this situation occurs, the system presents the user with a list of options. One way of presenting the options is, for example, displaying the cards in the virtual space. The user selects the card by touching the card they want to select. At this time, a physical card may be used as a means for indicating the option. Since the system is equipped with a means for detecting the position of the physical card, it is possible to simplify processing by having the user select the option using a physical card like a pointing stick, rather than detecting the user's hand and having the user determine the selected card from its position, as this eliminates the need to detect the position of the user's hand. Also, using a means for detecting the position of the physical card often reduces false detections.

The selection can be made by displaying option cards on the display, or by using various input means such as buttons. Any selection means or method may be used depending on the game to which it is applied.

As an example of selecting from a private collection where the order of cards is not disclosed, a case where cards are selected from a deck or the like will be described. The top real card of the deck is assigned to the selected virtual card. This allows the user to pick up the selected card by taking the top card of the deck. At this time, the cards may be displayed so that the contents of the selected card can be seen. For example, the top card of the deck is the selected card and is face up. At this time, the contents of the card may not be seen by the opponent. For example, the contents of the card are not displayed on the opponent's display means. The user may take the top real card of the deck, use the real card like the wand to select a card, and the selected virtual card may be assigned to the real card.

Regarding the state of the deck after the selected virtual card is assigned to the top real card of the deck, if a virtual card has already been assigned to a real card of the deck, the virtual card assignment of the top real card of the deck can be reproduced by replacing it with the real card to which the selected virtual card was assigned. After this, cards other than the selected card may be shuffled. The order of the deck after the selected card is assigned to the top of the deck may be freely set.

If there are no physical cards in the deck assigned, the selected card may be assigned to a card that is easy for the user to pick up, such as the top card of the deck. Even if cards are stacked, the top card is visible and can be recognized by the system using a camera or other device, so the system does not need to know the order of all the physical cards in the deck. If there are multiple selected cards, they can be processed one by one.

Alternatively, an unassigned real card can be used like the pointer described above to select a virtual card from the options, and the selected virtual card can be assigned to the real card used to select the card.

After the selection, any unassigned physical cards picked up by the user may be detected and the selected virtual card may be assigned to that card.

We will now explain the case where a card is selected from a set whose order is public. For example, a discard pile. As in the case of the deck, if the top card of the discard pile is changed to the selected card, the user can easily pick up the selected card.
In cases where the order of cards is public, the order of cards may have meaning depending on the game to which the system is applied, so the system changes the allocation of virtual cards so that the order of cards after the selected card is removed is maintained or the order conforms to the rules of the game to which the system is applied. In this case, the system records the order in which the real cards are placed in the discard pile so that the order of the real cards in the discard pile can be recorded.

We will explain an example in which the selected virtual card is assigned to the topmost real card while maintaining the sorting order. First, the virtual card assigned to the topmost real card is temporarily stored, and the selected virtual card is assigned. Next, the temporarily stored virtual card is swapped with the virtual card assigned to the real card one card below. Furthermore, the virtual card assigned to the real card one card below is swapped with the temporarily stored virtual card. This swapping with the temporarily stored virtual card can be repeated until the selected virtual card becomes the real card that was assigned.

The selected virtual card may be assigned to an unassigned real card. In this case, the unassigned real card to which the virtual card is assigned may be selected from cards belonging to another collection, such as the deck. In addition, in this case, the real card to which the selected card was assigned may be changed to an unassigned state.

The exchange of cards will now be explained.
A case where a real card to which a virtual card has already been assigned is changed will be described. For example, a card in the hand is exchanged with a card on the table, or the position of the card is exchanged. In this case, the assignment of the original real card and the assignment of the destination real card can be exchanged. If the destination card is an unassigned virtual card, the virtual card assigned to the original real card is changed to an unassigned state, and the set to which it belongs is changed to the set to which the destination virtual card belonged. For example, when exchanging an assigned card in the hand with an unassigned card in the deck, the real card in the hand is left as it is, and the assignment of the virtual card of the real card is changed, and the originally assigned virtual card is changed to an unassigned card in the deck.

Now, the case of moving a card to another collection will be described.
When moving a card from the tabletop or hand to a public set such as the discard pile, there is no need to perform an allocation change process. The user can simply place a real card from the tabletop or hand to the discard pile, so the system may display a message encouraging the user to do so. When moving an assigned card to an unassigned set, the target real card is put into an unassigned state, and the assigned virtual card is moved to the destination set.

When it is necessary to move a card to a distant location, such as during a game against an opponent in a remote location in a network match, the real card to be moved is changed to an unallocated state, and the virtual card to be moved is assigned to the opponent's unallocated real card. The real card to be assigned may be an unallocated real card that is not currently being used, such as a card from the opponent's deck. For example, the top card of the deck may be used. The user receiving the card may not use a real card, and the received card may be displayed and handled in the virtual space without being assigned to a real card.

We will now explain what happens if a user loses their physical card during play.
A case where a user drops a real object that should be in hand and is no longer able to handle it will be described. When a virtual object is displayed in virtual reality, in many cases the user will wear a virtual reality display means such as VR goggles. In such a state, it is difficult for the user to find or pick up a lost object. Therefore, when a real object is lost, the user can smoothly continue the game by allocating the virtual object that was assigned to the lost real object to another real object. When a lost real object is found during the game, the real object may be made available again as an unallocated real object. In this case, the system may instruct the user to place the real object in a specific location, such as the top of the deck.

Explain how the system detects the loss of a real object.
If the system is unable to detect a real object for a predetermined period of time using the real object detection means, it determines that the real object has been lost, and assigns a virtual object to a new real object. Only specific real objects may be subject to loss detection, for example, only real cards on the table or in a hand may be subject to loss detection. Hidden real objects, such as discarded cards where cards are stacked and cannot be seen, may not be subject to loss detection.

When the system detects the loss of a real object, it may display this to the user and perform the allocation process of a new real object. At this time, if the system is able to recognize the real object whose loss was detected, it may cancel the allocation process.

The user may notify the system that an object has been lost through user action.

A lost virtual object may be treated as a virtual object, without being assigned to a real object. A virtual object may be handled by a user's gesture or user input to an input means.

A method for handling a case where the number of virtual objects exceeds the number of real objects will be described.
If the number of virtual objects to be assigned exceeds the number of real objects, a real object from a set other than the set of real objects to be assigned may be used. For example, when drawing a card from deck A, if there are no real cards from deck A, a virtual card from deck A may be assigned to a real card from another deck B instead and used. Alternatively, an unused virtual card from the discard pile may be assigned to an assigned real card, and the originally assigned virtual card from the discard pile may be treated as a virtual card. When the discard pile is used, the assignment process to the real card is performed again.

A virtual display using virtual reality or augmented reality may be used instead. When a user operates a virtual object that is being displayed virtually, a process may be performed to change the assignment and assign the virtual object to a real object.

This section explains matters related to the recognition of real objects such as cards and pieces.
Here, the explanation will be mainly given using cards as an example, but the same can be applied to other three-dimensional objects such as pieces.

The position of the real object is detected by attaching markers (101, 102, 103, 104, 100) to the real object and detecting the identification ID, position, and orientation of the marker. The system uses the detected ID, position, and orientation information to display a virtual object associated with the real object on the virtual reality display means. In the case of a card, an image of a virtual card is displayed on the face of the real card captured by the camera, so that the real card appears to the user as a virtual card.

We will explain the placement of markers on real cards. When a user holds multiple cards in their hand, they may overlap. Also, parts of the card may be hidden by the hand holding the card. By placing multiple markers (101, 102, 103, 104) on one real object, the system can recognize the ID, position, and orientation of the real object even if part of it is hidden.

One example of marker placement is to place the markers at least on the corners of the cards as shown in Figure 1. In the case of square cards, a typical hand holding pattern is as shown in Figure 2. In this case, at least one of the four corners of overlapping cards will be visible, so by placing the markers as in the cards in Figure 2, it is possible to recognize cards that are partially hidden. In this case, different IDs can be used for the markers placed on one real object. The system changes the position at which the virtual card is displayed based on the ID information. For example, if a marker is detected in the top left corner of a card, the position of that marker will be displayed so that it matches the position corresponding to the marker in the top left corner of the virtual card.

The detection of the position of a marker within a real object may use the recognition result of the real object itself. In the case of a card, for example, the edge of the card, i.e., a straight line, is detected, and the positional relationship between the marker nearby and the edge of the card is used to detect which of the four corners of the real card the marker is placed on. For example, if the edge of the card is detected to the left and above the marker, the marker becomes the top left marker of the card. In such a case, it is possible to reduce the number of types of IDs for markers placed on real objects. In other words, it is sufficient to prepare at least one type of ID for one real object, and the marker on the real object can be identified from the shape of the real object based on the recognized marker, so the number of types of IDs used by the system can be reduced, which results in smaller markers, simpler patterns of markers, and easier identification of IDs. In this case, the marker includes orientation information.

In addition to markers, barcodes (111, 112, 113, 114, 115, 116) can also be used. In this case, the markers can be used to recognize position information and detect the identification ID of the real object. In the case of cards, placing a barcode near the edge of the card increases the number of situations where the cards can be detected when overlapping. Also, like barcodes 113 and 114, multiple barcodes can be placed on one side, or one barcode can be placed on one side.

Large markers, like marker 100, may be placed to make them easier to recognize. In general, when the size of the marker or barcode is small, there is a risk of false detection. When the marker is large, it is often hidden and cannot be recognized, but in the case of card games, when the card is used as a tableau card, the entire surface of the card is often visible, so this is effective.

These descriptions regarding the markers and barcodes on the physical cards can also be applied to three-dimensional objects including curved surfaces. Furthermore, the markers and barcodes may be a single color or a combination of multiple colors. Furthermore, the markers, barcodes and card surface 131 may use different colors or patterns for each ID of the physical card. In particular, by changing the color of the card surface for each ID, the card surfaces will be separated by color when the physical cards are stacked, which is useful for detecting the card surface of each physical card.

In the case of cards, different marker and barcode layouts may be used on the front and back of the card. It is not necessary to have a marker or barcode on the back. Also, the IDs of the marker and barcode on the back may be different for the front and back, or the same ID may be used. It is not necessary to distinguish between the front and back.

This section explains the area in which virtual objects are drawn. Here, we mainly use examples of cards, but it can also be applied to three-dimensional objects and curved surfaces.

We will now explain the case where the physical card in Figure 1 is used. The physical card in Figure 1 has markers at the top right 101, bottom right 102, bottom left 103, and top left 104. The edges of the card may also be provided with contours 132 that make the edges of the card easier to detect. The markers and contours 132 can use colors and patterns that are easy for a system such as a camera to recognize.

The system uses a camera to recognize the position of the marker.

The system detects card surface 131. Card surface 131 may have a color, pattern, or other material features that make it easy to detect. Alternatively, contour 132 may be detected, and the area surrounded by contour 132 that includes the marker may be recognized as card surface 131. In a case such as that shown in Figure 2, contour 132 can be regarded as the boundary between the card surfaces of the cards. The marker portion and contour portion may be treated as part of the card surface.

When an image of a real card recorded in the system is placed in a position and orientation corresponding to the marker captured by camera 402, the detected portion of the image captured by camera 402 that matches the placed real card may be regarded as the card face. Here, the portion of the card face visible to camera 402 matches the image of the placed real card, and the hidden portion does not match, so the system can detect the area in which the card face of the virtual card should be drawn. A pattern matching process may be performed to detect the matching portion. The acceptable degree of match may be adjusted as appropriate.

The system deforms the shape of the image of the card surface of the virtual card corresponding to the marker according to the detected marker's posture. An example is shown in FIG. 3. The system deforms the image of the card surface 133 by changing the magnification ratio of the detected marker 106 relative to the size of the recorded marker 105, and by changing the same as the change in posture of the marker 106 (134).

The system uses the detected card surface 131 as a mask and ANDs it with the card surface of the virtual card that has been deformed according to the magnification ratio and posture of the detected marker, and cuts out only the part of the card surface of the virtual card that corresponds to the part that is visible to the camera.

The system draws an image of the deformed card face. In the case of AR or MR, the deformed image 134 after the cutout process is drawn in a location corresponding to the position of the detected marker, so that the actual card in the image captured by the camera is displayed as a virtual card. In cases where it is not necessary to display real space, such as in VR, it is not necessary to perform the process of detecting the portion of the card face 131 of the virtual card that is visible to the camera, or the process of cutting out the card face of the virtual card described above.

The order of processing is not limited to the above order. For example, the order of image deformation of the card face and detection of the card face 131 may be reversed. In this case, the detected card face is deformed so that it appears as if the card face 131 were captured in a specified position, such as from the front, and the image of the virtual card is trimmed with a mask according to its shape, thereby extracting the visible portion of the card face that the system should draw.

The drawing of a virtual object will now be described.
Here, we will mainly explain the drawing of virtual cards in augmented reality (AR).

A virtual object is displayed in a virtual space according to the position and orientation of the real object. By displaying a virtual object on a video or image in real space, the real object can be made to appear as a virtual object.

The displayed virtual objects do not have to be based on real objects. For example, in the case of a card game, instead of simply displaying cards, it is possible to display CG images of characters corresponding to the cards. The characters may have animations and can move in accordance with the progress of the game.

In addition, information about the displayed virtual object may be displayed. For example, the status of the card or information about other cards that are influencing or being influenced by it may be displayed. One example of such a representation is to display a frame for displaying information near the virtual object, or to connect the displayed virtual objects with arrows or lines. Animation may be added to the image displayed on the card surface of the virtual object, or information may be displayed.

The displayed virtual object may be made transparent. For example, when a user is about to touch a real object, the displayed virtual object may be made transparent, allowing the user to grasp the shape of the real object, making it easier for the user to handle the real card. This is particularly useful when the shape of the real object and the shape of the displayed virtual object are different.

In addition to making the virtual object transparent, the shape of the virtual object may be changed to a shape similar to that of the real object. In this case, the displayed virtual object may be displayed transparently and overlaid. In addition to making the shape of the virtual object the same as the real object, only a part of it, such as a handle on the real object, may be changed to a shape similar to that of the real object.

The displayed virtual object may reflect the state of the real object.
For example, in a case where a real card has a shadow, the shadow can be displayed on the displayed virtual card to enhance the affinity of the representation with the real world. One example of a method for acquiring the state of a real object is to obtain the difference between the information of the real object recorded in the system and the state of the real object recognized by the system. For example, if the recognized real object is recognized as being darker than the recorded real object, the color of the displayed virtual object is darkened.

Next, changing the orientation of a virtual object will be described.
When displaying a virtual object, the virtual object may be displayed regardless of the orientation or position of the real object. In the case of a card, the orientation or front or back of the card may be changed. For example, when applied to a card game, when the user needs to change the orientation or front or back of a card according to the rules of the game, the orientation or front or back of the virtual card displayed by the system on behalf of the user may be changed. Also, in a card game, when the front of a card that is not normally visible to the opponent, such as a hand card, is visible to the opponent, the opponent's display means may process and display the card so that the information on the front side is not visible. One example is to display a specific image on the card face, or to display the back side.

In one example, when turning a face-down card face up, instead of flipping over the actual card, the user makes a gesture such as touching the actual card to reveal the face-up of the corresponding virtual card. Or, when the rules of the applicable game require a card to be turned face-up, the system may automatically reveal the face-up of the card.

The size of the displayed virtual object may be changed to a size independent of the real object.
For example, the size of the displayed virtual card is made larger than the size of the actual card.

When changing the display of a virtual object, animation may be used. By changing the display of a virtual object with animation, the user can intuitively grasp the change in the virtual object.

In the present invention, since the progress of the game can be managed by a computer, cheating can be prevented. Also, random elements such as card shuffling can be managed by a computer. For example, in the case of cards, it is possible to prevent the order of the cards from being sufficiently random depending on the shuffling method, resulting in certain cards remaining together. It is also possible to intentionally control elements that are essentially random according to the rules of the game. While providing the above features, the present invention allows users to play the game using real cards and pieces. It is possible to obtain the convenience that comes with making it a computer game, such as shuffling, card selection, automating game processing, and displaying various information such as the number of cards, while retaining the fun that users have of handling cards and pieces, which is lost in computer games.

It will also be possible to play against other players via the internet.

An example of the configuration is shown below. Control/program execution is performed by a combination of general computers, microcomputers, and circuits.
FIG. 4 shows an example of the system configuration when applied to a card game. The system includes real cards, a computer 401, a camera 402 as real object recognition means, and a board 403. The system also includes display means for displaying virtual cards to the user. An example of the display means is a head mounted display (HMD) 405, and the user can view a virtual space via the HMD 405, or an image observed by a camera provided in the HMD 405, in which the virtual cards are displayed. Here, the camera provided in the HDM 405 may be the camera 402 for recognizing real cards. A plurality of cameras 402 may be used, and the plurality of cameras may be used to complement each other's blind spots and improve the recognition accuracy of position and orientation.
The computer 401 is equipped with a game program 404, and the progress of the game is managed by the game program 404. The computer 401 also recognizes the type, position, and posture of real cards based on images from a camera 402, and displays virtual cards assigned by the game program 404 on an HMD 405, which is a display means.

On the board 403, cards to be placed, a deck, discards, and the like used in the game to which the system is applied are placed. FIG. 5 shows an example of the board. The board 403 in this example has a deck area 501, a discard area 502, a deck area 503, and a second deck area 504. Each area may have a marker to identify the location of the area. The marker of the area is hidden when a card is placed, but by recording the location of the area when no card is placed, the system can determine whether the card at that location is a deck, a discard, or a deck. If the initial state of the board is determined by the rules of the game, the board 403 and the markers on the board 403 are not provided, and the initial arrangement of the actual cards may be recorded to make the above determination. The HDM 405 may display the location of each area (501, 502, 503, 504) so that the user can identify the location of each area.

The real object or the board may be provided with a means for detecting user input, such as a contact sensor. For example, a touch sensor may be provided in the place 503 for cards on the board 403, and when a user inputs to the touch sensor, a predetermined action in the game to which the action is applied is executed according to the virtual card assigned to the real card placed on the place. One example is executing an attack in the game or a process specific to that real card, or displaying candidate actions so that the user can select and execute them.

When there is an opponent, the system is provided with a board 403 and HMD 405 for the opponent. Depending on the game to which the system is applied, a single board 403 may be shared by all users.

The opponent may be a user at a remote location connected via a network or the Internet. In this case, the user and the opponent each use the system shown in FIG. 4. The computer 401 is connected via a network, and a communication battle is played. In this case, the opponent's board 403 and virtual objects may not be provided, and the objects handled by the opponent, such as cards, may only be displayed virtually. Also, accurate position and posture information may not be obtained for the opponent's real objects. For example, the positions of the real cards in the opponent's hand do not need to be displayed accurately, as they move according to the opponent's user's movements, and they can be displayed in a predetermined location in the virtual space, for example. When the opponent places a real card from his or her hand onto the tableau, the movement of the real card does not need to be reproduced, and the virtual card may be moved using a predetermined animation or the like. The movement of the opponent's virtual card may be the same as when the opponent is a computer player in a general computer game.

User input such as the opponent's movements and voice may be converted into a specified action, a specified line, or a specified voice of a character in the virtual space. The conversion may be performed in the user's system, the opponent's system, or a system at the relay point of the communication. For example, when the opponent plays a card on the table, the character representing the opponent performs a specified action of playing a card on the table. This may also be accompanied by a line such as "playing a card on the table." Since the user's movements and voice are replaced with the character's specified movements and lines and transmitted to the opponent, this has the effect of increasing the user's anonymity from the communication opponent. Also, since the opponent is replaced with a game character, the user can enjoy a human-to-human battle while maintaining the worldview of the game.

Each component may be replaced with another component that contains similar effects or characteristics.

Instead of using the camera 401, other means for recognizing the ID, position, and orientation of the real object may be used. The real object itself may be equipped with a means for recognizing its own position. One example is a GPS. The real object may be equipped with a means for recognizing markers or barcodes on the board surface 403.

The present system may be implemented as a smartphone application. In that case, the camera 402 may be a camera provided on the smartphone. The smartphone may also function as an HMD 405. Virtual reality or augmented reality images may be rendered on the smartphone screen.

The function of one component in the present embodiment may be separated into multiple components, or the functions of multiple components may be integrated into one component. At least a part of the configuration of the present embodiment may be replaced with a known configuration having a similar function. At least a part of the configuration of the present embodiment may be omitted. At least a part of the configuration of the present embodiment may be added to or substituted for the configuration of another embodiment of the present embodiment. All aspects included in the technical idea identified only by the wording described in the claims are embodiments of the present invention.

The present invention is not limited to the above-described embodiment, and various modifications can be made.
In addition to the above-mentioned device, the present invention can also be realized in various forms, such as a system that includes the device as a component, a program for causing a computer to function as the device, a medium on which this program is recorded, a control method, a method, etc.

Alternatively, the program may be installed on a server and downloaded from the server by the device that executes it, or some or all of the processing may be performed on the server side.

The technology of the present application is applicable to various board games and card games. Furthermore, even without managing the allocation of virtual objects to real objects, the user can enjoy multiple types of games using only the system of the present application by simply changing the type of virtual object to be assigned depending on the game the user is playing. For example, in a card game, the user can enjoy a game using playing cards and Hanafuda cards by switching between playing cards and Hanafuda cards to be assigned to virtual cards. Furthermore, in a card game in which the user builds a deck (mountain of cards), the user can virtually manage and build the deck and select which deck to use at the start of the game to play.

10 User's hand, 100 Marker, 101 Marker top right, 102 Marker bottom right, 103 Marker bottom left, 104 Marker top left, 105 Recorded marker, 106 Detected marker, 111 Barcode top right, 112 Barcode bottom right, 113 Barcode bottom left, 114 Barcode top left, 115 Barcode bottom, 116 Barcode top, 131 Card surface, 132 Contour, 133 Card surface of virtual card, 134 Card surface of virtual card after transformation, 401 Computer, 402 Camera, 403 Board, 404 Game program, 405 Head mounted display, 501 Deck placement area, 502 Discard placement area, 503 Playing card placement area, 504 Deck 2 placement area

Claims (2)

A virtual space representation device comprising: an object recognition means for recognizing an identification ID, a position and an orientation of a real object , which is an object in the real world; and a virtual space display means, the virtual space representation device further comprising a camera capable of photographing the real object;
When an image of a real object recorded in the system is arranged in an image acquired from the camera based on information on the position and orientation of the real object recognized by the object recognition means,
A portion where the image of the actual card recorded in the system and placed in the image acquired from the camera matches the image of the camera is defined as a drawing area;
Displaying a virtual object associated with the identification ID in the drawing area
Virtual space representation device.
A difference is calculated between the state of the real object recognized by the object recognition means and the information of the real object recorded in the system;
Displaying the shadow of a real object on a virtual object,
Or/Further, the color of the virtual object is changed based on the color of the recognized real object. The virtual space representation device of claim 1.

Images