JP7651617B2 - Image display device, image display system, and image display method - Google Patents

Description

The present invention relates to an image display device, an image display system, and an image display method that display an image of a different space from the real space in which the user is actually located.

Technologies such as Virtual Reality (VR), which creates and displays image content of a virtual world using a computer or other device, allowing users to experience the sensation of being in that virtual world, and Augmented Reality (AR), which displays image content created using a computer or other device superimposed on real scenery, are well known and have been put to practical use using devices such as head mounted displays (HMDs) and smartphones.

As an example of a game that applies the above technology, Patent Document 1 discloses an entertainment system in which a player uses a mobile terminal to actually move around the real world and obtain actual information from real stores and facilities. In this game, "a map of the area around the user is displayed on the user's mobile terminal, and the user is allowed to select an event spot. Upon selecting an event spot, a quiz or mini-game is presented to the user. The user travels to the actual event spot using the map, gets hints to play the quiz or mini-game, and earns points when they complete the game."

JP 2002-49681 A

With VR and AR technologies, a user can experience being in another space by viewing images of a space located differently from the real space (hereafter referred to as "another space") while being in the space where the user is actually located (hereafter referred to as "real space").

When viewing an image of another space, for example, a landscape of a memorable place the user has visited in the past or a landscape of an interesting place that the user has not yet visited, simply playing back an image of that place does not allow the user to experience that place with a sense of realism. In order to enhance the sense of realism, the real space can be fused with the other space, for example, by viewing images of another place while actually moving around the current location, the user can experience the other place as if they were actually moving around the other place. Alternatively, when a user moves (walks or runs) along a set course in the real space, it can become boring to simply see the same old scenery in the real space. In such cases, if the user can view images of another space while moving, it can add a newness to the walk or run and eliminate the user's sense of boredom.

In Patent Document 1, a user plays a game in a virtual space using a mobile terminal, but the user moves through the real space while relying on a map displayed on the mobile terminal to obtain hints for a quiz, and the game can be considered to be a combination of a virtual space and a real space. However, the configuration described in Patent Document 1 does not combine a real space with another space, that is, it does not combine a space where the user is actually located with a space in a different location. Therefore, the user cannot experience an image of another space with a sense of realism, and the user cannot eliminate boredom when moving in the real space.

The object of the present invention is to provide an image display device, an image display system, and an image display method that enable a user to view an image of another space with a more realistic feeling, or to eliminate the sense of boredom felt when moving through real space.

The present invention relates to an image display device that is worn by a user and displays an image of another space that is located at a different position from the real space in which the user is actually located. The image display device comprises a position detection unit that detects the position of the user, an orientation/posture detection unit that detects the user's front direction and orientation (azimuth and elevation angle of the user's line of sight), a communications interface that acquires image data of the other space and course information including the shooting position from the outside, a memory that stores the image data and course information acquired by the communications interface, a display unit that displays the image of the other space stored in the memory, an audio output unit that outputs audio, and a control unit that controls the display of the image of the other space on the display unit. The control unit is characterized in that, based on the course information of the other space, it associates a course consisting of multiple points in the other space with a course consisting of multiple points in the real space, and when the user moves along the course in the real space, it displays images of the associated points in the other space according to the user's position.

According to the present invention, when a user moves along a course in real space, the scenery at a corresponding position in another space can be viewed as a still image or a video. This has the effect of providing a realistic experience as if the user were actually moving through another space, and eliminating the sense of boredom felt when moving through real space.

FIG. 1 is a diagram showing a configuration of an image display system 1 according to a first embodiment. FIG. 2 is a diagram showing the appearance of a user wearing the image display device 2 (head mounted display 19). FIG. 2 is a diagram showing an example of a display screen of the image display device 2. FIG. 13 is a diagram showing an example of acquiring images of each point on a course in a different space. FIG. 13 is a diagram showing an example of displaying a course setting in a real space and an image captured in another space. FIG. 13 is a diagram showing a process for registering course information for another space. FIG. 5B is a diagram showing the contents of course information registered by the process of FIG. 5A. FIG. 13 illustrates the process of displaying registered images by traversing a course in real space. FIG. 11 is a diagram showing an example of a course in a different space with an altitude difference according to the second embodiment. FIG. 13 is a diagram showing an example of a flat course with no altitude difference in real space. FIG. 13 is a diagram showing a course in another space according to the third embodiment. FIG. 1 is a diagram showing a course around a track as a course in real space. FIG. 1 is a diagram showing a user wearing an image display device 2 (smartphone 40). FIG. 4 is a diagram showing an example of a display screen of a smartphone 40. FIG. 4 is a diagram showing an example of a display screen of a smartphone 40. FIG. 4 is a diagram showing an example of a display screen of a smartphone 40.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same parts are generally designated by the same reference numerals, and the repeated description will be omitted.
Example 1

In the first embodiment, an image display system 1 is described that displays an image captured in another space when the user moves to a corresponding position in the real space.

[Image display system]
1 is a diagram showing a configuration of an image display system 1 according to a first embodiment. In the image display system 1, an image display device 2, an imaging device 3, and a Web server 4 are connected via a communication network 5. The communication network 5 includes a wireless communication network and the Internet. Here, a case where a head mounted display (HMD) is used as the image display device 2 will be described. The internal configuration of each device will be described below.

The image display device 2 has an image sensor 6a, a position detection unit 7a, a memory 8a, a communication interface 9a, a control unit 10a, an orientation/attitude detection unit 11a, a display unit 12a, and an audio output unit 13. The image sensor 6a includes an optical lens (not shown) and is a unit that captures still images or moving images, and the captured image data is stored in the memory 8a. The position detection unit 7a measures latitude, longitude, and altitude as position information using, for example, a global positioning system (GPS). The orientation/attitude detection unit 11a includes a geomagnetic sensor and a gyro sensor (not shown) and measures the azimuth angle (angle measured clockwise from the north direction) and the elevation angle (angle measured upward from the horizontal plane). The display unit 12a captures or generates and displays images such as images captured in another space, composite images with them, and courses. The communication interface 9a is connected to the communication network 5 and transmits and receives data between the web server 4 and the image capture device 3. The audio output unit 13 includes a speaker (not shown) and generates an audio signal by synthesis or the like to output audio. The control unit 10a is made up of a CPU and controls all the functional blocks of the image display device 2. The memory 8a stores a display control application, a web search application, a map application, and the like (not shown).

Here, the image display device 2 is equipped with an image sensor 6a and is configured to also serve as the image capture function of the image capture device 3 described next, but if it is used only for the image display function, the image sensor 6a may be omitted.

The imaging device 3 has an imaging element 6b, a position detection unit 7b, a memory 8b, a communication interface 9b, a control unit 10b, an orientation/attitude detection unit 11b, and a display unit 12b. Functional blocks with the same names as those in the image display device 2 described above have similar configurations and functions, so descriptions will be omitted.

The imaging device 3 is for photographing another space, and captures still images and videos using the imaging element 6, and stores the image data in the memory 8b. The position detection unit 7b and the orientation/posture detection unit 11b obtain information at the time of photographing (position, orientation, orientation, etc.), and store the information in the memory 8b as additional information (metadata) of the image data. In this case, the image data and the additional information are paired and stored as a single image file, making it easy to associate them. The stored image file is transmitted to the Web server 4 or the image display device 2 via the communication interface 9b and the communication network 5. Alternatively, it can be passed to the Web server 4 or the image display device 2 via a removable recording medium such as an SD card (not shown). The display unit 12b plays and displays the image at the time of photographing and the data stored in the memory 8b.

The Web server 4 has a communication interface 9c, a control unit 10c, and a memory 8c, and image files and Web page documents are stored in the memory 8c. Image data captured by the imaging device 3 and an image file containing its associated information are received via the communication interface 9c and imported into the memory 8c. The control unit 10c generates course information based on the acquired data and publishes it on the Internet as a Web page document. Methods for disclosing the course information include a method of directly indicating the latitude, longitude, and altitude as point information on the course, a method of specifying the shooting location as a point on the course from the associated information of the image file, and a combination of these. In this embodiment, the above course is called a course in another space. In this way, by publishing the course in another space as a Web page document, the image data in the other space can be used not only by the photographer but also by other users who are interested.

The operation of the image display system 1 of this embodiment is as follows. First, the image capture device 3 captures images at each point on the course in the other space, and the image data and the additional information of the capture points are transmitted to the Web server 4 and registered. This registration process may be performed by the user who owns the image display device 2, or by another user. When the user himself/herself registers, the image capture element 6a in the image display device 2 may be used, in which case the image capture device 3 is not necessary. Next, when the user wears the image display device 2 and moves along a course in the real space, the image of the other space registered in the Web server 4 is displayed on the image display device 2 according to the movement position. At that time, when the user specifies the desired position (start point and end point) to move in the real space on the map, the course in the real space is determined in which the distance and direction are approximated by a predetermined geometric relationship to the course in the other space, and the registered image of the other space is displayed at the movement point in the real space corresponding to the capture point in the other space.

[Image display device]
2A and 2B are diagrams showing an example of the appearance and display screen of the image display device 2. The image display device 2 is a see-through HMD 19. Fig. 2A shows a state in which a user 20 wears the see-through HMD 19 on his/her head. The display unit 12a of the see-through HMD 19 is configured such that a semi-transmissive prism 22 is disposed in front of one eye (right eye) 21a, and a reproduced image of a display element 23 on the side is irradiated onto the prism 22. In addition, an image sensor 6a is disposed in front of the see-through HMD 19, and a control unit (drive unit) 10a is disposed at the top.

Figure 2B shows an example of the display screen (display unit 12a) when the user 20 looks at the prism 22. A reconstructed image 24 (shown by a solid line) of another space by the display element 23 can be seen superimposed on a landscape image 25 (shown by a dashed line) of the front in the real space through the prism 22. When the reconstructed image 24 is displayed, its display position (display direction) is, for example, the center of the display screen 12a. Furthermore, by comparing the orientation information from the orientation and posture detection units 11a and 11b of the shooting direction of the image display device 2 and the imaging device 3, it is also possible to display the corresponding reconstructed image 24 when the direction of the user's line of sight in the real space matches the orientation of the shooting direction of the imaging device 3 in the other space.

In this example, the see-through HMD 19 is a monocular type in which the prism 22 and display element 23 are placed on one eye (right eye) 21a, but it can also be a binocular type in which they are placed on the other eye (left eye) 21b. The transmittance of the prism 22 is set appropriately depending on whether it is a monocular or binocular type so that both the landscape image 25 and the reproduced image 24 can be viewed in a balanced manner.

[Course in a different space]
3 is a diagram showing an example of acquiring images of each point on a course in another space. While moving along a course 26a (indicated by a dashed arrow) in another space, a photographer (the user of the image display device 2 or another user) photographs surrounding scenery to be recommended at each point on the course (scenery that will become a memory for the user) with the imaging device 3. Note that a route that can be traveled without overlapping in order to photograph each recommended object is called a recommended course.

Specifically, at the start point P111 of the recommended course 26a, the landmark gate 28 is photographed, and at point P112, a structure 29 is photographed. At the branching point P113, a landscape including the branching road is photographed. At point P114, a tower 30 is photographed, and at the end point P115 of the course, a torii gate 31 is photographed. If there are other recommended objects outside the recommended course 26a (points P111 to P115), for example, a stone monument 32 is photographed at point P116. The image data photographed at these points and their associated information (such as location information) are saved as image files in memory 8b and transmitted to the Web server 4. The data at each point is stored with a distinction made as to whether or not it is on the recommended course 26a.

[Real space course]
FIG. 4 is a diagram showing an example of setting a course in the real space and displaying an image taken in another space. A course 27a (indicated by a solid arrow) is set in the real space, and points P211 to P215 on the course are shown. Outside the course are shown by dashed lines 39a and 39b. The procedure for setting the course will be described. A user who wishes to move in the real space uses a map application or the like to display a map of the real space on the display unit 12a of the image display device 2, and specifies the start point P211 and end point P215 of the course. As a result, the start point P211 is associated with the start point P111 of the other space shown in FIG. 3, and the end point P215 is associated with the point P115 of the other space. Next, a course that can be moved in the real space is determined so that it has a predetermined geometric relationship (direction deviation and scale) that is approximately the same as the course in the other space. Naturally, since the conditions of roads and buildings are different between the real space and the other space, the shape of the course that the user can move in the real space does not strictly match the shape of the course in the other space. Therefore, routes and intermediate points that have similar geometric relationships are set. Details of the course setting will be explained in the PAD section below. As a result, intermediate points P212 to P214 in the real space are determined so as to correspond to intermediate points P112 to P114 of the course in another space shown in Figure 3, and a course 27a in the real space connecting these points is set.

As the user moves along course 27a in the real space and approaches each of the set points P211 to P215, photographed images 24a to 24e of the corresponding recommended objects (structures and scenery at branching points) taken in another space are displayed on the display unit 12a of the image display device 2. The direction of the user's line of sight is detected by the orientation/posture detection unit 11a, and the registered recommended objects are displayed when they are in the direction of the user's line of sight, thereby enhancing the sense of realism. Of these, point P213 corresponds to point P113, which is a branching point in another space. When approaching this branching point P213, a branching guide map is displayed in which not only the corresponding photographed image 24c but also an arrow 24c' indicating the branching direction registered in advance is image-combined.

While intermediate points P112 and P114 on the different space course 26a are not branching points, the corresponding points P212 and P214 on the real space course 27a require a change of course (right turn) at the immediately preceding intersection. Therefore, the position detection unit 7a of the image display device 2 detects that the user has reached that intersection, and the audio output unit 13 instructs the user by voice to turn right. This allows the user to travel without straying from the course 27a set in the real space. Note that if the user travels along the course of dashed lines 39a and 39b that deviates from the recommended course, an image of a stone monument 24f is displayed at the position of point P216.

As a result, when the user walks along course 27a in the real space, the user can see scenery images of a different space from the real space according to the moving position, and can experience the different space with a sense of realism. Furthermore, at branching points, the user can see a branching guide map superimposed on the scenery image, which has the effect of allowing the user to preview the route at the branching point.

[Processing PAD and Course Information]
5A is a diagram showing the process of registering course information for another space, and shows the process structure as a hierarchical problem analysis diagram (PAD). The control unit 10b of the imaging device 3 reads out the captured image data and its associated information from the memory 8b, and transmits them to the Web server 4 via the communication interface 9b. Since multiple courses are registered in the Web server 4, each is assigned a course number.

In step S11, image data of another space captured by the imaging device 3 and its associated information (such as location information) are registered as an image file. At this time, image files that are not on the recommended course may be included (for example, data of the stone monument 32 in FIG. 3).

In step S12, the following process is repeated for each point (P111 to P115) on the recommended course. In step S12a, an image file (images 28 to 31 at P111 to P115 in FIG. 3) to be registered for the specified point is selected. In step S12b, point information is set from the additional information of the selected image file. At this time, if there is information on the shooting location, this is used as the point information, and if there is no information on the shooting location, the point information is specified from a map application or the like and registered. In addition, related figures and symbols to be displayed superimposed on images such as branch guidance arrows are arbitrarily registered as one piece of additional information. In the example of FIGS. 3 and 4, arrow symbol 24c' that provides route guidance at branch point P213 (P113) is registered as additional information.

Figure 5B is a diagram showing the contents of the course information registered by the process of Figure 5A. Course information 44 includes a course number, course point information, and image file information. The course number is a number for identifying the course. Course point information is position information (latitude, longitude, altitude) of points from the start point to the end point of the course. Altitude can be omitted if it is not covered by the function of the position detection unit. Image file information is a list (or link) of image files 45 of images taken at points on or around the course. In addition, the image files 45 referenced from the image file information above include image data 16 and its associated information 17 (commonly known as metadata). The associated information 17 includes the shooting location and related figures and symbols.

Figure 6 shows the process of moving along a course in real space and displaying registered images, with the processing structure shown as a hierarchical PAD. The control unit 10a of the image display device 2 retrieves image data and its associated information from the Web server 4 via the communication interface 9a, sets a course in real space, and displays the corresponding image according to the moving position.

Step S21 is for specifying the course in another space in which the image to be displayed is stored. When the user inputs a course number (e.g., No. 1) in step S21a, the corresponding course information and image file are retrieved from the Web server 4 in step S21b.

In step S22, the user specifies a start point and a finish point in the real space. Methods for specifying the positions include a method of specifying the position from a map application, and a method of specifying the user's current position from the position detection unit 7a of the image display device 2.

Step S23 repeats the following process for each section defined between adjacent points on the course. That is, the end point (target point) of the section is set at the start point of the section, and when the user reaches the end point of the section, the registered image is displayed and the end point of the next section is set.

In step S23a, in order to set a course in the real space that is similar to the course in another space, the section end point is determined based on the geometric relationship between the two courses. Therefore, the linear direction and distance when the section start point Ps (the starting point in the first time) is expected to the course end point Pg (the finish point) in the other space and the real space are compared, and the direction difference and scale of the two are obtained. For example, the linear direction from the section start point Ps to the finish point Pg in the other space is eastward, and the distance is 5 km. On the other hand, the linear direction from the section start point Ps' to the finish point Pg' in the real space is northward, and the distance is 2 km. In this case, the direction difference of the real space with respect to the other space is rotated 90 degrees counterclockwise, and the scale is 0.4 times. Using these geometric relationships (direction difference and scale), the section end point Pe' in the real space corresponding to the section end point Pe in the other space is obtained. However, the position of the section end point Pe' determined in this way is not necessarily a place that can be reached in real space, and may overlap with an area where buildings exist, for example. Therefore, a map application in the real space is referenced to determine a position that can be reached close to position Pe' (for example, on a road), and this is set as the target section end point Pe". The map application is then referenced to determine the route from section start point Ps' to section end point Pe", and the course in real space is set.

In step S23b, it is determined whether the user has approached the section end point Pe". The user's moving position in real space is detected by the position detection unit 7a. Then, when the moving position reaches the vicinity of the section end point Pe", for example, within a range of 10% of the section distance from the section end point Pe", it is determined that the user has approached the section end point Pe".

If the result of the determination in step S23b is that the section end point is being approached, in step S23c, the registered image at the corresponding point in the other space is displayed on the display unit 12a. After this, in step S23d, it is repeatedly determined whether the section end point Pe" on the course in the real space has been reached. If it is determined that the section has ended, that position is set as the start point of the next section in the real space and the process returns to step S23.

Note that as a condition for determining the end of a section in step S23d, if there is a branching point on the map application and the branching point is not a dead end, then the branching point is considered to be the end of the section. Point P213 in FIG. 4 corresponds to this case. Other conditions for determining the end of a section include when the section distance in real space reaches a specified distance even if it is not a branching point, or when it is a point specified by the user. Points P212 and P214 in FIG. 4 correspond to this case.

Note that the real-space course 27a can be automatically created by performing a simulated drive on a map application. Furthermore, in order to create a real-space course 27a that matches the user's preferences from a course 26a in another space, the real-space course that was automatically created by the process in FIG. 6 may be manually modified before or during movement. For example, when point P213 is reached, a round-trip course 39a to point P216 outside the automatically created course 27a may be added.

Furthermore, when the difference in direction between the direction of travel on the course 26a in another space and the direction of travel on the course 27a in the real space exceeds a predetermined value, the audio output unit 13 may notify the user by voice, or a message may be displayed on the display unit 12a. This allows the user to move along the course while being aware that the difference between the course in the other space and the course in the real space has become large due to the geographical constraints of the real space.

Furthermore, when it is detected that the user's position has deviated from the course 27a in the real space by a predetermined value or more, the audio output unit 13 may emit an alarm sound or the display unit 12a may display a warning message. This allows the user to move along the course 27a in the real space without making any mistakes.

In the above explanation, the image captured by the imaging device 3 is registered in the Web server 4, and the image display device 2 retrieves and displays the registered image from the Web server 4. In contrast, when the user takes and displays the image himself, the image can be taken and saved on the image display device 2. Therefore, the imaging device 3 and Web server 4 can be omitted.

According to this embodiment, when a user moves along a course in real space, the scenery of the corresponding position in another space can be viewed as a still image or a moving image, so that the user can experience a sense of realism as if he or she were actually moving through the other space. In addition, the course in the real space along which the user moves is not only similar to the course in the other space, but is also flexibly set according to the geographical conditions such as roads in the real space, making it highly practical.
Example 2

In Example 2, we explain how to set a course in real space when the course in another space has an altitude difference and the course in real space is flat.

Figure 7 is a diagram showing an example of a course in another space with altitude differences. The other space is undulating, and the altitude differences are represented by contour lines 34 (shown by dotted lines), with altitude [m] written on each contour line. Another space course 26b (shown by solid arrows) is a course with altitude differences from start point P121 to end point P126. Point P122 has the highest altitude, and point P123 has the lowest altitude. Points P124 and P125 are highlands. Therefore, the walking speed decreases in the uphill sections (points P121 to P122, points P123 to P124). Conversely, the walking speed increases in the downhill sections (points P122 to P123, points P125 to P126). There is a hut 33 at point P124, and an image of that hut is registered. The same applies to the objects to be photographed at the other points, but a description of this will be omitted here.

Image data captured at each point is registered in the Web server 4 in the same manner as in Example 1, but as shown in the additional information 17 in FIG. 5B, altitude is added to the information on the shooting position obtained by the position detection unit 7b at the time of shooting. Altitude information can also be measured by an altitude sensor. Alternatively, it is possible to read the altitude value of the position from map information.

Figure 8 shows an example of a flat course 27b with no altitude difference in the real space, which corresponds to the course 26b in the other space in Figure 7. Since there is no altitude difference in the real space, the contour lines are omitted. The course 27b in the real space is from the start point P221 to the end point P226, and the direction of each section between adjacent points corresponds to the direction of each section of the course 26b in the other space with a predetermined direction difference. However, the distance of each section in the real space is set taking into consideration that the walking speed changes due to the altitude difference in the other space. In other words, the position information and altitude information of each point of the course 26b in the other space are obtained from the Web server 4, and the walking time for each section is assumed. Then, intermediate points are set on the course 27b so that the walking time for each section of the course 27b in the real space corresponds to the walking time for each section of the course 26b in the other space at a predetermined scale.

Specifically, the distance is increased for sections corresponding to uphill sections (points P221-P222, points P223-P224), and decreased for sections corresponding to downhill sections (points P222-P223, points P225-P226). When the user approaches a point in each section (for example, point P224), the position detection unit 7a detects it, and, as in Example 1, a registered image of another space (for example, 24g) is displayed on the display unit 12a.

In this way, in Example 2, the distances between points in the real space are corrected and set to take into account the altitude difference in the other space and to cancel out the difference in travel time for the user that accompanies the altitude difference, allowing the user to travel in the real space with a feeling that is closer to the course in the other space in terms of time required and fatigue level.

Here, we have assumed that the course in another space has an altitude difference and the course in real space is flat. However, conversely, if the course in another space is flat and the course in real space has an altitude difference, or if the altitude difference between the two courses is different, the course in real space can also be set taking walking time into consideration.
Example 3

In Example 3, an image display device 2 that allows a user to run around a circular course in real space while viewing an image of the course in another space will be described. Here, a case where a smartphone 40 is used as the image display device 2 will be described.

Figure 9 shows course 26c in another space. Here, a course used in a road race is assumed, starting at point P131 and finishing at point P135. At intermediate point P132, there is a building 35, and a photographed image of this building is registered on the web server 4.

Figure 10 is a diagram showing the course 27c in the real space. Here, the course is a circuit around a track for athletics. The shape of the course 27c in the real space is unrelated to the shape of the course 26c in the other space, and points are set at positions corresponding to the running distance. For example, the course starts at point P231 on the starting line 38, and the goal is the same point P235 after two laps around the track. The scale is calculated from the ratio of the total length of the course 26c in the other space to the course 27c in the real space, and the passing points on the course 27c in the real space are automatically determined as positions at a distance obtained by multiplying the distance from the start of the course 26c in the other space to the passing points by the scale. In this method of calculating distance, the image display device 2 itself must measure the running distance of the user on the course 27c in the real space. As an alternative method, a beacon is installed at the passing point position on the course 27c in the real space, and the reaching of the passing point is detected by the received signal from the beacon.

Here, the latter method using beacons will be explained. Assuming a race with multiple people, beacons 36a to 36d are placed at the start/finish and each passing point on course 27c in real space so that position information can be accurately measured. The receivable range where signals from the beacons can be received at or above a certain threshold is indicated by dashed line 37. The user starts at point P231 (beacon 36a) and proceeds counterclockwise, passing point P232 (beacon 36c), then completes one lap at the same point as the start (beacon 36a). The user then passes points P233 (beacon 36b) and P234 (beacon 36d), before completing two laps at point P235 (beacon 36a) where the finish line 38 is drawn, and reaches the finish line.

Figure 11 shows a user wearing an image display device. Here, a smartphone 40, which is an image display device 2, is attached to the body of the user 20 using a fixing device 41 so that the display screen can be viewed hands-free. The smartphone 40 uses a Bluetooth (registered trademark) function as the position detection unit 7a to detect signals emitted by the beacons 36a to 36d and is able to know the current location. When going around the track multiple times as in this example, it is possible to determine which of points P231 to P235 has been reached by combining this with the number of times the beacon signal has been received (i.e. the number of laps).

An application that associates points P231 to P235 on course 27c in the real world with points P131 to P135 on course 26c in another space is installed on smartphone 40. Image data on course 26c in another space is either downloaded in advance from web server 4 via communication network 5, or is kept in a state where it can be downloaded from web server 4 at any time in response to a request from the application.

A specific example will be described in which multiple users race on course 27c in real space. After launching the application on smartphone 40, a first group of athletes waits just before point P231, the starting point, within range 37 of reception of the signal from beacon 36a, and starts the race at 00 seconds of every minute. After the start, a second group of athletes similarly waits just before point P231. This method makes it possible to have a wave start, in which the athletes are divided into groups of several and start at the same time. The starter can either give the signal to start according to the posted time at 00 seconds of every minute, or simply watch over the athletes to make sure they don't get a false start.

When each runner passes through points P233 and P234 in the real world and approaches the finish point P235, the smartphone 40 measures the time when the received signal from the beacon 36a reaches a maximum as the finish time. The finish time of each runner is transmitted, for example, via the wireless communication network 5 to a counting device (not shown) of the race organizer and used to count the race rankings.

Figures 12A to 12C are diagrams showing examples of the display screen of the smartphone 40. Here, the display is shown when the user approaches point P232 in the real space corresponding to point P132 in the other space. In the real space course 27c, the vehicle goes straight near point P232, but in the other space course 26c, the vehicle turns right at point P132. As the user drives, the display unit 12a changes and displays image 24h of FIG. 12A, which shows a right turn in the other space, to image 24i of FIG. 12B. These images include buildings 35a, 35b and roads 42a, 42b taken in the other space. In the other space, the vehicle turns right at this point, but in the real space, the image is displayed in which the traveling direction is always seen as a straight line. In other words, since the correspondence between the traveling direction is not fixed between the other space and the real space, the shape of the course in the real space relative to the other space can be freely set. Note that the correspondence between the traveling direction is fixed, for example, in a state where the traveling direction is always shifted by 30 degrees clockwise.

Figure 12C shows an example of a display that reflects the user's forward direction and posture (the azimuth and elevation angles of the user's line of sight). Figure 12B shows a display image 24j in which the user twists his or her upper body to the left and looks down while passing by an image 35b of a building in another space in order to view it. In normal image display on a smartphone, even if the image has a wide viewing angle such as a 360° angle of view, only images within the range of the angle of view of the smartphone orientation that matches the direction of travel are displayed. Therefore, the orientation/posture detection unit 11a of the smartphone 40 can detect the movement of the user's upper body and display images within the range of the angle of view of that line of sight. As a result, in the display image 24j, an image 35c of a building is displayed directly in front of the user and an image 42c of a road is displayed below it.

According to the third embodiment, the present invention can be applied to competitions in which runners run around the same course while looking at scenery in different spaces. In particular, runners can easily become bored in long-distance races with many laps, but being able to run while looking at different scenery from time to time has the effect of eliminating that feeling of boredom.

Figure 11 shows an example in which the smartphone 40 is attached to the body using a fixing device 41 so that it can be viewed hands-free, but typically the smartphone 40 is placed in a pocket of a backpack or the like, and the smartphone 40 can be set to emit an alarm when approaching each point. When the user hears the alarm, it means that the user is approaching the next point, so the user can pick up the smartphone 40 and look at the screen of the smartphone 40.

In addition, in the third embodiment, not only still images but also videos can be displayed on the display unit. In the above explanation, a beacon is used, but at the very least, it is sufficient to be able to play videos only within the range where the beacon is received. Alternatively, if you want to display videos outside the beacon reception range, you can calculate the user's running speed between adjacent beacons and change the playback speed of the captured video accordingly.

In addition, if the real space is a circular course and passing points can be identified using only the position detection unit without using beacons, the video can be aligned based on the position coordinates obtained from the position detection unit and played back over the entire course. In that case, position information can be added to the captured video at regular time intervals or when passing through a point.

In the above embodiments, the user moves on foot or by running, but the user may also move by bicycle, a car that is not driven by the user, or public transportation. The user may also change the intermediate points or goal of the course during the movement. In addition, when displaying images on the image display device while moving within each section, the images may be images such as the scenery from a car window that have been photographed in advance in a different space and registered in a state accessible from the image display device.

Although several embodiments have been described above, the present invention is not limited to the above-mentioned embodiments and includes various modified examples. For example, the above-mentioned embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. It is also possible to add the configuration of one embodiment to the configuration of another embodiment. It is also possible to add, delete, or replace other configurations with respect to part of each embodiment.

1: Image display system, 2: Image display device, 3: Imaging device, 4: Web server, 5: Communication network, 6a, 6b: Imaging element, 7a, 7b: Position detection unit, 8a, 8b, 8c: Memory, 9a, 9b, 9c: Communication interface, 10a, 10b, 10c: Control unit, 11a, 11b: Orientation and attitude detection unit, 12a, 12b: Display unit, 13: Audio output unit, 19: Head mounted display (HMD), 20: User, 40: Smartphone.

Claims (2)

An image display method for displaying image information of a second space different from the first space by an image display device of a first space, comprising:
a course information acquisition step of acquiring course information including at least one set of section start point information and section end point information in the second space, and image information related to the section end point information ;
a map information display step of displaying map information of the first space;
a start point position input step of accepting a designation of a start point position on the map information ;
a recommended course and image display point setting step of setting a recommended course and the image display points along which the image display device can move in the first space based on the map information , the positional relationship in the second space between the section start point information and the section end point information and the positional relationship in the first space between the start point position and the image display point being on an approximate scale;
a position information acquiring step of acquiring position information of the image display device in the first space;
an image information display step of displaying image information related to the section end point information when the position information is in the vicinity of the image display point when the image display device moves on the recommended course from the start point position;
An image display method comprising: The image display device displays image information of a second space different from the first space in an image display device of a first space,
a position information acquisition unit that acquires position information of the image display device in the first space;
a communication interface that acquires course information from an external device, the course information including at least one set of section start point information and section end point information in the second space, and image information related to the section end point information;
an input interface that accepts designation of a start point position in the first space ;
a display unit for displaying map information and the image information;
A control unit;
Equipped with
The control unit is
Displaying map information of the first space on the display unit;
a positional relationship in the second space between the section start point information and the section end point information and a positional relationship in the first space between the start point position and an image display point are approximated in scale , and a recommended course along which the image display device can move in the first space and the image display points are set based on the map information ;
When the image display device moves on the recommended course from the starting point position, if the position information is in the vicinity of the image display point, image information related to the section end point information is displayed on the display unit.
1. An image display device comprising:

Images