JP7732370B2 - Information processing program and information processing method - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act (1) July 23, 2021 to September 14, 2021 https://vronline. jp/touken/ (2) September 24, 2021 to September 30, 2021 https://vronline. jp/touken/

This invention relates to an information processing program and information processing method that places two-dimensional models of artworks such as swords in a virtual three-dimensional space, allowing viewers to freely view the two-dimensional models of the artworks or to view them while receiving sequential explanations of the key points to appreciate.

A sword observation system has been known in the past that involves facing the tip of a sword toward a light source to observe the blade pattern of the sword, and displays an image of the sword captured from the observer's perspective (see paragraphs "0019" and "0020" and Figures 1-4 and 8 of Patent Document 1).

Japanese Patent Application Laid-Open No. 2017-72407

However, conventional sword observation systems use actual swords, so unless the schedules of the sword-photographer and the observer can match, the swords cannot be viewed, meaning that the viewer cannot freely view them at any time. Another problem is that the viewer cannot freely operate the camera.
In contrast to this, if a photograph of a sword is used instead of the actual sword, viewers can view it freely at any time, but there is a second problem in that, because a photograph is a two-dimensional image, it lacks the sense of three-dimensionality and reality compared to the actual sword.

The present invention therefore places a virtual artwork model, a virtual camera, and a virtual light source in a virtual three-dimensional space, allowing viewers to appreciate the virtual artwork model while freely operating the virtual camera and virtual light source. Furthermore, the present invention allows viewers to appreciate the virtual artwork model while receiving sequential explanations of the viewing points by switching to navigation mode, without having to check each viewing point individually.

An information processing program according to one aspect of the present invention is an information processing program executed by a computer of an information processing device capable of receiving input via an operation unit, and includes: a model placement unit that places a virtual artwork model, which is a two-dimensional model, within a virtual three-dimensional space; a camera placement unit that places a virtual camera in the virtual three-dimensional space to photograph the virtual artwork model; a virtual light source placement unit that places a virtual light source in the virtual three-dimensional space to irradiate the virtual artwork model with light; an image generation unit that generates a captured image by photographing the virtual three-dimensional space with the virtual camera; an image display unit that displays the captured image on a screen; a camera movement control unit that moves the virtual camera within the virtual three-dimensional space based on a camera movement operation performed on the operation unit; and a light source movement control unit that moves the virtual light source within the virtual three-dimensional space based on a light source movement operation performed on the operation unit. The computer is characterized by a mode selection unit that can select between an appreciation mode in which the virtual artwork model displayed on the screen is freely viewed while moving at least one of the virtual camera and the virtual light source based on at least one of the camera movement operation and the light source movement operation, and a navigation mode in which the virtual artwork model is viewed while sequentially receiving explanations of the appreciation points of the virtual artwork model; and a navigation control unit that, after switching to the navigation mode, displays on the screen at least the images captured by the virtual camera, the term names of the appreciation points, and an explanatory text for the appreciation points; displays the term name of the next appreciation point based on a progression operation on the operation unit; and, when the virtual camera arrives at the next appreciation point, displays the explanatory text for the next appreciation point, thereby causing the computer to function as a navigation control unit that progresses navigation.

An information processing program according to one aspect of the present invention further causes the computer to function as a mission generation unit that generates a mission for searching for a target area where the next viewing point is located using the virtual camera based on the camera movement operation during the navigation process; and a mission determination unit that, after the mission is generated, switches the movement of the virtual camera from automatic to manual, automatically moves the virtual camera from its position to the target area when the display range of the virtual camera enters a preset range for the target area, and displays the explanatory text for the next viewing point after the virtual camera arrives at the target area.

An information processing program according to one aspect of the present invention is characterized in that the virtual artwork model is generated from photographic data taken from the front and back sides of the Japanese sword.
An information processing method according to one aspect of the present invention is an information processing method for controlling a computer of an information processing device capable of receiving input via an operation unit, the information processing method including a model placement step of placing a virtual artwork model, which is a two-dimensional model, in a virtual three-dimensional space; a camera placement step of placing a virtual camera in the virtual three-dimensional space to photograph the virtual artwork model; a virtual light source placement step of placing a virtual light source in the virtual three-dimensional space to irradiate the virtual artwork model with light; an image generation step of generating a captured image by photographing the virtual three-dimensional space with the virtual camera; an image display step of displaying the captured image on a screen; a camera movement control step of moving the virtual camera in the virtual three-dimensional space based on a camera movement operation performed on the operation unit; a mode selection step for selecting between an appreciation mode in which the virtual artwork model displayed on the screen is freely appreciated while moving the virtual camera and the virtual light source based on the camera movement operation and the light source movement operation, and a navigation mode in which the virtual artwork model is appreciated while receiving explanations of appreciation points in the virtual artwork model in sequence; and a navigation control step for, after switching to the navigation mode, displaying on at least the screen the images taken by the virtual camera, the term names of the appreciation points, and an explanatory text for the appreciation points, and displaying the term name of the next appreciation point based on a progress operation on the operation unit, and when the virtual camera arrives at the next appreciation point, displaying the explanatory text for the next appreciation point, thereby progressing the navigation.

According to one aspect of the present invention, by placing a virtual artwork model, a virtual camera, and a virtual light source in a virtual three-dimensional space, a viewer can view the virtual artwork model while freely operating the virtual camera and virtual light source.
Furthermore, according to one aspect of the present invention, by switching to navigation mode, the viewer can appreciate the virtual artwork model while receiving explanations of the appreciation points one by one, without having to check each one.

1 is a schematic diagram illustrating an overall image of a virtual three-dimensional space according to an embodiment of the present invention; 1 is a schematic diagram showing an overall image of an information processing system according to an embodiment of the present invention; FIG. 3 is a block diagram showing an example of the configuration of the smart device shown in FIG. 2. FIG. 3 is a block diagram showing an example of the configuration of the server in FIG. 2 . FIG. 4 is a block diagram showing an example of the configuration of the processor unit in FIG. 3 . This is an explanatory diagram of sword terminology. 1A and 1B are explanatory diagrams of coordinates, in which FIG. 1A shows three-dimensional coordinates, and FIG. 1B shows an explanatory diagram of virtual three-dimensional coordinate groups. FIG. 10 is an explanatory diagram illustrating the relationship between an operation trajectory and a camera movement trajectory in the embodiment based on the present invention. 10A and 10B are explanatory diagrams for explaining a light source moving operation in the embodiment based on the present invention. FIG. 10 is an explanatory diagram showing an example of a display on a screen of a smart device according to an embodiment of the present invention. FIG. 11 is an explanatory diagram showing another display example corresponding to FIG. 10 . 1 is a perspective view illustrating an example of a remote control device according to an embodiment of the present invention.

(Virtual 3D space 60 in FIG. 1)
In FIG. 1, 60 denotes a virtual three-dimensional space, in which a virtual sword model 61, a virtual camera 62, and a virtual light source 63 are arranged.
The virtual sword model 61 is a two-dimensional model that imitates a real sword.
The virtual sword model 61 is placed on a plane parallel to the XZ axes in the virtual three-dimensional space 60 and is stationary.
In contrast to this, the virtual camera 62 and virtual light source 63 are arranged so as to be movable in the X, Y and Z axes directions relative to the stationary virtual sword model 61 .
Moreover, the display range 65 of the virtual camera 62 coincides with the range of the screen 70 of the smart device 30, which will be described later with reference to FIG.
Here, in the virtual three-dimensional space 60, the X axis is positioned in the width direction of the virtual sword model 61, the Y axis is positioned in the thickness direction of the virtual sword model 61, and the Z axis is positioned in the length direction of the virtual sword model 61. The X axis of the virtual three-dimensional space 60 corresponds to the left-right direction on the screen 70 of the smart device 30 shown in Figure 8, the Y axis corresponds to the thickness direction of the smart device 30, and the Z axis corresponds to the up-down direction of the screen 70.

(Virtual Sword Model 61)
The virtual sword model 61 is generated from two pieces of sword image data, for example, photographed from the front and back sides of a Japanese sword. The virtual sword model 61 is in a state in which the front image of the sword image data is attached to the front side of a plane having a front and back side, and the back image is attached to the back side of the plane.
Although a Japanese sword has been exemplified as the virtual sword model 61, the Japanese sword is an example of a work of art.

(work of art)
Artworks broadly include all art works (including art and crafts).
Artworks that are flat and have some unevenness are suitable. Japanese swords are made of metal, but even non-metallic objects with different textures can have those texture differences reflected in the virtual art model. Artworks include not only swords such as Japanese swords, but also paintings, folding screens with ink paintings, Buddhist embroideries, textiles, embroidery, folding screens, and gold coins (old coins).

(Information processing system 10 of FIG. 2)
The information processing system 10 includes a server 20, a smart device 30, and a personal computer 40 (hereinafter referred to as a "PC").
The smart device 30 is, for example, a smartphone or a tablet computer.
The PC 40 may be, for example, a desktop type, a notebook type, or a tablet type.
The server 20, the smart device 30, and the PC 40 are configured to be able to communicate with each other via the Internet.
In this embodiment, with this configuration, it is assumed that services such as viewing the virtual sword model 61 and navigation to viewing points will be provided. An application for realizing this service is installed on the smart device 30 or the like. Then, the service is provided to the user by transmitting model data of the virtual sword model 61 via the server 20.

(Hardware configuration of smart device 30 in FIG. 3)
The smart device 30 includes a processor unit 31, an internal storage device 32, a main memory 33, a communication unit 34, an operation unit 35, a display unit 36, and a gyro sensor 37. The smart device 30 may also include a camera, a speaker, a GPS, a light, etc.
The processor unit 31 controls the operation of the smart device 30 by executing a system program (not shown) for performing information processing, which will be described later, and for controlling the overall operation of the smart device 30.

The processor unit 31 may be equipped with a single processor or multiple processors. The internal storage device 32 stores various programs executed by the processor unit 31 and various data used by the programs.
The internal storage device 32 is, for example, a flash EEPROM or a hard disk drive. The main memory 33 temporarily stores computer programs and information. The communication unit 34 is connected to a network via wired or wireless communication and transmits and receives predetermined data to and from the server 20. The operation unit 35 is, for example, an input device for receiving operations from a user. The display unit 36 is typically a liquid crystal display device. Note that in the processing according to this embodiment, a touch panel integrated with an LCD screen is assumed as the operation unit 35 and the display unit 36. In other embodiments, a gyro sensor 37 or a predetermined pointing device other than a touch panel may be used as the operation unit 35.

(Hardware configuration of server 20 in FIG. 4)
The server 20 includes at least a processor unit 21 , an internal storage device 22 , a main memory 23 , a communication unit 24 , and a database 25 .
The processor unit 21 executes various programs for controlling the server 20. The internal storage device 22 stores various programs executed by the processor unit 21 and various data used by the processor unit 21. The main memory 23 temporarily stores computer programs and information. The communication unit 24 connects to a network via wired or wireless communication and transmits and receives predetermined data between the smart device 30 and other servers (not shown).

(Software configuration of smart device 30 in FIG. 5)
The processor unit 31 of the smart device 30 reads an information processing program and functions as a computer, acting as each unit shown in FIG.
The various units include: (1) a mode selection unit 200, (2) a model placement unit 201, (3) a camera placement unit 202, (4) a light source placement unit 203, (5) an image generation unit 204, (6) an image display unit 205, (7) a camera movement control unit 206, (8) a light source movement control unit 207, (9) a point sequence coordinate information acquisition unit 208, (10) a data receiving unit 209, (11) a navigation control unit 210, (12) a mission generation unit 211, (13) a mission determination unit 212, and (14) a model inversion unit 213.
Although each unit has been described using the processor unit 31 of the smart device 30, it may be executed by a computer in the processor unit (not shown) of the PC 40 in FIG.

(Mode selection unit 200 in FIG. 5)
The mode selection unit 200 is a means for selecting between an appreciation mode in which the virtual sword model 61 displayed on the screen 70 is freely viewed while moving at least one of the virtual camera 62 and the virtual light source 63 based on at least one of a camera movement operation and a light source movement operation, and a navigation mode in which the virtual sword model 61 is viewed while receiving sequential explanations of the viewing points of the virtual sword model 61.

(Model placement unit 201 in FIG. 5)
The model placement unit 201 is a means for placing a virtual sword model 61, which is a two-dimensional model, in a virtual three-dimensional space 60, as shown in FIG.

(Camera placement unit 202 in FIG. 5)
The camera placement unit 202 is a means for placing a virtual camera 62 for photographing a virtual sword model 61 in a virtual three-dimensional space 60, as shown in FIG.

(Light source arrangement unit 203 in FIG. 5)
The light source placement unit 203 is a means for placing a virtual light source 63 that irradiates the virtual sword model 61 with light within the virtual three-dimensional space 60, as shown in FIG.

(Image generation unit 204 in FIG. 5)
The image generating unit 204 is a means for generating a captured image by capturing an image of the virtual three-dimensional space 60 with the virtual camera 62 .

(Image display unit 205 in FIG. 5)
The image display unit 205 is a means for displaying a captured image on the screen 70 .

(Camera movement control unit 206 in FIG. 5)
The camera movement control unit 206 is a means for moving the virtual camera 62 within the virtual three-dimensional space 60 based on a camera movement operation performed on the operation unit 35 .
Here, the camera movement operation refers to a swipe operation on the screen 70, which will be described later with reference to FIG.
In addition, when the virtual sword model 61 is warped and a camera movement operation is performed to move the virtual camera 62 linearly in the longitudinal direction of the virtual sword model 61, the camera movement control unit 206 moves the virtual camera 62 in a curved manner along the blade 100 of the virtual sword model 61.
Specifically, the camera movement control unit 206 normally reflects the vertical (screen Y coordinate (up and down)) movement amount obtained by the swipe operation only in the Z value of the coordinates of the virtual camera 62 (for example, (0,0,0) → (0,0,5)), but corrects the Z value to an X value based on the point sequence coordinate information (for example, (0,0,0) → (3,0,5)).
As a result, as will be described later with reference to Figure 8, the virtual camera 62 is moved in a curve along the blade 100 of the virtual sword model 61, thereby preventing the virtual sword model 61 from going outside the screen 70 due to the influence of the warp 105 (see Figure 6).

(Light source movement control unit 207 in FIG. 5)
Light source movement control unit 207 is means for moving virtual light source 63 in virtual three-dimensional space 60 based on a light source movement operation on operation unit 35 .
Furthermore, light source movement control unit 207 moves virtual light source 63 in accordance with the amount of light source movement operation.
Here, the light source movement operation is, for example, a tilting operation of the smart device 30.
The tilting motion is detected by, for example, the gyro sensor 37 of the smart device 30.

(Point sequence coordinate information acquisition unit 208 in FIG. 5)
The point sequence coordinate information acquisition unit 208 is a means for acquiring point sequence coordinate information, which is the coordinates in the virtual three-dimensional space 60 of each point in the point sequence that is arranged at predetermined intervals along the blade of the virtual sword model 61 .
Here, the point sequence coordinate information refers to a virtual three-dimensional coordinate group 64, which will be described later with reference to FIG.

(Data receiving unit 209 in FIG. 5)
The data receiving unit 209 is a means for receiving model data of the virtual sword model 61 generated outside the information processing device, the virtual sword model 61 being generated from photographic data taken from both the front and back sides of the Japanese sword.
Here, the outside of the information processing device refers to, for example, the server 20 shown in Fig. 2. Although the server 20 is exemplified as the outside, the outside is not limited to this and may be another server (not shown).
The model data is received using, for example, the communication unit 34 shown in FIG.

(Navigation control unit 210)
After switching to navigation mode, the navigation control unit 210 displays at least on the screen 70 the images taken by the virtual camera 62, the viewing points, and explanatory text for the viewing points, and displays the next viewing point based on the progress operation on the operation unit 35.When the virtual camera 62 arrives at the next viewing point, the navigation control unit 210 is a means for progressing through navigation by displaying explanatory text for the next viewing point.
Here, the progress operation refers to a touch operation of the first button 91 or the second button 92 displayed on the screen 70, which will be described later with reference to FIG.
Although the navigation is performed using the display on the screen 70, the present invention is not limited to this, and for example, audio may also be used in combination.

(Mission generation unit 211 in FIG. 5)
The mission generation unit 211 is a means for generating a mission to search for a target area where the next viewing point is located by the virtual camera 62 based on a camera movement operation during navigation.

(Mission determination unit 212 in FIG. 5)
The mission determination unit 212 is a means for switching the movement of the virtual camera 62 from automatic to manual after a mission occurs, automatically moving the virtual camera 62 from its position to the target area when the display range 65 of the virtual camera 62 enters a preset range for the target area, and displaying an explanatory text for the next viewing point after the virtual camera 62 arrives at the target area.

(Model inversion unit 213 in FIG. 5)
The model inversion unit 213 is a means for inverting the virtual sword model based on a model inversion operation performed on the operation unit 35 .
Here, the model inversion operation refers to, for example, an action of shaking the smart device 30 from side to side.
The left and right shaking motion is detected by, for example, the gyro sensor 37 of the smart device 30.
Although a left-right shaking action has been exemplified as a model flipping operation, the present invention is not limited to this, and the virtual sword model 61 may be flipped by swiping the screen 70 left or right.
The virtual sword model 61 is inverted by calculating point A (a_x, 0, c_z) where the Z coordinate of the center point (c_x, 0, c_z) within the display range 65 (see FIG. 1) of the virtual camera 62 overlaps with the virtual three-dimensional coordinate group 64 of the virtual sword model 61, which will be described later using FIG. 7(a), on the plane of "y = 0" where the sword data, which will be described later using FIG. 7(a), is located, and the line X = a_x is used as the rotation axis.
The virtual sword model 61 rotates 180 degrees around the rotation axis.
Furthermore, the rotation axis changes each time depending on the position of the virtual camera 62 .
The rotation direction of the virtual sword model 61 coincides with the direction in which the smart device 30 is swung.

(Terminology for the sword 50 (virtual sword model 61) in FIG. 6, etc.)
The terminology for each part of the sword 50 and the distance between each part will be explained using FIG. 6.
The terms and distances for the sword 50 are: (1) blade 100, (2) nakago (stalk) 101, (3) kizami (edge) 102, (4) total length 103, (5) blade length 104, (6) curvature 105, (7) tip length 106, (8) tip length 107, (9) butanouchi (edge cut) 108, (10) motohaba (width at the base) 109, and (11) tsuna (ridge) 110.
Although the terms and distances for the sword 50 have been explained, the same terms and distances will also be used for the virtual sword model 61.
The curvature 105 means the "dimension" or "portion" of the longest part between the straight line connecting the tip of the cutting edge 102 and the ridge 110 and the ridge.

(3D coordinates 51 and virtual 3D coordinate group 64 in FIG. 7)
The three-dimensional coordinates 51 and the virtual three-dimensional coordinate group 64 will be explained with reference to FIG.
10A shows a reference point set in two-dimensional sword data, and the three-dimensional coordinates 51 of the reference point. The value of the three-dimensional coordinates 51 is (0.0, 0.0, 0.0). The three-dimensional coordinates 51 indicate the coordinates of the reference point on the X, Y, and Z axes.

10(b) shows a virtual three-dimensional coordinate group 64 of a sequence of points set at predetermined intervals along the blade of the virtual sword model 61. The virtual three-dimensional coordinate group 64 was generated in accordance with the two-dimensional sword data described above.
The virtual three-dimensional coordinate group 64 is, for example, (-1.8, 0.0, -36.0), (3.4, 0.0, 0.0), and (-1.5, 0.0, 28.0) in order from top to bottom.
The virtual three-dimensional coordinate group 64 is based on the three-dimensional coordinate 51, and on the X axis, moving leftward in FIG. 1B is assigned a "plus" value, and moving rightward is assigned a "minus" value.
The virtual three-dimensional coordinate group 64 is used by the camera movement control unit 206 to correct the movement of the virtual camera 62 .

(Relationship between operation trajectory 71 and camera movement trajectory 72 in FIG. 8)
For example, when a swipe operation is performed linearly upward on the screen 70 of the smart device 30 as shown on the left side of the figure, the operation locus 71 becomes a straight line.
On the other hand, because the Japanese sword has a curvature 105, the movement of the virtual camera 62 within the virtual three-dimensional space 60 is corrected by the camera movement control unit 206, and the camera moves in a curved line along the blade 100 of the virtual sword model 61. When this camera movement trajectory 72 is aligned with the operation trajectory 71 and displayed on the screen 70, it appears as shown on the left side of the figure.

(Zooming in and out of the virtual sword model 61)
As for the zoom operation, when a pinch operation is performed on the screen 70, the virtual camera 62 in FIG. 1 moves in the direction of the zoom-out arrow, the display range 65 becomes larger, and the captured image on the screen 70 becomes smaller.
Conversely, when a pinch-in operation is performed, the virtual camera 62 in FIG. 1 moves in the direction of the enlargement arrow, the display range 65 becomes smaller, and the captured image on the screen 70 is enlarged.

(Light source movement operation in FIG. 9)
The relationship between the tilting action of the smart device 30 and the movement of the virtual reflecting portion 80 of the light emitted from the virtual light source 63 will be described with reference to FIG. 9 .
Based on the position of the virtual light source 63 in the virtual three-dimensional space 60, a virtual light source fixed position 81 in the real world is set outside the smart device 30.
The tilt of the smart device 30 is then calculated using, for example, the gyro sensor 37, and the light source movement control unit 207 controls the position of the virtual light source 63 in the virtual three-dimensional space 60 so that it coincides with the virtual light source fixed position 81.

(Display example of screen 70 in FIGS. 10 and 11)
A display example in the navigation mode will be described with reference to FIGS.
10, the screen 70 of the display unit 36 displays an image of the virtual sword model 61 taken by the virtual camera 62, a display position 90 for displaying the name of the sword that is the basis for the virtual sword model 61, virtual points and their explanations, and a first button 91 (e.g., "back") and a second button 92 (e.g., "next") for operating the navigation. These displays are controlled by the image display unit 205.
It should be noted that the images and characters displayed on the screen 70 are not limited to the display example shown in FIG.

When the navigation mode starts, a viewing point (for example, a "stem" (nakago)) is displayed at the display position 90 as shown in FIG.
After the virtual point is displayed, touching the second button 92 (for example, "Next") causes the virtual camera 62 to move to the virtual point (for example, "Stem"), and when the virtual camera 62 arrives at the virtual point, the captured image is displayed, and an explanatory text about the virtual point is displayed at the display position 90, as shown in Fig. 11. This constitutes one cycle, and each time the second button 92 is touched, one cycle is repeated a predetermined number of times to progress the navigation.
The input operation is to touch the second "Next" button 92, which automatically moves the virtual camera 62. During this time, i.e., until the second "Next" button 92 is touched, the viewer is free to operate the virtual camera 62 manually.

(Mission occurs)
A mission will be generated during navigation.
The mission is to display "Let's zoom in on the stem area here" at display position 90 in FIG.
The viewer watches the mission and performs a camera movement operation to move the virtual camera 62. While watching the captured image, the viewer moves the virtual camera 62 to search for the target area where the viewing point is located.
Thereafter, when the virtual camera 62 arrives at the target area, an explanation of the virtual point is displayed at the display position 90 as shown in FIG.

On the other hand, if the virtual camera 62 does not arrive at the target area, the explanation of the virtual point is not displayed at the display position 90 .
The viewer then moves the virtual camera 62 further to search for the target area.
As a result, when the display range 65 of the virtual camera 62 enters a preset range for the target area, the virtual camera 62 automatically moves from that position to the target area. After the virtual camera 62 arrives at the target area, an explanatory text for the virtual point is displayed at the display position 90, as shown in FIG. 11. These are controlled by the mission determination unit 212.

An example of the remote control device 41 will be described with reference to FIG.
The remote control device 41 has a handle shape and includes a gyro sensor. The remote control device 41 is connected to a PC 40 or the like. The gyro sensor detects tilting or shaking of the remote control device 41 and transmits an electrical signal to the PC 40.
The virtual camera 62 and the virtual light source 63 are moved by tilting or shaking the remote control device 41 .

(Functions of each area)
The navigation control unit 210 provides various functions.
In addition to the "viewing mode" and "navigation mode," the smart device 30 also has (1) an appreciation note function, (2) a purchase/sword selection function, and (3) a bonus function, all of which are not shown.
The appreciation note function is a function that allows the user to store and view appreciation notes of virtual sword models 61 that have already been appreciated.
The purchase/sword selection function allows you to purchase a sword individually and have a portion of the profits returned to that sword.
The bonus feature allows you to store and view images, videos, columns, etc. of limited-edition products and bonuses.

(Processing for increasing the three-dimensional effect of the virtual sword model 61)
In this information processing program (hereinafter also referred to as the "sword appreciation app"), for example, in order to increase the three-dimensionality of the virtual sword model 61, an image containing surface unevenness information and gloss information in addition to a photograph of the sword (color information) is used.
Specifically, in addition to a photograph of the sword, a normal map that holds surface normal information corresponding to each pixel in the photograph, a reflection intensity map that holds gloss information, and a surface roughness (smoothness) map are used.
Based on this information, the amount of light hitting each pixel in the photo is changed to create a three-dimensional effect.
The unevenness is expressed using a technique called "normal mapping." In the sword appreciation app, normal mapping is used to make flat images appear pseudo-three-dimensional.
In addition to the pseudo-shape created by normal mapping, the gloss information that determines how each part reflects is expressed by using images of specular reflection information, surface roughness, or smoothness information to express differences in the reflection of the blade pattern and jigane.

(Features and Effects of the Embodiments)
The features of the embodiment are as follows.
(First feature)
The first feature is that the information processing program is executed by a computer of an information processing device (smart device 30) capable of receiving input from an operation unit 35, and includes a model placement unit 201 that places a virtual artwork model (virtual sword model 61), which is a two-dimensional model, in a virtual three-dimensional space 60; a camera placement unit 202 that places a virtual camera 62 that photographs the virtual artwork model (virtual sword model 61) in the virtual three-dimensional space 60; and a virtual light source 63 that irradiates light onto the virtual artwork model (virtual sword model 61) in the virtual three-dimensional space 60. 6, an image generation unit 204 that generates a captured image by capturing an image in the virtual three-dimensional space 60 with the virtual camera 62, an image display unit 205 that displays the captured image on the screen 70, a camera movement control unit 206 that moves the virtual camera 62 in the virtual three-dimensional space 60 based on a camera movement operation on the operation unit 35, and a light source movement control unit 207 that moves the virtual light source 63 in the virtual three-dimensional space 60 based on a light source movement operation on the operation unit.
Furthermore, light source movement control unit 207 is characterized in that it moves virtual light source 63 in accordance with the amount of light source movement operation.

(Effect of the first feature)
According to the first feature, by placing a virtual artwork model (virtual sword model 61), a virtual camera 62, and a virtual light source 63 in a virtual three-dimensional space 60, a viewer can freely view the virtual artwork model (virtual sword model 61) at any time while freely operating the virtual camera 62 and virtual light source 63.
Furthermore, according to the first feature, by moving the virtual light source 63 in accordance with the amount of operation of the light source movement operation, the position at which light is reflected in the virtual artwork model (virtual sword model 61) can be changed, thereby creating a three-dimensional effect in the virtual artwork model (virtual sword model 61).
Furthermore, according to the first feature, in addition to the "three-dimensional effect," the virtual art model 61 can also reproduce the textures that correspond to the characteristics of each art piece, such as the brush strokes in the case of a painting, the beautiful luster of an art piece that uses lacquer or gold leaf, and the delicate texture of each individual thread of embroidery in the case of an embroidered Buddha (a Buddhist statue painted with embroidery), and by making it possible to view the work by changing the way the light hits it, the user can be given the experience of viewing the real thing.
In particular, by using the virtual sword model 61 as a virtual artwork model, it is possible to express not only the "three-dimensional effect" but also the delicate texture of the sword surface, such as the "hamon" and "jigane" in a realistic manner.
The major feature of using the virtual sword model 61 is the quality and realism of the appearance, which makes it seem as if you are viewing the real thing.
The surface of a sword is very complex due to its material and the way it is made (forged), so unlike materials such as cloth, paper, or stone, it is a very difficult material to express because it is a matter of expressing the texture within the material "iron."
In addition, the real-time nature of the sword, in which the reflections of the sword change as you tilt the device, is an important element in expressing the realism of viewing a sword.
Furthermore, if the virtual sword model 61 is used and the light source movement operation is performed by "tilting" (gyro sensor), a more realistic viewing experience can be created, as if you were actually holding the sword in your hands and admiring the blade pattern, etc.

(Second characteristic point)
The second feature is that when the virtual sword model 61 is warped, the camera movement control unit 206 moves the virtual camera 62 in a curved manner along the blade 100 of the virtual sword model 61 when a camera movement operation is performed to move the virtual camera 62 linearly in the longitudinal direction of the virtual sword model 61.

(Effect of the second feature)
According to the second feature, by placing a virtual sword model 61, a virtual camera 62, and a virtual light source 63 in a virtual three-dimensional space 60, the viewer can freely view the virtual sword model 61 at any time while freely operating the virtual camera 62.
Furthermore, according to the second feature, when the virtual sword model 61 is warped, the virtual camera 62 can be moved in a curved line along the blade 100 of the virtual sword model 61, thereby preventing the virtual sword model 61 from moving outside the screen 70 due to the influence of the warp 105.

(Third characteristic point)
The third feature is that the computer further functions as a model inversion unit 213 that inverts the virtual sword model 61 inside out based on a model inversion operation on the operation unit 35 .

(Effect of the third feature)
According to the third feature, when viewing the virtual sword model 61, it is necessary to turn the virtual sword model 61 over, and at that time, by turning the virtual sword model 61 over, the back surface can be viewed.
Furthermore, according to the third feature, when the flipping operation is performed by "tilting" (gyro sensor), a more realistic viewing experience can be created, as if picking up the sword and looking at its backside.

(Fourth feature)
A fourth feature is that the virtual sword model 61 is generated from photographic data taken from both the front and back sides of the Japanese sword.

(Effect of the fourth feature)
According to the fourth feature, by generating the virtual sword model 61 from photographic data of a Japanese sword, the virtual sword model 61 can be generated realistically and the data volume can be reduced.

(Fifth characteristic point)
a camera placement step of placing a virtual camera 62 that photographs the virtual artwork model (virtual sword model 61) within the virtual three-dimensional space 60; a light source placement step of placing a virtual light source 63 that irradiates the virtual artwork model (virtual sword model 61) within the virtual three-dimensional space 60; an image generation step of generating a captured image by photographing the virtual three-dimensional space 60 with the virtual camera 62; an image display step of displaying the captured image on a screen 70; a camera movement control step of moving the virtual camera 62 within the virtual three-dimensional space 60 based on a camera movement operation performed on the operation unit 35; and a light source movement control step of moving the virtual light source 63 within the virtual three-dimensional space 60 along the virtual sword model 61 based on a light source movement operation performed on the operation unit 35.
The light source movement control step is characterized in that virtual light source 63 is moved in accordance with the amount of light source movement operation.

(Effect of the fifth feature)
The fifth feature provides the same effect as the first feature.

(Sixth characteristic point)
a camera placement unit 202 that places a virtual camera 62 that photographs the virtual artwork model (virtual sword model 61) in the virtual three-dimensional space 60; a light source placement unit 203 that places a virtual light source 63 that irradiates the virtual artwork model (virtual sword model 61) with light in the virtual three-dimensional space 60; an image generation unit 204 that generates a captured image by photographing the virtual three-dimensional space 60 with the virtual camera 62; an image display unit 205 that displays the captured image on a screen 70; a camera movement control unit 206 that moves the virtual camera 62 in the virtual three-dimensional space 60 based on a camera movement operation performed on the operation unit 35; a mode selection unit 200 that can select between an appreciation mode in which the virtual artwork model (virtual sword model 61) displayed on the screen 70 is freely appreciated while moving at least one of the virtual camera 62 and the virtual light source 63 based on at least one of the camera movement operation and the light source movement operation, and a navigation mode in which the virtual artwork model (virtual sword model 61) is appreciated while receiving explanations of the appreciation points of the virtual artwork model (virtual sword model 61) in sequence; and a navigation control unit 210 that, after switching to the navigation mode, displays on at least the screen 70 images taken by the virtual camera 62, the term names of the appreciation points, and an explanation of the appreciation point, and displays the term name of the next appreciation point based on a progression operation on the operation unit 35, and when the virtual camera 62 arrives at the next appreciation point, displays the explanation of the next appreciation point, thereby causing the computer to function as a navigation control unit 210 that progresses navigation.

(Effect of the Sixth Feature)
According to the sixth feature, by placing a virtual sword model, which is a two-dimensional model, a virtual camera, and a virtual light source in a virtual three-dimensional space, the viewer can view the virtual artwork model (virtual sword model 61) while freely operating the virtual camera and virtual light source.
Furthermore, according to one aspect of the present invention, by switching to navigation mode, the viewer can appreciate the virtual artwork model (virtual sword model 61) while receiving explanations of the appreciation points one by one, without having to check each one.

(Seventh characteristic point)
A seventh feature is that the computer further functions as a mission generation unit 211 that generates a mission to search for a target area where the next viewing point is located using the virtual camera 62 based on camera movement operations while navigation is in progress, and a mission determination unit 212 that switches the movement of the virtual camera 62 from automatic to manual after the mission is generated, automatically moves the virtual camera 62 from its position to the target area when the display range 65 of the virtual camera 62 enters a preset range for the target area, and displays an explanatory text for the next viewing point after the virtual camera 62 arrives at the target area.

(Effect of the Seventh Feature)
According to the seventh feature, by using the virtual camera 62 to search for the target area where the next viewing point is located based on the camera movement operation, it is possible to create an atmosphere in which the viewer feels as if they are visiting an art museum and viewing the exhibits while listening to an explanation from a curator, thereby adding an active and experiential element.
Furthermore, according to the seventh feature, the display range does not need to completely match the target area as long as it is within the set range, allowing for smoother viewing and creating the impression that the viewer is receiving support from a curator.

(Eighth characteristic point)
An eighth feature is that the virtual artwork model (virtual sword model 61) is generated from photographic data taken from the front and back sides of the Japanese sword.

(Effect of the Eighth Feature)
According to the eighth characteristic point,
According to the eighth feature, the same effect as that of the fourth feature is achieved.

(Ninth characteristic point)
a camera placement step of placing a virtual camera 62 that photographs the virtual artwork model (virtual sword model 61) in the virtual three-dimensional space 60; a light source placement step of placing a virtual light source 63 that irradiates the virtual artwork model (virtual sword model 61) in the virtual three-dimensional space 60; an image generation step of generating a captured image by photographing the virtual three-dimensional space 60 with the virtual camera 62; an image display step of displaying the captured image on a screen 70; a camera movement control step of moving the virtual camera 62 in the virtual three-dimensional space 60 based on a camera movement operation performed on the operation unit 35; a mode selection step that allows selection between an appreciation mode in which the virtual artwork model (virtual sword model 61) displayed on the screen 70 is freely appreciated while moving the virtual camera 62 and the virtual light source 63 based on the camera movement operation and the light source movement operation, and a navigation mode in which the virtual artwork model (virtual sword model 61) is appreciated while receiving explanations of appreciation points of the virtual artwork model (virtual sword model 61) in sequence; and a navigation control step that, after switching to the navigation mode, displays on at least the screen 70 images taken by the virtual camera 62, the term names of the appreciation points, and an explanation of the appreciation points, and displays the term name of the next appreciation point based on a proceeding operation on the operation unit 35 (for example, a touch operation of the first button 91 or the second button 92 in Figure 10), and when the virtual camera 62 arrives at the next appreciation point, displays the explanation of the next appreciation point, thereby progressing the navigation.

(Effect of the ninth feature)
According to the ninth feature, the same effect as that of the sixth feature is achieved.

REFERENCE SIGNS LIST 10 Information processing system 20 Server 21 Processor unit 22 Internal storage device 23 Main memory 24 Communication unit 25 Database 30 Smart device 31 Processor unit 32 Internal storage device 33 Main memory 34 Communication unit 35 Operation unit 36 Display unit 37 Gyro sensor 40 PC
41 Remote control device 50 Sword 51 Three-dimensional coordinates 60 Virtual three-dimensional space 61 Virtual sword model 62 Virtual camera 63 Virtual light source 64 Virtual three-dimensional coordinate group 65 Display range 70 Screen 71 Operation trajectory 72 Camera movement trajectory 80 Virtual reflection section 81 Virtual light source fixed position 90 Display position 91 First button 92 Second button 100 Blade 101 Tang 102 Tip 103 Overall length 104 Blade length 105 Curvature 106 Tip length 107 Tip length 108 Butt joint 109 Base width 110 Ridge section 200 Mode selection section 201 Model placement section 202 Camera placement section 203 Light source placement section 204 Image generation section 205 Image display section 206 Camera movement control section 207 Light source movement control unit 208 Point sequence coordinate information acquisition unit 209 Data receiving unit 210 Navigation control unit 211 Mission generation unit 212 Mission determination unit 213 Model inversion unit

Claims (4)

An information processing program to be executed by a computer of an information processing device capable of accepting input from an operation unit,
a model placement unit that places a virtual artwork model, which is a two-dimensional model, in a virtual three-dimensional space;
a camera placement unit that places a virtual camera that photographs the virtual artwork model within the virtual three-dimensional space;
a virtual light source arrangement unit that arranges a virtual light source in the virtual three-dimensional space to irradiate the virtual artwork model with light;
an image generation unit that generates a captured image by capturing an image of the virtual three-dimensional space with the virtual camera;
an image display unit that displays the captured image on a screen;
a camera movement control unit that moves the virtual camera within the virtual three-dimensional space based on a camera movement operation performed on the operation unit;
a light source movement control unit that moves the virtual light source in the virtual three-dimensional space based on a light source movement operation performed on the operation unit;
a mode selection unit that can select between an appreciation mode in which the virtual artwork model displayed on the screen is freely appreciated while moving at least one of the virtual camera and the virtual light source based on at least one of the camera movement operation and the light source movement operation, and a navigation mode in which the virtual artwork model is appreciated while receiving sequential explanations of the appreciation points of the virtual artwork model;
An information processing program that causes the computer to function as a navigation control unit that, after switching to the navigation mode, displays on at least the screen the image taken by the virtual camera, the term name of the viewing point, and an explanatory text for the viewing point, and displays the term name of the next viewing point based on a progress operation on the operation unit, and when the virtual camera arrives at the next viewing point, displays the explanatory text for the next viewing point, thereby progressing the navigation. a mission generating unit that generates a mission to search for a target area where the next viewing point is located by the virtual camera based on the camera movement operation during the progress of the navigation;
2. The information processing program according to claim 1, further causing the computer to function as a mission determination unit that switches the movement of the virtual camera from automatic to manual after the occurrence of the mission, automatically moves the virtual camera from its position to the target area when the display range of the virtual camera enters a predetermined range for the target area, and displays the explanatory text for the next viewing point after the virtual camera arrives at the target area. The information processing program according to claim 1 or 2, wherein the virtual artwork model is generated from photographic data taken from the front and back sides of the Japanese sword. An information processing method for controlling a computer of an information processing device capable of accepting input from an operation unit, comprising:
a model placement step of placing a virtual artwork model, which is a two-dimensional model, in a virtual three-dimensional space;
a camera placement step of placing a virtual camera in the virtual three-dimensional space to capture an image of the virtual artwork model;
a virtual light source placement step of placing a virtual light source in the virtual three-dimensional space to irradiate the virtual artwork model with light;
an image generation step of generating a captured image by capturing an image of the virtual three-dimensional space with the virtual camera;
an image display step of displaying the captured image on a screen;
a camera movement control step of moving the virtual camera within the virtual three-dimensional space based on a camera movement operation performed on the operation unit;
a light source movement control step of moving the virtual light source in the virtual three-dimensional space based on a light source movement operation performed on the operation unit;
a mode selection step for selecting between an appreciation mode in which the virtual artwork model displayed on the screen is freely appreciated while moving the virtual camera and the virtual light source based on the camera movement operation and the light source movement operation, and a navigation mode in which the virtual artwork model is appreciated while receiving sequential explanations of the appreciation points of the virtual artwork model;
An information processing method that causes the computer to execute a navigation control step in which, after transitioning to the navigation mode, the image taken by the virtual camera, the term name of the viewing point, and an explanatory text for the viewing point are displayed at least on the screen, the term name of the next viewing point is displayed based on a progress operation on the operation unit, and when the virtual camera arrives at the next viewing point, the explanatory text for the next viewing point is displayed to progress the navigation.