JP7760873B2 - Image generation device, image generation method, and image generation program - Google Patents

Description

The present invention relates to an image generation device, an image generation method, and an image generation program.

Technology for providing virtual reality is rapidly advancing in a variety of fields, allowing people to simulate experiences of worlds that are difficult to experience in real life.

As a technology related to this, Patent Document 1 discloses a virtual reality service provision system that enables presentation in a virtual reality world that is integrated with the real world. This system stores multiple virtual reality objects and information indicating virtual reality services available to the user. The system determines the placement positions of the virtual reality objects in a virtual space based on ambient situation information that indicates the user's surroundings. The system then generates coded data for the video to be displayed on the display panel of a head-mounted display worn by the user on their head, based on user state information that indicates the user's state and the placement positions of the virtual reality objects.

Patent Document 2 also discloses a virtual reality presentation system that takes into consideration the impact that a user's actions in a virtual reality world have on the real world. This system detects the user's surroundings and state, and generates an image of a virtual reality object based on the detected user state. The system displays the generated image and outputs audio associated with the displayed virtual reality object. The system then alerts the user using video or audio based on the detected surroundings of the user.

JP 2016-045814 A JP 2014-170330 A

People engaged in various occupations typically improve their occupational skills by accumulating work experience. Therefore, while it would be desirable to be able to efficiently accumulate work experience that will significantly improve their occupational skills, in reality, opportunities to gain such work experience are limited, making it difficult for individuals to accumulate the work experience they desire. Furthermore, it is particularly difficult to accumulate the desired work experience in dangerous occupations such as police work. One way to solve this problem would be to simulate the work experience using technology related to providing virtual reality, as described in Patent Documents 1 and 2 above. However, currently, there are no systems that realize such simulated work experiences, and Patent Documents 1 and 2 do not specifically mention technology that realizes simulated work experiences.

The primary objective of the present invention is to provide an image generation device that allows users to efficiently accumulate experience related to a job through a simulated job experience.

An image generation device according to one aspect of the present invention includes: an acquisition means for acquiring, when a first subjective image of virtual reality for simulating a job is presented to a user, behavioral information representing the behavior of the user in response to the first subjective image; a determination means for determining future changes in the state of an object included in the first subjective image that will occur due to the user's behavior, based on scenario determination criteria representing the relationship between the behavioral information and state change information representing the future changes in the state of the object; and a generation means for generating a second subjective image that follows the first subjective image, based on the state change information and image generation criteria, and presenting the generated second subjective image to the user.

In another aspect of achieving the above object, an image generation method according to one embodiment of the present invention includes, when a first subjective image of virtual reality for simulating a job is presented to a user, acquiring behavioral information representing the behavior of the user in response to the first subjective image by an information processing device, determining future changes in the state of an object included in the first subjective image that will occur due to the user's behavior based on scenario determination criteria representing the relationship between the behavioral information and state change information representing the future changes in the state of the object, generating a second subjective image that follows the first subjective image based on the state change information and image generation criteria, and presenting the generated second subjective image to the user.

In yet another aspect of achieving the above object, an image generation program according to one embodiment of the present invention causes a computer to execute the following steps when a first subjective image of virtual reality for simulating a job is presented to a user: an acquisition process for acquiring behavioral information representing the behavior of the user in response to the first subjective image; a determination process for determining future changes in the state of an object included in the first subjective image that will occur due to the user's behavior, based on scenario determination criteria that represent the relationship between the behavioral information and state change information representing the future changes in the state of the object; and a generation process for generating a second subjective image that follows the first subjective image, based on the state change information and image generation criteria, and presenting the generated second subjective image to the user.

Furthermore, the present invention can also be realized by a computer-readable, non-volatile recording medium on which such an image generation program (computer program) is stored.

This invention allows users to simulate a job, allowing them to efficiently accumulate experience related to that job.

1 is a block diagram showing a configuration of an image generation device 10 according to a first embodiment of the present invention. 10 is a diagram illustrating an example of a scenario setting menu screen that the acquisition unit 11 according to the first embodiment of the present invention displays on the screen 400 of the management terminal device 40. FIG. FIG. 2 is a diagram illustrating an example of a tree-structured graph representing the scenario determination criteria 142 according to the first embodiment of the present invention. FIG. 2 is a diagram illustrating an example of data of status change information 143 according to the first embodiment of the present invention. FIG. 10 is a diagram showing an example of the determination unit 12 according to the first embodiment of the present invention sequentially determining subsequent scenes in accordance with an action selected by an experiencer for a scene. 1 is a flowchart (1/2) showing the operation of the image generation device 10 according to the first embodiment of the present invention. 10 is a flowchart (2/2) showing the operation of the image generation device 10 according to the first embodiment of the present invention. FIG. 10 is a block diagram showing the configuration of an image generating device 30 according to a second embodiment of the present invention. FIG. 9 is a block diagram showing the configuration of an information processing device 900 that can realize an image generation device according to each embodiment of the present invention.

Embodiments of the present invention will be described in detail below with reference to the drawings.

First Embodiment
FIG. 1 is a block diagram showing the configuration of an image generating device 10 according to a first embodiment of the present invention. The image generating device 10 is an information processing device that generates educational image content (e.g., animation) for police officers, for example, by using virtual reality technology to enable police officers to simulate the investigation of a crime or accident. The educational image content is a subjective image from the perspective of a participant (police officer) experiencing the simulated investigation of a crime or accident. Hereinafter, in this embodiment, this subjective image may be simply referred to as an image. The subjective image may also include audio.

As shown in FIG. 1, the video generation device 10 is communicatively connected to a VR (Virtual Reality) goggles 21, a camera 22, a microphone 23, an input device 24, and a management terminal device 40. The VR goggles 21 are a head-mounted display worn by participants who view the educational video content described above to simulate the investigation of a crime or accident. Instead of the VR goggles 21, participants may also view the video content using a regular monitor.

The camera 22 captures the participant's actions (body movements, etc.) at specific scenes (timings) in the video content. The microphone 23 collects sounds made by the participant at specific scenes in the video content. The input device 24 is a device such as a joystick, mouse, or keyboard that the participant uses to perform input operations on the menu screen displayed on the screen of the VR goggles 21.

The management terminal device 40 is an information processing device such as a personal computer that is used by the administrator of the image production device 10 when inputting information into the image production device 10 or when checking information output from the image production device 10. The management terminal device 40 displays the information output from the image production device 10 on its screen 400. As will be described later, the management terminal device 40 is used, for example, by police training personnel when setting up scenarios for video content.

As shown in FIG. 1, the video generation device 10 includes an acquisition unit 11, a determination unit 12, a generation unit 13, and a storage unit 14. The acquisition unit 11, determination unit 12, generation unit 13, and storage unit 14 are examples of an acquisition means, a determination means, a generation means, and a storage means, respectively.

The storage unit 14 is, for example, a storage device such as RAM (Random Access Memory) or a hard disk 904, which will be described later with reference to FIG. 8. The storage unit 14 stores behavior information 141, scenario determination criteria 142, state change information 143, and image generation criteria 144. Details of the information stored in the storage unit 14 will be described later.

First, we will explain how the video generation device 10 according to this embodiment generates a scenario for educational video content to be presented to participants through input operations by an educator or other person.

The acquisition unit 11 displays a scenario setting menu on the screen 400 of the management terminal device 40 for generating scenarios for video content related to the investigation of various incidents or accidents.

Figure 2 is a diagram illustrating an example of a scenario setting menu screen that the acquisition unit 11 according to this embodiment displays on the screen 400 of the management terminal device 40. The acquisition unit 11 displays on the screen 400 a scenario setting menu screen that allows the user to input information for setting scene x (x is an identifier that can identify the scene), which is one of the scenes in the video content to be presented to the viewer. Note that a scene according to this embodiment represents a scene that is a unit that makes up the video content. The acquisition unit 11 displays on the screen, for example, a pull-down menu that allows the user to input information about the people, objects, and places that appear. However, it is assumed that the items shown in the pull-down menu have been provided in advance by, for example, the administrator (educator) of the video generation device 10.

The acquisition unit 11 acquires information representing the scenario settings entered by the educator through input operations on the management terminal device 40 using the pull-down menu. In the example shown in Figure 2, the educator has selected a suspicious person as a character, a car as a feature, and a public road as a location. Furthermore, the acquisition unit 11 may display icons representing the character, feature, and location selected by the educator on the screen 400, as shown in Figure 2, for example.

The acquisition unit 11 displays on the screen a pull-down menu into which information describing the actions of people and objects can be input for each person and object appearing in scene x in the video content set as described above. The acquisition unit 11 acquires information describing the movements in scene x that is input by the educator through an input operation on the management terminal device 40 using the pull-down menu. In the example shown in Figure 2, the educator has set a scene in which a car driven by a suspicious person stops while traveling on a public road as scene x in the video content.

Note that the scenario setting menu screen illustrated in Figure 2 is an example, and the acquisition unit 11 may display a scenario setting menu screen of a different form than that shown in Figure 2 on the screen 400 of the management terminal device 40.

The acquisition unit 11 displays a scenario setting menu, such as the one shown in FIG. 2, on the screen 400 for each scene included in the educational video content, and the educator performs an input operation to set the scene for each scene. The acquisition unit 11 generates or updates the state change information 143 (described below) based on the information entered through the input operation.

The acquisition unit 11 may also display on the screen 400 a menu screen for selecting the educational objective for the educational video content for which a scenario is to be created. The acquisition unit 11 may then narrow down the characters, character movements, objects, object movements, and locations selectable on the setting menu screen described above, according to the objective entered by the educator. In this case, the acquisition unit 11 prevents inappropriate options from being displayed in the pull-down menu, so that inappropriate scenes such as "a car driving in an incident that occurred inside a private home" or "a person swimming when there is no sea" are not set. However, in this case, it is assumed that criteria for displaying appropriate options for characters, character movements, objects, object movements, and locations according to the educational objective are stored in the storage unit 14.

Next, we will explain how the video generation device 10 according to this embodiment presents a subjective video, which is educational video content, to a user.

The acquisition unit 11 first displays a menu on the screen of the VR goggles 21 that allows the user to select video content depicting the investigation of a crime or accident that the user wishes to experience. The acquisition unit 11 acquires information input by the user, for example via the input device 24, in response to the displayed menu as identification information for scene 1, the first scene of the video content.

After the user selects scene 1 of the video content as described above, the generation unit 13 shown in FIG. 1 generates a video of scene 1 and displays the generated video on the screen of the VR goggles 21. In other words, the generation unit 13 generates a video (animation) of a car driven by a suspicious person traveling on a public road and stopping. The method for generating the video by the generation unit 13 will be described later.

At a predetermined timing (e.g., immediately before the completion of Scene 1) while the image of Scene 1 is being displayed on the screen of the VR goggles 21 by the generation unit 13, the acquisition unit 11 displays information on the screen of the VR goggles 21 superimposed on the image of Scene 1, prompting the user to select an action to be taken in the scene of Scene 1 (Action 1). The acquisition unit 11 may, for example, display options for Action 1 on the screen of the VR goggles 21.

The user selects at least one of the options displayed on the screen. The user may make this selection through an input operation using the input device 24, by speaking a word representing the option, or by moving their body in accordance with the option.

The acquisition unit 11 acquires information representing input operations performed by the experiencer using the input device 24 as behavioral information 141 representing action 1. The acquisition unit 11 also collects voice uttered by the experiencer using the microphone 23, performs voice recognition on the collected voice using existing voice recognition technology, and acquires the result as behavioral information 141 representing action 1. The acquisition unit 11 also acquires video of the experiencer via the camera 22, and performs image recognition on the acquired video using existing image recognition technology, and acquires the result as behavioral information 141 representing action 1. In this case, the behavioral information 141 also includes the movement of each part of the experiencer's body, the direction of their gaze, and so on.

The acquisition unit 11 stores the acquired behavior information 141 representing action 1 in the memory unit 14.

The determination unit 12 determines scene 2, which occurs as a result of action 1 and follows scene 1, based on behavioral information 141 representing action 1 acquired by the acquisition unit 11 and scenario determination criteria 142.

Figure 3 is a diagram illustrating a tree-structured graph representing the scenario determination criteria 142 according to this embodiment. In the graph illustrated in Figure 3, nodes represented by circles represent scenes in the video presented to the viewer, and edges represented by arrows connecting nodes represent actions that the viewer will take in each scene.

The scenario determination criteria 142 are generated or updated by the acquisition unit 11, for example, via input operations by an educator on the management terminal device 40. At this time, the acquisition unit 11 displays a graph representing the scenario determination criteria 142, such as the one illustrated in FIG. 3, on the screen 400 of the management terminal device 40 so that it can be generated or updated. The acquisition unit 11 acquires information representing, for example, input operations to create or delete nodes, as illustrated in FIG. 3, input operations to connect nodes with edges, and input operations to assign identifiers that can identify actions to each edge. The identifiers that can identify actions are information that can identify the actions of people, the movements of objects, and locations in the state change information 143 illustrated in FIG. 4, which will be described later. The acquisition unit 11 generates or updates the scenario determination criteria 142 based on the information representing the input operations described above.

According to the scenario determination criteria 142 illustrated in FIG. 3, scene 1 is set to one of multiple scenes 1, such as scene 1-1 or scene 1-2, depending on the video content selected by the participant. Then, depending on action 1 selected by the participant in scene 1, scene 2 following scene 1 will be one of multiple scenes 2, such as scene 2-1-1 or scene 2-1-2. Similarly, depending on action 2 selected by the participant in scene 2, scene 3 following scene 2 will be one of multiple scenes 3, such as scene 3-1-1-1 or scene 3-1-1-2.

Note that the scenario determination criteria 142 may be represented in a form other than the tree-structured graph illustrated in FIG. 3. The scenario determination criteria 142 may be, for example, a table representing the relationship between scenes and actions.

The future changes in the state of objects included in the video presented to the viewer for each scene represented by the scenario determination criteria 142 (i.e., the actions of the characters, the movements of the objects, and the location of the scene) are managed by state change information 143.

Figure 4 is a diagram illustrating data of the status change information 143 according to this embodiment. According to the status change information illustrated in Figure 4, scene 1-1 represented by the scenario determination criteria 142 is a scene in which a car driven by a suspicious individual stops while traveling on a public road. Similarly, scene 2-1-1 represented by the scenario determination criteria 142 is a scene in which a suspicious individual discards an item they are carrying, and scene 3-1-1-1 is a scene in which the suspicious individual behaves in an unsettled manner.

As explained with reference to Figures 3 and 4, the scenario determination criteria 142 represent the relationship between behavior information 141, which is represented as the actions selected by the viewer in each scene, and state change information 143, which represents future changes in the state of objects included in the video (the actions of the characters, the movements of the characters, and the location of the scene).

Furthermore, the scenario determination criteria 142 may be a learning model that is the result of learning the relationship between the actions (behavior information 141) taken by a participant in a certain situation and the changes in the situation (state change information 143) that result from those actions, for example, during the investigation of an actual incident or accident in the past.

Figure 5 is a diagram showing an example of the determination unit 12 according to this embodiment sequentially determining subsequent scenes depending on the action selected by the user for a scene. Based on the initial settings of the scenario selected by the user, the determination unit 12 determines scene 1-1 as scene 1 from among multiple scenes including scene 1-1, scene 1-2, etc. in the scenario determination criteria 142 illustrated in Figure 3. However, scene 1-1 is a scene in which a car driven by a suspicious person stops while traveling on a public road, as illustrated in Figures 4 and 5.

The determination unit 12 then determines scene 2-1-1 as scene 2 in response to the experiencer's selection of the action of instructing the suspicious person to get off (instructing the suspicious person to get off) as action 1 for the above-mentioned scene 1-1. That is, in this case, in the scenario determination criteria 142 illustrated in FIG. 3, the edge connecting scene 1-1 and scene 2-1-1 represents the action of instructing the suspicious person to get off. Note that scene 2-1-1 is a scene in which a suspicious person discards an item they are holding, as illustrated in FIGS. 4 and 5.

The determination unit 12 then determines scene 3-1-1-1 as scene 3 in response to the experiencer's selection of action 2, which is to gaze at an object that has fallen to the ground, for the above-mentioned scene 2-1-1. That is, in this case, in the scenario determination criteria 142 illustrated in FIG. 3, the edge connecting scene 2-1-1 and scene 3-1-1-1 represents the action of gazing at an object that has fallen to the ground. Note that scene 3-1-1-1 is a scene in which a suspicious person behaves restlessly, as illustrated in FIGS. 4 and 5.

Furthermore, in response to selecting the action of instructing a suspicious person to present their driver's license (instructing the suspicious person to present their driver's license) as action 3 for the above-mentioned scene 3-1-1-1, the determination unit 12 determines, in the scenario determination criteria 142, as scene 4 the subsequent scene connected to scene 3-1-1-1 by the edge representing that action.

The generation unit 13 generates a subjective video for each scene determined by the determination unit 12 as described above, based on the state change information 143 and the video generation criteria 144. In the example shown in Figure 5, the generation unit 13 first generates a video of scene 1-1, in which a car driven by a suspicious person stops while traveling on a public road.

Here, we will explain an example of a method for generating subjective video by the generation unit 13. In Figure 4, for the sake of convenience, the actions of people, movements of objects, and locations represented by the status change information 143 are shown in plain text, but in reality, the status change information 143 contains people, their actions, objects, movements of objects, locations, etc. as codes so that information processing can be performed on them. The video generation criteria 144 is then standard information that represents the relationship between these codes and the video related to those codes.

The generation unit 13 generates a video that will serve as the background of the scene by comparing the location code represented by the state change information 143 with the video generation standard 144. For example, if the location code represents a public road, the generation unit 13 generates a video (animation) of a typical public road indicated by the video generation standard 144 as the background of the scene.

The generation unit 13 generates a video representing the actions of a person or the movements of an object by comparing the codes of the person, the person's actions, the object, and the object's movements represented by the state change information 143 with the image generation standard 144. However, the state change information 143 has codes representing changes in state (movement) for each part of a person (e.g., hands, feet, head, torso, eyes, mouth, etc.) and each part of an object (for example, if the object is a car, the entire car, doors, windows, lights, etc.) so that the actions of a person or the movements of an object can be identified. The generation unit 13 then compares the codes representing changes in state for each part of a person or object with the image generation standard 144, thereby generating a video (animation) of a scene representing the actions of a person or the movements of an object using existing image generation technology.

The generation unit 13 displays the subjective video of the scene generated as described above on the screen of the VR goggles 21.

Next, the operation (processing) of the image generation device 10 according to this embodiment will be described in detail with reference to the flowcharts in Figures 6A and 6B.

The acquisition unit 11 displays a video content selection menu on the screen of the VR goggles 21 worn by the user (step S101). The acquisition unit 11 accepts an input operation by the user to select video content from the displayed video content selection menu (step S102). The image generation device 10 sets the variable i (i is a natural number) to "1" (step S103).

The determination unit 12 determines scene i based on the scenario determination criteria 142 from the selection of video content (when i = 1) or the experiencer's action i-1 (when i ≥ 2) (step S104). The generation unit 13 generates a subjective video of scene i based on the state change information 143 and video generation criteria 144 related to the determined scene i, and displays the generated subjective video on the screen of the VR goggles 21 (step S105).

The video generation device 10 checks whether the scenario determination criteria 142 indicate that scene i is the last scene (step S106). If scene i is the last scene (Yes in step S107), the entire process ends. If scene i is not the last scene (No in step S107), the acquisition unit 11 checks whether it is time to acquire behavioral information 141 representing the experiencer's action i (step S108).

If it is not time to acquire behavioral information 141 (No in step S109), the process returns to step S108. If it is time to acquire behavioral information 141 (Yes in step S109), the acquisition unit 11 displays information prompting the user to select action i to take in scene i on the screen of the VR goggles 21, superimposed on the video of scene i (step S110). The acquisition unit 11 acquires behavioral information 141 representing the user's performance of action i (step S111). The video generation device 10 adds "1" to variable i, and the process returns to step S104.

The image generation device 10 according to this embodiment enables the user to efficiently accumulate experience related to a job through a simulated experience of that job. This is because the image generation device 10 sequentially generates and presents subsequent subjective images to the user based on the user's actions in response to the virtual reality subjective images presented to the user to simulate the job, and based on scenario determination criteria 142 that indicate changes in the scenario resulting from the user's actions.

The effects achieved by the image generation device 10 according to this embodiment are described in detail below.

People working in various occupations usually improve their job skills by accumulating work experience. Therefore, it would be desirable to be able to efficiently accumulate work experience that will significantly improve their job skills. However, in reality, opportunities to gain such work experience are limited, making it difficult for individuals to accumulate as much work experience as they wish. Furthermore, it is particularly difficult to accumulate as much work experience as desired in dangerous occupations such as police work. Therefore, in order to solve this problem, it is important to enable people to experience simulated jobs, for example by using technology related to the provision of virtual reality.

In response to these issues, the image generation device 10 according to this embodiment includes an acquisition unit 11, a determination unit 12, and a generation unit 13, and operates as described above with reference to, for example, FIGS. 1 to 6B. That is, when a first subjective video of virtual reality for simulating a job is presented to a user, the acquisition unit 11 acquires behavioral information 141 representing the behavior of the user in response to the first subjective video. The determination unit 12 determines future changes in the state of an object included in the first subjective video, which will occur due to the user's behavior, based on scenario determination criteria 142 representing the relationship between the behavioral information 141 and state change information 143 representing future changes in the state of the object. The generation unit 13 then generates a second subjective video that follows the first subjective video based on the state change information 143 and image generation criteria 144, and presents the generated second subjective video to the user.

The image generation device 10 then uses scenario determination criteria 142 based on the actual work experience of many people in the past to present the user with a virtual reality subjective image that is equivalent to the actual job. This allows the image generation device 10 to enable the user to efficiently accumulate experience related to the job through a simulated experience of the job.

Furthermore, the image generation device 10 according to this embodiment may use a learning model that learns the association between behavioral information 141 and state change information 143 as the scenario determination criteria 142. This allows the image generation device 10 to more accurately enable the user to efficiently accumulate experience related to a job through a simulated experience of that job.

The image generation device 10 according to this embodiment may also have a function for simultaneously presenting subjective images to multiple participants. In this case, the acquisition unit 11 acquires behavioral information 141 relating to each of the multiple participants who are simultaneously presented with subjective images. The determination unit 12 generates state change information 143 from the behavioral information 141 relating to the multiple participants. The generation unit 13 presents each of the multiple participants with subjective images that include other participants other than the participant. By having such a function, the image generation device 10 can enable users to efficiently accumulate experience relating to a job, even for a job that requires multiple people to work together, for example.

Furthermore, the subjective video generated by the video generation device 10 according to this embodiment is not limited to video in which the experiencer is a police officer and experiences investigating a crime or accident. The subjective video generated by the video generation device 10 may be, for example, video in which the experiencer is a store clerk and experiences serving customers in a store, or video in which the experiencer is a teacher and experiences teaching students.

Second Embodiment
FIG. 7 is a block diagram showing the configuration of an image generating device 30 according to the second embodiment of the present invention.

The video generation device 30 according to this embodiment includes an acquisition unit 31, a determination unit 32, and a generation unit 33. The acquisition unit 31, the determination unit 32, and the generation unit 33 are examples of an acquisition means, a determination means, and a generation means, respectively.

When a first subjective video 331 of virtual reality for simulating a job is presented to the user, the acquisition unit 31 acquires behavioral information 311 that represents the user's behavior in response to the first subjective video 331. The first subjective video 331 is, for example, an image similar to the subjective video generated by the image generation device 10 according to the first embodiment and displayed on the screen of the VR goggles 21. The behavioral information 311 is, for example, information similar to the behavioral information 141 according to the first embodiment. The acquisition unit 31 operates in the same manner as, for example, the acquisition unit 11 according to the first embodiment.

The determination unit 32 determines future changes in the state of the object included in the first subjective video 331 that occur due to the action of the user, based on scenario determination criteria 322 that represent the relationship between the action information 311 and state change information 321 that represents future changes in the state of the object. The state change information 321 is, for example, information similar to the state change information 143 according to the first embodiment. The scenario determination criteria 322 are, for example, criteria similar to the scenario determination criteria 142 according to the first embodiment. The determination unit 32 operates in the same manner as the determination unit 12 according to the first embodiment, for example.

The generation unit 33 generates a second subjective video 332 that follows the first subjective video 331 based on the state change information 321 and the video generation standard 333, and presents the generated second subjective video 332 to the user. The video generation standard 333 is, for example, a standard similar to the video generation standard 144 according to the first embodiment. Like the first subjective video 331, the second subjective video 332 is, for example, a subjective video similar to the subjective video generated by the video generation device 10 according to the first embodiment and displayed on the screen of the VR goggles 21. The generation unit 33 operates in the same manner as the generation unit 13 according to the first embodiment, for example.

The image generation device 30 according to this embodiment enables the user to efficiently accumulate experience related to a job through a simulated experience of that job. This is because the image generation device 30 sequentially generates and presents subsequent subjective images to the user based on the user's actions in response to the virtual reality subjective images presented to the user to simulate the job, and based on scenario determination criteria 322 that indicate changes in the scenario resulting from the user's actions.

<Hardware configuration example>
In the above-described embodiments, each unit in the image generation device shown in Figures 1 and 7 can be realized by dedicated HW (Hardware) (electronic circuitry). In Figures 1 and 7, at least the following components can be considered as functional (processing) units (software modules) of a software program that includes instructions executed by a processor.
Acquisition units 11 and 31,
decision units 12 and 32;
generation units 13 and 33,
- Storage control function in the storage unit 14.

However, the division of each component shown in these drawings is for the sake of convenience, and various configurations are possible when implementing the system. An example of the hardware environment in this case will be explained with reference to Figure 8.

Figure 8 is a diagram illustrating an exemplary configuration of an information processing device 900 (computer) capable of realizing the image generation device according to each embodiment of the present invention. That is, Figure 8 shows the configuration of a computer (information processing device) capable of realizing the image generation device shown in Figures 1 and 7, and represents a hardware environment capable of realizing each function in the above-mentioned embodiments.

The information processing device 900 shown in FIG. 8 includes the following components.
・CPU (Central_Processing_Unit) 901,
・ROM (Read_Only_Memory) 902,
・RAM (Random_Access_Memory) 903,
Hard disk (storage device) 904,
communication interface 905,
Bus 906 (communication line),
A reader/writer 908 capable of reading and writing data stored on a recording medium 907 such as a CD-ROM (Compact Disc Read Only Memory),
Input/output interface 909 including a monitor, speaker, keyboard, etc.

In other words, the information processing device 900 having the above components is a general computer in which these components are connected via a bus 906. The information processing device 900 may have multiple CPUs 901, or may have a CPU 901 configured with multiple cores.

The above-described embodiment may also supply a computer program capable of realizing the following functions to the information processing device 900 shown in Figure 8. For example, the functions are the above-described configurations in the block diagrams (Figures 1 and 7) referred to in the description of the embodiment, or the functions of the flowcharts (Figures 6A and 6B). The functions of the image generation device according to this embodiment are then achieved by reading, interpreting, and executing the computer program into the CPU 901 of the hardware. The computer program supplied to the device may be stored in readable and writable volatile memory (RAM 903) or a non-volatile storage device such as ROM 902 or hard disk 904.

Furthermore, in the above case, the method for supplying the computer program to the hardware can employ currently common procedures. Examples of such procedures include installing the program into the device via a recording medium 907 such as a CD-ROM, or downloading it from an external source via a communications line such as the Internet. In such cases, the computer program supplied to the information processing device according to this embodiment can be considered to be composed of the code that constitutes the program, or the recording medium 907 on which that code is stored.

The present invention has been described above using the above-described embodiment as an exemplary example. However, the present invention is not limited to the above-described embodiment. In other words, the present invention can be applied in various aspects that are understandable to those skilled in the art within the scope of the present invention.

REFERENCE SIGNS LIST 10 Image generation device 11 Acquisition unit 12 Determination unit 13 Generation unit 14 Storage unit 141 Action information 142 Scenario determination criteria 143 Status change information 144 Image generation criteria 21 VR goggles 22 Camera 23 Microphone 24 Input device 30 Image generation device 31 Acquisition unit 311 Action information 32 Determination unit 321 Status change information 322 Scenario determination criteria 33 Generation unit 331 First subjective video 332 Second subjective video 333 Image generation criteria 40 Management terminal device 400 Screen 900 Information processing device 901 CPU
902 ROM
903 RAM
904 Hard disk (storage device)
905 Communication interface 906 Bus 907 Recording medium 908 Reader/writer 909 Input/output interface

Claims (10)

an acquisition means for acquiring, when a first subjective image of virtual reality for simulating a job is presented to the experiencer, behavioral information representing the experiencer's behavior in response to the first subjective image;
a determining means for determining a future change in state of an object, which is included in the first subjective video and which is caused by the action of the user and which is selected from candidate objects narrowed down according to the input educational objective , based on the action information and a scenario determining criterion which represents a future change in state of the object and indicates a relationship with state change information according to the educational objective ;
a generation means for generating a second subjective video subsequent to the first subjective video based on the status change information and a video generation standard that indicates a relationship between the status change information and a video , and presenting the generated second subjective video to the user;
An image generating device comprising: the acquiring means acquires the behavioral information representing at least one of the voice, the line of sight, and the movement of each body part of the experiencer;
The image generating device according to claim 1 . the acquiring means displays options for the experiencer's actions in the first subjective video, and acquires the action information through an input operation by the experiencer to select at least one of the options.
3. The image generating device according to claim 1. The determining means determines the behavior of the person who is the target and the movement of the object who is the target.
The image generating device according to any one of claims 1 to 3. the scenario determination criterion is a learning model that learns the association between the behavior information and the state change information;
The image generating device according to any one of claims 1 to 4. further comprising a storage means in which the scenario determination criteria and the image generation criteria are stored,
The image generating device according to any one of claims 1 to 5. the acquiring means acquires the behavioral information regarding each of the plurality of experiencers who are simultaneously presented with the first subjective video;
the determining means generates the state change information from the behavior information regarding the plurality of experiencers;
the generating means presents the second subjective video including the other experiencers to each of the plurality of experiencers;
7. An image generating device according to claim 1. The first subjective video and the second subjective video are videos in which the experiencer experiences investigating an incident or an accident when the experiencer is a police officer, or videos in which the experiencer experiences serving customers in a store when the experiencer is a store clerk, or videos in which the experiencer experiences teaching students when the experiencer is a teacher.
8. An image generating device according to any one of claims 1 to 7. By the information processing device,
When a first subjective image of virtual reality for simulating a job is presented to a user, behavioral information representing a behavior of the user in response to the first subjective image is acquired;
determining a future change in the state of an object selected from candidate objects included in the first subjective video and narrowed down according to the input educational objective, which change is caused by the action of the user, based on the behavior information and a scenario determination criterion that represents a relevance between the behavior information and state change information that represents the future change in the state of the object and corresponds to the educational objective ;
generating a second subjective video subsequent to the first subjective video based on the status change information and a video generation standard that indicates a relationship between the status change information and a video , and presenting the generated second subjective video to the user;
Video generation method. an acquisition process for acquiring, when a first subjective video of virtual reality for simulating a job is presented to the experiencer, behavioral information representing the experiencer's behavior in response to the first subjective video;
a determination process for determining a future change in state of an object, which is included in the first subjective video and is selected from object candidates narrowed down according to the input educational objective, and which is caused by the behavior of the user, based on the behavior information and a scenario determination criterion that represents a relevance between the behavior information and state change information according to the educational objective and indicates the future change in state of the object;
a generation process of generating a second subjective video subsequent to the first subjective video based on the status change information and a video generation standard that indicates a relationship between the status change information and a video , and presenting the generated second subjective video to the user;
An image generation program for running the above on a computer.