JP7600002B2 - Image generating device and image generating method - Google Patents

Description

The present invention relates to an image generating device and an image generating method.

A content viewing device is known that suppresses motion sickness caused by the movement of a moving body when the occupant of the moving body views viewing content (for example, Patent Document 1). In this content viewing device, a display control means causes an output device to display a behavior image corresponding to the behavior of the moving body in addition to the viewing content. In the behavior image, an object is displayed that gives an idea of the acceleration that occurs with the behavior of the moving body.

JP 2018-157319 A

The content viewing device described in Patent Document 1 has the problem that the objects displayed in the behavior image are not related to the moving object itself, which can hinder the sense of immersion for users who are not in the vehicle.

The problem that the present invention aims to solve is to provide an image generating device and an image generating method that can prevent the sense of immersion of a user who is not in a vehicle from being impaired.

The present invention solves the above problem by obtaining vehicle information including information on the vehicle's behavior from a device installed in the vehicle, generating a viewpoint image which is an image viewed in a predetermined direction from a virtual viewpoint within the vehicle cabin, in which the subject sways due to the vehicle's behavior, outputting a control signal for displaying on a display device installed in a space other than the vehicle cabin, obtaining information regarding the gaze point of a user viewing the display device, and generating a viewpoint image in which, when the vehicle surroundings image is set as a target object, the sway level of the vehicle surroundings image is lower than the sway level of the interior image, or, when the interior image is set as the target object, the viewpoint image in which the sway level of the interior image is lower than the sway level of the vehicle surroundings image .

The present invention makes it possible to prevent the sense of immersion of users who are not in the vehicle from being impaired.

1 is a block diagram of an image display system according to an embodiment of the present invention. FIG. 2 is an explanatory diagram for explaining a viewpoint image. 5 is a flowchart showing image display control executed by the image generating device according to the present embodiment. 10 is a flowchart showing an image display control executed by an image generating device according to a modified example. 13 is an example of a virtual space when the display device is incorporated in a head mounted display.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
In this embodiment, an image generating device and an image generating method according to the present invention will be described using an example in which the image generating device and the image generating method are applied to an image display system.

FIG. 1 is a block diagram of an image display system 100 according to this embodiment. The image display system 100 according to this embodiment is used in a scene where a virtual space that reproduces the space where user B is present is displayed to user A who is in a different space from user B. In this embodiment, user A is taken as an example of a user who is in remote space 30, which is a location away from vehicle 1, and user B is taken as an example of a user who is in vehicle 1 and is present in in-vehicle space 20. In the following description, for convenience, user A in remote space 30 will be referred to as remote user A, and user B in in-vehicle space 20 will be referred to as in-vehicle user B.

In this embodiment, the states of remote user A and in-vehicle user B are not particularly limited. For example, in FIG. 1, remote user A is shown in a seated state, but remote user A may be standing. Also, for example, in FIG. 1, in-vehicle user B is shown in a state where he is operating the steering wheel of vehicle 1, but in-vehicle user B is not the driver, and may be a passenger sitting in the passenger seat or the back seat.

In this embodiment, the image display system 100 is used in a system that uses virtual reality technology. As shown in FIG. 1, the remote user A and the in-car user B are in different spaces, so the remote user A cannot confirm what the in-car user B is viewing in the in-car space 20. However, in the virtual reality technology that allows the user to perceive a virtual space, which is a world created by a computer, as reality, the computer can create a virtual space for the remote user A as if he were in the in-car space 20. In the virtual space, the remote user A perceives as if he were in the in-car space 20 through at least his vision and hearing. Furthermore, the image generating device and image generating method according to the present invention change the virtual space according to the behavior of the vehicle 1, so that the remote user A can visually grasp the behavior of the vehicle 1 even while in the remote space 30. In addition, since the target object that the remote user A is gazing at does not shake as much as other objects, it is possible to prevent the remote user A from becoming sick due to changes in the virtual space caused by the behavior of the vehicle 1. Hereinafter, each component of the image display system 100 will be described with reference to FIG. 1.

As shown in FIG. 1, the image display system 100 includes an image generating device 10, an in-vehicle space 20, and a remote space 30. The remote space 30 is a space separated from the in-vehicle space 20. The remote space 30 may be, for example, the interior of a vehicle other than the vehicle 1 or a room in a building, but the remote space 30 is not particularly limited as long as it is a location separate from the in-vehicle space 20.

The vehicle interior space 20 is a space showing the interior of the vehicle 1. The vehicle 1 is provided with an on-board camera 21, an on-board sensor 22, and an on-board communication device 23. In FIG. 1, these devices are shown as being provided within the vehicle interior space 20, but the on-board camera 21, the on-board sensor 22, and the on-board communication device 23 are assumed to be mounted on the vehicle 1. The devices provided on the vehicle 1 are connected, for example, by a Controller Area Network (CAN) or other on-board network (such as Ethernet (registered trademark)) to transmit and receive information to and from each other.

The on-board camera 21 is an imaging device that captures images of the periphery of the vehicle 1. The on-board camera 21 is preferably installed so as to capture images of the range visible to the occupants of the vehicle 1, and may be composed of multiple cameras, such as a front camera that captures images of the front of the vehicle, a side camera that captures images of the sides of the vehicle, and a rear camera that captures images of the rear of the vehicle. The images captured by the on-board camera 21 are output to the on-board communication device 23 as peripheral image data of the vehicle 1. The on-board camera 21 may be any device or equipment that captures images of the range visible to the occupants of the vehicle 1, and there are no particular limitations on its form, number, characteristics, installation location, etc. For example, the on-board camera 21 may be composed of multiple cameras or multiple types of cameras.

For example, the functions of the in-vehicle camera 21 include an anti-shake function that prevents the captured image from shaking due to the shaking of the vehicle 1. The anti-shake function is a function that prevents the shaking of the vehicle 1 from being reflected in the captured image. When the in-vehicle camera 21 has the anti-shake function and the function is enabled, the in-vehicle camera 21 captures an image in which the shaking is suppressed even when the vehicle 1 is shaking. The in-vehicle camera 21 outputs information on whether or not it has the anti-shake function, and if it does, information on whether the function is enabled or disabled, to the in-vehicle communication device 23 as in-vehicle camera data.

The on-board sensor 22 is a sensor that detects the current state of the vehicle 1. The current state of the vehicle 1 is the current behavior of the vehicle 1. The current behavior of the vehicle 1 includes the current attitude of the vehicle 1. There is no particular limit to the type or number of the on-board sensors 22. In this embodiment, the acceleration and angular acceleration of the vehicle 1 are described as examples of parameters indicating the behavior of the vehicle 1, but the parameters indicating the behavior of the vehicle 1 may include at least one of the acceleration and angular velocity of the vehicle 1. As shown in FIG. 1, the on-board sensor 22 is composed of an acceleration sensor 22a and an angular acceleration sensor 22b.

The acceleration sensor 22a is a so-called three-axis acceleration sensor, which is a sensor that measures the acceleration on three axes of the vehicle 1. The three axes of the vehicle 1 are an axis (x-axis) along the longitudinal direction of the vehicle 1, an axis (y-axis) along the lateral direction of the vehicle 1, and an axis (z-axis) along the up-down direction of the vehicle 1. The acceleration sensor 22a may be, for example, a MEMS type acceleration sensor. The method of measuring the acceleration in the acceleration sensor 22a is not particularly limited. The acceleration of each axis of the vehicle 1 measured by the acceleration sensor 22a is output to the in-vehicle communication device 23 as acceleration data of the vehicle 1. The acceleration sensor 22a may be a device or equipment that measures the acceleration on three axes of the vehicle 1, and the form, number, characteristics, installation location, etc. of the acceleration sensor 22a are not particularly limited. For example, the acceleration sensor 22a may be composed of multiple acceleration sensors or multiple types of acceleration sensors.

The angular acceleration sensor 22b is a sensor that measures the rotational motion of the vehicle 1. The rotational motion of the vehicle 1 includes the rotational motion of the vehicle 1 in the roll direction, the pitch direction, and the yaw direction. The angular acceleration sensor 22b is composed of, for example, a roll angular acceleration sensor that measures the angular acceleration of the vehicle 1 in the roll direction, a pitch angular acceleration sensor that measures the angular acceleration of the vehicle 1 in the pitch direction, and a yaw angular acceleration sensor that measures the angular acceleration of the vehicle 1 in the yaw direction. Note that the method of measuring the angular acceleration in the angular acceleration sensor 22b is not particularly limited. The angular acceleration in each direction of the vehicle 1 measured by the angular acceleration sensor 22b is output to the in-vehicle communication device 23 as angular acceleration data of the vehicle 1. The angular acceleration sensor 22b may be a device or equipment that measures the angular acceleration in three directions of the vehicle 1, and its form, number, characteristics, installation location, etc. are not particularly limited. For example, the angular acceleration sensor 22b may be composed of multiple angular acceleration sensors or multiple types of angular acceleration sensors.

The in-vehicle communication device 23 is a device equipped with a wireless communication function for wirelessly transmitting and receiving data to and from the outside of the vehicle 1. An example of the in-vehicle communication device 23 is a telematics control unit (TCU). The in-vehicle communication device 23 uses a communication standard such as 4G/LTE or Wifi (registered trademark) by wireless communication function to connect to the Internet and transmit and receive various data to and from a server or system provided outside the vehicle 1. The in-vehicle communication device 23 receives peripheral captured image data and in-vehicle camera data of the vehicle 1 from the in-vehicle camera 21. The in-vehicle communication device 23 also receives acceleration data and angular acceleration data of the vehicle 1 from the in-vehicle sensor 22. The in-vehicle communication device 23 transmits peripheral captured image data, acceleration data, and angular acceleration data of the vehicle 1 to the image generating device 10 at a predetermined cycle.

The remote space 30 will now be described. As shown in FIG. 1, the remote space 30 is provided with a remote communication device 31, a display device 32, a remote camera 33, and a remote controller 34. The devices provided in the remote space 30 are connected, for example, by Ethernet (registered trademark) to transmit and receive information to each other.

The remote communication device 31 is a device equipped with a wireless communication function for wirelessly transmitting and receiving data to and from the outside of the remote space 30. An example of the remote communication device 31 is a router. The wireless communication function of the remote communication device 31 is similar to that of the in-vehicle communication device 23, so the description of the wireless communication function of the in-vehicle communication device 23 is used. In this embodiment, the remote communication device 31 transmits and receives data to and from the image generation device 10. Image data is input to the remote communication device 31 from the image generation device 10, and the remote communication device 31 outputs the image data to the display device 32. In addition, gaze point information of the remote user A is input to the remote communication device 31 from the remote controller 34, and the remote communication device 31 transmits the gaze point information of the remote user A to the image generation device 10.

The display device 32 displays the image generated by the image generating device 10. The image data generated by the image generating device 10 is input to the display device 32 via the remote communication device 31. In this embodiment, the display device 32 is the screen display portion of a head-mounted display worn by the remote user A. Note that the display device 32 is not limited to being provided on a head-mounted display, and may be, for example, a television or monitor having a display such as a liquid crystal panel.

The remote camera 33 captures an image of the remote user A viewing the display device 32. The remote camera 33 is preferably provided in a location where it can capture an image of the object viewed by the remote user A in the image displayed by the display device 32. For example, the remote camera 33 is provided on a head-mounted display worn by the remote user A together with the display device 32. There may be multiple remote cameras 33, and the remote camera 33 may be composed of, for example, a camera that captures an image displayed by the display device 32 (hereinafter, also referred to as a display image) and a camera that captures the eyes of the remote user A viewing the display device 32. Note that there may be multiple cameras that capture the image displayed by the display device 32 and the remote user A. The captured image captured by the remote camera 33 is output to the remote controller 34.

The remote controller 34 is a computer that is composed of a ROM (Read Only Memory) that stores a program for identifying an object that the remote user A is gazing at in the image displayed on the display device 32, a CPU (Central Processing Unit) that executes the program stored in the ROM, and a RAM (Random Access Memory) that functions as an accessible storage device. The remote controller 34 is provided, for example, in a head-mounted display worn by the remote user A together with the display device 32 and the remote camera 33. The remote controller 34 may be provided in a location separate from the display device 32 and the remote camera 33.

The remote controller 34 acquires an image captured by the remote camera 33, which shows the eyes of the remote user A, and an image displayed on the display device 32. The remote controller 34 identifies the gaze direction of the remote user A by image processing. The remote controller 34 also identifies an object located in the line of sight of the remote user A in the display image on the display device 32 by image processing. As a result, the remote controller 34 determines that the identified object is an object being gazed at by the remote user, and identifies the position of the object as the gaze point of the remote user A. Note that the method of identifying the gaze point of the remote controller 34 may be any identification method known at the time of filing of this application, and is not limited to the above identification method. The gaze point information of the remote user A, including the object being gazed at by the remote controller 34 and the position of the object in the display image on the display device 32, is output to the remote communication device 31.

The image generating device 10 will now be described. In this embodiment, the image generating device 10 is provided in a server (not shown) located outside the vehicle interior space 20 and the remote space 30. The image generating device 10 wirelessly transmits and receives data between the vehicle-mounted communication device 23 and the remote communication device 31 via a communication device (not shown) in the server. The location where the image generating device 10 is installed is not particularly limited, and for example, the image generating device 10 may be mounted on the vehicle 1. Also, for example, the image generating device 10 may be provided in the remote space 30.

The image generating device 10 is a computer that includes a ROM (Read Only Memory) that stores a program for generating and processing an image to be viewed by a remote user A, a CPU (Central Processing Unit) that executes the program stored in the ROM, and a RAM (Random Access Memory) that functions as an accessible storage device. The image generating device 10 includes a vehicle information acquisition unit 11, a storage unit 12, a gaze point information acquisition unit 13, an image generating unit 14, and a display control unit 15. The image generating device 10 can execute the functions of the vehicle information acquisition unit 11, the storage unit 12, the gaze point information acquisition unit 13, the image generating unit 14, and the display control unit 15 by executing the program stored in the ROM. Each function of the image generating device 10 will be described.

The vehicle information acquisition unit 11 acquires vehicle information including information on the behavior of the vehicle 1 from a device provided in the vehicle 1. Specifically, the vehicle information acquisition unit 11 acquires, via the in-vehicle communication device 23, an image of the surroundings of the vehicle 1 captured by the in-vehicle camera 21, information on the anti-shake function of the in-vehicle camera 21, and the acceleration and angular acceleration of the vehicle 1 detected by the in-vehicle sensor 22. For example, the vehicle information acquisition unit 11 acquires vehicle information from the vehicle 1 at a predetermined interval.

The storage unit 12 stores interior images that represent the interior of the vehicle 1. The interior images are used by the image generation unit 14 to generate a virtual space that reproduces the interior space 20 of the vehicle 1. The storage unit 12 stores multiple types of interior images, and stores interior images for each model and grade of the vehicle 1, for example. Examples of interior images include images of an instrument panel that includes a steering wheel, shift lever, navigation operation unit, air conditioning operation unit, dashboard, speedometer, etc. Note that the interior images are not limited to images of an instrument panel, and may be, for example, images of the driver's seat, passenger seat, rear seat, an image showing the inside of a door, or an image of a ceiling or floor.

The gaze point information acquisition unit 13 acquires information related to the gaze point of the remote user A viewing the display device 32. The gaze point information acquisition unit 13 acquires gaze point information of the remote user A from the remote controller 34 via the remote communication device 31. The gaze point information of the remote user A includes at least the subject that the remote user A is gazing at in the display image of the display device 32, and the position of the subject in the display image of the display device 32.

The image generating unit 14 generates a virtual space that reproduces the interior space 20 of the vehicle, and generates an image viewed from a predetermined virtual viewpoint. The image generating unit 14 also generates an image in which the subject sways in accordance with the behavior of the vehicle 1. In this embodiment, an image viewed in a predetermined direction from a virtual viewpoint in the cabin of the vehicle 1, in which the subject sways due to the behavior of the vehicle 1, is described as a viewpoint image.

The image generating unit 14 sets a virtual viewpoint at a predetermined position within the cabin of the vehicle 1 in order to generate a viewpoint image. There is no particular limitation as to where in the cabin the virtual viewpoint is set, but it is preferable that the virtual viewpoint be set at a position where, when an image is displayed on the display device 32, the remote user A viewing the image can feel as if he or she is riding in the vehicle 1. The image generating unit 14 sets the virtual viewpoint at a position corresponding to the head of an occupant of the vehicle 1, for example. Note that position information of the head of the occupant of the vehicle 1 is pre-stored in the memory unit 12.

Next, an example of a method for generating a viewpoint image will be described. When the image generating unit 14 sets a virtual viewpoint, it generates an image showing the situation around the vehicle 1 based on the captured image of the surroundings of the vehicle 1. Then, the image generating unit 14 reads out an interior image of the vehicle 1 seen from the virtual viewpoint from the storage unit 12, and generates a virtual space that reproduces the interior space 20 of the vehicle 1 by superimposing the interior image on the image showing the situation around the vehicle 1. Furthermore, the image generating unit 14 generates a viewpoint image by cutting out an image of a portion of the virtual space that is included in the range of the virtual field of view based on the virtual viewpoint, based on a preset range of the virtual field of view.

Figure 2 is an explanatory diagram for explaining the viewpoint image. Figure 2 (A) is an example of a virtual space showing the interior of the vehicle 1 generated by the image generation unit 14. In Figure 2 (A), A is an interior image of the vehicle 1 including the car navigation display a and steering wheel b, and B is a captured image of the surroundings of the vehicle 1. The image generation unit 14 generates a virtual space showing the interior of the vehicle 1 by superimposing and displaying the interior image A on the captured image of the surroundings B. Also, in Figure 2 (A), C indicates the part included in the range of the virtual field of view when looking ahead at the vehicle 1 from the virtual viewpoint.

Figure 2 (B) shows the viewpoint image C displayed on the display device 32. The display device 32 displays the viewpoint image C as an image of the area in front of the vehicle 1 viewed from a virtual viewpoint. In the example of Figure 2, the viewpoint image C includes an interior image A showing the state of the interior of the vehicle 1 as viewed from the virtual viewpoint, and a surrounding captured image B of the vehicle 1 showing the surroundings of the vehicle 1 as viewed from the virtual viewpoint.

Next, the movement of the subject in the viewpoint image will be described. The image generation unit 14 generates the viewpoint image such that, among the subjects appearing in the viewpoint image, at least the subject other than the subject gazed at by the remote user A sways due to the behavior of the vehicle 1. The image generation unit 14 sets, among the subjects appearing in the viewpoint image, the object gazed at by the remote user A as the target object. In this embodiment, the image generation unit 14 sets, as the target object, either the interior image or the captured image of the surroundings of the vehicle 1 that constitute the viewpoint image, based on the gaze point of the remote user A.

For example, when the object gazed upon by the remote user A is a subject that appears in the interior image, the image generating unit 14 sets the interior image as the target object. In the example of the viewpoint image shown in FIG. 2B, it is assumed that the remote user A is gazing at the display a. When the image generating unit 14 identifies the gaze point of the remote user A as the position of the display a from the gaze point information of the remote user A acquired by the gaze point information acquiring unit 13, it sets the interior image A including the display a as the target object. On the other hand, for example, when the object gazed upon by the remote user A is a subject that appears in the peripheral captured image of the vehicle 1, the image generating unit 14 sets the peripheral captured image of the vehicle 1 as the target object. In the example of the viewpoint image shown in FIG. 2B, it is assumed that the remote user A is gazing at other vehicles and traffic lights other than the vehicle 1 that appear in the peripheral captured image B. When the image generating unit 14 identifies the gaze point of the remote user A as the position of another vehicle or a traffic light based on the gaze point information of the remote user A acquired by the gaze point information acquiring unit 13, it sets the surrounding captured image B including the other vehicle or traffic light as the target object.

When the image generating unit 14 sets a target object in the interior image or the peripheral captured image of the vehicle 1, the image generating unit 14 generates a viewpoint image in which the rocking level of the target object is lower than the rocking level of objects other than the target object. The rocking level is the rocking width or speed at which the subject rocks in response to the behavior of the vehicle 1. For example, when the object that the remote user A is gazing at is included in the interior image and the target object is set in the interior image, the image generating unit 14 generates a viewpoint image in which the rocking level at which the interior image rocks in response to the behavior of the vehicle 1 is lower than the rocking level at which the peripheral captured image of the vehicle 1 rocks in response to the behavior of the vehicle 1. On the other hand, for example, when the object that the remote user A is gazing at is included in the peripheral captured image of the vehicle 1 and the target object is set in the peripheral captured image of the vehicle 1, the image generating unit 14 generates a viewpoint image in which the rocking level at which the peripheral captured image of the vehicle 1 rocks in response to the behavior of the vehicle 1 is lower than the rocking level at which the interior image rocks in response to the behavior of the vehicle 1. Note that as long as the sway level of the target object is lower than the sway level of objects other than the target object, there is no particular limitation as to whether the target object sways in response to the behavior of the vehicle 1.

In this embodiment, the image generating unit 14 changes the display mode of the object other than the target object in accordance with the behavior of the vehicle 1 in order to make the oscillation level of the target object lower than the oscillation level of the object other than the target object. The change in the display mode of the object other than the target object includes at least one of changing the size of the image, changing the display position of the image, and changing the tilt angle of the image. The image generating unit 14 sets the rate at which the display mode of the object other than the target object is changed to the same rate as the displacement amount of the virtual viewpoint in accordance with the behavior of the vehicle 1. In other words, the image generating unit 14 changes the display mode of the object other than the target object in direct proportion to the displacement amount of the virtual viewpoint. The change in the size of the image, the change in the display position of the image, and the change in the tilt angle of the image will each be described with an example.

An example of changing the size of an image will be described. For example, when the vehicle 1 decelerates and a backward acceleration occurs in the vehicle 1, a forward acceleration occurs in the occupant of the vehicle 1. In this case, the image generating unit 14 enlarges the object other than the target object by a predetermined magnification. As a result, a virtual space in which the object other than the target object is enlarged is generated. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses a state in which the distance between the virtual viewpoint and the object other than the target object has decreased. On the other hand, for example, when the vehicle 1 accelerates and a forward acceleration occurs in the vehicle 1, a backward acceleration occurs in the occupant of the vehicle 1. In this case, the image generating unit 14 reduces the object other than the target object by a predetermined magnification. As a result, a virtual space in which the object other than the target object is reduced is generated. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses a state in which the distance between the virtual viewpoint and the object other than the target object has increased.

An example of changing the display position of an image will be described. For example, when the vehicle 1 accelerates and the vehicle 1 sinks backward, the viewpoint of the occupant of the vehicle 1 is moved upward. In this case, the image generation unit 14 moves objects other than the target object downward by a predetermined distance. As a result, a virtual space is generated in which the display positions of objects other than the target object have moved downward. Then, the image generation unit 14 generates a viewpoint image from the generated virtual space that expresses the state in which objects other than the target object have moved downward as viewed from the virtual viewpoint. On the other hand, for example, when the vehicle 1 decelerates and the vehicle 1 sinks forward, the viewpoint of the occupant of the vehicle 1 is moved downward. In this case, the image generation unit 14 moves objects other than the target object upward by a predetermined distance. As a result, a virtual space is generated in which the display positions of objects other than the target object have moved upward. Then, the image generation unit 14 generates a viewpoint image from the generated virtual space that expresses the state in which objects other than the target object have moved upward as viewed from the virtual viewpoint.

An example of changing the inclination angle of an image will be described. For example, when the vehicle 1 is traveling around a curve in a left direction, if the vehicle 1 is tilted to the left, the vehicle occupant's posture will also be tilted to the left. In this case, the image generating unit 14 tilts everything other than the target object to the left (counterclockwise direction) by a predetermined angle. This generates a virtual space in which everything other than the target object is tilted to the left. The image generating unit 14 then generates a viewpoint image from the generated virtual space that expresses the state in which everything other than the target object is tilted to the left as seen from the virtual viewpoint. On the other hand, for example, when the vehicle 1 is traveling around a curve in a right direction, if the vehicle 1 is tilted to the right, the vehicle occupant's posture will also be tilted to the right. In this case, the image generating unit 14 tilts everything other than the target object to the right (clockwise direction) by a predetermined angle. This generates a virtual space in which everything other than the target object is tilted to the right. The image generating unit 14 then generates a viewpoint image from the generated virtual space that expresses the state in which everything other than the target object is tilted to the right as seen from the virtual viewpoint.

In this way, the image generating unit 14 changes the size, display position, and tilt angle of objects other than the target object in accordance with the behavior of the vehicle 1, thereby generating a viewpoint image in which objects other than the target object sway in accordance with the behavior of the vehicle 1. In other words, the image generating unit 14 executes at least one of a process of changing the size of objects other than the target object, a process of changing the display position of objects other than the target object, a process of changing the tilt angle of objects other than the target object, and a combination of these processes, as a process of shaking objects other than the target object in accordance with the behavior of the vehicle 1 (hereinafter also referred to as a shaking process).

The relationship between the sway level and the display mode of objects other than the target object will now be described. The image generating unit 14 sets the sway level of objects other than the target object based on the acceleration and angular acceleration of the vehicle 1, which are part of the vehicle information of the vehicle 1. For example, the image generating unit 14 sets a larger sway level the faster the acceleration of the vehicle 1 or the faster the angular acceleration of the vehicle 1, and executes a sway process in which objects other than the target object sway more. The image generating unit 14 generates at least one of a viewpoint image in which the sway width of the target object is smaller than the sway width of objects other than the target object, and a viewpoint image in which the speed of the target object is slower than the speed of objects other than the target object.

In the process of changing the size of objects other than the target object, the image generating unit 14 sets a larger enlargement rate of objects other than the target object or a smaller reduction rate of objects other than the target object as the shaking level increases. The image generating unit 14 also sets a shorter time required for objects other than the target object to change to the set enlargement rate or reduction rate as the shaking level increases. As a result, the larger the shaking level of objects other than the target object is, i.e., the greater the degree to which the vehicle 1 is shaking, the faster the size of objects other than the target object changes and the greater the rate at which the size changes.

In addition, in the process of changing the display position of objects other than the target object, the image generation unit 14 sets the distance that objects other than the target object will move to be longer the greater the sway level. Also, the image generation unit 14 sets the time required for objects other than the target object to move to the changed display position to be shorter the greater the sway level. As a result, the greater the sway level of objects other than the target object, i.e., the greater the degree to which the vehicle 1 is swaying, the faster the objects other than the target object will move and the longer their movement distance will be.

In addition, in the process of changing the tilt angle of objects other than the target object, the image generation unit 14 sets a larger angle at which objects other than the target object tilt as the sway level increases. In addition, the image generation unit 14 sets a shorter time required for objects other than the target object to tilt as the sway level increases. As a result, the speed at which objects other than the target object tilt increases and the tilt angle increases as the sway level of objects other than the target object increases, i.e., the greater the degree to which the vehicle 1 is swaying.

The display control unit 15 outputs a control signal for displaying the viewpoint image generated by the image generation unit 14 on a display device 32 provided in the remote space 30. For example, the display control unit 15 converts image data of the viewpoint image into a control signal that can be displayed by the display device 32. The display control unit 15 transmits the converted control signal to the remote communication device 31 via a communication device (not shown) provided in the server.

Next, the image display control flow executed by the image generating device 10 will be described using the flowchart shown in FIG. 3. FIG. 3 is a flowchart showing the image display control executed by the image generating device 10. The image generating device 10 executes the process of the flowchart shown in FIG. 3 at a predetermined cycle.

In step S101, the image generating device 10 acquires vehicle information from a device installed in the vehicle 1. The vehicle information includes an image of the surroundings of the vehicle 1 captured by the on-board camera 21, information on the anti-sway function of the on-board camera 21, acceleration in three axes of the vehicle 1 measured by the acceleration sensor 22a, and angular acceleration in three directions of the vehicle 1 measured by the angular acceleration sensor 22b. In step S101, the image generating device 10 acquires the vehicle information at a predetermined cycle, and ends the processing of step S101 after a predetermined time has elapsed.

In step S102, the image generating device 10 acquires gaze point information of the remote user A from the remote controller 34 via the remote communication device 31. The gaze point information of the remote user A includes the subject that the remote user A is viewing in the display image of the display device 32 and the position of the subject in the display image of the display device 32.

In step S103, the image generating device 10 determines whether the subject gazed at by the remote user A is included in either the interior image or the peripheral captured image of the vehicle 1 from the gaze point information of the remote user A acquired in step S102. When the image generating device 10 determines from the gaze point information of the remote user A that the subject gazed at by the remote user A is an object that appears in the interior image, the image generating device 10 sets the interior image as the target object. On the other hand, when the image generating device 10 determines from the gaze point information of the remote user A that the subject gazed at by the remote user A is an object that appears in the peripheral captured image of the vehicle 1, the image generating device 10 sets the peripheral captured image of the vehicle 1 as the target object. Methods for identifying the remote user A from the gaze point information of the remote user A include, for example, a method of identifying from the type of subject gazed at by the remote user A, and a method of identifying from the position of the gaze point of the remote user A in the display image of the display device 32. If the target object is set to the interior image, proceed to step S104; if the target object is set to the surrounding captured image of vehicle 1, proceed to step S108.

If the target object is set to an interior image in step S103, the process proceeds to step S104. In step S104, the image generating device 10 determines whether or not the shake prevention function of the vehicle-mounted camera 21 is enabled. For example, the image generating device 10 determines whether or not the shake prevention function of the vehicle-mounted camera 21 is enabled based on information related to the shake prevention function of the vehicle-mounted camera 21 among the vehicle information acquired in step S101. If it is determined that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S105, and if it is determined that the shake prevention function of the vehicle-mounted camera 21 is disabled, the process proceeds to step S106.

If it is determined in step S104 that the shake prevention function of the in-vehicle camera 21 is enabled, the process proceeds to step S105. In step S105, the image generating device 10 performs a shaking process on the peripheral image of the vehicle 1, which is other than the target object, because the interior image is set as the target object in step S103. Because it is determined in step S103 that the shake prevention function of the in-vehicle camera 21 is enabled, the peripheral image of the vehicle 1 does not shake or is less likely to shake in response to the shaking of the vehicle 1. For this reason, in this step, the image generating device 10 performs a shaking process on the peripheral image of the vehicle 1, and processes the peripheral image of the vehicle 1 so that it shakes in response to the shaking of the vehicle 1. The shaking process includes at least one of a process of changing the size of the peripheral image of the vehicle 1, a process of changing the display position of the peripheral image of the vehicle 1, and a process of changing the tilt angle of the peripheral image of the vehicle 1.

In step S106, the image generating device 10 sets a virtual viewpoint at a position corresponding to the head of an occupant of the vehicle 1, and generates a virtual space that reproduces the interior space of the vehicle 1 by superimposing an interior image on the peripheral captured image of the vehicle 1 (see FIG. 2(A)). The image generating device 10 then cuts out an image of a portion of the virtual space that is included in a virtual field of view based on the virtual viewpoint, thereby generating a viewpoint image that is an image viewed in a specified direction from the virtual viewpoint within the passenger compartment of the vehicle 1 (see FIG. 2(B)).

When the process proceeds from step S105 to step S106, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 that has been processed in step S105 to shake in accordance with the shaking of the vehicle 1. On the other hand, when the process proceeds from step S104 to step S106, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 captured by the on-board camera 21 itself. This is because the vibration prevention function of the on-board camera 21 is determined to be disabled in step S104, and therefore the captured image of the surroundings of the vehicle 1 captured by the on-board camera 21 shakes in accordance with the shaking of the vehicle 1.

In step S107, the image generating device 10 outputs the viewpoint image generated in step S108 to the display device 32 via the communication device provided in the server and the remote communication device 31. When the processing in step S108 ends, the image generating device 10 ends the processing of the flowchart shown in FIG. 3.

If the target object is set in the peripheral captured image of the vehicle 1 in step S103, the process proceeds to step S108. In step S108, the image generating device 10 determines whether or not the shake prevention function of the vehicle-mounted camera 21 is enabled. Since step S108 corresponds to step S104, the explanation of step S104 is used. If it is determined that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S109, and if it is determined that the shake prevention function of the vehicle-mounted camera 21 is disabled, the process proceeds to step S111.

If it is determined in step S108 that the shake prevention function of the in-vehicle camera 21 is enabled, the process proceeds to step S109. In step S109, the image generating device 10 performs a shaking process on the interior image, which is other than the target object, because the peripheral captured image of the vehicle 1 is set as the target object in step S103. The shaking process includes at least one of a process of changing the size of the interior image of the vehicle 1, a process of changing the display position of the interior image of the vehicle 1, and a process of changing the tilt angle of the interior image of the vehicle 1.

In step S110, the image generating device 10 generates a viewpoint image, which is an image seen in a predetermined direction from a virtual viewpoint in the cabin of the vehicle 1. Step S110 corresponds to step S106, and therefore the explanation of step S106 is used here. When the processing of step S110 ends, the process proceeds to step S107. The explanation of step S107 is the same as that given above.

If it is determined in step S108 that the shake prevention function of the in-vehicle camera 21 is disabled, the process proceeds to step S111. In step S111, the image generating device 10 reduces the shaking of the captured image of the vehicle 1, because the captured image of the vehicle 1 shakes in response to the shaking of the vehicle 1. The image generating device 10 executes a shake reduction process so that the shaking of the captured image of the vehicle 1 is reduced even if the vehicle 1 shakes. As a result, even if the vehicle 1 shakes, the object that the remote user A is gazing at is less likely to shake, so that the possibility of the remote user A becoming sick can be reduced. When the process in step S108 ends, the process proceeds to step S109. The explanations given above are used for the explanations from step S109 onwards. It should be noted that the image generating device 10 can be applied with a shake reduction process known at the time of filing of the present application.

As described above, the image generating device 10 according to the present embodiment includes a vehicle information acquisition unit 11 that acquires vehicle information including information on the behavior of the vehicle 1 from the on-board camera 21 and the on-board sensor 22 provided in the vehicle 1, an image generating unit 14 that generates a viewpoint image, which is an image viewed in a predetermined direction from a virtual viewpoint in the cabin of the vehicle 1, in which the subject sways due to the behavior of the vehicle 1, a display control unit 15 that outputs a control signal for displaying the image generated by the image generating unit 14 on a display device 32 provided in the remote space 30, and a gaze point information acquisition unit 13 that acquires information on the gaze point of the remote user A viewing the display device 32. In the present embodiment, the image generating unit 14 generates a viewpoint image in which the sway level of a target object corresponding to the gaze point of the remote user A among the subjects appearing in the viewpoint image is lower than the sway level of objects other than the target object. As a result, the remote user A who is not riding in the vehicle 1 can easily grasp the behavior of the vehicle 1 through vision without impeding the sense of immersion by viewing a viewpoint image in which objects other than the object he or she is gazing at sway in response to the behavior of the vehicle 1. In addition, the object that remote user A is gazing at is less likely to sway in response to the behavior of vehicle 1 than objects other than that object, which can prevent remote user A from becoming motion sick.

In addition, in this embodiment, the rocking level is the rocking width or speed at which the interior image or the captured image of the vehicle 1 rocks in response to the behavior of the vehicle 1. This allows the remote user A to intuitively grasp how the vehicle 1 is behaving from the objects captured in his or her peripheral vision.

Furthermore, in this embodiment, the image generating unit 14 generates at least one of a viewpoint image in which the oscillation width of the target object is smaller than the oscillation width of objects other than the target object, and a viewpoint image in which the speed of the target object is slower than the speed of objects other than the target object. This makes it possible to prevent objects included in the central visual field of the remote user A from swaying and objects included in the peripheral visual field of the remote user A from swaying, thereby preventing the remote user A from becoming dizzy.

In addition, in this embodiment, the vehicle information includes a peripheral image of the vehicle 1 captured by the onboard camera 21 installed in the vehicle, and the viewpoint image includes an interior image showing the interior of the vehicle as seen from a virtual viewpoint, and a peripheral image of the vehicle 1. The image generation unit 14 sets one of the interior image and the peripheral image of the vehicle 1 as a target object based on the gaze point of the remote user A. As a result, when the remote user A is gazing at the periphery of the vehicle 1, a viewpoint image can be generated in which the vibration level of the peripheral image of the vehicle 1 is lower than the vibration level of the interior image. Also, when the remote user A is inside the vehicle 1, a viewpoint image can be generated in which the vibration level of the interior image is lower than the vibration level of the peripheral image of the vehicle 1.

In addition, in this embodiment, when the image generating unit 14 sets the peripheral captured image of the vehicle 1 as the target object, it generates a viewpoint image in which the shaking level of the peripheral captured image of the vehicle 1 is lower than the shaking level of the interior image. As a result, when the remote user A is gazing at the periphery of the vehicle 1, the remote user A can intuitively grasp the behavior of the vehicle 1 by the shaking of the interior image while suppressing the remote user A from becoming sick.

Furthermore, in this embodiment, when the image generating unit 14 sets an interior image as the target object, it generates a viewpoint image in which the shaking level of the interior image is lower than the shaking level of the captured image of the surroundings of the vehicle 1. As a result, when the remote user A is gazing at the interior space of the vehicle 1, the remote user A can intuitively grasp the behavior of the vehicle 1 by the shaking of the captured image of the surroundings of the vehicle 1 while preventing the remote user A from becoming sick.

In addition, in this embodiment, the vehicle information of the vehicle 1 includes at least one of the acceleration and angular acceleration of the vehicle 1. This makes it possible to generate a viewpoint image in which the subject oscillates in accordance with the behavior of the vehicle 1.

The above-described embodiments are described to facilitate understanding of the present invention, and are not described to limit the present invention. Therefore, each element disclosed in the above embodiments is intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

For example, in the above-described embodiment, an interior image or a captured image of the surroundings of the vehicle 1 is given as an example of the target object, but the method is not limited to alternatively selecting an interior image or a captured image of the surroundings of the vehicle 1 to set the target object, and a predetermined range including the gaze point of the remote user A may be set as the target object. In this case, the image generating device 10 generates a viewpoint image in which the oscillation level of the predetermined range set as the target object is lower than the oscillation level outside the predetermined range. A description will be given of an image generating device 10 according to a modified example in which the image generating unit 14 is different from the above-described embodiment.

The image display control flow executed by the image generating device 10 according to the modified example will be described with reference to FIG. 4. FIG. 4 is a flowchart showing image display control executed by the image generating device 10 according to the modified example. Compared to the flowchart in FIG. 3, the flowchart in FIG. 4 replaces steps S103 to S106 and steps S108 to S110 with steps S201 to S204. Therefore, for the steps explained using the flowchart in FIG. 3, the explanation already given will be used and steps S201 to S204 will be explained.

In step S201, the image generating device 10 determines whether the anti-shake function of the vehicle-mounted camera 21 is enabled. Step S201 corresponds to step S104 and step S108 in FIG. 3, and therefore the explanations of step S104 and step S108 are used. If it is determined that the anti-shake function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S202, and if it is determined that the anti-shake function of the vehicle-mounted camera 21 is disabled, the process proceeds to step S204.

If it is determined in step S201 that the shake prevention function of the in-vehicle camera 21 is enabled, the process proceeds to step S202. In step S202, the image generating device 10 performs a shaking process on the outside of a predetermined range including the gaze point of the remote user A. For example, the image generating device 10 identifies the position of the subject that the remote user A is gazing at in the display image of the display device 32 from the gaze point information of the remote user A acquired in step S102. The image generating device 10 sets a predetermined range including the subject as the target object. The predetermined range may be a range including the subject that the remote user A is gazing at, and its shape, size, etc. are not particularly limited. For example, the predetermined range is stored in advance in the storage unit 12 as a range indicating the central visual field of the remote user A.

When the image generating device 10 sets a predetermined range including the gaze point of the remote user A as the target object, it executes a rocking process on the outside of the target object. For example, when the range of the central visual field of the remote user A is set as the target object, the image generating device 10 executes a rocking process on the object included in the peripheral visual field of the remote user A. In a modified example, the object of the rocking process is either an interior image, a captured image of the surroundings of the vehicle 1, or both an interior image and a captured image of the surroundings of the vehicle 1. The rocking process in step S201 is similar to the rocking process in steps S105 and S109 shown in FIG. 3. Therefore, the description of the rocking process will be based on the description of these steps.

In step S203, the image generating device 10 sets a virtual viewpoint at a position corresponding to the head of an occupant of the vehicle 1, and generates a virtual space that reproduces the interior space of the vehicle 1 by superimposing an interior image on the peripheral captured image of the vehicle 1. Then, the image generating device 10 generates a viewpoint image from the virtual space, which is an image viewed in a specified direction from the virtual viewpoint in the passenger compartment of the vehicle 1.

When the process proceeds from step S201 to step S204 and then to step S203, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 that has been processed in step S204 to shake in response to the shaking of the vehicle 1. On the other hand, when the process proceeds from step S201 to step S202 and then to step S203, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 itself captured by the on-board camera 21. This is because the vibration prevention function of the on-board camera 21 is determined to be disabled in step S201, and therefore the captured image of the surroundings of the vehicle 1 captured by the on-board camera 21 will shake in response to the shaking of the vehicle 1.

When the processing of step S203 is completed, the process proceeds to step S107, and the viewpoint image generated in step S203 is output to the display device 32.

If it is determined in step S201 that the shake prevention function of the in-vehicle camera 21 is enabled, the process proceeds to step S204. In step S204, the image generating device 10 reduces the shaking of the captured image of the periphery of the vehicle 1, because the captured image of the periphery of the vehicle 1 shakes in response to the shaking of the vehicle 1. Since step S204 corresponds to step S111 in FIG. 3, the explanation of step S111 is used here. When the process in step S204 ends, the process proceeds to step S202. The explanations given above are used for the explanations of steps S202 and onward.

As described above, the image generating unit 14 included in the image generating device 10 according to the modified example sets a predetermined range of the viewpoint image including the user's gaze point as a target object. This makes it possible to generate a viewpoint image in which the sway level of a subject included in the central visual field of the remote user A is lower than the sway level of a subject included in the peripheral visual field of the remote user A, for example. Since a viewpoint image that matches the actual central visual field and peripheral visual field of the remote user A can be generated, the remote user A can easily grasp the behavior of the vehicle 1 through vision without being hindered in the sense of immersion. Furthermore, objects included in the central visual field of the remote user A are less likely to sway in response to the behavior of the vehicle 1 compared to other objects, which makes it possible to prevent the remote user A from becoming sick.

The rocking process performed by the image generating device 10 according to the modified example may, for example, not set the rocking level uniformly for objects other than the target object, but may increase the rocking level the farther the object is from the gaze point of the remote user A. This makes it easier for the remote user A to visually recognize objects included in the central field of vision, since the closer the object is to the central field of vision, the less likely it is to rock in response to the behavior of the vehicle 1. Also, the farther the object is from the central field of vision, the more likely it is to rock in response to the behavior of the vehicle 1, making it easier for the remote user A to intuitively grasp the behavior of the vehicle 1 in his peripheral vision.

The oscillation process performed by the image generating device 10 according to the modified example may be, for example, a process in which the oscillation level is gradually changed over time by a morphing process so that no inflection point occurs in the difference in the oscillation level near the boundary between a predetermined range set as the target object and the range in question. Since the oscillation level changes continuously at the boundary between the central field of vision and the peripheral field of vision of the remote user A, it is possible to prevent the remote user A from feeling uncomfortable, for example, even if the vehicle 1 suddenly sways significantly.

In the above embodiment, the screen display portion of a head-mounted display worn by remote user A was used as an example of display device 32. In this case, the virtual viewpoint is not a preset viewpoint but the viewpoint of remote user A, and may be a viewpoint that moves in accordance with the movement of remote user A's head. In addition to the behavior of vehicle 1, image generating device 10 generates an interior image that matches the movement of remote user A's head, making it easier for remote user A to grasp the behavior of vehicle 1 even on a device that can be worn by remote user A. Note that FIG. 5 is an example of a virtual space when display device 32 is incorporated into a head-mounted display.

In the above embodiment, an image viewed from a virtual viewpoint toward the front of the vehicle 1 is described as a viewpoint image, but the direction viewed from the virtual viewpoint is not limited to the front of the vehicle 1. For example, an image viewed from the virtual viewpoint toward the left or right side of the vehicle 1 may be used as a viewpoint image.

In the above embodiment, an example was described in which an interior image of the vehicle 1 is stored in the storage unit 12, but the interior image of the vehicle 1 may be an image captured by a camera mounted on the vehicle 1 that captures an image of the interior of the vehicle 1.

In addition, in the above-described embodiment, the processes of steps S104 and S108 in FIG. 3 do not necessarily need to be executed. For example, the image generating device 10 may execute these processes only when outputting a viewpoint image to the display device 32 for the first time to remote user A.

In the above embodiment, the remote controller 34 executes the process of identifying the gaze point of the remote user A, but the present invention is not limited to this. Instead of the remote controller 34, the image generating device 10 may include a gaze point identification unit for identifying the gaze point of the remote user A. In this case, the image generating device 10 acquires an image captured by the remote camera 33, and identifies the gaze point of the remote user A based on the captured image.

REFERENCE SIGNS LIST 100: IMAGE DISPLAY SYSTEM 1: VEHICLE 21: ON-VEHICLE CAMERA 22: ON-VEHICLE SENSOR 22a: ACCELERATION SENSOR 22b: ANGULAR ACCELERATION SENSOR 23: ON-VEHICLE COMMUNICATION DEVICE 10: IMAGE GENERATION DEVICE 11: VEHICLE INFORMATION ACQUISITION UNIT 12: MEMORY UNIT 13: POINT OF GAZETTE INFORMATION ACQUISITION UNIT 14: IMAGE GENERATION DEVICE 15: DISPLAY CONTROL UNIT 31: REMOTE COMMUNICATION DEVICE 32: DISPLAY DEVICE 33: REMOTE CAMERA 34: REMOTE CONTROLLER

Claims (8)

A vehicle information acquisition unit that acquires vehicle information including information on a behavior of the vehicle from a device provided in the vehicle;
an image generating unit that generates, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in a cabin of the vehicle, in which a subject sways due to the behavior of the subject;
a display control unit that outputs a control signal for displaying the image generated by the image generation unit on a display device provided in a space different from the vehicle interior;
a gaze point information acquisition unit that acquires information regarding a gaze point of a user viewing the display device,
The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle,
the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image,
The image generating unit includes:
When the vehicle surroundings image is set to a target object among the subjects that corresponds to the user's gaze point based on the user's gaze point, an image generating device generates a viewpoint image in which the vibration level of the vehicle surroundings image is lower than the vibration level of the interior image . A vehicle information acquisition unit that acquires vehicle information including information on a behavior of the vehicle from a device provided in the vehicle;
an image generating unit that generates, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in a cabin of the vehicle, in which a subject sways due to the behavior of the subject;
a display control unit that outputs a control signal for displaying the image generated by the image generation unit on a display device provided in a space different from the vehicle interior;
a gaze point information acquisition unit that acquires information regarding a gaze point of a user viewing the display device,
The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle,
the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image,
The image generating unit includes:
When the interior image is set to a target object among the subjects that corresponds to the user's gaze point based on the user's gaze point, an image generating device generates a viewpoint image in which the shaking level of the interior image is lower than the shaking level of the vehicle surroundings image. 3. The image generating device according to claim 1,
The sway level is a sway width or speed at which the subject sways in response to the behavior of the image generating device. 4. The image generating device according to claim 3 ,
The image generation unit is an image generating device that generates at least one of the viewpoint images in which the oscillation range of the target object is smaller than the oscillation range of objects other than the target object, and the viewpoint image in which the speed of the target object is slower than the speed of objects other than the target object. The image generating device according to any one of claims 1 to 4 ,
The image generating unit is an image generating device that sets a predetermined range of the viewpoint image that includes the user's gaze point as the target object. The image generating device according to any one of claims 1 to 5 ,
The vehicle information includes at least one of an acceleration and an angular acceleration of the vehicle. 1. An image generating method executed by a processor to generate an image to be displayed on a display device provided in a space other than a vehicle interior, comprising:
Acquiring vehicle information including information on a behavior of the vehicle from a device provided in the vehicle;
generating, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in the interior of the vehicle, in which the subject sways due to the behavior of the subject;
Acquire information regarding a gaze point of a user viewing the display device;
a viewpoint image generating step of generating the viewpoint image in which a rocking level of a target object corresponding to a gaze point of the user among the subjects is made lower than the rocking level of an object other than the target object,
The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle,
the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image,
The viewpoint image generating step includes:
An image generating method including a step of lowering a vibration level of the vehicle surroundings image compared to the vibration level of the interior image when the vehicle surroundings image is set as the target object based on the user's gaze point . 1. An image generating method executed by a processor to generate an image to be displayed on a display device provided in a space other than a vehicle interior, comprising:
Acquiring vehicle information including information on a behavior of the vehicle from a device provided in the vehicle;
generating, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in the interior of the vehicle, in which the subject sways due to the behavior of the subject;
Acquire information regarding a gaze point of a user viewing the display device;
a viewpoint image generating step of generating the viewpoint image in which a rocking level of a target object corresponding to a gaze point of the user among the subjects is made lower than the rocking level of an object other than the target object,
The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle,
the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image,
The viewpoint image generating step includes:
An image generating method including a step of lowering the sway level of the interior image compared to the sway level of the vehicle surroundings image when the interior image is set as the target object based on the user's gaze point.

Images