JP7743701B2 - Image output method - Google Patents

Description

The present invention relates to an image output method for outputting a virtual space image viewed from a virtual viewpoint within a virtual space.

A device that assists with interior design has been proposed (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2001-142924

In the known technology described in Patent Document 1, after selecting a theme for the room, it is necessary to select the color and pattern of the interior materials and fixtures. Furthermore, the output image is a two-dimensional image, which lacks a sense of realism.

The present invention was made in light of these circumstances. Its purpose is to provide an image output method that allows the user to view the results of the interior design after selecting a theme. It also provides an image output method that can provide realistic three-dimensional images.

An image output method according to one aspect of the present application is characterized in that a computer performs the following processes: outputs multiple classifications related to residential interiors, accepts the selection of a classification, generates a virtual space that simulates a living space composed of interior components corresponding to the selected classification, acquires a virtual space image viewed from a virtual viewpoint within the generated virtual space, and outputs the acquired virtual space image.

In one aspect of the present application, after selecting a category (theme), interior design results can be viewed.

FIG. 1 is an explanatory diagram showing an example of the configuration of a VR system. FIG. 10 is an explanatory diagram illustrating an example of a theme DB. FIG. 10 is an explanatory diagram illustrating an example of a recommended combination DB. FIG. 10 is an explanatory diagram illustrating an example of a texture DB. FIG. 2 is an explanatory diagram illustrating an example of a room DB. FIG. 10 is an explanatory diagram illustrating an example of a vertex DB. FIG. 10 is an explanatory diagram illustrating an example of an edge DB. FIG. 10 is an explanatory diagram illustrating an example of a surface DB. FIG. 2 is an explanatory diagram illustrating an example of an object DB. 10 is a flowchart illustrating an example of a procedure of a main process. 10 is a flowchart illustrating an example of a procedure for generating a walk-through screen. 10 is a flowchart illustrating an example of a procedure for screen processing. 10 is a flowchart illustrating an example of a procedure for color selection processing. 10 is a flowchart illustrating an example of a procedure for environment switching processing. 10 is a flowchart illustrating an example of a procedure for viewpoint height change processing. 10 is a flowchart illustrating an example of a procedure for viewpoint switching processing. FIG. 10 is an explanatory diagram showing an example of a room selection screen. FIG. 10 is an explanatory diagram showing an example of a walk-through screen. FIG. 10 is an explanatory diagram showing an example of a color selection menu screen. FIG. 10 is an explanatory diagram showing an example of an environment switching screen. FIG. 10 is an explanatory diagram showing an example of a viewpoint height change screen. FIG. 10 is an explanatory diagram showing an example of a layout screen. FIG. 10 is an explanatory diagram showing an example of a building material selection screen. FIG. 10 is an explanatory diagram showing an example of an information screen. FIG. 10 is an explanatory diagram showing an example of an information screen. 10A and 10B are explanatory diagrams showing the state of opening and closing the storage. FIG. 10 is an explanatory diagram showing another example configuration of a VR system. FIG. 2 is a block diagram illustrating an example of a hardware configuration of a management server. FIG. 2 is a block diagram illustrating an example of the hardware configuration of a user terminal. FIG. 2 is an explanatory diagram illustrating an example of a user DB. FIG. 10 is an explanatory diagram illustrating an example of a case DB. FIG. 2 is an explanatory diagram illustrating an example of a history DB. 10 is a flowchart illustrating an example of a procedure for a review process. 10 is a flowchart illustrating an example of a procedure for a reference process. FIG. 10 is an explanatory diagram showing another example of the walk-through screen. FIG. 10 is an explanatory diagram illustrating an example of a training data DB. FIG. 10 is an explanatory diagram illustrating an example of a learning model. 10 is a flowchart illustrating an example of a procedure for a learning model generation process. 10 is a flowchart illustrating an example of a procedure for theme selection processing. FIG. 2 is a block diagram illustrating an example of functional units included in the image processing apparatus. 10 is a flowchart illustrating an example of a procedure for a proximity display process. FIG. 10 is an explanatory diagram showing an example of proximity display. FIG. 10 is an explanatory diagram showing another example of a history DB. 10 is a flowchart illustrating an example of a procedure for image display processing. FIG. 10 is an explanatory diagram illustrating an example of a bookmark DB. FIG. 10 is an explanatory diagram illustrating an example of a persona DB. 10 is a flowchart illustrating an example of a procedure for a recommendation mark assignment process. FIG. 10 is an explanatory diagram illustrating an example of a recommended furniture DB. 10 is a flowchart illustrating an example of a procedure for related information processing. 10 is a flowchart illustrating an example of a procedure for creating a moving image. FIG. 10 is an explanatory diagram showing an example of a travel route. FIG. 10 is an explanatory diagram illustrating an example of a template DB. 10 is a flowchart illustrating an example of a procedure for creating a presentation board. FIG. 10 is an explanatory diagram showing an example of a template. FIG. 10 is an explanatory diagram showing another example configuration of a VR system.

(Embodiment 1)
An embodiment will be described below with reference to the drawings. FIG. 1 is an explanatory diagram showing an example of the configuration of a VR system. VR is an abbreviation for Virtual Reality. The VR system 100 includes an image processing device 1, a controller 2, and a display 3. The image processing device 1 is configured as a desktop personal computer (PC), a notebook PC, a tablet computer, or the like. To display video images as realistic as possible, the image processing device 1 may be configured as a computer equipped with a graphics processing means with high 3D graphics processing capabilities, a so-called gaming PC. The image processing device 1 may also be configured as a multi-computer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer.

The controller 2 is, for example, a VR controller. The user uses the controller 2 to give instructions to the image processing device 1. The display 3 is a display device including a liquid crystal display panel or the like. It is desirable that the display 3 be a high-definition display such as a 4K display or an 8K display. The display 3 may also be a head-mounted display or VR goggles.

The image processing device 1 includes a control unit 11, a main memory unit 12, an auxiliary memory unit 13, an input I/F 15, a display I/F 16, a communication unit 17, and a reading unit 18. The control unit 11, the main memory unit 12, the auxiliary memory unit 13, the input I/F 15, the display I/F 16, the communication unit 17, and the reading unit 18 are connected by a bus B.

The control unit 11 has one or more arithmetic processing devices such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), etc. The control unit 11 realizes various functional units by reading and executing the control program 1P stored in the auxiliary storage unit 13.

The main memory unit 12 is a static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, etc. The main memory unit 12 mainly temporarily stores data required by the control unit 11 to perform arithmetic processing.

The auxiliary memory unit 13 is a hard disk or SSD (Solid State Drive), etc., and stores the control program 1P and various DBs (Databases) required for the control unit 11 to execute processing. The auxiliary memory unit 13 stores a theme DB 131, a recommended combination DB 132, a texture DB 133, a room DB 134, a vertex DB 135, an edge DB 136, a face DB 137, and an object DB 138. The auxiliary memory unit 13 may be an external storage device connected to the image processing device 1.

The input I/F 15 is a communication port that complies with communication standards such as PS/2 and Bluetooth (registered trademark). The input I/F 15 communicates with the controller 2. The input I/F 15 may also communicate with a keyboard or mouse (not shown).

The display I/F 16 is a display output port that complies with communication standards such as HDMI (registered trademark) (High-Definition Multimedia Interface), DisplayPort, and Thunderbolt. The display I/F 16 outputs an image signal to the display 3.

The communication unit 17 communicates with other computers via the network. The control unit 11 may use the communication unit 17 to download the control program 1P from another computer via the network N and store it in the auxiliary storage unit 13.

The reading unit 18 reads portable storage media 1a, including CD (Compact Disc)-ROM and DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 18 and store it in the auxiliary storage unit 13. The control unit 11 may also read the control program 1P from the semiconductor memory 1b.

Part of the processing performed by the image processing device 1, such as rendering, may be performed by a server computer with higher performance capabilities. The image processing device 1 receives the image created by rendering from the server computer and displays it on the display 3.

In the following description, "interior" includes interior decoration and furnishings. "Interior components" literally refer to the individual elements that make up an interior, such as the basic elements that make up a room, such as flooring, wall materials, and ceiling materials, fixtures and fittings such as doors and windows, decorative materials, furniture, system kitchens, curtains, and lighting fixtures. "Clothing materials" refer to interior finishing materials such as wall materials and ceiling materials, and finishing materials for built-in furniture. "Fitment furniture" refers to built-in furniture that is made to match the interior. In this embodiment, "decorative materials" refers to flooring and wallpaper materials.

Next, we will explain the database (DB) used in the VR system 100. Figure 2 is an explanatory diagram showing an example of a theme DB. The theme DB 131 stores information about interior themes. Theme is a word that succinctly expresses the idea used to create a sense of unity in the interior design. Theme can also be expressed as style or taste. The theme DB 131 includes a theme ID column, a theme name column, a summary column, and a description column. The theme ID column stores a theme ID that uniquely identifies a theme. The theme name column stores the theme name. The summary column stores a summary description of the theme. The description column stores a detailed description of the theme.

Figure 3 is an explanatory diagram showing an example of a recommended combination DB. The recommended combination DB 132 stores recommended combinations of flooring materials and cloth materials for each theme. The recommended combination DB 132 stores a theme ID column, a flooring material column, and a cloth material column. The theme ID column stores theme IDs. The flooring material column and cloth material column store the IDs of the flooring materials and cloth materials recommended for combination, for example, product models, respectively.

Figure 4 is an explanatory diagram showing an example of a texture DB. Texture DB 133 stores texture images. A texture is a repeating pattern applied to flooring or wallpaper materials. A texture image is an image of the repeating texture pattern. Texture DB 133 includes a texture ID column, a model column, a group column, an image column, and an application column. The texture ID column stores a texture ID that can uniquely identify a texture. The model column stores the product model of the flooring or wallpaper material corresponding to the texture. The group column stores the group to which the texture belongs. The image column stores texture images. The image column may store the file name of the texture image, and the file may be stored in the auxiliary memory unit 13. The application column stores the target to which the texture is applied. Note that texture images may include not only the patterns of flooring or wallpaper materials, but also general textures in computer graphics, i.e., images that represent the surface patterns of objects.

Figure 5 is an explanatory diagram showing an example of a room DB. Room DB 134 stores information about rooms represented as virtual three-dimensional spaces. Room DB 134 includes a room ID column, a name column, a theme column, a floor column, a wall column, a ceiling column, a fixture column, and a virtual camera column. The room ID column stores a room ID that can uniquely identify a room. The name column stores the name of the room. Examples of names include "A type," "B type," and "C type." The theme column stores the theme ID of the design theme applied at the time of initial display. The floor column stores information about the floors that make up the room. The floor information includes, for example, coordinate values indicating the position in virtual three-dimensional space of the surface representing the floor, and the texture ID of the texture to be applied as the floor material. The wall column stores information about the walls that make up the room. The wall information includes, for example, coordinate values indicating the position in virtual three-dimensional space of the surface representing the wall, and the texture ID of the texture to be applied as the wall material. The ceiling column stores information about the ceilings that make up a room. Ceiling information includes, for example, coordinate values that indicate the position in virtual three-dimensional space of the surface that represents the ceiling, and the texture ID of the texture to be applied as a finishing material. The ceiling column also stores information about the surfaces used to represent a dropped ceiling. The fixture column stores identification information for the fixtures that make up a room. The virtual camera column stores the position, shooting direction, etc. of a virtual camera that is set in advance in the virtual three-dimensional space. An image of the virtual three-dimensional space captured by the virtual camera becomes the output image from the image processing device 1. The virtual camera corresponds to the user's virtual viewpoint.

Figure 6 is an explanatory diagram showing an example of a vertex DB. The vertex DB 135 stores vertices that represent a virtual three-dimensional space. The vertex DB 135 includes a vertex ID string, an X coordinate string, a Y coordinate string, and a Z coordinate string. The vertex ID string stores a vertex ID that can uniquely identify a vertex. The X coordinate string stores the X coordinate of the vertex. The Y coordinate string stores the Y coordinate of the vertex. The Z coordinate string stores the Z coordinate of the vertex.

Figure 7 is an explanatory diagram showing an example of an edge DB. Edge DB 136 stores edges that represent a virtual three-dimensional space. Edge DB 136 includes an edge ID column, a start column, and an end column. The edge ID column stores an edge ID that can uniquely identify an edge. The start column stores the vertex ID of the vertex that indicates the start point of the edge. The end column stores the vertex ID of the vertex that indicates the end point of the edge. Edges are directed edges with a set direction.

Figure 8 is an explanatory diagram showing an example of a face DB. Face DB 137 stores faces that represent virtual three-dimensional space. Face DB 137 includes a face ID column, an edge 1 column, an edge 2 column, an edge 3 column, an edge 4 column, an edge 5 column, and an edge 6 column. The face ID column stores a face ID that can uniquely identify a face. Edge 1 column, edge 2 column, edge 3 column, edge 4 column, edge 5 column, and edge 6 column each store the edge IDs of the edges that make up the face. Since a face can be made up of a minimum of three edges, edges 4 to 6 may be NULL, indicating no values. Since edges have a set direction, tracing the edges that make up a face will form a complete perimeter. In other words, the vertices that make up a face are components of two edges, and are the start point of one edge and the end point of the other edge. The direction of the perimeter determines whether the face is front or back. When looking at a face, the perimeter that moves counterclockwise is the front, and the perimeter that moves clockwise is the back.

Figure 9 is an explanatory diagram showing an example of an object DB. Object DB 138 stores information about objects placed in a room. Objects include fixtures such as doors and windows. Object DB 138 includes an object ID column, a type column, a name column, a definition column, a cloth material column, a behavior column, and an icon column. The object ID column stores an object ID that can uniquely identify an object. The type column stores the type of object. Examples of types include fixtures, furniture, built-in furniture, or other. The name column stores the name of the object. The definition column stores the definition of the three-dimensional shape. For example, it stores the surface IDs of multiple surfaces that represent the object. The cloth material column stores information about surfaces to which cloth material is applied. The definition column includes information about the texture to be applied to surfaces that are not the target of cloth material application. The behavior column stores methods (functions) for changing the state of an object. For example, open() is a method for opening a door or a closet door. close() is a method for closing a door or a closet door. The icon column stores information about the icon used to operate an object. The icon can be, for example, a change icon (selection icon) or an information icon. The change icon is an icon that opens a screen for changing the texture of the object. The information icon is an icon that opens an information screen about the object. If no icon is assigned to the object, the icon column stores "none."

As shown above, when defining a virtual 3D space, first a vertex is defined. An edge is defined from two vertices. A face is defined from multiple edges. An object is defined from multiple faces. This is an example of a definition method known as a surface model or surface modeling. Definition methods are not limited to this. A surface model with a triangle as the smallest element may also be used, or a wireframe model, solid model, etc. may also be used.

Next, we will discuss information processing performed by the image processing device 1. Figure 10 is a flowchart showing an example of the procedure for main processing. Main processing is the main processing performed by the image processing device 1. The control unit 11 of the image processing device 1 displays a top screen on the display 3 (step S1). The control unit 11 displays a room select screen on the display 3 (step S2). The user selects a room on the room select screen. Each room has an interior design with a different theme. The control unit 11 generates a walk-through screen corresponding to the room selected by the user (step S3). When the user performs an operation on the walk-through screen, the control unit 11 determines whether the operation is a menu selection (step S4). If the control unit 11 determines that the operation is not a menu selection (NO in step S4), it executes processing on the walk-through screen (screen processing) (step S5) and returns the processing to step S4. If the control unit 11 determines that the operation is a menu selection (YES in step S4), it determines whether room select was selected (step S6). If the control unit 11 determines that Room Select has been selected (YES in step S6), it returns the process to step S2. If the control unit 11 determines that Room Select has not been selected (NO in step S6), it determines whether the top screen has been selected (step S7). If the control unit 11 determines that the top screen has been selected (YES in step S7), it returns the process to step S1. If the control unit 11 determines that the top screen has not been selected (NO in step S7), it determines whether a change in display settings has been selected (step S8). If the control unit 11 determines that a change in display settings has been selected (YES in step S8), it updates the screen in accordance with the change in settings (step S9) and returns the process to step S4. If the control unit 11 determines that a change in display settings has not been selected (NO in step S8), it determines whether End has been selected (step S10). If the control unit 11 determines that End has not been selected (NO in step S10), it displays a screen corresponding to the selected menu (step S11) and returns the process to step S4. If the control unit 11 determines that "End" has been selected (YES in step S10), it ends the main processing.

Figure 11 is a flowchart showing an example of the procedure for the walk-through screen generation process. The walk-through screen generation process corresponds to step S3 in Figure 10. The control unit 11 of the image processing device 1 obtains information about the room selected by the user on the room selection screen from the room DB 134 (step S21). The control unit 11 obtains recommended combination information from the recommended combination DB 132 using the theme ID included in the room information (step S22). The control unit 11 obtains flooring textures and cloth material textures from the texture DB 133 based on the recommended combination information (step S23). The control unit 11 obtains information about objects included in the virtual three-dimensional space based on the contents of the floor row, wall row, ceiling row, and fixture row in the room DB 134 (step S24). The control unit 11 generates a virtual three-dimensional space based on the object information (step S25). At this time, texture images obtained from the texture DB 133 are pasted onto the floor, walls, closet, etc. The control unit 11 acquires images of the virtual three-dimensional space based on the contents of the virtual camera sequence in the room DB 134 (step S26). The control unit 11 generates a walk-through screen that includes the acquired images (step S27). The control unit 11 outputs the generated walk-through screen (step S28) and returns the process to the caller.

Figure 12 is a flowchart showing an example of the screen processing procedure. The screen processing corresponds to step S5 in Figure 10. The control unit 11 of the image processing device 1 detects an object located at a position specified by the user on the walk-through screen (step S41). The control unit 11 determines whether the detected object is an icon (step S42). If the control unit 11 determines that the detected object is an icon (YES in step S42), it determines whether the icon is a change icon (step S43). If the control unit 11 determines that the icon is a change icon (YES in step S43), it displays a selection screen for changing the flooring material or wallpaper material (step S44). If the object is a floor surface, the control unit 11 displays a flooring material selection screen. If the object is a wall, closet, etc., the control unit 11 displays a wallpaper material selection screen. The user selects a flooring material or wallpaper material, or selects "Close." The control unit 11 determines whether the user's operation is an instruction to change the flooring material or floor material (step S45). If the control unit 11 determines that the instruction is to change the flooring or floor material (YES in step S45), it changes the texture of the floor, wall, etc. in accordance with the instruction (step S46). The control unit 11 acquires a virtual camera image after the change (step S47). The control unit 11 updates the walk-through screen based on the acquired image (step S48). The control unit 11 outputs the updated walk-through screen (step S49) and returns the process to the caller. If the control unit 11 determines that the icon is not an instruction to change the flooring or floor material (NO in step S45), it returns the process to the caller. If the control unit 11 determines that the icon is not a change icon (NO in step S43), it acquires information for displaying an information screen (step S50). For example, if an information icon attached to furniture is selected, it acquires information about the furniture. The control unit 11 generates a screen displaying the acquired information (step S51). The control unit 11 outputs the generated screen (step S52) and returns the process to the caller. If the control unit 11 determines that the detected object is not an icon (NO in step S42), it determines whether or not a viewpoint change has occurred (step S53). If the control unit 11 determines that a viewpoint change has occurred (YES in step S53), it performs a viewpoint change (step S54). The control unit 11 switches the virtual camera or changes the position of the virtual camera. Based on the changed virtual camera, the control unit 11 executes step S47 and subsequent steps. If the control unit 11 determines that a viewpoint change has occurred (NO in step S53), it returns the process to the caller. The viewpoint change in step S54 allows the user to freely change the viewpoint using the controller 2. The user feels as if they are walking around freely in a room represented as a virtual three-dimensional space.

Figure 13 is a flowchart showing an example of the color selection process. The color selection process is one of the processes corresponding to the screen update (step S9) in Figure 10. The control unit 11 creates a menu screen (step S61). The menu screen displays multiple recommended combinations of flooring materials and wallpaper materials and is created based on the recommended combination DB 132. It is desirable to display only combinations that correspond to the theme. The control unit 11 sets the display of the selection status on the menu screen so that the user can see the currently selected combination (step S62). The control unit 11 displays the generated menu screen (step S63). The user selects the changed combination on the menu screen or selects "Back." The control unit 11 determines whether the user has selected another combination (step S64). If the control unit 11 determines that the user has selected another combination (YES in step S64), it changes the floor and wall textures according to the selected combination (step S65). The control unit 11 acquires an image after the texture change (step S66). The control unit 11 updates the screen based on the acquired image (step S67). The control unit 11 outputs the updated screen (step S68) and returns the process to step S64. If the control unit 11 determines that the user has not selected another combination (NO in step S64), it clears the menu screen (step S69) and ends the process.

Figure 14 is a flowchart showing an example of the procedure for the environment switching process. The environment switching process is one of the processes corresponding to the screen update (step S9) in Figure 10. The control unit 11 displays a menu (step S81). Examples of menu options include "Day/Night Switch," "Open/Close Storage," and "Display Furniture." The user changes one of the menu options or selects "Back." The control unit 11 determines whether the day/night switch has been operated (step S82). If the control unit 11 determines that the day/night switch has been operated (YES in step S82), it changes the light source (virtual light source) (step S83). The day/night switch switches the time zone in the virtual three-dimensional space between daytime and nighttime. Daytime is assumed to be a sunny day. During daytime, the sun is used as the light source, and the lighting is expressed as if sunlight is pouring in through a window. During nighttime, the lighting is expressed as if the light source is a light source installed on the ceiling or wall. The control unit 11 acquires an image after the light source has been changed (step S84). The control unit 11 updates the screen based on the acquired image (step S85). Note that the day/night switching may also switch the view seen through the window. The control unit 11 outputs the updated screen (step S86) and returns the process to step S82. If the control unit 11 determines that the day/night switching has not been operated (NO in step S82), it determines whether the storage open/close operation has been performed (step S87). If the control unit 11 determines that the storage open/close operation has been performed (YES in step S87), it changes the open/closed state of a closet or other door (step S88). The control unit 11 changes the state by executing a method stored in the behavior column of the object DB 138. In addition to closets, desk and dresser drawers may also be opened and closed. The open state may have multiple stages, such as a 25% open state, a 50% open state, and a 100% open state. The control unit 11 executes step S84 and subsequent steps. If the control unit 11 determines that the storage open/close operation has not been performed (NO in step S87), it determines whether the furniture display has been operated (step S89). If the control unit 11 determines that the furniture display has been operated (YES in step S89), it changes the display or non-display of the furniture (step S90). The control unit 11 executes step S84 and subsequent steps. If the control unit 11 determines that the furniture display has not been operated (NO in step S89), it erases the menu (step S91) and ends the process.

Figure 15 is a flowchart showing an example of the procedure for changing the viewpoint height. The viewpoint height change process is one of the processes corresponding to the screen update (step S9) in Figure 10. The viewpoint height change process changes only the viewpoint height without changing the position in a plane parallel to the floor. Setting the viewpoint height to a value that takes the user's height into consideration makes it possible to output a more realistic image. Setting a low viewpoint height also makes it possible to view the room from a child's perspective. The control unit 11 displays a height change menu (step S101). The menu displays multiple different heights, with the current setting selected. The user can change the height or select "Back." The control unit 11 determines whether the height has been changed (step S102). If the control unit 11 determines that the height has been changed (YES in step S102), it changes the viewpoint (step S103). The control unit 11 changes the height of the virtual camera, which corresponds to the viewpoint. The control unit 11 acquires the changed image (step S104). The control unit 11 updates the screen based on the acquired image (step S105). The control unit 11 outputs the updated screen (step S106) and returns the process to step S102. If the control unit 11 determines that the height has not changed (NO in step S102), it erases the menu (step S107) and ends the process.

Figure 16 is a flowchart showing an example of the procedure for viewpoint switching processing. The viewpoint switching processing is one of the processes corresponding to the screen update (step S9) in Figure 10. The viewpoint switching processing is a process that changes the position in a plane parallel to the floor without changing the height of the viewpoint. The control unit 11 of the image processing device 1 displays a layout screen (step S121). The layout screen is a plan view showing the shape of the room and the positions of fixtures and fittings within the room. The layout screen shows the positions of multiple virtual cameras that can be specified as viewpoints using icons. The display mode of the currently selected virtual camera is different from the display mode of the other virtual cameras. The user changes the virtual camera or selects "Close." The control unit 11 determines whether the selection state has changed (step S122). If the control unit 11 determines that the selection state has changed (YES in step S122), it changes the viewpoint (step S123). The control unit 11 changes the virtual camera corresponding to the viewpoint. The control unit 11 acquires the changed image (step S124). The control unit 11 updates the screen based on the acquired image (step S125). The control unit 11 outputs the updated screen (step S126) and returns the process to step S122. If the control unit 11 determines that the selection state has not changed (NO in step S122), it erases the layout screen (step S127) and ends the process.

Next, examples of screens used in the above process are shown. Figure 17 is an explanatory diagram showing an example of a room select screen. The room select screen d01 is a screen for selecting a room to display. The room select screen d01 includes a top button d011 and multiple room select buttons d012. Selecting the top button d011 transitions to the top screen (not shown). Selecting the room select button d012 transitions to a walk-through screen displaying the selected room. In the example shown in Figure 17, each room is designed with a different theme, and the room select buttons d012 are labeled with the corresponding theme name (Calm, Flow, Advanced, Wastory, Tasty). The displayed theme is generated by the control unit 11 based on theme information stored in the theme DB 131. Note that when setting for a specific apartment, the room select button d012 may be labeled with a name indicating the room type, such as A-type or B-type.

Figure 18 is an explanatory diagram showing an example of a walk-through screen. The walk-through screen d02 includes a top button d021, a room select button d022, a color select button d023, an environment switch button d024, a viewpoint height switch button d025, a VR mode button d026, a print button d027, a floor plan button d028, a close button d029, a settings display d02A, an information icon d02B, and a change icon d02C. Selecting the top button d021 transitions to the top screen. Selecting the room select button d022 returns to the room select screen d01. Selecting the color select button d023 launches a color select process that allows you to change the flooring and wallpaper materials all at once. Selecting the environment switch button d024 launches an environment switch process that allows you to switch between day and night, open and close storage, etc. Selecting the viewpoint height switch button d025 launches a viewpoint switch process that allows you to change the viewpoint height. When the VR mode button d026 is selected, an image for VR goggles is output from the display I/F 16. The print button d027 sends a hard copy of the screen to a printer. When the floor plan button d028 is selected, the viewpoint switching process is activated and the layout screen is overlaid. When the close button d029 is selected, the top button d021 to the close button d029 are hidden. The settings display d02A displays various settings. The settings display d02A includes a theme d02A1, a combination d02A2, a viewpoint height d02A3, a selected camera icon d02A4, and a camera icon d02A5. The theme d02A1 displays the name of the selected theme. The combination d02A2 displays the type of flooring and wallpaper material. The viewpoint height d02A3 displays the height of the selected viewpoint. The selected camera icon d02A4 indicates the position of the virtual camera selected as the viewpoint on the room layout diagram. The camera icon d02A5 indicates the positions of selectable virtual cameras on the room layout diagram. When the information icon d02B is selected, an information screen displaying information about the item to which the icon is attached is overlaid. When the change icon d02C is selected, screen processing is started and a selection screen for changing the texture attached to the item to which the icon is attached is overlaid. The information icon d02B and change icon d02C are arranged by the control unit 11 according to the values of the icon column in the object DB 138.

Figure 19 is an explanatory diagram showing an example of a color selection menu screen. The color selection menu screen d03 (hereinafter simply referred to as the "menu screen d03") displays multiple recommended combinations (options) of flooring and wallpaper materials and is displayed during the color selection process. The menu screen d03 includes thumbnails d031, combination types d032, selection rectangles d033, check marks d034, and a back button d035. The thumbnails d031 show room design images when each recommended combination of flooring and wallpaper is applied. The combination types d032 indicate the product models of the combined flooring and wallpaper materials. The selection rectangle d033 indicates the currently selected combination. The check mark d034 indicates the combination currently applied to the walk-through screen d02. Selecting the back button d035 clears the menu screen d03. It is possible to change the flooring and wallpaper materials by selecting a combination other than the currently selected combination from the multiple recommended combinations displayed on the menu screen d03. The control unit 11 performs the change process for changing the texture of flooring materials based on the surface data stored in the floor column of the room DB 134, and for changing the texture of walls based on the surface data stored in the wall column of the room DB 134. For fixtures and furniture, the control unit 11 checks whether a change has been made by referencing the value in the cloth material column of the object DB, and changes the texture of the target surface based on the value in the definition column.

Figure 20 is an explanatory diagram showing an example of the environment switching screen. The environment switching screen d04 is a screen that displays the environment switching menu and is displayed during the environment switching process. The environment switching screen d04 includes a furniture area d041, a day/night area d042, a storage area d043, a selection rectangle d044, and a back button d045. The furniture area d041 can be switched between displaying and hiding furniture. When switching between displaying and hiding furniture, the control unit 11 determines whether each object placed in the room is furniture based on the value in the type column of the object DB 138, and switches between displaying and hiding objects determined to be furniture. The day/night area d042 can switch the lighting environment of the room between daytime and nighttime. The storage area d043 can open and close the doors of storage furniture such as a closet. The selection rectangle d044 indicates which of the furniture area d041, day/night area d042, or storage area d043 is the active area to be operated. Selecting the back button d045 clears the environment switching screen d04.

Figure 21 is an explanatory diagram showing an example of the viewpoint height change screen. The viewpoint height change screen d05 is a screen that displays selectable viewpoint heights and is displayed during the viewpoint height change process. The viewpoint height change screen d05 includes a selection rectangle d051, a check mark d052, and a back button d053. The selection rectangle d051 indicates the currently selected viewpoint height. The check mark d052 indicates the viewpoint height currently applied to the walk-through screen d02. When the back button d053 is selected, the viewpoint height change screen d05 is cleared. Figure 21A is an example of a walk-through screen when the viewpoint height is 150 cm. Figure 21B is an example of a walk-through screen when the viewpoint height is 60 cm. Note that the selectable viewpoint heights are values common to multiple rooms and stored in the auxiliary memory unit 13. The maximum viewpoint height may be a value corresponding to the ceiling height of the room and stored in the room DB 134, etc.

The viewpoint height shown in Figure 21 is an example. It is desirable to provide at least three viewpoint heights. The first viewpoint height is approximately 130 cm to 190 cm and is intended for an adult's viewpoint. The second viewpoint height is approximately 60 cm to 130 cm and is intended for a child's viewpoint. The third viewpoint height is approximately 10 cm to 60 cm and is a viewpoint that allows for a more realistic view of materials, particularly the texture of flooring. The third viewpoint height may be 50 cm or less, preferably 40 cm or less, or may be 15 cm or more, preferably 20 cm or more. By providing at least three viewpoint heights, it is possible to experience the interior design according to height and furthermore to experience the texture of materials in real time. The viewpoint heights that can be set for the adult viewpoint, child's viewpoint, and viewpoint for viewing materials may each be one or more. Furthermore, the viewpoint heights may be continuously adjustable using a track bar. In this case, it is desirable to display rough value ranges corresponding to the adult's viewpoint, the child's viewpoint, and the viewpoint for checking the material together with a track bar. Furthermore, when switching from the first viewpoint height or the second viewpoint height to the third viewpoint height, information about the flooring material, such as the name and model number, may be automatically displayed. When switching from the third viewpoint height to the first viewpoint height or the second viewpoint, the display of information about the flooring material, such as the name and model number, may be automatically hidden.

Figure 22 is an explanatory diagram showing an example of a layout screen. Figure 22A is an example of a layout screen showing the virtual camera position before the change. Figure 22B is an example of a layout screen showing the virtual camera position after the change. The layout screen d06 is a diagram showing the virtual camera positions that can be selected on the room floor plan and the position of the virtual camera currently selected as the viewpoint. The layout screen d06 includes a position mark d061, a viewpoint display d062, a field of view display d063, and a close button d064. The position mark d061 indicates the selectable virtual camera positions. The viewpoint display d062 indicates the selected virtual camera. The virtual camera position can be changed by selecting the position mark d061 with a mouse click, etc. The field of view display d063 indicates the field of view corresponding to the display on the walk-through screen d02. Selecting the close button d064 closes the layout screen d06.

Figure 23 is an explanatory diagram showing an example of a building material selection screen. The building material selection screen d07 is a selection screen for changing flooring or wallpaper materials, and is displayed when the change icon is selected in the screen processing. The building material selection screen d07 includes a building material display d071, a selection rectangle d072, a tone switch d073, a tone switch d074, and a close button d075. The building material display d071 shows the texture and type of each building material. If the selected change icon is attached to a floor, the building material display d071 displays the flooring material. If the selected change icon is attached to a wall, closet, etc., the building material display d071 displays the wallpaper material. The selection rectangle d072 shows the currently applied building material. Selecting the tone switch d073 or tone switch d074 changes the tone of the building material displayed in the building material display d071. Building materials are grouped according to color tone. The control unit 11 performs this grouping using values in the group column of the texture DB 133. Tones are, for example, white tone, light tone, gray tone, medium tone, dark tone, or deep tone. When the close button d075 is selected, the building material selection screen d07 is closed. Note that the building materials displayed in the building material display d071 may differ depending on the selected theme.

Figures 24 and 25 are explanatory diagrams showing examples of information screens. The information screen d08 is a screen that shows information about commercial materials, and is displayed when an information icon is selected in the screen processing. Figure 24 is an example of the information screen d08 that is displayed when an information icon attached to building materials is selected. Figure 25 is an example of the information screen d08 that is displayed when an information icon attached to furniture, etc. is selected. Information about commercial materials is stored in the auxiliary memory unit 13, and is read and displayed by the control unit 11 as appropriate.

Figure 26 is an explanatory diagram showing the opening and closing of storage furniture. Figure 26A shows the screen when the storage furniture door is closed. Figure 26B shows the screen when the storage furniture door is open. Opening and closing of storage furniture doors is performed using the environment switching process described above.

In this embodiment, multiple recommended combinations of flooring and wallpaper materials are defined for each theme, allowing for a variety of interior designs for each theme. Computer graphics using a virtual three-dimensional space allow the resulting design to be viewed with a sense of realism. Furthermore, by changing the recommended combinations, it is possible to make detailed changes to the design without changing the theme.

The lighting environment of the room can be switched between daytime and nighttime, allowing users to see how different lighting environments affect the perception of the design. The open/closed status of built-in furniture such as closets can be confirmed as images, allowing users to determine whether the furniture is positioned appropriately. Architectural materials can be changed on an interior component-by-component basis, allowing users to change parts of the design to suit their preferences.

After a user selects a room on the room select screen, an introduction screen introducing the theme used in the design of the selected room may be displayed before the walk-through screen is generated and displayed. This introduction screen makes it possible to propose a spatial concept that suits the user's lifestyle.

In this embodiment, the system is intended to support the interior design of rooms by real estate developers and architects who design homes, and interior designers commissioned by end users who purchase homes, but it is not limited to this. It may also be used by building material manufacturers to realistically represent the installation image of a wide variety of products in living spaces.

(Embodiment 2)
This embodiment relates to a configuration in which the operation history of the image processing device 1 is stored, thereby enabling a second user to refer to the contents of an interior design set by a first user. In this embodiment, the first user is an interior designer and the second user is an end user, but this is not limiting. For example, the first user may be an interior designer and the second user may be a real estate developer or an architect, or the first user may be a real estate developer or an architect and the second user may be an end user.

Figure 27 is an explanatory diagram showing another example configuration of a VR system. The VR system 100 includes an image processing device 1, a management server 4, and a user terminal 5. The image processing device 1, the management server 4, and the user terminal 5 are connected to each other via a network N so that they can communicate with each other.

As in embodiment 1, image processing device 1 is configured as a desktop PC (Personal Computer), notebook PC, tablet computer, or the like. Image processing device 1 is used by interior designers and the like who perform interior design.

The management server 4 is composed of a server computer, a workstation, etc. The management server 4 may be composed of a multi-computer consisting of multiple computers, a virtual machine constructed virtually using software, or a quantum computer. The management server 4 manages end-user information and the operation history of the image processing device 1.

The user terminal 5 is configured as a notebook PC, tablet computer, or the like. The user terminal 5 is used by an end user. Although only one image processing device 1 and one user terminal 5 are shown in FIG. 27, multiple devices may be used. A system including the management server 4 and image processing device 1 is referred to as the first system, and a system including the management server 4 and user terminal 5 is referred to as the second system.

Figure 28 is a block diagram showing an example hardware configuration of a management server. The management server 4 includes a control unit 41, a main memory unit 42, an auxiliary memory unit 43, and a communication unit 45. The control unit 41, the main memory unit 42, the auxiliary memory unit 43, and the communication unit 45 are connected by a bus B.

The control unit 41 has one or more processing units such as a CPU, MPU, GPU, etc. The control unit 41 functions as the management server 4 by reading and executing the control program 4P stored in the auxiliary storage unit 43.

The main memory unit 42 is an SRAM, DRAM, flash memory, etc. The main memory unit 42 mainly temporarily stores data required by the control unit 41 to perform arithmetic processing.

The auxiliary memory unit 43 is a hard disk or SSD, etc., and stores the control program 4P and various DBs required for the control unit 41 to execute processing. The auxiliary memory unit 43 stores a user DB 431, a case DB 432, a history DB 433, and a training data DB 434. The auxiliary memory unit 43 also stores a learning model 441. The auxiliary memory unit 43 may be an external storage device connected to the management server 4. The various DBs stored in the auxiliary memory unit 43 may also be stored in a database server or cloud storage different from the management server 4. Note that the training data DB 434 and the learning model 441 are not essential components in this embodiment.

The communication unit 45 communicates with the image processing device 1 and the user terminal 5 via the network. The control unit 41 may use the communication unit 45 to download the control program 4P from another computer via the network N and store it in the auxiliary storage unit 43.

Figure 29 is a block diagram showing an example of the hardware configuration of a user terminal. The user terminal 5 includes a control unit 51, a main memory unit 52, an auxiliary memory unit 53, a communication unit 54, an input unit 55, and a display unit 56. Each component is connected by a bus B.

The control unit 51 has one or more processing units such as a CPU, MPU, GPU, etc. The control unit 51 functions as the user terminal 5 by reading and executing the control program 5P stored in the auxiliary storage unit 53.

The main memory unit 52 is an SRAM, DRAM, flash memory, etc. The main memory unit 52 mainly temporarily stores data required by the control unit 51 to perform arithmetic processing.

The auxiliary storage unit 53 is a hard disk or SSD, etc., and stores the control program 5P and various DBs required for the control unit 51 to execute processing. The various DBs, etc. stored in the auxiliary storage unit 53 may be stored in a database server or cloud storage separate from the user terminal 5.

The communication unit 54 communicates with the management server 4 via the network. The control unit 51 may use the communication unit 54 to download the control program 5P from another computer via the network N and store it in the auxiliary memory unit 53.

The input unit 55 is a keyboard and a mouse. The display unit 56 includes an LCD panel or the like. The display unit 56 displays virtual space images acquired from the virtual three-dimensional space. The display unit 56 may also be a touch panel display integrated with the input unit 55. The user terminal 5 may also display on an external display device.

Next, we will explain the databases managed by the management server 4. Figure 30 is an explanatory diagram showing an example of a user DB. User DB 431 stores information about end users. User DB 431 includes a user ID column, an age column, a gender column, an occupation column, a number of family members column, and an annual income column. The user ID column stores a user ID that can uniquely identify an end user. The age column stores the age of the end user. The gender column stores the gender of the end user. The occupation column stores the occupation of the end user. The number of family members column stores the number of people living in the room including the end user. The annual income column stores the annual income of the end user. User DB 431 may also store other attributes that are useful for interior design, such as hobbies and favorite colors.

The following data may also be stored in user DB 431: information about the location and property selected by a real estate developer, architect, or end user, as well as information that reveals the end user's preferences and lifestyle. The former information can be used to link the location with the exterior of the property and make proposals. The latter information can be used to make more appealing proposals to end users. Examples of the latter information include favorite travel destinations (countries, places to stay), lodgings stayed at while traveling, books and magazines (fashion magazines, etc.) that are usually read, hobbies, places to buy clothes, favorite brands, etc.

Figure 31 is an explanatory diagram showing an example of a case DB. Case DB 432 stores each interior design case. Case DB 432 includes a case ID column, a person in charge ID column, a user ID column, a name column, and a status column. The case ID column stores a case ID that can uniquely identify a case. The person in charge ID column stores the ID of the interior design person, interior designer, etc. The user ID column stores the ID of the end user. The name column stores the name of the case. The name is set appropriately by the person in charge. The status column stores the status of the case. Examples of statuses include Under Consideration, Proposal, and Closed. Under Consideration is a state in which the design is being considered. Proposal is a state in which the end user approves the design content and is considering whether to place an order. Closed is a state in which the design has been solidified and the end user has signed a contract.

Figure 32 is an explanatory diagram showing an example of a history DB. The history DB (history memory unit) 433 stores the operation history of the image processing device 1. By referring to the operation history, it is possible to understand the progress and results of the interior design. The history DB 433 includes a history ID column, a date and time column, a user ID column, a person in charge ID column, a theme ID column, a flooring column, a cloth material column, and a change column. The history ID column records a history ID that can uniquely identify the history. The date and time column stores the date and time the history was stored. The user ID column stores the user ID of the end user. The person in charge ID column stores the ID of the person in charge of design. The theme ID column stores the theme ID of the selected theme. The flooring column stores the type of flooring material selected. The cloth material column stores the type of cloth material selected. The change column stores the changes made when part of the design is changed. For example, in the initial setting, the wallpaper is uniform throughout the room, but the wallpaper and the exterior of the kitchen counter in the living room and kitchen may have different designs. In this case, the change column stores the surface ID indicating the kitchen wall, the object ID indicating the kitchen counter, and the type of cloth material after the change.

Next, we will explain the information processing performed by the VR system 100. Figure 33 is a flowchart showing an example of the review process procedure. The review process is performed when a design engineer reviews an interior design. The engineer operates the image processing device 1 to log in to the VR system 100. The control unit 11 of the image processing device 1 sends a login request to the management server 4 (step S131). The control unit 41 of the management server 4 performs login processing (step S132). The login processing is, for example, authentication using an ID and password. The control unit 41 searches the case DB 432, creates a list of cases linked to the engineer, and sends it to the image processing device 1 (step S133). The control unit 11 of the image processing device 1 displays the received list on the display 3 (step S134). The engineer selects a case if he or she wants to review the design of a case included in the list, or selects "new" if he or she wants to review a new case. At that time, the engineer selects whether the review is for a specific end user or for a review for which an end user has not been determined. The control unit 11 determines whether the case is new (step S135). If the control unit 11 determines that the end user is new (YES in step S135), it determines whether to register the end user information (step S136). If the control unit 11 determines that the end user information is to be registered (YES in step S136), it registers the end user information (step S137). The control unit 11 transmits the end user information entered by the person in charge to the management server 4 and registers it in the user DB 431. YES in step S136 means that the person in charge has selected a review for a specific end user. If the control unit 11 determines that the end user information is not to be registered (NO in step S136), it proceeds to step S138. NO in step S136 means that the person in charge has selected a review for which the end user has not been confirmed. If the control unit 11 determines that the end user is not new (NO in step S135), it transmits a history request for the selected case to the management server 4 (step S139). The control unit 41 of the management server 4 receives the request (step S140). The control unit 41 acquires the history of the case from the history DB 433 and transmits it to the image processing device 1 (step S141). The control unit 41 receives the history (step S142). The control unit 41 proceeds to step S138. The control unit 11 executes the main processing (step S138). This main processing is the same as the main processing in embodiment 1, and therefore a description thereof will be omitted. Note that if the history has been received, the room selection screen is not displayed, and instead the walk-through screen is displayed. After the main processing, the control unit 11 transmits the history to the management server 4 (step S143). The history includes the theme ID, theme ID string, flooring material type, and wallpaper material type. The control unit 41 of the management server 4 receives the history (step S144). The control unit 41 updates the history DB 433 based on the received history (step S145). The control unit 41 transmits a notification of update completion to the image processing device 1 (step S146). The control unit 11 of the image processing device 1 receives the completion notification (step S147). The control unit 11 determines whether to end the processing (step S148). The control unit 11 determines whether to end the review processing (step S148). If the control unit 11 determines not to end the review processing (NO in step S148), the processing returns to step S134. If the control unit 11 determines to end the review processing (YES in step S148), the review processing ends.

Figure 34 is a flowchart showing an example of the procedure for the reference process. The reference process is executed when an end user references the contents of an interior design. The end user uses the user terminal 5 to log in to the VR system 100. The control unit 51 of the user terminal 5 sends a login request to the management server 4 (step S161). The control unit 41 of the management server 4 receives the login request (step S162). The control unit 41 uses the user ID included in the login request to search the history DB 433 and acquire the history (step S163). The control unit 41 transmits the acquired history to the user terminal 5 (step S164). The control unit 51 of the user terminal 5 receives the history (step S165). The control unit 51 executes main processing (step S166). This main processing is similar to the main processing in embodiment 1, so a description is omitted. Note that a room selection screen is not displayed here. Furthermore, a walk-through screen is generated and displayed using the received history. The end user checks the design content with the designer on the walk-through screen and changes the flooring and wallpaper materials as necessary. After the main processing ends, the control unit 51 sends any changes made by the end user as update information to the management server 4 (step S167). The control unit 41 of the management server 4 receives the update information (step S168). The control unit 41 updates the history in the history DB 433 based on the update information (step S169). The control unit 41 sends a notification of update completion to the user terminal 5 (step S170). After receiving the completion notification (step S171), the control unit 51 of the user terminal 5 ends the reference processing.

Figure 35 is an explanatory diagram showing another example of a walk-through screen. The walk-through screen d09 is almost identical to the walk-through screen d02 shown in Figure 18. The following explanation will mainly focus on the differences. The walk-through screen d09 includes a top button d091, a room select button d092, a color select button d093, an environment switch button d094, a viewpoint height switch button d095, a print button d096, a floor plan button d097, a close button d098, a settings display d099, and an information icon d09A. The functions of each button, etc. are the same as those of the walk-through screen d02, so explanations will be omitted.

The walk-through screen d09 shown in Figure 35 does not include the VR mode button and change icon present on the walk-through screen d02 shown in Figure 18. Because it cannot operate in VR mode, end users cannot freely change the viewpoint, but can only select a pre-set viewpoint. Also, because the change icon is not included, end users cannot change the cloth material individually. This ensures that the interior design created by the interior designer is conveyed to the end user with precision.

In this embodiment, the end user can view the contents of the interior design. The end user can take their time to review the design created by the designer.

In this embodiment, the designer creates a design using the image processing device 1, and the end user uses the user terminal 5 to refer to and review the design created by the designer, but this is not limited to this. The management server 4 may provide services or programs to the user terminal 5 so that the user terminal 5 can execute functions equivalent to those of the image processing device 1. This allows the end user to simulate an interior design on their own, without the help of an interior designer, etc. If the user terminal 5 is provided with all the same functions as the image processing device 1 and there is a risk that the end user will be confused by the multiple functions, it is desirable to provide a simplified version to the user terminal 5.

(Embodiment 3)
This embodiment relates to a form in which a theme to be proposed to an end user is output using a learning model. When the theme is output, it may be output so that it can be distinguished from other themes to be displayed. The interior designer can clearly recognize the theme. For example, a label may be attached to the theme to be displayed, or the theme may be displayed in a separate column from other themes. Alternatively, when the theme is output, it may be output so that it cannot be distinguished from other themes to be displayed. This allows the interior designer to select a theme without preconceptions. In this embodiment, training data is prepared in which themes selected by an end user are associated with the attributes of the end user. A learning model is generated based on the training data. New end user attributes are input into the generated learning model. A designer considers the interior design to propose based on the themes obtained from the learning model.

Figure 36 is an explanatory diagram showing an example of a training data DB. The training data DB 434 stores themes selected by the end user in association with the attributes of the end user. The training data DB 434 includes a user ID column, an age column, a gender column, an occupation column, a number of family members column, an annual income column, and a label column. The user ID column stores the user ID of the end user. The age column stores the age of the end user. The gender column stores the gender of the end user. The occupation column stores the occupation of the end user. The number of family members column stores the number of family members including the end user. The annual income column stores the annual income of the family including the end user. The label column stores the theme selected by the end user.

Figure 37 is an explanatory diagram showing an example of a learning model. Learning model 441 is a neural network that takes end user attributes as input and themes as output. The neural network is, for example, a CNN (Convolution Neural Network), and has an input layer that accepts end user input, an output layer that outputs themes, and an intermediate layer that extracts feature quantities of end user attributes.

The input layer has multiple neurons that accept multiple attribute values of the end user as input, and passes the input item values to the middle layer. The middle layer has a configuration in which convolution layers that convolve each attribute value input from the input layer and pooling layers that map the values convolved in the convolution layer are connected alternately, compressing the input information and ultimately extracting features. The output layer has multiple neurons that output the suitability of each theme to the end user, and outputs the suitability for each theme based on the features output from the middle layer.

In this embodiment, the learning model 441 is described as a CNN, but the learning model 441 is not limited to a CNN and may be a model constructed using other learning algorithms, such as a neural network other than a CNN, a Bayesian network, or a decision tree.

The management server 4 uses training data for learning. The training data is data that associates end user attribute information with themes selected by the end user. The training data is stored in the training data DB 434 as described above.

The management server 4 inputs the end user's attribute values contained in the training data into the input layer, undergoes calculation processing in the middle layer, and obtains information about themes suitable for the end user from the output layer. There are the same number of output nodes as there are themes. Each output node is associated with a theme, and outputs a value indicating the end user's suitability for each theme. Here, the sum of all suitability values is 1.

The management server 4 compares the fitness values output from the output layer with the labels included in the training data, i.e., the correct values, and optimizes the parameters used in the calculations in the intermediate layer so that the output value of the output node corresponding to the label approaches 1. These parameters include, for example, the weights (coupling coefficients) between neurons and the coefficients of the activation functions used in each neuron. There are no particular limitations on the method for optimizing the parameters, but for example, the management server 4 optimizes various parameters using the backpropagation method. The management server 4 performs the above learning process using all of the training data and generates a learned learning model 441.

Figure 38 is a flowchart showing an example procedure for the learning model generation process. The control unit 41 of the management server 4 creates training data from the user DB 431 and history DB 433 (step S191). The control unit 41 stores the training data in the training data DB 434. The control unit 41 selects one of the training data stored in the training data DB 434 (step S192). The control unit 41 performs learning (step S193). The learning process has already been explained, so it will not be described here. The control unit 41 determines whether there is unprocessed training data (step S194). If the control unit 41 determines that there is unprocessed training data (YES in step S194), it returns the process to step S192. If the control unit 41 determines that there is no unprocessed training data (NO in step S194), it stores the generated learning model 441 in the auxiliary storage unit 43 (step S195) and ends the process.

Figure 39 is a flowchart showing an example of the procedure for theme selection processing. The theme selection processing is a process that uses the learning model 441 to select a theme that suits the end user. For example, when a designer proposes an interior design to a new end user, they consider a design based on the theme selected by the theme selection processing.

The designer acquires the end user's attributes and sends them to the management server 4 via the image processing device 1. The control unit 41 of the management server 4 acquires the end user's attributes (step S201). The control unit 41 inputs the acquired attributes into the learning model 441 and acquires the theme to be selected from the output of the learning model 441 (step S202). The control unit 41 outputs the theme (step S203) and ends the process. The selected theme is sent to the image processing device 1. The designer selects the theme sent from the image processing device 1 and considers the design.

Although the theme selection process is performed by the management server 4, it is not limited to this. It may also be performed by the image processing device 1 used by the designer. In this case, a copy of the learning model 441 is stored in the auxiliary memory unit 13 of the image processing device 1.

A theme is selected based on the end user's attributes, but a learning model that suggests recommended combinations may also be generated based on the selected theme and the end user's attributes. This learning model is generated for each theme. Training data containing the end user's attributes and the recommended combinations selected by the end user is generated for each theme from historical data. The end user's attributes are the input, and the recommended combinations are the correct answer data. End user attributes are input into the learning model for each theme, and learning is performed so that the output of the learning model is the recommended combination.

In this embodiment, the theme is selected using the learning model 441, which makes it possible to increase the accuracy of theme selection, which is an early stage of design.

Figure 40 is a block diagram showing an example of functional units included in an image processing device. The image processing device 1 includes, as functional units, a classification output unit 11a, a reception unit 11b, a generation unit 11c, an acquisition unit 11d, and an image acquisition unit 11e. Each of these functional units is realized by the control unit 11 operating based on the control program 1P.

The classification output unit 11a outputs multiple classifications related to residential interiors. The reception unit 11b receives the selection of a classification. The generation unit 11c generates a virtual space that simulates a living space composed of interior elements according to the selected classification. The acquisition unit 11d acquires a virtual space image seen from a virtual viewpoint within the generated virtual space. The image acquisition unit 11e outputs the acquired virtual space image.

(Proximity display function)
As described above, when the viewpoint is set to the third viewpoint height (a height of approximately 10 cm to 60 cm), a display mode is achieved that allows the user to more realistically confirm the texture of flooring materials. Materials whose textures can be confirmed do not have to be limited to flooring materials. The following describes a proximity display function that allows the user to confirm the texture of materials other than flooring materials.

With the proximity display function, when the position of the virtual viewpoint enters a specified range based on an object in virtual space, the texture image attached to the surface of that object is switched to a higher resolution image. When the proximity display function is enabled, the texture image attached to objects that are not the target of proximity display may be low or medium resolution. High, medium, and low resolutions may be determined appropriately taking into account the hardware performance of the user terminal 5, etc. As an example of settings, 71 to 96 dpi is low resolution, around 200 dpi is medium resolution, and 350 to 400 dpi is high resolution.

Figure 41 is a flowchart showing an example of the procedure for the proximity display processing. The proximity display processing is executed after the viewpoint change in step S54 and image acquisition in step S47 in the screen processing shown in Figure 12. The proximity display processing is also executed after image acquisition in step S104 in the viewpoint height change processing shown in Figure 15. The control unit 11 of the image processing device 1 acquires the viewpoint of the virtual camera (step S211). The control unit 11 selects an object to be processed (step S212). The object is, for example, an object whose value in the cloth material column of object DB 138 is other than "none." The control unit 11 calculates the distance between the viewpoint position and the object (step S213). The control unit 11 determines whether the calculated distance is equal to or less than a predetermined threshold (step S214). If the control unit 11 determines that the calculated distance is equal to or less than the predetermined threshold (YES in step S214), the control unit 11 changes the texture attached to the surface of the object to a high-resolution one (step S215). If the control unit 11 determines that the calculated distance exceeds a predetermined threshold (NO in step S214), it determines whether there are any unprocessed objects (step S216). If the control unit 11 determines that there are any unprocessed objects (YES in step S216), it returns the process to step S212 and processes the unprocessed objects. If the control unit 11 determines that there are no unprocessed objects (NO in step S216), it ends the process.

Although the proximity display process is described as being executed after step S47 or step S104, this is not a limitation. It may also be processed integrally with image acquisition. When the viewpoint position of the virtual camera is changed, the appearance of each object as seen from the viewpoint position is recalculated. During this calculation, the distance between the viewpoint position and the object is calculated, and if the calculated distance is equal to or less than a threshold, the texture is made high resolution. This allows for efficient rendering processing.

Figure 42 is an explanatory diagram showing an example of proximity display. Figure 42 shows an example of proximity display for flooring materials. The proximity display may also display information such as the model, color name, material, price, or manufacturer name of the building material being displayed.

In the above explanation, it is assumed that the texture of the object (building materials such as flooring or wall materials) that is the subject of close-up display is changed to high resolution on the walk-through screen shown in Figure 18, but this is not limited to this. A window separate from the walk-through screen may be opened, and only the subject object may be displayed in 3D in that window. Furthermore, the window may allow the object to be rotated and zoomed in and out. It may also be possible to check how the object looks when illuminated by sunlight during the day and when illuminated by indoor lighting at night.

Although it has been stated that the texture of an object is changed to a higher resolution when the distance between the virtual camera's viewpoint and the object falls below a certain distance, this is not limited to this. The user may also explicitly specify an object to be displayed in close proximity using a pointing device, etc.

The close-up display function allows users to check the texture of individual materials (building materials). When displayed in 3D in a separate window, users can check the material as if they were holding it in their hands.

(Expansion of operation history)
In the second embodiment, a form of storing the operation history of the first user has been described. The operation history is stored in the history DB 433 shown in FIG. 32 , but the stored history may be expanded. In addition to the content explicitly shown in FIG. 32 , the history DB 433 may store a history of changes to the viewpoint height, an operation history of day/night switching, and the like. Furthermore, a virtual space image may be stored in association with each operation history. The operation history may be stored each time a user performs an operation, or may be stored at predetermined time intervals.

Next, we will discuss the use of extended operation history. Here, we assume the following operation. The users featured are a showroom guide and an interior coordinator (hereinafter also referred to as "coordinator"). The guide is assumed to be familiar with operating the VR system 100, but the coordinator is not. The coordinator visits the showroom and designs using the VR system 100 with the guide's assistance. Alternatively, the guide may operate the VR system 100 in place of the coordinator. The operation history here is stored in the history DB 433 as the coordinator's history, even if the guide operates the system. Furthermore, if the coordinator is not registered in the VR system 100, user registration is performed and a user ID is assigned to the coordinator.

Figure 43 is an explanatory diagram showing another example of a history DB. History DB 433 includes a history ID column, a date and time column, a sequence number column, a user ID column, a person in charge ID column, a theme ID column, a flooring column, a cloth material column, a change column, a viewpoint position column, a viewpoint height column, a day/night column, an image column, and a case ID column. The history ID column stores an ID that can uniquely identify a history. The date and time column stores the date and time the user logged in. The sequence number column indicates the sequence number of the history. The user ID column stores the ID of the user, in this case the coordinator. The person in charge ID column stores the ID of the guide. The theme ID column, flooring column, and cloth material column are the same as the columns with the same names shown in Figure 32, so their explanation is omitted. Note that if flooring and cloth materials are changed to flooring and cloth materials of different colors all at once by referring to the recommended combination DB 132, the history is stored in the flooring column and the cloth material column. The change column stores the change history of cloth materials (building materials such as flooring and wall materials). The viewpoint position column stores the history of changes to the viewpoint position. The viewpoint position column stores planar coordinates (x coordinate, y coordinate). The viewpoint height column stores the history of changes to the viewpoint position height (z coordinate). If a preset first viewpoint height, second viewpoint height, or third viewpoint height is selected, that information is stored in the viewpoint height column. In other cases, the viewpoint height column stores the z coordinate value. The day/night column stores the history of time zone changes. As mentioned above, during the day, the lighting is expressed as sunlight pouring in through a window, and during the night, the lighting is expressed as using lighting installed on the ceiling or wall as a light source. Therefore, switching between day and night changes the overall color of the virtual space. The image column stores virtual space images for each history. The image column may store the filename of the image file instead of the actual image data. The case ID column stores an ID that identifies the design case. By associating a series of histories with the same case ID, it is possible to identify the series of histories when referencing the history DB 433.

Once the design in the showroom is completed, an ID for the work and a project ID are issued. The project ID may be issued when the work begins. The operation history is associated with the user ID and project ID and stored in the history DB 433. A two-dimensional code is also issued to the coordinator. The two-dimensional code includes information such as the user ID, project ID, date, and a URL (Uniform Resource Locator) for accessing the VR system 100. A printout of the two-dimensional code and date, along with the virtual space image, is provided to the coordinator. This printout may also be provided to the end user who plans to purchase a home.

The coordinator can access the VR system 100 using a two-dimensional code. The VR system 100 provides the coordinator with a web application. The coordinator can use the VR system 100 using a web browser without having to install a new application on their device.

Next, we will explain the processing related to operation history. Figure 44 is a flowchart showing an example of the procedure for image display processing. Image display processing is a process in which the coordinator references the design content created in the showroom on the user's own terminal (user terminal). The coordinator receives the design in the showroom and has the camera (not shown) of the user terminal 5 read the two-dimensional code. The control unit 51 of the user terminal 5 analyzes the two-dimensional code and issues a login request to the VR system 100 using the URL obtained through the analysis (step S221). The control unit 41 of the management server 4 receives the login request (step S222). The login request includes the user ID, case ID, date, etc. The control unit 41 reads the history from the history DB 433 using the user ID, case ID, date, etc. included in the login request (step S223). The control unit 41 creates a list of the read history and sends it to the user terminal 5 (step S224). The control unit 51 of the user terminal 5 receives the list and displays it on the display unit 56 (step S225). It is desirable for the list to include a virtual space image corresponding to each history as a thumbnail. The coordinator uses the input unit 55 to select one of the histories. The control unit 51 sends the selection information to the management server 4 (step S226). The control unit 41 of the management server 4 receives the selection information (step S227). The control unit 41 acquires the virtual space image corresponding to the selected history and sends it to the user terminal 5 (step S228). The control unit 51 of the user terminal 5 receives the image and displays it on the display unit 56 (step S229), ending the process.

The images displayed during image display processing are snapshot images, and cannot be operated like the walk-through screen, but can be switched to the walk-through screen at the coordinator's instruction. In this case, the virtual space displayed on the walk-through screen is set based on the history. Alternatively, the snapshot images stored as history may be used only as thumbnails when displaying a list, and the walk-through screen based on the history may be reproduced and displayed. Since the application running on the coordinator's terminal is a web application, the functions available on the walk-through screen may be limited.

When displaying each history, not only the virtual space image but also the model number of the building material selected in the virtual space may be displayed. Furthermore, by linking with catalog information, it may be possible to display the price, form, features, etc. of the building material.

By checking the history, coordinators can confirm the progress of design considerations in the showroom. Furthermore, by using the virtual space images stored along with the history, coordinators can create materials for end users. Furthermore, by using the walk-through screen that reflects the history, coordinators can make changes to the design.

In addition, the history may store information about building materials that have been displayed in close proximity. When referencing the history, the building materials that have been displayed in close proximity can be included in the list, or a separate list of building materials that have been displayed in close proximity can be created.

(Bookmark function)
While the VR system 100 automatically stores history, the bookmark function stores the state of the virtual space at a certain point in time when explicitly instructed by the user. FIG. 45 is an explanatory diagram showing an example of a bookmark DB. The bookmark DB 435 includes a bookmark ID column, a project ID column, a name column, a date and time column, a user ID column, a person in charge ID column, a theme ID column, a flooring material column, a cloth material column, a change column, and an image column. In the bookmark DB 435, the date and time column through the change column each store data similar to the columns with the same names in the history DB 433, so their explanations are omitted. The bookmark ID column stores an ID that can uniquely identify a bookmark. The project ID column stores an ID for each design project handled by the coordinator. The name column stores the name of the bookmark. It is assumed that the user inputs a name, but if no input is made, a name generated by the management server 4 is stored. The image column stores the virtual space image at the time when bookmark storage is instructed. The case ID column and the image column are data columns that are also included in the extended history described above. Similar to the history DB 433, a viewpoint position column, a viewpoint height column, and a day/night column may be provided in the bookmark DB 435. A user instructs to save a bookmark by selecting a bookmark button. For example, the bookmark button may be added to the top button d021, room select button d022, etc., lined up on the left side of the walk-through screen d02 shown in FIG. 18. A pull-down menu may be displayed by right-clicking the mouse on the walk-through screen d02, and a command to save a bookmark may be given from the menu.

The process for calling up a bookmark is similar to the image display process shown in Figure 44, so a detailed explanation will be omitted. By using the bookmark function, the coordinator can memorize the status of the virtual space at a time specified by the coordinator.

(Recommended display)
The recommended display for building materials will now be described. There is a wide variety of building materials that can be selected on the building material selection screen shown in Figure 23. Therefore, even a coordinator may be confused about which building material to select. The recommended display is a display that helps in selecting building materials. A persona is set for each theme. Also, recommended building materials are set for each theme. Based on the attributes of the end user, the persona that is closest to the end user is determined. Recommended building materials are identified based on the theme that corresponds to the determined persona. The identified building materials are displayed with a mark or icon indicating that they are recommended.

Figure 46 is an explanatory diagram showing an example of a persona DB. Persona DB 139 stores persona information for each theme. Persona DB 139 includes a theme ID column, an age column, a gender column, an occupation column, a family composition column, and an annual income column. The theme ID column stores the theme ID of the theme corresponding to the persona. The age column stores the age of the persona. The gender column stores the gender of the persona. The occupation column stores the occupation of the persona. The family composition column stores the family composition of the persona. The annual income column stores the family composition of the persona.

Figure 47 is a flowchart showing an example of the procedure for the recommended mark assignment process. The recommended mark assignment process is executed when the building material selection screen d07 is generated. The control unit 11 of the image processing device 1 determines whether the end user's attributes have been acquired (step S241). If the control unit 11 determines that the end user's attributes have been acquired (YES in step S241), it determines the persona that most closely resembles the end user (step S242). The control unit 11 calculates the similarity between the end user and each persona using the end user's attributes and the persona's attributes. Since the calculation of similarity can be performed using publicly known technology, an explanation will be omitted. The persona with the highest similarity is determined to be the persona that most closely resembles the end user. The control unit 11 refers to the persona DB 139 to determine the theme corresponding to the persona (step S243). The control unit 11 refers to the recommended combination DB to identify the building materials that correspond to the determined theme (step S244). The control unit 11 assigns a recommendation mark to the identified building material (step S245) and ends the process. If the control unit 11 determines that the end user's attributes have not yet been acquired (NO in step S241), it determines whether a theme has been selected (step S246). If the control unit 11 determines that a theme has been selected (YES in step S246), it moves the process to step S244. If the control unit 11 determines that a theme has not yet been selected (NO in step S246), it ends the process.

The method of assigning recommended marks is not limited to the above-mentioned method. Marks may also be assigned to building materials that have a high affinity with the selected theme. Building materials that have a high affinity with the theme are determined by analyzing the operation histories of multiple users, etc. The correspondence between themes and building materials is then stored in advance in the auxiliary memory unit 13. Also, based on the operation history of the user (coordinator), marks may be assigned to building materials that the user is likely to like or that have been selected in the past. Furthermore, marks may also be assigned to building materials that manufacturers, etc., are particularly interested in promoting.

(Display related information)
The recommendation display displays recommended building materials based on the end user's attributes. The related information display displays information based on building materials, such as personas corresponding to building materials and information on furniture that is considered to be good to combine with building materials.

Figure 48 is an explanatory diagram showing an example of a recommended furniture DB. The recommended furniture DB 13A stores information on furniture that is considered to be suitable for combination with building materials. The recommended furniture DB 13A includes a model column, a product name column, a part number column, a color column, an image column, and a catalog column. The model column stores the model of the building material. The product name column stores the product name of the recommended furniture. Product names include sofas, dining tables, and beds. The product number column stores the product numbers of the recommended furniture. The color column stores the color of the furniture. The image column stores images of the recommended furniture. The image column may store the file names of the image files. The catalog column stores catalog information for the recommended furniture or the URL of a website where the catalog information is posted.

Figure 49 is a flowchart showing an example procedure for related information processing. Related information processing is a process that is executed when the user performs an operation such as right-clicking the mouse on a building material on the walk-through screen. The control unit 11 of the image processing device 1 accepts a command via a predetermined operation (step S251). The control unit 11 identifies the object that the user has selected as the target of operation from the position of the mouse pointer, etc., and determines whether it is the target of related information processing (step S252). If the control unit 11 determines that the identified object is the target of related information processing (YES in step S252), it displays a menu (step S253). The control unit 11 determines whether the user has completed menu selection (step S254). If the control unit 11 determines that the user has not completed menu selection (NO in step S254), it repeats the determination process. If the control unit 11 determines that the user has completed menu selection (YES in step S254), it determines whether the menu selected by the user is "Catalog" (step S255). If the control unit 11 determines that the menu selected by the user is "Catalog" (YES in step S255), it displays catalog information for the specified object, for example, a building material (step S256), and ends the process. For example, the catalog information is obtained from the auxiliary storage unit 13 or the manufacturer's website using the model of the building material as a key. If the control unit 11 determines that the menu selected by the user is not "Catalog" (NO in step S255), it determines whether the menu selected by the user is "Persona" (step S257). If the control unit 11 determines that the menu selected by the user is "Persona" (YES in step S257), it displays persona information (step S258), and ends the process. The control unit 11 obtains persona information in the following procedure. The control unit 11 references the recommended combination DB 132 to obtain a theme ID corresponding to the building material. The control unit 11 obtains persona information corresponding to the obtained theme ID from the persona DB 139. If the control unit 11 determines that the menu selected by the user is not "Persona" (NO in step S257), it determines whether the menu selected by the user is "Recommended Furniture" (step S259). If the control unit 11 determines that the menu selected by the user is "Recommended Furniture" (YES in step S259), it acquires and displays information about recommended furniture corresponding to the building material from the recommended furniture DB 13A (step S260), and ends the processing. If the control unit 11 determines that the menu selected by the user is not "Recommended Furniture" (NO in step S259), it ends the processing. Furthermore, if the control unit 11 determines that the identified object is not a target for related information processing (NO in step S252), it ends the processing.

The related information display function displays catalog information for building materials and persona information corresponding to the materials, allowing users to easily select building materials that seem appropriate for the end user. It also provides information on suitable furniture to combine with building materials, making it possible to recommend furniture purchases to end users.

In the above explanation, the related information display function was explained as being executed by right-clicking the building material with the mouse, but this is not limited to this. It may also be called up by clicking the information icon d02B shown in Figure 18.

(Recording function)
On the walk-through screen d02, by continuously changing the position of the virtual camera, it is possible to obtain a series of images that give the perception of walking around in a virtual space. The recording function is a function for creating videos by storing these images.

Figure 50 is a flowchart showing an example of the procedure for video creation processing. The control unit 11 of the image processing device 1 receives an instruction to start recording (step S271). The control unit 11 starts recording (step S272). The control unit 11 determines whether to end recording (step S273). If the control unit 11 determines not to end recording (NO in step S273), it repeats the determination. If the control unit 11 determines to end recording (YES in step S273), it stores the video file in the auxiliary storage unit 13 (step S274) and ends processing. The control unit 11 determines to end processing when, for example, the user issues an instruction to end processing.

Instructions to start/end recording can be made, for example, by adding a record button to the top button d021, room select button d022, etc. lined up on the left side of the walk-through screen d02 shown in FIG. 18, and the user can start recording by selecting the record button. While recording is in progress, the record button is changed to an end button and displayed, and recording ends when the user selects the end button. While recording, a pause button may be displayed in addition to the end button, allowing the user to pause and resume recording. When the mouse is right-clicked on the walk-through screen d02, a pull-down menu may be displayed, and instructions to start/end recording may be given from that menu.

The user may manually move the virtual camera's viewpoint position using a pointing device such as a 3D mouse, or the virtual camera may move at a predetermined speed along a pre-set route. In this case, the user must instruct the camera to start recording, but not to stop it. Even if the user does not instruct the camera to stop recording, the recording will end once the camera has completed moving along the predetermined route. Multiple routes may be set so that the user can select one. Different routes may be set for each theme. The camera may also stop mid-movement, display a close-up view of a building material specified by the user for a predetermined period of time, and then resume moving. The user may also be able to specify the height of the viewpoint and the speed of movement during movement. Videos may also be created by stitching together multiple virtual space images stored as operation history or bookmarks.

When creating a video from a predetermined route by stitching together multiple virtual space images, the management server 4 may perform the video creation process. The user can create the video by issuing instructions from the user terminal 5. At that time, if there is insufficient data in the management server 4, the missing data is obtained from the image processing device 1.

Figure 51 is an explanatory diagram showing examples of travel routes. Figure 51 shows three routes. Route 1 and Route 2 have the same starting position but different goal positions. Route 3 has a different starting position from Route 1 and Route 2, but the same goal position as Route 2. Note that the user may be able to change pre-set routes. The user may also be able to create new routes. Different pre-set routes may also be prepared for each theme.

(Presentation board creation function)
The presentation board is used when a coordinator proposes his/her own outfit to an end user (customer). A plurality of pieces of information are collected on one screen and displayed on the display unit 56 of the user terminal 5.

Figure 52 is an explanatory diagram showing an example of a template DB. Template DB 437 stores templates for creating presentation boards. Template DB 437 includes a theme ID column, pattern column, area 1 column, area 2 column, area 3 column, and template column. The theme ID column stores the theme ID of the theme corresponding to the template. The pattern column stores the pattern name of the template. Here, this is represented as a number. The area 1 column, area 2 column, and area 3 column store the data content to be embedded in each area. The image indicates a virtual space image. The description indicates an explanation of the coordination concept. The building materials indicate information about the building materials selected in the virtual space. The template column stores the template definition. It stores the size, position, etc. of each area.

Figure 53 is a flowchart showing an example procedure for the presentation board creation process. The user inputs a request to create a presentation board into the user terminal 5. The control unit 51 of the user terminal 5 sends the request to create a presentation board to the management server 4 (step S281). The control unit 41 of the management server 4 receives the request to create a presentation board (step S282). The control unit 41 acquires the user's operation history from the history DB 433 (step S283). The control unit 41 creates a list of the history and sends it to the user terminal 5 (step S284). The control unit 51 of the user terminal 5 receives the list and displays it on the display unit 56 (step S285). The user selects the history to use using the input unit 55. The control unit 51 accepts the selection (step S286). The control unit 51 sends the selected information to the management server 4 (step S287). The control unit 41 of the management server 4 receives the selected information (step S288). The control unit 41 acquires a theme corresponding to the selected history and acquires a template corresponding to the theme from the template DB 437 (step S289). The control unit 41 embeds data included in the history, such as the virtual space image and building material information, into the template to create a presentation board (step S290). The control unit 41 transmits the presentation board to the user terminal 5 (step S291). The control unit 51 of the user terminal 5 receives the presentation board (step S292). The control unit 51 displays the presentation board on the display unit 56 (step S293). The user inputs a concept description of the coordination, etc., via the input unit 55. The control unit 51 accepts the input (step S294). Note that the concept description, etc., may be input after step S286 and transmitted to the management server 4 in step S287. The control unit 51 embeds the accepted concept description in the presentation board and redisplays the presentation board on the display unit 56 (step S295). The user confirms the displayed content and enters a necessary command. The command may be "edit," "save and exit," etc. The control unit 51 receives a command (step S296). The control unit 51 determines whether the received command indicates completion (step S297). If the control unit 51 determines that the command does not indicate completion (NO in step S297), it executes the command (step S298) and returns the process to step S295. Note that, depending on the content of the command, the process may also return to step 285. If the control unit 51 determines that the command indicates completion (YES in step S297), it stores the presentation board in the auxiliary storage unit 53 (step S299) and ends the process.

Figure 54 is an explanatory diagram showing example templates. Figure 54 shows three types of templates. All templates have three types of embedded information, and the information types are the same. What differs between the templates is the position of the display area and the balance of size. A virtual space image is embedded in area 521. An explanation of the coordinator's concept is embedded in area 522. An image of the selected building material is embedded in area 523. Information displayed in area 523 may also include catalog information for the building material (model, price, form, features, etc.).

The presentation board creation function allows coordinators to easily create presentation boards for clients. Furthermore, the VR system 100 collects the necessary information according to a template, saving the coordinator the trouble of gathering information.

(Centralized data management)
In the above-described embodiment, the image processing device 1 or the management server 4 stores data for constructing the virtual three-dimensional space, user data, and user operation history data, but this is not limited to this. A centralized database server may be provided to facilitate the management, updating, and sharing of this data. Furthermore, a terminal for a salesperson who introduces the VR system 100 may also be included as part of the VR system 100 configuration.

Figure 55 is an explanatory diagram showing another example configuration of a VR system. The VR system 100 includes an image processing device 1, a management server 4, a user terminal 5, a database server 6, and a sales representative terminal 7. The image processing device 1, management server 4, user terminal 5, database server 6, and sales representative terminal 7 are connected to each other via a network N so that they can communicate with each other.

The configurations of the image processing device 1, management server 4, and user terminal 5 are the same as in embodiment 2, so detailed explanations will be omitted.

The database server 6 is composed of a server computer, a workstation, etc. The database server 6 may be composed of a multicomputer consisting of multiple computers, a virtual machine constructed virtually by software, or a quantum computer. The database server 6 stores data for constructing the virtual three-dimensional space, user information, and user operation history. In the above-described embodiment, the data for constructing the virtual three-dimensional space is stored in the image processing device 1. The user information and user operation history are stored in the management server 4. In this case, these data are stored in the database server 6.

The salesperson terminal 7 is configured as a notebook PC, tablet computer, or the like. The salesperson terminal 7 is a terminal used primarily by salespeople who sell building materials. Salespeople sell building materials by introducing the VR system 100 to coordinators. The salesperson terminal 7 stores explanatory materials for the VR system 100, etc.

The image processing device 1 is a computer with higher specifications than the user terminal 5 and the salesperson terminal 7. The image processing device 1 is used by a showroom guide who is familiar with operating the VR system 100. The user terminal 5 is used by a coordinator.

The image processing device 1, sales representative terminal 7, and user terminal 5 provide different functions as follows: All functions of the VR system 100 are available on the image processing device 1. The sales representative terminal 7 can demonstrate the VR system 100. The sales representative terminal 7 allows users to select a theme and view virtual space images designed for each selected theme. However, the viewpoint position of the virtual camera can only be selected from multiple pre-set positions. A web application is provided on the user terminal 5. The user terminal 5's main function is to create a presentation board using the results of the design performed on the image processing device 1.

The image processing device 1, management server 4, and database server 6 make up the showroom system as a subsystem. The salesperson system as a subsystem is made up of the salesperson terminal 7, management server 4, and database server 6. The coordinator system as a subsystem is made up of the user terminal 5, management server 4, and database server 6. Because the database server 6 centrally manages the data used in the VR system 100, data can be shared and linked between any of the showroom system, salesperson system, or coordinator system.

The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed.
The embodiments disclosed herein are to be considered as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

REFERENCE SIGNS LIST 100 VR system 1 Image processing device 11 Control unit 11a Classification output unit 11b Reception unit 11c Generation unit 11d Acquisition unit 11e Image acquisition unit 12 Main memory unit 13 Auxiliary memory unit 131 Theme DB
132 Recommended Combination DB
133 Texture DB
134 Room DB
135 Vertex DB
136 Side DB
137 side DB
138 Object DB
139 Persona DB
13A Recommended Furniture DB
15 Input I/F
16 Display I/F
17 Communication unit 18 Reading unit 1P Control program 1a Portable storage medium 1b Semiconductor memory 2 Controller 3 Display 4 Management server 41 Control unit 42 Main storage unit 43 Auxiliary storage unit 431 User DB
432 Case DB
433 History DB
434 Training Data DB
435 Bookmark DB
437 Template DB
441 Learning model 45 Communication unit 4P Control program 5 User terminal 51 Control unit 52 Main memory unit 53 Auxiliary memory unit 54 Communication unit 55 Input unit 56 Display unit 5P Control program B Bus N Network

Claims (12)

Output multiple classifications related to home interiors,
Accept the classification selection,
A virtual space is generated that simulates a living space composed of interior components according to the selected classification;
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
Output the acquired virtual space image,
an image output method, characterized in that a computer executes a process of outputting display setting information including the name of the classification, information on the building materials used for the interior components, and the height of the virtual viewpoint so as to be superimposed on the virtual space image. Input the attributes of an end user who is purchasing a home into a learning model that has been trained to output a classification of interior design when the attributes of the end user are input,
obtaining a classification from the learned model;
When outputting multiple classifications related to home interiors, output the acquired classifications,
Accept the classification selection,
A virtual space is generated that simulates a living space composed of interior components according to the selected classification;
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
An image output method, comprising: a computer executing a process for outputting an acquired virtual space image. When the virtual viewpoint is set within a predetermined distance from the interior component, the texture image of the interior component is switched;
3. The image output method according to claim 1, wherein the computer executes a process of displaying information about the interior components that are the display targets. Output multiple classifications related to home interiors,
Accept the classification selection,
A virtual space is generated that simulates a living space composed of interior components according to the selected classification;
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
Output the acquired virtual space image,
outputting a selection of a plurality of building materials with different colors according to the selected classification;
Accept the selection of the output building materials,
changing the color of the interior component in accordance with the received building materials;
acquiring the virtual space image corresponding to the changed virtual space;
and storing the acquired virtual space image in association with a change history of the building materials. Output multiple classifications related to home interiors,
Accept the classification selection,
A virtual space is generated that simulates a living space composed of interior components according to the selected classification;
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
Output the acquired virtual space image,
Get the time zone,
selecting a virtual light source corresponding to the acquired time period from a plurality of virtual light sources corresponding to a predetermined time period;
changing the color of the virtual space based on a selected virtual light source;
acquiring the virtual space image corresponding to the changed virtual space;
The image output method is characterized in that a computer executes a process of storing the acquired virtual space image in association with a change history of the building materials. Output multiple classifications related to home interiors,
Accept the classification selection,
A virtual space is generated that simulates a living space composed of interior components according to the selected classification;
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
Output the acquired virtual space image,
outputting at least three options of a height of the virtual viewpoint, including a first viewpoint, a second viewpoint lower than the first viewpoint, and a third viewpoint lower than the second viewpoint;
changing the height of the virtual viewpoint in response to the selection of the option;
acquiring the virtual space image corresponding to the changed virtual space;
and storing the acquired virtual space image in association with a history of changes to the height of the virtual viewpoint. Output multiple classifications related to home interiors,
Accept the classification selection,
A virtual space is generated that simulates a living space composed of interior components according to the selected classification;
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
A process of outputting the acquired virtual space image,
Input end-user attributes into the learning model,
obtaining a classification of interiors from the learning model;
An image output method, characterized in that, when outputting a plurality of classifications related to the interior of the house, a computer executes a process of outputting the acquired classification. Output classifications related to home interiors,
Accept the classification selection,
A virtual space is generated that simulates a living space composed of interior components according to the selected classification;
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
A process of outputting the acquired virtual space image,
Input end-user attributes into the learning model,
obtaining a classification of interiors from the learning model;
An image output method, characterized in that, when outputting the classification related to the interior of the house, a process of outputting the acquired classification is executed by a computer. A virtual space is generated that simulates a living space composed of interior components according to a classification of the interior of a house,
Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space,
A process of outputting the acquired virtual space image,
Input end-user attributes into the learning model,
obtaining a classification of interiors from the learning model;
an image output method, characterized in that a computer executes a process of using the acquired classification as a classification related to the interior of the house. 10. The image output method according to claim 7 , wherein the computer executes a process of outputting the acquired virtual space image to a screen that allows a virtual viewpoint in the virtual space to be changed. 11. The image output method according to claim 10 , wherein the computer executes a process of accepting, via the screen, a change in at least one of the position of the virtual viewpoint and the direction from the virtual viewpoint. 12. The image output method according to claim 7 , wherein the computer executes a process of outputting the acquired virtual space image onto a screen that allows a user to walk through the virtual space.