JP7620898B2 - Information processing device, information processing system, information processing method, and program - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies. Announced at the Japan Home & Building Show 2022 from October 26, 2022 to October 28, 2022. Posted on the website on September 22, 2023.

The present invention relates to an information processing device, an information processing system, an information processing method, and a program.

Constructing a home has traditionally been an important issue for many users, and several technologies have been proposed for constructing the home that the user desires. For example, a technology has been disclosed that can propose home plans and floor plans based on information input by the user (see Patent Document 1).

JP 2021-189657 A

Here, in conventional technologies including the technology described in the above-mentioned Patent Document 1, in many cases, house plans and the like are proposed based on the user's answers to questions. However, in general, although users have an abstract ideal image and preferences for the house they want to build, it is difficult for them to clearly communicate their image and desires for the house.

The aim of the present invention is to assist users in choosing a home in an intuitive and easy-to-understand way.

One aspect of the present invention is an information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, the information processing system including: a housing contractor database that stores the characteristics of homes constructed by housing contractors in association with the housing contractors for each of a plurality of housing contractors; an acquisition unit that acquires from the electronic device an image selected according to the user's preferences and text information indicating the user's requests for the image; a generation unit that generates a first feature related to the user's preferences based on the text information, generates a second feature based on the image that is a feature of the image related to what is contained in the image, and generates weights for each of the first feature and the second feature based on the text information; and a selection unit that selects a housing contractor from the plurality of housing contractors that corresponds to the user's preferences based on a comparison result of comparing the feature information indicating the user's preferences generated based on the weight, the first feature, and the second feature with the features of the home; an information processing device that constitutes the information processing system; an information processing method that includes each of the processes; and a program that causes a computer to execute each of the processes.
Another aspect of the present invention is an information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, the information processing system comprising: a housing developer database that stores the characteristics of homes constructed by a housing developer and the housing developer in association with each of a plurality of housing developers; an image information database that stores housing images of each home constructed by each of the plurality of housing developers; an acquisition unit that acquires an image according to the user's preferences from the electronic device; a feature acquisition unit that acquires features of the image based on the image; a selection unit that selects a housing developer according to the user's preferences from among the plurality of housing developers based on a comparison result obtained by comparing the features of the image with the features of the home; and an information provision unit that causes the electronic device to display the housing image extracted from the image information database based on the features of the image acquired by the feature acquisition unit when a new image is requested by the user's operation ; an information processing device that constitutes the information processing system; an information processing method that includes each of the processes; and a program that causes a computer to execute each of the processes.
Moreover, one aspect of the present invention is an information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, the information processing system including a housing contractor database that stores the characteristics of homes constructed by housing contractors in association with the housing contractors for each of a plurality of housing contractors, an acquisition unit that acquires an image according to the user's preferences from the electronic device, a feature acquisition unit that acquires the features of the image based on the image, a selection unit that selects a housing contractor according to the user's preferences from among the plurality of housing contractors based on a comparison result obtained by comparing the features of the image with the features of the home, and an information provision unit that causes the electronic device to display housing contractor information related to the housing contractor selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected, an information processing device that constitutes the information processing system, an information processing method that includes each of those processes, and a program that causes a computer to execute each of those processes.
Moreover, one aspect of the present invention is an information processing device capable of handling a database that associates the characteristics of homes constructed by a housing developer with the housing developer for each of a plurality of housing developers, the information processing device comprising: an acquisition unit that acquires an image selected according to a user's preference and text information indicating a user's desire for the image; a generation unit that generates a first feature related to the user's preference based on the text information, generates a second feature based on the image, which is a feature of the image related to what is contained in the image, and generates weights for each of the first feature and the second feature based on the text information; and a selection unit that selects a housing developer according to the user's preference from among the plurality of housing developers based on a comparison result of comparing the feature information indicating the user's preference generated based on the weight, the first feature, and the second feature with the features of the home; an information processing system including the information processing device; an information processing method including each of the processes; and a program that causes a computer to execute each of the processes.
Moreover, one aspect of the present invention is an information processing device capable of handling a database that associates the characteristics of homes constructed by a housing developer with the housing developer for each of a plurality of housing developers, and an image information database that stores housing images of each home constructed by each of the plurality of housing developers , and the information processing device comprises an acquisition unit that acquires images according to a user's preferences, a feature acquisition unit that acquires features of the image based on the image, a selection unit that selects a housing developer according to the user's preferences from among the plurality of housing developers based on a comparison result obtained by comparing the features of the image with the features of the home, and an information provision unit that, when a new image re-proposal is requested by the user's operation, causes the housing image extracted from the image information database based on the features of the image acquired by the feature acquisition unit to be displayed on a display unit , an information processing system including the information processing device, an information processing method including each of these processes, and a program that causes a computer to execute each of these processes.
Moreover, one aspect of the present invention is an information processing device capable of handling a database that associates the characteristics of homes constructed by a housing developer with the housing developer for each of a number of housing developers, and the information processing device includes an acquisition unit that acquires an image according to a user's preferences, a feature acquisition unit that acquires features of the image based on the image, a selection unit that selects a housing developer according to the user's preferences from among the multiple housing developers based on a comparison result between the features of the image and the features of the home, and an information provision unit that causes a display unit to display housing developer information related to the housing developer selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected, an information processing system including the information processing device, an information processing method including each of these processes, and a program that causes a computer to execute each of these processes.

The present invention can assist users in choosing a home in an intuitive and easy-to-understand way.

FIG. 1 is a system configuration diagram showing a configuration of an information processing system. FIG. 2 is a diagram illustrating an example of a hardware configuration of a server that configures the information processing system. 2 is a diagram illustrating an example of the functional configuration of a server, a user terminal, and a housing business terminal that constitute an information processing system. FIG. 13 is a diagram illustrating an example of an image displayed on a user terminal. FIG. 1 is a conceptual diagram showing an overview of a tagging method. FIG. 6 is an image diagram showing an overview of a tagging method, which is an example different from the example in FIG. 5 . FIG. 1 is an image diagram for explaining the concept of a user vector. FIG. 5 is a diagram showing an example of an image displayed on a user terminal, which is an example different from the example of FIG. 4 . 9 is a diagram showing an example of an image displayed on a user terminal, which is an example different from the examples of FIGS. 4 and 8 . FIG. 1 is an arrow chart showing an example of the flow of various processes executed by a server and a user terminal. 1 is a block diagram illustrating an example of the configuration of an information processing system. FIG. 13 is a diagram showing an example of generating tags to be added to a house image. FIG. 2 is a simplified diagram showing the stored contents of image information. FIG. 2 is a diagram showing the stored contents of housing business operator information in a simplified form. FIG. 13 is a diagram showing a display example of a selection screen. FIG. 13 is a diagram showing a display example of a selected business operator information screen. FIG. 13 is a diagram showing a display example of a selection screen. 13 is a flowchart showing an example of a housing business operator selection process. FIG. 2 is a diagram showing the stored contents of item information in a simplified manner. FIG. 13 is a diagram showing a display example of an edit screen. 13 is a flowchart showing an example of a house image editing process. FIG. 2 is a simplified diagram showing the stored contents of image information. FIG. 13 is a diagram showing a display example of a selection screen. 13 is a flowchart illustrating an example of an image selection process. FIG. 13 is a diagram showing a display example of a text input screen. FIG. 13 is a diagram showing a display example of a selection screen. 13 is a flowchart showing an example of a housing business operator selection process. FIG. 13 is a diagram showing a display example of a text input screen. FIG. 13 is a diagram showing a display example of a selection screen.

The following describes an embodiment of the present invention with reference to the drawings.

<Overview>
Prior to a specific description using FIG. 1, an example of a service (hereinafter, referred to as "this service") to which an information processing system according to one embodiment of the present invention is applied will be described.

This service is a service that suggests housing businesses that match the user's tastes and new housing images based on, for example, a housing image selected by the user. Note that housing businesses are various businesses related to the construction and design of houses, such as first-class architectural offices, house manufacturers, and construction companies.

In this service, an information processing system according to one embodiment of the present invention (hereinafter referred to as "this system") utilizes techniques such as statistical machine learning to analyze home images selected by the user and home images provided by home builders, select home builders that are close to the user's preferred home, and display them to the user. This allows the user to simply select a home image of their choice, select promising home builders who are likely to be able to build the home of their choice, and request the creation of home plans, design, etc.

Figure 1 is a system configuration diagram showing the configuration of an information processing system according to one embodiment of the present invention.

The information processing system shown in FIG. 1 includes a server 1, a user terminal 2, and housing business operator terminals 3-1 to 3-n (n is any natural number equal to or greater than 1). The server 1, the user terminal 2, and the housing business operator terminals 3-1 to 3-n are connected to each other via a predetermined network N such as the Internet. However, the network N is not a required component, and for example, NFC (Near Field Communication), Bluetooth (registered trademark), LAN (Local Area Network), etc. may be used. In the following explanation, when there is no need to particularly distinguish between the housing business operator terminals 3-1 to 3-n, they will be collectively referred to as housing business operator terminals 3.

Server 1 is composed of a general-purpose personal computer or the like, and is managed by the provider of this service. Server 1 executes various processes related to the provision of this service.

The user terminal 2 is composed of a smartphone, tablet, general-purpose PC, etc., and is used by a user who wishes to use this service. The user terminal 2 accepts various input operations by the user. The user terminal 2 also acquires various information sent from the server 1 and displays it on a display unit (not shown) etc.

The housing provider terminal 3 is configured with a general-purpose personal computer or the like, and is managed by various housing providers who wish to participate in this service. The housing provider terminal 3 transmits image information, etc. of the houses selected by each housing provider to the server 1.

<Hardware Configuration>
FIG. 2 is a diagram showing an example of a hardware configuration of a server constituting an information processing system according to an embodiment of the present invention.

As shown in FIG. 2, the server 1 includes a control unit 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input/output interface 15, an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.

The control unit 11 is composed of a microcomputer including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a semiconductor memory, and executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 to the RAM 13.

The RAM 13 also stores information necessary for the control unit 11 to execute various processes.

The control unit 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input/output interface 15 is also connected to this bus 14. An output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20 are connected to the input/output interface 15.

The output unit 16 outputs image information, audio information, etc. to various liquid crystal displays, speakers, etc. The image information and audio information output by the output unit 16 are output from various liquid crystal displays, speakers, etc. in a state that can be recognized by humans as images and audio.

The input unit 17 is composed of a keyboard, mouse, etc., and is used to input various information.

The storage unit 18 is configured with a hard disk drive (HDD) or a solid state drive (SSD) and stores various data. In this embodiment, for example, various information including various programs and various databases is stored.

The communication unit 19 controls communication with other information processing devices, etc. via a network N including the Internet.

The drive 20 is provided as necessary. Removable media 31, which may be a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately attached to the drive 20. Programs read from the removable media 31 by the drive 20 are installed in the storage unit 18 as necessary. The removable media 31 can also store various data stored in the storage unit 18 in the same way as the storage unit 18.

Here, the hardware configuration of the user terminal 2 and the housing business operator terminal 3 can basically be the same as the hardware configuration of the server 1, so the explanation will be omitted.

Figure 3 shows an example of the functional configuration of a server, a user terminal, and a housing business terminal that constitute an information processing system according to one embodiment of the present invention.

As shown in FIG. 3, in the control unit 11 of the server 1, an image information acquisition unit 60, a tag information acquisition unit 62, a vector generation unit 64, and a housing business selection unit 66 function by executing various programs, etc.

In addition, a housing provider DB400 is provided in one area of the memory unit 18 of the server 1. The housing provider DB400 stores vectors (hereinafter referred to as "provider vectors") corresponding to each housing provider that are generated based on housing images provided by each housing provider, as well as profiles provided by each housing provider. The method by which the system generates vectors for each housing provider can be basically the same as the method for generating vectors for users, which will be described later.

The image information acquisition unit 60 acquires information about an image including a house from a user related to the house as second image information. Specifically, the image information acquisition unit 60 acquires information about a favorite house image selected by the user (hereinafter referred to as "favorite image information") transmitted from the user terminal 2 via the communication unit 19.

The tag information acquisition unit 62 acquires information regarding classification based on a predetermined criterion for the second image information. In other words, the tag information acquisition unit 62 acquires information regarding the tags assigned to each of the favorite images.

Here, a tag is a criterion for classifying each image based on the characteristics of the image, etc. The act of classifying each image based on the tag criteria is called tagging. This tagging is performed automatically for all home images handled by this system based on the results of learning by statistical machine learning that was performed in advance, and the results are stored in advance in a database (not shown).

Therefore, the tag information acquisition unit 62 acquires information about tags attached to favorite images stored in the database (not shown).

Specifically, in this embodiment, for example, the following five types of criteria are adopted as tags.
(1) Location: A classification standard for locations included in each house image. For example, information about locations within the house that are actually included in each house image, such as the exterior of the house as a whole, the entrance, the stairs, the living room, the kitchen, the bathroom, and the rooms, is added as a tag.
(2) Style This is a classification standard for the style of the location included in each house image. If the exterior of the house is a wooden exterior that is close to nature, for example, information such as "natural" or "woody" is added as a tag. Note that, as a specific classification of the style, for example, various style standards known in various fields of architecture, such as "urban,""darktone,""woody,""natural,""Japanesestyle,""vintage,""old folk house,""modern," and "simple" are adopted.
(3) Material: This is a classification standard for the materials contained in each house image. For example, information about characteristic materials such as "wood,""woodgrain,""tile,""concrete/mortar,""steel/iron,""designwallpaper,""glass," and "stainless steel" are added as tags.
(4) Item This is a classification criterion for whether or not each house image includes a wood stove or solar panels. For example, if each house image includes a wood stove or solar panels, it is tagged as "Yes (plus)," and if each house image does not include a wood stove or solar panels, it is tagged as "No (minus)."
(5) Emotions These are classification criteria related to the atmosphere of each house image and the emotions felt when viewing it. Emotion criteria such as "family gathering/warmth,""openness,""luxurious/elegant/luxurious,""calm,""cool/chic,""flashy,""casual/pop,""solid,""sophisticated/stylish," and "cute" are adopted. For example, information regarding the results of evaluating these emotion criteria for each house image on a five-point scale from 0 to 4 may be added to each house image as a tag.

The vector generation unit 64 generates a predetermined vector based on the results of learning using statistical methods. Specifically, the vector generation unit 64 generates a vector (hereinafter referred to as a "user vector") indicating the user's desired housing preferences based on information related to tags attached to favorite images acquired by the tag information acquisition unit 62. Details of the generated vector will be described later with reference to FIG. 7, etc.

The housing business selection unit 66 compares the user vector generated by the vector generation unit 64 with the business vectors of each housing business stored in the housing business DB 400 to determine a housing business that can be recommended to the user. The housing business selection unit 66 may select one housing business corresponding to the business vector that is closest to the user vector, or may select multiple housing businesses corresponding to business vectors that are closest in distance in order of proximity.

In addition, the housing business selection unit 66 transmits information about the selected housing business (hereinafter referred to as "selected business information") to the user terminal 2.

As shown in FIG. 3, in the control unit 200 of the user terminal 2, a display image management unit 220, an image selection reception unit 222, and a housing business information acquisition unit 224 function by executing various programs, etc.

The display image management unit 220 generates and manages various images to be displayed on a display unit (not shown) of the user terminal 2. Specifically, the display image management unit 220 generates various images for the user to select a favorite image and various images for presenting the housing business information (described later) to the user, and controls the display of the images on a display unit (not shown) of the user terminal 2.

The image selection receiving unit 222 receives information on images including houses from users related to houses. Specifically, the image selection receiving unit 222 receives a selection of a favorite house image (hereinafter referred to as a "favorite image") selected by the user. The image selection receiving unit 222 also transmits favorite image information related to the received selection of favorite image to the server 1. Note that a method in which a user actually selects a favorite image will be described later with reference to FIG. 4 etc.

The housing business information acquisition unit 224 acquires the selected business information sent from the server 1 via the communication unit 210.

Figure 4 is a diagram showing an example of an image displayed on a user terminal constituting an information processing system according to one embodiment of the present invention. Specifically, Figure 4 shows an example of an image displayed when a user selects a favorite image.

In the example of FIG. 4, multiple house images selected by the user as favorite images are displayed. Note that the favorite images displayed in FIG. 4 are selected by the user from house images presented in advance.

In the example of FIG. 4, the selection of a favorite image can be easily cancelled by, for example, clicking on a favorite button (e.g., display areas A1 to D1) displayed together with the house image. In this embodiment, the selection (and deletion) of a favorite image is performed, for example, in this manner.

Next, the tagging method will be specifically described with reference to Figures 5 and 6. Figures 5 and 6 are conceptual diagrams showing an overview of the tagging method according to one embodiment of the present invention.

In the example of Figure 5, an example of a house image selected as a favorite image is displayed. As described above, in this system, the target house image is automatically tagged based on the results of learning using statistical machine learning.

Specifically, in the example of Figure 5, the living room is captured in an image of a house, so "living room" is assigned as the location tag. And because the overall structure is natural and wooden (S1 in Figure 5), "natural" and "woody" are assigned as style tags. Similarly, "wood grain" is assigned as the material tag. Because there is no wood stove or solar panels, "none (minus)" is assigned as the item tag. Finally, taking into account the impression given by the overall atmosphere, emotion tags such as "family gathering/warmth" and "calming" may be assigned.

Here, the system can perform tagging at multiple levels, particularly in tagging locations. That is, in addition to tagging a wide area, the system can also assign tags that indicate detailed locations. Specifically, in the example of FIG. 5, it is possible to tag locations included in the image of the house, such as "open-plan kitchen (S2 in FIG. 5)," "dining table (S3 in FIG. 5)," and "stairs (S4 in FIG. 5)."

Similarly, in the example of Figure 6, tags such as "Exterior," "Garden/Courtyard," "Wood Deck," and "Gable Roof" are assigned as location tags, tags such as "Dark Tones" as style tags, tags such as "Siding" as material tags, the tag "None (negative)" as item tags, and tags such as "Cool/Chic" and "Sophisticated/Stylish" as emotion tags.

Figure 7 is an image diagram to explain the concept of user vectors according to one embodiment of the present invention.

In the example of Figure 7, the following numerical values (vectors) are displayed based on the tagging results: location (0.2), style (0.8), material (0.5), item (0.2), and emotion (0.5). Note that the vectors are generated, for example, by converting the tags assigned to each favorite image selected by the user into numerical values and then statistically processing the numerical values. Therefore, the generated user vector can be evaluated as a comprehensive vector representation of the user's housing preferences.

Figure 8 shows an example of an image displayed on a user terminal constituting an information processing system according to one embodiment of the present invention, and is a diagram showing an example different from the example shown in Figure 4.

Specifically, in the example of FIG. 8, new home image candidates are displayed to the user based on the selected favorite image. These new home images are selected by comparing the user vector generated by the system with vectors generated from any home image (such as home images provided by a housing developer or any home image acquired in advance by the system). While reviewing these home images, if the user finds a home image that suits their taste, they can easily add it to their favorite images by clicking a favorite button (e.g., display areas E1 to H1) displayed along with the home image. By adding the newly added favorite image in this way and generating a user vector, the system can improve the accuracy of responding to the user's tastes.

Figure 9 shows an example of an image displayed on a user terminal constituting an information processing system according to one embodiment of the present invention, and shows an example different from the examples in Figures 4 and 8.

Specifically, in the example of FIG. 9, candidates for selected housing businesses are displayed along with information such as the name of the housing business and the area they cover based on the selected favorite image. The user can select the housing business to which they wish to request the construction of a house while checking the displayed candidates for housing businesses. In the example of FIG. 9, the user can check detailed information (e.g., company information, registered profile, etc.) of each housing business by clicking a details confirmation button (e.g., display areas I1 and J1). In the example of FIG. 9, the user can request detailed information for each housing business by clicking a request for information button (e.g., display areas I2, J2, M1) and then clicking an arbitrary decision button (e.g., display area K1). For example, when this request for information is made, the information is sent to each housing business, and each housing business sends the specified information to the user.

Figure 10 is an arrow chart showing an example of the flow of various processes executed by a server and a user terminal constituting an information processing system according to one embodiment of the present invention.

In step S1, the image selection receiving unit 222 of the user terminal 2 receives the selection of a favorite image selected by the user.

In step S2, the image selection receiving unit 222 of the user terminal 2 transmits favorite image information regarding the received favorite image selection to the server 1.

In step S21, the image information acquisition unit 60 of the server 1 acquires the favorite image information selected by the user and transmitted from the user terminal 2 via the communication unit 19.

In step S22, the tag information acquisition unit 62 of the server 1 acquires information about the tags assigned to each of the favorite images.

In step S23, the vector generation unit 64 of the server 1 generates a user vector indicating the user's desired housing preferences based on information about the tags assigned to the favorite images acquired by the tag information acquisition unit 62.

In step S24, the housing business selection unit 66 of the server 1 compares the user vector generated by the vector generation unit 64 with the business vectors of each housing business stored in the housing business DB 400 to determine a housing business that can be recommended to the user.

In step S25, the housing business selection unit 66 of the server 1 transmits selected business information regarding the selected housing business to the user terminal 2.

In step S3, the housing business information acquisition unit 224 of the user terminal 2 acquires the selected business information sent from the server 1 via the communication unit 210.

In step S4, the display image management unit 220 of the user terminal 2 generates various images etc. for presenting the housing business information to the user, and executes control to display the images on a display unit (not shown) of the user terminal 2.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and modifications and improvements that can achieve the object of the present invention are included in the present invention.

Here, we further explain the unique advantages of this system. This system generates vectors based on an image selected by the user (e.g., a favorite image) and can further recommend home images with high similarity. Therefore, unlike existing systems, the user does not need to clearly verbalize the atmosphere, characteristics, etc., and can automatically receive recommendations for new home images and housing businesses by simply selecting a home image of their choice. In addition, this system is updated each time the user selects a new home image as a favorite image, thereby improving the accuracy of the selection. These are the distinctive advantages of this system.

Although the present system has been described as using the generated vectors to determine similarity, this is not limited to this. For example, the present system may generate arbitrary criteria or numerical values based on the tagging results, and use these to select new home images or recommend home developers.

In the above embodiment, the favorite images are all described as being selected from house images pre-installed in the system, but this is not limited thereto. The system may, for example, have a function for accepting uploads of favorite house images from users. In this case, the system may appropriately tag newly uploaded house images.

In the above embodiment, the system has been described as automatically assigning tags based on the results of learning using statistical machine learning, but this is not limited to this. For example, the system may assign tags based on any criteria, regardless of whether statistical machine learning is applied. Also, the system may simply accept input of the results of tagging performed by a person and acquire the results as information.

In the above embodiment, the system has been described as adopting five criteria for tagging: "location," "style," "material," "item," and "emotion." However, this is not limited to these criteria. The system may adopt any other criteria as the tagging criteria.

In the above embodiment, the system has been described as generating a user vector by taking into account all of the criteria of "location," "style," "material," "item," and "emotion," but this is not limited thereto. For example, the system may generate a user vector by taking into account only any of the criteria of "location," "style," "material," "item," and "emotion."

In the above embodiment, the system has been described with examples of criteria for tagging based on the criteria of "location," "style," "material," "item," and "emotion," and examples of tags to be assigned, but these are all examples and are not limiting. The system may tag based on any criteria other than the above examples, and may employ any tags other than the above examples.

Although not described in the above embodiment, the timing at which the user terminal 2 transmits the favorite image information and the timing at which tagging or the like is performed based on the favorite image information acquired by the server 1 are both arbitrary. For example, the present system may adopt a specification in which various processes are performed when a display area (not shown) is clicked by the user, or a specification in which various processes are performed constantly when various conditions are updated by the user's operation.

Although the above-mentioned embodiments (particularly the embodiments of Figures 5 and 6) have been merely simplified, the present system can employ results obtained by hierarchical learning, particularly in the tagging of locations. Specifically, by performing learning while taking into account the tagging content, including the hierarchy, shown in the examples of Figures 5 and 6, the present system can further improve the learning accuracy.

Although the above-mentioned embodiments (particularly the embodiments of FIG. 4 and FIG. 8) have been described simply, the present system may have more extensive user management functions than simply allowing the addition and deletion of favorite images. Specifically, for example, the present system may have a customer information management function, a work progress management function, a reservation function for events, etc., a communication function such as SNS, a function for accepting comments from users (information regarding preferences and requests for housing, the planned construction site, budget, etc.), etc.

Although not explained in the above embodiment, the tagging criteria of this system do not need to be treated equally for all criteria, and weighting may be adopted. For example, in terms of emotion criteria, the priority of a tag such as "family gathering/warmth" may be set relatively high compared to the priority of a tag such as "calming."

Although not described in the above embodiment, the present system may also have a function for interacting with the user. Specifically, for example, the present system may be able to improve the accuracy of matching by presenting questions to the user and having the user input answers to those questions.

Although the above embodiment has been a simplified explanation, the vectors generated by this system may be quantified in any manner. Specifically, this system may have numerical values as more specific tag items, such as location vectors such as the living room (0.7), dining room (0.6), kitchen, etc., style vectors such as Western style (1.0), Japanese style (0.0), and emotion vectors such as cool (1.0), cute (0.0), etc.

Although not described in the above embodiment, the present system may use the following elements, for example, to select a new home image or recommend a home builder.
(1) The system may use image features, for example, to select new home images or recommend home builders.
(2) The system may use text (such as letters and numbers) contained in an image, for example, to select a new home image or recommend a home builder.
When using these elements, the system may use any of them as a criterion for tagging.

The above-mentioned series of processes can be performed by either hardware or software.

In other words, the functional configurations in FIG. 3 and the like are merely examples and are not particularly limited.
That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not limited to the example in Fig. 3 etc. Furthermore, the location of the functional block is not limited to the example in Fig. 3 etc. and may be arbitrary.

Furthermore, a single functional block may be configured as hardware alone, software alone, or a combination of both.

In addition, the number and users of the various hardware components that make up this system are optional, and the system may also be configured to include other hardware.

For example, the various hardware components and the number of users that make up this system are not limited to the above examples and are arbitrary. Furthermore, this system may be configured by including and combining not only the various hardware components described above, but also other hardware, etc.

For example, the system may be configured to execute various processes via the Web, etc., as in a so-called cloud-based system, and send the results of the processes to various hardware.

When a series of processes is executed by software, the programs that make up the software are installed on a computer or other device from a network or recording medium.

The computer may also be a computer that is built into dedicated hardware. The computer may also be a computer that is capable of executing various functions by installing various programs.

In other words, for example, any computer, any mobile terminal such as a smartphone, etc. may be freely adopted for the various hardware in the above-mentioned embodiments.

In addition, the recording medium containing such a program may be configured not only as a removable medium (not shown) provided separately from the device body in order to provide the program to the user, but also as a recording medium provided to the user in a state where it is already installed in the device body.

In this specification, the steps of writing a program to be recorded on a recording medium may be executed in parallel or individually, as well as in chronological order. Also, some of the steps described in the above embodiment may be omitted.

In addition, in this specification, the term "system" refers to an overall device that is composed of multiple devices, multiple means, etc.

The effects of this embodiment are achieved even when these other embodiments are adopted. This embodiment can also be combined with other embodiments, and other embodiments can also be combined with each other as appropriate.

In other words, an information processing device to which the present invention is applied is sufficient if it has the following configuration, and can take a variety of different forms.

That is, the first aspect of the present invention is an information processing device for processing images relating to a house, comprising: a storage means for storing information relating to an image including the house from a construction company relating to the house as first image information; a second image information acquisition means for acquiring information relating to an image including the house from a user relating to the house as second image information; and a classification information acquisition means for acquiring information relating to a classification based on a predetermined criterion relating to the second image information as classification information.

In addition, in the first aspect, the information processing device may further include a first determination means for determining construction businesses that can be recommended to the user based on the classification information acquired by the classification information acquisition means.

In addition, in the first aspect, the information processing device may further include a second determination means for determining new house image candidates that can be presented to the user based on the classification information acquired by the classification information acquisition means.

In addition, in the first aspect, classified information can be obtained based on at least one of location-related criteria, style-related criteria, material-related criteria, item-related criteria, and emotion-related criteria.

In addition, in the first aspect, the classification information acquisition means can acquire information regarding tags assigned to images related to the second image information based on the results of learning using a statistical method.

In addition, in the first aspect, the classification information acquisition means can generate a numerical value that serves as a decision criterion by taking into account the results of learning using statistical methods.

In addition, in the first aspect, the classification information acquisition means can perform learning including a learning method that utilizes a hierarchical structure.

A second aspect of the present invention is an information processing method executed by a computer for processing images relating to a house, the information processing method including a storage means for storing information relating to an image including the house from a construction company relating to the house as first image information, a second image information acquisition means for acquiring information relating to an image including the house from a user relating to the house as second image information, and a classification information acquisition means for acquiring information relating to a classification based on a predetermined criterion relating to the second image information as classification information.

A third aspect of the present invention is a program for causing a computer for processing images relating to a house to execute processing including a storage means for storing information relating to an image including the house from a construction company relating to the house as first image information, a second image information acquisition means for acquiring information relating to an image including the house from a user relating to the house as second image information, and a classification information acquisition means for acquiring information relating to a classification based on a predetermined criterion relating to the second image information as classification information.

This allows users to intuitively and easily select the housing provider of their choice.

As described above, in this embodiment, it is possible to realize various services for proposing housing businesses and new home images that match the user's tastes based on the home image selected by the user. Various aspects of these services will be described below. Note that parts common to the above-described embodiments will be omitted from illustrations and explanations as appropriate, or will be described with the same reference numerals.

[Example of configuration of information processing system]
Fig. 11 is a block diagram showing a configuration example of an information processing system IS1. The information processing system IS1 corresponds to the information processing system shown in Fig. 1 and Fig. 3. The server 1 corresponds to the server 1 shown in Fig. 1 to Fig. 3. The user terminal 2 corresponds to the user terminal 2 shown in Fig. 1 and Fig. 3.

The information processing system IS1 is composed of a server 1 and a user terminal 2. For example, the server 1 and the user terminal 2 are connected via a predetermined network N (see FIG. 1). Note that communication between these devices is performed using wired communication or wireless communication. Furthermore, communication between these devices may be performed directly between the devices in addition to communication via the network N.

[Server configuration example]
The server 1 includes a communication unit 310, a control unit 320, and a storage unit 330. The communication unit 310, the control unit 320, and the storage unit 330 correspond to the units with the same names shown in Figs.

The communication unit 310 exchanges various types of information with other devices using at least one of wired communication and wireless communication. For example, the communication unit 310 executes a reception process for receiving various types of information transmitted from the user terminal 2, a transmission process for transmitting various types of information to the user terminal 2, etc.

The control unit 320 includes an information acquisition unit 321, a tag generation unit 322, a vector generation unit 323, an extraction unit 324, and an information provision unit 325.

The information acquisition unit 321 receives various information sent from the user terminal 2 and supplies each piece of information that has been received to each part.

The tag generating unit 322 generates tags for information (e.g., house images, other images) output from the information acquiring unit 321, and outputs information (tag information) related to the generated tags to the vector generating unit 323. For example, when the information output from the information acquiring unit 321 is a house image stored in an image information DB (Data Base) 340, the tag generating unit 322 acquires tag information 343 (see FIG. 13) associated with the house image. When the information output from the information acquiring unit 321 is another image, the tag generating unit 322 can generate tags based on tag information stored in the tag information DB 360. When the information output from the information acquiring unit 321 is another image, the tag generating unit 322 can generate tags using an AI (Artificial Intelligent) model. The tag generating method will be described in detail with reference to FIG. 12, etc.

The vector generation unit 323 generates a user vector according to the preferences of the user of the user terminal 2 based on the tag information (vector components) output from the tag generation unit 322, and outputs the generated user vector to the extraction unit 324. The method of generating the user vector will be described in detail with reference to FIG. 15 etc.

The extraction unit 324 extracts desired information from each DB stored in the storage unit 330 based on the information output from each unit, and outputs each extracted information to the information provision unit 325. For example, when a user vector is output from the vector generation unit 323, the extraction unit 324 compares the user vector with the housing business vector stored in the housing business DB 350, and extracts a housing business that meets the preferences of the user of the user terminal 2 based on the similarity of each vector. Also, for example, when the provision of a new housing image is requested from the user terminal 2, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the tag information 343 (vector) stored in the image information DB 340, and extracts a housing image that meets the preferences of the user of the user terminal 2 based on the similarity of each vector. The method of generating each of these user vectors will be described in detail with reference to FIG. 15, etc.

The information providing unit 325 executes transmission control to transmit information output from each unit to the user terminal 2.

The memory unit 330 is a storage medium that stores various types of information. For example, the memory unit 330 stores various types of information required for the control unit 320 to perform various processes (e.g., a control program, image information 340 (see FIG. 13), a housing business DB 350 (see FIG. 14), a tag information DB 360, and an AI model). The memory unit 330 also stores various types of information acquired via the communication unit 310. For example, a ROM, a RAM, an SRAM (Static Random Access Memory), a HDD, an SSD, or a combination of these can be used as the memory unit 330.

Image information DB340 is a database for storing house images provided by housing developers in association with the feature quantities of multiple tags generated based on these images. Image information DB340 will be described in detail with reference to FIG. 13.

Housing business DB350 is a database for storing various information related to housing businesses in association with each other. Housing business DB350 will be described in detail with reference to FIG. 14.

The tag information DB 360 is a database for storing information used when tag information is generated by the tag generation unit 322. For example, when tag information is generated using an AI model, information about the AI model is stored.

[Example of user terminal configuration]
The user terminal 2 includes a communication unit 410, a position information acquisition unit 420, an image acquisition unit 430, a sound acquisition unit 440, a control unit 450, a UI (User Interface) unit 460, and a storage unit 470. The user terminal 2 is realized by, for example, a device such as an information processing device such as a smartphone, a tablet terminal, or a personal computer, or an electronic device. Note that each unit included in the user terminal 2 is an example, and some of these may be omitted, or other members may be added. For example, when the user terminal 2 is a personal computer, the position information acquisition unit 420 and the image acquisition unit 430 can be omitted or can be external devices.

The communication unit 410 exchanges various types of information with other devices using wired or wireless communication based on the control of the control unit 450.

The location information acquisition unit 420 acquires location information on the location where the user terminal 2 is located, and outputs the acquired location information to the control unit 450. The location information acquisition unit 420 can be realized, for example, by a GPS receiver that receives a GPS (Global Positioning System) signal and calculates location information based on the GPS signal. The calculated location information includes data on the position, such as latitude, longitude, and altitude, at the time of receiving the GPS signal. A location information acquisition device that acquires location information by other location information acquisition methods may be used. For example, location information may be derived using access point information by a wireless LAN present in the vicinity, and a location information acquisition device that acquires this location information may be used. For example, location information may be derived using location information of a base station used for calls and communications, and a location information acquisition device that acquires this location information may be used. For example, location information may be derived using a position estimation technology by a navigation function, and a location information acquisition device that acquires this location information may be used. For example, the device's position can be estimated based on sensor information from various sensors (e.g., acceleration sensors, gyro sensors) and map information.

The image acquisition unit 430 captures an image of a subject and generates an image (image data) based on the control of the control unit 450, and outputs the generated image to the control unit 450. The image acquisition unit 430 is composed of, for example, an imaging element (image sensor) that receives light from the subject collected by a lens (not shown), and an image processing unit that performs predetermined image processing on the image data generated by the imaging element. For example, a CCD (Charge Coupled Device) type or a CMOS (Complementary Metal Oxide Semiconductor) type imaging element can be used as the imaging element.

The sound acquisition unit 440 acquires sounds around the user terminal 2 under the control of the control unit 450, and outputs sound information related to the acquired sounds to the control unit 450. As the sound acquisition unit 440, for example, one or more microphones or sound acquisition sensors can be used.

The control unit 450 controls each unit based on various programs stored in the storage unit 470. The control unit 450 is realized by, for example, a processing device such as a CPU. Each process executed by the control unit 450 will be described in detail with reference to Figures 15 to 18.

The storage unit 470 is a storage medium that stores various types of information. For example, the storage unit 470 stores various types of information (e.g., a control program, an image information DB 800 (see FIG. 22), a housing business application, and device identification information) required for the control unit 450 to perform various processes. The storage unit 470 also stores various types of information acquired via the communication unit 410. For example, a ROM, a RAM, an SRAM, a HDD, an SSD, or a combination of these can be used as the storage unit 470. The storage unit 470 may be a removable storage unit or may be a storage unit built into the user terminal 2. The user terminal 2 may also be provided with both a removable storage unit and a built-in storage unit.

The UI unit 460 includes a reception unit 461 and an output unit 462.

The reception unit 461 receives various operations from the user and outputs the received operation contents to the control unit 450. The reception unit 461 and the output unit 462 may be configured as a touch panel that allows the user to input operations by touching or approaching the display surface with their finger, or may be configured as a separate user interface. When configured as a separate user interface, various operation members such as switches, buttons, and keyboards can be used as the reception unit 461. Also, a user interface may be created in which an image corresponding to the operation unit is projected onto a wall or the like, and the image can be used to perform operations (for example, by the user pointing at it). Also, a sound acquisition unit 440 or the like that receives various operations based on the voice emitted by the user may be used as the reception unit 461.

The output unit 462 outputs various information based on the control of the control unit 450. For example, the output unit 462 can be configured with a display unit and a sound output unit. This display unit displays various images based on instructions from the control unit 450. As the display unit, for example, a display panel such as an organic EL (Electro Luminescence) panel or an LCD (Liquid Crystal Display) panel can be used. The sound output unit outputs various sounds based on instructions from the control unit 450. As the sound output unit, for example, one or more speakers can be used. Note that the display unit, sound output unit, sound acquisition unit 440, and reception unit 461 are examples of user interfaces, and some of these may be omitted, or other user interfaces may be used.

[Tagging example]
Here, tagging that is assigned to image information stored in image information DB 340 will be described. For example, image information uploaded to server 1 by a housing developer using the service in this embodiment is stored in image information DB 340. That is, image information uploaded to server 1 by the housing developer is registered as a house image. When a house image is registered in this manner, a tag assignment process is executed for the house image. Then, the registered house image and the tag generated for it are associated with each other and stored in image information DB 340.

As described above, a tag is a criterion for classifying images (house images) based on the properties of each image, and also refers to the item that serves as the criterion for the classification. In other words, tagging a house image can be referred to as giving meaning to the house image. A tag also refers to an item that serves as the criterion for extracting the features (feature amount) of an image (house image). A tag can also be referred to as information for identifying an image (house image). Note that the label shown below also refers to information that represents the features, attributes, etc. of an image (house image). In this way, a tag and a label have roughly the same or similar meanings, and therefore, below, a tag and a label may be used to mean the same thing.

In addition, in this embodiment, information in which a tag and a corresponding feature (feature amount) are associated will be described as tag information. However, tag information can also be referred to as a tag dictionary, metadata, associated information, additional information, etc. In addition, in this embodiment, an example will be described in which a quantified feature amount is used as a feature corresponding to a tag.

Figure 12 (A) shows an example of tags that can be added to a house image. As mentioned above, this shows an example of using the tags "location," "style," "material," "item," and "emotion."

A location tag is a label that indicates a location included in a house image. Note that the same house image may contain multiple locations. For example, if the same house image contains an "entrance" and a "staircase," then multiple locations will be included. There may also be a hierarchical structure with a certain location (e.g., the living room) and other locations (a split-level floor) linked to it. For example, location tags related to locations included in a house image may include the exterior of the entire house, the entrance, the stairs, the living room, the kitchen, the bathroom, rooms, etc. Note that location tags may be expressed in multiple levels (e.g., five levels from 0 to 4, a range from 0 to 1), for example.

A style tag means a label that indicates the style of a place included in a house image. Note that a house or the like included in the same house image may have multiple styles. For example, if the same house image includes "natural" and "Japanese style," multiple styles will be included. Also, for example, if the exterior of a house included in a house image is a wooden exterior that is close to nature, information such as "natural" and "woody" is assigned as a style tag. Also, as described above, it is possible to adopt various style standards known in various fields of architecture, such as "urban," "dark tone," "woody," "natural," "Japanese style," "vintage," "old folk house," "modern," and "simple," as style tags. Note that style tags can be expressed in multiple stages (for example, five stages from 0 to 4, a range of 0 to 1). For example, when expressed in a range of 0 to 1, if a modern house image includes woody elements, evaluation values such as modern: 0.7 and woody: 0.4 can be assigned.

A material tag is a label that indicates the material contained in a house image. Note that the same house image may contain multiple materials. For example, if the same house image contains "wood" and "glass," it will contain multiple materials. For example, information about characteristic materials such as "wood," "wood grain," "tile," "concrete/mortar," "steel/iron," "designer wallpaper," "glass," and "stainless steel" that are contained in the house image is assigned as a material tag. Note that material tags can also be expressed in multiple levels (for example, five levels from 0 to 4, a range from 0 to 1).

Item tags refer to labels that represent items such as a wood stove or solar panels. Note that multiple items may be included in the same house image. For example, if the same house image includes a "wood stove" and a "solar panel", multiple items will be included. For example, if a wood stove is included in the house image, an item tag indicating "wood stove included" is added as a wood stove item, and if a solar panel is included in the house image, an item tag indicating "solar panel included" is added as a solar panel item. On the other hand, if a wood stove is not included in the house image, an item tag indicating "no wood stove included" is added as a wood stove item, and if a solar panel is not included in the house image, an item tag indicating "no solar panel included" is added as a solar panel item. Note that it is also possible to add labels that represent items other than wood stoves and solar panels as item tags. Also, item tags can be expressed in multiple stages (for example, five stages from 0 to 4, a range from 0 to 1).

An emotion tag is a label that represents the mood and visual emotion of a house image. This emotion tag can be expressed in multiple stages (for example, five stages from 0 to 4, a range of 0 to 1). For example, emotion standards such as "family gathering/warmth," "openness," "luxurious/elegant/luxurious," "calm," "cool/chic," "flashy," "casual/pop," "solid," "sophisticated/stylish," and "cute" are adopted as emotions related to house images. Then, for example, information regarding the results of evaluating each of these emotion standards in the range of 0 to 1 for the house image is assigned as an emotion tag.

[Example of tagging done by humans]
When a housing business operator uploads a house image to the server 1, it is possible to execute a tagging process in which tags are manually assigned to the house image. For example, a person can judge the location, style, material, items, and emotions related to the house image contained in the house image, and the above-mentioned tag information can be assigned to the house image based on the judgment result. However, when tags are assigned manually in this manner, personnel with specialized knowledge are required at the time of registration, and it is often difficult to obtain such personnel. Therefore, an example of executing the tagging process using artificial intelligence (AI) or the like is shown below. Note that in this embodiment, an example of assigning tags mainly using AI is described.

[Example of tagging using AI]
[Examples of images and teacher labels used for learning]
FIG. 12B shows an example of an image IMG1 used for learning. Note that image IMG1 is an example of an image (e.g., a house image) used for learning. That is, when performing a tagging process that uses AI to tag a house image, various images such as image IMG1 can be used as teacher data. Note that image IMG1 is a simplified image for ease of explanation.

In addition, in this embodiment, an example of generating an AI model for each tag pattern (location, style, material, item, emotion) is shown. That is, an example of generating five AI models that have learned each of location, style, material, item, and emotion is shown. Note that the generation of AI models is not limited to this, and one AI model may be generated for all tag patterns (location, style, material, item, emotion), or multiple AI models may be generated for two to four patterns out of all tag patterns (location, style, material, item, emotion).

When learning image IMG1, specific tags are assigned to image IMG1 as teacher labels. Then, the tag patterns (location, style, material, item, emotion) that correspond to each AI model to be trained are extracted and used for learning.

An example of a teacher label for image IMG1 is shown below. That is, an example of a tag to be added as a teacher label to image IMG1 used for learning is shown below. Note that the contents shown below ((1) to (5)) are simplified contents for ease of explanation, and are not limited to these.
(1) Location tags: Living room: Other, Split floor (2) Style tags: Natural, Woody (3) Material tags: Wood/wood grain, Carpet (4) Item tags: Not applicable (5) Emotion tags:
Family gatherings and warmth: 0.875
High-class, elegant, luxurious: 0.125
Cool/chic: 0.375
Casual Pop: 0.5
Sophisticated/Stylish: 0.75
Openness: 0.375
Calm: 0.75
Flashy: 0
Heavy feeling: 0
Cute: 0.375

As shown in FIG. 12A, the tags to be assigned are predetermined. Each tag is a multi-label that allows overlapping. For example, each of the location tag, style tag, and material tag is set to "1" if the corresponding item is included in image IMG1, and set to "0" if the corresponding item is not included in image IMG1. That is, in the above example, since image IMG1 contains "living room" and "skip floor", (1) the location tag "living room" is assigned a "1", the other location tag "skip floor" is assigned a "1", and the other location tags are assigned a "0". Furthermore, since image IMG1 contains "natural" and "woody", (2) the style tags "natural" and "woody" are assigned a "1", and the other style tags are assigned a "0". Furthermore, since image IMG1 contains "wood/wood grain" and "carpet," (3) the material tags "wood/wood grain" and "carpet" are each assigned a "1," and the other material tags are assigned a "0."

Also, for example, an emotion tag is a tag that quantifies the degree to which it applies, between 0 and 1. In this case, an emotion tag of 0 means that the degree of the corresponding emotion is the lowest, and an emotion tag of 1 means that the degree of the corresponding emotion is the highest.

Also, for example, an item tag is a tag that indicates whether or not a corresponding thing is present in an image. That is, the item tag is set to "1" if the corresponding thing is contained in image IMG1, and is set to "0" if the corresponding thing is contained in image IMG1. That is, in the above example, since image IMG1 does not contain an object that corresponds to the item tag, a "0" is assigned to each item in the (4) item tag. That is, it is assigned that image IMG1 does not contain an object that corresponds to the item tag.

Similarly, for images other than image IMG1 used for learning (e.g., house images), specific tags are assigned as teacher labels and used for learning.

Note that, while an example has been shown in which a value of 0 or 1 is assigned to each of the location tag, style tag, material tag, and item tag, at least one of these can also be expressed in multiple levels (for example, in the range of 0 to 1) in the same way as the emotion tag. As mentioned above, for example, an image of a modern house with woody elements can have style tags such as modern: 0.7, woody: 0.4, etc.

[Example of generating an AI model used for learning]
An AI model (for example, a machine learning model generated by machine learning) is generated using the above-mentioned teacher data (for example, image IMG1) and teacher labels (for example, each tag information). The learning shown in this embodiment means finding a regularity behind a large amount of data based on these data. The AI model generated by the learning shown in this embodiment is generated by various learning algorithms. For example, the teacher data (for example, image IMG1) and teacher labels (for example, each tag information) are read and learned in advance, and this AI model can be used for tagging processing. For example, CNN (Convolutional Neural Network), ViT (Vision Transformer), etc. can be used as the AI model. These can be executed by installing a predetermined application on the server 1.

[Example of output from a trained AI model]
As described above, an AI model is generated for each tag pattern (location, style, material, item, emotion), so five AI models are prepared for each tag pattern. For this reason, the house images registered in the server 1 are input to the five AI models, respectively. As a result, the five AI models output the results of tagging for location, style, material, item, and emotion, respectively. In this way, using the five AI models, it is possible to output the input image with tag information related to the set tag attached. It is also possible to output the input image other than the house image with tag information related to the set tag attached.

[Example of configuration of image information DB]
FIG. 13 is a simplified diagram showing the contents of the image information stored in the image information DB 340. As shown in FIG.

Image information DB340 is a database for storing house images provided by housing developers in association with the features of multiple tags generated based on these images.

Specifically, image information 342, tag information 343, and housing business identification information 344 are stored in image information DB 340 in association with image identification information 341.

Image identification information 341 is identification information for identifying each home image prepared by the housing developer to be provided to the user. Note that, for ease of explanation, FIG. 13 shows an example in which only numerical values are stored as identification information in image identification information 341, but other identification information may be used.

Image information 342 is information about images prepared by the housing developer to provide to users. Note that, for ease of explanation, FIG. 13 shows an example in which only images are stored in image information 342, but various attribute information stored in association with the images may also be included.

Tag information 343 is tag information generated for the image stored in image information 342. Specifically, tag information assigned by the tag assignment process described above is stored. Among this tag information, emotion and style are used when generating a user vector. Among this tag information, location, material, and item can be used to narrow down searches for housing images, housing businesses, etc.

For example, when displaying house images stored in the image information DB 340 on the UI unit 460 (see FIG. 11) of the user terminal 2, it is possible to use it to narrow down house images that the user likes. For example, if a user is particular about living rooms, it is possible to display house images limited to "living rooms" on the UI unit 460 based on the location tag associated with the house images. In this case, the user can select a house image that he or she likes from among the house images displayed on the UI unit 460 (house images with "1 (or a number greater than 0)" assigned to the location tag "living room"). This makes it possible to generate a user vector (described later) that better reflects the user's preferences. Also, for example, if a user likes rooms with stoves, it is possible to display house images limited to house images with "stoves" on the UI unit 460 based on the item tag associated with the house images. In this case, the user can select a house image that he or she likes from among the house images displayed on the UI unit 460 (house images with "1 (or a number greater than 0)" assigned to the item tag "stove". This makes it possible to generate a user vector that better reflects the user's preferences. For example, if a user likes rooms with exposed concrete, it is possible to display on the UI unit 460 house images limited to house images with "concrete" based on the material tag associated with the house image. In this case, the user can select a house image that he/she likes from among the house images displayed on the UI unit 460 (house images with "1 (or a value greater than 0)" assigned to the material tag "concrete"). This makes it possible to generate a user vector that better reflects the user's preferences. For example, if a user wants to be recommended only home builders who handle solar panels, it is possible to display on the UI unit 460 house images limited to house images with "solar panels" based on the item tag associated with the house image. In this case, the user can select a house image that he/she likes from among the house images displayed on the UI unit 460 (house images with "1 (or a value greater than 0)" assigned to the item tag "solar panel"). This makes it possible to more appropriately recommend home builders who handle solar panels. Note that illustrations and other explanations regarding the narrowing down of searches for home images, home builders, etc. will be omitted.

The housing business identification information 344 is identification information for identifying the housing business that provided the image stored in the image information 342. Note that, for ease of explanation, FIG. 13 shows an example in which the name of the housing business is stored in the housing business identification information 344, but other information capable of identifying the housing business (e.g., a serial number, a code) may also be used.

Note that the information shown in FIG. 13 is an example of information stored in the image information DB 340, and other information may be stored in association with the information, and some of the information may be omitted as necessary.

[Example of the configuration of the housing business information DB]
14 is a simplified diagram showing the contents of the housing developer information stored in the housing developer DB 350. The housing developer DB 350 corresponds to the housing developer DB 400 shown in FIG.

Housing business DB350 is a database for storing various information related to housing businesses in association with each other.

Specifically, the housing business identification information 351, the profile information 352, and the tag information 353 are associated with each other and stored in the housing business DB 350.

The housing business operator identification information 351 is identification information for identifying the housing business operator. Note that the housing business operator identification information 351 corresponds to the housing business operator identification information 344 shown in FIG. 13. Also, similar to the example shown in FIG. 13, in order to facilitate explanation, FIG. 14 shows an example in which the name of the housing business operator is stored in the housing business operator identification information 351, but other information capable of identifying the housing business operator (e.g., a serial number, a code) may be used.

Profile information 352 is various information related to the housing business whose identification information is stored in housing business identification information 351. For example, the address of the housing business, introduction information about the housing business, etc. are stored. This profile information can be edited by the housing business as appropriate.

The tag information 353 is tag information assigned to a housing business operator whose housing business operator identification information is stored in the housing business operator identification information 351. Only information related to "style" and "emotion" from the tag information 343 shown in FIG. 13 is stored as this tag information. In addition, this tag information may be set appropriately by the housing business operator, or may be stored based on the calculation results of each tag information assigned to multiple housing images uploaded to the server 1 by the housing business operator. For example, it is possible to store the average value of each element related to "style" and "emotion" from each tag information assigned to multiple housing images uploaded to the server 1 as each element related to "style" and "emotion". In addition, the calculation method of each element related to "style" and "emotion" may be the same as the calculation method of the user vector described later. This tag information is used when selecting a housing business operator to be recommended to the user.

Note that the information shown in FIG. 14 is an example of information stored in the housing business operator DB 350, and other information may be stored in association with it, and some of this information may be omitted as necessary.

[Example of selecting favorite images]
15 is a diagram showing an example of a selection screen 500 displayed on the UI unit 460 of the user terminal 2. The selection screen 500 is displayed on the UI unit 460 based on the control of the control unit 450.

The selection screen 500 displays a house image display area 501 that displays multiple house images 503 to 508, and a decision button 502. Specifically, the information providing unit 325 of the server 1 transmits selection screen information for displaying the selection screen 500 to the user terminal 2 based on a request from the user terminal 2. When the user terminal 2 receives the selection screen information, the control unit 450 causes the UI unit 460 to display the selection screen 500 based on the received selection screen information. Note that other house images can be displayed in the house image display area 501 based on a user operation.

In addition, when the reception unit 461 of the UI unit 460 receives a user operation, the control unit 450 executes various processes based on the user operation. For example, when the user performs a selection operation (e.g., a touch operation) to select a favorite house image displayed in the house image display area 501, the favorite house image selected by the selection operation is placed in a selected state. For example, the favorite house image selected by the selection operation is displayed in a different manner from other house images (e.g., the favorite button 509 (star area) is painted black, surrounded by a thick line, or colored in a specific color) so that the user can understand the selection state. FIG. 15 shows an example in which the favorite button 509 (star area) is painted black to place the house image 503 in a selected state. Note that after the user performs a selection operation, it is possible to display only the house image selected by the user. This example is shown in FIG. 4. In this case, it is also possible to cancel the selection state of the house image by performing a selection operation (e.g., a click operation) on the favorite button (star area) as shown in FIG. 4.

Furthermore, after one or more house images are selected by the user, if the user performs a selection operation (e.g., a touch operation) to select the decision button 502, the control unit 450 transmits image selection information related to the one or more house images selected by the user to the server 1. When the server 1 receives the image selection information, the information acquisition unit 321 outputs the received image selection information to the tag generation unit 322 and the vector generation unit 323.

[Example of user vector generation]
The tag generation unit 322 acquires, from the image information DB 340, tag information associated with one or more house images corresponding to the image selection information transmitted from the user terminal 2. Specifically, the tag generation unit 322 acquires, from tag information 343 in the image information DB 340, one or more pieces of tag information associated with image identification information 341 corresponding to the one or more house images corresponding to the image selection information, and outputs the acquired tag information to the vector generation unit 323.

Next, the vector generation unit 323 generates a user vector for the user who transmitted the image selection information based on the tag information output from the tag generation unit 322. Note that when there is one house image corresponding to the image selection information transmitted from the user terminal 2, it is possible to use the tag information associated with that one house image as the user vector.

Here, in order to properly acquire the preferences of a user who has viewed a house image, it is considered that the appearance of the house image and the impression obtained from the house image are important. Therefore, in this embodiment, an example is shown in which a user vector is generated according to the user's preferences using information on style tags and emotion tags from among the various tag information. As described above, information on location tags, material tags, and item tags can be used to narrow down searches. Note that the example of generating a user vector shown here is just one example, and a user vector may be generated using at least one of style tags, emotion tags, and other tag information (tag information on location tags, material tags, and item tags).

For example, the vector generation unit 323 can perform an addition process on the elements that make up each of the style tags and emotion tags among the vectors corresponding to the tag information output from the tag generation unit 322, and generate a user vector by dividing the result of the addition by the number of vectors that were added.

For example, assume that the user has selected house image 3. Of the tag information (tag vector) associated with these three selected house images, the elements that make up the style tag are [a1, ... , an1], [b1, ... , bn1], [c1, ... , cn1], and the elements that make up the emotion tag are [x1, ... , xn2], [y1, ... , yn2], [z1, ... , zn2]. Here, n1 means the number of items in the style tag, and n2 means the number of items in the emotion tag. The components (numeric values) of each element are values in the range from 0 to 1.

In this case, the vector generation unit 323 generates a user vector for the style tag by dividing the sum of the components of the three multidimensional vectors ([a1, ..., an1], [b1, ..., bn1], [c1, ..., cn1]) by 3, i.e. [(a1+b1+c1)/3, ..., (an1+bn1+cn1)/3]. Similarly, the vector generation unit 323 generates a user vector for the emotion tag by dividing the sum of the components of the three multidimensional vectors ([x1, ..., xn2], [y1, ..., yn2], [z1, ..., zn2]) by 3, i.e. [(x1+y1+z1)/3, ..., (xn2+yn2+zn2)/3]. Then, the vector generation unit 323 sets the combination of the user vector related to the style tag and the user vector related to the emotion tag as the user vector. Specifically, the user vector is [(a1+b1+c1)/3, ..., (an1+bn1+cn1)/3, (x1+y1+z1)/3, ..., (xn2+yn2+zn2)/3]. In this case, each element of the user vector corresponds to each element of the business operator vector.

In this way, the average value of multiple house images selected by the user can be used as the user vector. However, this embodiment is not limited to this, and it is possible to generate a user vector using other calculation methods. For example, it is possible to adopt a calculation method in which the average is calculated after using the softmax function only for the style tag of each house image. Also, for example, it is possible to adopt a calculation method in which the average is calculated using the Max function. Also, in this embodiment, an example is shown in which a user vector is generated using elements that constitute each of the style tag and emotion tag, but this is not limited to this, and the user vector may be generated using at least one element other than the style tag and emotion tag, or the user vector may be generated using other tag information.

[Example of housing business selection]
Based on the user vector generated by the vector generation unit 323, the extraction unit 324 extracts housing businesses that meet the user's preferences corresponding to the user vector, from among the housing businesses whose housing business information is stored in the housing business DB 350. For example, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the business vector stored in the tag information 353 of the housing business DB 350, and extracts housing businesses that meet the user's preferences based on the comparison result.

As described above, the business vector (tag information 353) is composed of a combination of a style vector and an emotion vector. Here, an example is shown in which the business vector (tag information 353) of a certain business is expressed as [S1, ..., Sn1, E1, ..., En2]. Here, n1 means the number of items in the style tag, and n2 means the number of items in the emotion tag. Furthermore, the components (numeric values) of each element correspond to the values of the tag information 353.

In this case, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the operator vector (tag information 353) [S1, ..., Sn1, E1, ..., En2] of a certain operator, and determines the similarity between the two based on the comparison result. For example, if the user vector is [(a1+b1+c1)/3, ..., (an1+bn1+cn1)/3, (x1+y1+z1)/3, ..., (xn2+yn2+zn2)/3], the extraction unit 324 compares the user vector with the operator vector [S1, ..., Sn1, E1, ..., En2] of a certain operator.

For example, the extraction unit 324 calculates the cosine similarity between the user vector and each business vector for each business vector. A publicly known calculation method can be used to calculate the cosine similarity. When calculating the cosine similarity, it is preferable to normalize each vector. Then, the extraction unit 324 extracts the housing business corresponding to the business vector with the highest cosine similarity (i.e., the smallest cosine value with the user vector) as the housing business according to the user's preference. Note that a high cosine similarity means that the distance is short in vector space. In addition, when multiple housing businesses are provided to the user, the extraction unit 324 can extract a predetermined number (for example, about 2 to 10) of housing businesses with high calculated cosine similarity. The number of housing businesses extracted in this way may be specified based on a user operation using the user terminal 2.

The housing business operator extraction process may be performed using other calculation methods. For example, the extraction unit 324 calculates the difference value (for each corresponding component) between each component constituting the user vector and each component constituting each business vector (the component corresponding to each component of the user vector) for each business vector, and calculates the sum of these difference values for each business vector. The extraction unit 324 may then extract the housing business operator corresponding to the business vector with the smallest sum of the difference values as the housing business operator according to the user's preferences.

In this way, one housing provider corresponding to the provider vector that is most similar to the user vector may be selected, or multiple housing providers corresponding to a predetermined number of provider vectors that have a high similarity to the user vector may be selected in order of similarity. In other words, one housing provider corresponding to the provider vector that is closest to the user vector may be selected, or multiple housing providers corresponding to provider vectors that are close in distance may be selected in order of proximity.

The extraction unit 324 also outputs the extracted information about the housing business owner (e.g., profile information 352, image information 342) to the information provision unit 325.

Then, the information providing unit 325 transmits selected housing business operator information (e.g., profile information 352, image information 342) relating to the extracted housing business operators to the user terminal 2.

[Example of selected business operator information display]
16 is a diagram showing an example of a selected business information screen 510 displayed on the UI unit 460 of the user terminal 2. The selected business information screen 510 is displayed on the UI unit 460 based on the control of the control unit 450.

The selected business information screen 510 displays information about the housing business extracted by the extraction unit 324 of the server 1 (e.g., profile information 352, image information 342). For example, the selected business information screen 510 displays a profile information display area 511, a housing image display area 512 that displays multiple housing images 516 to 518 associated with the housing business extracted by the extraction unit 324 of the server 1, a re-propose new image button 514, and a decision button 515. Specifically, the information providing unit 325 of the server 1 transmits screen information for displaying the selected business information screen 510 to the user terminal 2. When the user terminal 2 receives the screen information, the control unit 450 causes the UI unit 460 to display the selected business information screen 510 based on the received screen information. In addition, the housing image display area 512 can display other housing images associated with the extracted housing business based on a user operation.

In addition, when a user operation is received by the reception unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. For example, after checking the profile displayed in the profile information display area 511 and the multiple house images 516 to 518 displayed in the house image display area 512, when the user decides that the housing business operator of the profile displayed in the profile information display area 511 is the preferred housing business operator, the user performs a selection operation (e.g., a touch operation) to select the decision button 515. When the selection operation of the decision button 515 is performed, the control unit 450 transmits decision information indicating that the selection operation has been performed to the server 1. When the server 1 receives the decision information, the control unit 320 executes a predetermined process to enable some kind of exchange between the housing business operator corresponding to the received decision information and the user. As this predetermined process, for example, at least one of a transmission process that transmits information about the user (information with the user's consent) to the housing provider corresponding to the determined information, and a transmission process that transmits information for accessing the housing provider corresponding to the determined information (e.g., access information such as a telephone number, email address, URL, SNS, etc.) to the user is executed. This allows the user to easily and quickly access a housing provider that meets their preferences.

Here, it is also assumed that after checking the profile displayed in the profile information display area 511 and the multiple home images 516 to 518 displayed in the home image display area 512, the user may request a re-proposal of the housing business owner for the profile displayed in the profile information display area 511. For example, a user who has checked the address included in the profile displayed in the profile information display area 511 may request a re-proposal because the location is far away. Also, a user who has checked the multiple home images 516 to 518 displayed in the home image display area 512 may request a re-proposal because there is no image to their liking.

In this way, when the user requests a re-proposal of the housing business owner of the profile displayed in the profile information display area 511, the user performs a selection operation (e.g., a touch operation) to select the re-proposal button 514 for a new image. When the selection operation of the re-proposal button 514 for a new image is performed, the control unit 450 transmits re-proposal request information indicating that the selection operation has been performed to the server 1. When the server 1 receives the re-proposal request information, the extraction unit 324 extracts a housing image to be newly provided to the user from the image information DB 340 based on the user vector generated by the vector generation unit 323. For example, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the tag information (style, emotion) stored in the tag information 343 of the image information DB 340, and based on the comparison result, newly extracts a housing image according to the user's preferences. Specifically, the extraction unit 324 calculates the difference value between each component constituting the user vector and each component constituting each tag information (each vector) for each tag information, and calculates the sum of these difference values for each tag information. The extraction unit 324 then extracts a house image corresponding to a predetermined number of tag information pieces with a small total difference value as a new house image according to the user's preferences. Note that, using the cosine similarity of each vector to be compared, a house image corresponding to a predetermined number of tag information pieces with a high cosine similarity may be extracted as a new house image according to the user's preferences. Also, a new house image according to the user's preferences may be extracted using other calculation methods.

The extraction unit 324 also outputs house image information (e.g., image information 342) relating to the extracted new house image to the information provision unit 325.

[Example of new house image selection]
17 is a diagram showing an example of a selection screen 520 displayed on the UI unit 460 of the user terminal 2. The selection screen 520 is displayed on the UI unit 460 based on the control of the control unit 450.

Selection screen 520 displays a house image display area 521 that displays multiple house images 523 to 528, and a decision button 522. Note that selection screen 520 is a display screen that corresponds to selection screen 500 shown in FIG. 15, and house image display area 521 and decision button 522 correspond to house image display area 501 and decision button 502. For this reason, detailed explanations of these will be omitted. In addition, the selection operation and decision operation for each image are similar to the example shown in FIG. 15, so detailed explanations of these will be omitted.

Furthermore, after one or more home images are selected by the user, if the user performs a selection operation (e.g., a touch operation) to select the decision button 522, the control unit 450 transmits image selection information related to the one or more home images selected by the user to the server 1. Note that the tag generation process by the tag generation unit 322 using the image selection information, the vector generation process by the vector generation unit 323, and the housing business operator extraction process by the extraction unit 324 are the same as in the above-mentioned example.

In this way, a new house image can be selected and provided to the user using a user vector generated based on a house image selected by the user (favorite image). Furthermore, by performing each of these processes multiple times, it is possible to refine the recommendations to the user. Furthermore, a new house image can be selected using past calculation results (e.g., a user vector from one or several times ago).

[Server operation example]
Fig. 18 is a flow chart showing an example of a housing business operator selection process by the server 1. This housing business operator selection process is executed by the control unit 320 (see Fig. 11) based on a program stored in the storage unit 330 (see Fig. 11). This housing business operator selection process is executed continuously for each control period. This housing business operator selection process will be described with appropriate reference to Figs. 1 to 17.

In step S601, the information providing unit 325 transmits home image information to the user terminal 2 based on a home image request from the user terminal 2. This home image information is information for displaying a selection screen 500 (see FIG. 15) on the UI unit 460 of the user terminal 2. For example, the user terminal 2 may be able to receive the provision of the service in this embodiment by installing a specific application (housing business operator app) on the user terminal 2. In this case, the control unit 450 of the user terminal 2 transmits a home image request to the server 1 by starting the housing business operator app on the user terminal 2.

In step S602, the information acquisition unit 321 determines whether or not image selection information (favorite image information) has been received from the user terminal 2. For example, if the user performs a process of selecting a house image on the selection screen 500 (see FIG. 15) and then selects the decision button 502, the control unit 450 of the user terminal 2 transmits the image selection information to the server 1. If image selection information has been received, the process proceeds to step S603. On the other hand, if image selection information has not been received, monitoring continues.

In step S603, the tag generation unit 322 acquires, from the image information DB 340, tag information associated with one or more house images corresponding to the image selection information received in step S602. Note that the method of acquiring this tag information is the same as the acquisition method described above.

In step S604, the vector generation unit 323 generates a user vector for the user of the user terminal 2 that transmitted the image selection information, based on the tag information acquired in step S603. Note that the method for generating this user vector is the same as the generation method described above.

In step S605, the extraction unit 324 extracts and selects a housing provider that meets the preferences of the user corresponding to the user vector from among the housing providers whose housing provider information is stored in the housing provider DB 350, based on the user vector generated in step S604. The method of selecting (extracting) this housing provider is the same as the extraction method described above.

In step S606, the information providing unit 325 transmits selected housing business operator information (e.g., profile information 352, image information 342) relating to the housing business operator selected in step S605 to the user terminal 2. When the user terminal 2 receives this selected housing business operator information, the control unit 450 of the user terminal 2 causes the UI unit 460 to display the selected business operator information screen 510 (see FIG. 16).

In step S607, the information acquisition unit 321 determines whether or not decision information has been received from the user terminal 2 that transmitted the selected housing business information in step S606. For example, when the user selects the decision button 515 on the selected business information screen 510 (see FIG. 16), the control unit 450 of the user terminal 2 transmits the decision information to the server 1. If decision information has been received, the process proceeds to step S608. On the other hand, if decision information has not been received, the process proceeds to step S609.

In step S608, the control unit 320 executes a predetermined process for introducing the housing provider (selected housing provider) corresponding to the decision information received in step S607 to the user of the user terminal 2. This predetermined process is the same as the predetermined process described above.

In step S609, the information acquisition unit 321 determines whether or not re-proposal request information has been received from the user terminal 2 that sent the selected housing business information in step S606. For example, when the user selects the Re-propose a new image button 514 on the selected business information screen 510 (see FIG. 16), the control unit 450 of the user terminal 2 sends re-proposal request information to the server 1. If re-proposal request information has been received, the process proceeds to step S610. On the other hand, if re-proposal request information has not been received, the process returns to step S607.

In step S610, the extraction unit 324 extracts new house images to be provided to the user from the image information DB 340 based on the user vector generated in step S604.

In step S611, the information providing unit 325 transmits house image information (e.g., image information 342) relating to the new house image extracted in step S611 to the user terminal 2. This house image information is information for displaying the selection screen 520 (see FIG. 17) on the UI unit 460 of the user terminal 2.

[Example of effect]
Here, it is assumed that the user has an abstract ideal image and preferences for a preferred home, but has difficulty in clearly verbalizing the image and desires of the home. For this reason, if the user is asked to answer questions such as the image and desires of the home, and a housing business is proposed based on the user's answers, there is a risk that an appropriate housing business may not be proposed. Therefore, in this embodiment, it is possible to appropriately acquire the user's preferences by selecting a housing image according to the user's preferences from among a plurality of housing images displayed on the user terminal 2. Then, a user vector is generated using tag information related to the housing image according to the user's preferences, and a housing business is proposed to the user based on the comparison result between this user vector and the housing business vector, so that it is possible to propose an appropriate housing business according to the user's preferences. In other words, it is possible to support the user's housing selection in an intuitive and easy-to-understand manner.

[Example of house image editing by user]
Here, it is assumed that none of the house images provided to the user match the user's preferences. In this case, it is assumed that the user will have difficulty selecting a preferred house image. Therefore, it is conceivable that the house images provided to the user can be temporarily edited to match the user's preferences, making it easier for the user to select a preferred house image.

[Example of Item Information DB Configuration]
19 is a diagram showing, in a simplified form, the stored contents of the item information stored in the item information DB 370. The item information DB 370 may be stored in the storage unit 330 of the server 1, or may be stored in an external device other than the server 1 for use.

Item information DB370 is a database for storing items for editing images provided by housing developers in association with the features of multiple tags generated for these items.

Specifically, image information 372 and tag information 373 are stored in item information DB 370 in association with item identification information 371.

Item identification information 371 is identification information for identifying each item prepared by the service provider. Note that, for ease of explanation, FIG. 19 shows an example in which only the name indicating the item is stored as identification information in item identification information 371, but other identification information may be used.

Image information 372 is information about an item image prepared by the service provider so that the user can edit it. Note that, for ease of explanation, FIG. 19 shows an example in which only the image is stored in image information 372, but various attribute information (e.g., size information) stored in association with the image may also be included.

Tag information 373 is tag information generated for the item image stored in image information 372. This tag information is used when generating a user vector. Note that FIG. 19 shows an example in which tag information with the same content as tag information 343 shown in FIG. 13 is stored, but it is also possible to store only the necessary information and omit the other information.

[Example of a screen where a user edits a house image]
20 is a diagram showing an example of an edit screen 700 displayed on the UI unit 460 of the user terminal 2. The edit screen 700 is displayed on the UI unit 460 based on the control of the control unit 450.

The editing screen 700 displays a house image display area 701 that displays multiple house images 703 to 706, a decision button 702, and an item display area 710. Specifically, the information providing unit 325 of the server 1 transmits editing screen information for displaying the editing screen 700 to the user terminal 2. When the user terminal 2 receives the editing screen information, the control unit 450 causes the UI unit 460 to display the editing screen 700 based on the received editing screen information. Note that the house image display area 701 corresponds to the house image display area 501 shown in FIG. 15.

The item display area 710 is an area that displays each of the item images 711 to 718 whose item information is stored in the item information DB 370 (see FIG. 19). The user can generate an image of the user's preference by performing a moving operation to move a desired item displayed in the item display area 710 to a desired house image displayed in the house image display area 701. For example, if the UI unit 460 is configured with a touch panel, the user can perform an operation (e.g., a drag and drop operation) to move the desired item to the position of the desired house image while touching it. For example, the item image 712 indicating a potted plant can be placed in the house image 703 by the user performing an operation to move the item image 712 to a desired position in the house image 703. The transition of this movement is shown diagrammatically by the arrow 720. In addition, the house image in which the item image is placed (the edited house image) is in a selected state. In addition, the image synthesis process in which the item image is placed in the house image and synthesized can use a known image processing. In addition, various image processes (e.g., zooming in and out, adjusting the angle in three dimensions, additional editing of images) may be performed on each item image based on user operations. Publicly known image processing may be used for each of these image processes. Furthermore, the item images placed in the house image may be appropriately transformed in relation to other components in the house image using publicly known image processing techniques. For example, the item images may be zoomed in and out based on the size of each object in the house image, and the angle of the item image may be adjusted based on the inclination of each object in the house image.

In this way, if no home image matching the user's preferences is displayed in the home image display area 701, it is possible to edit a portion of each home image displayed in the home image display area 701 to generate a home image matching the user's preferences. Note that the home image generated by this editing is an image generated temporarily in order to select a home builder, and is deleted after the home builder selection process is completed.

In this way, when a user operation is received by the reception unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. For example, when a selection operation (e.g., a touch operation) is performed by the user to select a house image to be edited from among the house images displayed in the house image display area 701, the preferred house image selected by the selection operation is placed in a selected state.

In addition, after the user edits one or more house images, if the user executes a selection operation (e.g., a touch operation) to select the decision button 702, the control unit 450 transmits edited image information regarding the one or more house images edited by the user to the server 1. When the server 1 receives the edited image information, the information acquisition unit 321 outputs the received edited image information to the tag generation unit 322 and the vector generation unit 323.

[Example of user vector generation]
The tag generation unit 322 acquires tag information for one or more edited images corresponding to the edited image information transmitted from the user terminal 2 from the image information DB 340 and the item information DB 370. Specifically, the tag generation unit 322 acquires tag information associated with the original house image of each edited image from the image information DB 340, and acquires tag information associated with items added to each edited image by editing from the item information DB 370. Then, the tag generation unit 322 outputs each piece of acquired tag information to the vector generation unit 323.

Next, the vector generation unit 323 generates a user vector for the user who sent the edited image information based on the tag information output from the tag generation unit 322. In this case, the method of generating a user vector is the same as that described above, except that tag information associated with the item is used, so a detailed description will be omitted here.

Here, it is considered that the edited image edited by adding items by the user reflects the user's preferences more than the unedited house image. Therefore, the user vector may be generated by weighting the tag information associated with the edited image edited by adding items by the user higher than other tag information. It is also assumed that the edited image may have multiple items added. In the case of such an edited image, it is considered that the user's preferences are further reflected. Therefore, for example, the weighting for the edited image and the weighting for the added items may be increased according to the number of items added to the edited image to generate a user vector. In other words, the weighting for the edited image and the weighting for the added items may be changed based on various information regarding the items added to the edited image (for example, the number of items, the proportion of item images in the house image) to generate a user vector. This makes it possible to generate a user vector that reflects the user's preferences.

As shown in FIG. 20, when a house image is edited by adding an item to the house image, it is assumed that the style and emotion will differ depending on the position of the item in the house image. Therefore, using the AI model related to style and emotion described above, the tag generation unit 322 may generate new tag information related to style and emotion for the edited image. In this case, it is possible to generate more appropriate tag information according to the editing results edited by the user.

[Example of user terminal operation]
Fig. 21 is a flow chart showing an example of a house image editing process by the user terminal 2. This house image editing process is executed by the control unit 450 (see Fig. 11) based on a program stored in the storage unit 470 (see Fig. 11). This house image editing process is constantly executed at each control cycle. This house image editing process will be described with appropriate reference to Figs. 1 to 20.

In step S751, the control unit 450 transmits edit screen request information for requesting edit screen information to the server 1. For example, when a user operation is performed to request an edit screen, the control unit 450 transmits the edit screen request information to the server 1.

In step S752, the control unit 450 determines whether or not editing screen information has been received from the server 1. If editing screen information has been received, the process proceeds to step S753. On the other hand, if editing screen information has not been received, the process continues monitoring.

In step S753, the control unit 450 causes the UI unit 460 to display an editing screen (e.g., editing screen 700 (see FIG. 20)) based on the editing screen information received in step S752.

In step S754, the control unit 450 determines whether or not a user operation has been accepted by the acceptance unit 461 for the editing screen displayed in step S753. If a user operation has been accepted, the process proceeds to step S755. On the other hand, if a user operation has not been accepted, the process continues to monitor.

In step S755, the control unit 450 determines whether the user operation received in step S754 is an editing operation for editing the house image displayed on the editing screen. For example, if a move operation is performed to move a desired item displayed in the item display area 710 (see FIG. 20) to a desired house image displayed in the house image display area 701 (see FIG. 20), it is determined that an editing operation has been performed as the user operation. If the user operation is an editing operation, the process proceeds to step S756. On the other hand, if the user operation is not an editing operation, the process proceeds to step S757.

In step S756, the control unit 450 executes an editing process to edit the house image based on the user operation (editing operation) accepted in step S754. For example, when a move operation is performed to move an item displayed in the item display area 710 to the house image displayed in the house image display area 701, image processing is executed to move the item in response to the move operation and composite it into the house image. Also, for example, when a delete operation is performed to delete an item moved to the house image, image processing is executed to delete the item in response to the delete operation.

In step S757, the control unit 450 determines whether the user operation received in step S754 is a decision operation that is performed after one or more house images have been edited. For example, if a selection operation (e.g., a touch operation) that selects the decision button 702 (see FIG. 20) is performed, it is determined that a decision operation has been performed as the user operation. If the user operation is a decision operation, the process proceeds to step S759. On the other hand, if the user operation is not a decision operation, the process proceeds to step S758.

In step S758, the control unit 450 executes a predetermined process based on the user operation received in step S754. For example, when a selection operation is performed to select the house image displayed in the house image display area 701, image processing is executed to select the house image in response to the selection operation.

In step S759, the control unit 450 transmits edited image information regarding one or more house images edited by the user to the server 1.

In this way, by allowing the user to edit a home image according to their preferences from among the multiple home images displayed on the user terminal 2, it is possible to more appropriately express the user's preferences based on the user's subjective editing. This makes it possible to suggest appropriate housing providers that match the user's preferences based on the user's editing results. In other words, it is possible to support the user's home selection in an intuitive and easy-to-understand manner.

In the above, an example of editing a house image using an image (item image) has been shown, but the house image may be edited using text information input by the user (for example, manual input, voice input, gesture input). For example, it is possible to input "I want to put a piano" or "I want to put a sofa" as text information. In this case, for example, it is possible to reflect the input text information on the selected house image. For example, it is possible to generate an image related to the input text information using a generative AI (for example, an image generation AI (for example, stable diffusion, midjourney)) that has learned about each style. Then, it is possible to generate a desired edited image by synthesizing the generated image (image related to the text information) with the selected house image. In this case, for example, a known image synthesis technique can be used. This makes it possible to generate a desired edited image by following the atmosphere of the style related to the input text information. In this case, for example, depending on the weight of the input text information (for example, the number of characters), the weight associated with the edited image and the weight associated with the input text information may be increased to generate a user vector. This makes it possible to generate a user vector that reflects the user's preferences.

[Example using user-acquired images]
As described above, it is possible that none of the house images provided to the user match the user's preferences. In this case, it is possible that the user has difficulty selecting a preferred house image. Therefore, it is possible to select a housing business operator that matches the user's preferences by generating a user vector that matches the user's preferences using an image acquired by the user.

[Example of configuration of image information DB]
FIG. 22 is a simplified diagram showing the contents of the image information stored in the image information DB 800. As shown in FIG.

The image information DB 800 is a database stored in the storage unit 470 (see FIG. 11) of the user terminal 2. The image information DB 800 also stores image information acquired by the image acquisition unit 430. For example, if a smartphone is used as the user terminal 2, the camera mounted on the smartphone corresponds to the image acquisition unit 430.

When a user performs an image acquisition operation (e.g., a still image shooting operation, a video shooting operation), the control unit 450 of the user terminal 2 records the image acquired by the image acquisition unit 430 based on the acquisition operation as an image file (still image file, video file) in the image information DB 800. For example, as shown in FIG. 22, the image file is stored in the image information 802.

In this case, the control unit 450 associates the location information (e.g., latitude and longitude) acquired by the location information acquisition unit 420 of the user terminal 2 with the image file and stores it based on the timing of the acquisition operation. For example, as shown in FIG. 22, the acquired location information is stored in the location information 803 in association with the image file. For example, when a smartphone is used as the user terminal 2, a GPS device installed in the smartphone corresponds to the location information acquisition unit 420.

The control unit 450 also stores the audio information acquired by the audio acquisition unit 440 of the user terminal 2 in association with the image file based on the timing of the acquisition operation. For example, as shown in FIG. 22, the acquired audio information is stored in audio information 804 in association with the image file. For example, when a smartphone is used as the user terminal 2, the microphone mounted on the smartphone corresponds to the audio acquisition unit 440. When a still image is captured, audio information for a predetermined time based on the capture timing can be stored in association with the still image. When a video is captured, the acquired audio information can be used to use a representative image of the video (for example, the first frame, the frame with the highest characteristic) as a still image.

For example, a user uses the user terminal 2 to take pictures of a house, the interior of the house, scenery, or other subjects that suit the user's preferences, and the images that suit the user's preferences are stored in the image information DB 800. It is then possible to generate a user vector according to the user's preferences using the image information stored in the image information DB 800. Note that when a stationary device such as a personal computer is used as the user terminal 2, it is possible that the position information acquisition unit 420 and the image acquisition unit 430 are not provided. In this case, an external device may be used as the position information acquisition unit 420 and the image acquisition unit 430, and image information acquired using an imaging device (e.g., a smartphone, a digital still camera, a digital video camera (e.g., a camera-integrated recorder)) can be acquired and stored in the image information DB 800, and this can be used.

[Example of user's selection of favorite image]
23 is a diagram showing an example of a selection screen 810 displayed on the UI unit 460 of the user terminal 2. The selection screen 810 is displayed on the UI unit 460 based on the control of the control unit 450.

The selection screen 810 displays an image display area 820 that displays multiple images 821 to 826, a selected image display area 830 that displays a selected image, a position information designation area 811, an audio information designation area 812, and a decision button 813. Specifically, the information providing unit 325 of the server 1 transmits selection screen information for displaying the selection screen 810 to the user terminal 2. When the user terminal 2 receives the selection screen information, the control unit 450 causes the UI unit 460 to display the selection screen 810 based on the received selection screen information.

The image display area 820 is an area for displaying each of the images 821 to 826 whose image information is stored in the image information DB 800 of the user terminal 2. That is, the control unit 450 acquires each piece of image information stored in the image information DB 800, and displays it in the image display area 820 of the selection screen 810. Note that the images to be displayed here are not limited to house images, and various types of images can be displayed.

The user can select an image of the user's preference by performing a moving operation to move a desired image displayed in the image display area 820 to the selected image display area 830. For example, if the UI unit 460 is configured with a touch panel, the user can perform an operation (e.g., a drag-and-drop operation) to move the desired image to the selected image display area 830 while touching it. For example, the user can place the image 822 in the selected image display area 830 by performing an operation to move the image 822 including a coffee cup to the selected image display area 830. The transition of this movement is shown diagrammatically by an arrow 840. Note that the image already displayed in the selected image display area 830 and another image may be composited using a known image processing technique to create a composite image. Also, one or more images may be edited using each of the editing techniques described above.

The selected image display area 830 is an area that displays each of the images 831, 832, and 822 selected based on a user operation. The selected image display area 830 may display each of the house images that are selected on the screens shown in FIG. 15 and FIG. 17, each of the edited images edited on the editing screen 700 shown in FIG. 20, and the like. When various images are displayed in this manner, it may be possible to appropriately perform operations to delete, add, and edit the various images based on a user operation.

Figure 23 shows an example in which house images 831 and 832, which were selected in Figure 15, are displayed in the selected image display area 830.

The location information designation area 811 is an area that is operated when specifying whether or not to use location information associated with the image displayed in the image display area 820. For example, when a selection operation is performed to select the pull-down button in the location information designation area 811 after an image moved to the selected image display area 830 is selected, a selection area for selecting use or non-use is displayed, and the user performs the operation desired in this selection area. For example, when it is designated to use the location information associated with the selected image, it is possible to generate a user vector using the location information associated with that image. On the other hand, when it is designated not to use the location information associated with the selected image, it is possible to generate a user vector without using the location information associated with that image.

The audio information designation area 812 is an area that is operated when specifying whether or not to use audio information associated with the image displayed in the image display area 820. For example, when an image moved to the selected image display area 830 is selected and then a selection operation is performed to select the pull-down button in the audio information designation area 812, a selection area for selecting use or non-use is displayed, and the user performs the operation desired in this selection area. For example, when it is designated to use the audio information associated with the selected image, it is possible to generate a user vector using the audio information associated with that image. On the other hand, when it is designated not to use the audio information associated with the selected image, it is possible to generate a user vector without using the audio information associated with that image.

In this way, when a user operation is accepted by the accepting unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. Also, when the user executes a selection operation (e.g., a touch operation) to select the decision button 813 after moving one or more images to the selected image display area 830, the control unit 450 transmits selected image information related to the one or more images selected by the user to the server 1. This selected image information includes image information, position information (if use is specified), and audio information (if use is specified). When the server 1 receives the selected image information, the information acquisition unit 321 outputs the received selected image information to the tag generation unit 322 and the vector generation unit 323.

[Example of user vector generation]
The tag generating unit 322 generates tag information for one or more selected images corresponding to the selected image information transmitted from the user terminal 2. When a house image stored in the image information DB 340 is selected as the selected image, the tag generating unit 322 acquires the tag information of the house image from the image information DB 340.

In addition, if an image stored in the image information DB 800 of the user terminal 2 is a selected image, the tag generation unit 322 generates tag information related to style and emotion for that image. For example, it is possible to generate tag information related to style and emotion for that image using the AI model related to style and emotion described above.

In addition, if at least one of location information (if its use is specified) and audio information (if its use is specified) is included in the selected image information, it is possible to generate tag information relating to style and emotion for each piece of information. For example, a database that associates location information (e.g., latitude and longitude) with tag information relating to style and emotion (e.g., information previously set for each region) can be prepared in advance in the storage unit 330 of the server 1, and tag information corresponding to the location information included in the selected image information can be obtained using this database.

In addition, for example, a database that associates location information (e.g., latitude and longitude) with the characteristics and environmental images of a certain location (e.g., Okinawa is "hot" and Kamakura is "ancient capital" and "calm") can be prepared in advance in the storage unit 330 of the server 1, and this database can be used to obtain the characteristics and environmental images (e.g., Okinawa is "hot" and Kamakura is "ancient capital" and "calm") that correspond to the location information included in the selected image information, and text information related to these characteristics and environmental images (e.g., "hot", "ancient capital", "calm"). In this case, tag information can be generated from the text information using an AI model described below.

In addition, for example, a database that associates audio information with tag information related to style and emotion (for example, information pre-set for each type of sound (e.g., "noise," "quiet," "river babbling," "child's voice")) can be prepared in advance in the storage unit 330 of the server 1, and tag information corresponding to the audio information included in the selected image information can be obtained using this database.

For example, it is also possible to convert voice information into text information. In this case, it is possible to generate tag information from the text information using an AI model described below. Tag information may also be generated using an AI model that has been trained using location information, voice information, etc.

Next, the vector generation unit 323 generates a user vector for the user who sent the image selection information, based on the tag information output from the tag generation unit 322. Note that the method for generating this user vector is similar to the method for generating the user vector described above, and therefore a detailed description thereof will be omitted here.

In the above, an example has been shown in which tag information is generated using an AI model that has been generated by learning in advance, and a user vector is generated using this tag information, but other AI models may be used. For example, an LLM (large language model) may be used as the AI model. For example, ChatGPT (Generative Pre-trained Transformer), Bard, Llama2 (Large Language Model Meta AI 2), multimodal LLM, etc. may be used as the LLM. Note that these are only examples, and other AI models may be used. For example, tag information may be generated for various images using a multimodal LLM. Note that text information may be attached to a house image (e.g., a property image). In such a case, tag information related to the text information may be generated using an AI model, and the tag information obtained by combining this tag information with tag information related to the house image (e.g., a property image) may be used to generate a user vector.

[Example of user terminal operation]
Fig. 24 is a flowchart showing an example of image selection processing by the user terminal 2. This image selection processing is executed by the control unit 450 (see Fig. 11) based on a program stored in the storage unit 470 (see Fig. 11). This image selection processing is executed constantly for each control period. This image selection processing will be described with appropriate reference to Figs. 1 to 23.

In step S851, the control unit 450 transmits selection screen request information for requesting selection screen information to the server 1. For example, when a user operation is performed to request a selection screen, the control unit 450 transmits the selection screen request information to the server 1.

In step S852, the control unit 450 determines whether or not selection screen information has been received from the server 1. If selection screen information has been received, the process proceeds to step S853. On the other hand, if selection screen information has not been received, the process continues monitoring.

In step S853, the control unit 450 obtains image information (e.g., image information 802) to be displayed on the editing screen based on the selection screen information received in step S852 from the image information DB 800 (see FIG. 22).

In step S854, the control unit 450 causes the UI unit 460 to display a selection screen (e.g., selection screen 810 (see FIG. 23)) based on the selection screen information received in step S852.

In step S855, the control unit 450 determines whether or not a user operation has been accepted by the acceptance unit 461 for the selection screen displayed in step S854. If a user operation has been accepted, the process proceeds to step S856. On the other hand, if a user operation has not been accepted, the process continues to monitor.

In step S856, the control unit 450 determines whether the user operation received in step S855 is a selection operation for selecting an image displayed on the selection screen. For example, if a move operation is performed to move an image displayed in the image display area 820 (see FIG. 23) to the selected image display area 830 (see FIG. 23), it is determined that a selection operation has been performed as the user operation. If the user operation is a selection operation, the process proceeds to step S857. On the other hand, if the user operation is not a selection operation, the process proceeds to step S858.

In step S857, the control unit 450 executes a selection process to select an image based on the user operation (selection operation) accepted in step S855. For example, when a move operation is performed to move an image displayed in the image display area 820 (see FIG. 23) to the selected image display area 830 (see FIG. 23), image processing is executed to move the image in response to the move operation and set it as the selected image. Also, for example, when a delete operation is performed to delete an image that has been moved to the selected image display area 830 as the selected image, image processing is executed to delete the image in response to the delete operation from the selected image display area 830.

In step S858, the control unit 450 determines whether the user operation accepted in step S855 is a decision operation that is performed after one or more images are selected. For example, if a selection operation (e.g., a touch operation) that selects the decision button 813 (see FIG. 23) is performed, it is determined that a decision operation has been performed as the user operation. If the user operation is a decision operation, the process proceeds to step S860. On the other hand, if the user operation is not a decision operation, the process proceeds to step S859.

In step S859, the control unit 450 executes a predetermined process based on the user operation received in step S855. For example, when a selection operation is performed to select the house image displayed in the selected image display area 830, image processing is executed to select the house image in response to the selection operation.

In step S860, the control unit 450 transmits selected image information regarding one or more images selected by the user to the server 1.

In this way, by allowing the user to select images according to the user's preferences from among the images acquired using the user terminal 2, it is possible to more appropriately express the user's preferences based on the user's proactive image collection. This makes it possible to suggest appropriate housing providers according to the user's preferences based on the results of the user's collection. In other words, it becomes possible to support the user in selecting a home in an intuitive and easy-to-understand manner.

[Example using text information]
In the above, an example of selecting a housing business operator using mainly house images has been shown. However, it is assumed that the house image that the user prefers may not be presented to the user. Therefore, in the following, an example of selecting a house image using text information will be shown.

[Example of generating user vectors based on text information]
25 is a diagram showing an example of a text input screen 900 displayed on the UI unit 460 of the user terminal 2. The text input screen 900 is displayed on the UI unit 460 under the control of the control unit 450.

On the text input screen 900, a house image display area 901 that displays multiple house images 902 to 904, a text information input field 905, a re-proposal button 906, and a decision button 907 are displayed. Specifically, the information providing unit 325 of the server 1 transmits screen information for displaying the text input screen 900 to the user terminal 2 based on a request from the user terminal 2. When the user terminal 2 receives the screen information, the control unit 450 causes the UI unit 460 to display the text input screen 900 based on the received screen information. Note that other house images can be displayed on the text input screen 900 based on user operations.

When a user operation is received by the reception unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. Here, it is assumed that the house image display area 901 does not display the user's favorite house image. In such a case, it is possible for the user to input (for example, manual operation or voice input) characters related to the favorite house into the text information input field 905. FIG. 25 shows an example in which the user inputs "I like houses that have the warmth of wood." Note that this is just one example, and characters related to the favorite house may be, for example, "traditional English-style houses," "houses like Buckingham Palace," or "I like houses with a more modern design."

Furthermore, if the user performs a selection operation (e.g., a touch operation) to select the Re-suggest button 906 after characters are entered in the text information input field 905, the control unit 450 transmits text information corresponding to the characters entered by the user to the server 1. When the server 1 receives the text information, the information acquisition unit 321 outputs the received text information to the tag generation unit 322 and the vector generation unit 323. Note that, if one or more house images are selected by the user, the control unit 450 transmits image selection information related to the selected one or more house images to the server 1 together with the text information.

[Example of user vector generation]
The tag generation unit 322 generates tag information corresponding to the text information transmitted from the user terminal 2. When one or more house images are selected by the user, image selection information regarding the selected one or more house images is transmitted from the user terminal 2 together with the text information. In this case, it is possible to generate a user vector taking into consideration the image selection information. Examples of this are shown in Figs. 28 and 29.

For example, the tag generating unit 322 can generate tag information corresponding to text information using a predetermined database (e.g., a dictionary database) for converting text information into tag information. For example, a dictionary database in which character strings are associated with features of each element constituting tag information (e.g., features related to style and emotion) can be used. Then, the tag generating unit 322 extracts character strings included in the dictionary database from among the character strings included in the sentence corresponding to the text information transmitted from the user terminal 2, and extracts features of tag information corresponding to the extracted character strings from the dictionary database. Also, when multiple character strings are extracted, the tag generating unit 322 extracts features of tag information corresponding to each character string. Then, the tag generating unit 322 aggregates (e.g., calculates an average value) the features of one or more extracted tag information to generate tag information corresponding to the text information transmitted from the user terminal 2. Then, the tag generating unit 322 outputs the generated tag information to the vector generating unit 323.

It is also possible to use an AI model to generate tag information (user vector) from text information. For example, it is possible to use an AI model that has learned multiple pieces of text information to which predetermined tags that associate sentences with the characteristics of each element that makes up the tag information (e.g., characteristics related to style and emotion) are attached as teacher labels. For this learning method, the learning method shown in FIG. 12 can be applied.

As an AI model, for example, LLM can be used. As an LLM, for example, ChatGPT, Bard, Llama2, etc. can be used. Note that these are only examples, and other AI models may be used. For example, based on text information input by a user, condition information to output features related to a house image for each of a plurality of tags can be input to the LLM as instruction information (prompt), and the output information in response to this can be tag information. Here, the multiple tags are tags related to style and emotion, as described above.

Here, an example of a prompt that suggests tags using ChatGPT is shown. For example, a sentence constituting the text information sent from the user terminal 2 can be used to input instruction information (prompt) to the LLM to output a score (0 to 1) regarding the relevance with each of a plurality of tags (style tags, emotion tags), and the corresponding output information can be used as the feature quantity for each of the plurality of tags. The style tags may be the same as those described above, and may have elements such as Western, Japanese, modern, American, antique, country, simple, natural, Nordic, industrial, woody, etc. The emotion tags may be the same as those described above, and may have elements such as family gathering/warmth, high-class/elegant/luxury, cool/chic, casual/pop, refined/stylish, open feeling, calm, flashy, profound feeling, cute, etc.

Next, the vector generation unit 323 generates a user vector for the user who transmitted the text information based on the tag information output from the tag generation unit 322. In this case, the tag information generated by the tag generation unit 322 can be used as the user vector. Note that when multiple pieces of text information are transmitted from the user terminal 2, tag information is generated for each piece of text information, so that it is possible to generate a user vector using multiple pieces of tag information, as in the example shown in FIG. 15.

[Example of house image selection based on text information]
Next, the extraction unit 324 extracts a house image to be newly provided to the user from the image information DB 340 based on the user vector generated by the vector generation unit 323. This extraction method is similar to the method of extracting a new house image when the "Re-propose a new image" button 514 (see FIG. 16) is selected, and therefore a description thereof will be omitted here.

The extraction unit 324 also outputs house image information (e.g., image information 342) related to the extracted house image to the information provision unit 325. The information provision unit 325 then transmits the house image information to the user terminal 2, and the user terminal 2 displays the house image information.

[Example of new house image selection]
Fig. 26 is a diagram showing a display example of a selection screen 910 displayed on the UI unit 460 of the user terminal 2. The selection screen 910 is displayed on the UI unit 460 based on the control of the control unit 450. Note that the selection screen 910 is a modified version of a part of the message at the top of the selection screen 520 shown in Fig. 17, and the rest is common to the selection screen 520. For this reason, the parts common to the selection screen 520 are denoted by the same reference numerals.

In this way, it is possible to select a new house image and provide it to the user using the user vector generated based on the text information input by the user. Note that a text information input field 905 (see FIG. 25) may be provided on the selection screen 910 to allow further input of text information. In this case, when the user inputs text information, it is possible to re-suggest a new house image to the user based on that text information. Furthermore, by performing each of these processes multiple times, it is possible to refine the recommendations to the user.

The above describes an example of accepting text information input by the user when there is no preferred image among the house images presented to the user, when there is a request for a particular house image, etc. However, this is not limited to this. For example, it is possible to accept text information input by the user before the house images are presented, and to select a house image to present to the user using a user vector generated based on this text information.

[Server operation example]
Fig. 27 is a flow chart showing an example of a housing business operator selection process by the server 1. This housing business operator selection process is a partial modification of the housing business operator selection process shown in Fig. 18. Specifically, steps S931 to S934 have been added. Other than this addition, the housing business operator selection process is the same as that shown in Fig. 18, so the same reference numerals are used to designate parts that are common to Fig. 18, and description thereof will be omitted.

In step S931, the information acquisition unit 321 determines whether or not text information has been received from the user terminal 2. For example, if the user inputs text information on the text input screen 900 (see FIG. 25) and then selects the re-suggest button 906, the control unit 450 of the user terminal 2 transmits the text information to the server 1. If text information has been received, the process proceeds to step S932. On the other hand, if text information has not been received, the process returns to step S602.

In step S932, the tag generation unit 322 and the vector generation unit 323 generate tag information (user vector) based on the text information received in step S931. The method of generating this user vector is the same as the method described above.

In step S933, the extraction unit 324 extracts a new house image to be provided to the user from the image information DB 340 based on the user vector generated in step S932.

In step S934, the information providing unit 325 transmits house image information (e.g., image information 342) relating to the new house image extracted in step S933 to the user terminal 2. This house image information is information for displaying the selection screen 910 (see FIG. 26) on the UI unit 460 of the user terminal 2.

[Example of house image re-recommendation based on house image and text information]
In the above, an example of presenting a house image to a user using a user vector generated based on text information has been shown. Here, after presenting a plurality of house images to a user, it is assumed that the user selects a house image that the user likes from among them, and receives requests from the user regarding those house images through text information. Therefore, in the following, an example of generating a new user vector based on the house image selected by the user and the user's request regarding the house image presented to the user will be shown.

[Example of generating a user vector based on selected house images and text information]
Fig. 28 is a diagram showing a display example of a text input screen 950 displayed on the UI unit 460 of the user terminal 2. Note that the text input screen 950 is similar to the text input screen 900 shown in Fig. 25, and differs in that the content entered in the text information input field 905 and that the house images 902 and 903 are in a selected state (favorite buttons 951 and 952 are blacked out), and the rest is common to the text input screen 900. For this reason, the parts common to the text input screen 900 are indicated with the same reference numerals. Also, the process of transmitting each piece of information to the server 1 is similar to the example shown in Fig. 25.

As described above, it is possible that the house image display area 901 may not display the user's preferred house image, that there may be few house images that the user likes, or that a house image that is slightly different from the user's preferred house image may be displayed. In such cases, it is possible for the user to input text related to the preferred house in the text information input field 905 by inputting (e.g., manual operation or voice input). Figure 28 shows an example in which the user has input "Please recommend a house with a slightly brighter atmosphere." Note that this is just one example, and text related to the preferred house may also be inputted, for example, "Please make the original recommendation result blank and recommend an image of a house with a brighter atmosphere."

[Example of user vector generation]
The tag generation unit 322 generates tag information corresponding to each of the image selection information and text information transmitted from the user terminal 2. The generation of tag information based on image selection information is similar to the example shown in Fig. 15 etc. The generation of tag information based on text information is similar to the example shown in Fig. 25. The generation of user vectors based on tag information generated by the tag generation unit 322 (user vectors based on image selection information, user vectors based on text information) is also similar to the generation examples described above.

[Example of determining the reflection ratio of image selection information and text information based on user requests]
Here, we show an example of generating a user vector C for selecting a new house image using a user vector (user vector A) generated based on image selection information and a user vector (user vector B) generated based on text information.

For example, it is possible to generate a user vector C using a preset ratio (reflection ratio of user vector A and user vector B). For example, it is possible to obtain a user vector C by the following formula 1 using the reflection ratio α of user vector A and user vector B. Note that the calculation method according to formula 1 is only one example, and the user vector C may be obtained by other calculation methods.
User vector C = (1 - α) × user vector A + α × user vector B ... Equation 1

The vector generation unit 323 generates user vector A and user vector B based on the tag information output from the tag generation unit 322, and generates user vector C based on the reflection ratio α and user vector A and user vector B. Note that the reflection ratio α may be set by the user or by the service provider in this embodiment.

The reflection ratio α may be set based on at least one of the text information. For example, it is possible to set the reflection ratio α using an AI model. For example, it is possible to use an AI model that has learned multiple pieces of text information to which a specific tag that associates the reflection ratio with the request sentence is added as a teacher label.

As an AI model, for example, LLM can be used. As an LLM, for example, ChatGPT, Bard, Llama2, etc. can be used. Note that these are only examples, and other AI models may be used. For example, based on text information input by a user, condition information for outputting the reflection ratio α can be input to the LLM as instruction information (prompt), and the output information in response to this can be the reflection ratio α.

Here, an example of a prompt that uses ChatGPT to output the reflection ratio α is shown. For example, it is possible to input instruction information (prompt) to the LLM to determine the extent to which the content of a sentence constituting the text information sent from the user terminal 2 should be reflected, determine the ratio (0 to 1) and output the result, using the sentence, and to input the corresponding output information as the reflection ratio α. In this case, instruction information (prompt) to output the estimated basis for the reflection ratio α may be input to the LLM.

For example, as an example of how to determine the reflection ratio α (example of the basis for estimating the reflection ratio α), if the user inputs "Please blank out the original recommendation result and recommend an image of a house with a bright atmosphere," it is possible to use the estimation basis of the reflection ratio α that the reflection ratio of the content of the sentence is appropriate as 1.0, since it is a request to ignore the original recommendation result. Also, for example, if the user inputs "Please recommend a house with a brighter atmosphere," it is possible to use the estimation basis of the reflection ratio α that the reflection ratio of the content of the sentence is appropriate as about 0.3, since it is a request to maintain the original recommendation result to a certain extent and change it a little more. Also, for example, if the user inputs "I like houses with a bright atmosphere," it is difficult to determine whether the original recommendation result should be maintained, so it is possible to use the estimation basis of the reflection ratio α that the reflection ratio of the content of the sentence is appropriate as about 0.5.

[Example of house image selection based on text information]
Next, the extraction unit 324 extracts a house image to be newly provided to the user from the image information DB 340 based on the user vector C generated by the vector generation unit 323. This extraction method is similar to the method of extracting a new house image when the "Re-suggest a new image" button 514 (see FIG. 16) is selected, and therefore will not be described here.

The extraction unit 324 also outputs house image information (e.g., image information 342) related to the extracted house image to the information provision unit 325. The information provision unit 325 then transmits the house image information to the user terminal 2, and the user terminal 2 displays the house image information.

The above shows an example of extracting a housing image to be newly provided to the user based on the user vector C, but a housing provider that matches the user's preferences may be extracted from the housing provider DB 350 based on the user vector C. In this case, selected housing provider information (e.g., profile information 352, image information 342) related to the extracted housing provider is transmitted to the user terminal 2 and displayed on the user terminal 2. For example, the selected provider information screen 510 (see FIG. 16) is displayed on the UI unit 460.

In addition, a text information input field 905 (see FIGS. 25 and 28) may be provided on the selected business operator information screen 510 (see FIG. 16), and a user vector B may be generated according to the user's preferences using the text information input field 905, and a housing image extraction process, housing business operator extraction process, etc. may be performed using the already generated user vector A (user vector generated based on image selection information) and user vector B. In this case, each piece of extracted information is transmitted to the user terminal 2 and displayed on the user terminal 2.

[Example of new house image selection]
Fig. 29 is a diagram showing a display example of a selection screen 910 displayed on the UI unit 460 of the user terminal 2. The selection screen 960 is displayed on the UI unit 460 based on the control of the control unit 450. Note that the selection screen 960 is a modified version of a part of the message at the top of the selection screen 520 shown in Fig. 17, and the rest is common to the selection screen 520. For this reason, the parts common to the selection screen 520 are denoted by the same reference numerals.

In this way, a new house image can be selected and provided to the user using the user vector C generated based on the text information input by the user and the house image selected by the user. Note that a text information input field 905 (see Figures 25 and 28) may be provided in the selection screen 960 to allow further input of text information. In this case, when text information is input by the user, a new house image can be provided to the user based on that text information. Furthermore, by performing each of these processes multiple times, it is possible to refine the recommendations to the user.

[Server operation example]
For an example of the operation of the server 1, the housing business operator selection process shown in FIG. 27 can be applied. However, in step S602, it is determined whether only image selection information has been received. Also, in step S932, a user vector is generated based on the image selection information and text information (or only the text information). That is, a user vector C for selecting a new housing image is generated using a user vector (user vector A) generated based on the image selection information and a user vector (user vector B) generated based on the text information. In this case, the user vector C can be generated using the above-mentioned formula 1.

The processes shown in Figures 18, 21, 23, and 27 are examples for implementing this embodiment, and the order of some of the process steps may be changed within the scope of implementing this embodiment, some of the process steps may be omitted, and other process steps may be added. In addition, in each of these processes, an example is shown in which the control unit 450 controls the display state of the display screen displayed on the user terminal 2 based on information transmitted from the server 1, but the display state of the display screen displayed on the user terminal 2 may be controlled based on control on the server 1 side.

In this way, the user can request improvements to the proposed house image from the server 1, a request vector (user vector B) can be generated based on the content of the request, and the user vector can be updated using this request vector. The updated user vector is user vector C. In this way, by updating the user vector based on the content of the user's request, the user can specifically communicate his or her own requests, and can more directly influence the recommendation results of house images. As a result, it is possible to improve user satisfaction and maintain the user's interest.

[Example of providing relevant information to housing businesses]
Here, when a housing business operator is selected as one that matches the user's preferences, information to that effect is sent to that housing business operator. However, some information on the user's preferences (e.g., statistical information, user tendency information) may also be provided to housing businesses that were not selected. For example, information on the type of the selected housing image and the number of selected housing images may be provided to a housing business operator associated with a housing image that has been selected by a user. Also, for example, in the example shown in FIG. 20, information on the type of the edited housing image, the items used in the edit, and the number of edits may be provided to a housing business operator associated with an edited housing image. These may be used as statistics on the overall preferences of housing businesses, or may be used for analyzing user tendencies, etc.

[Example of executing processing in other devices or systems]
In the above, an example has been shown in which the tag information generation process, the vector generation process, the selection process, and the like are executed in the server 1 (or the information processing system IS1), but all or part of each of these processes may be executed in other devices. In this case, the information processing system is configured by each device that executes part of each of these processes. For example, at least part of each process can be executed using an information processing device managed by another service company, a device that can be used by a user (e.g., a user terminal 2, a smartphone, a tablet terminal, a personal computer), various information processing devices such as a server that can be connected via a predetermined network such as the Internet, and various electronic devices. In addition, the information stored in each DB may be managed in an external storage device and retrieved from the external storage device for use.

Furthermore, a part (or all) of the information processing system capable of executing the functions of the server 1 (or the information processing system IS1) may be provided by an application that can be provided via a predetermined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[Example of configuration and effect of this embodiment]
The information processing system IS1 is an information processing system including a user terminal 2 (an example of an electronic device) used by a user, and a server 1 (an example of an information processing device) capable of providing image information to the user terminal 2. The information processing system IS1 includes a housing contractor DB 350 (an example of a housing contractor database) that associates and stores housing contractors with housing characteristics related to the style and emotion of a house constructed by the housing contractor for each of a plurality of housing contractors, an information acquisition unit 321 (an example of an acquisition unit) that acquires from the user terminal 2 an image selected according to the user's preferences and text information indicating requests for the image, and generates respective characteristics (first characteristics) of the style and emotion related to the user's preferences based on the text information, and based on the image, calculates what is included in the image (for example, a characteristic related to the house). The image processing apparatus includes a tag generating unit 322 that generates image features (second features) related to the style and emotion of an object, and generates weights (e.g., reflection ratio α) for the first and second features based on the text information, a vector generating unit 323 (an example of a feature generating unit), and an extraction unit 324 (an example of a selection unit) that selects a housing business operator according to the user's preferences from among a plurality of housing businesses based on a comparison result obtained by comparing feature information (e.g., user vector C) indicating the user's preferences generated based on the weights, the first feature, and the second feature with the features of the house (e.g., housing business operator vector). The first feature is tag information (or a user vector A corresponding thereto) generated based on image information, and the second feature is tag information (or a user vector B corresponding thereto) generated based on text information. The user vector C is calculated by Equation 1 based on the user vector A, the user vector B, and the reflection ratio α.

According to this configuration, it is possible to select a housing provider that matches the user's preferences using a user vector C that is generated based on the text information entered by the user and the house image selected by the user. This makes it possible to propose an appropriate housing provider that matches the user's preferences. In other words, it is possible to support the user in selecting a house in an intuitive and easy-to-understand manner.

The information processing system IS1 is an information processing system including a user terminal 2 (an example of an electronic device) used by a user and a server 1 (an example of an information processing device) capable of providing image information to the user terminal 2. The information processing system IS1 includes a housing business DB 350 (an example of a housing business database) that associates and stores housing business operators with the characteristics of the house related to the style and emotion of the house constructed by the housing business operator for each of a plurality of housing businesses, an information acquisition unit 321 (an example of an acquisition unit) that acquires an image according to the user's preferences from the user terminal 2, a tag generation unit 322 that generates image characteristics related to the style and emotion of the object contained in the image (e.g., an object related to the house) based on the image, a vector generation unit 323 (an example of a feature generation unit), and an extraction unit 324 (an example of a selection unit) that selects a housing business operator according to the user's preferences from among a plurality of housing businesses based on a comparison result obtained by comparing the characteristics of the image (e.g., tag information, user vector) with the characteristics of the house in the housing business DB 350 (e.g., housing business operator vector). The information processing method according to this embodiment is an information processing method including each of these processes. Moreover, the program according to this embodiment is a program that causes a computer to execute each of these processes. In other words, the program according to this embodiment is a program that causes a computer to realize each of the functions that can be executed by the server 1.

According to this configuration, it is possible to acquire an image of the user's preference from the user terminal 2, and based on the results of comparing the image features related to the style and emotion of the house etc. contained in the acquired image with the house features in the housing provider DB 350, select a housing provider that matches the user's preference from among multiple housing providers. This makes it possible to propose an appropriate housing provider that matches the user's preference. In other words, it is possible to support the user in selecting a home in an intuitive and easy-to-understand manner.

The information acquisition unit 321 (an example of an acquisition unit) acquires a plurality of images according to the user's preferences from the user terminal 2 (an example of an electronic device). The information processing system IS1 further includes a vector generation unit 323 (an example of a feature information generation unit) that generates a user vector (an example of feature information) indicating the user's preferences regarding style and emotion based on the characteristics of each of the plurality of images. The extraction unit 324 (an example of a selection unit) selects a housing provider according to the user's preferences based on the comparison result obtained by comparing the user vector with the characteristics of the housing (housing provider vector) in the housing provider DB 350.

According to this configuration, it is possible to acquire multiple images of the user's preferences from the user terminal 2, and based on the results of comparing a user vector generated based on the characteristics of each of the acquired multiple images with the characteristics of the homes in the housing provider DB 350, select a housing provider that matches the user's preferences from among multiple housing providers. This makes it possible to propose a housing provider that is more appropriate to the user's preferences. In other words, it is possible to support the user's home selection in an intuitive and easy-to-understand manner.

The information processing system IS1 further includes an information providing unit 325 that transmits display information for displaying housing business information related to the housing business selected by the extraction unit 324 (an example of a selection unit) to the user terminal 2 (an example of an electronic device). The user terminal 2 also includes a control unit 450 that executes display control to display the housing business information on a UI unit 460 (an example of a display unit) based on the display information.

This configuration allows the user to easily visually check, on the UI unit 460 of the user terminal 2, which housing provider is selected from among multiple housing providers based on the user's preferred image, according to the user's preference. In other words, it is possible to support the user in selecting a home in an intuitive and easy-to-understand manner.

The information processing system IS1 further includes an image information DB340 (image information database) that stores housing image information for each of the homes constructed by each of the multiple housing developers in association with each of the multiple housing developers, and an information providing unit 325 that transmits display information for displaying the housing image information stored in the image information DB340 to a user terminal 2 (an example of an electronic device). The information acquiring unit 321 (an example of an acquiring unit) acquires from the user terminal 2, based on the housing image information, a housing image selected based on a user operation by the user from among the housing images displayed on the user terminal 2 (an example of an electronic device), as an image according to the user's preferences.

With this configuration, it is possible to display the house image information stored in the image information DB 340 on the UI unit 460 of the user terminal 2. This allows the user to easily visually check the house image information displayed on the UI unit 460 of the user terminal 2, and to easily select the house image information of their choice from among them. In other words, it is possible to support the user in making a house selection in an intuitive and easy-to-understand manner.

The user terminal 2 (an example of an electronic device) also includes a control unit 450 that executes display control for displaying a house image corresponding to the house image information on the UI unit 460 (an example of a display unit) based on display information transmitted from the server 1 (an example of an information processing device), and transmission control for transmitting edited image information relating to an edited house image in which the house image displayed on the UI unit 460 is edited based on user operation to the server 1. The information acquisition unit 321 (an example of an acquisition unit) of the information processing system IS1 also acquires an image corresponding to the edited image information transmitted from the user terminal 2 as an image according to the user's preferences.

According to this configuration, by enabling editing of the house image displayed on the user terminal 2 according to the user's preferences, it is possible to more appropriately express the user's preferences based on the user's subjective editing. Therefore, it is possible to propose appropriate housing providers according to the user's preferences based on the user's editing results. In other words, it is possible to support the user in selecting a house in an intuitive and easy-to-understand manner.

The user terminal 2 (an example of an electronic device) also includes an image information DB 800 (an example of an image information database) that stores images acquired by the image acquisition unit 430, and a control unit 450 that executes display control to display images stored in the image information DB 800 on a UI unit 460 (an example of a display unit), and transmission control to transmit selected image information relating to an image selected from the images displayed on the UI unit 460 based on a user operation to the server 1 (an example of an information processing device). The information acquisition unit 321 (an example of an acquisition unit) of the information processing system IS1 also acquires an image corresponding to the selected image information transmitted from the user terminal 2 as an image according to the user's preferences.

According to this configuration, the user can select images that match the user's preferences from among images acquired using the user terminal 2 (or other external device), making it possible to more appropriately express the user's preferences based on the user's proactive image collection. This makes it possible to propose appropriate housing providers that match the user's preferences based on the results of the user's collection. In other words, it becomes possible to support the user's housing selection in an intuitive and easy-to-understand manner.

In addition, the tag generation unit 322 (an example of a feature generation unit) of the server 1 (an example of an information processing device) extracts image features related to style and emotion using an AI model that extracts features related to what is contained in the target image.

With this configuration, it is possible to extract image features related to style and emotion for any image sent from the user terminal 2 according to the user's preferences. This makes it possible to suggest appropriate housing providers according to the user's preferences. In other words, it becomes possible to support the user in selecting a home in an intuitive and easy-to-understand way.

The information processing system IS1 further includes an image information DB 340 (image information database) that stores housing image information on each of the housing businesses constructed by each of the housing businesses in association with each of the housing businesses, and an information providing unit 325 that transmits display information for displaying the housing image information stored in the image information DB 340 to the user terminal 2 (an example of an electronic device). The information acquiring unit 321 (an example of an acquiring unit) acquires from the user terminal 2 a housing image selected based on a user operation by the user from among the housing images displayed on the UI unit 460 of the user terminal 2 based on the housing image information as an image according to the user's preferences. When a re-proposal request for re-proposal of a new housing business other than the selected housing business is transmitted from the user terminal 2, the information providing unit 325 transmits display information for displaying the housing image information extracted from the housing image information stored in the image information DB 340 as a housing image according to the user's preferences to the user terminal 2 based on the image features (e.g., tag information, user vector) generated by the tag generating unit 322 and the vector generating unit 323 (an example of a feature generating unit).

According to this configuration, the user can use the user terminal 2 to make a re-proposal request to request a re-proposal of a new housing provider other than the one proposed to the user. This makes it possible to easily request a re-proposal of a housing provider that meets the user's preferences, even if a housing provider that is not to the user's preference is proposed. In this case, since the housing image used to select a housing provider can be selected based on the characteristics of an image once selected by the user, it is possible to provide the user with a housing image that is close to the user's preferences. This makes it possible to propose a more appropriate housing provider that meets the user's preferences.

The server 1 (an example of an information processing device) includes an information acquisition unit 321 (an example of an image acquisition unit) that acquires an image according to the user's preferences, a tag generation unit 322 that acquires features related to the image, a vector generation unit 323 (an example of a feature acquisition unit), and an extraction unit 324 (an example of a selection unit) that selects a housing company according to the user's preferences from among a plurality of housing companies based on a comparison result obtained by comparing the features related to the image with the features of each housing company related to houses constructed by each of the plurality of housing companies. The information processing method according to this embodiment is an information processing method that includes each of these processes. The program according to this embodiment is a program that causes a computer to execute each of these processes. In other words, the program according to this embodiment is a program that causes a computer to realize each function that the server 1 can execute. For example, the tag generation unit 322 acquires features (tag information) related to the image acquired by the information acquisition unit 321. The vector generation unit 323 generates a user vector based on the tag information generated by the tag generation unit 322.

The processes in this embodiment are executed based on a program for causing a computer to execute various processing procedures. This embodiment can also be understood as an embodiment of a program that realizes the function of executing each of these processes, and a recording medium that stores the program. For example, the program can be stored in each storage device of each device by an update process for adding new functions to an information processing device, electronic device, etc. This makes it possible to cause each updated device to execute each of the processes shown in this embodiment.

Although the embodiments of the present invention have been described above, the above embodiments merely show some of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

1...server, 11...control unit, 60...image information acquisition unit, 62...tag information acquisition unit, 64...vector generation unit, 66...housing business selection unit, 400...housing business DB, 2...user terminal, 200...control unit, 220...display image management unit, 222...image selection reception unit, 224...housing business information acquisition unit, 3-1 to 3-n...housing business terminal, IS1...information processing system, 310...communication unit, 320...control unit, 321...information acquisition unit, 322...tag generation unit, 323...vector generation unit, 324...extraction unit, 325...information provision unit, 330...storage unit, 340...image information DB, 350...housing business DB, 360...tag information DB, 410...communication unit, 420...location information acquisition unit, 430...image acquisition unit, 440...sound acquisition unit, 450...control unit, 460...UI unit, 461...reception unit, 462...output unit, 470...storage unit

Claims (21)

An information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device,
a housing developer database that stores the characteristics of houses constructed by a housing developer and the housing developer in association with each of a plurality of housing developers;
an acquisition unit that acquires from the electronic device an image selected according to the user's preference and text information indicating a request for the image;
a generation unit that generates a first feature related to a preference of the user based on the text information, generates a second feature that is a feature of the image related to what is included in the image based on the image, and generates weights for the first feature and the second feature based on the text information;
an information processing system comprising: feature information indicating the user's preferences generated based on the weights, the first feature, and the second feature; and a selection unit that selects a housing business operator from among the multiple housing businesses based on a comparison result of comparing the feature information with the features of the housing. 2. The information processing system according to claim 1,
The housing business database stores, as the characteristics of the house, characteristics related to the style and emotion of the house;
An information processing system in which the generation unit generates, as the first features, features of style and emotion related to the user's preferences, and generates, as the second features, features related to the style and emotion of what is included in the image. An information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device,
a housing developer database that stores the characteristics of houses constructed by a housing developer and the housing developer in association with each of a plurality of housing developers;
an image information database for storing images of each of the houses constructed by each of the plurality of housing developers;
an acquisition unit that acquires an image according to the user's preference from the electronic device;
A feature acquisition unit that acquires features of the image based on the image;
a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the characteristics of the image with the characteristics of the house ;
an information providing unit that, when a new image re-proposal is requested by an operation of the user, causes the electronic device to display the house image extracted from the image information database based on the features of the image acquired by the feature acquiring unit;
An information processing system comprising: An information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device,
a housing developer database that stores the characteristics of houses constructed by a housing developer and the housing developer in association with each of a plurality of housing developers;
an acquisition unit that acquires an image according to the user's preference from the electronic device;
A feature acquisition unit that acquires features of the image based on the image;
a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the image characteristics with the housing characteristics ;
an information providing unit that causes the electronic device to display housing business operator information related to the housing business operator selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected;
An information processing system comprising: 5. The information processing system according to claim 3,
The housing business database stores, as the characteristics of the house, characteristics related to the style and emotion of the house;
The feature acquisition unit is an information processing system that generates and acquires features related to style and emotion of the objects contained in the image as features of the image. 6. The information processing system according to claim 5 ,
The acquisition unit acquires a plurality of images according to the user's preferences from the electronic device;
a feature information generating unit configured to generate feature information indicating the user's preferences regarding the style and the emotion based on features of each of the plurality of images;
The selection unit is an information processing system that selects a housing business operator that meets the user's preferences based on a comparison result obtained by comparing the characteristic information with the characteristics of the home. 4. The information processing system according to claim 3 ,
the information providing unit transmits to the electronic device display information for displaying housing business operator information related to the housing business operator selected by the selection unit;
The electronic device is an information processing system including a control unit that executes display control to display the housing business owner information on a display unit based on the display information. 5. The information processing system according to claim 4 ,
An image information database is further provided that stores images of houses constructed by each of the plurality of housing developers in association with each of the plurality of housing developers,
the information providing unit transmits display information for displaying the house image stored in the image information database to the electronic device;
The acquisition unit is an information processing system that acquires from the electronic device a house image selected from the house images displayed on the electronic device based on the display information based on user operation by the user as an image according to the user's preferences. 9. The information processing system according to claim 8 ,
The electronic device includes:
display control for displaying the house image on a display unit based on the display information;
a control unit that executes a transmission control to transmit edited image information regarding an edited house image in which the house image displayed on the display unit is edited based on a user operation to the information processing device;
The acquisition unit acquires an image corresponding to the edited image information transmitted from the electronic device as an image according to the user's preferences. 5. The information processing system according to claim 3,
The electronic device includes:
an image database for storing images acquired by the image acquisition unit;
a control unit that executes a display control for displaying images stored in the image database on a display unit, and a transmission control for transmitting selected image information related to an image selected based on a user operation from among the images displayed on the display unit to the information processing device,
The acquisition unit acquires an image corresponding to the selected image information transmitted from the electronic device as an image according to the user's preference. 6. The information processing system according to claim 5 ,
The feature acquisition unit is an information processing system that extracts and acquires features of the image related to the style and the emotion using an AI model that extracts features related to what is contained in the target image. 9. The information processing system according to claim 8 ,
The information processing system includes an information providing unit that, when a request for re-proposition of a new image is sent from the electronic device in response to a user operation using the operation image , sends display information to the electronic device for displaying a house image extracted from the house images stored in the image information database as an image that corresponds to the user's preferences based on the features of the image acquired by the feature acquisition unit. An information processing device capable of handling a database that stores characteristics of houses constructed by a housing developer in association with the housing developer for each of a plurality of housing developers,
an acquisition unit that acquires an image selected according to a user's preference and text information indicating a request for the image;
a generation unit that generates a first feature related to a preference of the user based on the text information, generates a second feature that is a feature of the image related to what is included in the image based on the image, and generates weights for the first feature and the second feature based on the text information;
and a selection unit that selects a housing business operator according to the user's preferences from among the multiple housing businesses based on a comparison result of feature information indicating the user's preferences generated based on the weight, the first feature, and the second feature and the features of the house. An information processing device capable of handling a database that stores characteristics of houses constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, and an image information database that stores images of houses constructed by each of the plurality of housing developers ,
An acquisition unit that acquires an image according to a user's preference;
A feature acquisition unit that acquires features of the image based on the image;
a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the image characteristics with the housing characteristics ;
an information providing unit that displays, on a display unit, the house image extracted from the image information database based on the features of the image acquired by the feature acquiring unit, when a new image re-proposition is requested by an operation of the user;
An information processing device comprising: An information processing device capable of handling a database that stores characteristics of houses constructed by a housing developer in association with the housing developer for each of a plurality of housing developers,
An acquisition unit that acquires an image according to a user's preference;
A feature acquisition unit that acquires features of the image based on the image;
a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the image characteristics with the housing characteristics ;
an information providing unit that displays on a display unit housing business operator information related to the housing business operator selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected;
An information processing device comprising: An information processing method to be executed by a computer capable of handling a database that associates and stores characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers, comprising:
a generation process for generating a first feature related to the user's preference based on the text information when an image selected according to a user's preference and text information indicating a request for the image are acquired, generating a second feature that is a feature of the image related to what is included in the image based on the image, and generating weights for the first feature and the second feature based on the text information;
an information processing method including: a selection process for selecting a housing business operator according to the user's preferences from among the plurality of housing businesses based on a comparison result of feature information indicating the user's preferences generated based on the weights, the first feature, and the second feature and the features of the housing. An information processing method to be executed by a computer capable of handling a database that stores characteristics of homes constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, and an image information database that stores images of homes constructed by each of the plurality of housing developers , comprising:
an acquisition process for acquiring an image according to a user's preference based on the image and acquiring a feature of the image;
a selection process for selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics ;
an information providing process for displaying, on a display unit, the house image extracted from the image information database based on the features of the image acquired in the acquisition process, when a new image re-proposition is requested by an operation of the user;
An information processing method comprising: An information processing method to be executed by a computer capable of handling a database that associates and stores characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers, comprising:
an acquisition process for acquiring an image according to a user's preference based on the image and acquiring a feature of the image;
a selection process for selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics ;
an information providing process in which housing business operator information related to the housing business operator selected in the selection process and an operation image for performing a user operation to request re-proposition of a new image to be selected are displayed on a display unit;
An information processing method comprising: A program to be executed by a computer capable of handling a database that associates and stores the characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers,
a generation step of generating a first feature related to the user's preference based on the text information when an image selected according to a user's preference and text information indicating a request for the image are acquired, generating a second feature that is a feature of the image related to what is included in the image based on the image, and generating weights for the first feature and the second feature based on the text information;
A program that causes a computer to execute feature information indicating the user's preferences generated based on the weights, the first feature, and the second feature, and a selection procedure that selects a housing operator that meets the user's preferences from among the multiple housing operators based on a comparison result of comparing the feature information with the features of the housing. A program to be executed by a computer capable of handling a database that stores characteristics of homes constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, and an image information database that stores images of homes constructed by each of the plurality of housing developers ,
an acquisition step of acquiring an image according to a user's preference based on the image and acquiring a feature of the image;
a selection step of selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics ;
an information providing step of displaying, on a display unit, the house image extracted from the image information database based on the features of the image acquired in the acquisition step, when a new image re-proposal is requested by an operation of the user;
A program that causes a computer to execute the following. A program to be executed by a computer capable of handling a database that associates and stores the characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers,
an acquisition step of acquiring an image according to a user's preference based on the image and acquiring a feature of the image;
a selection step of selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics ;
an information providing step of displaying on a display unit housing business operator information related to the housing business operator selected in the selection step and an operation image for performing a user operation to request re-proposition of a new image to be selected;
A program that causes a computer to execute the following.

Images