JP7828590B2 - Information processing device and information processing program - Google Patents

Description

This invention relates to an information processing device and an information processing program.

Conventionally, technologies are known that perform searches based on search queries entered by users and provide search results to those users. For example, a known technology involves receiving a natural language search query from a user, receiving a natural language search execution command from the user, obtaining information indicating the status of the natural language search execution command, and using the obtained information to process the query.

Japanese Patent Publication No. 2022-108035

However, the conventional technologies described above do not always guarantee that users will receive search results that address the ambiguity of their desired search criteria.

This application aims to provide users with search results that address the ambiguity of the search target they desire.

The information processing device according to this application comprises: a receiving unit that receives input information entered by a user of the search system; a generating unit that inputs the input information into a machine learning model, causes the machine learning model to generate question information indicating a question for identifying the search target desired by the user, obtains response information indicating a response to the question information, generates a search query according to the response information, generates output information corresponding to the search results corresponding to the search query; and an output control unit that outputs the output information.

According to one embodiment, search results can be provided to the user after supplementing the ambiguity of the search target desired by the user.

Figure 1 is a diagram illustrating the outline of the search process related to the conventional technology. Figure 2 is a diagram illustrating the overview of the search process according to the embodiment. Figure 3 shows an example of the configuration of an information processing system according to the embodiment. Figure 4 shows an example of the configuration of an information processing device according to the embodiment. Figure 5 is a diagram illustrating an example of information processing according to the embodiment. Figure 6 shows an example of a prompt according to the embodiment. Figure 7 shows an example of a prompt according to the embodiment. Figure 8 shows an example of a prompt according to the embodiment. Figure 9 shows an example of a prompt according to the embodiment. Figure 10 shows an example of a prompt according to the embodiment. Figure 11 is a flowchart showing the information processing procedure by the information processing device according to the embodiment. Figure 12 is a diagram illustrating an example of information processing related to a modified example. Figure 13 is a hardware configuration diagram showing an example of a computer that implements the functions of an information processing device.

The following describes in detail, with reference to the drawings, the embodiments for implementing the information processing apparatus and information processing program according to this application (hereinafter referred to as "embodiments"). Note that these embodiments do not limit the information processing apparatus and information processing program according to this application. Furthermore, the same parts are denoted by the same reference numerals in each of the following embodiments, and redundant descriptions are omitted.

(Embodiment)
[1. Introduction]
Figure 1 is a diagram illustrating the outline of a search process according to the prior art. In Figure 1, a terminal device 10 of user U1 using the search system 200 transmits a search query entered by user U1 to the search system 200. The search system 200 receives the search query from user U1. Specifically, the search system 200 obtains the search query from the terminal device 10. When the search system 200 obtains the search query, it performs a search based on the search query and transmits the search results to the terminal device 10. The terminal device 10 receives the search results from the search system 200.

Figure 2 is a diagram illustrating the overview of the search process according to the embodiment. Figure 2 differs from Figure 1 in that the information processing device 100, according to the embodiment, plays a role in connecting the user U1 who uses the search system 200 with the search system 200 by using a machine learning model M1. Specifically, when user U1 uses the search system 200, the object they want to search for (hereinafter sometimes referred to as "search target") may be ambiguous. For example, one case of an ambiguous search target is when the name of the object to be searched is unknown. Another case of an ambiguous search target is when only some of the characteristics of the object to be searched are known. In response to this, the information processing device 100 uses the machine learning model M1, which plays a role in complementing the ambiguity of the search target when user U1 uses the search system 200, to complement the ambiguity of the search target desired by user U1.

Specifically, the information processing device 100 inputs input information (for example, a search query containing ambiguous information) entered by user U1 using the search system 200 into the machine learning model M1 to generate information to complement the ambiguity of the search target desired by user U1. More specifically, as information to complement the ambiguity of the search target desired by user U1, the information processing device 100 generates question information indicating questions to identify the search target desired by user U1. In other words, as information to complement the ambiguity of the search target desired by user U1, the information processing device 100 generates question information indicating questions to clarify the search target desired by user U1. For example, the information processing device 100 inputs input information into the machine learning model M1 to generate question information indicating questions to identify the search target desired by user U1. In other words, the question information is information indicating what should be asked of user U1 in order to identify the search target desired by user U1. In other words, the question information is information that indicates the content of the questions posed to user U1 in order to identify the search target desired by user U1. For example, the machine learning model M1 may be a language model that generates and outputs information in response to the input information. For example, the machine learning model M1 may be a Large Language Model (LLM). For example, the information processing device 100 inputs input information into the machine learning model M1 and causes the machine learning model M1 to generate question information in response to the input information. For example, the information processing device 100 inputs text-based input information into the machine learning model M1. The information processing device 100 also causes the machine learning model M1 to generate text-based question information.

In Figure 2, the terminal device 10 of user U1 transmits input information entered by user U1 using the search system 200 to the information processing device 100. For example, the input information may be information entered by user U1 with some search intent. For example, the input information may be a search query containing ambiguous information. The information processing device 100 receives the input information from user U1. The information processing device 100 acquires the input information. Upon acquiring the input information, the information processing device 100 inputs the input information into the machine learning model M1, causing the machine learning model M1 to generate question information indicating a question to identify the search target desired by user U1. In this way, the information processing device 100 generates question information by causing the machine learning model M1 to generate the question information. When the information processing device 100 generates question information, it transmits the question information to the terminal device 10. In this way, the information processing device 100 can compensate for the ambiguity of the search target desired by user U1 by generating question information according to the input information and transmitting the generated question information to the terminal device 10.

Although not shown in the diagram, when the terminal device 10 receives question information, it displays the received question information on the screen. The terminal device 10 also transmits response information, indicating the answer to the question, to the information processing device 100 as input information. The information processing device 100 acquires the response information from the terminal device 10. Upon acquiring the response information, the information processing device 100 inputs the response information into the machine learning model M1, causing the machine learning model M1 to generate a search query corresponding to the response information. In this way, the information processing device 100 generates a search query by causing the machine learning model M1 to generate the search query. Furthermore, by acquiring response information corresponding to the question information and generating a search query corresponding to the response information, the information processing device 100 can generate a search query that complements the ambiguity of the search target desired by the user U1.

Furthermore, in Figure 2, the information processing device 100 inputs the search query generated by the machine learning model M1 to the search system 200. When the search system 200 receives the search query, it executes the search based on the query and transmits the search results to the information processing device 100. The information processing device 100 retrieves the search results from the search system 200. Also, when the information processing device 100 receives the search results, it inputs the search results to the machine learning model M1, causing the machine learning model M1 to generate output information corresponding to the search results. In this way, the information processing device 100 generates output information by causing the machine learning model M1 to generate it. The information processing device 100 then transmits the output information to the terminal device 10. The terminal device 10 receives the output information from the information processing device 100.

As described above, the information processing device 100 receives input information entered by user U1 using the search system 200. The information processing device 100 also inputs the input information into the machine learning model M1, causing the model M1 to generate question information indicating a question to identify the search target desired by user U1. It then obtains response information indicating a response to the question information, generates a search query corresponding to the response information, and generates output information corresponding to the search results that correspond to the search query. The information processing device 100 also outputs the output information. In this way, the information processing device 100 can provide user U1 with search results that correspond to the search query while resolving any ambiguity regarding the search target desired by user U1. Therefore, the information processing device 100 can provide user U1 with search results that resolve any ambiguity regarding the search target desired by user U1.

[2. Configuration of the Information Processing System]
An example configuration of the information processing system 1 according to the embodiment will be described using Figure 3. Figure 3 is a diagram showing an example configuration of the information processing system 1 according to the embodiment. As shown in Figure 3, the information processing system 1 includes a terminal device 10, an information processing device 100, and a search system 200. The terminal device 10, the information processing device 100, and the search system 200 are connected to each other via a network N, either by wire or wireless means, enabling communication between them. The network N is, for example, a WAN (Wide Area Network) such as the Internet. Note that the information processing system 1 shown in Figure 3 may include multiple terminal devices 10, multiple information processing devices 100, and multiple search systems 200.

The terminal device 10 is an information processing device used by user U1 who uses the search system 200. The terminal device 10 can be implemented as, for example, a smartphone, a tablet, a notebook PC (Personal Computer), a desktop PC, a mobile phone, or a PDA (Personal Digital Assistant). The terminal device 10 displays information received from the information processing device 100, etc., using a web browser or application. In the example shown in Figure 2, the terminal device 10 is a smartphone.

The information processing device 100 is an information processing device that performs information processing according to the embodiment, and can be implemented, for example, by a server device or a cloud system. In the example in Figure 2, the information processing device 100 receives input information entered by user U1 using the search system 200. The information processing device 100 also inputs the input information into a machine learning model M1, causing the machine learning model M1 to generate question information indicating a question for identifying the search target desired by user U1, obtains response information indicating a response to the question information, generates a search query according to the response information, and generates output information corresponding to the search results that correspond to the search query. The information processing device 100 also outputs the output information.

The search system 200 is an information processing device that provides search services, and can be implemented, for example, by a server device or a cloud system. In the example shown in Figure 2, when the search system 200 receives a search query, it performs a search based on the query and transmits the search results to the information processing device 100.

[3. Configuration of Information Processing Equipment]
An example of the configuration of the information processing device 100 according to the embodiment will be described using Figure 4. Figure 4 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. The information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

(Communications Department 110)
The communication unit 110 is implemented using a NIC (Network Interface Card), an antenna, etc. The communication unit 110 is connected to various networks by wired or wireless means, and performs information transmission and reception with, for example, a terminal device 10 or a search system 200.

(Storage unit 120)
The memory unit 120 is implemented by, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disc. Specifically, the memory unit 120 stores various types of data. For example, the memory unit 120 may store various types of information received by the receiving unit 132. The memory unit 120 may also store various types of information generated by the generation unit 133. The memory unit 120 may also store various types of information acquired by the generation unit 133. For example, the memory unit 120 stores various programs. For example, the memory unit 120 stores an information processing program according to the embodiment. The memory unit 120 may also store information related to various machine learning models. For example, the memory unit 120 stores information related to a machine learning model M1, which is a large-scale language model or a visual language model. The memory unit 120 also stores information related to an image generation model M2. The memory unit 120 also stores information related to an image recognition model M3. Furthermore, the memory unit 120 stores information related to the speech recognition model M4. The memory unit 120 may also store information related to machine learning models that recognize information acquired by various sensors.

(Control unit 130)
The control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in the memory device inside the information processing device 100 using RAM as the working area. Alternatively, the control unit 130 is a controller and can be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

The control unit 130 has an instruction unit 131, a reception unit 132, a generation unit 133, and an output control unit 134 as functional units, and may realize or execute the information processing operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Figure 4; other configurations are also acceptable as long as they perform the information processing described later. Furthermore, each functional unit represents a function of the control unit 130 and does not necessarily have to be physically distinct.

(Instruction unit 131)
The instruction unit 131 instructs the machine learning model M1 to identify the search target desired by the user of the search system, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query. Specifically, the instruction unit 131 may instruct the machine learning model M1, which is a language model that generates and outputs information according to the input information, to identify the search target desired by the user, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query. More specifically, the machine learning model M1 may be a language model that has been trained to estimate and output the next token from an input token sequence. For example, the machine learning model M1 may be a large-scale language model (LLM). For example, the machine learning model M1 may be OpenAI's gpt-3.5 or gpt-4.

Figure 5 is a diagram illustrating an example of information processing according to the embodiment. In Figure 5, the instruction unit 131 instructs the machine learning model M1 to identify the search target desired by user U1 using the search system 200, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query (step S11). Figure 5 illustrates the case where the machine learning model M1 is a large-scale language model. Specifically, the instruction unit 131 inputs a prompt to the machine learning model M1 instructing it to identify the search target desired by user U1, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query. Hereafter, "instructing the model to identify the search target desired by user U1 using the search system 200, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query" may be referred to as "instructing as described above."

For example, the instruction unit 131 gives the above-described instructions to a machine learning model M1 (for example, OpenAI's gpt-3.5 or gpt-4) that can receive system prompts and user prompts separately. For example, the instruction unit 131 gives the above-described instructions by inputting a system prompt to the machine learning model M1 that instructs it to identify the search target desired by user U1, generate a search query corresponding to the identified search target, and obtain the search results corresponding to the generated search query. For example, the instruction unit 131 may give the above-described instructions by inputting the prompt P1 shown in Figure 6 to the machine learning model M1. Figure 6 is a diagram showing an example of a prompt according to the embodiment. The prompt P1 shown in Figure 6 is a system prompt containing the following text: "Assist users of the search system. Engage in conversation with the user to clarify the object they wish to search for. After considering and structuring search information that accurately represents the clarified object, call the search system. The following tools can be used in conversation with the user and in considering and structuring search information: Image recognition tool (Input: Image file name, Output: Text describing what is in the image). Image generation tool (Input: Text, Output: Image based on the text). When the user provides information they wish to search for, engage in conversation while clearly stating your own consideration process."

For example, the instruction unit 131 instructs the machine learning model M1 to identify the search target desired by user U1 by inputting a prompt P1 containing the text shown in Figure 6, "Assist users of the search system. Clarify the target the user wishes to search for by engaging in conversation with the user." Furthermore, the instruction unit 131 instructs the machine learning model M1 to generate a search query corresponding to the identified search target and retrieve search results corresponding to the generated search query by inputting another prompt P1 containing the text shown in Figure 6, "Consider and configure search information that accurately represents the clarified target."

(Reception desk 132)
The reception unit 132 receives input information entered by users of the search system. In Figure 5, the reception unit 132 receives input information entered by user U1 of the search system 200 (step S12). For example, the reception unit 132 may receive input information from user U1's terminal device 10. For example, the reception unit 132 may receive input text entered by user U1 as input information. For example, the input text may be a sentence. Also, when the reception unit 132 receives input information, it may output the input information to the generation unit 133.

(Generation unit 133)
The generation unit 133 inputs the input information into a machine learning model, causes the machine learning model to generate question information indicating a question to identify the search target desired by the user, obtains response information indicating a response to the question information, generates a search query according to the response information, and generates output information corresponding to the search results that correspond to the search query. For example, the generation unit 133 may obtain the input information from the reception unit 132. When the generation unit 133 obtains the input information, it generates question information according to the input information. The generation unit 133 generates question information corresponding to the input information. Specifically, the generation unit 133 inputs the input information into a machine learning model, causing the machine learning model to generate question information indicating a question to identify the search target desired by the user. In Figure 5, the generation unit 133 inputs the input information into the machine learning model M1, causing the machine learning model M1 to generate question information indicating a question to identify the search target desired by user U1 (step S13). More specifically, the generation unit 133 inputs a prompt to the machine learning model M1, along with the input information, instructing it to generate question information that indicates a question for identifying the search target desired by the user, causing the machine learning model M1 to generate question information corresponding to the input information. For example, the generation unit 133 may input the input text entered by the user U1 as input information to the machine learning model M1, and cause the machine learning model M1 to generate question text corresponding to the input text as question information. For example, the question text may be a sentence. Furthermore, if the generation unit 133 has caused the machine learning model M1 to generate question information, it may output the question information to the output control unit 134.

For example, the generation unit 133 may input a prompt P2 to the machine learning model M1 along with the input information, which includes the text shown in the upper part of Figure 7: "Is the information entered by the user sufficient to perform the search? Answer Yes or No." Figure 7 is a diagram showing an example of a prompt according to the embodiment. Furthermore, if the machine learning model M1 outputs "No" in response to the input of prompt P2, the generation unit 133 may input a prompt P3 to the machine learning model M1, which includes the text shown in the middle part of Figure 7: "Is it necessary to use a tool to ask the user for clarification? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "No" in response to the input of prompt P3, the generation unit 133 may input a prompt P7 to the machine learning model M1, which includes the text shown in the upper part of Figure 9: "What information should be asked of the user for clarification? Write the wording to ask the user." For example, the generation unit 133 may input prompt P7 to the machine learning model M1 as a prompt instructing it to generate question information indicating a question to identify the search target desired by the user. Figure 9 shows an example of a prompt according to the embodiment. The generation unit 133 may obtain the text output from the machine learning model M1 in response to the input of prompt P7 as question information. In this way, the generation unit 133 may cause the machine learning model M1 to generate question information in the form of text. The generation unit 133 may then generate question information in the form of text. The generation unit 133 may also input prompts P2 to P11 as system prompts to the machine learning model M1.

Furthermore, if the machine learning model M1 outputs "Yes" in response to prompt P2, the generation unit 133 may input prompt P4 to the machine learning model M1 containing the text "Do you need to use a tool to perform the search? Answer Yes or No." as shown in the lower part of Figure 7. Also, if the machine learning model M1 outputs "No" in response to prompt P4, the generation unit 133 may input prompt P8 to the machine learning model M1 containing the text "What information are you searching for? Output text or file names." as shown in the middle part of Figure 9. The generation unit 133 may also obtain the text or file names output by the machine learning model M1 as a search query in response to prompt P8. In this way, the generation unit 133 may cause the machine learning model M1 to generate a search query in text form. In this way, the generation unit 133 may cause the machine learning model M1 to generate a search query in text form. In this way, the generation unit 133 may generate a search query in text form.

(Output control unit 134)
The output control unit 134 outputs various information. For example, the output control unit 134 may acquire various information from the generation unit 133. When the output control unit 134 acquires various information, it may output the various information. In Figure 5, the output control unit 134 acquires question information from the generation unit 133. The output control unit 134 also outputs the question information. In Figure 5, when the output control unit 134 acquires question information, it outputs the question information (step S14). For example, the output control unit 134 may output the question information to the terminal device 10 of user U1. The output control unit 134 may transmit the question information to the terminal device 10 of user U1. When the terminal device 10 receives the question information, it may display the question information on the screen. The terminal device 10 may also transmit the information entered by user U1 to the information processing device 100 within a predetermined time after displaying the question information on the screen.

Furthermore, the reception unit 132 receives response information indicating a response to the question information. Specifically, the reception unit 132 receives response information entered by the user. In Figure 5, the reception unit 132 receives response information entered by user U1 (step S15). For example, the reception unit 132 may receive response information from user U1's terminal device 10. For example, the reception unit 132 may receive information entered into the terminal device 10 as response information within a predetermined time after the terminal device 10 displays the question information on the screen. For example, the reception unit 132 may receive the response text entered by user U1 as response information. Also, when the reception unit 132 receives response information, it may output the response information to the generation unit 133.

Furthermore, the generation unit 133 acquires response information from the reception unit 132. When the generation unit 133 acquires response information, it generates a search query based on that information. Specifically, the generation unit 133 inputs the response information into a machine learning model to cause the machine learning model to generate the search query. In Figure 5, the generation unit 133 inputs the response information into the machine learning model M1 to cause the machine learning model M1 to generate the search query (step S16). More specifically, the generation unit 133 inputs a prompt to the machine learning model M1 along with the response information, instructing it to generate a search query corresponding to the response information, causing the machine learning model M1 to generate the search query corresponding to the response information. For example, the generation unit 133 may input the response text entered by user U1 as the response information into the machine learning model M1, and cause the machine learning model M1 to generate search text corresponding to the response text as the search query. For example, the search text may be a sentence.

For example, the generation unit 133 may input a prompt P2 to the machine learning model M1 along with the response information, which includes the text shown in the upper part of Figure 7: "Is the information entered by the user sufficient to perform the search? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "Yes" in response to the input of prompt P2, the generation unit 133 may input a prompt P4 to the machine learning model M1, which includes the text shown in the lower part of Figure 7: "Is it necessary to use a tool to perform the search? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "No" in response to the input of prompt P4, the generation unit 133 may input a prompt P8 to the machine learning model M1, which includes the text shown in the middle part of Figure 9: "What information are you searching for? Output text or file names." For example, the generation unit 133 may input prompt P8 to the machine learning model M1 as a prompt instructing it to generate a search query corresponding to the response information. Furthermore, the generation unit 133 may obtain the text and filename output from the machine learning model M1 as search queries in response to the input of prompt P8. In this way, the generation unit 133 may cause the machine learning model M1 to generate search queries in the form of text. In this way, the generation unit 133 may cause the machine learning model M1 to generate search queries in the form of text. In this way, the generation unit 133 may generate search queries in the form of text.

Furthermore, if the machine learning model M1 outputs "No" in response to prompt P2, the generation unit 133 may input prompt P3 to the machine learning model M1 containing the text shown in the middle of Figure 7, "Is it necessary to use a tool to ask the user for clarification? Answer with Yes or No." Also, if the machine learning model M1 outputs "No" in response to prompt P3, the generation unit 133 may input prompt P7 to the machine learning model M1 containing the text shown in the upper part of Figure 9, "What information should be asked of the user for clarification? Write the wording to ask the user." The generation unit 133 may also obtain the text output by the machine learning model M1 in response to prompt P7 as new question information. Furthermore, if the generation unit 133 has caused the machine learning model M1 to generate new question information, it may output the new question information to the output control unit 134.

Furthermore, in Figure 5, when the generation unit 133 generates a search query corresponding to the response information, it inputs the generated search query to the search system 200 (step S17). For example, the generation unit 133 may transmit the generated search query to the search system 200. The generation unit 133 also obtains the search results corresponding to the search query (step S18). For example, the generation unit 133 may obtain the search results corresponding to the search query from the search system 200. The generation unit 133 may receive the search results corresponding to the search query from the search system 200.

Furthermore, when the generation unit 133 obtains search results, it generates output information corresponding to the search results based on those results. Specifically, the generation unit 133 inputs the search results into a machine learning model and causes the machine learning model to generate output information corresponding to the search results. In Figure 5, the generation unit 133 inputs the search results into the machine learning model M1 and causes the machine learning model M1 to generate output information corresponding to the search results (step S19). More specifically, along with the search results, the generation unit 133 inputs a prompt to the machine learning model M1 instructing it to generate output information corresponding to the search results, causing the machine learning model M1 to generate output information corresponding to the search results. For example, when the generation unit 133 obtains search results, it may input a prompt P9 to the machine learning model M1 along with the search results, which includes the sentence "Is it necessary to use a tool to determine whether the search results are in line with your search intent? Answer Yes or No," as shown in the lower part of Figure 9. Also, if the machine learning model M1 outputs "No" in response to the input of prompt P9, the generation unit 133 may determine whether the similarity between the search query and the search results exceeds a predetermined threshold. For example, the generation unit 133 may determine whether the similarity between the text search query and the text search result is above a predetermined threshold. If the generation unit 133 determines that the similarity between the text search query and the text search result is above a predetermined threshold, it may determine that the search result is in line with the search intent. Furthermore, if the generation unit 133 determines that the search result is in line with the search intent, it may input a prompt P11 to the machine learning model M1 that includes the following text, as shown in the lower part of Figure 10: "Summarize or paraphrase the search results to return them to the user. Include the wording to be returned to the user as the search result." Figure 10 is a diagram showing an example of a prompt according to the embodiment. For example, the generation unit 133 may input prompt P11 to the machine learning model M1 as a prompt instructing it to generate output information corresponding to the search result. The generation unit 133 may obtain the text output from the machine learning model M1 in response to the input of prompt P11 as output information. For example, the generation unit 133 may obtain a text that summarizes the text search result as output information. Furthermore, the generation unit 133 may obtain output information that paraphrases the search results, which are text. In this way, the generation unit 133 may cause the machine learning model M1 to generate the output information, which is text. In this way, the generation unit 133 may generate the output information, which is text.

Furthermore, if the generation unit 133 determines that the similarity between the text search query and the text search results is not above a predetermined threshold (i.e., below a predetermined threshold), it may determine that the search results do not conform to the search intent. If the generation unit 133 determines that the search results do not conform to the search intent, it may input a prompt P10 to the machine learning model M1 containing the text shown in the upper part of Figure 10, "Modify the search information to make the search results appropriate. Do you need to use a tool? Answer Yes or No." Also, if the machine learning model M1 outputs "No" in response to the input of prompt P10, the generation unit 133 may generate a new search query. For example, the generation unit 133 may generate a search query different from the search query for which the similarity to the search results was determined to be below a predetermined threshold. For example, the generation unit 133 may again input a prompt P8 to the machine learning model M1 containing the text shown in the middle part of Figure 9, "What information are you searching for? Output text or file names." Furthermore, the generation unit 133 may acquire the text and filename output again from the machine learning model M1 in response to the input of prompt P8 as a new search query.

Furthermore, the output control unit 134 outputs output information based on the search results. In Figure 5, the output control unit 134 may acquire output information from the generation unit 133. When the output control unit 134 acquires output information, it outputs the output information to the user U1's terminal device 10 (step S20). For example, the output control unit 134 may transmit the output information to the user U1's terminal device 10.

[4. Processing Procedure]
Figure 11 is a flowchart showing the information processing procedure by the information processing device according to the embodiment. In Figure 11, the instruction unit 131 of the information processing device 100 instructs the machine learning model to identify the search target desired by the user, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query (step S101). The reception unit 132 of the information processing device 100 receives input information entered by the user using the search system (step S102). The generation unit 133 of the information processing device 100 inputs the input information into the machine learning model to cause the machine learning model to generate question information (step S103). The generation unit 133 also obtains response information (step S104). The generation unit 133 also inputs the response information into the machine learning model to cause the machine learning model to generate a search query (step S105). The generation unit 133 also inputs the search results into the machine learning model to cause the machine learning model to generate output information (step S106). Furthermore, the output control unit 134 of the information processing device 100 outputs output information (step S107).

[5. Variations]
The processing according to the above-described embodiment may be carried out in various other forms besides those described above.

In the embodiment described above, the case in which the generation unit 133 generates text was explained, but the information generated by the generation unit 133 is not limited to text. For example, the generation unit 133 may generate images in addition to text. For example, the generation unit 133 may have an image generation model M2, which generates images corresponding to text from text, generate an image. The generation unit 133 may generate an image by having the image generation model M2 generate an image. Specifically, the generation unit 133 may have a machine learning model M1 generate a first input text, which is text to be input to an image generation model M2 that generates images corresponding to text from text, input the first input text to the image generation model M2, have the image generation model M2 generate a first generated image, which is an image corresponding to the first input text, input the first generated image to the machine learning model M1, and have the machine learning model M1 generate question information. For example, the generation unit 133 may determine which tool should be used to generate question information, generate input information for the tool, input the generated input information to the tool, and input a prompt instructing the tool to generate question information based on the output information of the tool. For example, the generation unit 133 may input a prompt instructing it to determine which tool to use to generate question information from among the tools pre-specified as available tools by the instruction unit 131. For example, the generation unit 133 may input a prompt instructing it to determine which tool to use to generate question information from among the image generation model and image recognition model.

For example, the generation unit 133 may input a prompt P2 to the machine learning model M1 along with the input information, which includes the text shown in the upper part of Figure 7: "Is the information entered by the user sufficient to perform the search? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "No" in response to the input of prompt P2, the generation unit 133 may input a prompt P3 to the machine learning model M1, which includes the text shown in the middle part of Figure 7: "Is it necessary to use a tool to ask the user again? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "Yes" in response to the input of prompt P3, the generation unit 133 may input a prompt P5 to the machine learning model M1, which includes the text shown in the upper part of Figure 8: "What tool should be used? Answer with the tool name and input." Figure 8 is a diagram showing an example of prompts according to the embodiment. For example, the generation unit 133 may determine which tool to use to generate question information, generate input information for the tool, input the generated input information into the tool, and input prompt P5 to the machine learning model M1 as a prompt instructing the model to generate question information based on the tool's output information.

Furthermore, the generation unit 133 may obtain text that identifies the image generation model M2, which is the tool name output from the machine learning model M1 in response to the input of prompt P5. The generation unit 133 may also obtain the first input text output from the machine learning model M1 as input to the image generation model M2 in response to the input of prompt P5. If the generation unit 133 obtains text that identifies the image generation model M2, it may refer to the storage unit 120 based on the text that identifies the image generation model M2 to obtain the image generation model M2. The generation unit 133 may also input the first input text to the image generation model M2 and cause the image generation model M2 to generate a first generated image, which is the image corresponding to the first input text.

Furthermore, the generation unit 133 may input a prompt P6 to the machine learning model M1 along with the first generated image, containing the text shown in the lower part of Figure 8: "Is it necessary to use any further tools? Answer Yes or No." Also, if the machine learning model M1 outputs "No" in response to the input of prompt P6, the generation unit 133 may input a prompt P7 to the machine learning model M1 along with the first generated image, containing the text shown in the upper part of Figure 9: "What information should be asked of the user? Write the question to ask the user." The generation unit 133 may also obtain the question text output by the machine learning model M1 in response to the input of the first generated image and prompt P7 as question information. In this way, the generation unit 133 may generate question information based on the image generated by the image generation model M2. Furthermore, the output control unit 134 may output the question text along with the first generated image.

Figure 12 is a diagram illustrating an example of information processing related to a modified example. In Figure 12, user U1 has a search intent to find a shop where they can keep a rare dog they saw on television. However, since user U1 does not know the breed of the rare dog they saw on television, user U1 inputs the vague search query "shop where I can keep a rare dog" into their terminal device 10. Terminal device 10 transmits the vague search query "shop where I can keep a rare dog" to the information processing device 100. The receiving unit 132 of the information processing device 100 receives the vague search query "shop where I can keep a rare dog" as input information. The generation unit 133 of the information processing device 100 inputs the vague search query "shop where I can keep a rare dog" into a large-scale language model, which is a machine learning model M1, and causes the machine learning model M1 to generate the text "images of XX and YY" to be input into the image generation model M2. Here, XX and YY are the names of rare dog breeds. Furthermore, the generation unit 133 inputs the text "Images of XX and XX" to the image generation model M2, causing the image generation model M2 to generate images corresponding to XX and XX. The generation unit 133 also inputs the images corresponding to XX and XX generated by the image generation model M2 to the machine learning model M1, causing the machine learning model M1 to generate the question sentence "Is it the dog in either of the images?". The output control unit 134 outputs the question sentence "Is it the dog in either of the images?" along with the images corresponding to XX and XX generated by the image generation model M2 to the terminal device 10. The terminal device 10 also transmits the response sentence "The image on the right!" input by user U1 to the information processing device 100. Here, the image on the right corresponds to the image of XX. The generation unit 133 also inputs the response sentence "The image on the right!" to the machine learning model M1, causing the machine learning model M1 to generate the search query "Pet shops where I can buy XX". Furthermore, the generation unit 133 inputs the search query "Pet shops where I can buy XX" into the search system 200 and obtains a list of pet shops where dogs of the XX breed can be purchased as search results. The generation unit 133 then inputs the obtained list of shops into the machine learning model M1 and causes the machine learning model M1 to generate the output information "The shop ZZ in YY seems good!". The output control unit 134 of the information processing device 100 outputs the output information "The shop ZZ in YY seems good!" to the terminal device 10.

Furthermore, in the above-described modification, the generation unit 133 was described as generating question information based on an image generated by the image generation model M2. However, the generation unit 133 may also generate a search query based on an image generated by the image generation model M2. Specifically, the generation unit 133 causes a machine learning model to generate a second input text, which is text to be input to an image generation model that generates an image corresponding to text from text. The generation unit 133 inputs the second input text to the image generation model to generate a second generated image, which is an image corresponding to the second input text. The generation unit 133 inputs the second generated image to the machine learning model to generate a search query. For example, the generation unit 133 may determine which tool should be used to generate the search query, generate input information for the tool, input the generated input information into the tool, and input a prompt instructing the tool to generate a search query based on the tool's output information. For example, the generation unit 133 may input a prompt instructing the tool to determine which tool should be used to generate the search query from among the tools pre-specified as available tools by the instruction unit 131. For example, the generation unit 133 may receive a prompt instructing it to determine which tool should be used to generate the search query from among the image generation model and image recognition model.

For example, the generation unit 133 may input a prompt P2 to the machine learning model M1 along with the response information, which includes the text shown in the upper part of Figure 7: "Is the information entered by the user sufficient to perform the search? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "Yes" in response to the input of prompt P2, the generation unit 133 may input a prompt P4 to the machine learning model M1, which includes the text shown in the lower part of Figure 7: "Is it necessary to use a tool to perform the search? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "Yes" in response to the input of prompt P4, the generation unit 133 may input a prompt P5 to the machine learning model M1, which includes the text shown in Figure 8: "What tool should be used? Answer with the tool name and input." For example, the generation unit 133 may determine which tool to use to generate the search query, generate input information for the tool, input the generated input information into the tool, and input prompt P5 to the machine learning model M1 as a prompt instructing the model to generate the search query based on the tool's output information. For example, the generation unit 133 may input prompt P5 to the machine learning model M1 as a prompt instructing the model to generate a second input text, which is text to be input to the image generation model M2.

Furthermore, the generation unit 133 may acquire text that identifies the image generation model M2, which is the tool name output from the machine learning model M1 in response to the input of prompt P5. The generation unit 133 may also acquire a second input text output from the machine learning model M1 as input to the image generation model M2 in response to the input of prompt P5. If the generation unit 133 has acquired text that identifies the image generation model M2, it may refer to the storage unit 120 based on the text that identifies the image generation model M2 to acquire the image generation model M2. The generation unit 133 may also input the second input text to the image generation model M2 and have the image generation model M2 generate a second generated image, which is the image corresponding to the second input text.

Furthermore, the generation unit 133 may input a prompt P6 to the machine learning model M1 along with the second generated image, containing the text shown in the lower part of Figure 8, "Is it necessary to use any further tools? Answer Yes or No." Also, if the machine learning model M1 outputs "No" in response to prompt P6, the generation unit 133 may input a prompt P8 to the machine learning model M1 along with the second generated image, containing the text shown in the middle part of Figure 9, "What information are you searching for? Output text or file names." The generation unit 133 may also obtain the search text and file names (for example, the file names corresponding to the second generated image) output by the machine learning model M1 in response to the input of the second generated image and prompt P8 as a search query. In this way, the generation unit 133 may generate a search query based on the image generated by the image generation model M2. The generation unit 133 may also input the search text and file names to the search system 200.

Furthermore, in the embodiment described above, the generation unit 133 inputs input text entered by user U1 as input information to the machine learning model M1 to generate question information. However, the generation unit 133 may input information other than text as input information to the machine learning model M1. For example, the generation unit 133 may input an image as input information to the machine learning model M1. Specifically, the generation unit 133 inputs an input image, which is an image included in the input information, to an image recognition model M3 that generates text that describes the content of an image from an image. The image recognition model M3 generates text that corresponds to the input image, and the generation unit 133 inputs the text to the machine learning model M1 to generate question information. For example, the image recognition model M3 may be a visual language model (VLM). For example, the generation unit 133 may determine the tool to be used to generate question information, generate input information for the tool, input the generated input information into the tool, and input a prompt instructing the tool to generate question information based on the output information of the tool.

For example, the reception unit 132 may accept an input image entered by user U1 as input information. For example, the reception unit 132 may accept an input image and input text as input information. The reception unit 132 may accept an input image which is an image included in the input information. Also, when the reception unit 132 accepts input information, it may output the input information to the generation unit 133. Also, the generation unit 133 may input a prompt P2 to the machine learning model M1 along with the input image, which includes the text shown in the upper part of Figure 7, "Is the information entered by the user sufficient to perform a search? Answer Yes or No." Also, if the machine learning model M1 outputs "No" in response to the input of prompt P2, the generation unit 133 may input a prompt P3 to the machine learning model M1 which includes the text shown in the middle part of Figure 7, "Is it necessary to use a tool to ask the user again? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "Yes" in response to prompt P3, the generation unit 133 may input prompt P5 to the machine learning model M1, which includes the text "Which tool should be used? Please answer with the tool name and input." as shown in Figure 8. For example, the generation unit 133 may determine which tool should be used to generate the question information, generate the tool's input information, input the generated input information into the tool, and input prompt P5 to the machine learning model M1 as a prompt instructing it to generate the question information based on the tool's output information.

Furthermore, the generation unit 133 may obtain text that identifies the image recognition model M3, which is the tool name output from the machine learning model M1 in response to the input of prompt P5. The generation unit 133 may also obtain the filename of the input image output from the machine learning model M1 as input to the image recognition model M3 in response to the input of prompt P5. If the generation unit 133 obtains text that identifies the image recognition model M3, it may refer to the storage unit 120 based on the text that identifies the image recognition model M3 to obtain the image recognition model M3. If the generation unit 133 obtains the filename of the input image, it may refer to the storage unit 120 based on the filename to obtain the input image. The generation unit 133 may also input the input image to the image recognition model M3 and have the image recognition model M3 generate an input sentence corresponding to the input image.

Furthermore, the generation unit 133 may input a prompt P6 to the machine learning model M1 along with the input text, which includes the text shown in the lower part of Figure 8: "Is it necessary to use any further tools? Answer Yes or No." Also, if the machine learning model M1 outputs "No" in response to prompt P6, the generation unit 133 may input a prompt P7 to the machine learning model M1 along with the input text, which includes the text shown in the upper part of Figure 9: "What information should be asked of the user? Write the wording to ask the user." The generation unit 133 may also obtain the question text output by the machine learning model M1 in response to the input text and prompt P7 as question information. In this way, the generation unit 133 may generate question information based on the input text generated by the image recognition model M3.

Furthermore, while the above-described modification explained a case where the generation unit 133 generates question information based on the text generated by the image recognition model M3, the generation unit 133 may also generate a search query based on the text generated by the image recognition model M3. Specifically, the generation unit 133 inputs a response image, which is an image included in the response information, to the image recognition model M3, which generates text describing the content of an image from an image. The generation unit 133 then causes the image recognition model M3 to generate a response text, which is text corresponding to the response image, and inputs the response text into a machine learning model to cause the machine learning model to generate a search query. For example, the generation unit 133 may determine the tool to be used to generate the search query, generate input information for the tool, input the generated input information into the tool, and input a prompt instructing the tool to generate a search query based on the output information of the tool.

For example, the reception unit 132 may receive a response image input by user U1 as response information. For example, the reception unit 132 may receive a response image and a response text as response information. The reception unit 132 may receive a response image, which is an image included in the response information. Also, when the reception unit 132 receives response information, it may output the response information to the generation unit 133. The generation unit 133 may also input a prompt P2 to the machine learning model M1 along with the response image, which includes the text shown in the upper part of Figure 7, "Is the information entered by the user sufficient to perform the search? Answer Yes or No." Also, if the machine learning model M1 outputs "Yes" in response to the input of prompt P2, the generation unit 133 may input a prompt P4 to the machine learning model M1, which includes the text shown in the lower part of Figure 7, "Is it necessary to use a tool to perform the search? Answer Yes or No." Furthermore, if the machine learning model M1 outputs "Yes" in response to prompt P4, the generation unit 133 may input prompt P5 to the machine learning model M1, which includes the text shown in Figure 8: "Which tool should be used? Please provide the tool name and input." For example, the generation unit 133 may determine which tool should be used to generate the search query, generate input information for that tool, input the generated input information into the tool, and input prompt P5 to the machine learning model M1 as a prompt instructing the model to generate the search query based on the tool's output information.

Furthermore, the generation unit 133 may obtain text that identifies the image recognition model M3, which is the tool name output from the machine learning model M1 in response to the input of prompt P5. The generation unit 133 may also obtain the filename of the response image output from the machine learning model M1 as input to the image recognition model M3 in response to the input of prompt P5. If the generation unit 133 obtains text that identifies the image recognition model M3, it may refer to the storage unit 120 based on the text to obtain the image recognition model M3. If the generation unit 133 obtains the filename of the response image, it may refer to the storage unit 120 based on the filename to obtain the response image. The generation unit 133 may also input the response image to the image recognition model M3 and have the image recognition model M3 generate a response sentence corresponding to the response image.

Furthermore, the generation unit 133 may input a prompt P6 to the machine learning model M1 along with the response text, which includes the text shown in the lower part of Figure 8: "Is it necessary to use any further tools? Answer Yes or No." Also, if the machine learning model M1 outputs "No" in response to prompt P6, the generation unit 133 may input a prompt P8 to the machine learning model M1 along with the response text, which includes the text shown in the middle part of Figure 9: "What information are you searching for? Output text or file names." The generation unit 133 may also obtain the search text and file names (for example, file names corresponding to the response image) output by the machine learning model M1 in response to the response text and prompt P8 as a search query. In this way, the generation unit 133 may generate a search query based on the response text generated by the image recognition model M3.

Furthermore, the generation unit 133 may generate question information based on the recognition information generated by the speech recognition model M4, which generates recognition information indicating the recognition result of the speech data from the speech data, in the same manner as in the case of the image recognition model M3. For example, the generation unit 133 may generate question information based on the text generated by the speech recognition model M4, which generates recognition information that is text indicating the content of the speech data from the speech data. For example, the generation unit 133 may input input speech data, which is the speech data included in the input information, to the speech recognition model M4, which generates recognition information corresponding to the input speech data, and then input the input recognition information to the machine learning model M1, causing the machine learning model M1 to generate question information. For example, the generation unit 133 may input input speech data to the speech recognition model M4, which generates recognition information that is text indicating the content of the speech data from the speech data, causing the speech recognition model M4 to generate input speech text, which is text corresponding to the input speech data, and then input the input speech text to the machine learning model M1, causing the machine learning model M1 to generate question information.

Furthermore, the generation unit 133 may generate a search query based on the recognition information generated by the speech recognition model M4, which generates recognition information indicating the recognition result of the speech data from the speech data, in the same manner as in the case of the image recognition model M3. For example, the generation unit 133 may generate a search query based on the text generated by the speech recognition model M4, which generates recognition information that is text indicating the content of the speech data from the speech data. The generation unit 133 inputs response speech data, which is speech data included in the response information, to the speech recognition model M4, which generates recognition information corresponding to the response speech data, and inputs the response recognition information to the machine learning model M1, causing the machine learning model M1 to generate a search query. For example, the generation unit 133 may input response speech data to the speech recognition model M4, which generates recognition information that is text indicating the content of the speech data from the speech data, and causes the speech recognition model M4 to generate response speech text, which is text corresponding to the response speech data, and inputs the response speech text to the machine learning model M1, causing the machine learning model M1 to generate a search query.

Furthermore, in the embodiment described above, the generation unit 133 determined whether the similarity between a text-based search query and a text-based search result is above a predetermined threshold. However, the generation unit 133 may also determine whether the similarity between a search query and a search result with different modals is above a predetermined threshold. Here, the case where the modals of the search query and the search result are different includes cases where either the search query or the search result is text, and the other is a modal other than text (e.g., image or audio data). Specifically, the storage unit 120 may store information about the multimodal model M5, which is a machine learning model that determines the similarity between text and non-text modals. For example, if the machine learning model M1 outputs "Yes" in response to the input of prompt P9, the generation unit 133 may input prompt P5 to the machine learning model M1 containing the text "What tool should be used? Answer with the tool name and input," as shown in Figure 8.

Furthermore, the generation unit 133 may obtain text that identifies the multimodal model M5, which is the tool name output from the machine learning model M1 in response to the input of prompt P5. The generation unit 133 may also obtain the filenames of the search query and search results output from the machine learning model M1 as input to the multimodal model M5 in response to the input of prompt P5. If the generation unit 133 obtains text that identifies the multimodal model M5, it may refer to the storage unit 120 based on the text that identifies the multimodal model M5 to obtain the multimodal model M5. If the generation unit 133 obtains the filenames of the search query and search results, it may input the search query and search results into the multimodal model M5 and determine the similarity between the search query and the search results. The generation unit 133 may also determine whether the similarity between the search query and the search results is above a predetermined threshold.

Furthermore, while the above-described embodiment explained the case where the machine learning model M1 is a large-scale language model, the machine learning model M1 may be a language model trained to estimate and output the next token from the input image and token sequence. For example, the machine learning model M1 may be a Visual Language Model (VLM). For instance, when the generation unit 133 receives an input image as input information, it can input the input image into the machine learning model M1, which is a visual language model, without using the image recognition model M3, and generate question information corresponding to the input image. Similarly, when the generation unit 133 receives a response image as response information, it can input the response image into the machine learning model M1, without using the image recognition model M3, and generate a search query corresponding to the response information. For example, the machine learning model M1 may be CoCa (Contrastive Captioners are Image-Text Foundation Models), BLIP (Bootstrapping Language-Image Pre-training), BLIP2, GIT (Generative Image to Text Transformer), etc.

[6. Effects]
As described above, the information processing device 100 according to the embodiment includes a reception unit 132, a generation unit 133, and an output control unit 134. The reception unit 132 receives input information entered by a user using the search system. The generation unit 133 inputs the input information into a machine learning model, causes the machine learning model to generate question information indicating a question to identify the search target desired by the user, obtains response information indicating a response to the question information, generates a search query according to the response information, and generates output information corresponding to the search results that correspond to the search query. The output control unit 134 outputs the output information.

Thus, the information processing device 100 can generate question information in response to input information entered by a user of the search system, and output the generated question information, thereby enabling it to compensate for the ambiguity of the search target desired by the user. Furthermore, the information processing device 100 can acquire response information in response to the question information and generate a search query in response to the response information, thereby generating a search query that compensates for the ambiguity of the search target desired by the user. As a result, the information processing device 100 can provide the user with search results that compensate for the ambiguity of the search target desired by the user. Moreover, because the information processing device 100 can provide the user with search results that compensate for the ambiguity of the search target desired by the user, it can contribute to achieving Sustainable Development Goal (SDG) 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

Furthermore, the output control unit 134 outputs the question information. The reception unit 132 receives the response information entered by the user. The generation unit 133 inputs the response information into the machine learning model and causes the machine learning model to generate a search query.

This allows the information processing device 100 to generate a search query that complements the ambiguity of the search target desired by the user, based on the response information entered by the user.

Furthermore, the generation unit 133 inputs the search results into a machine learning model, causing the machine learning model to generate output information.

This allows the information processing device 100 to provide the user with search results that compensate for any ambiguity in the search target desired by the user.

Furthermore, the generation unit 133 causes a machine learning model to generate a first input text, which is text to be input to an image generation model that generates images corresponding to text from text. The first input text is then input to the image generation model to generate a first generated image, which is an image corresponding to the first input text. Finally, the first generated image is input to the machine learning model to generate question information.

This allows the information processing device 100 to, for example, generate an image corresponding to the input information entered by the user, and generate question information based on the generated image. Therefore, it can appropriately compensate for the ambiguity of the search target desired by the user.

Furthermore, the generation unit 133 causes a machine learning model to generate a second input text, which is text to be input to an image generation model that generates images corresponding to text from text. The second input text is then input to the image generation model to generate a second generated image, which corresponds to the second input text. Finally, the second generated image is input to the machine learning model to generate a search query.

This allows the information processing device 100 to, for example, generate an image corresponding to the response information input by the user, and generate a search query based on the generated image. Therefore, it can appropriately compensate for the ambiguity of the search target desired by the user.

Furthermore, the generation unit 133 receives the input image, which is an image included in the input information, as input to an image recognition model that generates text describing the content of an image from an image. The image recognition model generates the input text, which is text corresponding to the input image. The input text is then input to a machine learning model, which generates the question information.

This allows the information processing device 100 to, for example, recognize the content of an image input by the user and generate appropriate question information.

Furthermore, the generation unit 133 inputs a response image, which is an image included in the response information, to an image recognition model that generates text describing the content of an image from an image. The image recognition model generates a response text, which is text corresponding to the response image. The response text is then input to a machine learning model, which generates a search query.

This allows the information processing device 100 to, for example, recognize the content of an image input by the user and generate an appropriate search query.

Furthermore, machine learning models are either Large Language Models (LLMs) or Visual Language Models (VLMs).

This allows the information processing device 100 to generate appropriate question information and search queries using a large-scale language model or a visual language model.

Furthermore, the information processing device 100 also includes an instruction unit 131. The instruction unit 131 instructs the machine learning model to identify the search target desired by the user, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query.

This allows the information processing device 100 to assign the machine learning model the roles of identifying the search target desired by the user, generating a search query corresponding to the identified search target, and obtaining search results corresponding to the generated search query.

[7. Hardware Configuration]
Furthermore, the information processing device 100 according to the above-described embodiment is realized by a computer 1000 having a configuration such as that shown in Figure 13. Figure 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, controlling various components. The ROM 1300 stores boot programs executed by the CPU 1100 when the computer 1000 starts up, as well as programs dependent on the computer 1000's hardware.

The HDD 1400 stores programs executed by the CPU 1100, as well as data used by such programs. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input/output interface 1600. The CPU 1100 acquires data from input devices via the input/output interface 1600. Furthermore, the CPU 1100 outputs the generated data to output devices via the input/output interface 1600.

The media interface 1700 reads programs or data stored in the recording medium 1800 and provides them to the CPU 1100 via the RAM 1200. The CPU 1100 loads such programs from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded programs. The recording medium 1800 can be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical Disk), tape media, magnetic recording media, or semiconductor memory.

For example, when the computer 1000 functions as an information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, these programs may be obtained from other devices via a predetermined communication network.

Although several embodiments of this application have been described in detail based on the drawings, these are illustrative examples, and the present invention can be implemented in various modified and improved forms based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the invention.

[8. Other]
Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those illustrated; all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content remains consistent.

100 Information processing device 110 Communication unit 120 Storage unit 130 Control unit 131 Instruction unit 132 Reception unit 133 Generation unit 134 Output control unit

Claims (12)

A reception unit that receives input information entered by users of the search system,
A generation unit inputs the aforementioned input information into a machine learning model, causes the machine learning model to generate question information indicating a question for identifying the search target desired by the user, obtains response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to the search results corresponding to the search query.
An output control unit that outputs the aforementioned output information,
Equipped with,
The generating unit is
The system determines whether the similarity between the search query and the search results is above a predetermined threshold , and if it determines that the similarity is below the predetermined threshold , it generates a new search query different from the previous search query.
Information processing device. A reception unit that receives input information entered by users of the search system,
A generation unit inputs the aforementioned input information into a machine learning model, causes the machine learning model to generate question information indicating a question for identifying the search target desired by the user, obtains response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to the search results corresponding to the search query.
An output control unit that outputs the aforementioned output information,
Equipped with,
The generating unit is
A prompt instructing the machine learning model to determine which tool to use to generate the question information or search query from among the tools presented in advance as available tools, wherein the prompt instructs the machine learning model to generate input information for the tool, input the generated input information for the tool into the tool, and generate the question information or search query based on the output information of the tool.
Information processing device. The output control unit,
Output the aforementioned question information,
The aforementioned reception unit is
The response information entered by the user is received,
The generating unit is
The response information is input to the machine learning model to cause the machine learning model to generate the search query.
The information processing apparatus according to claim 1 or 2. The generating unit is
The search results are input to the machine learning model, and the machine learning model generates the output information.
The information processing apparatus according to claim 1 or 2. The generating unit is
The machine learning model generates a first input text, which is text to be input to an image generation model that generates an image corresponding to the text from the text; the machine learning model generates a first generated image, which is an image corresponding to the first input text, by inputting the first input text into the image generation model; and the machine learning model generates the question information by inputting the first generated image into the machine learning model.
The information processing apparatus according to claim 1 or 2. The generating unit is
The machine learning model generates a second input text, which is text to be input to an image generation model that generates an image corresponding to the text from the text; the machine learning model generates a second generated image, which is an image corresponding to the second input text, by inputting the second generated image into the machine learning model, and the machine learning model generates the search query.
The information processing apparatus according to claim 1 or 2. The generating unit is
An image recognition model that generates text describing the content of an image is input an input image, which is an image included in the input information, to cause the image recognition model to generate text corresponding to the input image, and the input text is input to the machine learning model to cause the machine learning model to generate the question information.
The information processing apparatus according to claim 1 or 2. The generating unit is
A response image, which is an image included in the response information, is input to an image recognition model that generates text describing the content of an image from an image; a response text, which is text corresponding to the response image, is generated by the image recognition model; the response text is input to the machine learning model; and the machine learning model generates the search query.
The information processing apparatus according to claim 1 or 2. The aforementioned machine learning model is either a Large Language Model (LLM) or a Visual Language Model (VLM).
The information processing apparatus according to claim 1 or 2. The machine learning model further includes an instruction unit that instructs the model to identify a search target desired by the user, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query.
The information processing apparatus according to claim 1 or 2. A reception procedure for receiving input information entered by users of the search system,
A generation procedure comprising: inputting the aforementioned input information into a machine learning model; causing the machine learning model to generate question information indicating a question for identifying the search target desired by the user; obtaining response information indicating a response to the question information; generating a search query corresponding to the response information; and generating output information corresponding to the search results that correspond to the search query;
An output control procedure for outputting the aforementioned output information,
Have the computer run it,
The aforementioned generation procedure is:
The system determines whether the similarity between the search query and the search results is above a predetermined threshold , and if it determines that the similarity is below the predetermined threshold , it generates a new search query different from the previous search query.
Information processing program. A reception procedure for receiving input information entered by users of the search system,
A generation procedure comprising: inputting the aforementioned input information into a machine learning model; causing the machine learning model to generate question information indicating a question for identifying the search target desired by the user; obtaining response information indicating a response to the question information; generating a search query corresponding to the response information; and generating output information corresponding to the search results that correspond to the search query;
An output control procedure for outputting the aforementioned output information,
Have the computer run it,
The aforementioned generation procedure is:
A prompt instructing the machine learning model to determine which tool to use to generate the question information or search query from among the tools presented in advance as available tools, wherein the prompt instructs the machine learning model to generate input information for the tool, input the generated input information for the tool into the tool, and generate the question information or search query based on the output information of the tool.
Information processing program.