JP7849496B2 - Selection-predictive neural network system - Google Patents

Description

This specification relates to generating responses to query inputs using neural networks.

A neural network is a machine learning model that utilizes one or more layers of nonlinear units to predict an output for a given input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or output layer. Each layer of the network generates an output from the received input according to the current values of its respective set of parameters.

This specification describes a system implemented as a computer program on one or more computers at one or more locations, which generates responses to queries about the environment.

In particular, the system receives a context input containing contextual information about the environment, and a query input containing queries about the environment.

The system then uses the contextual input to generate a response to the query, such as a natural language response.

The system generates a response by repeatedly alternating between selection steps and inference steps. As a result, the system can generate trace data that provides interpretable information in natural language, characterizing how the system arrived at its response.

The specific embodiments of the subject matter described herein may be implemented to achieve one or more of the following advantages:

One problem with using neural network-based methods to make control decisions, for example, to control an agent or a manufacturing plant, is that it is often difficult to infer why a particular decision was made; in other words, the neural network is a "black box" that is only entrusted with performing the task. Similarly, when diagnosing failures in mechanical systems, knowing the reasons behind the responses can be useful.

The system implementation described can address this problem by providing a reasoning trace that can be displayed to the user, or otherwise provided, or stored for later review. This can be particularly useful when control decisions or fault diagnosis relate to the safe operation of machine agents or manufacturing plants. The system implementation described can provide the reasoning trace as a sequence of logical steps presented as natural language statements, for example, leading from a query to a response in a causal chain, in a human-interpretable natural language.

More specifically, the technique described generates a response by repeatedly alternating between two steps: 1) selection, which involves choosing a subset of relevant information sufficient to perform a single step of inference; and 2) inference, which involves looking only at the limited information provided for the inference by the selection output and using that information to infer new intermediate pieces of evidence along the way to the final answer. This ensures that the intermediate inference outputs (and, optionally, the selection output) provide an interpretable inference trace that justifies the final answer. Furthermore, the inferences produced by the technique described are causal, because each step follows and depends on the previous step, and each inference is performed independently based solely on the limited information provided by the selection output, without direct access to the query input or previous inference steps. As a result, a high-quality response is obtained, simultaneously providing an interpretable natural language "trace" of the inferences performed by the system while generating the response.

Details of one or more embodiments of the subject matter of this specification are described in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims.

This is a diagram illustrating an exemplary choice inference system. This diagram shows a selection inference system used to control an agent that interacts with its environment. This figure shows a simplified example of the selection and inference steps. This is a flowchart illustrating an exemplary process for generating a response to a query. This is a flowchart illustrating an exemplary process for generating a selected output.

The same reference number and name in various drawings refer to the same element.

This specification describes a system implemented as a computer program on one or more computers in one or more locations that uses contextual information about the environment to generate responses to queries about that environment.

Figure 1A shows an exemplary selection-inference system 100. The selection-inference system 100 is an example of a system that can be implemented as a computer program on one or more computers in one or more locations, where the systems, components, and techniques described below are implemented.

The selection inference system 100 receives a query input 110 and a context input 120, and can use the context input 120 to generate a response 150 to the query input 110.

Query input 110 contains queries related to the environment. For example, each query may be a text fragment in natural language.

Context input 120 contains context information that provides context about the environment. More specifically, the context information in context input 120 contains one or more natural language statements, each representing a rule or fact about the environment.

In some implementations, the environment is a real-world environment, and system 100 facilitates reasoning in the real-world environment, for example, to control the real-world system within the environment in principle and logically. As will be described later, in some implementations, system 100 can also provide trace data containing causal explanations of why a particular response 150 was generated, which can promote trust in the system, especially in environments where safety is critical.

For example, in some implementations, the environment is a real-world environment, and response 150 is used to control a machine agent, such as a robot or an autonomous or semi-autonomous vehicle, operating within that real-world environment to perform a task.

Figure 1B shows an example of the operation of the selection prediction system 100 when the environment is a real-world environment navigated by the machine agent 102.

In the example in Figure 1B, agent 102 is shown as a vehicle. However, more generally, the mechanical agent 102 could be any suitable agent controlled by a control system as the agent navigates the real-world environment.

Agent 102 includes one or more sensors 104 that capture observations of the environment, for example, at specified time intervals, as Agent 102 navigates the environment.

For example, observational records may include, for instance, images, object position data, and sensor data, such as from image, distance, or position sensors, or from actuators, for capturing observational records as an agent interacts with its environment. For example, in the case of a robot, observational records may include, for instance, data characterizing the robot's current state, such as joint position, joint velocity, joint force, torque, or acceleration, such as gravity-compensated torque feedback, and one or more of the global or relative orientation of an item the robot grasps. In the case of a robot or other mechanical agent or vehicle, observational records may similarly include, one or more of position, linear or angular velocity, force, torque, or acceleration, and the global or relative orientation of one or more parts of the agent. Observational records may be defined in one, two, or three dimensions, and may be absolute and/or relative observational records. Observation records may include, for example, detected electronic signals such as motor current or temperature signals, and/or image or video data from, for example, a camera or LiDAR sensor, or data from, for example, the agent's sensors or data from sensors placed separately from the agent in the environment.

Agent 102 is also associated with a control system 106 that generates control signals to control Agent 102 using observation records generated by sensor 104. Specifically, the control system 106 first determines the appropriate actions that Agent 102 should take as part of performing a specified task, such as navigating to a specific location, identifying a specific object, moving a specific object to a given location, or somehow manipulating a specific object. Then, it generates control signals to cause Agent 102 to perform these actions, thereby guiding Agent 102 through a planned trajectory through the environment.

The control system 106 can be deployed mounted on agent 102, or deployed separately from agent 102, and can transmit control signals to agent 102 via a data communication network.

The control signal may be a control input for controlling the agent. For example, when the agent is a robot, the control signal may be, for example, torque for the robot's joints or a higher-level control command. As another example, when the agent is an autonomous or semi-autonomous land, air, or sea vehicle, the control signal may include actions to control the vehicle's navigation, e.g., steering, and movement, e.g., braking and/or acceleration. For example, the control signal may be, for example, torque to a control surface or other control element, e.g., the vehicle's steering control element, or a higher-level control command.

In other words, control signals may include, for example, position, velocity, or force/torque/acceleration data for one or more joints of a robot or a part of another mechanical agent.

In these examples, as with the control system 106, system 100 can be deployed mounted on agent 102, or it can be deployed separately from agent 102.

In these implementations, system 100 is used to provide an additional control layer on top of control system 106, and system 100 or another component can determine the query input 110 based on information received by the control system.

In other words, the query input 110 may be determined by receiving control signals from the agent control system 106, and then, for example, one or more natural language queries concerning agent 102 in the environment may be determined from the control signals.

In some implementations, the agent control system 106 is, for example, an autonomous or semi-autonomous control system that autonomously or semi-autonomously controls the navigation of agent 102.

In some other implementations, the agent control system 106 has an interface for receiving control commands, for example, from a human operator.

In both of these applications, the described system 100 may be used to provide an additional control layer, for example, for safety purposes. For instance, the described system 100 may be used to prevent control of the machine agent 102 in a way that is dangerous or violates one or more rules.

One or more rules concerning the control of the agent may be explicitly included, for example, as part of the context input, or implicitly included, for example, as natural language statements when training the neural network used by system 100, particularly when each of these includes a language model, as will be described later. As an example, such rules may include rules concerning the permitted movement of a vehicle, such as traffic rules. As another example, such rules may include rules concerning decisions that should be made to ensure the safe behavior of the machine agent, for example, to prevent harm to the machine agent or to a human.

Therefore, each query input 110 may relate to an action to be performed by the agent, such as an action being considered by the control system 106.

For example, the query input 110 can define the action to be performed by the machine agent 102 in the form of a question, such as "Should I turn left?" or "Is it safe for the agent to turn left?"

As another example, query input 110 could ask what action should be performed by the machine agent, for example, "Which way should the agent turn?"

In the case of a robot, the query input 110 often relates to a specific subtask among a set of subtasks that should be performed to carry out a task, for example, "What to do next?" or "Should I pick up object X?". The subtasks themselves may include a series of basic elemental actions, such as a robot's moving parts opening a gripper.

Generally, the query input 110 contains one or more natural language queries. More specifically, the query input 110 may contain a natural language description that defines the information that the system should provide in response. In other words, the query input 110 explicitly or implicitly determines what is required in the response.

The response 150 to the query input 110 is used to control the machine agent 102 in the real-world environment. More specifically, the response 150 is used to control the actions that the machine agent 102 should perform.

For example, response 150 may prevent an action that would otherwise be performed; that is, the response may determine whether the action defined by the query input will be performed.

As another example, if the query input implicitly or explicitly requests that an action be determined, response 150 may define the action to be performed.

In these implementations, obtaining the context input 120 may involve obtaining one or more observational records of the real-world environment, which may include one or more observational records of the agent, since the environment includes the agent. The observational records may be obtained from one or more sensors, but are not required to be the agent's sensor 104. As described above, the observational records may include still or moving images, which, as used herein, include LIDAR point clouds and/or other sensor data from one or more sensors detecting the state of the environment or the agent.

One or more observation records are processed, for example, by a first machine learning model to generate natural language representations of the one or more observation records, i.e., natural language text describing the observation records, which is included in the context input 120.

There are many different types of machine learning models that can be used to achieve this. For example, so-called visual-language models are generally configured to describe images or videos using natural language, for example, to perform image or video captioning tasks. More generally, such models can perform many different types of image processing tasks by formulating the task as a text generation problem, for example, to detect or classify objects in an image or video. Accordingly, other machine learning models can be trained to generate natural language text describing data from other types of sensors, for example, to represent physical location or force as natural language statements describing an agent or the environment or a part thereof. One or more natural language representations of observation records are used to provide one or more of the natural language statements in the context input 120.

Furthermore, the context input 120 may include one or more rules regarding the current location of agent 102 in the environment, which are not directly generated from the observation records produced by sensor 104.

For example, context input 120 could have rules of common knowledge, either manually designed or derived from a trained machine learning model, weights from a driving rules manual, or obtained from a knowledge graph or the internet. Examples of such rules include: "A car's electrical system will short-circuit if wet," "A car will end up there if it is driving in a certain direction," and "If a car's electrical system is broken, it is not safe to drive." Other examples of such rules include: "The speed limit at the current location is 30 mph," "Right turns are permitted after stopping at this red light," and "Turning across a double yellow line is prohibited."

Therefore, in these examples, system 100 can be used to evaluate the possible outcomes of actions being considered by control system 106, and the actions are then sent as control signals to agent 102.

In some other implementations, the environment is a real-world environment including a manufacturing plant, for example, a manufacturing plant for producing chemical, bio, or mechanical products, or food. To the extent used herein, “manufacturing” a product includes refining starting materials to create a product, or processing starting materials to remove contaminants, for example, to produce a cleaned or regenerated product. A manufacturing plant may comprise multiple manufacturing units, such as containers for chemical or biological substances, or machinery for processing solid or other materials. The manufacturing units are configured such that intermediate versions or components of a product are movable between manufacturing units during the manufacturing process, for example, via pipes or mechanical conveyances. In implementations, a system is used to control one or more of the manufacturing units, or to control the movement of intermediate versions or components of a product between manufacturing units.

Therefore, in these implementations, obtaining the context input 120 may then involve obtaining one or more observational records of the manufacturing unit or movement from one or more sensors. The sensors may include any type of sensor configured to detect the manufacturing unit or movement, such as mechanical movement or electrical conditions such as force, pressure, temperature, current, voltage, frequency, impedance, quantity, level, flow rate/transfer rate or flow/transfer path of one or more materials, physical or chemical conditions such as physical state, shape or configuration or chemical state such as pH, the configuration of the unit, such as the mechanical configuration of the unit, or valve configuration; an image or video sensor for capturing image or video observational records of the manufacturing unit or movement; or any other suitable type of sensor. In the implementation, one or more observational records are processed to generate natural language representations of the one or more observational records, for example, as described above. The natural language representations of the one or more observational records are used to provide one or more natural language statements of the contextual information.

A query input may relate to an action that controls the operation or movement of one or more manufacturing units. A response to a query input is used to control the operation or movement of one or more manufacturing units. For example, a response to a query input may be used to control energy or other resource usage, for example, to minimize it, or to control manufacturing to achieve a desired quality or characteristic of the product. For example, an action may include an action that controls an item of equipment in the plant, or an action that changes a setting affecting the movement of a manufacturing unit, product, intermediate, or their components, for example, by adjusting or turning on/off an item of equipment or a manufacturing process.

In some implementations, a manufacturing plant has a factory control system for controlling manufacturing units or for controlling their movement. Query inputs can be generated by receiving control signals from the factory control system and generating one or more natural language queries for query input from the control signals. As previously mentioned, the factory control system may be autonomous, semi-autonomous, or human-controlled.

As previously mentioned, the system may implement rules, for example, to control or limit the allocation of energy or other resources, or to ensure the target quality or characteristics of a product, or to restrain the operation of a factory, for example, a manufacturing unit, within safe limits.

In some implementations, the environment is a real-world environment, and the method is used to diagnose failures in a machine system operating in that real-world environment. Then, obtaining context input may involve obtaining one or more observational records of the machine system (here, including observational records of the machine system's operation) from one or more sensors, for example, as described above. These are processed to generate natural language representations of one or more observational records used to provide one or more natural language statements of contextual information, for example, as described above. In these implementations, the query input concerns the operation of the machine system, and the response to the query input is used to identify failures in the machine system. For example, the query input may include general queries such as "Is the system working correctly?" or "What's wrong with the system?" or specific queries such as "Is component X faulty?".

As another example, the environment may be an educational environment, and for instance, the system may be deployed as part of an educational software program that helps a user learn or practice one or more corresponding skills. In these examples, the context input 120 may contain natural language statements describing or referencing a scenario or scene in a real-world or imaginary environment, and the query input 110 may be a question about the scenario or scene that prompts logical reasoning. As described below, the trace data generated by the system 100 as part of generating the response 150 can be used to provide the user with insights into the logical reasoning required to generate the response 150. In other words, the trace data generated by the system 100 can be used to generate explanations of questions and the corresponding responses to help a user learn one or more skills.

As a concrete example, an educational software program can generate questions and accompanying contextual information (or receive this data from an external source). An example of such a question might be, "Why do astronauts need oxygen backpacks?" System 100 (given the context of basic facts) should generate a reasoning trace like the following to help the user understand the concept: "(1) Humans need oxygen to survive. There is no oxygen in space. Therefore, humans need an oxygen supply to survive in space. (2) Oxygen can be supplied to astronauts by oxygen backpacks, and humans need an oxygen supply to survive in space. Therefore, humans should use oxygen backpacks to help astronauts survive in space."

As another example, the environment may be an information retrieval environment, and for example, the system may be deployed as part of a search engine or other software that allows a user to retrieve information from a corpus of documents, for example, the Internet or another electronic document corpus. In these examples, the query input 110 may be any appropriate natural language query, and the context input 120 may contain relevant statements identified from the corpus of documents, i.e., by searching the corpus using conventional information retrieval techniques. The system 100 can then use the context input 120 to generate the response 150 to the query input 110, i.e., to generate the response 150 to the query input 110 so that the user can view a trace of the logical reasoning required to increase the user's confidence in the accuracy of the response 150 generated by the system 100.

Returning to the explanation in Figure 1A, to generate response 150, system 100 uses selection neural network 130 and inference neural network 140.

More specifically, system 100 generates a response by performing multiple update iterations.

In each update iteration, the system 100 performs a selection step by using the selection neural network 130 to select an appropriate subset of natural language statements from the context input 120.

System 100 then performs an inference step to generate a new natural language statement representing a new fact, i.e., a “predicted” fact, using a selected appropriate subset and the inference neural network 140. The fact is called “predicted” because it does not exist in the context input 120 and is determined by System 100 from the context input 120.

In each update iteration other than the last update iteration, system 100 updates the context input 120 using the new fact, that is, by adding a natural language statement representing the new fact to the context input 120.

In the final update iteration, system 100 uses the new facts to generate response 150. For example, system 100 may provide, as response 150, a natural language output derived from the new natural language statement generated in the final update iteration.

Therefore, system 100 iteratively adds new facts to the context input 120, and then uses the new facts generated in the last update iteration to generate the response 150 to the query input 110.

More specifically, the selection neural network 130 is a neural network configured to process a selection input, which includes the context input 120 (at the current update iteration) and the query input 110, to generate a selection output containing one or more natural language statements from the context input 120. Since the context input 120 is updated in each iteration, the selection output can identify different natural language statements in different update iterations.

The inference neural network 140 is a neural network configured to process an inference input, including a selected output, and generate an inference output containing a natural language statement representing new facts about the update iteration. Since the selected output can differ for different update iterations, the inference output can also differ across update iterations.

In some implementations, the inference input does not include either the context input 120 or the query input 110. That is, the inference neural network 140 only has access to a suitable subset of the context input 120 contained within the selected output, and does not have access to the remaining natural language statements in either the context input 120 or the query input 110.

Therefore, in each update iteration, the system 100 processes the selection input, which includes the context input 120 (at the time of the update iteration) and the query input 110, using the selection neural network 130 to generate a selection output for the update iteration that includes one or more natural language statements from the context input 120. These natural language statements may include statements that were included in the original context input obtained by the system 100, statements that were added to the context input in a previous update iteration, or both.

In some implementations, the selection neural network 130 is configured to generate a text sequence containing one or more natural language statements from the context input 120, i.e., to perform regression estimation. In some of these implementations, the system 100 may utilize constrained sampling to ensure that the tokens generated by the neural network 130 are either (i) a valid continuation of some natural language statement in the context input 120, (ii) the start of another natural language statement in the context input 120, or, optionally, (iii) one or more predetermined tokens that indicate, for example, a separator between natural language statements or the end of a regression-estimated text sequence.

In some other implementations, the neural network 130 can be configured to generate a text sequence containing placeholder references to natural language statements in the context input 120, for example, generating the sentence "I sent 1. I know I sent 4," where "I sent 1" is a reference to the first sentence in the context input 120, and "I sent 4" is a reference to the fourth sentence in the context input 120. The system can then substitute these into an actual sentence to generate a selected output.

Furthermore, in other implementations, each selection step includes multiple internal steps. In each internal step, the selection neural network 130 is used to add a new statement to the selection output for the selection step. This is explained in more detail below with reference to Figure 3.

System 100 then uses the inference neural network 140 to process the inference input, which includes the selected output for the update iteration, and generates an inference output that includes a natural language statement representing the new facts about the update iteration. As described above, if the update iteration is not the last update iteration, System 100 updates the context input 120 to include a natural language statement in the inference output for the update iteration.

Thanks to the way System 100 generates Response 150, System 100 can also provide trace data 160 as output, which provides an interpretable natural language summary of the causal inference performed by System 100 to generate Response 150. In particular, System 100 alternates between two steps: 1) selection, which involves choosing a subset of relevant information sufficient to make a single-step inference; and 2) inference, which looks only at the limited information provided to the inference by the selection output and uses that information to infer new intermediate pieces of evidence on the way to the final answer. Therefore, the system ensures that the intermediate inference outputs (and, optionally, the selection output) provide an interpretable inference trace to legitimize the final answer. Furthermore, the inferences produced by System 100 are causal, since each step follows and depends on the previous step, and each inference is made independently based only on the limited information provided by the selection output, without the query input or direct access to the previous inference step.

In some implementations, system 100 provides trace data 160 along with response 150, for example, to the user. In some other implementations, system 100 stores trace data 160 associated with data that identifies response 150.

For example, trace data 160 can later be accessed by a user requesting to see the "reasoning" performed by system 100, which resulted in a specific control signal being sent to the machine agent.

As a simplified example, if context input 120 indicates that there are pedestrians near the road the agent is traveling on, and query input 110 is "what action should be taken," for example, "continue driving" or "stop driving," then the trace data may include selection outputs such as "there is someone crossing the road" and "knowing that it is not safe to drive when someone is crossing the road," and the inference could be "therefore, it is not safe to drive."

As another simplified example, when context input 120 indicates that there is a lake near the vehicle, and query input 110 asks whether the vehicle should drive towards the lake, the trace data 160 may include intermediate inferences such as "Therefore, the car will end up at the lake" → "The lake means it will get wet" → "Getting wet means a short circuit" → "A short circuit means it is unsafe," and the response 150 could be "It should not go forward."

The selection neural network 130 can generally have any suitable architecture that allows the neural network to be used to map the selection input to an appropriate subset of the context input 120. Similarly, the inference neural network 140 can generally have any suitable architecture that allows the neural network to be used to map the inference input to a new natural language statement.

As concrete examples, both the selection neural network 130 and the inference neural network 140 can be language model neural networks, respectively. Generally, a language model neural network is a neural network that, given a text prompt containing a sequence of tokens in natural language, is trained to generate the next token in the sequence. This process can be repeated to expand the text prompt, one token at a time, to generate a natural language output, i.e., to autoregressively generate a natural language output token by token. At each "time step," the language model neural network processes the current sequence to generate a probability distribution of tokens in the vocabulary. The next token can then be selected using the probability distribution, for example, by sampling from the distribution using nuclear sampling or another sampling technique, or by selecting the token with the highest probability. The tokens in the vocabulary may include any combination of various tokens, such as words, subwords, letters, punctuation marks and other symbols, as well as numbers. Generally, language model neural networks are trained on a corpus of text consisting of tokens from a vocabulary (and optionally, other tokens that can be mapped to tokens not present in the given vocabulary) to predict the next token in a sequence of tokens from training data.

It is surprising, but well done, that large-scale language model neural networks can perform tasks they were not explicitly trained to perform. For example, these neural networks can perform translation tasks (assuming the training corpus contains words in different languages), arithmetic, and many other tasks. Language model neural networks can be made to perform specific tasks by providing a natural language description of the desired response as input or “prompt.” The prompt may be a few-shot prompt, where several queries, e.g., 1 to 10 examples and illustrative outputs, are given in the text prior to the actual query. Alternatively, or further, language model neural networks may be “fine-tuned” to perform specific tasks by obtaining pre-trained language model neural networks trained on the large-scale corpus described above, and then further training some of all of these language model neural networks with a relatively small number of examples specific to the type of task to be performed.

Therefore, a trained language model neural network can perform the types of control and diagnostic tasks described. If the system should adhere to rules when generating responses, these rules may be included in the contextual information, for example, in prompts and/or as statements in the training data corpus, or in the data used to fine-tune the language model neural network.

In other words, in some implementations, the selection neural network 130 and the inference neural network 140 are the same, pre-trained language model neural networks. In these implementations, each selection input contains a certain type of fusion shot prompt that causes the language model to generate a selection output, and each inference input contains a different type of fusion shot prompt that causes the language model to generate an inference output.

For example, a fusion shot prompt for the inference neural network 140 may include one or more exemplary pairs of inference inputs and inference outputs arranged according to a predetermined syntax (followed by a subset of context inputs identified by the corresponding selection outputs), and a fusion shot prompt for the selection neural network 130 may include one or more exemplary pairs of selection inputs and selection outputs arranged according to a predetermined syntax (followed by context inputs and query inputs arranged according to the predetermined syntax).

For example, the Fusion Shot prompt within each selection input may take the following form:
#nShotPrompt
#Example 1.
<Context 1>
<Query 1>
#Example Selection
<fact>. <fact>[and <fact>]* know. Therefore,
...
#Problems that need to be solved.
<context>
<Query>

In this example, statements following '#' are not included in the prompt (or are optional) and are only included in the example for illustrative purposes; <context> represents natural language text in the context input; <query> represents the query input; "..." indicates that one or more additional examples in the same format follow the first example; each <fact> is a natural language statement copied from the corresponding context; and [and <fact>]* means that the system can select multiple facts for each inference step, where the total number of facts in a given inference step is a hyperparameter.

As another example, the fusion shot prompt within each inference input may take the following form.
#nshot guessing prompt
#Example 1.
<Fact>. We know that <Fact> [and <Fact>]*. Therefore, <New Fact>.
...
#Problems that need to be solved.
<Output of the selection step>. Therefore,

In this example, the <new fact> in each example within the prompt is the inferred fact generated ("inferred") from the <fact> in the example, and the <selection step output> is one or more facts in the selection output for the selection step, formatted in the same way as the example in the Fusion Shot prompt.

The language model neural network may be a large-scale language model neural network, for example, one with more than 1 billion, 10 billion, or 100 billion trained parameters. The language model neural network may be trained with more than 10 billion, 100 billion, or 1 trillion words or tokens representing words or other text tokens, for example, subwords (also known as "wordpieces").

In some other implementations, both the selection neural network 130 and the inference neural network 140 have the same architecture and are language model neural networks pre-trained on the same large corpus, but the selection neural network 130 is fine-tuned on a first dataset of exemplary selection input and selection output pairs, and the inference neural network 140 is fine-tuned on a second dataset of exemplary inference input and selection input pairs, or both. For example, by fine-tuning the selection neural network 130, it may be possible to more effectively generate outputs that include placeholder references from context inputs to facts, as described above. In some of these implementations, each selection input and each inference output each include their respective fusion shot prompts, while in some other implementations, fusion shot prompts are not included.

In some implementations, the language model neural network is an autoregressive transformer neural network, characterized by having a series of self-attention neural network layers. Each self-attention neural network layer has an attention layer input for each element of the input, and is configured to apply an attention mechanism to the attention layer input to generate an attention layer output for each element of the input; many different attention mechanisms can be used. In some implementations, the language model neural network may be a mixed-expert model.

Figure 1C shows a simplified example of the selection and inference steps performed when system 100 is used to control a vehicle, such as an autonomous or semi-autonomous vehicle.

In the example in Figure 1C, the query input 176 is "Should the vehicle turn left?", and the context input 174 (at the time of the exemplary selection step shown in Figure 1C) is "There is a lake on the left, the lake has a lot of water, and the water could damage the vehicle." To perform the selection step, system 100 generates a selection input containing a k-shot prompt 172 (a portion of the exemplary context for one of the examples in the k-shot prompt 172, an exemplary query input for the example, and an exemplary selection output for the example are shown in Figure 1C), context input 174, and query input 176. System 100 processes the selection input using the selection neural network 130 to generate a selection output 180 stating "There is a lot of water in the lake, and therefore the water could damage the vehicle."

System 100 then generates an inference input, including a k-shot prompt 182 (one example of a k-shot prompt 182 is shown in Figure 1C), and a selection output 180. System 100 processes the inference input using the inference neural network 140 to generate a new inferred fact, namely, an inference output 182 stating, "Entering the lake may damage the vehicle." System 100 then adds the new inferred fact to the context input 174, 188, for use in the next selection step.

Figure 2 is a flowchart of an exemplary process 200 for generating a response to a query input. For convenience, the process 200 is described as being carried out by a system consisting of one or more computers in one or more locations. For example, a selection inference system appropriately programmed according to this specification, such as the selection inference system 100 in Figure 1A, can carry out the process 200.

The system obtains context input containing context information (step 202). The context information includes one or more natural language statements, each representing a fact or rule about the environment.

The system receives a query input containing one or more queries about the environment (step 204).

The system then generates a response to the query input, such as a natural language output, by performing update iterations until the termination criteria are satisfied. For example, the termination criteria may be satisfied after a threshold number of iterations have been performed. Alternatively, the termination criteria may be satisfied if the same selected output is produced in a threshold number of consecutive update iterations. Another example is if the same inferred output is produced in a threshold number of consecutive update iterations. Yet another example is that the system may include a stopping input containing the query input, the most recent inferred output, and optionally, one or more of (i) pre-generated selected outputs, inferred outputs, or both, or (ii) context inputs, and a stopping neural network trained to process the stopping input and generate an instruction on whether the most recent inferred output is a valid response to the query input. Another example is that the system may determine whether the most recent inferred output is a valid response to the query input by applying, for example, one or more rules based on string matching between the inferred output and the query input.

In each update iteration, the system processes the context input and the selection input, which includes the query input, using a selection neural network to generate a selection output for the update iteration that contains one or more natural language statements from the context input (step 206). In other words, the selection output generally contains an appropriate subset of natural language statements from the context input.

The system then uses an inference neural network to process the inference input, which includes selected outputs for the update iteration, and generates an inference output containing natural language statements representing new facts about the update iteration (step 208).

In each update iteration other than the last one, i.e., if the system determines after performing the update iteration that the termination criteria are not satisfied, the system updates the context input to include the natural language statement in the inferred output for the update iteration (step 210).

In the final update iteration, i.e., when the system determines that the termination criteria are satisfied after performing the update iteration, the system generates a response using the natural language statements in the inferred output for the final update iteration (step 212). The system may also generate an inference trace containing the inferred outputs for each update iteration other than the final one, and optionally, the natural language statements in the selected outputs for the update iterations (and may provide this as part of the response or store it for future use).

Figure 3 is a flowchart illustrating an exemplary process 300 for generating a selection output in a given update iteration. For convenience, the process 300 is described as being performed by a system consisting of one or more computers in one or more locations. For example, a selection prediction system appropriately programmed according to this specification, such as the selection prediction system 100 in Figure 1A, can perform the process 300.

In the example in Figure 3, the system generates a selection output by selecting each natural language statement from the context input in each of one or more sequences of selection iterations. For example, the system may perform a predetermined number of selection iterations to add a predetermined number of natural language statements to the selection output. Alternatively, the system may perform selection iterations until some other criterion is satisfied.

In each selection iteration, the system generates an input for the selection iteration, which includes a context input, a query input, and any natural language statement selected in any previous selection iteration preceding the selection iteration in the sequence (step 302). For example, when the selection input to the selection neural network includes a fusion shot prompt as described above, the system can modify the fusion shot prompt (which already includes the context input and query input) to include, at the corresponding location in a given syntax, a natural language statement selected in any previous selection iteration preceding the selection iteration in the sequence.

For each of the natural language statements in the contextual information, the system then uses a selection neural network to process the input for selection iterations and determine the likelihood assigned to the natural language statement by the selection neural network (step 304).

For example, the system can select natural language statements from the context input that have not yet been selected in previous selection iterations, as a set of natural language statements.

As described above, the selection neural network may be a language model neural network that predicts the probability distribution for a given text token, given the current text token in a sequence. In this case, the system can determine the likelihood assigned to a natural language statement by combining, for example, multiplying, the probabilities assigned to each text token in the natural language statement by the selection neural network. The probability assigned to a given token is the probability for that token in the probability distribution generated by processing a combined sequence, for example, an input for selection iteration and any tokens preceding the given token in the natural language statement, using a selection neural network conditioned on the input to the selection neural network.

The system then selects the natural language statement with the highest likelihood from the set of natural language statements (step 306).

This specification uses the term “configured” in relation to system and computer program components. A system consisting of one or more computers being configured to perform a particular operation or action means that the system has installed software, firmware, hardware, or a combination thereof that causes the system to perform the operation or action when it is running. A computer program being configured to perform a particular operation or action means that the program contains instructions that, when executed by a data processing device, cause the device to perform the operation or action.

The subject matter and functional operating embodiments described herein can be implemented in digital electronic circuit mechanisms, or in tangibly implemented computer software or firmware, or in computer hardware including the structures disclosed herein and their structural equivalents, or in combination of one or more of these. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., as one or more modules of computer program instructions encoded on a tangible non-temporary storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage board, a random or serial access memory device, or a combination of one or more of these. Alternatively or additionally, the program instructions may be encoded on artificially generated propagating signals, such as mechanically generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, machines, and equipment for processing data, including, for example, a programmable processor, a single computer, or multiple processors or computers. A device may be a special-purpose logic circuit mechanism, such as an FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit), or may further include these. Optionally, in addition to hardware, a device may include code that creates an execution environment for computer programs, such as processor firmware, a protocol stack, a database management system, an operating system, or code constituting one or more of these, or a combination thereof.

Computer programs are sometimes called, or referred to as, programs, software, software applications, apps, modules, software modules, scripts, or code, and may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages. They can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A program may, but does not, correspond to a file in a file system. A program may be stored in a file portion that holds one or more scripts stored in other programs or data, such as a markup language document; in a single file dedicated to the program in question; or in multiple collaborative files, such as files that store one or more modules, subprograms, or parts of code. Computer programs may be deployed to run on one computer or on multiple computers located in one location, or they may be distributed across multiple locations and interconnected by data communication networks.

In this specification, the term “database” is used broadly to refer to any collection of data, but data does not need to be structured in any particular way, or not structured at all, and can be stored on one or more storage devices in different locations. Therefore, for example, an index database may contain multiple collections of data, each of which can be organized and accessed in different ways.

Similarly, in this specification, the term “engine” is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components and installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine, while in other cases, multiple engines may be installed and running on the same computer or on multiple computers.

The processes and logic flows described herein may be implemented by one or more programmable computers executing one or more computer programs to perform their functions by acting on input data and generating outputs. The processes and logic flows may be implemented by special-purpose logic circuit mechanisms, such as FPGAs or ASICs, or by a combination of special-purpose logic circuit mechanisms and one or more programmed computers.

A computer suitable for running computer programs may be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Generally, the central processing unit will receive instructions and data from read-only memory, random-access memory, or both. Essential elements of a computer are the central processing unit for executing or running instructions, and one or more memory devices for storing instructions and data. The central processing unit and memory can be complemented by or incorporated into special-purpose logic circuit elements. Generally, a computer may include one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks, or may be operably coupled to mass storage devices to receive data from or transfer data, or both. However, a computer is not required to have such devices. Furthermore, a computer can be incorporated into other devices, such as, to name just a few, mobile phones, personal digital assistants (PDAs), mobile audio or video players, game consoles, Global Positioning System (GPS) receivers, or portable storage devices, such as Universal Serial Bus (USB) flash drives.

Computer-readable media suitable for storing computer program instructions and data include, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks, and all forms of non-volatile memory, media, and memory devices.

To provide user interaction, embodiments of the subject matter described herein can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user, as well as a keyboard and a pointing device such as a mouse or trackball to which the user can provide input to the computer. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback, and user input can be received in any form, including acoustic, voice, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from a device used by the user, for example, by sending a web page to a web browser on the user's device in response to a request received from a web browser. The computer can also interact with the user by sending text messages or other forms of messages to a personal device, such as a smartphone running a messaging application, and receiving response messages from the user in return.

Data processing devices for implementing machine learning models may include, for example, special-purpose hardware accelerator units for processing machine learning training or production, i.e., the common and numerical computation parts of the workload, such as inference.

Machine learning models can be implemented and deployed using machine learning frameworks, such as the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework.

Embodiments of the subject matter described herein can be implemented in a computing system that includes, for example, a data server as a backend component, or a middleware component, for example, an application server, or a client computer having a frontend component, for example, a graphical user interface or web browser or application for a user to interact with the implementation of the subject matter described herein, or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected by digital data communications of any form or medium, such as communication networks. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

A computing system may include a client and a server. The client and server are generally separate from each other and typically interact through a communication network. The client-server relationship arises from computer programs running on each computer and having a client-server relationship with each other. In some embodiments, the server transmits data, such as an HTML page, to a user device, for example, to display data to a user interacting with a device acting as a client, and to receive user input from the user. Data generated on the user device, such as the results of user interaction, may be received from the device to the server.

This specification includes many specific implementation details, but these should not be construed as limiting within the scope of any invention or within the scope of what may be claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Furthermore, some features described herein in the context of separate embodiments can be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable partial combination in multiple embodiments. Moreover, features are described above as working in several combinations, and may even be initially claimed in this way, but one or more features from a claimed combination may, in some cases, be removed from that combination, and the claimed combination may cover partial combinations or variations of partial combinations.

Similarly, while operations are shown in the drawings and described in the claims in a specific order, this should not be understood as requiring such operations to be performed in the specific illustrated order or sequentially, or requiring all described operations to be performed to achieve the desired result. In some situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged in multiple software products.

This document describes specific embodiments of the subject matter. Other embodiments are within the scope of the following claims. For example, the actions described in the claims may be performed in a different order and still achieve the desired results. As an example, the process shown in the accompanying drawings does not necessarily require a specific illustrated order or sequence to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

100 Selection Prediction System, System
102 Machine Agent, Agent
104 Sensors
106 Control systems, agent control systems
130 Selected Neural Networks, Neural Networks
140 Inferential Neural Networks

Claims (22)

A method carried out by one or more computers,
A step of obtaining a context input that includes context information, wherein the context information includes one or more natural language statements, each representing a fact or rule about the environment;
The steps include receiving a query input that includes a query relating to the aforementioned environment,
The process includes the step of generating a response to the query input by performing update iterations until the termination criteria are satisfied, wherein in each update iteration,
Using a selection neural network, process the selection input, which includes the context input and the query input, to generate a selection output for the update iteration, which includes one or more of the natural language statements from the context input.
Using an inference neural network, process the inference input, which includes the selected output for the update iteration, to generate an inference output that includes a natural language statement representing a new fact for the update iteration,
In each update iteration except the last update iteration,
Updating the context input to include the natural language statement in the inference output for the update iteration,
Methods that include... The method according to claim 1, further comprising the step of providing, as a response, (i) a natural language output derived from the natural language statement in the inference output for the last update iteration, and (ii) an inference trace containing the natural language statement in the inference output for each update iteration other than the last update iteration. The method according to claim 2, further comprising the inference trace for the selected output for the update iteration. The method according to claim 1 , wherein the inference input does not include the context input or the query input. The environment is a real-world environment, and the method is used to control a machine agent acting in the real-world environment to perform a task.
Obtaining the context input includes obtaining one or more observational records of the real-world environment from one or more sensors, and processing the one or more observational records to generate a natural language representation of the one or more observational records.
The aforementioned query input relates to the action to be performed by the agent,
The aforementioned method,
A step of using the natural language representation of one or more observation records to provide one or more of the natural language statements of the context information,
The further step includes using the response to the query input to control the machine agent in the real-world environment,
The method according to claim 1 . The machine agent has an agent control system for controlling the actions of the machine agent, the query input includes one or more natural language queries, and the step of receiving the query input is:
Receiving control signals from the aforementioned agent control system,
This includes generating one or more natural language queries from the control signals,
The method according to claim 5. The method according to claim 5, wherein the machine agent includes an autonomous or semi-autonomous vehicle that navigates within the real-world environment, and the action includes an action for controlling the movement of the vehicle within the real -world environment. The aforementioned environment is a real-world environment,
The context input is derived from observational records characterizing the current state of the real-world environment, generated from measurement results from at least one or more sensors configured to detect the real-world environment.
The aforementioned query input includes data that characterizes the agent's planned navigation.
The response to the query input characterizes the action to be performed by the agent in response to the observation record,
In particular, the agent is a robot or an autonomous vehicle.
The method according to claim 1 . The method according to claim 8, further comprising the step of controlling the navigation of the agent based on the response to the query input. The environment is a manufacturing plant for producing a product, the manufacturing plant comprises a plurality of manufacturing units, the plurality of manufacturing units are configured such that intermediate versions or components of the product are movable between the manufacturing units during the production of the product, and the method is used to control one or more of the manufacturing units, or to control the movement of the intermediate versions or components of the product between the manufacturing units.
Acquiring the context input includes acquiring one or more observation records of the manufacturing unit or the movement from one or more sensors, and processing the one or more observation records to generate a natural language representation of the one or more observation records.
The query input relates to an action that controls the movement of one or more of the manufacturing units, or the movement of the units.
The aforementioned method,
The steps include using the natural language representation of one or more observation records to provide one or more of the natural language statements of the contextual information,
The further step includes using the response to the query input to control the operation of one or more of the manufacturing units or to control the movement of the manufacturing units,
The method according to claim 1 . The manufacturing plant has a factory control system for controlling the manufacturing units or for controlling the movement, the query input includes one or more natural language queries, and the step of receiving the query input is:
Receiving control signals from the aforementioned factory control system,
This includes generating one or more natural language queries from the control signals,
The method according to claim 10. The environment is a real-world environment, and the method is used to diagnose a malfunction in a mechanical system operating in the real-world environment.
The step of obtaining the context input includes obtaining one or more observation records of the machine system from one or more sensors, and processing the one or more observation records to generate a natural language representation of the one or more observation records.
The query input relates to the operation of the machine system,
The aforementioned method,
The steps include using the natural language representation of one or more observation records to provide one or more of the natural language statements of the contextual information,
The further step includes using the response to the query input to identify a fault in the machine system,
The method according to claim 1 . The method according to claim 1 , wherein the query input includes a natural language description that defines the information that the response should provide. The process further includes the step of generating at least one of the natural language statements in the context input by processing the observation record using a first machine learning model configured to process the observation record and generate natural language text describing the observation record,
especially,
The step of generating at least one of the natural language statements in the context input by processing the observation record using a second machine learning model configured to process the observation record using (i) the observation record, (ii) the natural language text describing the observation record, or (iii) both, to generate natural language text characterizing one or more rules for determining the new fact related to the observation record, further comprising the step of generating at least one of the natural language statements in the context input,
The method according to claim 1 . The method according to claim 1 , wherein the termination criterion is satisfied when a threshold update iteration is performed. The step of using a selection neural network to process a selection input, including the context input and the query input, to generate a selection output for the update iteration, which includes one or more of the natural language statements in the context information,
Each of one or more sequences of selective iterations includes selecting a natural language statement from the context information, wherein in each selective iteration,
To generate an input for the selection iteration, wherein the input includes the context input, the query input, and any natural language statement selected in any previous selection iteration preceding the selection iteration in the sequence,
This includes selecting each of the natural language statements by processing the input for the selection iteration using the selection neural network,
The method according to claim 1 . Selecting each of the natural language statements of the context information for the selection iteration is,
For each of the sets of natural language statements in the aforementioned context information,
The process involves using the selection neural network to process the input for the selection iteration and determining the likelihood assigned to the natural language statement by the selection neural network.
This includes selecting the natural language statement with the highest likelihood from the set of natural language statements.
The method according to claim 16. The aforementioned selection neural network and the aforementioned inference neural network are the same neural network.
The aforementioned selection input includes a first fusion shot prompt,
The aforementioned guess input includes a second different fusion shot prompt,
The method according to claim 16 . The method according to claim 18, wherein generating the input for the selection iteration includes modifying the first fusion shot prompt to include the natural language statement selected in any previous selection iteration preceding the selection iteration in the sequence. The method according to claim 1 , wherein the selection neural network and the inference neural network are the same pre-trained language model neural network, and in particular the language model neural network has more than one billion trained parameters. One or more computer-readable storage media that, when executed by one or more computers, stores instructions causing the one or more computers to perform the operation described in any one of claims 1 to 20. A system comprising one or more computers and one or more storage devices for storing instructions, wherein, when an instruction is executed by the one or more computers, the system causes the one or more computers to perform the operation described in any one of claims 1 to 20.