JP7659540B2 - SYSTEM AND METHOD FOR MANAGING VOICE QUERIES USING PRONUNCIATION INFORMATION - Patent application - Google Patents

Description

The present disclosure relates to a system for managing voice queries, and more specifically, to a system for managing voice queries based on pronunciation information.

In a conversation system, when a user utters a voice query to the system, the speech is converted to text using an automatic speech recognition (ASR) module. This text then forms the input to the conversation system, which determines the response to the text. For example, when a user says "show me Tom Cruise movies," the ASR module converts the user's speech to text and issues it to the conversation system. The conversation system only acts on the text it receives from the ASR module. Sometimes, in this process, the conversation system loses the pronunciation details of the words or sounds contained in the user's query. The pronunciation details can provide information that can be useful for search, especially when the same word has two or more pronunciations and the pronunciations correspond to different meanings.

This disclosure describes systems and methods for conducting searches and predicting a user's intended search query based on multiple context inputs when a user speaks a query word. The search may be based on multiple context inputs including, for example, user search history, user likes and dislikes, general trends, pronunciation details of the query words, and any other suitable information. An application receives the voice query and generates a text query representing the voice query. The application uses pronunciation information that may be included in metadata associated with the text query included in the text query or that may be included in metadata of entities in a database to more accurately retrieve search results. In some embodiments, the application generates metadata based on text-to-speech and speech-to-text conversions to improve reachability of entities from the search query.
The present invention provides, for example, the following:
(Item 1)
1. A method for responding to a voice query, the method comprising:
Receiving a voice query at an audio interface;
extracting, using control circuitry, one or more keywords from the voice query;
generating a text query based on the one or more keywords using the control circuitry; and
identifying an entity, the identifying the entity being based on the text query and metadata about the entity, the metadata comprising one or more alternative text representations of the entity, the one or more alternative text representations being based on a pronunciation of an identifier associated with the entity;
Retrieving a content item associated with the entity; and
A method comprising:
(Item 2)
13. The method of claim 1, wherein the one or more alternative textual representations comprise a phonemic representation of the entity.
(Item 3)
3. The method of any of claims 1 and 2, wherein the one or more alternative textual representations comprise alternative spellings of the entity that are based on pronunciation.
(Item 4)
4. The method of claim 1, wherein the one or more alternative text representations of the entity comprise text strings generated based on a previous speech-to-text transformation.
(Item 5)
The one or more alternative text representations comprise a plurality of alternative text representations, each alternative text representation in the plurality of alternative text representations:
Converting the first text representation into an audio file;
converting the audio file into a second text representation;
Generated by
5. The method of any of items 1-4, wherein the second text representation is not identical to the first text representation.
(Item 6)
6. The method of any of items 1-5, wherein identifying the entity is further based on user profile information.
(Item 7)
7. The method of any of items 1-6, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 8)
Identifying the entity includes:
identifying the plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities, a respective score based on comparing the respective one or more alternative text representations to the text query;
selecting said entity by determining a maximum score;
8. The method according to any one of items 1 to 7, comprising:
(Item 9)
9. The method of claim 1, further comprising generating a plurality of text queries, the plurality of text queries comprising the text query, each text query of the plurality of text queries being generated based on a respective setting of a speech-to-text module of the control circuitry.
(Item 10)
identifying a respective entity based on each of the plurality of text queries;
determining a respective score for each of the entities based on a comparison of the respective text queries to metadata associated with the respective entities;
identifying said entity by selecting a maximum score of said respective scores;
10. The method of claim 9, further comprising:
(Item 11)
1. A system for responding to a voice query, the system comprising:
Memory,
Means for implementing the steps of the method according to any one of items 1 to 10;
The system comprises:
(Item 12)
A non-transitory computer readable medium having instructions encoded thereon that, when executed by control circuitry, enable the control circuitry to perform the steps of the method of any of items 1-10.
(Item 13)
11. A system for responding to voice queries, said system comprising means for implementing the steps of the method according to any of items 1-10.

These and other objects and advantages of the present disclosure will become apparent upon consideration of the following detailed description taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout.

FIG. 1 shows a block diagram of an illustrative system for generating text queries in accordance with some embodiments of this disclosure.

FIG. 2 shows a block diagram of an illustrative system for retrieving content in response to a voice query, according to some embodiments of this disclosure.

FIG. 3 shows a block diagram of an illustrative system for generating phonetic information according to some embodiments of this disclosure.

FIG. 4 is a block diagram of an illustrative user equipment in accordance with some embodiments of the disclosure.

FIG. 5 shows a block diagram of an illustrative system for responding to voice queries in accordance with some embodiments of this disclosure.

FIG. 6 shows a flowchart of an illustrative process for responding to a voice query based on pronunciation information, according to some embodiments of the present disclosure.

FIG. 7 shows a flowchart of an illustrative process for responding to a voice query based on alternative expressions according to some embodiments of this disclosure.

FIG. 8 shows a flowchart of an illustrative process for generating metadata about an entity based on pronunciation, according to some embodiments of the present disclosure.

FIG. 9 shows a flowchart of an illustrative process for retrieving content associated with an entity of a voice query, according to some embodiments of the present disclosure.

In some embodiments, the present disclosure is directed to a system configured to receive a voice query from a user, analyze the voice query, and generate a text query (e.g., a transcription) for searching content or information. The system responds to the voice query based in part on the pronunciation of one or more keywords. For example, in the English language, there are multiple words that are spelled the same but have different pronunciations. This may be especially true for people's names. Some examples include:
To illustrate, a user may say into the system's audio interface, "Show me Louis'interview." The system may generate illustrative text queries such as:
Option 1) "Show me Louis Freeh's interview with Fraud Magazine."
Option 2) "Show me the interview with Lewis Black that aired on CBS."
The resulting text query depends on how the user spoke the word "Louis." If the user pronounced it "LOO-ee," the system would select option 1 or apply a heavier weight to option 1. If the user pronounced it "LOO-his," the system would select option 2 or apply a heavier weight to option 2. If the pronunciation was not taken into account, it is likely that the system would not be able to respond to the voice query accurately.

In some circumstances, a voice query that includes a partial name of a person may cause ambiguity in correctly detecting that person (e.g., referred to as a "non-deterministic person search query"). For example, if a user utters "show me movies starring Tom" or "show me interviews with Louis", the system would need to determine whether the user is asking about Tom or Louis/Louie/Lewis. In addition to the pronunciation information, the system may analyze one or more contextual inputs, such as, for example, user search history (e.g., previous queries and search results), user likes/dislikes/preferences (e.g., from user profile information), general trends (e.g., of multiple users), popularity (e.g., among multiple users), any other suitable information, or any combination thereof. The system retains the pronunciation information in a suitable form (e.g., in the text query itself or in metadata associated with the text query) so that it is not lost after the automatic speech recognition (ASR) process.

In some embodiments, for pronunciation information to be used by the system, the information field in which the system searches must contain the pronunciation information for comparison to the query. For example, the information field may include information about the entity that includes the pronunciation metadata. The system may perform a phoneme transcription process that takes the user's voice query as input and converts it to text that, when read back, sounds phonetically correct. The system may be configured to use the output of the phoneme transcription process and the pronunciation metadata to determine search results. In an illustrative example, the pronunciation metadata stored for the entity may include the following:

In some embodiments, the present disclosure is directed to a system configured to receive a voice query from a user, analyze the voice query, and generate a text query (e.g., a transcription) for searching for content or information. The information fields that the system searches include pronunciation metadata, alternative text representations of entities, or both. For example, when a user issues a voice query to the system, the system first converts the speech to text using an ASR module. The resulting text then forms an input to a conversational system (e.g., performing an action in response to the query). To illustrate, if a user says, "Show me a Tom Cruise movie," the ASR module converts the user's speech to text and issues a text query to the conversational system. If an entity corresponding to "Tom Cruise" exists in the data, the system matches it with the text "Tom Cruise" and returns the appropriate results (e.g., information about Tom Cruise, content featuring Tom Cruise, or a content identifier thereof). An entity may be referred to as "reachable" when it exists in the data (e.g., in an information field) and can be accessed directly using the entity title. Reachability is essential for the system to perform search operations. For example, an entity may be referred to as "unreachable" if a piece of data (e.g., a movie, artist, television series, or other entity) exists in the system and associated data is stored, but a user cannot access the information. An unreachable entity in the data system represents a failure of the search system.

The system may identify one or more entities or content items among the plurality of stored information. In some embodiments, the system generates an audio file based on a first text string representing the entity or content item. Based on the first text string and at least one speech criterion, the system may generate a second text string based on the audio file using a speech-to-text module. The system compares the text strings and stores the second text string if it is not identical to the first text string. In some embodiments, the system generates metadata including results from the text-to-speech-to-text conversion to anticipate possible misidentifications when responding to voice queries during search operations. The metadata may include alternative representations of the entity to improve reachability.

FIG. 1 shows a block diagram of an illustrative system 100 for generating text queries, according to some embodiments of the present disclosure. System 100 includes an ASR module 110, a conversation system 120, pronunciation metadata 150, user profile information 160, and one or more databases 170. For example, ASR module 110 and conversation system 120, which may be included together in system 199, may be used to implement a query application.

A user may voice a query 101, including the utterance "Show me that interview with Louis from last week," into an audio interface of the system 199. The ASR module 110 is configured to sample, condition, and digitize the received audio input, analyze the resulting audio file, and generate a text query. In some embodiments, the ASR module 110 retrieves information from the user profile information 160 to aid in generating the text query. For example, speech recognition information about a user may be stored in the user profile information 160, and the ASR module 110 may use the speech recognition information to identify the speaking user. In a further example, the system 199 may include the user profile information 160 stored in a suitable memory. The ASR module 110 may determine pronunciation information about the voiced word "Louis." Because there are more than one pronunciation for the text word "Louis," the system 199 generates the text query based on the pronunciation information. Further, the sound "Loo-his" may be converted to text as "Louis" or "Lewis," and thus the context information may help identify the correct entity of the voice query (e.g., Lewis as in Lewis Black as opposed to Louis as in Louis Farrakhan). In some embodiments, conversation system 120 is configured to generate a text query, respond to a text query, or both, based on the recognized words from ASR module 110, the context information, user profile information 160, pronunciation metadata 150, one or more databases 170, any other information, or any combination thereof. For example, conversation system 120 may generate a text query and then compare the text query to pronunciation metadata 150 for multiple entities to determine a match. In a further example, the conversation system 120 may compare one or more recognized words to the pronunciation metadata 150 for multiple entities to determine matches and then generate a text query based on the identified entities. In some embodiments, the conversation system 120 generates a text query with accompanying pronunciation information. In some embodiments, the conversation system 120 generates a text query with embedded pronunciation information. For example, the text query may include a phoneme representation of a word such as "loo-ee" rather than the correct grammatical representation "Louis." In a further example, the pronunciation metadata 150 may include one or more reference phoneme representations to which the text query may be compared.

User profile information 160 may include user identification information (e.g., name, identifier, address, contact information), user search history (e.g., previous voice queries, previous text queries, previous search results, feedback regarding previous search results or queries), user preferences (e.g., search settings, favorite entities, keywords included in two or more queries), user likes/dislikes (e.g., entities followed by the user in a social media application, user entered information), other users connected to the user (e.g., friends, family, contacts in a social networking application, contacts stored on the user device), user voice data (e.g., audio samples, signatures, speech patterns, or files to identify the user's voice), any other suitable information about the user, or any combination thereof.

The one or more databases 170 include any suitable information for generating text queries, responding to text queries, or both. In some embodiments, the phonetic metadata 150, the user profile information 160, or both may be included in the one or more databases 170. In some embodiments, the one or more databases 170 include statistical information (e.g., search history, content consumption history, consumption patterns) about a plurality of users. In some embodiments, the one or more databases 170 include information about a plurality of entities, including people, places, objects, events, content items, media content associated with the one or more entities, or combinations thereof.

FIG. 2 shows a block diagram of an illustrative system 200 for retrieving content in response to a voice query, according to some embodiments of the present disclosure. The system 200 includes a speech processing system 210, a search engine 220, an entity database 250, and user profile information 240. The speech processing system 210 may identify audio files and analyze the audio files for phonemes, patterns, words, or other elements from which keywords may be identified. In some embodiments, the speech processing system 210 may analyze the audio input and identify words in the time domain, the spectral domain, or both. For example, the speech processing system 210 may analyze the audio input in the time domain and determine periods in which speech occurs (e.g., to eliminate periods of pauses or silence). The speech processing system 210 may then analyze each period in the spectral domain and identify phonemes, patterns, words, or other elements from which keywords may be identified. The speech processing system 210 may output the generated text query, one or more words, pronunciation information, or a combination thereof. In some embodiments, the speech processing system 210 may read data from the user profile information 240 for voice recognition, speech recognition, or both.

A search engine 220 receives the output from the speech processing system 210 and combines it with the search settings 221 and the context information 222 to generate a response to the text query. The search engine 220 may use the user profile information 240 to generate, modify, or respond to the text query. The search engine 220 uses the text query to search among data in a database of entities 250. The database of entities 250 may include metadata associated with a number of entities, content associated with a number of entities, or both. For example, the data may include an identifier for the entity, details describing the entity, a title referring to the entity (e.g., may include a phonetic representation or an alternative representation), a phrase associated with the entity (e.g., may include a phonetic representation or an alternative representation), a link associated with the entity (e.g., an IP address, a URL, a hardware address), a keyword associated with the entity (e.g., may include a phonetic representation or an alternative representation), any other suitable information associated with the entity, or any combination thereof. Once the search engine 220 has identified one or more entities that match the keywords of the text query, identified one or more content items that match the keywords of the text query, or both, the search engine 220 may then provide the information, content, or both to the user as a response 270 to the text query. In some embodiments, the search settings 221 include databases, entities, types of entities, types of content, other search criteria, or any combination thereof, that affect the generation of the text query, the retrieval of search results, or both. In some embodiments, the context information 222 includes genre information (e.g., to further narrow the search field), keywords, database identification (e.g., databases likely to contain the targeted information or content), type of content (e.g., by date, genre, title, format), any other suitable information, or any combination thereof. The response 270 may include, for example, content (e.g., a displayed video), information, a list of search results, links to content, any other suitable search results, or any combination thereof.

FIG. 3 illustrates a block diagram of an illustrative system 300 for generating pronunciation information according to some embodiments of the present disclosure. The system 300 includes a text-to-speech engine 310 and a speech-to-text engine 320. In some embodiments, the system 300 determines the pronunciation information independent of a text or voice query. For example, the system 300 may generate metadata about one or more entities (e.g., metadata stored in a database of pronunciation metadata 150 of the system 100 or entities 250 of the system 200, etc.). The text-to-speech engine 310 may identify a first text string 302 that may include an entity name or other identifier that is likely to be included in a voice query. For example, the text-to-speech engine 310 may identify a “name” field of the entity metadata rather than an “ID” field because a user is more likely to speak a voice query that includes a name rather than a numeric or alphanumeric identifier (e.g., a user speaks “Louis” rather than “Wiki04556”). The text-to-speech engine 310 generates an audio output 312 at a speaker or other audio device based on the first text string. For example, the text-to-speech engine 310 may use one or more settings to specify voice details (e.g., male/female voice, accent, or other details), playback speed, or any other suitable settings that may affect the generated audio output. The speech-to-text engine 320 receives an audio input 313 from the audio output 312 at a microphone or other suitable device (e.g., in addition to or instead of an audio file that may be stored) and generates a text conversion of the audio input 313 (e.g., in addition to or instead of storing an audio file of the recorded audio). The speech-to-text engine 320 may use the processing settings to generate a new text string 322. The new text string 322 is compared to the first text string 302. If the new text string 322 is identical to the text string 302, no metadata needs to be generated since the voice query may result in a conversion to the exact text query. If new text string 322 is not identical to text string 302, this may indicate that the voice query was incorrectly converted to a text query. Thus, if new text string 322 is not identical to text string 302, speech-to-text engine 320 includes new text string 322 in the metadata associated with the entity with which text string 302 is associated. System 300 may identify multiple entities and generate metadata for each entity that includes the resulting text strings (e.g., new text string 322, etc.) from text-to-speech engine 310 and speech-to-text engine 320. In some embodiments, for a given entity, text-to-speech engine 310, speech-to-text engine 320, or both may generate two or more new text strings using two or more settings. Thus, since the two or more text strings are different from text string 302, each new text string may then be stored in the metadata. For example, different pronunciations or pronunciation interpretations resulting from different settings may generate different new text strings, which may be stored for voice queries from different users. By generating and storing alternative representations (e.g., text string 302 and new text string 322), system 300 may update metadata and enable more accurate searches (e.g., improving entity reachability and search accuracy).

In an illustrative example, for an entity, system 300 identifies the title and related phrases, passes each phrase through text-to-speech engine 310, saves a respective audio file, and then passes each respective audio file through speech-to-text engine 320 to obtain an ASR transcript (e.g., new text string 322). If the ASR transcript differs from the original phrase (e.g., text string 302), system 300 adds the ASR transcript to the entity's related phrases (e.g., as stored in metadata). In some embodiments, system 300 does not require any manual work and may be fully automated (e.g., no user input is required). In some embodiments, system 300 is alerted when a user issues a query and does not get the desired results. In response, a human manually identifies what should be the correct entity for the query. Incorrect results are stored and provide information for future queries. System 300 addresses potential inaccuracies at the metadata level, not at the system level. The analysis of the text string 302 for many entities can be exhaustive and automatic, such that all erroneous examples are identified and resolved up front (e.g., prior to a user's voice query). The system 300 does not require a user to provide a voice query in order to generate erroneous examples (e.g., alternative expressions). The system 300 can be used to emulate a user's interaction with the query system and anticipate potential sources of error in conducting a search.

The user may access content, applications (e.g., for interpreting voice queries), and other features from, for example, one or more of their devices (i.e., user equipment or audio equipment), one or more network-connected devices, one or more electronic devices having a display, or combinations thereof. Any of the illustrative techniques of this disclosure may be implemented by a user device, a device that provides a display to a user, or any other suitable control circuitry configured to respond to voice queries and generate display content to a user.

FIG. 4 illustrates a generalized embodiment of an illustrative user device. User equipment system 401 may include a set-top box 416 that includes or is communicatively coupled to a display 412, audio equipment 414, and a user input interface 410. In some embodiments, display 412 may include a television display or a computer display. In some embodiments, user input interface 410 is a remote control device. Set-top box 416 may include one or more circuit boards. In some embodiments, one or more circuit boards include processing circuitry, control circuitry, and storage (e.g., RAM, ROM, hard disk, removable disk, etc.). In some embodiments, the circuit boards include input/output paths. Each of user equipment device 400 and user equipment system 401 may receive content and data via input/output (hereinafter "I/O") paths 402. I/O paths 402 may provide content and data to control circuitry 404, which includes processing circuitry 406 and storage 408. The control circuitry 404 may be used to send and receive commands, requests, and other suitable data using the I/O paths 402. The I/O paths 402 may connect the control circuitry 404 (specifically, the processing circuitry 406) to one or more communication paths (described below). The I/O functions may be provided by one or more of these communication paths, but are shown as a single path in FIG. 4 to avoid overcomplicating the drawing. Although a set-top box 416 is shown in FIG. 4 for illustrative purposes, any suitable computing device having processing circuitry, control circuitry, and storage may be used in accordance with the present disclosure. For example, the set-top box 416 may be replaced or supplemented by a personal computer (e.g., notebook, laptop, desktop), a network-based server hosting user-accessible client devices, a non-user-owned device, any other suitable device, or any combination thereof.

The control circuitry 404 may be based on any suitable processing circuitry, such as the processing circuitry 406. As referred to herein, the processing circuitry should be understood to mean a circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or a supercomputer. In some embodiments, the processing circuitry is distributed across multiple separate processors or processing units, such as multiple processing units of the same type (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, the control circuitry 404 executes instructions for an application stored in memory (e.g., the storage device 408). Specifically, the control circuitry 404 may be instructed by the application to perform the functions discussed above and below. For example, an application may provide instructions to control circuitry 404 to generate a media guide display. In some implementations, any actions performed by control circuitry 404 may be based on instructions received from the application.

In some client/server-based embodiments, the control circuitry 404 includes communications circuitry suitable for communicating with an application server or other network or server. Instructions for performing the functionality described above may be stored on the application server. The communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, an Ethernet card, or a wireless modem for communicating with other equipment or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communications network or path. In addition, the communications circuitry may include circuitry (described in more detail below) that enables peer-to-peer communication of user equipment devices or communication of user equipment devices at remote locations from each other.

The memory may be an electronic storage device such as storage device 408 that is part of control circuitry 404. As referred to herein, the phrase "electronic storage device" or "storage device" should be understood to mean any device for storing electronic data, computer software, or firmware, such as any combination of random access memory, read-only memory, hard drives, optical drives, solid state devices, quantum storage devices, game consoles, game media, or any other suitable fixed or removable storage devices. Storage device 408 may be used to store various types of content described herein and media guidance data described above. Non-volatile memory may also be used (e.g., to initiate boot-up routines and other instructions). Cloud-based storage may be used, for example, to supplement storage device 408 or in place of storage device 408.

A user may use the user input interface 410 to send instructions to the control circuitry 404. The user input interface 410, the display 412, or both may include a touch screen configured to provide a display and receive tactile input. For example, the touch screen may be configured to receive tactile input from a finger, a stylus, or both. In some embodiments, the equipment device 400 may include a front-facing screen and a rear-facing screen, multiple forward screens, or multiple angled screens. In some embodiments, the user input interface 410 includes a remote control device having one or more microphones, buttons, keypads, any other components configured to receive user input, or combinations thereof. For example, the user input interface 410 may include a handheld remote control device having an alphanumeric keypad and options. In a further example, the user input interface 410 may include a handheld remote control device having a microphone and control circuitry configured to receive and identify voice commands and transmit information to the set-top box 416.

Audio equipment 414 may be provided as integrated with other elements of each of user device 400 and user equipment system 401, or may be a stand-alone unit. Audio components of videos and other content displayed on display 412 may be played through speakers of audio equipment 414. In some embodiments, the audio may be distributed to a receiver (not shown), which processes the audio and outputs it through speakers of audio equipment 414. In some embodiments, for example, control circuitry 404 is configured to provide audio cues to the user or other audio feedback to the user using speakers of audio equipment 414. Audio equipment 414 may include a microphone configured to receive audio input, such as voice commands and utterances (including, for example, voice queries). For example, a user may speak letters or words, which are received by the microphone and converted to text by control circuitry 404. In a further example, a user may vocalize commands, which are received by the microphone and recognized by control circuitry 404.

The application (e.g., for managing voice queries) may be implemented using any suitable architecture. For example, a standalone application may be implemented entirely on each of user device 400 and user equipment system 401. In some such embodiments, instructions for the application are stored locally (e.g., in storage 408) and data for use by the application is downloaded on a periodic basis (e.g., from an out-of-band feed, from an Internet resource, or using another suitable approach). Control circuitry 404 may read instructions for the application from storage 408, process the instructions, and generate any of the displays discussed herein. Based on the processed instructions, control circuitry 404 may determine the content of an action to perform when input is received from user input interface 410. For example, movement of a cursor on the display up/down may be indicated by the processed instructions when input interface 410 indicates that the up/down button has been selected. The application and/or any instructions for implementing any of the embodiments discussed herein may be encoded on a computer-readable medium. A computer-readable medium includes any medium capable of storing data. A computer-readable medium may be transient, including, but not limited to, a propagating electrical or electromagnetic signal, or may be non-transient, including, but not limited to, volatile and non-volatile computer memory or storage devices such as hard disks, floppy disks, USB drives, DVDs, CDs, media cards, registered memory, processor cache, random access memory (RAM), etc.

In some embodiments, the application is a client/server based application. Data for use by a thick or thin client implemented on each of the user device 400 and the user equipment system 401 is retrieved on demand by issuing requests to a server remote from each of the user equipment device 400 and the user equipment system 401. For example, the remote server may store instructions for the application in a storage device. The remote server may use circuitry (e.g., control circuitry 404) to process the stored instructions and generate the displays discussed above and below. The client device may receive the display generated by the remote server and display the contents of the display locally on the user device 400. In this manner, the processing of the instructions is performed remotely by the server, while the resulting display, which may include text, a keyboard, or other visual objects, is provided locally on the user device 400. The user device 400 may receive inputs from a user via the input interface 410 and transmit those inputs to the remote server for processing and generating corresponding displays. For example, the user device 400 may transmit a communication to a remote server indicating that an up/down button was selected via the input interface 410. The remote server may process instructions according to the input and generate a display of the application corresponding to the input (e.g., a display that moves a cursor up/down). The generated display is then transmitted to the user device 400 for presentation to the user.

In some embodiments, the application is downloaded and interpreted or otherwise launched by an interpreter or virtual machine (e.g., launched by the control circuitry 404). In some embodiments, the application may be encoded in ETV Binary Interchange Format (EBIF) and received by the control circuitry as part of a suitable feed and interpreted by a user agent running on the control circuitry 404. For example, the application may be an EBIF application. In some embodiments, the application may be defined by a series of JAVA-based files that are received and launched by a local virtual machine or other suitable middleware executed by the control circuitry 404.

5 illustrates a block diagram of an illustrative network arrangement 500 for responding to voice queries, in accordance with some embodiments of the present disclosure. The illustrative system 500 may represent a situation in which a user provides a voice query at a user device 550, views content on a display of the user device 550, or both. In the system 500, there may be more than one type of user device, although only one is shown in FIG. 5 to avoid overcomplicating the drawing. In addition, each user may utilize more than one type of user device, and may also utilize more than one of each type of user device. The user device 550 may be the same as the user device 400 of FIG. 4, the user equipment system 401, any other suitable device, or any combination thereof.

The user device 550, illustrated as a wireless-enabled device, may be coupled to the communications network 510 (e.g., connected to the Internet). For example, the user device 550 is coupled to the communications network 510 via a communications path (which may include, for example, an access point). In some embodiments, the user device 550 may be a computing device coupled to the communications network 510 via a wired connection. For example, the user device 550 may also include a wired connection to a LAN or any other suitable communications link to the network 510. The communications network 510 may be one or more networks including the Internet, a cellular network, a mobile voice or data network (e.g., a 4G or LTE network), a cable network, a public switched telephone network, or any other type of communications network or combination of communications networks. The communications path may include one or more communications paths, such as a satellite path, a fiber optic-based path, a cable path, a path supporting Internet communications, a free-space connection (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths. Although no communication paths are depicted between user device 550 and network device 520, these devices may communicate with each other directly via communication paths such as those described above and other short-range point-to-point communication paths such as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or other short-range communications via wired or wireless paths. BLUETOOTH is a certification mark owned by Bluetooth SIG, Inc. The devices may also communicate with each other directly through an indirect path via communication network 510.

The system 500 as shown includes a network device 520 (e.g., a server or other suitable computing device) coupled to a communication network 510 via a suitable communication path. Communications between the network device 520 and the user device 550 may be exchanged via one or more communication paths, but are shown in FIG. 5 as a single path to avoid overcomplicating the drawing. The network device 520 may include a database and one or more applications (e.g., an application server, as a host server). Multiple network entities may exist and communicate with the network 510, but only one is shown in FIG. 5 to avoid overcomplicating the drawing. In some embodiments, the network device 520 may include one source device. In some embodiments, the network device 520 implements an application that communicates with instances of the application in many user devices (e.g., the user device 550). For example, an instance of a social media application may be implemented on user device 550, and application information may be communicated to and from network device 520, which may store profile information about the user (e.g., so that current social media feeds are available on devices other than user device 550). In a further example, an instance of a search application may be implemented on user device 550, and application information may be communicated to and from network device 520, which may store profile information about the user, search history from multiple users, entity information (e.g., content and metadata), any other suitable information, or any combination thereof.

In some embodiments, the network device 520 includes one or more types of stored information, including, for example, entity information, metadata, content, historical communication and search records, user preferences, user profile information, any other suitable information, or any combination thereof. The network device 520 may include an application host database or server, plug-ins, software developer kits (SDKs), application programming interfaces (APIs), or other software tools configured to provide software (e.g., to be downloaded to the user device), remotely launch software (e.g., to host applications accessed by the user device), or otherwise provide application support to applications of the user device 550. In some embodiments, information from the network device 520 is provided to the user device 550 using a client/server approach. For example, the user device 550 may pull information from a server, or the server may push information to the user device 550. In some embodiments, an application client residing on user device 550 may initiate a session with network device 520 and retrieve information as needed (e.g., as data becomes stale or as the user device receives a request from the user to receive data). In some embodiments, the information may include user information (e.g., user profile information, user-created content). For example, the user information may include current and/or historical user activity information, such as content transactions in which the user is involved, searches performed by the user, content consumed by the user, whether the user interacts with social networks, any other suitable information, or any combination thereof. In some embodiments, the user information may identify patterns of a given user over a period of time. As illustrated, network device 520 includes entity information for multiple entities. Entity information 521, 522, and 523 include metadata for the respective entities. The entities for which metadata is stored in network device 520 may be linked to one another, referenced to one another, described by one or more tags in the metadata, or a combination thereof.

In some embodiments, the application may be implemented on the user device 550, the network device 520, or both. For example, the application may be implemented as a software or set of executable instructions that may be stored on storage of the user device 550, the network device 520, or both, and executed by the control circuitry of the respective devices. In some embodiments, the application may include an audio recording application, a speech-to-text application, a text-to-speech application, a voice-recognition application, or a combination thereof, implemented as a client/server based application, where only the client application resides on the user device 550 and the server application resides on a remote server (e.g., the network device 520). For example, the application may be implemented partially on the user device 550 (e.g., by the control circuitry of the user device 550) as a client application, and partially on the remote server as a server application that runs on the control circuitry of the remote server (e.g., the control circuitry of the network device 520). When executed by the control circuitry of the remote server, the application may instruct the control circuitry to generate a display and transmit the generated display to the user device 550. The server application may instruct the control circuitry of the remote device to transmit the data for storage on the user device 550. The client application may instruct the control circuitry of the receiving user device to generate an application display.

In some embodiments, the deployment of system 500 is a cloud-based deployment. The cloud provides access to services such as information storage, retrieval, messaging, or social networking services, among other examples, and to any of the content described above with respect to user devices. Services can be provided in the cloud through a cloud-computing service provider or through other providers of online services. For example, cloud-based services can include storage services, sharing sites, social networking sites, search engines, or other services where user-sourced content is distributed for viewing by others on connected devices. These cloud-based services can enable user devices to store information in the cloud and receive information from the cloud, rather than storing information locally and accessing the locally stored information. Cloud resources can be accessed by user devices using, for example, a web browser, messaging applications, social media applications, desktop applications, or mobile applications, and can include audio recording applications, speech-to-text applications, text-to-speech applications, voice-recognition applications, and/or any combination of those access applications. User device 550 may be a cloud client that relies on cloud computing for application delivery, or user device 550 may have some functionality without access to cloud resources. For example, some applications running on user device 550 may be cloud applications (e.g., applications delivered as a service over the Internet), while other applications may be stored and run on user device 550. In some embodiments, user device 550 may receive information from multiple cloud resources simultaneously.

In an illustrative example, a user may speak a voice query into user device 550. The voice query is recorded by an audio interface of user device 550, sampled and digitized by application 560, and converted into a text query by application 560. Application 560 may also include a pronunciation along with the text query. For example, one or more words of the text query may be represented by phonemic symbols rather than proper spelling. In a further example, pronunciation metadata may be stored along with the text query including a phonemic representation of one or more words of the text query. In some embodiments, application 560 transmits the text query and any suitable pronunciation information to network device 520 for searching in a database of entities, content, metadata, or combinations thereof. Network device 520 may identify entities associated with the text query, content associated with the text query, or both, and provide the information to user device 550.

For example, a user may speak "Watch Tom Cruise movies" into the microphone of user device 550. Application 560 may generate a text query "Tom Cruise movies" and transmit the text query to network device 520. Network device 520 may identify the entity "Tom Cruise" and then identify movies that are linked to the entity. Network device 520 may then transmit the content (e.g., video file, trailer, or clip), content identifier (e.g., movie title and image), content address (e.g., URL, website, or IP address), any other suitable information, or any combination thereof, to user device 550. Because the pronunciation of "Tom" and "Cruise" is generally unambiguous, application 560 does not need to generate pronunciation information in this situation.

In a further example, a user may speak "show me an interview with Louis" into the microphone of user device 550, where the user pronounces the name Louis as "loo-ee" rather than "loo-ihs". In some embodiments, application 560 may generate a text query "interview with Louis" and transmit the text query to network device 520 along with metadata including the phonetic representation as "loo-ee". In some embodiments, application 560 may generate a text query "interview with Loo-ee" and transmit the text query to network device 520, where the text query itself includes pronunciation information (e.g., in this example, the phonetic representation). Because the name Louis is common, there may be many entities that include this identifier. In some embodiments, network device 520 may identify entities that have metadata including a pronunciation tag that has "loo-ee" as a phonetic representation. In some embodiments, the network device 520 may read trending searches, the user's search history, or other contextual information to identify entities that the user is likely to refer to. For example, the user may have previously searched for "FBI," and the entity Louis Freeh (e.g., former Director of the FBI) may include metadata that includes tags related to "FBI." Once an entity is identified, the network device 520 may then transmit to the user device 550 the content (e.g., a video file or clip of the interview), a content identifier (e.g., a file title and still images from the interview), a content address (e.g., a URL, website, or IP address for streaming one or more video files of the interview), any other suitable information related to Louis Freeh, or any combination thereof. Because the pronunciation of "Louis" may be ambiguous, the application 560 may generate pronunciation information in such circumstances.

In an illustrative example, a user may speak "William Djoko" into the microphone of user device 550. Application 560 may generate a text query that may not correspond to the correct spelling of the entity. For example, the voice query "William Djoko" may be converted to text as "William gjoka." This incorrect text conversion may result in difficulty in identifying the correct entity. In some embodiments, metadata associated with the entity William Djoko includes alternative representations based on pronunciation. Metadata for the entity "William Djoko" may include pronunciation tags (e.g., "related phrases"), as shown in Table 1.
Although the text query may include an incorrect spelling, the correct entity may be identified because the metadata associated with the correct entity includes the variant. Thus, network device 520 may include entity information including alternative expressions and thus identify the correct entity in response to a text query including the phrase "William gjoka". Once the entity is identified, network device 520 may then transmit to user device 550 the content (e.g., an audio or video file clip), a content identifier (e.g., a song or album title and a still image from a concert), a content address (e.g., a URL, website, or IP address for streaming one or more audio files of music), any other suitable information related to William Djoko, or any combination thereof. Because the name "Djoko" may be translated incorrectly from speech, application 560 may generate phonetic information for identifying the correct entity in such a situation for storage in the metadata.

In the illustrative example above, the reachability of the entity William Djoko is improved by storing alternative representations, particularly since the ASR process may result in grammatically incorrect textual translations of the entity name.

In an illustrative example, metadata may be generated based on the pronunciation, not in response to a user's voice query, but for later reference (e.g., via a text query or other search and retrieval process). In some embodiments, network device 520, user device 550, or both may generate metadata based on the pronunciation information. For example, user device 550 may receive user input of an alternative representation of an entity (e.g., based on previous search results or speech-to-text conversion). In some embodiments, network device 520, user device 550, or both may automatically generate metadata about an entity using a text-to-speech module and a speech-to-text module. For example, application 560 may identify a text representation of an entity (e.g., a text string of the entity's name), input the text representation to the text-to-speech module, and generate an audio file. In some embodiments, the text-to-speech module includes one or more settings or criteria with which the audio file is generated. For example, the settings or criteria may include language (e.g., English, Spanish, Mandarin), accent (e.g., regional or language-based), voice (e.g., a particular person's voice, male voice, female voice), speed (e.g., play time of the relevant portion of the audio file), pronunciation (e.g., with respect to multiple phoneme variants), any other suitable settings or criteria, or any combination thereof. Application 560 then inputs the audio file into a speech-to-text module and generates a resultant text representation. If the resultant text representation is not identical to the original text representation, application 560 may store the resultant text representation in metadata associated with the entity. In some embodiments, application 560 may repeat this process for different settings or criteria, thus generating different text representations that may be stored in the metadata. The resultant metadata includes the original text representation along with variants generated using text-to-speech-to-text conversion to anticipate likely variants. Thus, when application 560 receives a voice query from a user and the transcription to text does not exactly match the entity identifier, application 560 may still identify the correct entity. Additionally, application 560 does not need to analyze the text query for pronunciation information because the metadata includes the variants (e.g., the analysis is performed in advance rather than in real time).

Application 560 may include any suitable functionality, such as, for example, audio recording, speech recognition, speech-to-text conversion, text-to-speech conversion, query generation, search engine functionality, content retrieval, display generation, content presentation, metadata generation, database functionality, or combinations thereof. In some embodiments, aspects of application 560 are implemented across two or more devices. In some embodiments, application 560 is implemented on a single device. For example, entity information 521, 522, and 523 may be stored in memory storage of user device 550 and accessed by application 560.

FIG. 6 shows a flowchart of an illustrative process 600 for responding to a voice query based on pronunciation information, according to some embodiments of the present disclosure. For example, the query application may perform process 600 implemented on any suitable hardware, such as user device 400 of FIG. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof. In a further example, the query application may be an instance of application 560 of FIG. 5.

In step 602, the query application receives a voice query. In some embodiments, the audio interface (e.g., audio equipment 414, user input interface 410, or a combination thereof) may include a microphone or other sensor that receives the audio input and generates an electronic signal. In some embodiments, the audio input is received at an analog sensor that provides an analog signal that is conditioned, sampled, and digitized to generate an audio file. The audio file may then be analyzed by the query application in steps 604 and 606. In some embodiments, the audio file is stored in memory (e.g., storage device 408). In some embodiments, the query application includes a user interface (e.g., user input interface 410) that allows a user to record, play, modify, crop, visualize, or otherwise manage the audio recording. For example, in some embodiments, the audio interface is configured to receive audio input at all times. In a further example, in some embodiments, the audio interface is configured to receive audio input when a user provides an instruction to the input interface (e.g., by selecting a soft button on a touch screen to start an audio recording). In a further example, in some embodiments, the audio interface is configured to receive audio input and begin recording when speech or other suitable audio signals are detected. The query application may include any suitable conditioning software or hardware to convert the audio input into a stored audio file. For example, the query application may apply one or more filters (e.g., low-pass, high-pass, notch, or band-pass filters), amplifiers, digitizers, or other conditioning to generate an audio file. In a further example, the query application may apply any suitable processing, such as compression, transformation (e.g., spectral transformation, wavelet transformation), normalization, equalization, truncation (e.g., in the time or spectral domain), any other suitable processing, or any combination thereof, to the conditioned signal to generate an audio file. In some embodiments, in step 602, the control circuitry receives the audio file from a separate application, from a separate module of the query application, based on user input, or any combination thereof. For example, in step 602, the control circuitry may receive the voice query as an audio file that is stored in a storage device (e.g., storage device 408) for further processing (e.g., steps 604-612 of process 600).

In step 604, the query application extracts one or more keywords from the voice query of step 602. In some embodiments, the one or more keywords may represent the complete voice query. In some embodiments, the one or more keywords include only significant words or portions of the utterance. For example, in some embodiments, the query application may identify words in the utterance and select some of those words as keywords. For example, the query application may identify words and select from among those words those words that are not prepositions. In a further example, the query application may identify as keywords only words that are at least three characters long. In a further example, the query application may identify keywords as phrases that contain two or more words (e.g., to be more descriptive and provide more context), which may be useful for narrowing down the potential search field of related content. In some embodiments, the query application uses any suitable criteria for identifying keywords from the audio input, such as, for example, words, phrases, names, places, channels, media asset titles, or other keywords. The query application may process the words using any suitable word detection technique, speech detection technique, pattern recognition technique, signal processing technique, or any combination thereof. For example, the query application may compare a series of signal templates to a portion of the audio signal to find out if a match exists (e.g., whether a particular word is included in the audio signal). In a further example, the query application may apply learning techniques to better recognize words in the voice query. For example, the query application may use past data as a training set to gather feedback from users regarding multiple requested content items in the context of multiple queries, and thus make recommendations and retrieve content. In some embodiments, the query application may store snippets (i.e., short duration clips) of recorded audio during the detected speech and process the snippets. In some embodiments, the query application stores relatively large segments of the speech (e.g., greater than 10 seconds) as audio files and processes the files. In some embodiments, the query application may process the speech and detect words by using ongoing calculations. For example, a wavelet transform may be performed on the speech in real time, albeit with some time delay, to provide a continuous computation of the speech pattern (e.g., that may be compared to a reference to identify words). In some embodiments, a query application may detect the words and the user who spoke the words (e.g., speech recognition) in accordance with the present disclosure.

In some embodiments, in step 604, the query application adds the detected word to a list of words detected in the query. In some embodiments, the query application may store these detected words in memory. For example, the query application may store the words in memory as a set of ASCII characters (i.e., an 8-bit code), a pattern (e.g., indicating the speech signal criteria used to match the word), an identifier (e.g., a code for the word), a string, any other data type, or any combination thereof. In some embodiments, the media guidance application may add words to memory as they are detected. For example, the media guidance application may append the newly detected word to a string of previously detected words, add the newly detected word to a cell array of previously detected words (e.g., increase the cell array size by 1), create a new variant corresponding to the newly detected word, create a new file corresponding to the newly created word, or store one or more words detected in step 604.

At step 606, the query application determines pronunciation information for the one or more keywords of step 604. In some embodiments, the pronunciation information includes a phonemic representation (e.g., using the International Phonetic Alphabet) of the one or more keywords. In some embodiments, the pronunciation information includes one or more alternative spellings of the one or more keywords to incorporate the pronunciation. In some embodiments, at step 606, the control circuitry generates metadata associated with the text query that includes the phonemic representation.

At step 608, a query application generates a text query based on the one or more keywords from step 604 and the pronunciation information from step 606. The query application may generate the text query by placing the one or more keywords in a preferred order (e.g., in the order in which they were spoken). In some embodiments, the query application may omit one or more words from the voice query (e.g., short words, prepositions, or any other words determined to be relatively less important). The text query may be generated as a file (e.g., a text file) and stored in a suitable storage device (e.g., storage device 408).

At step 610, the query application identifies an entity among the multiple entities in the database based on the text query and the stored metadata about the entity. In some embodiments, the metadata includes a pronunciation tag. In some embodiments, the query application may identify the entity by identifying a metadata tag of a content item that corresponds to the entity. For example, the content item may include a movie that has tags about actors in the movie. If the text query includes actors, the query application may determine a match and, based on the match, identify the entity as associated with the content item. To illustrate, the query application may first identify the entity (e.g., search among the entities) and then retrieve the content associated with the entity, or the query application may first identify the content (e.g., search among the content) and determine whether the entity associated with the content matches the text query. A database arranged by entity, by content, or both may be searched by the query application.

In some embodiments, the query application identifies an entity based on user profile information. For example, the query application may identify an entity based on already identified entities from a previous voice query. In a further example, the query application may identify an entity based on popularity information associated with the entity (e.g., based on searches across multiple users). In some embodiments, the query application identifies an entity based on user preferences. For example, if one or more keywords match a preferred entity name or identifier in the user profile information, the query application may identify or weight the entity more heavily.

In some embodiments, the query application identifies the entities by identifying a plurality of entities (e.g., using metadata stored for each entity), determining a respective score for each respective entity of the plurality of entities based on comparing the respective pronunciation tags to the text query, and selecting the entity by determining a maximum score. The score may be based on the number of matches identified between keywords of the text query and metadata associated with the entities or content items.

In some embodiments, the query application identifies two or more entities (e.g., associated metadata) among the plurality of entities based on the text query. The query application may identify content items associated with some or all of the entities of the query. In some embodiments, the query application identifies the entities by comparing at least a portion of the text query to metadata tags stored for each entity and identifying matches.

At step 612, the query application retrieves a content item associated with the entity. In some embodiments, the query application identifies a content item, downloads a content item, streams a content item, generates a content item for display, or a combination thereof. For example, a voice query may include "show me the latest Tom Cruise movies" and the query application may provide a link to the movie "Mission Impossible: Fallout" from which the user may select to view the video content. In some embodiments, the query application may retrieve a number of content items associated with the entity that matches the text query. For example, the query application may retrieve a number of links, video files, audio files, or other content, or a list of identified content items in accordance with this disclosure.

7 shows a flowchart of an illustrative process 700 for responding to a voice query based on alternative expressions, according to some embodiments of the present disclosure. For example, the query application may perform process 700 implemented on any suitable hardware, such as user device 400 of FIG. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof. In a further example, the query application may be an instance of application 560 of FIG. 5.

In step 702, the query application receives a voice query. In some embodiments, an audio interface (e.g., audio equipment 414, user input interface 410, or a combination thereof) may include a microphone or other sensor that receives audio input and generates an electronic signal. In some embodiments, the audio input is received at an analog sensor that provides an analog signal that is conditioned, sampled, and digitized to generate an audio file. The audio file may then be analyzed by the query application in step 704. In some embodiments, the audio file is stored in memory (e.g., storage device 408). In some embodiments, the query application includes a user interface (e.g., user input interface 410) that allows a user to record, play, modify, crop, visualize, or otherwise manage the audio recording. For example, in some embodiments, the audio interface is configured to receive audio input at all times. In a further example, in some embodiments, the audio interface is configured to receive audio input when a user provides an instruction to the user interface (e.g., by selecting a soft button on a touch screen to start an audio recording). In a further example, in some embodiments, the audio interface is configured to receive audio input and begin recording when speech or other suitable audio signals are detected. The query application may include any suitable conditioning software or hardware for converting the audio input into a stored audio file. For example, the query application may apply one or more filters (e.g., low-pass, high-pass, notch, or band-pass filters), amplifiers, digitizers, or other conditioning to generate an audio file. In a further example, the query application may apply any suitable processing to the conditioned signal, such as compression, transformation (e.g., spectral transformation, wavelet transformation), normalization, equalization, truncation (e.g., in the time or spectral domain), any other suitable processing, or any combination thereof, to generate an audio file. In some embodiments, at step 702, the control circuitry receives the audio file from a separate application, from a separate module of the query application, based on user input, or any combination thereof. For example, step 702 may include receiving the voice query as an audio file that is stored in a storage device (e.g., storage device 408) for further processing (e.g., steps 704-710 of process 700).

In step 704, the query application extracts one or more keywords from the voice query of step 702. In some embodiments, the one or more keywords may represent the complete voice query. In some embodiments, the one or more keywords include only significant words or portions of the utterance. For example, in some embodiments, the query application may identify words in the utterance and select some of those words as keywords. For example, the query application may identify words and select from among those words those words that are not prepositions. In a further example, the query application may identify only words that are at least three characters long as keywords. In a further example, the query application may identify keywords as phrases that contain two or more words (e.g., to be more descriptive and provide more context), which may be useful for narrowing down the potential search field of related content. In some embodiments, the query application uses any suitable criteria for identifying keywords from the audio input, such as, for example, words, phrases, names, places, channels, media asset titles, or other keywords. The query application may process the words using any suitable word detection technique, speech detection technique, pattern recognition technique, signal processing technique, or any combination thereof. For example, the query application may compare a series of signal templates to a portion of the audio signal to find out if a match exists (e.g., whether a particular word is included in the audio signal). In a further example, the query application may apply learning techniques to better recognize words in the voice query. For example, the query application may use past data as a training set to gather feedback from users regarding multiple requested content items in the context of multiple queries, and thus make recommendations and retrieve content. In some embodiments, the query application may store snippets (i.e., short duration clips) of recorded audio during the detected speech and process the snippets. In some embodiments, the query application stores relatively large segments of the speech (e.g., greater than 10 seconds) as audio files and processes the files. In some embodiments, the query application may process the speech and detect words by using ongoing calculations. For example, a wavelet transform may be performed on the speech in real time, albeit with some time delay, to provide a continuous computation of the speech pattern (e.g., that may be compared to a reference to identify words). In some embodiments, a query application may detect the words and the user who spoke the words (e.g., speech recognition) in accordance with the present disclosure.

In some embodiments, in step 704, the query application adds the detected word to a list of words detected in the query. In some embodiments, the query application may store these detected words in memory. For example, the query application may store the words in memory as a set of ASCII characters (i.e., an 8-bit code), a pattern (e.g., indicating the speech signal criteria used to match the word), an identifier (e.g., a code for the word), a string, any other data type, or any combination thereof. In some embodiments, the media guidance application may add words to memory as they are detected. For example, the media guidance application may append the newly detected word to a string of previously detected words, add the newly detected word to a cell array of previously detected words (e.g., increase the cell array size by 1), create a new variant corresponding to the newly detected word, create a new file corresponding to the newly created word, or store one or more words detected in step 704.

At step 706, a query application generates a text query based on the one or more keywords of step 704. The query application may generate the text query by placing the one or more keywords in a preferred order (e.g., in the order in which they were spoken). In some embodiments, the query application may omit one or more words of the voice query (e.g., short words, prepositions, or any other words determined to be relatively less important). The text query may be generated as a file (e.g., a text file) and stored in a suitable storage device (e.g., storage device 408).

At step 708, the query application identifies the entity based on the text query and metadata about the entity. The metadata includes an alternative text representation of the entity based on the pronunciation. In some embodiments, the query application may identify the entity by identifying a metadata tag of the content item that corresponds to the alternative representation of the entity. For example, the content item may include a movie with tags for actors in the movie, the tags including alternative spellings (e.g., derived from a system such as system 300 or otherwise included in the metadata). If the text query includes the actors, the query application may determine a match and, based on the match, identify the entity as associated with the content item. To illustrate, the query application may first identify the entity (e.g., search among the entity) and then retrieve the content associated with the entity, or the query application may first identify the content (e.g., search among the content) and determine whether the entity associated with the content matches the text query. Databases arranged by entity, by content, or both may be searched by the query application. The query application may determine a match when one or more words of the text query match an alternative representation of the entity (e.g., as stored in metadata associated with the entity).

In some embodiments, the query application identifies an entity based on user profile information. For example, the query application may identify an entity based on already identified entities from a previous voice query. In a further example, the query application may identify an entity based on popularity information associated with the entity (e.g., based on searches across multiple users). In some embodiments, the query application identifies an entity based on user preferences. For example, the query application may identify or weight an entity more heavily if one or more keywords match an alternative representation of a preferred entity name or identifier in the user profile information.

In some embodiments, the query application identifies the entities by identifying a plurality of entities (e.g., with metadata stored for each entity), determining a respective score for each respective entity of the plurality of entities based on comparing the respective metadata to the text query, and selecting the entity by determining a maximum score. The score may be based on the number of matches identified between keywords of the text query and metadata associated with the entities or content items.

In some embodiments, the query application identifies two or more entities (e.g., associated metadata) among the plurality of entities based on the text query. The query application may identify content items associated with some or all of the entities of the query. In some embodiments, the query application identifies the entities by comparing at least a portion of the text query to metadata tags stored for each entity and identifying matches.

At step 710, the query application retrieves a content item associated with the entity. In some embodiments, the query application identifies a content item, downloads a content item, streams a content item, generates a content item for display, or a combination thereof. For example, a voice query may include "show me the latest Tom Cruise movies" and the query application may provide a link to the movie "Mission Impossible: Fallout" from which the user may select to view the video content. In some embodiments, the query application may retrieve a number of content items associated with the entity that matches the text query. For example, the query application may retrieve a number of links, video files, audio files, or other content, or a list of identified content items in accordance with this disclosure.

8 shows a flowchart of an illustrative process 800 for generating metadata about an entity based on a pronunciation, according to some embodiments of the present disclosure. For example, an application may perform the process 800 implemented on any suitable hardware, such as user device 400 of FIG. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof. In a further example, the application may be an instance of application 580 of FIG. 5. In a further example, system 300 of FIG. 3 may perform the illustrative process 800.

In step 802, the application identifies an entity of a plurality of entities for which information is stored. In some embodiments, the application selects an entity based on a predefined order. For example, the application may select entities in alphabetical order to perform a portion of process 800. In some embodiments, the application identifies an entity when metadata about the entity is created. For example, the application may identify an entity when the entity is added to a database (e.g., a database of entities). In some embodiments, the application identifies an entity when a search operation misidentifies the entity and thus an alternative representation may be desirable to prevent further misidentification. In some embodiments, the application identifies an entity based on user input. For example, a user may indicate to the application (e.g., in a suitable user interface) an entity based on an incorrect search result, an unreachable entity, or an error observed in the search results. In some embodiments, the application need not identify an entity in response to an error in the search results or a predefined order. For example, the application may randomly select an entity in the entity database and proceed to step 804. In some embodiments, the application may identify an entity based on the popularity of the entity in search queries. For example, greater search effectiveness may be achieved by determining alternative representations for more common entities such that more search queries are correctly answered. In a further example, the application may identify less common or more ambiguous entities and prevent inaccessibility of those entities since fewer search queries may specify these entities. The application may apply any suitable criteria to determine the entities to identify. In some embodiments, the application may identify more than one entity in step 802, and thus steps 804-810 may be performed for each identified entity. In some embodiments, the application may identify content items rather than or in addition to entities. For example, the application may identify an entity such as a movie, and then identify all other significant entities associated with that entity and undergo steps 804-810.

In step 804, the application generates an audio file based on the first text string and at least one speech criterion. The first text string describes the entity identified in step 802. For example, as illustrated in FIG. 3, the application may include a text-to-speech engine 310, which may be configured to generate the audio file. The application may generate audio output from a speaker or other suitable sound generating device that may be detected by a microphone or other suitable detection device. The application may apply one or more settings or speech criteria in generating and outputting the audio file. For example, aspects of the generated "speech" may be adjusted or otherwise selected based on any suitable criteria. In some embodiments, the at least one speech criterion includes a pronunciation setting (e.g., the way one or more syllables, letter groups, or words are pronounced, or the phonemes to be used). In some embodiments, the at least one speech criterion includes a language setting (e.g., defining a language, accent, regional accent, or other linguistic information).

In an illustrative example involving multiple speech criteria, the application may generate respective audio files based on the first text string and the respective speech criteria, generate respective second text strings based on the respective audio files, compare each second text string to the first text string, and store each second text string (e.g., in metadata associated with the entity) if it is not identical to the first text string.

In an illustrative example, the application may convert a first text string into a first audio signal, generate speech at a speaker based on the audio signal, detect the speech using a microphone, generate a second audio signal, process the audio signal, and generate an audio file. In some embodiments, the application generates the speech at the speaker based on at least one speech setting of the text-to-speech module.

In step 806, the application generates a second text string based on the audio file. The second text string should match the first text string and describe the entity identified in step 802, aside from differences that may result from text-to-speech or speech-to-text conversion. For example, as illustrated in FIG. 3, the application may include a speech-to-text engine 320, which may be configured to receive the audio input or the generated file thereof and convert the audio into a transcript (e.g., a text string). The application may receive the audio input at a microphone or other suitable sound detection device. The application may apply one or more settings in receiving, tuning, and converting the audio file to text. For example, aspects of tuning and converting the detected "speech" may be tuned or otherwise selected based on any suitable criteria.

In an illustrative example, the application generates a playback of an audio file on a speaker and uses a microphone to detect the playback, generate an audio signal, and convert the audio signal to a second text string by identifying one or more words. In some embodiments, the application converts the audio signal to the second text string based on at least one text setting of the speech-to-text module.

In step 808, the application compares the second text string to the first text string. In some embodiments, the application compares each character of the first and second text strings to determine a match. In some embodiments, the application determines the extent to which the first and second text strings match (e.g., the percentage of text strings that match, the number of differences that exist, the number of keywords that match or do not match). The application may use any suitable technique to determine whether the first and second text strings are identical, similar, or different, and the extent to which they are similar or different.

In step 810, the application stores the second text string if it is not identical to the first text string. In some embodiments, the application stores the second text string in metadata associated with the entity. In some embodiments, step 810 includes the application updating existing metadata based on one or more text queries. For example, once the query is answered and the search results are evaluated, the application may update the metadata to reflect new learnings. If the second text string is determined to be identical to the first text string, no new information is gained by storing the second text string. However, an indication of the comparison of step 808 may be stored in the metadata to increase confidence in the reachability of the entity via the voice query. For example, if the second text string is identical to the first text string, it may serve to validate the existing metadata for the voice-based query.

9 shows a flowchart of an illustrative process 900 for retrieving content associated with an entity of a voice query, according to some embodiments of the present disclosure. For example, the query application may perform the process 900 implemented on any suitable hardware, such as user device 400 of FIG. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof. In a further example, the query application may be an instance of application 560 of FIG. 5.

In step 902, the query application receives an audio signal at an audio interface. The system may include a microphone or other audio detection device and may record an audio file based on the audio input to the device.

In step 904, a query application analyzes the audio signal of step 902 to identify speech. The query application may apply any suitable decimation, conditioning (e.g., amplification, filtering), processing (e.g., in the time or spectral domain), pattern recognition, algorithms, transformations, any other suitable actions, or any combination thereof. In some embodiments, the query application uses any suitable techniques to identify words, sounds, phrases, or combinations thereof.

In step 906, the query application determines whether a voice query has been received. In some embodiments, the query application determines that a voice query has been received based on parameters of the audio signal. For example, periods of no speech before and after the query may delimit the scope of the voice query in the recording. In some embodiments, the query application identifies keywords in the order in which they were spoken, applies sentence or query templates to the keywords, and extracts a text query. For example, the placement of nouns, proper nouns, verbs, adjectives, adverbs, and other parts of speech may provide indications of the beginning and end of a voice query. The query application may apply any suitable criteria in analyzing the audio signal to extract text. In step 908, the query application generates a text query based on the results of steps 904 and 906. In some embodiments, in step 908, the query application may store the text query in suitable storage (e.g., storage 408). If, in step 906, the query application determines that a voice query has not been received or otherwise that a text query cannot be generated based on the analyzed audio of step 904, the query application may return to step 902 and proceed with detecting audio until a voice query is received.

In step 910, a query application accesses a database for entity information. The query application uses the text query of step 908 to search among the information in the database. The query application may apply any suitable search algorithm to identify information, entities, or content in the database.

In step 912, the query application determines whether an entity in the database of step 910 matches the text query of step 908. The query application may identify and evaluate multiple entities to find a match. In some embodiments, the text query includes two or more entities, and the query application searches through the content to determine content items that have the entity associated in their metadata (e.g., by comparing the metadata tags of the content items with the text query). In some circumstances, the query application may be unable to identify a match, and in response may continue the search, search in another database, modify the text query (e.g., return to step 908 (not shown)), return to step 904 and modify the settings used in step 904 (not shown), return an indication that no search results were found, provide any other suitable response, or any combination thereof. In some embodiments, the query application may identify multiple entities, content, or both that match the text query. Step 914 includes the query application identifying content associated with the text query of step 908. In some embodiments, steps 914 and 910 may be reversed and the query application may search within the content based on a text query. In some embodiments, the entity may include a content identifier, and thus steps 910 and 914 may be combined.

At step 916, the query application retrieves the content associated with the text query of step 908. At step 916, for example, the query application may identify the content items, download the content items, stream the content items, generate the content items or a list of content items (e.g., or a list of links to the content items) for display, or a combination thereof.

The above-described embodiments of the present disclosure are presented for purposes of illustration, not limitation, and the present disclosure is limited only by the claims that follow. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and that the flow charts or examples related to one embodiment may be combined with any other embodiment in a suitable manner, performed in a different order, or performed in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and/or methods described above may be applied to or used in accordance with other systems and/or methods.
This specification discloses embodiments including, but not limited to, the following:
1. A method for responding to a voice query, the method comprising:
Receiving a voice query at an audio interface;
extracting one or more keywords from the voice query using control circuitry;
determining pronunciation information for one or more keywords using control circuitry;
generating a text query based on the one or more keywords and the pronunciation information using control circuitry;
identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
Retrieving a content item associated with an entity; and
A method comprising:
(Item 2) The method of item 1, wherein the pronunciation information comprises one phoneme of one or more keywords.
(Item 3) The method of item 1, wherein identifying the entity is further based on user profile information.
(Item 4) The method of item 3, wherein identifying an entity is based on a previously identified entity from a previous voice query.
(Item 5) The method of item 1, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 6) Identifying an entity includes:
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective pronunciation tags to the text query;
selecting an entity by determining a maximum score;
2. The method according to claim 1, comprising:
(Item 7) The method of item 1, wherein the entity is a first entity, and further includes identifying a second entity among the multiple entities based on the text query and second metadata related to the second entity, and the content item is associated with the first entity and the second entity.
(Item 8) The method of item 1, wherein identifying an entity among a plurality of entities in a database includes comparing at least a portion of the text query to tags of stored metadata and identifying a match.
(Item 9) The method of item 1, wherein a first keyword of the one or more keywords is associated with two or more pronunciations of the first keyword.
10. The method of claim 1, wherein the pronunciation information comprises a phonetic representation of a first keyword of the one or more keywords.
11. A system for responding to voice queries, comprising:
an audio interface for receiving a voice query;
a control circuit coupled to the audio interface;
The control circuit includes:
Extracting one or more keywords from the voice query;
determining and extracting pronunciation information for one or more keywords;
generating and extracting a text query based on one or more keywords and pronunciation information;
Identifying and extracting an entity from a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
and retrieving a content item associated with the entity.
(Item 12) The system of item 11, wherein the pronunciation information comprises a phoneme of one of the one or more keywords.
(Item 13) The system of item 11, wherein the control circuitry is further configured to identify the entity based on user profile information.
Item 14. The system of item 13, wherein the control circuitry is further configured to identify an entity based on previously identified entities from a previous voice query.
(Item 15) The system of item 11, wherein the control circuitry is further configured to identify the entity based on popularity information associated with the entity.
(Item 16) The control circuit includes:
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective pronunciation tags to the text query;
12. The system of claim 11, further configured to identify the entities by selecting the entity by determining a maximum score.
(Item 17) The system described in Item 11, wherein the entity is a first entity, the control circuit is further configured to identify a second entity among the multiple entities based on the text query and second metadata related to the second entity, and the content item is associated with the first entity and the second entity.
(Item 18) The control circuit is further configured to identify an entity among a plurality of entities in the database by comparing at least a portion of the text query with tags of stored metadata and identifying a match, as described in Item 11.
(Item 19) The system of item 11, wherein a first keyword of the one or more keywords is associated with two or more pronunciations of the first keyword.
20. The system of claim 11, wherein the pronunciation information comprises a phonetic representation of a first keyword of the one or more keywords.
21. A non-transitory computer readable medium having instructions encoded thereon, the instructions, when executed by a control circuit,
Receiving a voice query at an audio interface;
Extracting one or more keywords from the voice query;
determining pronunciation information for one or more keywords;
generating a text query based on one or more keywords and pronunciation information;
identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
and retrieving a content item associated with the entity.
22. The non-transitory computer-readable medium of claim 21, wherein the pronunciation information comprises a phoneme of one of the one or more keywords.
23. The non-transitory computer-readable medium of claim 21, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify an entity based on user profile information.
(Item 24) The non-transitory computer-readable medium of item 23, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify an entity based on previously identified entities from a previous voice query.
(Item 25) The non-transitory computer-readable medium of item 21, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify an entity based on popularity information associated with the entity.
26. The method of claim 25, further comprising: encoding instructions that, when executed by the control circuitry,
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective pronunciation tags to the text query;
22. The non-transitory computer readable medium of claim 21, wherein the control circuitry identifies the entities by: determining a maximum score; and selecting an entity.
(Item 27) The non-transitory computer-readable medium of item 21, wherein the entity is a first entity and further comprises encoded instructions which, when executed by the control circuit, cause the control circuit to identify a second entity among the plurality of entities based on the text query and second metadata related to the second entity, and the content item is associated with the first entity and the second entity.
(Item 28) The non-transitory computer-readable medium of item 21, further comprising encoded instructions which, when executed by the control circuit, cause the control circuit to identify an entity among a plurality of entities in the database by comparing at least a portion of the text query with tags of stored metadata and identifying a match.
(Item 29) The non-transitory computer-readable medium of item 21, wherein a first keyword of the one or more keywords is associated with two or more pronunciations of the first keyword.
30. The non-transitory computer-readable medium of claim 21, wherein the pronunciation information comprises a phonemic representation of a first keyword of the one or more keywords.
31. A system for responding to voice queries, comprising:
A means for receiving a voice query;
means for extracting one or more keywords from the voice query;
means for determining pronunciation information for one or more keywords;
means for generating a text query based on one or more keywords and pronunciation information;
means for identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
and means for retrieving a content item associated with the entity.
32. The system of claim 31, wherein the pronunciation information comprises a phoneme of one of the one or more keywords.
33. The system of claim 31, wherein the means for identifying an entity comprises means for identifying an entity based on user profile information.
34. The system of claim 33, wherein the means for identifying an entity comprises means for identifying an entity based on a previously identified entity from a previous voice query.
35. The system of claim 31, wherein the means for identifying an entity comprises means for identifying an entity based on popularity information associated with the entity.
(Item 36) The means for identifying an entity comprises:
means for identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
means for determining, for each respective entity of the plurality of entities, a respective score based on comparing the respective pronunciation tag to the text query;
and means for selecting an entity by determining a maximum score.
(Item 37) The system described in Item 31, wherein the entity is a first entity, and further comprising means for identifying a second entity among the multiple entities based on the text query and second metadata related to the second entity, and the content item is associated with the first entity and the second entity.
(Item 38) The system of item 31, wherein the means for identifying an entity among a plurality of entities in the database includes means for comparing at least a portion of the text query with tags of stored metadata and identifying a match.
(Item 39) The system of item 31, wherein a first keyword of the one or more keywords is associated with two or more pronunciations of the first keyword.
40. The system of claim 31, wherein the pronunciation information comprises a phonetic representation of a first keyword of the one or more keywords.
41. A method for responding to a voice query, the method comprising:
Receiving a voice query at an audio interface;
extracting one or more keywords from the voice query using control circuitry;
determining pronunciation information for one or more keywords using control circuitry;
generating a text query based on the one or more keywords and the pronunciation information using control circuitry;
identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
and retrieving a content item associated with the entity.
42. The method of claim 41, wherein the pronunciation information comprises a phoneme of one of the one or more keywords.
(Item 43) The method of any of Items 41-42, wherein identifying the entity is further based on user profile information.
44. The method of claim 41, wherein identifying an entity is based on a previously identified entity from a previous voice query.
(Item 45) The method of any of Items 41-44, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 46) Identifying an entity comprises:
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective pronunciation tags to the text query;
and selecting an entity by determining a maximum score.
(Item 47) A method according to any of Items 41-46, wherein the entity is a first entity, and further includes identifying a second entity among the plurality of entities based on the text query and second metadata relating to the second entity, and the content item is associated with the first entity and the second entity.
(Item 48) The method of any of Items 41-47, wherein identifying an entity among a plurality of entities in a database includes comparing at least a portion of the text query to stored metadata tags and identifying a match.
(Item 49) The method of any of Items 41-48, wherein a first keyword of the one or more keywords is associated with two or more pronunciations of the first keyword.
50. The method of any of claims 41-49, wherein the pronunciation information comprises a phonetic representation of a first keyword of the one or more keywords.
51. A method for responding to a voice query, comprising:
Receiving a voice query at an audio interface;
extracting one or more keywords from the voice query using control circuitry;
generating a text query based on the one or more keywords using control circuitry;
identifying an entity based on the text query and metadata about the entity, the metadata comprising one or more alternative text representations of the entity based on a pronunciation of an identifier associated with the entity;
and retrieving a content item associated with the entity.
52. The method of claim 51, wherein the one or more alternative text representations comprise a phonemic representation of the entity.
53. The method of claim 51, wherein the one or more alternative text representations comprise alternative spellings of the entity based on pronunciation.
(Item 54) The method of item 51, wherein one or more alternative text representations of an entity comprise a text string generated based on a previous speech-to-text conversion.
The one or more alternative text representations comprise a plurality of alternative text representations, each of the plurality of alternative text representations comprising:
Converting the first text representation into an audio file;
52. The method of claim 51, wherein the audio file is generated by converting the audio file into a second text representation, the second text representation being not identical to the first text representation.
(Item 56) The method of item 51, wherein identifying the entity is further based on user profile information.
57. The method of claim 51, wherein identifying an entity is further based on popularity information associated with the entity.
(Item 58) Identifying an entity comprises:
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective one or more alternative text representations to the text query;
and selecting an entity by determining a maximum score.
(Item 59) The method of item 51, further comprising generating a plurality of text queries, the plurality of text queries comprising a text query, each text query of the plurality of text queries being generated based on a respective setting of a speech-to-text module of the control circuit.
(Item 60)
identifying a respective entity based on each of the plurality of text queries;
determining a respective score for each entity based on a comparison of each text query to metadata associated with each entity;
and identifying the entity by selecting a maximum score of the respective scores.
61. A system for responding to voice queries, comprising:
an audio interface for receiving a voice query;
a control circuit;
The control circuit includes:
Extracting one or more keywords from the voice query;
generating a text query based on one or more keywords;
identifying an entity based on the text query and metadata about the entity, the metadata comprising one or more alternative text representations of the entity based on a pronunciation of an identifier associated with the entity;
and retrieving a content item associated with the entity.
(Item 62) The system of item 61, wherein the one or more alternative text representations comprise a phonetic representation of the entity.
(Item 63) The system of item 61, wherein the one or more alternative text representations comprise alternative spellings of the entity based on pronunciation.
(Item 64) The system of item 61, wherein one or more alternative text representations of an entity comprise a text string generated based on a previous speech-to-text conversion.
The one or more alternative text representations comprise a plurality of alternative text representations, and the control circuitry:
Converting the first text representation into an audio file;
62. The system of claim 61, configured to generate each of the multiple alternative text representations by converting the audio file to a second text representation, the second text representation not being identical to the first text representation.
(Item 66) The system of item 61, wherein the control circuit is further configured to identify the entity based on user profile information.
67. The system of claim 61, wherein the control circuitry is further configured to identify the entity based on popularity information associated with the entity.
(Item 68) The control circuit comprises:
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective one or more alternative text representations to the text query;
62. The system of claim 61, further configured to identify the entities by: determining a maximum score and selecting the entity.
(Item 69) The system described in Item 61, wherein the control circuit is further configured to generate a plurality of text queries, the plurality of text queries comprising a text query, the control circuit comprising a speech-to-text module, and each text query of the plurality of text queries is generated based on a respective setting of the speech-to-text module.
(Item 70) A control circuit comprising:
identifying a respective entity based on each of the plurality of text queries;
determining a respective score for each entity based on a comparison of each text query to metadata associated with each entity;
and identifying the entity by selecting a maximum score of the respective scores.
71. A non-transitory computer readable medium having instructions encoded thereon that, when executed by a control circuit,
Receiving a voice query at an audio interface;
Extracting one or more keywords from the voice query;
generating a text query based on one or more keywords;
identifying an entity based on the text query and metadata about the entity, the metadata comprising one or more alternative text representations of the entity based on a pronunciation of an identifier associated with the entity;
and retrieving a content item associated with the entity.
72. The non-transitory computer-readable medium of claim 71, wherein the one or more alternative text representations comprise a phonemic representation of the entity.
73. The non-transitory computer-readable medium of claim 71, wherein the one or more alternative text representations comprise alternative spellings of the entity based on pronunciation.
(Item 74) The non-transitory computer-readable medium of item 71, wherein one or more alternative text representations of an entity comprise a text string generated based on a previous speech-to-text conversion.
7. The one or more alternative text representations comprise a plurality of alternative text representations, further comprising encoded instructions that, when executed by the control circuitry, cause the control circuitry to:
Converting the first text representation into an audio file;
72. The non-transitory computer-readable medium of claim 71, generating each alternative text representation of the plurality of alternative text representations by converting the audio file to a second text representation, the second text representation not being identical to the first text representation.
(Item 76) The non-transitory computer-readable medium of item 71, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify an entity based on user profile information.
(Item 77) The non-transitory computer-readable medium of item 71, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify an entity based on popularity information associated with the entity.
7. The method of claim 7, further comprising: encoding instructions that, when executed by the control circuitry,
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective one or more alternative text representations to the text query;
selecting an entity by determining a maximum score;
72. The non-transitory computer readable medium of claim 71, wherein the control circuit identifies an entity by
(Item 79) The non-transitory computer-readable medium of item 71, further comprising encoded instructions which, when executed by the control circuit, cause the control circuit to generate a plurality of text queries, the plurality of text queries comprising a text query, each text query of the plurality of text queries being generated based on a respective setting of a speech-to-text module of the control circuit.
80. The method of claim 80, further comprising: encoding instructions that, when executed by the control circuitry,
identifying a respective entity based on each of the plurality of text queries;
determining a respective score for each entity based on a comparison of each text query to metadata associated with each entity;
and identifying the entity by selecting a maximum score from the respective scores.
81. A system for responding to voice queries, comprising:
Means for receiving a voice query at an audio interface;
means for extracting one or more keywords from the voice query;
means for generating a text query based on one or more keywords;
means for identifying an entity based on a text query and metadata about the entity, the metadata comprising one or more alternative text representations of the entity based on a pronunciation of an identifier associated with the entity;
and means for retrieving a content item associated with the entity.
(Item 82) The system of item 81, wherein the one or more alternative text representations comprise a phonetic representation of the entity.
(Item 83) The system of item 81, wherein the one or more alternative text representations comprise alternative spellings of the entity based on pronunciation.
(Item 84) The system of item 81, wherein one or more alternative text representations of an entity comprise a text string generated based on a previous speech-to-text conversion.
The one or more alternative text representations comprise a plurality of alternative text representations, each of the plurality of alternative text representations comprising:
means for converting the first text representation into an audio file;
82. The system of claim 81, wherein the audio file is generated by a means for converting the audio file into a second text representation, the second text representation being not identical to the first text representation.
(Item 86) The system of item 81, wherein the means for identifying an entity further comprises means for identifying an entity based on user profile information.
87. The system of claim 81, wherein the means for identifying an entity further comprises means for identifying an entity based on popularity information associated with the entity.
(Item 88) The means for identifying an entity includes:
means for identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
means for determining, for each respective entity of the plurality of entities based on comparing each of the one or more alternative text representations to the text query;
and means for selecting an entity by determining a maximum score.
(Item 89) The system of item 81, further comprising means for generating a plurality of text queries, the plurality of text queries comprising a text query, each text query of the plurality of text queries being generated based on a respective setting of a speech-to-text module of the control circuit.
(Item 90)
means for identifying respective entities based on respective text queries of the plurality of text queries;
means for determining a respective score for each entity based on a comparison of each text query to metadata associated with each entity;
and means for identifying the entity by selecting a maximum score of the respective scores.
91. A method for responding to a voice query, comprising:
Receiving a voice query at an audio interface;
extracting one or more keywords from the voice query using control circuitry;
generating a text query based on the one or more keywords using control circuitry;
identifying an entity based on the text query and metadata about the entity, the metadata comprising one or more alternative text representations of the entity based on a pronunciation of an identifier associated with the entity;
and retrieving a content item associated with the entity.
(Item 92) The method of item 91, wherein the one or more alternative text representations comprise a phonemic representation of the entity.
(Item 93) The method of any of Items 91-92, wherein the one or more alternative text representations comprise alternative spellings of the entity based on pronunciation.
(Item 94) The method of any of Items 91-93, wherein one or more alternative text representations of an entity comprise a text string generated based on a previous speech-to-text conversion.
The one or more alternative text representations comprise a plurality of alternative text representations, each of the plurality of alternative text representations comprising:
Converting the first text representation into an audio file;
95. The method according to any of claims 91-94, wherein the audio file is generated by converting the audio file into a second text representation, the second text representation being not identical to the first text representation.
(Item 96) A method according to any of Items 91-95, wherein identifying the entity is further based on user profile information.
(Item 97) The method of any of Items 91-96, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 98) Identifying an entity comprises:
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining, for each respective entity of the plurality of entities based on comparing the respective one or more alternative text representations to the text query;
and selecting an entity by determining a maximum score.
(Item 99) A method according to any of Items 91-98, further comprising generating a plurality of text queries, the plurality of text queries comprising a text query, each text query of the plurality of text queries being generated based on respective settings of a speech-to-text module of the control circuit.
(Item 100)
identifying a respective entity based on each of the plurality of text queries;
determining a respective score for each entity based on a comparison of each text query to metadata associated with each entity;
and identifying the entity by selecting a maximum score of the respective scores.
101. A method for generating entity metadata for a voice query, the method comprising:
identifying an entity of a plurality of entities in which information is stored;
generating an audio file based on a first text string and at least one speech criterion using a text-to-speech module, the first text string describing an entity;
generating a second text string based on the audio file using a speech-to-text module;
comparing the second text string to the first text string;
and if not identical to the first text string, storing the second text string in metadata associated with the entity.
(Item 102) The method of item 101, wherein at least one speech criterion comprises a pronunciation setting.
(Item 103) The method of item 101, wherein at least one speech criterion comprises a language setting.
The at least one speech criterion comprises a plurality of speech criteria, and the method includes:
generating respective audio files based on the first text string and respective speech criteria using a text-to-speech module;
generating a respective second text string based on the respective audio file using a speech-to-text module;
comparing each second text string to the first text string;
and storing the respective second text string in metadata associated with the entity if the respective second text string is not identical to the first text string.
(Item 105) The method of item 101, further comprising updating metadata based on one or more text queries.
(Item 106) The method of item 101, further comprising storing a phonetic representation of the first text string in metadata associated with the entity.
Item 107. Generating an audio file based on a first text string includes:
Converting a first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
Detecting speech using a microphone and generating a second audio signal;
Item 102. The method of item 101, comprising processing the audio signal to generate an audio file.
(Item 108) The method of item 107, wherein generating speech in the speaker is further based on at least one speech setting of the text-to-speech module.
Generating a second text string based on an audio file includes:
generating a playback of the audio file on a speaker;
detecting the playback using a microphone and generating an audio signal;
and converting the audio signal into a second text string by identifying one or more words.
(Item 110) The method of item 109, wherein converting the audio signal into a second text string is based on at least one text setting of a speech-to-text module.
11. A system for generating entity metadata for a voice query, the system comprising: a control circuit;
The control circuit includes:
identifying an entity of a plurality of entities in which information is stored;
generating, using an audio interface coupled to the control circuitry, an audio file based on a first text string and at least one speech criterion, the first text string describing an entity;
generating a second text string based on the audio file using the audio interface;
comparing the second text string to the first text string;
and if not identical to the first text string, storing the second text string in metadata associated with the entity.
(Item 112) The system of item 111, wherein at least one speech criterion comprises a pronunciation setting.
(Item 113) The system of item 111, wherein at least one speech criterion comprises a language setting.
(Item 114) The at least one speech criterion comprises a plurality of speech criteria, and the control circuitry:
generating, using an audio device, respective audio files based on the first text string and the respective speech criteria;
generating, using an audio device, respective second text strings based on the respective audio files;
comparing each second text string to the first text string;
and storing the respective second text string in metadata associated with the entity if the respective second text string is not identical to the first text string.
(Item 115) The system of item 111, wherein the control circuit is further configured to update the metadata based on one or more text queries.
(Item 116) The system of item 111, wherein the control circuitry is further configured to store a phonetic representation of the first text string in metadata associated with the entity.
(Item 117) An audio device includes a speaker and a microphone, and a control circuit includes:
Converting a first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
Detecting speech using a microphone and generating a second audio signal;
112. The system of claim 111, further configured to generate an audio file based on the first text string by processing the audio signal and generating the audio file.
(Item 118) The system of item 117, wherein the control circuit is further configured to generate speech at the speaker based on at least one speech setting.
(Item 119) An audio device includes a speaker and a microphone, and a control circuit includes:
generating a playback of the audio file on a speaker;
detecting the playback at a microphone and generating an audio signal;
112. The system of claim 111, further configured to generate a second text string based on the audio file by identifying one or more words and converting the audio signal into a second text string.
(Item 120) The system of item 119, wherein the control circuit is further configured to convert the audio signal into a second text string based on at least one text setting of the speech-to-text module.
121. A non-transitory computer-readable medium having instructions encoded thereon, the instructions, when executed by a control circuit,
identifying an entity of a plurality of entities in which information is stored;
generating an audio file based on a first text string and at least one speech criterion, the first text string describing an entity;
generating a second text string based on the audio file;
comparing the second text string to the first text string;
and if the second text string is not identical to the first text string, storing the second text string in metadata associated with the entity.
(Item 122) The non-transitory computer-readable medium of item 121, wherein at least one speech criterion comprises a pronunciation setting.
(Item 123) The non-transitory computer-readable medium of item 121, wherein at least one speech criterion comprises a language setting.
12. The at least one speech criterion comprises a plurality of speech criteria, and further comprises encoded instructions that, when executed by the control circuitry,
generating respective audio files based on the first text string and the respective speech criteria;
generating a respective second text string based on the respective audio file;
comparing each second text string to the first text string;
and if the second text string is not identical to the first text string, storing the respective second text string in metadata associated with the entity.
(Item 125) The non-transitory computer-readable medium of item 121, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to update metadata based on one or more text queries.
(Item 126) The non-transitory computer-readable medium of item 121, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to store a phonetic representation of the first text string in metadata associated with the entity.
12. The method of claim 11, further comprising: encoding instructions that, when executed by the control circuitry,
Converting a first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
Detecting speech using a microphone and generating a second audio signal;
Item 122. The non-transitory computer readable medium of item 121, further comprising a control circuit for processing an audio signal and generating an audio file.
(Item 128) The non-transitory computer-readable medium of item 127, further comprising encoded instructions which, when executed by the control circuit, cause the control circuit to generate speech at a speaker based on at least one speech setting of the text-to-speech module.
129. The method of claim 128, further comprising: encoding instructions that, when executed by the control circuitry,
generating a playback of the audio file on a speaker;
detecting the playback using a microphone and generating an audio signal;
and converting the audio signal into a second text string by identifying one or more words.
(Item 130) The non-transitory computer-readable medium of item 129, further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to convert the audio signal into a second text string based on at least one text setting of the speech-to-text module.
131. A system for generating entity metadata for a voice query, the system comprising:
means for identifying an entity of a plurality of entities for which information is stored;
means for generating an audio file based on a first text string and at least one speech criterion, the first text string describing an entity; and
means for generating a second text string based on the audio file;
means for comparing the second text string to the first text string;
and means for storing the second text string in metadata associated with the entity if the second text string is not identical to the first text string.
(Item 132) The system of item 131, wherein at least one speech criterion comprises a pronunciation setting.
(Item 133) The system described in Item 131, wherein at least one speech criterion comprises a language setting.
(Item 134) The at least one speech criterion comprises a plurality of speech criteria, and the system:
means for generating respective audio files based on the first text string and the respective speech criteria;
means for generating respective second text strings based on the respective audio files;
means for comparing each second text string to the first text string;
and means for storing the respective second text string in metadata associated with the entity if the respective second text string is not identical to the first text string.
(Item 135) The system of item 131, further comprising means for updating metadata based on one or more text queries.
136. The system of claim 131, further comprising means for storing a phonetic representation of the first text string in metadata associated with the entity.
137. The means for generating an audio file based on a first text string comprises:
means for converting a first text string into a first audio signal;
means for generating speech at a speaker based on the audio signal;
means for detecting speech using a microphone and generating a second audio signal;
and means for processing the audio signal and generating an audio file.
(Item 138) The system of item 137, wherein the means for generating speech at a speaker further comprises means for generating speech at a speaker based on at least one speech setting of the text-to-speech module.
139. The means for generating a second text string based on an audio file includes:
means for generating a playback of the audio file on a speaker;
means for detecting the playback using a microphone and generating an audio signal;
and means for converting the audio signal into a second text string by identifying one or more words.
(Item 140) The system described in Item 139, wherein the means for converting the audio signal into a second text string includes means for converting the audio signal into a second text string based on at least one text setting of the speech-to-text module.
141. A method for generating entity metadata for a voice query, the method comprising:
identifying an entity of a plurality of entities in which information is stored;
generating an audio file based on a first text string and at least one speech criterion using a text-to-speech module, the first text string describing an entity;
generating a second text string based on the audio file using a speech-to-text module;
comparing the second text string to the first text string;
and if not identical to the first text string, storing the second text string in metadata associated with the entity.
(Item 142) The method of item 141, wherein at least one speech criterion comprises a pronunciation setting.
(Item 143) The method of any of Items 141-142, wherein at least one speech criterion comprises a language setting.
The at least one speech criterion comprises a plurality of speech criteria, and the method further comprises:
generating respective audio files based on the first text string and respective speech criteria using a text-to-speech module;
generating a respective second text string based on the respective audio file using a speech-to-text module;
comparing each second text string to the first text string;
Item 144. The method of any of items 141-143, further comprising: storing the respective second text string in metadata associated with the entity if it is not identical to the first text string.
(Item 145) The method of any of Items 141-144, further comprising updating metadata based on one or more text queries.
146. The method of claim 141, further comprising storing a phonetic representation of the first text string in metadata associated with the entity.
14. Generating an audio file based on a first text string includes:
Converting a first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
Detecting speech using a microphone and generating a second audio signal;
Processing the audio signal to generate an audio file.
(Item 148) The method of item 147, wherein generating speech in the speaker is further based on at least one speech setting of the text-to-speech module.
Item 149. Generating a second text string based on an audio file includes:
generating a playback of the audio file on a speaker;
detecting the playback using a microphone and generating an audio signal;
and converting the audio signal into a second text string by identifying one or more words.
(Item 150) The method of item 149, wherein converting the audio signal into a second text string is based on at least one text setting of a speech-to-text module.

Claims (10)

1. A method of responding to a voice query for execution by a computing system , the method comprising:
receiving a voice query at an audio interface , the computing system ;
extracting one or more keywords from the voice query using control circuitry ;
generating a text query based on the one or more keywords , using the control circuitry;
the computing system identifying an entity, the identifying the entity based on the text query and metadata about the entity, the metadata comprising one or more alternative text representations of the entity, the one or more alternative text representations based on a pronunciation of an identifier associated with the entity;
the computing system retrieving a content item associated with the entity. The method of claim 1, wherein the one or more alternative textual representations comprise a phonemic representation of the entity. The method of any of claims 1 and 2, wherein the one or more alternative textual representations comprise alternative spellings of the entity based on pronunciation. The method of any of claims 1 to 3, wherein the one or more alternative text representations of the entity comprise text strings generated based on a previous speech-to-text transformation. The one or more alternative text representations comprise a plurality of alternative text representations, each alternative text representation in the plurality of alternative text representations:
said computing system converting the first textual representation into an audio file;
the computing system converting the audio file into a second text representation;
The method of any of claims 1 to 4, wherein the second textual representation is not identical to the first textual representation. The method according to any of claims 1 to 5, wherein said computing system's identifying said entities is further based on user profile information or on popularity information associated with said entities. The computing system identifying the entity comprises:
the computing system identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining , by the computing system, a respective score for each entity of the plurality of entities based on comparing the respective one or more alternative textual representations to the text query;
The method according to any of claims 1 to 6, comprising: the computing system selecting the entity by determining a maximum score. 8. The method of claim 1, further comprising: the computing system generating a plurality of text queries, the plurality of text queries comprising the text query, each text query of the plurality of text queries being generated based on a respective setting of a speech-to-text module of the control circuitry. identifying , by the computing system, a respective entity based on each of the plurality of text queries;
determining , by the computing system, a respective score for each of the entities based on a comparison of the respective text query to metadata associated with the respective entities;
The method of claim 8 , further comprising: the computing system identifying the entity by selecting a maximum score of the respective scores. 1. A system for responding to a voice query, the system comprising:
Memory,
and means for implementing the steps of the method according to any one of claims 1 to 9.