JP7343588B2 - System and method for customizing and compositing video feeds on client devices - Google Patents

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62/819,428, “SYSTEMS AND METHODS FOR COMPOSITING A VIDEO FEED AT A CLIENT DEVICE,” filed March 15, 2019.

This application is a continuation in part of U.S. patent application Ser. Patent application filed in 2019 Claiming priority based on U.S. Provisional Patent Application No. 62/795,020 “SYSTEMS AND METHODS FOR PARTITIONING A VIDEO FEED TO SEGMENT LIVE PLAYER ACTIVITY” filed on January 22, 2019, February 2019 US filing on the 6th Claims priority under Provisional Patent Application No. 62/802,186 ``SYSTEMS AND METHODS FOR PARTITIONING A VIDEO FEED TO SEGMENT LIVE PLAYER ACTIVITY.'' The above-mentioned applications are herein incorporated by reference for all purposes.

News broadcasts and videos of sporting events (such as American football and soccer) are some of the most popular forms of entertainment. Streaming video feeds on user devices is a popular method. Traditional video streams utilize multiple layers of data that are compiled remotely and communicated around the world. For example, a broadcast room receives and compiles a video feed, an audio feed, and graphics to create a compiled video feed, which is then communicated to a remote device. Compiled video feeds consume large amounts of data because the audio and graphics cannot be separated from the video portion of the feed. Additionally, instead of optionally receiving a portion of the video feed, the end user may choose to either receive the compiled video feed or not receive the video feed at all; Unable to personalize video feed.

Techniques (such as systems, processors, and computer program products) are disclosed for providing customized synthetic video feeds. In various embodiments, a process for customizing and synthesizing a video feed includes: receiving a background video feed from a remote server; receiving content associated with one or more user-specific characteristics; determining one or more data elements based at least in part on the content. The processing includes by matching at least corresponding portions of the one or more data elements to corresponding portions of the background video feed to create a composite video feed customized to the one or more user-specific characteristics. generating and displaying a composite video feed on a display device of a client device.

In various embodiments, a process for customizing and compositing a video feed receives a background video feed from a remote server and extracts a first frame from timestamped metadata associated with a first frame of the background video feed. determining a first timestamp of a frame and receiving content associated with one or more user-specific characteristics. The content includes time-stamped metadata associated with the background video frame and data having a timestamp within a time range determined by the first timestamp. The processing comprises determining one or more data elements based at least in part on the received content. Processing includes determining at least a corresponding portion of the one or more data elements from a corresponding background video feed based at least in part on a match between a timestamp of the respective data element and a timestamp of the respective background video frame. The method includes generating a composite video feed customized to one or more user-specific characteristics, including by matching the portions. The processing comprises displaying the composite video feed on a display device of the client device. The generated composite video feed includes time-aligned frames in which each time-aligned frame is overlaid with a data element that is time-coincident with a respective background video frame. Videos customized and compiled using the techniques disclosed herein are personalized to the end user.

Various embodiments of the invention are disclosed in the following detailed description and accompanying drawings.

FIG. 1 is a block diagram illustrating one embodiment of a system for providing customized synthetic video feeds.

2 is a flowchart illustrating one embodiment of a process for customizing and compositing video feeds.

2 is a flowchart illustrating one embodiment of a process for customizing and compositing video feeds.

FIG. 2 is a diagram illustrating an example of a customized and synthesized news broadcast on a client device.

FIG. 2 is a diagram illustrating an example of a customized and synthesized news broadcast on a client device.

Diagram illustrating an example of a sports event customized and synthesized on a client device.

Diagram illustrating an example of a sports event customized and synthesized on a client device.

Diagram illustrating an example of a sports event customized and synthesized on a client device.

FIG. 1 is a block diagram illustrating one embodiment of a system for customizing and compositing video feeds.

FIG. 1 is a block diagram illustrating one embodiment of a system for customizing and compositing video feeds.

FIG. 1 is a block diagram illustrating one embodiment of a system for customizing and compositing video feeds.

1 is a block diagram illustrating one embodiment of a tracking device. FIG.

FIG. 1 is a block diagram illustrating one embodiment of a tracking device management system.

FIG. 1 is a block diagram illustrating an embodiment of a statistical system.

FIG. 1 is a block diagram illustrating an embodiment of an odds management system.

FIG. 1 is a block diagram illustrating one embodiment of a user device.

1 is a diagram illustrating an example environment including a playing field that includes components of a tracking system, according to an embodiment of the present disclosure. FIG.

FIG. 2 is a functional diagram illustrating a computer system programmed to customize and synthesize video feeds in accordance with some embodiments.

2 is a flowchart illustrating one embodiment of a process for customizing and compositing video feeds for live sports.

Diagram showing an example of a composite video feed customized for an American football game.

Diagram showing an example of a composite video feed customized for an American football game.

The present invention relates to a process, an apparatus, a system, a composition of matter, a computer program product embodied on a computer readable storage medium, and/or a processor (stored in and/or by a memory coupled to a processor). It may be implemented in a variety of forms, including a processor configured to execute the provided instructions. These implementations, or any other forms that the invention may take, are referred to herein as techniques. In general, the order of steps in the disclosed processes may be varied within the scope of the invention. Unless otherwise noted, a component such as a processor or memory described as being configured to perform a task may be referred to as a general component temporarily configured to perform a task at a given time; It may be implemented as a specific component manufactured to perform a task. As used herein, the term "processor" shall refer to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention is provided below with reference to drawings that illustrate the principles of the invention. Although the invention has been described in connection with such embodiments, it is not limited to any one embodiment. The scope of the invention is limited only by the claims, and the invention includes many alternatives, modifications, and equivalents. In the following description, many specific details are set forth in order to provide a thorough understanding of the invention. These details are for purposes of illustration, and the invention may be practiced according to the claims without some or all of these specific details. For the sake of brevity, technical matters well known in the art related to the present invention have not been described in detail so as not to unnecessarily obscure the present invention.

FIG. 1 is a block diagram illustrating one embodiment of a system for providing customized composite video feeds. The example system includes one or more client devices 150, a server 100, and one or more content creators 140. Each client device and contact creator is configured to communicate with server 100 via a communications interface (not shown).

One or more user devices 150 facilitate end user interaction with the various systems of this disclosure. The user device may be any type of computing device including, but not limited to, a smartphone, tablet, OTT box, desktop computer, or smart TV. An example of a user device is shown in FIG. End users interact with other system components by sending requests to server 100 to view video content, such as news broadcasts, live sporting events, or television programming. The server responds with the requested content, and portions of the video feed may be customized to the particular user's preferences using techniques detailed below. In various embodiments, the server 100 stores a user profile 102 that includes user-specified interests (e.g., during registration), user habits such as browsing history, and well-known machine learning. recording user preferences, which may be based on other user characteristics, which may be learned over time using various techniques, such as The server 100 also stores content that can be pushed to the client device periodically or sent to the client device in response to a query for data. Background content 104 includes content created or captured by content creators 140 (such as background video feeds), as described in more detail below. Further content 109 includes data elements (such as interactive data elements) that can be used to enhance the background video feed. Data elements may be extracted from a background video feed (e.g., to isolate or highlight scenes of a particular player on an American football team), generated (e.g., game/player statistics), or May be created in combination.

As an example, assume that user A who watches a news broadcast is interested in the live stock price of company X. A newscast is presented within User A's device and customized to display Company A's stock price. Assume that user B is not interested in company X but is interested in company Y. In User B's device, a newscast is presented and customized to display Company Y's stock price. As another example, suppose user A is watching a live American football game and is interested in player U. An American football game is presented within User A's device and customized to emphasize Player U, for example, by displaying footage of the game focusing on Player U's movements on the field. User B is not interested in player U, but is interested in player V. In User B's device, an American football game is presented and customized to highlight Player V.

One or more contact creators 140 create content (video, audio, text, images, etc.) that is distributed to client devices 150. As non-limiting examples, content creators create news broadcasts such as financial news, television programs in various fields such as reality television, and sporting events such as live sports broadcasts. For example, one or more cameras 140-1 through 140-S capture images and/or video of an event (which may be a live event) that is later utilized by the system of the present disclosure. In some embodiments, camera 140 includes one or more high resolution cameras. As a non-limiting example, the one or more high resolution cameras include cameras with 1080p resolution to 8K resolution.

The components described above are interconnected, optionally through a communications network. Elements within dashed boxes are optionally combined into a single system or device. Of course, other topologies of computer systems are also possible. For example, in some embodiments, any of the illustrated devices and systems may actually constitute several computer systems that are linked in a network or in a cloud computing environment. It may be a virtual machine or a container. Further, in some embodiments, rather than relying on a physical communication network, the illustrated devices and systems wirelessly transmit information between each other. Accordingly, the example topology shown in FIG. 1 is merely to describe the features of one embodiment of the present disclosure as will be readily understood by those skilled in the art.

Having described the general topology of the system, methods for compositing and customizing video feeds (eg, video presentations) at client devices are described below.

FIG. 2 is a flowchart illustrating one embodiment of a process for customizing and compositing video feeds. The processing may be performed by a client device such as user device 1 (150-1) or user device R (150-R) shown in FIG.

Processing begins with receiving a background video feed (step 202). The background video feed may be any type of content including, but not limited to, live events, news broadcasts, or advertisements. A video feed of a live event may be received while the live event is occurring, or a video feed of that event may be received after the live event has occurred, such as a recording of the live event (e.g., the video feed of the live event is is video on demand (VOD)). The news broadcast example is further described with respect to FIGS. 4A and 4B. The example of a live sporting event is further described with respect to FIGS. 5A-5C.

In some embodiments, the video feed includes an audio portion (eg, audio captured by the respective broadcast device 140). Thus, in some embodiments, the video feed received by each client device includes a composite of frames and audio (e.g., the video feed received by the client device includes an audio portion and an image portion). . However, the present disclosure is not limited thereto. For example, in some embodiments, each client device receives a video feed (eg, frames) and an audio feed (eg, audio segments or portions) as separate feeds. Separating the audio portion of a video feed before receiving the feed on each user device allows the end user at the user device to separate the audio portion of the composite video feed and/or closed captioning, depending on that user's user preference settings. Allows you to customize, enable or disable parts. For example, audio tracks (different languages, background music, narration, etc.) can be tailored to user characteristics. The background video feed may be received together with the content received at step 204, or may be received separately from (unrelated to) the content received at step 204.

The process receives content associated with one or more user-specific characteristics (step 204). A client device receives content from a remote server. Customization may be performed by the server and/or client device. In some embodiments, the client device determines what content to request based on user-specific characteristics. For example, a client device looks up content that is locally associated with a user's preferences and sends a request for interesting content to a server. In some embodiments, it is the server that decides what content to send based on user-specific characteristics. For example, a client may send a user identifier (such as a username or demographic) to a server, and the server may utilize a lookup table or other database that maps user characteristics to content of interest to that particular user. The user identifier is then used to look up content that is of interest to the user.

In various embodiments, content is pushed by the server to the client device at intervals determined by the server and/or user/client device settings. In various embodiments, content is pulled from a server by a client device using a content request. For example, a client device uses a communication interface to request content from a remote server. In various embodiments, the request is made automatically on behalf of the user when the user begins using the user device (or a client application on the user device). For example, when a user logs into a fantasy sports app, TV program viewing app, etc., a request for content is relayed to a server. In various embodiments, the request is initiated by a user via a user interface on a client device. For example, while watching a television program using a client device, a user may select an object of interest (such as a player or team at a sporting event, the ticker symbol of a company's stock, or a character on a reality TV show). Generate requests for content (additional information) about your subject of interest.

In some embodiments, the plurality of data elements received from the computer system include the plurality of data elements requested by the query. However, the present disclosure is not limited thereto. For example, in some embodiments, a first plurality of data elements is requested by a query, and accordingly, a second plurality of data elements are received from the computer system (e.g., only some data elements are available). Nevertheless, as detailed below, in some embodiments each data element received in response to a query includes associated timestamped metadata that is within (e.g., includes) a particular time range. (For example, a time range is from 1:04 PM to 1:06 PM on a particular day, and accordingly, the data elements received as a result of a query are from 1:04 PM to 1:06 PM on that particular day.) (including metadata associated with the time in between). In some embodiments, the plurality of data elements are received from one or more data stores (e.g., historical training data store 214, context store 228, etc.) of the system shown in FIG. 1, FIG. 6, or FIG. Ru. In some embodiments, the plurality of data elements are stored locally on the user device. In some embodiments, the plurality of data elements includes one or more data elements stored locally on the user device and on the computer system shown in FIG. 1, FIG. 6, or FIG. one or more data elements stored in a data store 214, a context store 228, etc.).

Processing determines one or more data elements based at least in part on the received content (step 206). The data element may be determined based on client device preference settings (eg, user preference settings). For example, if user preference settings indicate that the user is interested in the weather, weather data elements are included in the composite video feed. The data elements may be interactive, allowing the user to view additional content customized to the user's preferences by selecting the data element. Thus, the query includes a request for weather data elements to be overlaid with frames of the video feed. User preference settings may be stored within the user device or on a remote server.

User preference settings may be determined in a variety of ways. For example, a machine learning process determines trends in particular settings by analyzing user activity over a period of time. As another example, user interaction with affordance areas of graphical data elements overlaid on a video feed suggests user preferences. A user can interact with a graphic data element (e.g., an embedded graphic) (such as by touching a scrolling ticker on a video feed) to obtain further information about the ticker or change the state from visible state to OFF state. Having query request data elements according to user preference settings allows the composite video feed to be personalized for each user. This query also optimizes the amount of data received by each user device, since data elements that the user suggests that they exclude (ie, do not request) are not received by the user devices. Additionally, it allows end users to further personalize their viewing experience by allowing them to interact with the synthetic video feed and change which data elements are overlaid on the feed. do.

Processing generates a composite video feed customized to user-specific characteristics, such as by matching at least corresponding portions of the one or more data elements to corresponding portions of the background video feed (step 208). . Data elements may be aligned to corresponding portions of the background video feed, either asynchronously or time-synchronously, as detailed below.

The process displays the composite video feed on the display device of the client device (step 209). The composite video feed is displayed on a client device's user interface in conjunction (e.g., overlaid) with a background video feed that is broadcast to a general audience to display information tailored to the interests of a particular user. can be output.

In some embodiments, the communication (receiving steps 202 and 206 and requesting step 204) is performed using a first exchange protocol, which receives a portion of the background video feed and includes data elements and/or or provide a communication system for assembling portions of video feeds to form a composite video feed (eg, a video presentation). In some embodiments, the first exchange protocol is a Hypertext Transfer Protocol (HTTP) live streaming protocol (eg, HLS protocol). In some embodiments, the first exchange protocol is Real Time Messaging Protocol (RTMP). In some embodiments, the first exchange protocol delays reception of the video feed (eg, introduces latency into reception) by a dynamic period of time (eg, a period from 10 seconds to 60 seconds).

In some embodiments, the background video feed is received continuously (e.g., each client device continuously communicates with the computer system shown in FIG. 1 to receive the video feed). . In some embodiments, the video feed is received by each user device in packets or blocks (eg, each user device receives the video feed in 10 second blocks). In various embodiments, time-synchronized alignment is performed as detailed with respect to FIG.

In some embodiments, receiving a video feed, sending a content request, or a combination thereof is performed according to a client device's bandwidth determination. For example, in some embodiments, the client device's bandwidth is dynamic and changes depending on multiple factors (eg, network bandwidth, broadcast system bandwidth, etc.).

In some embodiments, the method further comprises sizing the visualization buffer (eg, visualization buffer module) using the bandwidth of the client device (eg, user device 150). In some embodiments, sizing the visualization buffer using the client device's bandwidth increases the size of the visualization buffer in response to the determination of reduced bandwidth. Furthermore, determining the size of the visualization buffer using the client device's bandwidth reduces the size of the visualization buffer in response to determining the bandwidth improvement. This visualization buffer sizing allows a certain number of time-aligned frames to be dynamically posted to the visualization buffer. For example, in some embodiments, reducing the number of time-aligned frames posted to the visualization buffer reduces load on the client device.

In some embodiments, receiving a video feed, sending a query, or a combination thereof is performed according to a client device's bandwidth determination. For example, in some embodiments, the client device's bandwidth is dynamic and changes depending on multiple factors (eg, network 106 bandwidth, broadcast system 150 bandwidth, etc.).

FIG. 3 is a flowchart illustrating one embodiment of a process for customizing and compositing video feeds. Processing may be performed by a server (such as server 100 shown in FIG. 1).

Processing begins with receiving a request for content associated with one or more user-specific characteristics (step 352). The server uses the communication interface to send and receive data to remote client devices.

The process determines content based on the request, such as by looking up content associated with the request (step 354). Content may include data and/or metadata, where the data may be directly output or post-processed by the client for output in a composite video feed. As non-limiting examples, data includes graphics, text, or audio. Metadata defines how data is output within the composite video feed, such as visual effects (such as alpha glow), speed, direction, refresh rate, termination conditions, and interaction rules.

Processes can record requests or user preferences by using profiles. For example, if a request is accompanied by user-specific characteristics, the characteristics are stored in a user profile that records the particular user's interests and/or behavior. As discussed above, in some embodiments, the server uses the user identifier sent by the client device to identify the user by utilizing a lookup table or other database that maps user characteristics to content of interest to a particular user. to look up content that is of interest to the user.

The process transmits the determined content to the remote client device in response to the content request (step 356). Content may be transmitted via any type of communication interface or network.

Processes for customizing and compositing video feeds are applied in a variety of settings. The figure below shows an example of a news broadcast.

FIG. 4A shows an example of a news broadcast customized and synthesized on a client device. In this example, the customized data element is scrolling text being displayed in a news broadcast. Background video 403 is a video of a news announcer reading a news bulletin about global public health developments. News broadcasts may be pre-recorded or live, and techniques for customization are applicable in either situation. As shown in the figure, the background video has various parts, including an inset frame ("Breaking News") that can display additional video (such as content from a reporter in the field interviewing various people). have

A lower text scroll 407 displays the current stock price. Breaking news has an impact on the stock prices of various companies, lowering the stock prices of cruise lines (CRUS) and airlines (WINGS), while raising the stock prices of telework companies (TLWK) and cleaning products companies (SANI). Users who watch newscasts may be particularly interested in some of those companies, for example, if they have invested in TLWK (as indicated by data element 405).

Text scroll 407 is an example data element that is an example of one or more data elements aligned with a portion of a background video feed. Together, background video 403 and text scrolling 407 form a composite video. The user can interact with this video to obtain further information of interest. For example, a user can select data element 405 to display additional video or information about TLWK, as shown in the following figure.

Video timestamp instantiates text scrolling (crawling). Data for crawling is periodically requested from the server based on personal user profiles and video timestamps, downloaded to the client, and then rendered locally. Formatting for crawling may be included as markup. The crawling behavior (speed, direction, interaction rules, refresh period, termination conditions (what causes the crawl to end) can be included in the download as various code commands.

FIG. 4B shows an example of a customized and synthesized news broadcast on a client device. In this example, the customized data element is a branched video that displays detailed reports on subjects of interest to the user. A background video feed or a synthetic video feed may be updated in response to user interaction with one or more data elements (eg, a particular stock ticker) of the synthetic video feed. FIG. 4B shows the state of the user interface after the user selects the data element (module) 405. Modules (such as frame elements) are locally rendered data. Formatting for modules can be embedded markup. A user touching or clicking on one of the modules (data element 405) triggers the opening of a new video stream (eg, data element 409). A graph of the company's sales this year compared to last year over the same period is displayed. The graph may be accompanied by other multimedia, such as a narration explaining when TLWK stock price is rising. The manner in which customized data elements are displayed is illustrative only and not intended to be limiting. For example, the data element 409 may be displayed in other parts of the user interface (such as a "Breaking News" insert), may fill the entire screen, or may be displayed in a larger/smaller space within the user interface. may be snapped to occupy. Other customized data elements can also be displayed. For example, a user can view a data element for TLWK and another data element for SANI at the same time. This allows the user to follow many objects/objects of interest while watching background video at the same time. For example, a user can study a detailed report on TWLK while a background video plays, and if another breaking news affects SANI's stock price, the user can easily exit the detailed report on TWLK. Then, they can turn their attention to SANI.

The data elements may also be other types of multimedia, such as audio tracks in language and/or music customized for a particular user. As another example, a background video feed may be an advertisement in which data elements such as prices (e.g. text strings) are tailored to the user, e.g. based on the user's geographic location (tax , which may vary by region), or customized/targeted based on the cost of living or perceived value of a particular user.

Processes for customizing and compositing video feeds are applied in a variety of settings. The diagram below shows an example of a live sporting event.

5A-5C illustrate examples of sports events customized and synthesized on a client device. In some embodiments, the live event is a sporting event (eg, a sporting event between a first team and a second team). For example, in some embodiments, the live event is an American football game, a soccer game, a tennis game, a water polo game, etc. between a first team and a second team. The video feed includes multiple frames (eg, frames captured by respective cameras 140). The video feed may include a fixed number of frames (eg, 16, 24, 28, or 60 frames per second) or a dynamic number of frames. The dynamic number of frames per second of the video may depend on the network and/or client device bandwidth.

User preference settings indicate preferences for one or more graphic data elements (eg, 509 or 511) to be overlaid on the video feed (503) and/or the orientation of the video feed or preferred camera. For example, in some embodiments, user preference settings specify that the user receives a video feed (e.g., a landscape camera view) from a first camera rather than a second camera (e.g., a partial landscape camera view). Show that you like to do something. Additionally, user preference settings indicate preferences for displaying graphical data elements 509 that specify various information regarding the respective subject 505. User preference settings may include, for example, ball location (e.g., ball position information), current ball carrier, target travel path (e.g., ball travel path in play), game clock data element 507, play clock data. Indicates a preference for displaying elements and/or graphical data elements, such as game status data elements.

In some embodiments, the user preference settings indicate a preference for displaying graphical data elements provided by the betting platform (eg, betting module 718). In some embodiments, user preference settings indicate a preference for displaying data elements that provide text-based information (eg, weather information, news, etc.). Thus, users can personalize and interact with the video feed to tailor their viewing experience to their particular preferences. It also allows the user device to monitor the data received from the computer system (e.g., the system shown in Figure 1) by preventing streams of data that the user is not interested in from being communicated unnecessarily. Allows to optimize quantities.

The plurality of data elements may include a plurality of match clock data elements (e.g., an array of match clock elements and associated metadata), a match state data element (e.g., an array of match state elements and associated metadata), a location information data element (e.g. , one or more object location information and an array of associated metadata), graphical data elements (e.g., multiple embedded graphics, multiple textual data elements such as news and/or weather, and associated metadata), textual data elements (eg, news articles, weather information, etc.), scores of sporting events with matching timestamp values, and so on.

The plurality of data elements may include one or more data element arrays. For example, in some embodiments, the plurality of data elements includes game clock data and an array of associated timestamps. In some embodiments, the plurality of data elements includes an array of play clock data and associated timestamps. In some embodiments, each data element within the plurality of data elements is provided as an array of data. Nevertheless, Table 1 illustrates an example game clock data element array, an example game state data element array, and timestamp metadata associated with each array as stored by the computer system illustrated in FIG.

The techniques disclosed herein find application in a sporting event streaming setting. The following figure illustrates an example system and process for capturing a background video feed and determining content that can be customized and displayed with the background video feed to respond to user preferences.

FIG. 6 is a block diagram illustrating one embodiment of a system for customizing and compositing video feeds. System 600 includes additional elements compared to FIG. 1 to facilitate video customization and compositing of live sporting events. System 600 includes a communications interface 107 and a processor 100 (which is similar to server 100 of FIG. 1 unless otherwise specified). Communication interface 107 is configured to receive time-stamped location information of one or more participants of one or both of the first set of participants and the second set of participants in the competition. In various embodiments, time-stamped location information is captured by a telemetry tracking system during a competition. In this example, the telemetry tracking system is comprised of tracking devices 300-1 through 300-P, anchor devices 120-1 through 120-Q, and optionally cameras 140-1 through 140-S. The elements of are managed by a tracker management system 400 as described in detail below.

The processor 100 is connected to a communication interface 107 and is configured to, for example, communicate with a first set of participants and a first set of participants at a certain point in time and/or as of a certain point in time while a current competition is in progress. The method is configured to calculate a first covariate parameter for each of the one or more participants in one or both of the two sets. Each first covariate parameter is derived from time-stamped location information of a corresponding participant of the first or second set of one or more participants of the current competition at that time.

In various embodiments, processor 100 includes a tracking management system 400 for tracking multiple objects and a statistics system 500 for managing various statistics. Tracking device management system 400 facilitates management of one or more tracking devices 300 as well as one or more anchor devices 120 of the system. Statistics system 500 stores and/or generates various statistics for use in predicting outcomes in competitions, such as live sporting events, and placing bets on various situations or outcomes in sporting events and other similar activities. Provide odds for. In various embodiments, the tracking management system 400 and the statistics system 500 include software engines or modules running on the processor 100 and/or include separate or potentially distinct systems (each system (including and/or executing on one or more processors).

In various embodiments, system 600 includes an odds management system 600 for managing odds and a plurality of user devices 700-1 through 700-R. Although odds management system 600 is shown external to processor 100, in some embodiments the odds management system is included within the processor. Odds management system 600 facilitates determining odds for outcomes at sporting events and managing various models associated with predicting outcomes at live events.

In some embodiments, the system comprises one or more user devices 700 that facilitate end user interaction with various systems of the present disclosure (such as odds management system 600). Further, in some embodiments, system 600 includes one or more cameras 140 that capture live images and/or video of live events that are later utilized by the system of the present disclosure. In some embodiments, camera 140 includes one or more high resolution cameras. As non-limiting examples, the one or more high resolution cameras include cameras with 1080p resolution, 1440p resolution, 2K resolution, 4K resolution, or 8K resolution.

The components described above are interconnected, optionally through a communication network. Elements within dashed boxes are optionally combined into a single system or device. Of course, other topologies for computer system 600 are possible. For example, in some embodiments, any of the illustrated devices and systems may actually constitute several computer systems that are linked in a network or in a cloud computing environment. It may be a virtual machine or a container. Further, in some embodiments, rather than relying on a physical communication network 106, the illustrated devices and systems wirelessly transmit information between each other. Accordingly, the example topology shown in FIG. 6 is merely to describe the features of one embodiment of the present disclosure as will be readily understood by those skilled in the art.

In some embodiments, the communication network 106 includes a tracking device management system 400 that manages one or more tracking devices 300 and one or more anchors 120, a statistics system 500, an odds management system 600, and one or more users. Device 700 and one or more cameras 140 are interconnected with each other and with optional external systems and devices. In some examples, communication network 106 optionally includes the Internet, one or more local area networks (LANs), one or more wide area networks (WANs), other types of networks, or A combination of networks.

Examples of networks 106 include the World Wide Web (WWW), intranets and/or wireless networks (such as cellular telephone networks), wireless local area networks (LANs) and/or metropolitan area networks (MANs), and other wireless communication networks. devices, including. Wireless communications optionally utilize any of a number of communication standards, protocols, and technologies, including: Global System for Mobile Communications (GSM), Extended Data GSM Environment (EDGE), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Evolution Data Only (EV-DO), HSPA , HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE802.11a, IEEE802.11ac, IEEE802.11ax, IEEE802.11b, IEEE802.11g, and/or IEEE802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g. Internet Message Access Protocol (IMAP) and/or Post Office Protocol (POP)), instant messaging (e.g. Extensible Messaging and Presence Protocol (XMPP) ), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS), or as of the filing date of this specification. Any other suitable communications protocol, including communications protocols that have not yet been developed.

In various embodiments, processor 100 includes a machine learning engine 210 (see FIG. 7A) that facilitates predicting the outcome of a competition. In the following figure, an example of a processor 100 with a machine learning engine is described in more detail.

7A and 7B are block diagrams illustrating one embodiment of a system for customizing and compositing video feeds. As shown in FIG. 7A, anchor device array 120 receives telemetry data 230 from one or more tracking devices 300. To minimize errors in receiving telemetry from one or more tracking devices 300, anchor device array 120 preferably includes at least three anchor devices. By including at least three anchor devices 120 in the anchor device array, each ping (e.g., telemetry data 230) received from each tracking device 300 includes combined data from at least three anchors receiving the respective pings. can be triangulated using Further details and information regarding systems and methods for receiving pings from tracking devices and optimization thereof are detailed below with particular reference to at least FIGS. 3 and 4.

In the illustrated example, telemetry data 230 received by anchor array 120 from one or more tracking devices 300 includes position telemetry data 232 . Position telemetry data 232 provides position data for each tracking device 300, which data describes the position of the tracking device within a spatial domain. In some embodiments, this position telemetry data 232 is provided as one or more Cartesian coordinates (e.g., an However, in other embodiments, any coordinate system (eg, polar coordinates, etc.) that describes the location of each tracking device 300 is used.

Telemetry data 230 received by anchor array 120 from one or more tracking devices 300 includes motion telemetry data 234. Movement telemetry data 234 provides data regarding various movements of the respective tracking device. In some embodiments, this motion telemetry data 234 is provided as a velocity of the respective tracking device 300, an acceleration of the respective tracking device, and/or a jerk of the respective tracking device. Further, in some embodiments, one or more of the above values are determined from an accelerometer of a respective tracking device 300 (e.g., accelerometer 317 of FIG. 8) and/or is derived from the position telemetry data 232 of . Further, in some embodiments, telemetry data 230 received by anchor array 120 from one or more tracking devices 300 includes biometric telemetry data 236. Biometric telemetry data 236 provides biometric information regarding each subject associated with a respective tracking device 300. In some embodiments, this biometric information includes the subject's heart rate, temperature (eg, skin temperature, temporal temperature, etc.), and the like.

In some embodiments, anchor array 120 communicates the telemetry data described above (eg, position telemetry 232, motion telemetry 234, biometric telemetry 236) to telemetry analysis system 240. Accordingly, in some embodiments, telemetry parsing system 240 communicates telemetry data (eg, data stream 244) to machine learning engine 210 and/or real-time data packager 246 for further processing and analysis.

In some embodiments, real-time data packager 246 connects one or more data sources (e.g., telemetry analysis system 240, game statistics input) by using one or more timestamps associated with the respective data. system 250, machine learning engine 210, etc.). For example, in some embodiments, the data source provides data associated with a real world clock timestamp (e.g., an event occurring at and associated with real world time 1:17 P.M.). do. In some embodiments, the data source provides data associated with a game clock timestamp associated with a live sporting event (eg, an event that occurs with 2 minutes and 15 seconds left in the second quarter). Further, in some embodiments, the data source provides data associated with both real world clock timestamps and match clock timestamps. Synchronization of data sources using time stamps allows the designers of the present disclosure to provide services with added accuracy, especially when placing bets on the outcome of live events. For example, in some embodiments, the data provided to user device 700 (e.g., streaming data 280 and/or direct data 282 in FIG. Describe. communications received from and/or to the user device to determine whether the end user of the user device 700 places a bet within a predetermined time frame (e.g., before the snap of the ball on the next play); Match clock and real-world time data are analyzed and the bet is confirmed, rejected, or held for further consideration.

In some embodiments, machine learning engine 210 receives data from various sources of the present disclosure to predict future outcomes at live sporting events and generate statistics for analysis and utilization. . For example, in some embodiments, the data sources for machine learning engine 210 include position data formation classifier 212 (hereinafter referred to as ``neural network''). For example, in some embodiments, formation classifier 212 analyzes telemetry data 230 to analyze a player's pre-snap formation. Analysis of the pre-snap telemetry data 230 allows the formation classifier 212 to determine various states and conditions of the game, such as downs of the game, violations of positional rules within the game (e.g., offsides, illegal motion, etc.) Make it. Further, in some embodiments, formation classifier 212 analyzes telemetry data 230 received after the start of play to further generate data and information regarding how each formation develops (e.g. , predicted running routes vs. actual running routes, predicted blocking assignments vs. action blocking assignments, player speed over a play, distance between two players over a play, etc.).

In some embodiments, machine learning engine 210 includes historical training data store 214. Historical data store 214 may store data for each particular sport (e.g., sports historical data 508 of FIG. 10), each particular team associated with a particular sport (e.g., team historical data 510 of FIG. 10), and/or Provides historical data and information regarding each particular player associated with a sport and/or team (eg, player historical data 514 of FIG. 10). In some embodiments, this data is initially used as a training data set for machine learning engine 210. However, the present disclosure is not limited to the above, as this data may be used to further enhance the features and services provided by the machine learning engine 210 and other systems of the present disclosure.

Further, in some embodiments, machine learning engine 210 comprises various models 220 that are used to predict future outcomes of sporting events and provide analysis of sporting events. In some embodiments, model 220 of machine learning engine 210 includes a predicted point model 222. Predicted points model 222 provides a numerical probability of receiving points for a particular play in an event. In some embodiments, the model 220 of the machine learning engine 210 determines either the probability of each participating team in the event winning or the probability of a given point difference between winning and losing teams in the event. including a winning probability model 224 that provides a winning probability model 224. Further, in some embodiments, the model 220 of the machine learning engine 210 includes a wins above replacement (WAR) model 226 for each player. The WAR model 226 provides values for the contribution that each player brings to the corresponding team (e.g., player 1 contributes a value of 1 to each team, player 2 contributes a value of 2 to each team, etc.). player 2 is more valuable to the team).

In some embodiments, machine learning engine 210 includes a context store 228. Situation store 228 is a cache of various situation details and/or statistics that are quickly accessed during an actual match scenario. Quick access to context store 228 prevents lag caused by querying different databases and systems (eg, location data formation classifier 212, historical training data store 214, etc.) to obtain the same information. Further details and information regarding the machine learning engine and its included components (such as the various data stores and models described above) are detailed below with particular reference to at least FIGS. 10 and 11.

Machine learning engine 210 communicates various odds and outputs of various databases and models to odds management system 600. Upon communication with the machine learning engine 210, the odds management system 600 provides various bets and predicted odds for future events at the sporting event to the user device 700, as well as to reflect the current status and statistics of the match. to update these odds in real time.

As shown in FIG. 7B, in some embodiments, system 600 includes a game statistics input system 250. Game statistics input system 250 includes at least in-play data 254 (in the American football example, data describing the state of the game during a given play (e.g., a weak-side receiver ran a post route)); The end-of-play data 256 is configured to provide the state of the game after a given play (e.g., the play resulted in a first down at the opposing 42-yard line). Some embodiments In some embodiments, statistics input system 250 data is associated with the world clock and match clock 242 and communicated accordingly to telemetry analysis system 240 and/or machine learning engine 210. A statistics input system 250 is included in the formation classifier 212.

In some embodiments, various data is communicated to application programming interface (API) server 260. This data may include streaming data 244, end of play data 256, data from odds management system 600, or a combination thereof. Accordingly, API server 260 may communicate with various components of the system to provide the various features and services of the present disclosure (e.g., game flow, requests for statistics, wagering on plays, etc.). Facilitates communication between multiple user devices 700 and master statistics database 270. Communication between API server 260 and one or more user devices 700 may include providing streaming data 280 and/or direct data 282 to each user device 700 through communication network 106 and receiving various data from each user device. and receiving a request 284. As non-limiting examples, streaming data 280 includes tracking "telemetry" data, such as player xyz coordinates or player accelerometer data, and direct data 282 includes clocks, scores, or remaining timeouts.

In some embodiments, master statistics database 270 includes some or all of the statistics known to machine learning engine 210 that are available to the user. The master statistical value database is updated periodically, such as every time one play is completed or every several plays are completed. For example, in some embodiments, only a portion of the statistics known to the machine learning engine 210 are stored in the master statistics database 270 because it is desired that they be available to the user. However, the present disclosure is not limited thereto. For example, in some embodiments, master statistics database 270 is included in machine learning engine 270. Elements within dashed boxes are optionally combined into a single system or device.

Having generally described the infrastructure of system 600, an example tracking device 300 will now be described with reference to FIG. 3.

FIG. 8 is a block diagram illustrating one embodiment of a tracking device. In various embodiments, a tracking device (hereinafter also referred to as a "tracker") may optionally include one or more processing units (CPUs) 374, memory 302 (e.g., random access memory), and one or more controllers 388. one or more selectively accessed magnetic disk storage and/or persistence devices 390; a network or other communications interface (which may include RF circuitry) 384; an accelerometer 317; and one or more optional an optical intensity sensor 364, an optional input/output (I/O) subsystem 366, and one or more communication buses 313 for interconnecting the aforementioned components and powering the aforementioned components. and a power supply section 376 for. In some embodiments, data in memory 302 is seamlessly shared with non-volatile memory 390 using well-known computer techniques (such as caching). In some embodiments, memory 302 and/or memory 390 are actually external to the tracking device but connected to the Internet, an intranet, a network, or other form of network or electronic communication using network interface 384. It may be hosted on a computer that is electronically accessible by tracking device 300 via a cable (shown as element 106 in FIG. 7).

In various embodiments, the tracking device 300 shown in FIG. 8 includes, in addition to the accelerometer 317, a magnetometer and/or a magnetometer to obtain information regarding the position and/or orientation (e.g., portrait or landscape) of the tracking device 300. or a GPS (or GLONASS or other global navigation system) receiver.

The tracking device 300 shown in FIG. Device 300 optionally has more or fewer components than shown in the figures, optionally combines two or more components, or optionally has a different configuration or arrangement of components. I hope you understand that. The various components shown in FIG. 8 may be implemented in hardware, software, firmware, or a combination thereof, such as one or more signal processing and/or application specific integrated circuits.

Memory 302 of tracking device 300 shown in FIG. 8 optionally includes high speed random access memory and further optionally includes one or more magnetic disk storage devices, flash memory devices, or other non-volatile Includes non-volatile memory such as solid-state memory devices. Access to memory 302 by other components of tracking device 300 (such as CPU 374) is optionally controlled by memory controller 388.

In some embodiments, CPU 374 and memory controller 388 are optionally implemented on a single chip. In some other embodiments, CPU 374 and memory controller 388 are implemented on separate chips.

Radio frequency (RF) circuitry in network interface 384 sends and receives RF signals (also referred to as electromagnetic signals). In some embodiments, the RF circuit 384 converts electrical signals to form electromagnetic signals and uses the electromagnetic signals to connect communication networks and other communication devices (one or more anchor devices 120 and/or tracking devices). management system 400, etc.). RF circuitry 384 optionally includes well-known circuitry for performing these functions, such as an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, etc. , subscriber identity module (SIM) cards, memory, and the like. In some embodiments, RF circuitry 384 optionally communicates with communication network 106.

In some embodiments, network interface (comprising RF circuitry) 384 operates with ultra-wideband (UWB) technology, which allows tracking device 300 to be used as an anchor device in a crowded spatial area, such as a live sporting event. Enables communication with array 120. In some embodiments, the tracking device 300 has a low power (e.g., about 1 milliwatt (mW)) send a signal. As used herein, these communications and transmissions are hereinafter referred to as "pings." For a discussion of UWB, Jiang et al. , 2000 “Ultra-Wide Band technology applications in construction: a review”, Organization, Technology and Management in Cons See truction 2(2), 207-213.

In some embodiments, power supply 358 includes a power management system, one or more power sources (e.g., batteries), a recharging system, a power outage detection circuit, a power converter or inverter, and a power status indicator ( For example, a light emitting diode (LED)) and any other components associated with power generation, management, and distribution of such tracking device 300 are optionally included. In some embodiments, telemetry data 230 includes information regarding the power supply 358 of each tracking device 300, such as battery consumption or an estimated time before the tracking device requires additional power.

In some embodiments, the memory 302 of the tracking device 300 for tracking each object stores:
- The operating system 304 (for example, ANDROID (registered trademark), iOS, DARWIN, RTXC, LINUX, UNIX (registered trademark), OS X, WINDOWS (registered trademark), or an embedded operating system (such as VxWorks)) is commonly comprises various software components and/or drivers to control and manage various system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components. ,
a tracking device identifier module 305 that stores data used to identify each tracking device 300, such as a tracking device identifier 306 and an optional tracking device group identifier 307;
- A tracking device ping module 308 that stores data and information associated with the ping rate of each tracking device, the tracking device ping module 308 including:
o an instantaneous ping rate 310 that describes the current ping rate at which each tracking device 300 is currently operating;
o a minimum ping rate 312 that describes the minimum ping rate at which each tracking device 300 can operate;
o maximum ping rate 314, which describes the maximum ping rate at which each tracking device 300 can operate;
o a threshold ping rate 316 that describes the minimum ping rate at which each tracking device 300 can operate, and
○Variable ping speed flag 318.

The tracking device identifier module 305 is configured to identify each tracking device 300 from a plurality of tracking devices (eg, tracking device 1 300-1, tracking device 2 300-3, ..., tracking device P 300-P). Store related information. In some embodiments, the information stored by the tracking device identifier module 305 includes a tracking device identifier (ID) 306 that includes a unique ID (e.g., a serial number or code) representing each tracking device 300. . In some embodiments, tracking device ID module 305 includes a tracking device group ID 307 that assigns each tracking device 300 to one or more tracking device groups (eg, tracking device group 418-2 in FIG. 9). Additionally, in some embodiments, the pings communicated by each tracking device 300 include data in a tracking device ID module 305 that allows anchor device array 120 to identify pings received from two or more tracking devices. enable. Further details and information regarding grouping of tracking devices 300 are detailed below with particular reference to at least FIG.

Tracking device ping module 308 stores data and information regarding various ping parameters and conditions of each tracking device 300 and facilitates ping management. For example, in some embodiments, the tracking device ping module 308 manages the instantaneous ping rate 310 of each tracking device 300 (eg, manages the instantaneous ping rate 310 to be 10 hertz (Hz)). In some embodiments, the tracking device 300 has one or more ping rate limits (one or more of a minimum ping rate 312 and a maximum ping rate 314) that define the maximum and minimum ping rates at which the tracking device 300 may transmit pings. (e.g. both). For example, in some embodiments, the minimum ping rate 312 and/or maximum ping rate 314 are based on bandwidth limitations, the number of active tracking devices 300, and the type of anticipated activity (e.g., sports and/or events). type, expected activity of the subject, etc.) by the tracking device management system 400. When configured with one or both ping rate limits, the tracking device ping module 308 operates to adjust the instantaneous ping rate 310 between a minimum ping rate 312 and a maximum ping rate 314. Accordingly, automatic optimization of tracking management system 400 may be used in conjunction with automatic ping rate adjustment of tracking device 300. In some embodiments, tracking device ping module 308 is configured to compare motion detected from accelerometer 317 to a predetermined threshold 316. Accordingly, ping module 308 increases instantaneous ping rate 310 (eg, until instantaneous ping rate 310 reaches maximum ping rate 314) in accordance with a determination that the detected motion is greater than predetermined threshold 316. Similarly, ping module 308 reduces instantaneous ping rate 310 (eg, until instantaneous ping rate 310 reaches minimum ping rate 312) in accordance with a determination that the detected motion is less than threshold ping rate 316.

In some embodiments, the ping module 310 includes a variable ping rate flag 318 (tracking device configured (e.g., wirelessly configured) by management system 400. For example, the tracking device management system 400 may set the variable ping rate flag 318 to "false" for one or more tracking devices 300 that are associated with players who are not currently participating on the playing field; In this case, the instantaneous ping rate 310 remains low even if the player is actively warming up, for example. Tracking device management system 400 sets variable ping rate flag 318 to "true" for one or more players actively participating on the playing field. Further, in some embodiments, each tracking device 300 is dynamically configured based on the respective tracking device's location. For example, a determination that the tracking device 300 is off the field of play (e.g., when the player is not actively participating in the game) versus a determination that the tracking device 300 is within the field of play (e.g., when the player is not actively participating in the game) is actively participating in the match).

System 600 (e.g., anchor array 120, telemetry analysis system 240, tracking Device management system 400, etc.) increases reliability in tracking objects co-located with tracking devices.

As mentioned above, in some embodiments, each tracking device 300 provides telemetry data 230 that is received and communicated by various anchors 120 proximate to the respective tracking device 300. This telemetry data may include positional telemetry data 232 (e.g., X, Y, and/or Z coordinates), motion telemetry data 234 (e.g., velocity, acceleration, and/or jerk), and/or biometric telemetry data. It includes data 236 (eg, heart rate, physical attributes of the player such as shoulder width, etc.).

In some embodiments, each subject in a match is equipped with two or more tracking devices 300 to increase the accuracy of the data received from the tracking devices for the subject. For example, in some embodiments, each subject is equipped with tracking devices 300 on both the left and right shoulders, with each such tracking device functioning normally and having line-of-sight to at least a portion of anchor 120. Accordingly, in some embodiments, data from left and right tracking devices 300 are combined with their telemetry data 230 to form a single timestamped object. This single object combines position data from both tracking devices 300 to create a centerline representation of each player's position. Furthermore, the position calculated with this centerline provides a more accurate representation of the center of the player's position on the playing field. Additionally, prior to creating a single player object as described above, using relative position data from two tracking devices 300 placed on the player's left and right shoulders, the system 600 determines whether the player is facing The orientation (eg, rotation) can be determined. In various embodiments, the inclusion of rotation data allows the task of creating an avatar from data created by recording telemetry data 230 during a match, and/or for future events within the match or the end of the match itself. The task of establishing sophisticated covariates that can be used to better predict outcomes is greatly facilitated.

In some embodiments, tracking device 300 includes any or all of the circuitry, hardware components, and software components found in the device shown in FIG. For the sake of brevity and clarity, only some of the possible components of the tracking device 300 are illustrated to better highlight additional software modules installed on the tracking device 300.

FIG. 9 is a block diagram illustrating one embodiment of a tracking device management system. Tracking device management system 400 is associated with one or more tracking devices 300 and anchors 120. Tracked device management system 400 includes one or more processing units (CPUs) 474, a peripheral interface 470, a memory controller 488, a network or other communication interface 484, and memory 402 (e.g., random access memory). A user interface 478 including a display 482 and an input 480 (e.g., keyboard, keypad, touch screen, etc.) and an output/input (I/O) subsystem 466 and one or more components for interconnecting the components described above. A plurality of communication buses 413 and a power supply system 476 for supplying power to the above-mentioned components are provided.

In some embodiments, input 480 is a touch-sensitive display (such as a touch-sensitive surface). In some embodiments, user interface 478 includes one or more soft keyboard embodiments. Soft keyboard embodiments may include standard (QWERTY) and/or non-standard configurations of symbols on the displayed icons.

Tracking device management system 400 is only one example of a system that may be used in engaging various tracking devices 300, and tracking device management system 400 may optionally include more or fewer components than shown. It is to be understood that the invention may include any combination of components, optionally a combination of two or more components, or optionally a different configuration or arrangement of components. The various components shown in FIG. 9 may be implemented in hardware, software, firmware, or a combination thereof, such as one or more signal processing and/or application specific integrated circuits.

Memory 402 optionally includes high speed random access memory, and optionally non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Including memory. Access to memory 402 by other components of management system 400 (such as CPU 474) is optionally controlled by memory controller 488.

Peripheral interface 470 may be utilized to connect management system input and output peripherals to CPU 474 and memory 402. One or more processors 474 execute various software programs and/or sets of instructions stored in memory 402 to perform various functions for management system 400 and to process data. do.

In some embodiments, peripheral interface 470, CPU 474, and memory controller 488 are optionally implemented on a single chip. In some other embodiments, they are optionally implemented on separate chips.

In some embodiments, power system 476 includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power outage detection circuit, and a power converter or inverter. A power status indicator (eg, a light emitting diode (LED), etc.) and any other components associated with generating, managing, and distributing power for the portable device are optionally included.

As shown in FIG. 9, the memory 402 of the tracking device management system preferably stores:
- The operating system 404 (e.g., ANDROID, iOS, DARWIN, RTXC, LINUX, UNIX, OS , power management, etc.) and facilitate communication between various hardware and software components;
- A tracking device manager module 406 for facilitating management of one or more tracking devices 300, the tracking device manager module comprising:
o Tracking device identifier store 408 for storing related information associated with each tracking device 410-1, such as tracking device identifier 306 and tracking device ping rate 414;
o Tracked device grouping store 416 to facilitate management of one or more tracked device groups 307.

Tracking device identifier store 408 includes information about each tracking device 410-1, such as the tracking device identifier (ID) 306 of each tracking device 300 and the tracking device group 307 with which the respective tracking device is associated. For example, in some embodiments, a first tracking device group 307-1 is associated with the respective subject's left shoulder and a second tracking device group 307-2 is associated with the respective subject's right shoulder. Additionally, in some embodiments, a third tracking device group 307-3 is associated with a respective target first position (e.g., receiver, defensive end, safety, etc.) and a fourth tracking device group 307-4 is associated with a respective target first position (e.g., receiver, defensive end, safety, etc.). , associated with the second position. Grouping 307 of tracking devices 300 allows particular groups to be designated with particular ping rates (eg, faster ping rates for running backs). The grouping 307 of the tracking devices 300 also allows certain groups to be isolated from other tracking devices not associated with their respective groups, which may result in telemetry data provided by the group's tracking devices. This is useful when looking at the expression of 230. Further information regarding tracked devices and tracked device management systems can be found in US Pat. No. 9,950,238, "Object Tracking System Optimization and Tools."

FIG. 10 is a block diagram illustrating one embodiment of a statistical system. Statistics system 500 stores and determines various statistical values in accordance with this disclosure. The statistics system 500 includes one or more processing units (CPUs) 574, a peripheral interface 570, a memory controller 588, a network or other communication interface 584, a memory 502 (e.g., random access memory), and a display 582. and a user interface 578 including an input portion 580 (e.g., keyboard, keypad, touch screen, etc.) and an output/input (I/O) subsystem 566 and one or more for interconnecting the aforementioned components. A communication bus 513 and a power supply system 576 for powering the aforementioned components are provided.

In some embodiments, input 580 is a touch-sensitive display (such as a touch-sensitive surface). In some embodiments, user interface 578 includes one or more soft keyboard embodiments. Soft keyboard embodiments may include standard (eg, QWERTY) and/or non-standard configuration symbols on the displayed icons.

Statistical system 500 is only one example of a system that may be used in viewing and determining various statistical values, and statistical system 500 may optionally have more or fewer components than shown in the figures. It is to be understood that one may optionally combine two or more components, or optionally have a different configuration or arrangement of components. The various components shown in FIG. 10 may be implemented in hardware, software, firmware, or a combination thereof, such as one or more signal processing and/or application specific integrated circuits.

Memory 502 optionally includes high speed random access memory, and optionally non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Including memory. Access to memory 502 by other components of statistical system 500 (such as CPU 574) is optionally controlled by memory controller 588.

Peripheral interface 570 may be utilized to connect management system input and output peripherals to CPU 574 and memory 502. One or more processors 574 execute various software programs and/or sets of instructions stored in memory 502 to perform various functions for statistical system 500 and to process data. do.

In some embodiments, peripheral interface 570, CPU 574, and memory controller 588 are optionally implemented on a single chip. In some other embodiments, they are optionally implemented on separate chips.

In some embodiments, power system 576 includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power outage detection circuit, and a power converter or inverter. A power status indicator (eg, a light emitting diode (LED), etc.) and any other components associated with generating, managing, and distributing power for the portable device are optionally included.

As shown in FIG. 10, the remote user device's memory 502 preferably stores:
- The operating system 504 (e.g., ANDROID, iOS, DARWIN, RTXC, LINUX, UNIX, OS , power management, etc.) and facilitate communication between various hardware and software components;
- location information classifier 212 for determining and analyzing player formation;
- a historical training data store 214 for storing various statistics regarding each sport 508, where each sport 508 includes various team historical data 510 for one or more teams 512; Includes various player statistics 514 for player 516, as well as
● A situation store 228 for storing data about player formations and match situations.

The positional formation classifier 212 (sometimes simply referred to as the formation classifier) provides information regarding the various states and formations of the player at any given point in the game. For example, in some embodiments, formation classifier 212 analyzes telemetry data 230 to determine a pre-snap formation. Accordingly, once the formation is determined and the telemetry data 230 is analyzed, subcategories of the formation may be determined (eg, an I-formation with different subcategories defining different running backs). Further, in some embodiments, the formation classifier 212 acts as a virtual referee and determines whether a violation, such as a player offside, neutral zone infraction, illegal motion, illegal formation, etc., occurred during a match or play. judge. In some embodiments, formation classifier 212 analyzes one or more of various formations in an American football game, such as a first table of offensive formations, a second table of defensive formations, and a third table of special team formations. Provide a table. In some embodiments, the above table of formations provides some or all of the formations described by Table 2, Table 3, and Table 4.

Further, in some embodiments, the formation classifier 212 determines the ball carrier by comparing the telemetry data 230 provided by the ball to the telemetry data of the player closest to the ball. Similarly, in some embodiments, the determination of which team has the ball is similarly performed. Further, in some embodiments, formation classifier 212 determines whether a player is within the boundaries of a match by analyzing telemetry data 230 extracted from the player and comparing it to known boundary lines of the playing field. Determine whether or not. In this manner, formation classifier 212 analyzes telemetry data 230 to provide a box score and/or automatic color commentary for the game.

Although formation classifier 212 is labeled "neural net," formation classifier 212 module need not perform team formation classification using a neural network classifier. In some embodiments, the formation classifier 212 module utilizes virtually any classification scheme that can read team formations from telemetry data. For example, in some embodiments, formation classifier 212 performs team formation classification using a nearest neighbor algorithm. In another embodiment, formation classifier 212 uses clustering to perform team formation classification. In some embodiments, the formation type resolution by formation classifier 212 is based on current live match results (e.g., wins/losses, point spreads, etc.), as disclosed with respect to the methods and features described with respect to FIGS. etc.) is used as a covariate in statistical models that predict

More specifically, in some embodiments, the formation classifier 212 uses a logistic regression algorithm, a neural network algorithm, a support vector machine (SVM) algorithm, a naive Bayes algorithm, a nearest neighbor algorithm, a boosted tree algorithm, a random forest algorithm, Or based on a decision tree algorithm.

As non-limiting examples, formation classifier 212 may employ a logistic regression algorithm, a neural network algorithm, a support vector machine (SVM) algorithm, a naive Bayes algorithm, a nearest neighbor algorithm, a boosted tree algorithm, a random forest algorithm, or a decision tree algorithm. Based on. When utilized for classification, SVM separates a given set of binary labeled data training sets at the hyperplane that is maximally distant from the labeled data. In cases where linear separation is not possible, the SVM can work with "kernel" techniques that automatically implement non-linear mapping to feature space. The hyperplane found by the SVM in the feature space corresponds to the nonlinear decision boundary in the input space. Tree-based methods partition the feature space into a set of rectangles and then fit a model (like a constant) to each rectangle. In some embodiments, the decision tree is a random forest regression. One particular algorithm that can function as a formation classifier 212 for an instant method is a classification and regression tree (CART). Other specific decision tree algorithms that can function as a formation classifier 212 for instant methods include, but are not limited to, ID3, C4.5, MART, and Random Forest.

In some embodiments, historical data store 214 stores statistics related to each sport 508, each team 510 within a sports league, and each player 512. As mentioned above, in some embodiments, data stored in historical data store 214 is utilized as a training set of data for machine learning engine 210 and/or formation classifier 212. For example, in some embodiments, the data stored in the historical data store 214 may include other data sets from similar leagues (e.g., using college football statistical data if the player is a professional rookie). used as initial data set at the start of the league, such as when extrapolated from the league, or when new statistics are generated (e.g., a previously unknown statistic becomes important) used to create data points. Additionally, in some embodiments, data from previously played matches is stored within historical data store 214.

In some embodiments, context store 228 includes data stored within one or more databases of machine learning engine 210 as a cache of information. This cache of context store 228 allows data to be queried and utilized at high speed without the need to query respective databases. In some embodiments, situation store 288 creates a new cache of data for each match. However, the present disclosure is not limited thereto.

FIG. 11 is a block diagram illustrating one embodiment of an odds management system. Odds management system 600 stores and determines various odds in accordance with this disclosure. Odds management system 600 includes one or more processing units (CPUs) 674, a peripheral interface 670, a memory controller 688, a network or other communication interface 684, memory 602 (e.g., random access memory), and a display. 682 and a user interface 678 including an input 680 (e.g., keyboard, keypad, touch screen, etc.) and an output/input (I/O) subsystem 666 and one or more for interconnecting the components described above. communication bus 613 and a power supply system 676 for supplying power to the above-mentioned components.

In some embodiments, input 680 is a touch-sensitive display (such as a touch-sensitive surface). In some embodiments, user interface 778 includes one or more soft keyboard embodiments. Soft keyboard embodiments may include standard (QWERTY) and/or non-standard configurations of symbols on the displayed icons.

Odds management system 600 is only one example of a system that may be used in viewing and determining various statistics, and odds management system 600 may optionally include more or fewer components than shown. However, it is to be understood that the invention may optionally combine two or more components, or optionally have a different configuration or arrangement of components. The various components shown in FIG. 11 may be implemented in hardware, software, firmware, or a combination thereof, such as one or more signal processing and/or application specific integrated circuits.

Memory 602 optionally includes high speed random access memory, and optionally non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Including memory. Access to memory 602 by other components of odds management system 600 (such as CPU 674) is optionally controlled by memory controller 688.

Peripheral interface 670 may be utilized to connect management system input and output peripherals to CPU 674 and memory 602. One or more processors 674 execute various software programs and/or sets of instructions stored in memory 602 to perform various functions for odds management system 600 and to process data. Execute.

In some embodiments, peripheral interface 670, CPU 674, and memory controller 688 are optionally implemented on a single chip. In some other embodiments, they are optionally implemented on separate chips.

In some embodiments, power system 676 includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power outage detection circuit, and a power converter or inverter. A power status indicator (eg, a light emitting diode (LED), etc.) and any other components associated with generating, managing, and distributing power for the portable device are optionally included.

As shown in FIG. 11, the remote user device's memory 602 preferably stores:
- The operating system 604 (e.g., ANDROID, iOS, DARWIN, RTXC, LINUX, UNIX, OS , power management, etc.) and facilitate communication between various hardware and software components;
- A modeling engine 200 for storing one or more prediction or outcome models, comprising:
o a predicted point model module 222 for determining predicted point values for scenarios during a match;
o a probability of winning model 224 for determining the probability of winning a match, and
○ Replacement player ratio contribution model module 226 for each player to determine;
o a real-time match status module 614 for receiving and communicating information associated with the currently occurring match; and
o Odds management module 616 to facilitate management of various odds and betting systems.

As previously mentioned, modeling engine 200 includes various algorithms and models used to generate statistics and predict outcomes in sporting events. In some embodiments, these models include a predicted points model 222 that provides numerical values for each play of the game. For example, if a drive in a game that leads to a touchdown has a play that includes a 5-yard rush, a 94-yard pass, and a 1-yard rush, even if the 1-yard rush led to a touchdown, the 94-yard rush The path has a much more important role within the drive. Thus, in some embodiments, a 5-yard rush is assigned a predicted point value of 0.5, a 94-yard pass is assigned a predicted point value of 5.5, and a 1-yard rush is assigned a predicted point value of 1. is assigned a predicted point value of , where higher values indicate more important or game-defining plays. In some embodiments, modeling engine 200 uses telemetry data collected in accordance with this disclosure to determine the outcome of a game (e.g., win or loss, as disclosed with respect to the methods and features described with respect to FIGS. 2 and 3). point spreads, etc.).

In some embodiments, real-time match situation module 614 receives information associated with situations occurring during the match. This information is then utilized in adjusting the various weights and values within the model described above. For example, if a quarterback twists his ankle and must make every play from a shotgun position, this immobility of the quarterback will be reflected in the game model 220 through the real-time game situation module 614.

FIG. 12 is a block diagram illustrating one embodiment of a user device. The user device is a remote user device 700 associated with an end user according to this disclosure. User device 700 includes one or more processing units (CPUs) 774, a peripheral interface 770, a memory controller 788, a network or other communication interface 784, memory 702 (e.g., random access memory), and a display 782. and a user interface 778 including an input 780 (e.g., keyboard, keypad, touch screen, etc.), an input/output (I/O) subsystem 766, an optional accelerometer 717, and an optional GPS 719. , an optional audio circuit 772, an optional speaker 760, an optional microphone 762, and one or more optional sensors, such as for detecting the intensity of contact to the user device 700. 764 (e.g., a touch-sensitive surface such as a touch-sensitive display system of device 700) and/or optical sensors and one or more communication buses 713 for interconnecting the aforementioned components and powering the aforementioned components. A power supply system 776 is provided.

In some embodiments, input 780 is a touch-sensitive display (such as a touch-sensitive surface). In some embodiments, user interface 778 includes one or more soft keyboard embodiments. Soft keyboard embodiments may include standard (QWERTY) and/or non-standard configurations of symbols on the displayed icons.

User device 700 is but one device example of a multi-function device that may be used by an end user, and user device 700 optionally has more or fewer components than shown in the figures, and optionally includes It is to be understood that two or more of the components may be combined or, optionally, have different configurations or arrangements of the components. The various components shown in FIG. 12 may be implemented in hardware, software, firmware, or a combination thereof, such as one or more signal processing and/or application specific integrated circuits.

Memory 702 optionally includes high speed random access memory, and optionally non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Including memory. Access to memory 702 by other components of user device 700 (such as CPU 774) is optionally controlled by memory controller 788.

Peripheral interface 770 may be utilized to connect management system input and output peripherals to CPU 774 and memory 702 . One or more processors 774 execute various software programs and/or sets of instructions stored in memory 702 to perform various functions for user device 700 and to process data. do.

In some embodiments, peripheral interface 770, CPU 774, and memory controller 788 are optionally implemented on a single chip. In some other embodiments, they are optionally implemented on separate chips.

In some embodiments, audio circuitry 772, speaker 760, and microphone 762 provide an audio interface between the user and device 700. Audio circuitry 772 receives audio data from peripheral interface 770 , converts the audio data to electrical signals, and transmits the electrical signals to speaker 760 . Speaker 760 converts electrical signals into sound waves that can be heard by humans. Audio circuit 772 also receives electrical signals converted from sound waves by microphone 762 . Audio circuitry 772 converts the electrical signal to audio data and transmits the audio data to peripheral interface 770 for processing. Audio data is optionally retrieved from and/or transmitted to memory 702 and/or RF circuitry 784 by peripheral interface 770 .

In some embodiments, power system 776 includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power outage detection circuit, and a power converter or inverter. A power status indicator (eg, a light emitting diode (LED), etc.) and any other components associated with generating, managing, and distributing power for the portable device are optionally included.

As shown in FIG. 12, the remote user device's memory 702 preferably stores:
- The operating system 704 (e.g., ANDROID, iOS, DARWIN, RTXC, LINUX, UNIX, OS , power management, etc.) and facilitate communication between various hardware and software components;
- an electronic address 706 used to identify a particular user device during communication with the various systems and devices of this disclosure;
- a user information store 708 that stores relevant information about each user associated with the corresponding user device 700 (such as user access information including username, user password, and access token);
● a match feed module 710 for viewing various representations of the match and for viewing various statistics associated with the match, including a whiteboard feed module 712, an avatar feed module 714, and a video feed module 716; as well as
● A betting module 718 that facilitates betting on match scenarios.

In some embodiments, the betting module 718 uses telemetry data collected in accordance with this disclosure to determine the current Predict the outcome of a match (e.g., win, point spread, etc.). In some embodiments, the betting module 718 uses telemetry data collected in accordance with this disclosure to provide odds for future in-game events in the current live game.

FIG. 13 illustrates an example environment that includes a playing field that includes components of a tracking system, in accordance with one embodiment of the present disclosure. This tracking system may be included in content creator 140 to track and/or capture video of the event. An example environment (eg, stadium 906) 900. Environment 900 includes a playing field 902 on which a game (eg, an American football game) is played. Environment 900 includes a region 904 that includes a playing field 902 and an area directly surrounding the playing field (e.g., an area that includes subjects not participating in the game, such as subjects 930-1 and 940-1). . Environment 900 includes an anchor device array 120 (e.g., anchor device 1201-1, anchor device 120-2,... , anchor device 120-Q). As shown in FIG. 9, in some embodiments, the array of anchor devices communicates (eg, via communication network 106) with a telemetry analysis system 240 (eg, tracker management system 400 of FIG. 9). Additionally, in some embodiments, one or more cameras 140 (eg, camera 140-1) capture images and/or video of the sporting event, which are used in forming the virtual reenactment. In FIG. 13, marker 930 represents the object of the first team in the game, while marker 940 represents the object of the second team in the game.

FIG. 14 is a functional diagram illustrating a computer system programmed to customize and synthesize video feeds, according to some embodiments. Obviously, other computer system architectures and configurations may be used to implement the video compositing/customization techniques described. Computer system 1400, which includes various subsystems as described below, includes at least one microprocessor subsystem (also referred to as a processor or central processing unit (CPU) 1402). For example, processor 1402 can be implemented by a single-chip processor or multiple processors. In some embodiments, processor 1402 is a general purpose digital processor that controls the operation of computer system 1400. In some embodiments, processor 1402 further comprises one or more co-processors or dedicated processors (eg, graphics processors, network processors, etc.). Using instructions retrieved from memory 1410, processor 1402 receives and manipulates input data received at input devices (e.g., image processing device 1406, I/O device interface 1404) and output devices (e.g., control the output and display of data on display 1418);

Processor 1402 is bi-directionally coupled to memory 1410, which may include, for example, one or more random access memories (RAM) and/or one or more read only memories (ROM). As is well known to those skilled in the art, memory 1410 can be used as general storage, temporary (eg, scratchpad) memory, and/or cache memory. Memory 1410 further includes programming instructions and data in the form of data objects and text objects, in addition to other data and instructions for processing performed on processor 1402, for storing input data and processed data. Can be used to store. Additionally, as is well known to those skilled in the art, memory 1410 typically stores basic operating instructions, program codes, data, and objects used by processor 1402 to perform functions (e.g., programmed instructions). Be prepared. For example, memory 1410 may include any suitable computer-readable storage medium, as described below, depending on, for example, whether data access needs to be bidirectional or unidirectional. For example, processor 1402 can also store and retrieve frequently needed data directly and very quickly from a cache memory included in memory 1410.

A removable mass storage device 1412 provides additional data storage capacity for computer system 1400 and is optionally coupled to processor 1402 in a bidirectional (read/write) or unidirectional (read only) manner. . Fixed mass storage 1420 may also provide additional data storage capacity, for example. For example, storage devices 1412 and/or 1420 may include magnetic tape, flash memory, PC cards, portable mass storage devices such as hard drives (e.g., magnetic, optical, or solid state drives), holographic storage devices, and , and other storage devices. Mass storage 1412 and/or 1420 generally stores additional programming instructions, data, etc. that are not typically utilized by processor 1402. It is understood that the information maintained in mass storage 1412 and 1420 may be incorporated in a standard manner into a portion of memory 1410 (eg, RAM) as virtual memory, if desired.

In addition to allowing processor 1402 to access the storage subsystem, bus 1414 may be used to provide access to other subsystems and devices. As shown, these may include a display 1418, a network interface 1416, an input/output (I/O) device interface 1404, an image processing device 1406, and other subsystems and devices. For example, image processing device 1406 may include a camera, scanner, etc. I/O device interface 1404 may include a touch screen (e.g., a capacitive touch-sensitive screen that supports gesture interpretation), a microphone, a sound card, speakers, a keyboard, a pointing device (e.g., mouse, stylus, human finger), a global A device interface for interacting with a positioning system (GPS) receiver, an accelerometer, and/or any other suitable device interface for interacting with system 1400 may be included. Multiple I/O device interfaces may be used with computer system 1400. The I/O device interface allows the processor 1402 to transmit data as well as, more typically, other devices such as keyboards, pointing devices, microphones, touch screens, transducer card readers, tape readers, voice or handwriting recognition devices. , biometric readers, cameras, portable mass storage devices, and other computers).

Network interface 1416 allows processor 1402 to connect to another computer, computer network, or telecommunications network using a network connection, as shown. For example, through network interface 1416, processor 1402 can receive information (e.g., data objects or program instructions) from another network, or output information to another network in the course of performing method/processing steps. be able to. Information is often represented as a series of instructions executed on a processor and may be received from or output to another network. An interface card (or similar device) and appropriate software implemented (e.g., executed/implemented) by processor 1402 may be used to connect computer system 1400 to external networks and transfer data according to standard protocols. can. For example, various processing embodiments disclosed herein may be executed on processor 1402 or with a remote processor that shares portions of the processing over a network (such as the Internet, an intranet, or a local area network). network, etc.). Additional mass storage devices (not shown) may be connected to processor 1402 through network interface 1416.

Furthermore, various embodiments disclosed herein further relate to computer storage products that include computer-readable media having program code for performing various computer-implemented operations. Computer-readable media includes any data storage device that can store data that can later be read by a computer system. Examples of computer readable media include, but are not limited to: Magnetic media such as disks and magnetic tape, optical media such as CD-ROM disks, magneto-optical media such as optical disks, and application specific integrated circuits (ASICs), programmable logic devices (PLDs), and ROM/RAM. A specially configured hardware device, such as a device. Examples of program code include, for example, machine code produced by a compiler, or files containing high-level code (eg, scripts) that can be executed using an interpreter.

The computer system illustrated in FIG. 14 is but one example of a computer system suitable for use with the various embodiments disclosed herein. Other computer systems suitable for such use may include more or fewer subsystems. In some computer systems, subsystems may share components (e.g., for devices with touch screens, such as smartphones, tablets, etc., the I/O device interface 1404 and display 1418 may share touch-sensitive screen components that both display output to the user). Additionally, bus 1414 is an example of any interconnection scheme that functions to connect subsystems. Other computer architectures having different configurations of subsystems may be utilized.

Another example process for providing customized composite video data with live sports content will now be described.

FIG. 15 is a flowchart illustrating one embodiment of a process for customizing and compositing video feeds for live sports. Each of the steps are similar to those described in FIG. 2 unless otherwise specified.

Referring to block 1501, the process receives a video feed of a live event. For example, a live event includes a match between two players or two teams. In various embodiments, the process receives a background video feed from a remote server. A video feed of a live event is called a background video feed because it can be played in the background of a user interface displayed on a client device.

Referring to block 1503, processing determines a first timestamp for a first frame of the background video feed from timestamped metadata associated with the first frame. Each frame of the video feed is associated with time-stamped metadata that corresponds to a point in time (eg, a reference point in time, a local point in time, etc.). In some embodiments, each portion of the video feed (eg, graphics/text portion, audio portion) is associated with time-stamped metadata. Associating a video feed with time-stamped metadata can be composited (e.g., overlaid) with the video feed and played back in chronological order from one or more data sources (game state data elements, game clock data elements, historical data elements, etc.). Enables alignment (e.g. synchronization) of data elements from one element to another (e.g. elements). Accordingly, each data element is stored with time-stamped metadata during transmission of the video feed so that the video feed and the data element can be combined as they occur during events (including live events).

According to the determination of the timestamp of each frame, a query is communicated (e.g., sent) to the computer system (e.g., via an API server for requesting data elements stored by the system shown in FIG. 1). communicated). In some embodiments, the query includes (eg, specifies) a time range restriction. For example, in some embodiments, the query is for one or more data elements within a time range that are stored by the computer system. In some embodiments, the time range is based on determined timestamps of each frame. For example, in some embodiments, the time range ranges from the time of the determined timestamp to a future point in time (e.g., from the time of the determined timestamp to 10 seconds in the future, 20 seconds in the future , 30 seconds in the future, ..., up to 60 seconds in the future, etc.). In some embodiments, the time range is a dynamic time range. For example, in some embodiments, the dynamic time range depends on the bandwidth of the respective user device and/or the bandwidth of the video feed module 716 (e.g., the size and/or bandwidth of the visualization buffer module 720). do. If the bandwidth of the respective user device 700 and/or video feed module 716 is determined to be relatively low, dynamic time ranges may be used to ensure that queries do not overload the user device and/or video feed module 716. Shortened so that less data is received in response.

Referring to block 1505, the process queries the remote system for data elements within a time range determined by the first timestamp. In some embodiments, the plurality of data elements includes a plurality of game clock data elements (eg, an array of game clock elements and associated metadata). In some embodiments, the plurality of data elements includes a plurality of match state data elements (eg, an array of match state elements and associated metadata). In some embodiments, the plurality of data elements includes a plurality of location information data elements (eg, an array of one or more subject location information and associated metadata). In some embodiments, the data elements include graphical data elements (eg, embedded graphics, textual data elements such as news and/or weather, and associated metadata). In some embodiments, the data elements include text data elements (eg, news articles, weather information, etc.). In some embodiments, each match state data element is a score of a sporting event at a matching timestamp value. In some embodiments, the plurality of data elements includes a combination of the above data elements.

In some embodiments, the method further comprises sizing the visualization buffer (eg, visualization buffer module 720) using the bandwidth of the client device (eg, user device 700). In some embodiments, sizing the visualization buffer using the client device's bandwidth increases the size of the visualization buffer in response to the determination of reduced bandwidth. Furthermore, determining the size of the visualization buffer using the client device's bandwidth reduces the size of the visualization buffer in response to determining the bandwidth improvement. This visualization buffer sizing allows a certain number of time-aligned frames to be dynamically posted to the visualization buffer. For example, in some embodiments, reducing the number of time-aligned frames posted to the visualization buffer reduces load on the client device.

Referring to block 1507, the method comprises receiving a plurality of data elements from a remote computer system responsive to the communicated query. In some embodiments, the plurality of data elements received from the computer system include the plurality of data elements requested by the query. However, the present disclosure is not limited thereto. For example, in some embodiments, a first plurality of data elements is requested by a query, and accordingly, a second plurality of data elements are received from the computer system (e.g., only some data elements are available). Still, in some embodiments, each data element received in response to a query includes associated timestamped metadata that is within (e.g., includes) a particular time range (e.g., the time range is from 1:04 PM to 1:06 PM on a particular day, and accordingly, the data elements received as a result of the query were associated with a time between 1:04 PM and 1:06 PM on that particular day. (including metadata). In some embodiments, multiple data elements are received from one or more data stores of the system (eg, historical data store 214, status store 228, etc.). In some embodiments, the plurality of data elements are stored locally on the user device. In some embodiments, the plurality of data elements includes one or more data elements stored locally on the user device and one or more data elements stored on the computer system (e.g., historical data store 214, status store 228, etc.). and one or more data elements.

Referring to block 1509, processing aligns individual frames of the video feed based on matching data elements and timestamps associated with the frames, respectively, to obtain time-aligned frames. Individual data elements of the plurality of data elements received as a result of the query (e.g., communicated via API server 260) are matched with individual frames of the plurality of frames of the video feed. Each frame and each data element matches each timestamp associated with each frame in the plurality of frames with each timestamp associated with each data element in the plurality of data elements. Aligned based on For example, with respect to the example shown in Table 4, each of the array of game clock data elements and the array of game status data elements are received by the user device from the computer system. Using the timestamped metadata associated with each of these array data elements, the corresponding match clock data element and match state data element are aligned with each frame of the video feed that has the same timestamp. be done. This alignment results in multiple time aligned frames. For example, a frame of a video feed is determined to have a timestamp of 00:00:06. Accordingly, a corresponding match clock data element and a corresponding match state data element with associated timestamped metadata of 00:00:06 are aligned with that frame to obtain a time aligned frame. Each time-aligned frame of the plurality of time-aligned frames is overlaid with one or more data elements of the plurality of data elements coincident in time with the respective time-aligned frame. In some embodiments, overlaying the one or more data elements includes compositing embedded graphics on the video feed (e.g., embedded scrolling ticker, embedded scoreboard, embedded weather conditions, etc.) ). In some embodiments, overlaying includes associating each data element with an affordance region, as described in more detail below.

In some embodiments, based on a match between each timestamp associated with a respective frame of the plurality of frames and each timestamp associated with a respective data element of the plurality of data elements. The step of aligning each data element of the plurality of data elements with each frame of the plurality of frames includes aligning each game clock data element with a timestamp value that matches the respective frame with each of the plurality of frames. It includes the step of identifying among the clock data elements. A respective game clock data element (e.g., data element 1304-4 in FIG. 17) is overlaid onto the respective frame, thereby forming one time-aligned frame of the one or more time-aligned frames. do.

In some embodiments, based on a match between each timestamp associated with a respective frame of the plurality of frames and each timestamp associated with a respective data element of the plurality of data elements. The step of aligning each data element of the plurality of data elements with each frame of the plurality of frames includes aligning each match state data element with a timestamp value matching the respective frame with the respective match state data element of the plurality of frames. It includes the step of identifying among the state data elements. Each match state data element (e.g., data elements 1304-1, 1304-2, 1304-3, 1304-5, etc.) is overlaid onto a respective frame, thereby providing one or more time-aligned frames. form one time-aligned frame.

In some embodiments, the multiple data elements include multiple location information data elements (eg, telemetry data 230). Each location information data element of the plurality of location information data elements is time-stamped location information of one or more players in the first plurality of players or the second plurality of players. Time-stamped location information is captured by a telemetry tracking system (eg, tracker management system 400) at a live event. Thus, based on the match between each timestamp associated with a respective frame of the plurality of frames and each timestamp associated with a respective data element of the plurality of data elements, The step of aligning each data element of the elements with each frame of the plurality of frames includes aligning each location data element with a timestamp value that matches the respective frame in the plurality of location data elements. It includes the step of identifying from. Each location information data element is overlaid onto a respective frame, thereby forming one time-aligned frame of the one or more timed frames.

In some embodiments, a telemetry tracking system (eg, telemetry management system 400) comprises multiple tracking devices (eg, tracking device 300). Each tracking device is configured to provide multiple signals (eg, telemetry data 230). Each player of the first and second plurality of players is associated with one or more tracking devices of the plurality of tracking devices. Three or more receivers (eg, anchor device 120) are configured to receive signals provided by multiple tracking devices. A computer system (e.g., telemetry management system 400 and/or telemetry analysis system 240) generates time-stamped signals for each player of the first and second plurality of players based on the plurality of signals received by the three or more receivers. The device is configured to determine location and/or motion information (eg, location telemetry 232, motion telemetry 234, biometric telemetry 236). This determination step builds a plurality of location information data elements.

In some embodiments, overlaying the one or more data elements for each time-aligned frame of the plurality of timed frames further comprises overlaying each time-aligned frame with a corresponding affordance region within an application executing on the client device. associating the time-aligned frames of the time-aligned frames. For example, in some embodiments, if the data element includes a graphic (eg, an embedded graphic), the affordance region is associated with the graphic. Each affordance area is configured to allow an end user of a respective user device 700 to interact with a respective data element. For example, in some embodiments, when an end user interacts with an affordance region associated with a respective data element (e.g., provides touch input via input portion 780 of respective user device 700), Decisions are made according to that interaction. In some embodiments, the decision is to display a menu (e.g., a menu or selection of statistics, a menu of team season results, etc.), e.g., in some embodiments, the end user Interacting with data element 210 of FIG. 10 causes a menu to be displayed on the user device (eg, overlaid on a video feed) indicating a selection of statistics associated with the respective subject. In some embodiments, interaction by an end user with an overlaid data element changes the display state (eg, visible or on and hidden or off) of the data element.

In some embodiments, matching each frame with one or more respective data elements is performed within a tolerance range of 0.01 seconds to 0.1 seconds. For example, alignment is performed within a tolerance of 0.0167 seconds (eg, 60 frames/second). For example, if each frame is determined to have a timestamp of {00 hours: 02 minutes: 25 seconds: 39 milliseconds} and the alignment tolerance is 5 milliseconds, then {00 hours: 02 minutes: 25 Each data element containing associated timestamped metadata for a time range from {00 hours: 02 minutes: 25 seconds: 46 milliseconds} is aligned with a respective frame. However, the present disclosure is not limited thereto. For example, in some embodiments, if each frame is determined to have a timestamp of {00 hours 02 minutes: 25 seconds: 39 milliseconds} and the alignment tolerance is 10 milliseconds, then 00 hours :02 minutes: 25 seconds: 39 milliseconds} to {00 hours: 02 minutes: 25 seconds: 49 milliseconds} each data element containing associated timestamped metadata is aligned with its respective frame. .

Referring to block 1511, the time-aligned frame containing the overlaid data elements is posted to a visualization buffer (eg, visualization buffer module 720). As mentioned above, the visualization buffer provides a queue (eg, a prioritized queue) of frames, data elements, and/or time-aligned frames to form a composite video feed.

In some embodiments, the visualization buffer holds time-aligned frames between one second and one minute, including one or more time-aligned frames. In some embodiments, the visualization buffer holds time-aligned frames between 20 seconds and 5 minutes, including one or more time-aligned frames.

In various embodiments, processing determines one or more data elements based at least in part on the received content. The processing includes matching at least a corresponding portion of the one or more data elements to a background video feed based at least in part on a match between a timestamp of the respective data element and a timestamp of the respective background video frame. A composite video feed is generated that is customized to one or more user-specific characteristics, such as by matching portions of the video feed. The processing displays a composite video feed on the display device of the client device, where each time-aligned frame is overlaid with a data element that is time-matched with the respective background video frame. Contains time-aligned frames.

In various embodiments, the process receives content associated with one or more user-specific characteristics via a communication interface, where the content includes time-stamped metadata associated with background video frames. and data having a timestamp that is within the time range determined by the first timestamp. Based on user profile and/or previous behavior, different metadata may be available to different users, examples of which are described above.

The diagram below shows an example of a background video and overlay data element, where the background video is footage from a live sporting event and the overlay data element is a scene from a live sporting event. There are various statistics, player information, etc. that can be interacted with.

16 and 17 illustrate an example of a composite video feed customized for an American football game. In some embodiments, one or more user preference settings indicate preferences for one or more graphic data elements overlaid on the video feed and/or the orientation or preferred camera of the video feed. For example, in some embodiments, user preference settings may be configured such that the user can view the video feed from the first camera 140-1 rather than the second camera 140-2 (e.g., the partial landscape camera view of FIG. 17). (e.g., the full landscape camera view of FIG. 16). Additionally, in some embodiments, user preference settings indicate a preference for displaying graphical data elements 1210 that specify various information about the respective subject. In some embodiments, the user preference settings indicate a preference for displaying graphical data elements 1220 that identify the location of the ball (eg, ball location information). In some embodiments, the user preference settings indicate a preference for displaying a graphical data element 1230 indicating the current ball carrier. In some embodiments, the user preference settings indicate a preference for displaying graphical data elements 1240 that indicate the path of movement of the object (eg, the path of movement of a ball in play). In some embodiments, user preference settings may be configured in game clock data element 1304-4, play clock data element 1304-5, and/or game state data element (e.g., elements 1304-1, 1304-2, and and/or indicates a preference for displaying graphical data elements 1304 that provide match status information, such as 1304-3). In some embodiments, the user preference settings indicate a preference for displaying graphical data elements 1300-B that provide a betting platform (eg, betting module 718). In some embodiments, user preference settings indicate a preference for displaying data elements 1306 that provide text-based information (eg, weather information, news, etc.). Thus, users can personalize and interact with the video feed to tailor their viewing experience to their particular preferences. This also allows the user device to optimize the amount of data received from the computer system by preventing streams of data that the user is not interested in from being communicated unnecessarily. .

Although the embodiments described above have been described in some detail for ease of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not intended to be limiting.
[Application example 1] A client device,
a communication interface;
A processor connected to the communication interface,
Receive background video feed from remote server,
receiving content associated with one or more user-specific characteristics via the communication interface;
determining one or more data elements based at least in part on the received content;
by matching at least corresponding portions of the one or more data elements to corresponding portions of the background video feed, the composite video feed customized to the one or more user-specific characteristics. generate,
a processor configured to display the composite video feed on a display device of the client device;
A device comprising:
[Application Example 2] The device according to Application Example 1,
the remote server looks up the content based at least in part on the one or more user-specific characteristics;
A device configured to send the content to the device via the communication interface.
[Application Example 3] The device according to Application Example 1, wherein the one or more data elements are interactive, and generating the composite video feed includes combining the interactive data element with the background video feed. device, including overlaying it on top of it.
[Application Example 4] In the device according to Application Example 1, the received content is
outputting data in the composite video feed including at least one of graphics, text, or audio;
metadata regarding how the data is output within the composite video feed;
including, devices.
Application Example 5. The device of Application Example 1, further comprising the display device, the display device configured to render the composite video feed.
Application Example 6 The device of Application Example 1, wherein the background video feed includes a video feed of a live sporting event.
[Application Example 7] The device according to Application Example 1, wherein the data element includes at least one of a game clock, a game status, and location information.
[Application Example 8] The device according to Application Example 1,
The background video feed includes a video feed of a live sporting event between a first team including a first plurality of players and a second team including a second plurality of players;
The data element includes a time-stamped location information data element associated with one or more players of the first plurality of players or the second plurality of players, the time-stamped location information data element , captured by a telemetry tracking system at a live sporting event,
Matching the data element to a corresponding portion of the background video feed comprises:
identifying each location information data element from the location information data element having a timestamp value that matches the respective frame;
The device includes overlaying the respective location information data elements onto the respective frames to form time-aligned frames in the composite video feed.
[Application Example 9] The device according to Application Example 1, wherein the communication interface periodically requests the content and periodically receives a portion of the background video feed. A device configured to do at least one of the following:
[Application Example 10] The device according to Application Example 1,
the data element includes text;
The device wherein rendering the composite video feed on the display device includes displaying the scrolling of the text along with the background video feed.
[Application Example 11] The device according to Application Example 11, wherein at least one of the scroll speed, direction, refresh rate, termination condition, and interaction rule is configured to control the content received from the remote server. A device based at least in part on
Application Example 12: The device of Application Example 1, wherein the processor further comprises, in response to user interaction with the one or more data elements of the composite video feed, the background video feed or A device configured to update at least one of the composite video feeds.
[Application example 13] The device according to application example 1,
the background video feed is an advertisement;
The determined data element includes a price customized for a particular user.
[Application Example 14] The device according to Application Example 1, wherein the data element includes audio in a language customized for the specific user.
[Application Example 15] The device according to Application Example 1, wherein the data element includes music customized to the specific user.
[Application example 16] A client device,
a communication interface;
A processor connected to the communication interface,
Receive background video feed from remote server,
determining a first timestamp of the first frame from timestamped metadata associated with a first frame of the background video feed;
receiving content associated with one or more user-specific characteristics via the communication interface;
The content includes time-stamped metadata associated with a background video frame and data having a timestamp within a time range determined by the first timestamp;
determining one or more data elements based at least in part on the received content;
at least a corresponding portion of the one or more data elements based at least in part on a match between a timestamp of the respective data element and a timestamp of the respective background video frame; generating a composite video feed customized to the one or more user-specific characteristics, including by matching the one or more user-specific characteristics;
displaying the composite video feed on a display device of the client device, the generated composite video feed having each time-aligned frame overlaid with a data element time-coincident with a respective background video frame; a processor configured to include aligned frames;
A device comprising:
[Application example 17] A server,
A processor,
receiving a request for content associated with one or more user-specific characteristics;
determining content based on the request, including by looking up content based at least in part on the request;
transmitting the determined content to a remote client device in response to the content request;
The client device is configured to generate a composite video feed customized to the one or more user-specific characteristics, such as by matching data elements to corresponding portions of the background video feed. a processor with
a memory coupled to the processor and configured to store the one or more user-specific characteristics in a user profile;
A server comprising:
Application Example 18: The server of Application Example 17, wherein the remote client device is configured to determine the data element based at least in part on the received content.
[Application example 19] A method, comprising:
Receive background video feed from remote server,
requesting content associated with one or more user-specific characteristics;
receiving content from the remote server in response to the content request;
determining a data element based at least in part on the received content;
generating a composite video feed customized to the one or more user-specific characteristics, including by matching the data elements to corresponding portions of the background video feed;
outputting the composite video feed on a display of a client device;
A method of providing.
[Application Example 20] A computer program product, which is embodied in a persistent computer-readable storage medium,
computer instructions for receiving a background video feed from a remote server;
computer instructions for requesting content associated with one or more user-specific characteristics;
computer instructions for receiving content from the remote server in response to the content request;
computer instructions for determining a data element based at least in part on the received content;
computer instructions for generating a composite video feed customized to the one or more user-specific characteristics, including by matching the data elements to corresponding portions of the background video feed;
computer instructions for outputting the composite video feed on a display of a client device;
A computer program product comprising:

Claims (21)

A client device,
a communication interface;
A processor connected to the communication interface,
Receive background video feed from remote server,
receiving content associated with one or more user-specific characteristics via the communication interface, wherein the content associated with the one or more user-specific characteristics is associated with a particular portion of the background video feed; the specific data element for which the database is queried; the time range or the particular frame range is based at least in part on the bandwidth of the client device;
determining one or more data elements based at least in part on the received content;
by aligning at least a corresponding portion of the one or more data elements with a corresponding portion of the background video feed; and overlaying the one or more data elements on the background video feed. generating a composite video feed customized to the one or more user-specific characteristics;
a processor configured to display the composite video feed on a display device of the client device;
A device comprising: 2. The device according to claim 1,
the remote server looks up the content based at least in part on the one or more user-specific characteristics;
A device configured to send the content to the device via the communication interface. 2. The device of claim 1, wherein the one or more data elements are interactive. 2. The device of claim 1, wherein the received content comprises:
outputting data in the composite video feed including at least one of graphics, text, or audio;
metadata regarding how the data is output within the composite video feed;
including, devices. 2. The device of claim 1, further comprising the display device, the display device configured to render the composite video feed. The device of claim 1, wherein the background video feed includes a video feed of a live sporting event. 2. The device of claim 1, wherein the data element includes at least one of a game clock, game status, or location information. 2. The device according to claim 1,
The background video feed includes a video feed of a live sporting event between a first team including a first plurality of players and a second team including a second plurality of players;
The data element includes a time-stamped location information data element associated with one or more players of the first plurality of players or the second plurality of players, the time-stamped location information data element , captured by a telemetry tracking system at a live sporting event,
Matching the data element to a corresponding portion of the background video feed comprises:
identifying from said location information data elements each location information data element having a timestamp value that matches a respective frame;
The device includes overlaying the respective location information data elements onto the respective frames to form time-aligned frames in the composite video feed. 2. The device of claim 1, wherein the communication interface at least one of periodically requests the content and periodically receives a portion of the background video feed. The device is configured as follows. 2. The device according to claim 1,
the data element includes text;
The device wherein rendering the composite video feed on the display device includes displaying the scrolling of the text along with the background video feed. 11. The device of claim 10, wherein at least one of the scrolling speed, direction, refresh rate, termination condition, and interaction rules depend at least in part on the content received from the remote server. Based on the device. 2. The device of claim 1, wherein the processor is further configured to control the background video feed or the composite video feed in response to user interaction with the one or more data elements of the composite video feed. A device that is configured to update at least one of the following: 2. The device according to claim 1,
the background video feed is an advertisement;
The determined data element includes a price customized for a particular user. 14. The device of claim 13, wherein the data element includes audio in a language customized for the particular user. 14. The device of claim 13, wherein the data element includes music customized to the particular user. A client device,
a communication interface;
A processor connected to the communication interface,
Receive background video feed from remote server,
determining a first timestamp of the first frame from timestamped metadata associated with a first frame of the background video feed;
receiving content associated with one or more user-specific characteristics via the communication interface;
The content associated with the one or more user-specific characteristics is relative to one or more databases of one or more data elements related to a particular time range or a particular frame range of the background video feed. obtained based at least in part on one or more queries;
the particular time range or the particular frame range for which the database is queried is based at least in part on the bandwidth of the client device;
The content includes time-stamped metadata associated with a background video frame and data having a timestamp within a time range determined by the first timestamp;
determining one or more data elements based at least in part on the received content;
at least a corresponding portion of the one or more data elements based at least in part on a match between a timestamp of the respective data element and a timestamp of the respective background video frame; and overlaying the one or more data elements on the background video feed, the composite video feed customized to the one or more user-specific characteristics. generate,
displaying the composite video feed on a display device of the client device, the generated composite video feed having each time-aligned frame overlaid with a data element time-coincident with a respective background video frame; a processor configured to include aligned frames;
A device comprising: A server,
A processor,
receiving a request for content associated with one or more user-specific characteristics;
The content associated with the one or more user-specific characteristics may be associated with one or more databases of one or more data elements relating to a particular time range or a particular frame range of a background video feed. or based at least in part on multiple queries;
the particular time range or the particular frame range for which the database is queried is based at least in part on client device bandwidth;
determining content based on the request, including by looking up content based at least in part on the request;
transmitting the determined content to the client device in response to the content request;
The client device comprises: by aligning a data element with a corresponding portion of the background video feed; and overlaying the one or more data elements onto the background video feed; a processor configured to generate a composite video feed customized to one or more user-specific characteristics;
a memory coupled to the processor and configured to store the one or more user-specific characteristics in a user profile;
A server comprising: 18. The server of claim 17, wherein the client device is configured to determine the data element based at least in part on the received content. A method,
Receive background video feed from remote server,
requesting content associated with one or more user-specific characteristics;
receiving content from the remote server in response to the content request;
The content associated with the one or more user-specific characteristics is relative to one or more databases of one or more data elements related to a particular time range or a particular frame range of the background video feed. obtained based at least in part on one or more queries;
the particular time range or the particular frame range for which the database is queried is based at least in part on client device bandwidth;
determining a data element based at least in part on the received content;
the one or more data elements by aligning the data elements with corresponding portions of the background video feed; and overlaying the one or more data elements on the background video feed; Generates a synthetic video feed customized to your unique characteristics,
outputting the composite video feed on a display of the client device;
A method of providing. A computer program that causes a computer to
computer instructions for receiving a background video feed from a remote server;
computer instructions for requesting content associated with one or more user-specific characteristics;
computer instructions for receiving content from the remote server in response to the content request;
The content associated with the one or more user-specific characteristics is relative to one or more databases of one or more data elements related to a particular time range or a particular frame range of the background video feed. obtained based at least in part on one or more queries;
the particular time range or the particular frame range for which the database is queried is based at least in part on client device bandwidth;
computer instructions for causing a data element to be determined based at least in part on the received content;
by aligning the data element with a corresponding portion of the background video feed; and overlaying the one or more data elements on the background video feed, computer instructions for generating a synthetic video feed customized to user-specific characteristics;
computer instructions for outputting the composite video feed on a display of the client device;
A computer program comprising: The client device according to claim 1,
the background video feed includes a video feed of a live sporting event;
at least one of the databases includes information processed concurrently with the live sporting event by a machine learning engine;
At least one database comprises a predicted outcome associated with the live sporting event, the predicted outcome being based at least in part on a machine learning model used by the machine learning engine.

Images