NZ620132B2 - Crowd-sourced video rendering system - Google Patents

Abstract

Disclosed is a method for graphic rendering over multiple clients (82, 84). The method includes distributing rendering tasks to connected client nodes (82, 84) having capable graphics processing units by transmitting viewport state data objects and a unique spatial location to each of the clients (82, 84); performing path tracing at each of the clients (82, 84) from the starting point of their unique spatial locations, and transmitting their rendered output back to the server (22). The server (22) generates a composite rendered output from the individual rendered outputs received by the participating clients (82, 84), and then transmits the composite to all connected clients (82, 84) for display. Thus, as the number of client nodes (82, 84) increases, the scene is rendered more rapidly and at higher quality. 2, 84); performing path tracing at each of the clients (82, 84) from the starting point of their unique spatial locations, and transmitting their rendered output back to the server (22). The server (22) generates a composite rendered output from the individual rendered outputs received by the participating clients (82, 84), and then transmits the composite to all connected clients (82, 84) for display. Thus, as the number of client nodes (82, 84) increases, the scene is rendered more rapidly and at higher quality.

Description

WO 23069 CROWD-SOURCED VIDEO RENDERING SYSTEM 2012/050165 TECHNICAL FIELD The t sure generally relates to distributed, crowd-sourced graphics rendering of applications, and more specifically relates to a server utilizing the graphics sing units of one or more connected s to m diffuse light rendering, receiving the rendered scenes from the one or more connected clients, generating a composite scene from the received scenes, and transmitting the composite scene to all connected clients.

BACKGROUND A client-server architecture, in general, is a distributed computing ecture that partitions tasks or work loads between s, which may be considered as “service ers”, and clients, which may be considered as “service requesters” or “service consumers”.

Often, the servers and the clients are connected via a computer network and various types of data may be transmitted between individual servers and individual clients bi-directionally over the computer network. Traditionally, servers usually have more resources and greater performance capabilities than the clients. A server may share its resources with one or more clients, such as performing certain tasks for the clients (i.e., providing services to the clients). Because a server typically has more resources than a client, the server may complete a task, especially a resourcedemanding task, much faster than the client is able to.

Data exchanged between a server and a client may be represented using any suitable data format and transmitted using any suitable communications protocol. For e, when an application is executed on a server for a client, the output of the application may be represented using a structured document, such as a HyperText Markup Language (HTML) document or an Extensible Markup Language (XML) document. The server may it the HTML or XML nt, which includes the data that represent the output of the application, to the client over a HyperText Transfer Protocol (HTTP) connection between the server and the client. The client, upon receiving the HTML or XML document, may consume the document and render the output of the application y using the HTML or XML document, such as in a web browser executed on the client. 2012/050165 SUMMARY The present disclosure generally s to a crowd-sourced graphics rendering system that distributes rendering tasks among one or more connected clients that transmit rendered output back to a server, which composites the client output into a single rendered output for transmission and display to all connected clients.

In the context of server-side graphics rendering, a single server may include an array of graphics processing units (GPUs) to render graphics objects and stream the output to clients. r, rendering complex 3D environments is expensive in terms of processor cycles, and onal GPUs or s must be added to adequately provide rendered frames or scenes to attached clients. Thus, traditional server-side cs rendering encounters significant scalability issues. It may be prohibitively expensive for the provider to add servers or GPUs to meet the s of an increasing number of clients; conversely, overinvestment in server or GPU infrastructure s in underutilization when demand falls below capacity.

Additionally, in recent years, the number of er-level devices having GPUs capable of quickly ming relatively complex 3D rendering has increased dramatically. For e, whereas the number of personal computers possessing GPUs capable of performing path tracing of a 3D scene in under a minute may have been limited to a handful of computers in the early 2000’s, currently over forty-million personal computers possess GPUs with this capability. ore it is desirable to leverage these client-side GPUs to offload graphics rendering tasks from the server to the clients, in effect crowd-sourcing the rendering of a given frame or environment.

In one embodiment, a computer system hosts a plurality of application instances, each ation instance corresponding to a remote client application. The er system ins a network connection to each of the remote client applications for which an application instance is hosted. In particular embodiments, the application instance is a software stub, as further discussed below. In particular embodiments, the computer system transmits render assets to the client applications for computers having sufficiently powerful GPUs to contribute to rendering. In particular embodiments, the computer system transmits render assets in the form of viewport state data objects (VSDOs) as fiarther discussed below. In particular embodiments, the computer system also transmits or assigns a unique spatial position to each client participating in the crowd-sourced rendering process. In particular embodiments, the client s utilize the 2012/050165 received render assets and the unique viewpoint to render the frame or environment. In particular embodiments, the client devices perform path tracing to render the e lighting of a scene or environment. After completing its assigned rendering task, each participating client device transmits its rendered output back to the computer system. In particular embodiments, the rendered output is a frame from a particular viewpoint. In particular embodiments, the ed output is a lightmap for the diffilse lighting of the environment. In particular ments, the rendered output is a modified VSDO. Upon receiving the rendered outputs, the computer system generates a composite rendered output for transmission back to all connected clients, even those that did not participate in the crowd-sourced rendering process.

In particular embodiments, each connected client views the scene from the exact same viewpoint, and the er system transmits a composite frame from the specific viewpoint to each connected client for display. In particular embodiments, each connected client views the scene from a different perspective, and the computer system transmits a composite render asset for the individual s to use in their own rendering engine. In particular environments, the render asset is a light map or VSDO containing light map data representing the diffilse lighting of the nment. Thus, clients may render the scene from different viewpoints using the render assets while retaining high-quality radiosity lighting. In particular embodiments, the clients render the hanging specular lighting locally and in conjunction with the received render assets containing the crowd-sourced diffuse lighting for the environment. Thus, as the number of client nodes possessing capable GPUs increases, any particular scene may be rendered more y or at higher graphics quality.

In an ment, a method for generating a composite render is disclosed. The method comprises transmitting to one or more clients, by a processor, common render assets, a unique spatial position ent from other transmitted spatial positions and one or more instructions to generate a render using the common render assets and its unique spatial position or using the common render assets and a respective unique spatial position that was transmitted to each of the one or more clients. In an embodiment, each unique l position is ted In an embodiment, the ctions to generate a render comprise instructions to perform path tracing on the render assets from the unique spatial position wherein the render is a light map for a portion of or an entire environment containing the render assets. In an embodiment the render is a frame, and the one or more instructions further comprise a common viewpoint from which to render the frame. In an embodiment, the render assets are viewport state data objects (VSDOs) and the render is the set of rendering assets enhanced by diffuse lighting data calculated as a result of the path tracing.

The method fiarther comprises receiving, by the processor, from each of the one or more clients, a render and generating, by the processor, a composite render from the received renders. The composite render thus generated is transmitted by the processor to the one or more clients. In an embodiment, generating, by the processor, the composite render comprises adding, by the processor, the ed light maps to generate a composite light map of a portion of or the entire environment containing the render assets. In an embodiment, generating, by the processor, the composite render fiarther comprises adding the received frames to generate a composite frame.

A method of receiving a ite render is disclosed. The method comprises receiving, by a sor from a server, render , a unique l position and one or more instructions to generate a render using the received render assets and the unique spatial position.

The method filrther ses generating, by the processor, the render in accordance with the one or more instructions, itting, by the processor, the generated render to the server and receiving, by the processor from the server, a composite render. In an embodiment, the method fiarther ses rendering, by the processor, specular ng from a server-independent viewpoint, ing, by the processor, the rendered specular lighting with the ed composite render and outputting, by the processor, the combined ed specular lighting and received composite render to a display.

In an embodiment, a system for generating a composite render is disclosed. The system comprises, a graphics processing unit, a processor and a storage medium for tangibly storing thereon program logic for execution by the processor. In an embodiment, the program logic comprises common render asset transmission logic, executed by the processor, to transmit common render assets to one or more clients, spatial position transmission logic, ed by the processor, to transmit, to each of the one or more clients, a unique spatial position different from other transmitted spatial positions and instruction transmission logic executed by the processor, to transmit, to each of the one or more clients, one or more instructions to generate a render using the common render assets and its unique spatial position. In an embodiment, the program logic further ses receiving logic, executed by the sor, to e, from each of the one or more clients, a render, composite render generating logic, executed by the processor, to te a ite render from the received renders and composite render transmitting logic, executed by the processor, to transmit the composite render to the one or more clients.

In an embodiment, a system for receiving a composite render is disclosed. The system ses, a graphics processing unit, a processor and a storage medium for tangibly storing thereon program logic for execution by the processor. In an embodiment, the program logic comprises receiving logic, executed by a processor, for receiving from a server, render assets, a unique spatial position and one or more instructions to generate a render using the received render assets and the unique spatial position. The programming logic fiarther comprises render generating logic, executed by the processor, for generating the render in accordance with the one or more instructions, transmitting logic, executed by the processor, for transmitting the generated render to the server and composite render receiving logic, executed by the processor for receiving from the server, a composite render. In an embodiment, the system further comprises ng rendering logic, executed by the processor, for ing ar lighting from a server-independent viewpoint. The system also comprises combining logic, executed by the sor, for combining the ed specular lighting with the received composite render and outputting logic, executed by the processor, for outputting the combined rendered specular lighting and received ite render to a display.

In an embodiment, the instruction transmission logic to te a render further comprises instructions to perform path tracing on the render assets from the unique spatial position wherein the render is a light map for a portion of or an entire nment containing the render assets. In an embodiment, the composite render generating logic filrther comprises adding logic to add the ed light maps to generate a composite light map of a portion of or the entire environment containing the render assets. In an embodiment, the composite render generating logic further comprises adding logic, executed by the processor, to add the received frames to generate a composite frame.

These and other embodiments whose features can be combined will be apparent to those of ordinary skill in the art by reference to the following detailed description and the anying drawings. 2012/050165 BRIEF DESCRIPTION OF THE DRAWINGS FIGURE 1 illustrates an example client-server system implementing a crowdsourced graphics rendering system.

FIGURE 2 illustrates a method of distributing the rendering of a frame from a fixed int to a plurality of clients.

FIGURE 3 rates example received client frames for a given scene.