AU781270B2

AU781270B2 - Monitoring device and method

Info

Publication number: AU781270B2
Application number: AU16417/01A
Authority: AU
Inventors: Andrew Steven Russell
Original assignee: Individual
Current assignee: Individual
Priority date: 2000-01-26
Filing date: 2001-01-24
Publication date: 2005-05-12
Anticipated expiration: 2021-01-24
Also published as: AU1641701A; GB2358919B; GB2358919A; CA2332156A1; GB0001629D0

Description

AUSTRATLA

Patents Act 1990 COMPLETE SPECIFICATION STANDARD PATENT Applicant(s): ANDREW STEVEN RUSSELL Invention Title: MONITORING DEVICE AND METHOD The following statement is a full description of this invention, including the best method of performing it known to me/us: 1 MONITORING DEVICE AND

DESCRIPTION

This invention relates to a monitoring device and, more specifically, to a light monitoring device which may be used during film making, as well as to a corresponding method.

Current advances in computer technology are such that computer-generated images are routinely incorporated into cinema films, television programs, promotional videos and commercials.

15 In order to produce a shot in which a computer generated image is incorporated, the background and any "real" actors, props arid scenery are first filmed. The film is then transferred, in electronic form, to a computer where it is stored.

A computer artist, who has previously generated the required images, will then incorporate those images into the film in what is known in the art as a "virtual studio". The complex motion of the generated image as well as the interaction of the so generated "virtual" image with the "real" actors, landscape and properties requires a great deal of expertise to' perfect. Indeed, a large number of factors have to be addressed when such an image is created.

In order to convince an audience that the virtual image is a real and integral component of the finished film a polished effect must be achieved by the artist.

This requires that the "real" components interact with the "virtual" parts and that the "virtual" parts look as if they are a part of the "real" scene.

2 An important factor in achieving a convincing effect is the play of light over the "real" and "virtual" components. Thus a computer artist will instruct the computer to determine the position of the light source and the computer will calculate the effect that the light would have had if the virtual component had been present in the real scene. The computer is then instructed to apply that lighting effect to the virtual component, and make corresponding virtual alterations to the film. Such alterations take into account the relationship between the light source and the virtual component, such as the casting of a shadow by the virtual component on to a real aspect of the scene.

15 Such techniques are normally most effective in open "real" scenes with a single light source. Such as during daylight in a wide plain for example, or at night in total r near total darkness illuminated by distinct artificial lighting. However, whilst it may be a relatively simple S 20 task for the computer to calculate the lighting effect in such circumstances, in others it is an almost impossible task.

For example, in scenes where there are many diffuse 25 light sources and in ones where light is reflected off of different surfaces it is difficult for the computer to accurately calculate the effect that the virtual component would have in the "real" scene. For example, light reflected off of a partially reflective surface placed behind the virtual component may result in a slight back lighting effect.

Whilst a frame of film in isolation may appear to correctly locate the virtual component, once several frames are run together the brain of a viewer begins to detect the minute differences between the computer enhanced or altered image and that expected from learned 3 experience. Thus, the images are interpreted and understood to have been generated by a computer and computer artist.

It is to be understood that a reference herein to a prior art publication does not constitute an admission that the publication forms a part of the common general knowledge in the art in Australia, or any other country.

According to a first aspect of the invention there is provided a light monitoring device comprising a plurality of spatially distributed light sensitive elements and a recording means, wherein: S 15 the output from each element is recordable as data by the recording means; the so recorded output data from each of the elements .comprises between them a record of the spatial 20 distribution in light conditions around the device; oO.* S• the recording means is arranged to record the data in computer readable form and to transfer the data to an 2 output port of the device such that it is loadable into a S. 25 memory of a computer; and .e the loaded data is processable by a program running on the computer, the computer program being able to construct the light conditions within a virtual studio.

The terms lighting conditions or light conditions, as used herein, refers to inter alia the spatial light distribution and spatial light intensity distribution.

Such terms refer to the three dimensional distribution of light which is experienced by an object at a particular location or at a particular point in time. Thus, the intensity of light and/or the brightness of the light at point around and about the object comprise the spatial 4 distrihutinn. The ryreate tc number of points around the object at which the light intensity and/or brightness is detected and measured the "finer", which is to say the greater the resolution, the spatial distribution.

Preferably, the temporal variations in the output of each element is recordable as data by the recording means, the so recorded output data also comprising between them a record of the temporal variations in light conditions around the device.

Preferably also, the plurality of elements is mounted in or on a housing.

The housing may be an oblate spheroid or it is of spherical, pyramidal or polyhedral form. The elements may be mounted in a regular array in or on the housing.

o o The output of each element may be recordable in real 20 time or at distinct intervals.

Each element may comprise a photodiode, each of which oo: has associated therewith an analogue to digital converter (ADC). The ADCs may be accommodated within the housing, the output from each of which may be recordable by the oo.. recording means.

The recording means is accommodated within the housing or alternatively external thereto.

The computer program may additionally construct a temporal framework for the light conditions.

Additionally, the program may be able to impose the light conditions upon a virtual object in order that the virtual object is lit according to the conditions at the time of detection. The program may also be able to impose 5 the temporal variation of the light conditions upon the virtual object.

A second aspect of the invention provides a lighting system comprising a light monitoring device and a computer, wherein the output from each of a plurality of light sensitive elements comprising the device is loadable into a memory of the computer, a program running on the computer being able to extract the Memorised data and construct a virtual spatial distribution of light related to that detected by the elements, and impose the light conditions upon a virtual object within a virtual studio.

The temporal variation of the output from each light 15 sensitive element may be recorded and the program may be able to impose the temporal variation of light conditions upon the virtual object.

According to a third aspect of the invention there is S 20 provided a method of creating a realistically lit virtual object comprising the steps of; detecting light intensity with a plurality of spatially arranged light sensitive elements; transferring the output from each of the elements, or at least data corresponding thereto, to a computer; constructing virtual light conditions o corresponding to that detected by the elements; and imposing the virtual light conditions upon a virtual object.

The method may further comprise the steps of; recording the temporal variation of output from each element; constructing a virtual light condition which varies according to the temporal variation in output from each element; and imposing the temporally varying condition upon an animated virtual object such that the lighting upon the object varies in accordance with variations detected by the elements.

6 In order that the various aspects of the invention may be fully understood embodiments of the invention, in accordance therewith, will now be described by way of example only and with reference to the accompanying drawings in which: Figure 1 is an isometric view of a first embodiment of the inventive device; Figure 2 is an isometric view of a second embodiment of the inventive device; and Figure 3 is a schematic diagram of a lighting system S 15 comprising the second embodiment.

S"Referring firstly to Figure 1, there is shown a light monitoring device 1 comprising a cuboid housing 10 having, in the centre of each face thereof, a light detecting S 20 element 5. Light impinges against an element 5 which, as a result, produces an output proportional to the brightness or intensity of light impinging thereon.

Referring now to Figure 2, there is shown a second light monitoring device 20 comprising a spherical housing having mounted thereon, in a regular array, a plurality of light detecting elements 25. The housing is also provided with a handle 35 from which the housing 30 may depend or be supported.

Figure 3 shows a schematic diagram of alighting system, generally indicated at 55, comprising the light monitoring device 20. Light, as represented by the arrows lip, impinges upon the light detecting elements causing a signal, proportional to the intensity or brightness of light experienced, to be outputted from each element 25. The signal from each element 25 is stored in 7 recording means 40 accommodated within the housing 30. The signal can be converted from that output from each element to a signal suitable for recording by a dedicated converter 45. The output from each element may be analogue in nature, in which case the converter 40 may be an analogue to digital converter (ADC) to provide a digital signal to be recorded by the recording means The data held within the recording means 40 is transferred to an external recording means 50 by a hard wire connection 48 which may be plugged into a suitable output port 38 situated on the housing 30, or preferably on the handle 35. The so recorded data may be loaded into a memory 60 of a computer 100, wherein it is converted S 15 into a virtual light condition by a program operating within the computer 100.

A computer artist, via the user interface 70, may request that the computer 100 imposes the virtual light condition, such as the spatial light intensity distribution, upon a virtual object within a virtual studio, and subsequently view the results via the interface If an animated virtual object is required, the temporal variation in output from each element 25 is recorded. The computer 100 ,converts the data into a temporally defined light condition, for example a spatial light intensity distribution, which may be imposed upon the virtual animated virtual object. The computer 100 calculates the effect of the light as it would play over the surface of the object as it moves, thereby lighting the object, within the virtual studio, in accordance with the lighting conditions detected by the elements If the temporal variation in output from each element is required, care is taken to ensure that the 8 converters 45 are able to convert the s gni with thi required time resolution, and that the recording means similarly record at a sufficient rate.

In some cases the "on board" recording means 40 can be dispensed with and the signal from each element 25 sent directly to the external recording means 50. In such a case the connection 48 allows the output from each element to be sent along a dedicated channel and recorded at the external means The light sensitive elements 5, 25 can be any suitable element known in the art, although the favoured element 5, 25 comprises a photodiode array. Such two dimensional arrays allow each photodiode to simultaneously register the light impinging thereon and a signal is gathered thereby. The associated electronics are housed within the housing 10, 30, as are, if required, filters to e ensure a high signal to noise ratio.

S In order to create a film, video or commercial :-"incorporating a virtual object within a "real" setting, :o•the setting is filmed with any real components required.

Thus, actors perform their roles interacting as necessary with the scene and with the as yet imaginary virtual component. Well established techniques, such as "blue screening", may be used to allow an actor to take the place of the virtual object within the real scene. Thus the proxy virtual object can interact with the real components of the scene, knock things over, cause splashes and so on.

Whilst the scene is being filmed, a light monitoring device 5, 20 is used to detect and subsequently record the actual lighting conditions present. The device 5, detects the spatial distribution of light present at the time of filming. Moreover, it may be placed at the 9 location which it is intended the virtual object will occupy to detect and record the lighting conditions at that point in space and time.

As stated above, the data are converted into, for example, a virtual spatial light distribution in the virtual studio, whereupon the distribution may be imposed upon the virtual object. In situations where the virtual object has a complicated surface the computer calculates how that surface will interact with the light conditions and alters the image accordingly. Similarly, the effect that the virtual object has on the light experienced by the other components in the shot may also be calculated and alterations made to the shot within the virtual studio.

S" Such processing steps may be automated, requiring little or no input from the computer artist once the e.

position of the virtual object within the "real" scene has been established.

The light monitoring device, its associated electronics and the computer software and hardware enable a computer artist to give a computer generated image, 25 which is to be incorporated into "real" scene, a much more o natural appearance as the lighting is essentially the same as that experienced on the actual "shoot". Thereby providing a much more convincing finished film.

It is noted that the housing 10, 30 may be of any suitable shape, an oblate spheroid, a sphere, pyramid or any polyhedron for example. It can be suspended from the ceiling, for example, hand held or mounted on a suitable stand. However, when setting up the device 1, 20 the alignment of the array of elements 5, 25 with respect to a fixed point in the scene is recorded. This enables the accurate location of the virtual light conditions within 10 the virtual scene. Furthermore, the ight sensitive elements 5, 25 may be situated in an irregular array over the surface of the housing 10, 30. The elements may be positioned at the corners of an object rather than on or in the faces as in the first embodiment.

In certain cases, the housing maybe the shape of the virtual object to be incorporated into the film. Thereby enabling the device to "capture", as close as possible, the lighting conditions which would have been experienced by the virtual object, if it had been present at the time of filming. The housing may therefore be a moving or static model of the virtual object as required.

Whilst the above exemplified embodiments relate to uses within the film or video production industries many other uses are envisaged. For example, light monitoring as part of a scientific study. In such cases the light sensitive elements need not record light in the visible 20 part of the spectrum. Suitable infrared and ultra violet detectors may be used to provide a spatial distribution of intensities in those respective parts of the spectrum.

In the claims which follow and in the preceding description of the invention, except where the context requires otherwise due to express language or necessary implication, the word "comprise" or variations such as "comprises" or "comprising" is used in an inclusive sense, i.e. to specify the presence of the stated features but not to preclude the presence or addition of further features in various embodiments of the invention.

Claims

2. A system according to Claim 1, wherein the temporal variation of the output from each light sensitive element is recordable.
3. A system according to Claim 1 or 2, wherein the program is able to impose the temporal variation of light conditions upon the virtual object.
4. A method of creating a realistically lit virtual object comprising the steps of: detecting light intensity with a plurality of ogle spatially arranged light sensitive elements; transferring the output from each of the elements, or at least data corresponding thereto, to a computer; constructing virtual light conditions corresponding to that detected by the elements; and imposing the virtual light conditions upon a virtual object. 12 A method according to Claim 4 further comprising the steps of: recording the temporal variation of output from each element; constructing a virtual light condition which varies according to the temporal variation in output from each element; and imposing the temporarily varying condition upon an animated virtual object such that the lighting upon the object varies in accordance with variations detected by the elements. S6. A light monitoring device comprising a plurality of spatially distributed light sensitive elements and a 15 recording means, wherein: the output from each element is recordable as data by the recording means; the so recorded output data from each of the elements comprises between them a record of the spatial S 20 distribution in light conditions around the device; the recording means is arranged to record the data in computer readable form and to transfer the data to an output port of the device such that it is loadable into a memory of a computer; and said loaded data is processable by a program running on the computer, the computer program being able to construct the light conditions within a virtual studio.
7. A device according to Claim 6, wherein the temporal variations in the output of each element is recordable as data by the recording means, the so recorded output data also comprising between them a record of the temporal variations in light conditions around the device. 13
8. A device according to Claim 6 or 7, wherein the plurality of elements is mounted in or on a housing.
9. A device according to Claim 8, wherein the housing is an oblate spheroid or it is of spherical, pyramidal or polyhedral form. A device according to Claim 8 or 9, wherein the elements are mounted in a regular array in or on the housing.
11. A device according to any of Claims 6 to 10, wherein the output of each element is recordable in real time. S
12. A device according to any of Claims 6 to 11, wherein the output of each element is recordable at distinct intervals. 20 13. A device according to any of Claims 6 to 12, wherein S* each element comprises a photodiode, each of which has associated therewith an analogue to digital converter eooe (ADC)
14. A device according to Claim 13, wherein each ADC is accommodated within the housing, the output from each of which is recordable by the recording means. A device according to any of Claims 6 to 14, wherein the recording means is accommodated within the housing.
16. A device according to Claim 15 further comprising external recording means. 14
17. A device according to Claim 16, wherein the external recording means is provided by a computer.
18. A device according to any one of Claims 6 to 17, wherein the computer program is able to construct a temporal framework for the light conditions.
19. A device according to any of Claims 6 to 18, wherein the computer program is able to impose the light conditions upon a virtual object in order that the virtual objection is lit according to the conditions at the time of detection.
20. A device according to Claim 18 or 19, wherein the program is able to impose the temporal variation of the light conditions upon the virtual object. i 21. A lighting system substantially as hereinbefore 20 described with reference to the accompanying drawings.
22. A method of creating a realistically lit virtual object substantially as hereinbefore described with oooreference to the accompanying drawings.
23. A light monitoring device substantially as hereinbefore described with reference to the accompanying drawings. DATED this 7 th Day of March 2005 ANDREW STEVEN RUSSELL By their Patent Attorneys GRIFFITH HACK