WO 2006/116536 PCT/US2006/015892
ELECTRONIC PROJECTION SYSTEMS AND METHODS
Related Application Data
[0001] This application claims priority to U.S. Provisional Application No.
60/674,981, filed April 26, 2005, which is incorporated in its entirety herein by
reference.
Field of the Invention
[0002] This invention generally relates to the field of projection displays and more
particularly to the field of electronic projection systems comprising two or more
projectors whose output is combined to form a composite image.
Background
[0003] Increasingly there is a need for motion picture producers and exhibitors to
differentiate their product at motion picture theatre multiplexes from that of
competitors and to differentiate the theatre experience from that which customers can
obtain at home. One approach is to provide images that are larger, sharper and
brighter than what viewers can experience elsewhere.
[0004] A number of attempts have been made over the years to improve the
performance of film based projectors by tiling multiple projectors together (e.g.
Cinerama in the 1950s) or by using a larger 5 perforation 70mm film format (e.g.
Todd AO or Cinemascope). The applicant, IMAX Corporation, successfully
developed a higher performance motion picture system using a 15 perforation 70mm
film format; enabled by a rolling loop film transportation mechanism.
[0005] Another approach to differentiate the performance of film based projectors
is to exhibit 3D motion pictures. This approach has been commercialized by various
organizations including the applicant over the years. Typically 3D presentation
requires two filmstrips, one for each eye, and two separate projectors to display the
images contained on the fihnstrips. Sometimes it may be desirable to convert such a
system so that a standard 2D motion picture can be shown, and in the case of a two
projector system it is straight forward; one projector can be switched off while the
other is used. We shall see that the invention disclosed below has the benefit of
improving performance by using the second projector in 2D operation rather than
letting it sit idle.

WO 2006/116536 PCT/US2006/015892
[0006] An emerging trend within the motion picture industry is to replace standard
film based projection with state of the art electronic projectors for a variety of reasons
including cost savings in motion picture distribution, and presentation of live events
in real time. A disadvantage of current electronic projectors is that they are limited in
resolution and light output required for large immersive screens. This is mainly due
to manufacturing economics and the current emphasis on electronic projectors that
propose to compete with standard 35mm film based projection only. One approach to
deal with the resolution and light output limits of electronic projectors is to tile or
combine the output of multiple separate projectors to form one large composite image
at the display screen surface. A number of patents have been granted discussing
various methods of tiling or stitching together the images of separate electronic
projectors including:
[0007] U.S. Patent No. 5,956,000 discloses a method of combining N projectors
together to form a composite image where the sub-images overlap and where the
overlap areas are modulated to compensate for the increased brightness in those
regions. The sub images are also corrected for misalignments.
[0008] U.S. Patent No. 6,115,022 involves the use of a three-dimensional array of
smoothing factors that is applied to the blending of overlapped image seams as well as
to other composite image artifacts.
[0009] U.S. Patent No. 6,456,339 discloses a method of generating a projector to
screen map by combining the results of a camera to screen mapping with a camera to
projector mapping. The projector to screen map is used to produce a pixel correcting
function that in turn is used to warp images to correct for misalignments and to
correct for illuminance and color artifacts in the region of the screen where the images
overlap.
[0010] U.S. Patent No. 6,222,593 describes a multiple projector system that tiles
images together to achieve a high resolution display. Images are captured from a
camera and parameters are calculated to permit warping the output of each of the
projectors via analytic expressions.
[0011] U.S. Patent Nos. 6,568,816 and 6,760,075 describes a projection system,
which has a single light source that supplies light to multiple projection heads whose
2

WO 2006/116536 PCT/US2006/015892
output are sub-images that overlap to form composite images. The single light source
ensures that colorimetery matching problems between the sub-images are eliminated.
[0012] U.S. Patent No. 6,570,623 discloses the use of blending frames located
between the projection lenses and the display screen to control the brightness of the
images in the overlapping region, and further discloses the use of an adaptive
technique using a camera based iterative algorithm to fine tune the blending of
overlapped images.
[0013] U.S. Patent No. 6,771,272 discloses a graphics system comprising pixel
calculation units and a sample buffer that is used to correct for display non-
uniformities, such as seam overlap brightness by appropriately scaling pixel values
prior to projection.
[0014] U.S. Patent No. 6,733,138 describes a system of forming a mosaic image
from multiple projectors by projecting registration images from each to form a union
registration image. This registration image is then used to generate a projective
matrix, which is used to warp individual source images to achieve a unified composite
image. The brightness from each of the projectors is weighted in the overlap regions
to minimize seam visibility.
[0015] U.S. Patent No. 6,804,406 describes a composite image display method
using display to screen and screen to camera spatial transformation functions as well
as a spatial luminance transfer function to pre-warp image segments prior to
projection. An inverse of the spatial luminance function is used to blend colors in the
tiled composite image.
[0016] U.S. Patent No. 6,814,448 discloses a composite image display system
which uses test images and means of sensing to determine correction data that is used
to provide for a uniform level of illumination, overlap regions included, across the
entire display screen surface.
[0017] All of these tiling techniques use various combinations of optical and
electronic image correction to ensure that the overlapped region is indistinguishable
from non-overlapped regions. Electronic image correction sacrifices the number of
bits available to display images (bit depth) because some of the available image bits
are used to correct for non-uniformities in brightness and color. In order to correct for
brightness, color mismatches and spatial misalignments of pixels between the
3

WO 2006/116536 PCT/US2006/015892
projectors a calibration technique, which measures the image on the screen to
determine the required correction must be employed.
[0018] Conventional methods to achieve tiling require warping of images from
every projector in the system. Each projector carries its own set of distortions that
need to be eliminated in order to prevent artifacts near or within the overlap region.
The removal of all distortions requires a mapping onto absolute screen coordinates,
which is done through analytic expressions.
[0019] In the process of equalizing brightness and color between the two
projectors, the output from each color channel must be adjusted. This adjustment is
subtractive and leads to a lower light output of the combined system. These displays
that use tiling must be frequently recalibrated primarily due to the reduction in
brightness or changes in color that occur as the lamps age.
[0020] As well, these patents listed above do not address the unique requirements
of projecting 3D stereoscopic motion picture images. Foremost 3D projection
requires two separate and coded channels of image data to be projected, one for each
eye's (left and right) point of view. In a tiled system the only way to achieve separate
left and right eye images without modification of the system is to multiplex left and
right eye images in time. As such the display duration of each frame is halved with
the first portion devoted to displaying left eye images and the second portion for '
displaying right eye images. While this approach is possible, in one implementation,
it requires expensive alternate eye shutter glasses to be worn by audience members.
The need for alternate eye glasses can be eliminated with the use of a fast acting
polarization converting element to switch the polarization of images for the right and
left eyes thus allowing passive polarizing glasses to be worn by the audience, see for
example, U.S. Patent No. 4,281,341. Whether alternate eye shutter glasses are used or
a fast acting polarizer is employed, time multiplexing the left and right eye images
sacrifices brightness. As well, these methods place a higher demand on the electronic
projectors to show content at faster frame rates and results in a reduced bit depth of
the projected images.
[0021] There are also alternative approaches to project 3D in a tiled projection
system that would require modification to placement of images on the screen. In the
case of a two projector system, this would require that the output of the two projectors
4

WO 2006/116536 PCT/US2006/015892
be fully overlapped. A passive 3D technique may then be used (polarizers or color
filters) to separate left and right eye images. However, converting a system that
requires images to be tiled for 2D operation and overlapped for 3D operation within a
short time period time between 2D and 3D motion picture screenings would be
complex and cost prohibitive.
[0022] A preferred approach for combining the output of two or more projectors
used for 3D and 2D presentations is to completely overlap the two images. When
images are completely superimposed, differences in brightness and color between the
two projectors do not appear as local discontinuities that are readily detectable by the
human eye. As such, a completely superimposed image does not suffer the loss in
image bit depth and brightness incurred in a tiled display to achieve the required
uniformity and does not require calibration to ensure overlapped and non-overlapped
regions are indistinguishable. In a fully overlapped system the only calibration that is
required is the measurement of the spatial distortions that cause pixel misalignments
among the pixels projected from different projectors. A projection system that
superimposes images is thus more robust due to insensitivity to changes in brightness
and color of the images that occur as the system is used.
[0023] The following patents discuss various embodiments of fully overlapped
component projectors achieved by electronically warping the image data.
U.S. Patent No. 6,456,339. In one embodiment of this patent, the images of two
projectors having a small pixel fill factor are completely overlapped to produce a
super resolution display. U.S. Patent No. 6,222,593 describes an embodiment where
their warping system is used to superimpose two images that may be used to increase
2D light levels or may be used for 3D applications.
[0024] U.S. Patent Application No. 2004/0239885 discloses a super resolution
composition method that uses a derived projector correspondence map to a target
surface. All component images are warped to the target surface and then an algorithm
working in the spatial frequency domain optimizes the image quality. This
optimization process depends on the image being displayed and is iterative making it
unsuitable for real time motion picture projection.
[0025] The following patents describe methods for increasing the resolution of a
display by superimposing with a half pixel offset between the component images
5

WO 2006/116536 PCT/US2006/015892
without warping the images electronically. Offset may be defined to be a vector
displacement with two orthogonal components.
[0026] U.S. Patent No. 5,490,009 discloses a method of enhancing the horizontal
and/or vertical resolution of a display device by simultaneously combining the output
of two or more offset spatial light modulators.
[0027] U.S. Patent No 6,222,593 is primarily focused on methods for tiling, but
does mention the possibility of superposition of images to increase light levels and to
allow the system to be used for 3D presentations.
[0028] U.S. Patent No. 6,231,189 discloses a dual polarization optical projection
system capable of 2D and 3D presentations in which separate component images are
combined prior to projection through a single projection lens. The resulting images
are fully overlapped on the projection screen and can be used to increase the
brightness of the display, increase the resolution of the display by imposing a fixed
offset of one image relative to the other of less than one pixel, or project stereoscopic
images using the orthogonal polarization of the component images to distinguish left
and right eye images.
[0029] Other patents, such as for example, U.S. Patent Nos. 6,231,189 and
5,490,009, disclose methods to achieve higher brightness and resolution by
superimposing projectors with a fixed sub-pixel offset relative to each other. In order
to achieve the fixed offset when projecting on a curved screen, the images must be
combined through a single projection lens as disclosed in U.S. Patent No. 6,231,189.
This negates the possibility of using off-the-shelf projectors. In addition, there are
considerable challenges involved to mechanically register pixels with a fixed sub-
pixel offset and maintain this offset over repeated use. In particular, when
illuminating large screens the amount of light that must travel through the system
results in thermal cycling that makes pixel registration more challenging.
[0030] To overcome challenges of maintaining a fixed sub-pixel registration
required when combining multiple projectors to enhance brightness and increase
resolution, certain patents or published patent applications such as, for example, U.S.
Patent Application No. 2004/0239885 and U.S. Patent Nos. 6,456,339, 6,814,448, and
6,570,623 disclose methods of image warping. These warping methods use a
calibration method to measure the spatial misalignments between different projectors.
6

WO 2006/116536 PCT/US2006/015892
These calibration methods calculate a correspondence map between the projectors and
a screen co-ordinate system for warping the image data to correct for geometric
distortions. The distortions result from optical or projection point differences among
the projectors. The calibration methods disclosed work on the premise of being able
to calculate absolute screen positions. Absolute screen positions are required to
correct distortions caused by projection points that deviate significantly from normal
incidence relative to the screen or where the intended application is sensitive to
distortions. In order to convert images taken by a camera to absolute screen
positions, the distortion of the camera and the relationship of the camera to the screen
must be known. In these systems both images are warped to the absolute screen
coordinates. If multiple cameras are used, then the calibration of the cameras to the
screen must be extremely accurate in order to ensure correct warping of the images to
achieve pixel registration. As disclosed in the prior art, this requires moving a
physical test target across the screen surface. In a large cinema projection system this
method of calibration is not practical.
[0031] The above patents do not address fee needs that a motion picture cinema
projection system must fulfill to be successful against competing display
technologies. In particular, systems that require the determination of absolute screen
coordinates to superimpose images are unnecessarily complex and impractical to
implement in a cinema theatre environment. They do not take advantage of the fact
that, in a theatre environment, the image from a projector has relatively low distortion
and may be projected essentially without modification onto the screen. This arises
from the fact that the optical axis of the projection system is near normal incidence to
the screen in a typical theatre environment. In addition, an immersive cinematic
experience requires a large field of view that can't be seen all at once. In this
situation, distortions occur gradually relative to the viewer's gaze and are not
noticeable.
[0032] There are additional requirements that are not easily met by existing art
when the cinema projector must show both 2D and 3D presentations or when one
presentation is a mixture of 2D and 3D formats. In some cases a custom projection
system must be designed. In other cases, one must resort to using expensive shutter
7

WO 2006/116536 PCT/US2006/015892
glasses or incur the light loss of 3D methods that use time multiplexing to distinguish
left and right eye images.
[0033] Existing art does not take advantage of the different requirements of 2D
and 3D presentation and the display properties required for an immersive experience.
There is a difference between optimal brightness for 2D compared to 3D projection.
In 3D projection a trade-off exists between brightness and perceived cross-talk
between left and right eyes. Cross-talk occurs when a right-eye image leaks into the
left eye or vise versa. This ghosting artifact is more apparent when the screen
brightness is increased. As a result, the optimal brightness for 3D projection is
generally lower than that required for 2D projection.
[0034] In addition to providing enhancements to the two modes of presentation, a
successful system must also provide: a high quality of presentation in these modes; be
cost effective; be easy to set up and calibrate; allow for a quick conversion from one
mode to the other, and be easy to maintain.
[0035] The needs described above require a unique and optimal combination of the
physical arrangement, calibration and mapping of the component projectors. This
combination of elements is the subject of this patent and will be discussed in more
detail below.
Summary of the Invention
[0036] Embodiments of the present invention provide an electronic projection
system that is capable of improving image fidelity by combining the output of
multiple projectors by warping images to sub-pixel accuracy relative to a reference
(master) projector. Embodiments of the present invention include methods and
systems in which multiple projectors are configured in a master-slave relationship.
The slave projector(s) images are warped to sub-pixel accuracy relative to the master
projector to achieve higher light levels and improved image fidelity. Further
differentiation is obtained by using these same projectors to display either
stereoscopic (3D) motion pictures or enhanced 2D motion pictures.
[0037] In one embodiment of the present invention, the images of the slave
projectors are homographically mapped to the master projector. This is distinct from
conventional projection systems where all projectors are homographically mapped to
8

WO 2006/116536 PCT/US2006/015892
absolute screen coordinates by means of a camera, which is homographically mapped
to the screen. Unlike these conventional systems, one embodiment of the present
invention does not attempt to get rid of all distortions of the master projector, but
instead maps all slave projectors to match the image of the master projector. The
relative warping disclosed in this invention greatly simplifies the system needed to
achieve superposition of images to sub-pixel accuracy.
[0038] In an embodiment of the present invention, empirical data is used to
achieve superposition to sub-pixel accuracy. Conventional projection systems are
generally focused on removing all aberrations in the system by mapping to absolute
screen coordinates. In order to achieve this mapping, analytic expressions are used by
such systems to correct and remove the distortions in all the projected images. In
contrast, one embodiment of the present invention uses a pixel correspondence map
that is derived empirically to serve as the means for warping. An advantage of this
method is that superposition to sub-pixel accuracy can be achieved even in regions
where higher order distortions occur and analytical modeling is difficult. An example
of such a distortion is the deformation of a vinyl screen when mounted upon a frame
that is curved in the horizontal direction. In order to remove all creases, the vinyl
must be stretched across the frame. This stretching causes the screen to deviate from
a cylindrical shape. Furthermore the shape will change with time as the screen
relaxes. By using an empirical method to achieve superposition, the ability to achieve
sub-pixel registration is independent of the ability to model higher order distortions.
[0039] Briefly, according to one aspect of this invention, a projection system and
method are described that optimally use two electronic projectors to provide both a
3D stereoscopic mode and a 2D projection mode of operation. Switching between
these modes of operation can occur between presentations and within presentations.
The two projectors are physically set up with their image fields substantially or
completely overlapped. A camera is used to measure projected test patterns on the
screen. The camera images are used to calculate a pixel correspondence map that
maps the projected pixel location of one projector to projected pixel location of
another projector with sub-pixel accuracy.
[0040] In a first embodiment of a 2D mode of operation, one projector is selected
as a master and projects the source image without warping. This is advantageous due
9

WO 2006/116536 PCT/US2006/015892
to the savings in computation that is significant for projecting motion picture
presentations. As well, there is an image quality advantage in a system that does not
require warping of the master image. Warping requires interpolation and can
introduce artifacts into the image thus reducing image fidelity compared to an image
that is not warped. The second projector (slave) projects an image that is warped to
match the image from the master projector by selective sampling of the source image
using the pixel correspondence map. The resulting composite image is twice as
bright and exhibits an improvement in image fidelity due to a reduction in image
artifacts as will be explained below.
[0041] In a second embodiment of a 2D mode of operation, the power of the
projection lamps can be reduced to prolong the life of said lamps while still
maintaining a screen brightness equal or greater to the brightness from a single
projector.
[0042] In a third embodiment of a 2D mode of operation, the total overlapping of
images is a means to provide a higher frame rate presentation using projection
technology that would otherwise be considered too slow to do so.
[0043] In a fourth embodiment of a 2D mode of operation, the master projector
projects a pre-distorted image so that the resultant image on the screen is free of first
order distortions. Examples of first order effects are keystone distortion and barrel or
pincushion lens distortion. As per other 2D embodiments, the slave projectors) are
warped to sub-pixel resolution to match the master projector image.
[0044] In the 3D mode of operation, the second eye of the stereoscopic pair is
warped to match the first eye to eliminate the need for high precision optics to match
left and right eye image sizes. Failure to match image sizes results in distortions that
degrade the stereoscopic effect and may lead to eye fatigue.
[0045] In a second embodiment of a 3D stereoscopic mode of operation, each
electronic projector combines three separate color channels, two of which have one
polarization orientation with the third having an orthogonal orientation of
polarization, into a unified beam of light. This is a common output polarization
configuration for a liquid crystal based modulator system where the color channels are
combined through a polarization sensitive x-Cube. The polarization states of all the
channels in the two projectors are rendered orthogonal to one another by a suitable
10

WO 2006/116536 PCT/US2006/015892
optical means. In so doing, one color channel in the second projector has identical
polarization to the two complementary color channels in the first projector. The
remaining three color channels from the two projectors have identical polarization
states that are orthogonal to the first three. The color channels having identical
polarization are combined to form the stereoscopic images, one for each eye. Because
each eye's image is a composite image with two color channels from one projector
and a third from a second projector one of the components must be warped, using the
pixel correspondence map, so that there is an accurate overall registration of the color
channels. This color dependent warping technique eliminates the need for additional
polarizers or polarization converting elements, which introduce light loss and changes
in color.
Brief Description of Drawings
[0046] These and other features, aspects, and advantages of the present invention
are better understood when the following Detailed Description is read with reference
to the accompanying drawings:
Figure 1 is a schematic showing the overall components of the projection
system according to one embodiment of the present invention;
Figure 2 is a schematic showing the operational flow of image data in the 2D
mode of operation according to one embodiment of the present invention;
Figure 3 is a schematic showing the operational flow of image data in the 3D
mode of operation according to one embodiment of the present invention;
Figure 4 is a schematic showing the operational flow of image data when a
high resolution source is used to project 2D images according to one embodiment of
the present invention;
Figure 5 is a schematic showing the operational flow of image data when a
high resolution source is used to project 3D images according to one embodiment of
the present invention;
Figure 6 is a schematic showing the operational flow of image data in the
alternate 3D mode were color channel have different polarizations according to one
embodiment of the present invention;
11

WO 2006/116536 PCT/US2006/015892
Figure 7 is a schematic showing the operational flow of image data in the
alternate 3D mode were color channel have different polarizations and all colors of
the slave projector are warped according to one embodiment of the present invention;
Figure 8 is a flow chart showing the steps performed in generating the pixel
correspondence map to warp one image into alignment with the other according to
one embodiment of the present invention;
Figure 9 is an example of a test pattern used for calibration according to one
embodiment of the present invention;
Figure 10 is a schematic illustrating the relative and varying offset of one
projected image with the other according to one embodiment of the present invention;
Figure 11 shows an illustrative shape of one possible sampling function used
to warp one image with respect from another using the pixel offset map according to
one embodiment of the present invention;
Figure 12 shows an illustrative fractional pixel offset map as a function of
screen position according to one embodiment of the present invention;
Figure 13 shows illustrative plots showing pixilated output of single and
superimposed projectors for sinusoidal input signals of different frequencies
according to one embodiment of the present invention;
Figure 14 shows an illustrative comparison of MTF for single and
superimposed projectors with 100% fill factor according to one embodiment of the
present invention;
Figure 15 shows illustrative artifacts as a function of spatial frequency for
superimposed images with different offsets according to one embodiment of the
present invention;
Figure 16 shows illustrative artifacts from a single projector compared to
artifacts from a system with superimposed images averaged over all pixel offsets
according to one embodiment of the present invention;
Figure 17 shows an illustrative operational flow of image data when a high
frame rate mode is used according to one embodiment of the present invention;
Figure 18 shows an illustrative timing diagram for the high frame rate mode
according to one embodiment of the present invention;
12

WO 2006/116536 PCT/US2006/015892
Figure 19 shows an illustrative timing diagram for the high frame rate mode
with increased brightness according to one embodiment of the present invention;
Figure 20 shows an illustrative timing diagram for a high frame rate mode of
operation with shutter for reduced motion artifacts according to one embodiment of
the present invention; and
Figure 21 shows an illustrative timing diagram for operation with double
shutter for reduced artifacts according to one embodiment of the present invention.
Detailed Description
[0047] Referring to Figure 1, a convertible projection system (1) is shown
composed of two separate electronic motion picture projectors (3,4). Other
embodiments of the projection system may contain more than two electronic motion
picture projectors. The projectors are not limited to a particular electronic technology
and may in fact be based on DMD (deformable mirror device) technology, LC (liquid
crystal) reflective or LC transmissive technology, or any other existing or emerging
electronic projection technology. In Figure 1 the two projectors (3,4) are shown in
their preferred embodiment of one placed above the other; however they may also be
arranged in other positions relative to one another. The two projectors, regardless of
their physical arrangement, project their images onto a projection screen (2) so that
they are substantially superimposed or overlapped.
[0048] In other embodiments, more than two projectors may be used. For
example, more than two projectors superimposed to project onto a single area.
Alternatively, another embodiment includes both tiling and superposition where any
point on the screen is illuminated by at least two projectors.
[0049] An alignment camera (5) is positioned to record test images projected by
both projectors (3,4) onto the screen (2) to calculate the pixel correspondence map
(Fig 2,21) used by the warping algorithm in the warping unit (Fig 2,20). Once the
two projectors (3,4) are physically positioned and correspondence map (Fig 2,21)
calculated, the projection system (1) can be rapidly and easily switched from 2D and
3D modes of projection by simply changing the electronic data sent to the projector.
As stated above, color and brightness changes that occur as the system ages do not
have any first order effects on image quality for 2D presentations. For 3D
13

WO 2006/116536 PCT/US2006/015892
presentations, the brightness of the projectors must be matched to ensure high quality
presentations.
[0050] The embodiment as shown in Figure 1 uses a single camera for calibration.
However, other embodiments may use multiple cameras to serve the same function.
[0051] Re-calibration of this preferred embodiment is only required when the
relative position of the projectors (3,4) to the screen (2) is changed. This may occur
due to a physical movement of the projector or due to a change in screen position.
Changes in position of the image on the screen can be caused by changes in the
optical path or modulator positions. The immersiveness of the presentation is
enhanced by curving and tilting the screen towards the audience. Screens are
normally made of vinyl that is stretched over a frame located at the perimeter of the
screen. The position of the screen may change over time as the vinyl stretches and
sags.
[0052] Figures 2 and 3 illustrate schematically the flow of image data from storage
to projection for each projector in both the 2D and 3D stereoscopic modes of
operation. In the 2D mode shown in Figure 2, a single source image buffer (10)
contains source image data for one image that is to be projected on a projection screen
for viewing by a theatre audience. Source image data buffer (10) is continuously
being refreshed by new images at a suitable display rate, typically 24 frames per
second, from an external image storage unit not shown. Image data from source
image data buffer (10) is transferred simultaneously and in parallel to a master
projector image data buffer (11), and to an image warping unit (20). The warping unit
(20) warps the source image data in accordance with a pixel correspondence map (21)
generated in calibration process (as described below and shown in Figure 8). Once the
source image has been warped by the warping unit (20), it is transferred to a slave
projector image buffer (12). Then the image data in the buffers (11,12) are
simultaneously transferred to the projectors (3,4) for projection on to the screen. The
images are substantially or completely overlapped. The slave projector image is
warped with sub-pixel accuracy to match the master projector image. The composite
image on the screen is without noticeable defects and exhibits an improvement in
image quality over that which would result from only one of the projectors being
used. In particular the composite image will exhibit superior brightness and an
14

WO 2006/116536 PCT/US2006/015892
improvement in image fidelity due to a reduction in image artifacts for superimposed
images. If the increased brightness on the screen is in excess of what is needed to
achieve the desired presentation quality, the power of the projection lamps can be
reduced to prolong their life and reduce operating costs.
[0053] Figure 3 shows image data flow in the 3D stereoscopic mode of operation.
In this mode each projector has a unique source of image data, one set of images
corresponding to left eye images and the other to right eye images. A source image
buffer (13) transfers the left eye image data to a master projector image buffer (11)
where it is stored temporarily pending transfer to the master projector (3). A separate
source image buffer (14) transfers right eye image data to a second image buffer (12)
via the warping unit (20) where it is stored temporarily until both it and the master
image in buffer (11) are transferred to projectors 4 and 3 respectively and then
projected onto a screen to form a single 3D image. The warping is done on one of the
images to eliminate the need for high precision optics needed to match left and right
eye image sizes. The output from projectors 3 and 4 are coded (not shown) so that
only left eye images are seen by the left eyes of audience members and only right eye
images by their right eyes. Common coding techniques include, but are not limited
to, polarization, time multiplexing and color.
[0054] Figure 4 illustrates a second embodiment of the 2D mode of the invention
where the source image data is provided at a higher resolution (17) than what either of
the two projectors can project. This data is resized (19) for the master projector (3) to
match the desired display resolution of the projector. The pixel correspondence map
(21) is re-scaled (18) to match the high resolution source. This high resolution pixel
correspondence map is used by the warping engine (20) to sample the source image
data for the slave projector. The output of the warping engine (20) is matched to the
resolution that the projector can project. The resulting composite image projected on
the screen may have a higher fidelity because of the image quality improvement that
occurred as a result of resizing projected image data from higher resolution source
data. Figure 5 shows the 3D mode of this embodiment.
[0055] Figure 6 depicts the second embodiment of a 3D mode of operation of the
inventive system for use with electronic projectors that use orthogonal polarizations to
combine the three separate color channels (usually red, green and blue) into one
15

WO 2006/116536 PCT/US2006/015892
composite beam for projection. In this type projector the output light has one color
channel polarized orthogonal to the other two. If the 3D mode of the first embodiment
were used with this type of projector the images seen by viewers would be scrambled.
One color from the right eye image would be visible in the viewers left eye, while one
color from right eye image would be visible in the left. This scrambling would destroy
the stereoscopic effect.
[0056] In this second embodiment, a polarization converter (22) is positioned
within the optical path of projector 4; in this case it shown between the projector and
the screen. This converter (22) changes the polarization of all of the component color
channels of projector (4) into an orthogonal polarization state to the respective
component colors from projector (3). Those skilled in the art will realize that the
converter (22) is a 14 wave plate. Alternatively 1/4 wave plates with suitable
orientation of their fast axes may be placed in front or within each of the projectors to
achieve the same result. Further, the projectors themselves may be designed to emit
light that is orthogonal in polarization.
[0057] Without loss of generality, assume, for example, that it is the green channel
that is orthogonally polarized to the red and blue channels. In Figure 6 the green
color channels of both the left and right eye image data source buffers (13, 14) are
transferred to a warping unit 20. The red and blue image data is transferred directly
from the source buffer (13) to the projector buffer (i i) for the left eye image data and
between buffer (14) and buffer (12) for the right eye image data. The right eye green
image data is warped using the pixel correspondence map (21) so that when it is
projected by projector (3) it will align spatially with red and blue channel projected by
projector (4). The left eye green image data is warped using the pixel correspondence
map (21) so that when it is projected by projector (4) it will align spatially with red
and blue channel projected by projector (3). After warping the image data for the
green channels are transferred to projector image buffers (11,12) and combined with
the red and blue channels of the opposite eye image data before being simultaneously
transferred to the projectors (3,4). When the images are combined on the screen (2),
the right eye image color channels will all have the identical polarization, while left
eye image color channels will all have a polarization that is orthogonal to the right eye
image.
16

WO 2006/116536 PCT/US2006/015892
[0058] A significant benefit of this second embodiment of the invention is that it
overcomes the need to convert light from the master projector (3) into a single
polarization state and convert light from the slave projector (4) into a single
polarization state orthogonal to the projector (3). This conversion of polarization can
be accomplished either through the addition of a polarizer which has the adverse
affect of reducing brightness by approximately by a minimum of 50% because the
polarization axis of the polarizer must be 45 degrees from the polarization states of
the output light. Alternatively, a filter may be added that rotates the polarization of
one color channel while leaving the other channels relatively unaffected. In this case,
a polarizer may be needed to eliminate unwanted light to reduce cross-talk between
left and right eye images. The addition of this clean-up polarizer between has the
adverse affect of reducing brightness by approximately 15% and may require further
losses to ensure white point accuracy and color uniformity across the screen are
maintained. In the 2D mode of operation this second embodiment would operate in a
similar manner as the first embodiment shown in Figure 2. No special arrangements
would be required to account for the different polarizations of the color channels
because the viewers would not be wearing polarization sensitive glasses.
[0059] The embodiment shown in Figure 6 does not warp all the image
information to the slave projector, hence, cannot correct for aberrations in the slave
projector lens relative to the master projector lens. Figure 7 shows another version of
this second embodiment of the invention. The difference in this case is that all of the
color channels from the slave projector are warped in addition to one channel from the
master projector eliminating the need for high precision optics required for projection.
The ability to warp all colors of one eye image relative to the other eye permits the
introduction of an offset on the screen between the two images in order to improve the
performance of 3D presentations. Those skilled within the art will realize that are
other versions of this second embodiment that are covered within the scope of this
invention.
[0060] The various embodiments of this invention use similar components. It is
possible to switch between 3D and 2D embodiments by simply changing the source
images. To switch from a 3D mode to a 2D mode the source images would be
duplicated in both source buffers or one source buffer would be turned off and one
17

WO 2006/116536 PCT/US2006/015892
image would sent to all projectors as shown in Figure 2. Switching back to a 3D mode
would be the reverse process.
[0061] It is important to note that embodiments of this invention also include a
Master projector, as shown in Figures 2 through 7, that pre-distorts images to lessen
or eliminate first order distortions, such as keystone distortion and barrel or
pincushion lens distortion, of the projected image. The pre-distortion may be
achieved by preprocessing the source content or it may be a real-time correction that
occurs in the electronic data path of the Master projector. In one embodiment, the
first order corrections to the Master projector's images is accomplished through an
analytic description of the distortions. In a method identical to that previously
described, the slave projector images are then warped to achieve sub-pixel registration
with the Master projector images.
[0062] In any of the embodiments disclosed herein, there may be a need to modify
the amount of warping to achieve accurate pixel registration between the Slave and
Master projectors as the system warms up. Absorption of the light, as it propagates
through the optical elements in the system, can cause slight changes in relative pixel
locations of the Slave and Master projectors. These changes are measurable and are
found to be repeatable. In order to avoid waiting until the system reaches thermal
equilibrium before presenting light on the screen, the pixel correspondence map can
be updated, based on prior measurements, as the projection system runs in order to
ensure accurate registration throughout the presentation.
[0063] In cases where the thermal effects are not completely predictable, dynamic
changes in pixel alignment cause errors that can be compensated by updating the pixel
correspondence map while the system is displaying 2D or 3D presentations. The map
is updated by measuring test patterns that are adapted to be inserted in the
presentation images using a method not easily detected by the viewers, such as
placing the calibration points near the edge of the screen or making the points blend
into the content. There are numerous means by which the calibration information
may be embedded into the content many of which would require processing multiple
frames to extract the calibration signal. This calibration information is not limited to
the full array of dots that are used in the calibration prior to the presentation but may
18

WO 2006/116536 PCT/US2006/015892
be a subset that allows one to correct for small changes in pixel positions across the
screen as the presentation progresses.
(0064] In any of the embodiments disclosed herein the calibration camera (5) as
shown in Figure 1 may be used to calibrate other aspects of the projected image
including color and luminance. With appropriate test patterns the camera may be used
to measure absolute color and spatial variation in color of the image projected on the
screen. The warping unit may have additional algorithms to use the measured color
maps to correct the average color in any or all slave and master projectors, spatial
variation in color in any or all the slave and master projectors or match the color of
the slave projector to the master projector. Similarly, the camera may also be used to
measure the absolute luminance of the image on the screen, the spatial variation of
luminance or the difference in luminance between the projectors. The measured
luminance maps may be used by the warping unit to electronically adjust the image
data to change the average luminance of the projected image in any or all slave and
master projectors, adjust the luminance distribution in any or all the slave and master
projectors or match the luminance of the master and slave projectors.
[0065] Figure 8 is a flowchart illustrating the steps performed to calibrate the pixel
correspondence map (21) used by the warping engine (20) to warp images in various
embodiments of this invention. The procedure measures the position of a pixel in an
image fiom slave projector (4) relative to the corresponding pixel from the master
projector (3). Corresponding pixels have the same index co-ordinates in the projector
image buffers (11,12). Step 30 involves generating a test pattern (Figure 9) of a grid
of dots (60), each several pixels in diameter, exhibiting a Gaussian luminance
distribution. The calibration can be performed once with test patterns with white dots
or it can be performed for each color channel of the projectors. In this case, three sets
of test patterns are generated, one for each color channel. By repeating the calibration
for each color, this invention overcomes lack of convergence between color channels
that may occur as the system heats up under the high flux levels needed to illuminate
a large screen or that may occur as the system is thermally cycled due to repeated use
over time. Differences in convergence (alignment of the color channels in the
projected image) between projectors will result in errors in the pixel correspondence
19

WO 2006/116536 PCT/US2006/015892
map if each color is assumed to be the same. Repeating the calibration for each color
eliminates these errors.
[0066] In step 32, an image is taken of the screen 2 with the alignment camera (5)
with both projectors in an off state so that only ambient light not originating from the
projectors is measured. Steps 34 and 36 involve taking an image of screen (2) with
projector (3) and projector (4) displaying the test pattern (Figure 9) respectively. In
step 38, the ambient light image is subtracted from the images captured in steps 34
and 36 to eliminate the effect of ambient light. In step 40 the centroid co-ordinates of
each dot in the test pattern projected by each projector is calculated in the camera
image. The test pattern dots (60) cover multiple pixels in the camera image. As a
result the dots centroid co-ordinates can be calculated with sub-pixel accuracy using a
variety of methods. A simple method involves calculating the sum of each pixel's co-
ordinates in the images of the dot weighted by the luminance of the pixel. Those
skilled in the art will realize that there are other methods of calculating the centroids
with sub-pixel accuracy, including fitting the camera pixels luminance values to the
function used to generate the dot test pattern; in this case a Gaussian. Those skilled in
the art will realize that other dot patterns may also be used to achieve sub-pixel
accuracy.
[0067] In step 42 the centroids of the dots from one projector camera image are
matched with the camera image of the other projector. In step 44 the offsets between
these centroids, in camera pixels, are calculated by subtracting co-ordinates of the
master projector dot centroid from the slave projector dot centroid. A conversion
between camera pixel scale and projector pixel scale can be calculated by comparing
the measured number of camera pixels between dots to the number of projector pixels
between dots on the test pattern. This scale is used to convert the offsets into
projector pixels in step 46. The pixel correspondence map 20 is completed in step 48
by calculating the offset for each projector pixel by interpolating between the offsets
generated for each pair of test pattern dots. Steps 34 to 48 are repeated for each color
channel if required.
[0068] Often the screens used in a 3D presentation will have a significant gain; i.e.
screen is not Lambertian. The variation in brightness across this screen due to the
high gain can pose a significant problem for the alignment camera (Fig 1, 5), which is
20

WO 2006/116536 PCT/US2006/015892
of limited dynamic range in sensing luminance. As well, the location of the centroid
of each of the dots can be affected by screen gain thus leading to a slight error in the
pixel correspondence map in the direction of increasing screen gain. To over come
both of these problems, the illumination of the test pattern is adjusted to compensate
for the variations in screen gain to make the image seen by the camera of uniform
luminance. This reduces the dynamic range of the images taken by the camera
allowing the entire screen to be imaged with a camera with a single exposure. In
addition it removes the systematic error in the pixel correspondence map caused by
the variation in screen luminance. Alternatively by knowing how the gain varies
across the screen, this error may be fixed by accounting for the gain variation in the
calculation of the pixel centroids.
[0069] Referring now to Figure 10, an array of pixels projected by the master
projector is depicted schematically by grid 50. A second array of overlapping
projected pixels from a second slave projector is depicted by grid 51. The second
pixel array 51 is shown having a varying horizontal and vertical offset from pixel
array 50. This variability is typical of overlapping images projected from the two
separate projectors. Alpha-numeric indexing values are shown on the horizontal and
vertical edges of the arrays to help describe how the second pixel array (51) is warped
in accordance with the invention. Pixels projected from the slave projector are
identified by the prime symbol (').
[0070] To illustrate the process of image warping, the image data value of a single
pixel, D4' of the overlapping slave projector image (51), is discussed below. Pixel
D4' shown to be offset relative to the master projector pixel D4 by a pixel width of
approximately -0.1 pixel in the horizontal or x direction and -0.1 pixel in the vertical
or y direction. These relative offset values would in fact be derived from the
calibration process discussed previously and would be stored in the pixel
correspondence map (20). Together the offset values and a sampling function to be
described below serve to warp or transform the image so that it conforms more
precisely to the image projected by the master projector. In other words the slave
projector image data for each pixel is electronically shifted to compensate for the
physical discrepancies in the overlap of the two images.
21

WO 2006/116536 PCT/US2006/015892
[0071] The sampling function (49) is shown in Figure 11. In a preferred
embodiment the warping function is a Lanczos window function, but other types of
sampling functions may also be effectively used. Warping sampling function 49 as
depicted in Figure 11 as one dimensional for illustrative purposes.
[0072] The points D2, D3, D4 and D5 on the sampling function refer to the
relative contribution of each of those pixels in source image data (10), to warped pixel
D4' calculated by the warping unit (20). The coordinate system in Figure 11
represents the pixel position of the slave projector with origin at D4\ Note that the
position of D4 in Figure 11 shows the -0.1 pixel offset in the horizontal direction as
shown in Figure 10. Another pass of the sampling function (49) in the vertical or y
direction is required to calculate the final image data for D4'.
[0073] Since the offsets between the projector results from a number of different
types of distortion (optics variation, projection point differences) the relative offset
between pixels varies across the screen. Figure 12 shows the fractional pixel offset, in
the horizontal direction, between pixels on a flat screen from two projectors that are in
rough alignment to one another. The fractional offset ranges from Vi pixel as shown
in the white areas to 0 pixels as shown in the black areas. This image was created by
taking the calculated offsets between the projectors and subtracting the integer pixel
offsets. It is clear from this example that the image enhancement is not constant
across the screen. If the two projectors were more carefully aligned and if the
projection lenses were better matched to one another, the variation across the screen
may be decreased. However, if the screen is not flat but curved the distinct projection
points of the two projectors make it impossible to achieve a uniform offset across the
screen even if the projectors are perfectly aligned and the projection lenses are
perfectly matched.
[0074] Now described in more detail is the increase in image fidelity when two
projected images with an offset are superimposed. Figure 13 shows, in one
dimension, how a sinusoidal signal image is represented by sampled pixels that are
projected by the projection system. In each sub-plot the horizontal axis represents
pixel position and the vertical axis represents relative pixel illuminance. The top,
middle and bottom rows of the sub-plots show the sampled response to a sinusoidal
input signal at wavelengths of 10, 2 and 1.43 pixels respectively. The left, center and
22

WO 2006/116536 PCTAJS2006/015892
right columns of the sub-plots show the output of one projector, a second projector
offset by 1/2 a pixel from the first and the superimposed output of the two projectors
respectively. In the latter case, the resultant was divided by two to keep the results on
the same scale. For descriptive purposes, the sinusoidal signal amplitude, shown as a
dashed line in the plots, is taken to be unity. The signal amplitude at each pixel is
obtained by taking the magnitude of the sinusoidal signal at the center of the pixel as
is illustrated by the solid circles (80) in each of the plots. The solid lines represent the
illuminance output of the projection systems. These solid lines show that the fill factor
of each pixel has been assumed to be 100%; i.e. there are no gaps between the pixels.
[0075] The top row of Figure 13 shows that for a signal frequency below the
Nyquist limit of the sampling frequency, the sinusoidal input signal is well described
by the pixilated outputs of the individual projectors. The superimposed projection
system shown at the far right is seen to follow the input sinusoidal signal more
accurately than either of the two individual projectors.
[0076] The middle row shows the results when at the Nyquist limit. Here, the
projector output depends critically upon the phase of the sinusoidal signal relative to
the sampled pixels. The left plot shows a maximum amplitude output when the
pixels are sampled exactly in phase with the sinusoidal signal while the middle plot
shows no variation in output when the pixels are sampled 90 degrees out of phase
with the sinusoidal signal. Once again the plot at the right shows the response when
the two projectors are superimposed.
[0077] The bottom row shows results when the frequency of the sinusoidal signal
exceeds the Nyquist limit. Here it is clear that the lack of sufficient sampling for the
single projector systems, shown by the plots at the left and the middle, leads to a poor
representation of the original sinusoidal signal. However, when the outputs of these
projectors are superimposed, as shown in the plot on the right, the increased amount
of sampling yields an output that does a much better job of representing the original
sinusoidal signal, albeit at a lower amplitude.
[00781 Figure 14 compares the modulation transfer function (MTF) of a single
projector (90) and projection system consisting of two projectors superimposed with a
1/2 pixel offset (91). The graph shows the response of the system to sinusoidal
modulations ranging in spatial frequency from 0 to .9 expressed in units of reciprocal
23

WO 2006/116536 PCT/US2006/015892
pixels. It follows that the Nyquist limit (92) is given by a frequency of 0.5 in this
figure. The MTF curves were calculated by determining the best fit sinusoid, at the
same frequency as the input sinusoid, to the pixilated output signals examples of
which are shown in Figure 13. The best fit at each frequency was averaged over all
phases of the input sinusoidal signal wave relative to the projector pixels. This
allowed the determination of the average best fit amplitude reported in Figure 14 and
its uncertainty.
[0079] Figure 14 also shows that the MTF of both the single projection system
(90) and the one in which two images are superimposed (91) is very nearly the same
except near the Nyquist limit (92). As commented in the description of Figure 13, at
the Nyquist limit the single projector system's output is strongly dependent on the
phase of the sinusoidal signal relative to the pixels. The resulting uncertainty in
amplitude is shown by the error bar (93) located near the Nyquist limit in Figure 14.
It may come as a surprise that this error bar (93) extends to amplitudes greater than
the input frequency amplitude. This may be explained by looking back at the middle
left plot of Figure 13. A best fit sinusoidal signal, that minimizes the residual
between the fit and the output from the projector, is of amplitude greater than the
input signal.
[0080] As one moves away from the Nyquist frequency, the uncertainty of the
amplitude decreases as is illustrated on the plot by the smaller error bar (94) at f = 0.8
for a single projector. The fitted amplitude uncertainty for the system with two
projectors superimposed is, on average, an order of magnitude smaller than this error
bar (94) over the entire frequency range including frequencies at or near the Nyquist
limit.
[0081] Figure 14 does not show any improvement in MTF due to superposition of
two projectors with 100% pixel fill factor however we must consider image artifacts
in order to understand the benefits of superposition. Image artifacts may be quantified
by determining how well the pixilated output from the projection system is fit by the
sinusoidal function. The goodness of fit may be determined by the variance between
the pixilated output and the sinusoidal fit. Figure 15 shows a plot of the square root of
the average variance divided by the phase averaged amplitude of the fitted sinusoid
shown in Figure 14. Each line in the graph is labeled by the fractional pixel offset of
24

WO 2006/116536 PCT/US2006/015892
one projector relative to the other. The largest amount of image artifacts is shown for
the superimposed system with zero pixel offset between projectors (labeled 0.0). This
is equivalent to the amount of artifacts from a single projector. The least amount of
image artifacts is shown for the superimposed system with 1/2 pixel offset between
projectors (labeled 0.5). Near f=1, the artifacts of this system are approximately lOx
smaller than the zero pixel offset artifacts. At frequencies at and below the Nyquist
limit (labeled 100), the artifacts are approximately 2x smaller than the zero pixel
offset artifacts. The other lines on the graph show image artifacts for offsets of 0.4,
0.3 and 0.1 pixels are less than image artifacts with zero pixel offset.
[0082] Assuming superposition varies across the screen as shown in Figure 12, it is
reasonable to argue that the average performance may be obtained by averaging the
improvements over all possible pixel offsets. Figure 16 shows this average
performance compared to that of a single projector. Here it is seen that the artifacts
are 27 to 42% less than the artifacts from a projection system with zero pixel offset
[0083] Warping the slave image as described above and superimposing it with the
master image improves the image fidelity due to a reduction in image artifacts. This
enhancement results from the pixels from one projector being offset from the pixels of
the other projector and image warped to fill in the appropriate pixel value. The
enhancement is a maximum when the offset is 1/2 pixel and a minimum at 0 pixel
offset as show in Figure 15.
[0084] The discussion above has been for a system in which the pixels have a
100% fill factor. When pixels have a low fill factor such as is disclosed in U.S. Patent
No. 6,456,339, the mechanism by which resolution is increased is distinct from the
improvement in image fidelity that we have considered above. When multiple
projectors are superimposed and positioned such that the pixels of one projector emit
light within the gaps between pixels of the other projector, a true increase in
resolution occurs. In the particular case where two projectors are superimposed with
the pixels of one projector halfway between those of the other projector, it is not hard
to see that there is a doubling of the Nyquist limit and resolution of the system. The
invention disclosed here deals with the more subtle increase in perceived resolution or
image fidelity that occurs for systems with high fill factors where the light output
from the pixels overlap regardless of what offset exists. This transition occurs at fill
25

WO 2006/116536 PCT/US2006/015892
factors greater than 25% where the size of the pixel is greater than 50% of the
interpixel distance.
[0085] An additional benefit is possible when projected images can be made to
accurately overlap each other. By overlapping higher frame rate images projected
from two or more slower responding projection systems (such as LCD projectors) it is
possible to get a higher frame rate presentation without the degradation artifacts
associated with slower responding projection systems. This may be achieved while
maintaining or improving overall image brightness.
[0086] With this flexible projection system, a number of other embodiments of the
invention are possible. Figure 17 illustrates an embodiment of the invention where a
high frame rate display is achieved using two lower frame rate projectors with the
addition of mechanical or electronic shutter (26,27) that alternately blocks the light
from each projector. In this embodiment the shutter is located between the projector
and screen, but it may also be located in other positions inside the projector. In this
embodiment the high frame rate (doubled this example) projection is divided up; the
even frames are displayed by the master projector (3) and the odd frames are
displayed by the second projector (4) as depicted by the projection frame sequence
shown in Figure 18. The even frames are projected at a different time than the odd
frames; while the even frames are projected the odd frame projector image is blocked.
In this embodiment, the odd frames are warped relative to the even frames before
display. The warping can be accomplished by generating a pixel correspondence map
as described above and using this to warp the odd image frames.
[0087] This embodiment requires exact matching of the projector brightness and
color balance between projected images to avoid flicker. Techniques are already
known involving the use of a camera to feedback luminance spatial profile
information (magnitude and spectral composition), which is then used to modify the
signal to the electronic projector to shape an image luminance profile. In this case the
luminance profile of the slave projector (and/or the master projector) can be shaped so
that the luminance profiles of both projectors match. This luminance shaping
information can be stored and used to modify the image data to the projector as
outlined in U.S. Patent No. 5,386,253 to Fielding. One trade-off of using two lower
frame rate projectors to achieve a higher frame rate is that screen image brightness is
26

WO 2006/116536 PCT/US2006/015892
reduced since only the light of one projector is being projected onto the screen at any
one time. To compensate for this it is possible to regain some of the screen brightness
by increasing the duty cycle (53) of the projected image, as shown in Figure 19,
without significantly degrading the effect of the higher frame rate. The flicker (54)
caused by this increase in duty cycle will be at twice the frame rate of the projector, in
the preferred embodiment this would be 48Hz, which is not readily detectable by the
human eye.
[0088] Figure 20 shows another embodiment of the projection system designed
where motion artifacts are reduced in a multiple projector system. In electronic
projectors where the light output is sustained over the entire frame time, motion blur
results (such as, for example, as described in Kurita, SID DIGEST 2001, 986). By
forcing the projector to output light intermittently, the motion artifacts are reduced.
Figure 20 shows the frame period 67 to be divided into an exposure 65 and a blanking
interval 66. The blanking interval can provide two benefits. The first is to reduce
motion artifacts by forcing the projector to emit light intermittently. The second is to
eliminate artifacts due to the response time of the projector.
[0089] Graph 82 in figure 20 characterizes the extent of motion blur that becomes
apparent when a moving object is observed sequentially over a series of image
frames. In a simplified explanation of this curve the perceived image position error is
the amount of positional error between an actual moving object and what is shown on
the display. The perceived image position error increases from the moment the
display initially shows the moving object's position up to when the next image frame
displays the object's updated position. The degree of positional error in curve 87 of
the moving object will become apparent to the observer watching the moving object
on the display as image blur. The longer the period 67 of the image frame the more
blur the same displayed object in motion will appear to have to the display viewer. In
the case of slower responding image projectors, such as LC type, the image pixel
takes a finite time to transition to its value in the subsequent frame. The cross hatched
area 88 represents the range of errors the pixel value may have during this transition
period.
[0090] The timing shown in figure 20 shows the shutter blocking a portion 66 of
the projected screen image frame 67 prior to the subsequent frame. This action has
27

WO 2006/116536 PCT/US2006/015892
created a reduction in the amount of error that leads to image blur. In this situation the
reduction is equivalent to the area 89. The advantage will be a reduction in motion
blur, however, this is at the cost of reduced brightness.
[0091] The blocking interval 66 can also be shifted to block out a portion of the
projected image during part of or all of the pixel transition time. The advantage will
be a reduction of motion artifacts but also at the cost of reduced brightness.
[0092] Despite there being a reduction in image brightness to get a reduction in
motion artifacts this loss of image brightness is somewhat compensated by the
brightness gained from using multiple projectors to overlap images on the screen.
[0093] By adjusting the shutter blocking period or by shifting the shutter blocking
period or a combination of both it is possible for scenes with much motion to get an
overall perceived presentation improvement by finding the optimum trade-off
between reducing motion artifacts with image brightness.
[0094] Figure 21 shows another embodiment where the frame rate of the projector
is low enough that the intermittent projection of light results in flicker that is
noticeable to the human eye. In this case the exposure is divided into two periods 68
and 69 and likewise the blanking period would be divided into 70 and 71. In this
scenario by adjusting the shutter blocking period or by shifting the shutter blocking
period or a combination of both it is possible for scenes with much motion to get an
overall perceived presentation improvement by finding the optimum trade-off
between reducing motion artifacts with image brightness.
[0095] In situations where 2D images have been converted to a stereoscopic pair
of images for 3D projection, the synthetic eye is often rendered at a lower resolution
to reduce costs and render times. In this situation the higher resolution image would
be sent to the slave projector (4) via the warping engine to ensure the image quality
enhancement contributed by the slave projector is maximized.
[0096] In situations when a stereoscopic pair of images is acquired, one image in
the stereo pair can be used to electronically enhance the resolution of the other image
using an appropriate image enhancement algorithm. In this situation the enhanced
image would then be sent to the projector that uses the warped image (4) to ensure the
image quality enhancement contributed by the second projector is maximized.
28

WO 2006/116536 PCT/US2006/015892
[0097] A digital enhancement can be applied to a 2D image to improve its
resolution. This higher resolution image may be used in the embodiment shown in
Figure 4 to improve image quality.
[0098] As stated earlier, warping requires interpolation that causes a slight
degradation in image quality. In another embodiment of this application, image
fidelity is restored by applying digital enhancement algorithms to the warped image.
[0099] The foregoing description of embodiments of the invention has been
presented only for the purpose of illustration and description and is not intended to be
exhaustive or to limit the invention to the precise forms disclosed. Many
modifications and variations are possible in light of the above teaching. The
embodiments were chosen and described in order to explain the principles of the
invention and their practical application so as to enable others skilled in the art to
utilize the invention and various embodiments with various modifications as are
suited to the particular use contemplated.
29

WO 2006/116536 PCT/US2006/015892
That which is claimed:
1. A method of generating composite images with a projection system
comprising a first projector and at least a second projector, comprising:
generating a correspondence map of pixels for images by determining offsets
between pixels from at least a second image from the second projector and
corresponding pixels from a first image from the first projector;
receiving a source image;
warping the source image based at least in part on the correspondence map to
produce a warped image; and
displaying the source image by the first projector and displaying the warped
image by the second projector to create a composite image.
2. The method of claim 1, wherein the correspondence map is empirically
generated.
3. The method of claim 1, wherein the first image and the second image are the
same test pattern.
4. The method of claim 1, wherein the source image is superimposed on the
warped image to create the composite image.
5. The method of claim 1, wherein the second projector comprises multiple
projectors and multiple warped images are produced for display on the multiple
projectors.
6. The method of claim 1, wherein the source image and the warped image are
part of a two-dimensional presentation.
7. The method of claim 1, wherein the source image and the warped image are
part of a three-dimensional presentation.
30

WO 2006/116536 PCT/US2006/015892
8. The method of claim 1, wherein the correspondence map comprises offsets for
each projector pixel.
9. The method of claim 1, wherein the correspondence map is generated with
sub-pixel accuracy.
10. The method of claim 1, wherein the correspondence map is generated by
projecting a test pattern with the first projector, projecting the test pattern with the
i
second projector, and measuring positions of second pixels in the test pattern from the
second projector relative to corresponding first pixels in the test pattern from the first
projector.
11. The method of claim 10, wherein the test pattern is adjusted to compensate for
the screen gain.
12. The method of claim 10, wherein a separate test pattern is generated and a
separate correspondence map is calculated for each color.
13. The method of claim 10, wherein a test pattern is generated and a
correspondence map calculated for white light
14. The method of claim 10, wherein the test pattern comprises an array of dots
comprised of at least one pixel.
15. The method of claim 10, wherein the test pattern comprises at least one of
multiple pixel dots with a uniform illumination distribution, multiple pixel dots with a
non-uniform illumination distribution, or multiple pixel dots with a Gaussian
distribution profile.
16. The method of claim 1, wherein the correspondence map is updated during a
presentation.
31

WO 2006/116536 PCT/US2006/015892
17. The method of claim 1, wherein the source image is pre-distorted to remove
keystone distortion before display.
18. The method of claim 17, wherein the source image is pre-distorted to remove
lens distortion before display.
19. The method of claim 4, wherein power to lamps in the first projector and the
second projector is reduced.
20. The method of claim 1, wherein the first projector and the second projector
each comprise a plurality of color channels, the first projector and the second
projector each comprise projection lenses that output light with different polarization
states for the plurality of color channels, and the warping is performed on one color
channel from each of the first projector and the second projector.
21. The method of claim 1, wherein the first projector and the second projector
each comprise a plurality of color channels, the first projector and the second
projector each comprise projection lenses that output light with different polarization
states for the plurality of color channels, and the warping is performed on one color
channel from the first projector and all color channels for the second projector.
22. The method of claim 7, wherein a conversion between a two-dimensional
presentation and a three-dimensional presentation comprises changing a flow of
images to the first projector and the second projector.
23. The method of claim 7, wherein images in the three-dimensional presentation
are coded to allow stereoscopic images to be separated by glasses worn by viewers.
24. The method of claim 7, wherein the source image is of higher resolution than a
display resolution of the first projector and the second projectors.
32

WO 2006/116536 PCT/US2006/015892
25. The method of claim 24, wherein the source image comprises a three-
dimensional stereoscopic pair of a high-resolution image and a lower resolution
image, wherein the high-resolution image is warped.
26. The method of claim 24, wherein a three-dimensional image is electronically
enhanced and the enhanced image is displayed by the second projector.
27. The method of claim 1, wherein a source image is electronically enhanced and
the enhanced source image is displayed by the second projector.
28. The method of claim 1, further comprising shuttering the first projector and
the second projector to establish a blanking interval.
29. The method of claim 28, wherein the blanking interval's length of time is
capable of adjustment
30. The method of claim 29, wherein the adjustment is performed during the
display of the source image and the warped image.
31. The method of claim 28, wherein the blanking interval is synchronized to the
display of the source image and the warped image.
32. The method of claim 31, wherein timing of a beginning of the blanking
interval is capable of adjustment.
33. The method of claim 1 wherein, color is matched between the first projector
and the second projector.
34. The method of claim 1 wherein, luminance is matched between the first
projector and the second projector.
33

WO 2006/116536 PCT/US2006/015892
35. The method of claim 6 wherein, brightness of the composite image is greater
than brightness of the source image.
36. The method of claim 6 wherein, fidelity of the composite image is greater than
fidelity of the source image.
37. A projection system capable of generating composite images, comprising:
a first projector;
at least a second projector;
an alignment camera capable of recording at least a first image produced by
the first projector and at least a second image produced by the second projector used
for generating a correspondence map of pixels for images by determining offsets
between pixels from the second image and corresponding pixels from the first image;
a source image buffer capable of supplying a source image; and
a warping unit capable of warping the source image based at least in part on
the correspondence map to produce a warped image,
wherein the source image is displayed by the first projector and the warped
image is displaying by the second projector to create a composite image.
38. The projection system of claim 37, wherein the projection system is capable of
displaying cinema presentations
39. The projection system of claim 37, wherein the projection system is capable of
displaying two-dimensional presentations and three-dimensional presentations.
40. The projection system of claim 37, wherein the correspondence map is
generated by an empirical method.
41. The projection system of claim 37, wherein the first image and the second
image are the same test pattern.
34

WO 2006/116536 PCT/US2006/015892
42. The projection system of claim 37, wherein the source image is superimposed
on the warped image to create the composite image.
43. The projection system of claim 37, further comprising a first shutter for the
first projector and a second shutter for the second projector, wherein each shutter is
capable of inserting a blanking interval between frames to reduce motion artifacts.
44. A method of generating composite images with a projection system
comprising a first projector and at least a second projector, comprising:
receiving an image sequence comprising at least a first image frame and a
second image frame;
warping the second image frame relative to the first image frame to produce a
warped image frame; and
displaying the first image frame with the first projector at a first time interval
and displaying the warped image frame with the second projector at a second time
interval, wherein the second time interval is not the same as the first time interval.
45. The method of claim 44, further comprising shuttering the second projector at
the first time interval and shuttering the first projector at the second time interval to
temporally separate ihe display of the first image frame from the warped image frame.
46. The method of claim 45, wherein a shutter interval is capable of adjustment.
47. The method of claim 46, wherein the adjustment is performed when the first
image or the second image is being displayed.
48. The method of claim 44, wherein the shuttering of the second projector is
synchronized to the first image and the shuttering of the first projector is synchronized
to the second image.
49. The method of claim 48, wherein timing of the shuttering is capable of
adjustment.
35

Embodiments of the present invention comprise electronic projection systems and methods. One embodiment of the present invention comprises a method of creating composite images with a projection system comprising a first projector and at least a second projector, comprising generating a correspondence map of pixels for images by determining offsets between pixels from at least a second image from the second projector and corresponding pixels from a first image from the first projector, receiving a source image, warping the source image based at least in part on the correspondence map to produce a warped image, and displaying the source image by the first projector and displaying the warped image by the second projector to create a composite image.