QTVRLingoEngine
©2000/01 Joachim Baur, Medienwerkstatt Schorndorf
A short table of contents:
Update of April 30th, 2001• improved lingo (faster
rendering and, even more important, now absolutely correct dewarping of
the panorama!). The update has of course been integrated into the main project, downloadable further below, but can also be applied separately for those of you who already dowloaded the 3+MBs of the prior (first) version. Click on these links to download just the updated "QTVRLingo"-castlib (just paste it over the existing one): Macintosh
Version (.sit-Archiv, 10 KB) |
The QTVRLingoEngine is a small collection of scripts for rendering interactive cylindrical QTVR-panoramas using Macromedia Director without the installation of QuickTime or any additional Xtras.
This is especially useful for web-based panos, other benefits are:
AND
The QTVRLingoEngine can NOT display
cubicVRs, as integrated in QuickTime 5.
For this purpose, please take a look at the famous ShockTimeVR
by Nonoche.
A CAVEAT FIRST:
This documentation is not in any way complete or self-explanatory ;-) It is meant to explain the basic principles and functions, but you should know your way around lingo if you want to really understand how it works or modify the engine for your own purposes! Sorry that I didn't comment on more of the code and the underlying math, but this is a non-commercial sparetime-project. Pay me and I'll answer anything ;-)
I had the urge to code the QTVRLingoEngine
because as a graphic designer I was partly unsatisfied with the original QTVRs
functionality, render methods and navigation methods.
So I wanted to have panos with a completely customized user-interface, with
animations in them, sound etc, and all lingo-based for maximum flexibility...
If you want to see what it's all about and how it works, I have made a sample director movie with 2 panos in it (one 360° and one 180°) including hotspots and some animations to play with.
Click below to see the Shockwave
8 version in your browser:
QTVRLingoEngine_example.htm
(300 KB)
You can download the unprotected
source-movie of the .dcr here:
Macintosh
version (.sit-archive, 1.3 MB)
Windows
version (.zip-archive, 1.5 MB)
2.1 QTVR-controls
Klicking on the buttons
|
|
|
|
|
causes the pano to move in that direction as long as the mouse is pressed.
You can also just click on the QTVR itself and drag in any direction to pan and tilt the pano.
A single click on this button
 |
shows the hotspot-areas (marked
in blue) while the mouse is pressed.
If the ShowHotspots-button is doubleclicked, the hotspot-areas stay visible
until the button is doubleclicked again.
 |
The SwingTo-function calculates the shortest distance between the current and the target position of the panorama. The motion is not very smooth and can certainly be improved, if anybody wants to take a look at it...
2.2 Rendering Settings
Except for those QTVR-control buttons, there are three sets of radiobuttons that control general rendering properties:
This sets the corresponding Director-property. The recalc-time should improve with this property set to true, more noticably so on slow machines ;-)
This controls the math that is applied when displaying the panorama:
 |
None just draws the sourcepict-image with linear scaling - the edges within the panorama-image are bend, tilting just moves the pano up and down |
 |
Partial corrects the edges in the pano,
so the image looks right when panning to the left and right. The tilt
buttons just move the pano up and down though |
 |
Full corrects the the edges in the
pano and applies perspective distortion when tilting. This is the default
mode and most CPU-intense, of course |
The QTVRs edges are feathered - this is achieved by a greenish bitmap with a feathered hole (alpha channel) in the center. If this masking bitmap is hidden (Off), you can see the complete sprite-quads.
2.3 Built-in hotspot-actions to play with
The two lamps in the first (green) room can be switched on and off independently from each other by clicking on them. The phone is ringing when the pano is turned towards it (the volume depends on the current pan-position). If you click on the phone, it stops ringing. You can click on the door to go to another room (blue) with a transition. This second room is just a partial pano with no hotspot-functionality in it except for the door back to the green room.
The QTVR-sourcePict is cut into vertical slices using imaging lingo. In the example.dcr, the sourcePict was 440 pixels high and 1440 pixels wide for the complete 360° pano (green room). This sourcepict was then cut up into 72 slices, each one 20 pixels wide. 20 sprite-channels with a slice-member attached to each one of them at runtime are used to render the pano on stage.
For each panorama used in the director-movie a lingo-object is created that has its own properties, so it is also possible to display more than one panorama at the same time.
So let's take a tour of the example.dir...
3.1 First a short description of the castlibs:
Contains the interface elements that are used in the example.dir - flash-buttons
(with built-in rollover- and active-states), text-fieldmembers and so on.
Contains the sliced up sourcePicts of the green room and the blue room.
The first 72 members (named greenRoom 1..72) were cut from the 360°-panorama
of the green room, followed by 36 members (named blueRoom 1..36) that were cut
from the blueRoom-sourcePict. The blue room is a partial panorma, covering only
180°.
After those you find the bitmaps that are used for the animations in the green
room: the members named "greenRoomDark" consist of the slices with
the two lamps turned off, the number after "greenRoomDark" shows their
position within the original panorama (the floorlamp ranges from slice number
25 to slice 40 and the desklamp-light is visible in slices number 56 to 62.
The other slices in the original sourcePict don't change when the lamps are
switched off).
The bitmaps are each 20 pixels wide by 440 pixels high and have16bit colordepth.
For the 300K-shockwave-version they were JPEG-compressed with a quality setting
of 40.
Contains the sliced up hotspot-sourcepicts of the green room and the blue room.
They have the same dimensions as the sourcePict-slices in the other cast (20
by 440 pixels) but their colordepth is 8bit. The palette of the bitmaps does
not matter, but it is vital that "Trim White Space" is DISABLED when
importing the hotspot-sourcePict!
First in the cast there are the 72 "greenRoomHotspots", followed by
72 "greenRoomHotspots ... showHS". Those are copies of the first 72
members, reduced to 1bit colordepth with all hotspotareas filled black. When
the "ShowHotSpots"-button is clicked during runtime, these "...showHS"-members
are brought onto the stage, and their sprites are assigned a blend value of
50 and a blue spritecolor - you can of course use any color you like to mark
the hotspot-areas.
All scripts and behaviors that are used in the movie for rendering and animating
the Lingo-QTVR.
3.2 The QTVRLingo-Cast
For each pano that is displayed on stage, on "startmovie" a different
lingo-object (instance of the parentscript "QTVRLingoEngine") is created
in memory. The references to these objects must be stored in global variables.
To create such a QTVR-object, the following lines are necessary:
gBlueRoom
= new(script "QTVRLingoEngine")
gCurrQTVR = gBlueRoom
initTheBlueRoom
gCurrQTVR.initNode()
The handler "initTheBlueRoom" sets all the "privately owned" properties of the blue room panorama. After these initial settings are complete, the "initNode()"-handler of the QTVR-object is called, which takes care of some other settings and calculations to prepare the display of this QTVR. The QTVR itself is not yet drawn on stage!
These scripts set properties of the QTVR-object stored in the global variable
"gCurrQTVR". I know that this is not the proper way of coding objects
(externally changing the internal properties of an object), but for convenience
sake and because I'm a graphic designer and don't have to know about these things
I decided that it is easiest for me to do it this way.
The "heart" of this whole thingie - here the actual math takes place
that renders the QTVR on stage...
3.3 How the QTVR-controls work
3.3.1 Some button is pressed (e.g. "Pan Left")
The button-behaviors themselves do not initiate a recalculation/redrawing of the pano directly. They just set flags to indicate user-interaction. So the "Pan left"-behavior just sets the pPanFlag of the gCurrQTVR-object to 1 when the mouse is down (you can also set the flag on mouseEnter of course, to create buttons that respond to rollover rather than to clicks)
3.3.2 The "exitFrame"-handler of the "frameLoop"-script is called
All the different types of movement are checked one after the other to see if the corresponding flags are set and to recalculate the QTVR accordingly:
checkPanning()
checkTilting()
checkZooming()
checkRotation()
The QTVR on the stage is not yet updated!
If the pSwingFlag is set to 1 (there is an automatic "SwingTo"-motion happening) all the other button flags are ignored until the swing is complete.
3.3.3 The "QTVRredraw()"-handler of the gCurrQTVR-object is called if there were any changes/movements
The QTVR-display on stage is refreshed
(similar to the "updateStage"-command)
If one of the check...()-handlers recalculates any values, it changes the pRedrawFlag
of the gCurrQTVR-object to indicate that an "updateStage" of the QTVR
is necessary. Depending on the type of movement, the pRedrawFlag is changed
from #none to either #spritesOnly or #quadsAndSprites.
#spritesOnly means that only a "Pan"-action has happened, in this
case it is not necessary to calculate any new sprite quads, just the members
of the sprites are exchanged so the user gets the impression of the pano spinning
to the left or right. All the other motions require a (more CPU-intense) recalculation
of the sprite-quads, which is indicated by the #quadsAndSprites-state of the
pRedrawFlag.
3.3.4 Checking for Hotspots
When "CheckHSRollOver" is called, the current mouseposition within the distorted spritequads has to be checked against the corresponding hotspot-slice to see if the mouse is over a hotspotarea. So after finding the corresponding hotspot-member (the 8bit-hotspot bitmap), "mapStagetoMember" is used to determine which pixel (x,y) the mouse is currently over. If the color of this pixel in the hotspot-bitmap is any other than white, we have hit a hotspot (the hotspot-number is determined by the palette index of the hotspot's color, therefore it doesn't matter WHICH palette the hotspot-bitmap uses)
3.3.5 Click-and-drag-functionality
In order for the familiar click-and-drag-spinning of a pano to work, the "SliceScript"-behavior must be attached to all the slice sprites. This behavior calls the "Trigger"-handler of the QTVR-object when the mouse is down on any of the slices. The handler first checks to see if the mouse was clicked on a hotspot and otherwise pans/tilts the pano in a repeat-loop towards the relative position of the mouse to the clickloc while the mouse is down.
What I missed most of all when working with Standard-QTVR-panoramas was an easy method of putting animations into the panorama-image - not just superimposed QuickTime-sprites or QTVR-objects, but background-animations that are rendered with the correct perspective/distortion when the user is panning, tilting and zooming the QTVR.
To make such animations possible, the QTVRLingoEngine uses an internal list (pSliceMemList) for reference to the single slice-sprite.members, so that any of the slices can be dynamically exchanged at any time.
4.1 Timed background animations
In order to create looping animations (e.g. the telephone ringing), it is easiest to create a timeout()-object that exchanges some members in the pSliceMemList each time it is called. The timeout-object for the phone is created in the "DisplayNode"-FrameScript:
timeout("telephone").new(500, #telephoneAnim)
so that it calls the "telephoneAnim"-handler
(located in the "HotSpot Handlers"-MovieScript) every 500 milliSeconds.
It plays the ringing sound once with each animation loop and changes the members
for slices number 63, 64 and 65 with each pass.
As soon as the telephone is clicked (or the blue room is drawn on stage), the
timeout-object is cleared and the animation stops.
4.2 Triggered animations / hotspot-actions
When the user clicks on a hotspot, the "hotSpotTriggerHandler" located in the "HotSpot Handlers"-MovieScript is called by the QTVR-object. The hotspot-properties are passed so the hotSpotTriggerHandler can decide what to do for each hotspot individually.
Hotspots can also be enabled/disabled at runtime by calling the "enableHS"-handler of the current QTVR-Object:
gCurrQTVR.enableHS(85,1) -- enables Hotspot number 85
gCurrQTVR.enableHS(85,0) -- disables Hotspot number 85
4.3 Transitions to other nodes
To prepare for a smooth node-switch, backbuffer-sprites are utilized. They are located in sprite-channels underneath the slice-sprites, so they are not visible as long as the slice-sprites are present on stage. The "switchNodes"-handler of the active QTVR-object fills the backbuffer with the slices of the new QTVR while copying the quads of the current slice-sprites and then jumps to the "goAnotherNode"-marker where the current QTVR slice-sprites are not present any more. This way, the new room can be entered using a transition and the exact same pan, tilt and fov-values of the current room.
Copy the "QTVRLingoEngine"-cast to your director-movie to start with
5.1 Setting up the Casts
Import your sourcePicts for the panorama and the hotspots (if there are any). I used separate castlibs for both sources, accordingly named "Slices" and "Hotspots". Please note when importing:
Once the sourcePicts are imported, use the "sliceSourcePict"-handler (located in the "Tools"-script of the "QTVRLingoEngine"-castlib) to let director automatically create the single slices using imaging lingo. Just type the following into the message window (substitute your own parameters for SliceCount, SrcPictMember and SrcPictCLib!):
sliceSourcePict SliceCount, SrcPictMember, SrcPictCLib
for example:
sliceSourcePict
72, 1, "Slices"
creates 72 new bitmap-slice-members
out of member 1 of castlib "Slices"
The hotspot-slices are created using the same method.
After creating the Hotspot-Slices, it is necessary to create copies of those slices which are just black-and-white if you're planning to use the "ShowHotSpots"-function in your project (if not, don't bother) Type into the message window:
createShowHSMembers FirstHSmember, HotSpotCLib, SliceCount
for example:
createShowHSMembers 147, "Hotspots",
36
creates 36 new showHotspot-slice-member by duplicating the 36 hotspot-slice-members
starting at member 147 of castlib "Hotspots"
There is another handler available
that builds a list off all the hotspot-numbers present in your hotspot-sourcepict
- useful, if you don't have a list of these numbers handy.
Type into the message window:
buildHSlist
And to complete the hotspot-section,
it is of course possible to modify (or even create) the hotspot-sourcepict using
director's own paint-window. Turn onion-skinning on to see the panorama-sourcepict
dimmed beneath the hotspot-bitmap. Now erase or draw anything you like...
If there is no hotspot-sourcepict present at all and you have to create it from
scratch, duplicate the panorama-sourcepict, change its colordepth to 8bit and
turn the "Trim White Space"-option off. Select All in the paint window
and fill with white. Now again use onion-skinning to paint hotspots over the
right areas.
5.2 Setting up the score
There are three groups of sprite-dummies
the QTVRLingoEngine works with (they must be arranged in this order in the score,
group 1 underneath group 2 underneath group 3). Dummy means that you can put
any odd bitmap in the sprite-channels and position it out of view (off the stage-area)
Group 1: sprites for the backbuffer-slices (only needed for switching
from one node to another)
Group 2: sprites for the panorama-slices (always needed)
Group 3: sprites for the showhotspot-slices (only needed for the "showHotspots"-feature)
All groups must consist of the same number of sprite-channels, you can omit groups 1 and 3 if you don't use these features in your project.
THAT'S IT. Almost ready to go - just a few lines of lingo to adjust:
Open the "initBlueRoom"- or the "initGreenRoom"-script. Change all the values to correspond with your own QTVR (pFirstSprite, pSpriteCount, etc). Then duplicate this member for each further panorama you want to display, give the handler a unique name and adjust the few params that differ from QTVR to QTVR (pFirstSliceMem, etc). In the "startMovie"-handler be sure to create an instance of the QTVRLingoEngine for each QTVRpano and call the corresponding init-handlers you just created.
6.1 Distorted (bend) edges within
the panorama-image
Compared to the 100%-straight edges that Apple's MoviePlayer renders, the
edges in QTVRLingo-panos are slightly crooked. I am not sure if this is due
to the math that I'm using to un-distort the sourcePict-slices being wrong or
due to the fact that the slices themselves have a width of 20 pixels - 1 pixel
wide slices would be ideal of course, but would also need 400 sprite-channels
instead of the 20 sprite-channels now used to render the QTVR on stage.
The April 30th-Update fixes this rendering bug!
6.2 QTVRLingo-sprites must be
placed underneath all other sprites in the score
The slice-sprites, the backbuffer-sprites and the showhotspot-sprites all
have to be in sprite-channels underneath all other interface-elements (or their
z-index has to be manipulated by lingo) plus the border outside the desired
QTVR viewrect has to be covered in some way to hide the off-viewrect parts of
the sprite-quads...
Alternatively, a MIAW with only a pano in it can be used.
6.3 Quad-calculation problems
When changing the FoV (zooming in and out) without any tilting, 1 pixel wide
vertical gaps are visible at certain zoomlevels, always at the same locations
within the viewrect. The math shouldn't be the cause of the problem, because
then the gaps would show up in the right half of the pano simultaneously, as
the caculations are the same for both halves (thanks to symmetry). Also, when
I do a "put" of the sprite-quads in the message-window, the adjacent
edges of the sprite with the gap inbetween have the correct quad-coordinates...
So I think the gaps must be due to internal rounding differences when director
draws the quads onto stage. Changing the floatprecision did not make any difference,
either.
My solution is to not allow a tilt of 1.0 (=no tilt), but always have a little
tilt (0.95/1.05) to prevent the gaps from showing up.