VRML Behaviors

This document is part of a collection of VRML2.0 history at http://www.mitra.biz/vrml/vrml2/mw_history.html
This version: Mitra <mitra@mitra.biz>, 2 Dec 95.

History and Dependencies

This paper draws heavily on discussions with, and the proposals by Sony [6], [7] [8] and SDSC [9], and is an attempt to merge the best ideas from these and my own papers.

This paper also reflects a number of concerns brought up by SGI, both in their behaviors paper [11], and in meetings.

Their are several other papers that should be considered part of this proposal, they are mostly orthogonal to this paper.

• API: the API between browser and Interpreter/Apple
• TimeSensors: Time event generator
• Sensors: Click and Drag sensors etc
• Interpolators: for animation
• Pseudo Nodes: Access to browser state.
• Prototypes: Prototyping and instancing.
• Frames: A proposal for changing Separators (this paper will need some minor rewriting if the Frames proposal is accepted.)

Changes since 12 Nov 95

• First - check out the other papers that are part of this proposal, much stuff has been moved there.
• Removed changed Separator version of SciptNode, the Separator change is likely to take a different form as documented in the frames proposal.
• Added eventIn and eventOut to ScriptNode to support authoring and optimisation
• Added optimisation flags to ScriptNode to allow control of functionality v. optimisation.
• Added discussion of culling and optimisation
• Extended FieldEvent to be equivalent to ROUTE - either could be chosen.
• Discuss replacing tiny language with FieldEvent or ROUTE.
• Removed EventHandlers - they are replaced by the Sensors to handle things like PICK and DRAG, and by the flags on Script nodes that control events such as CULL and LOD.

Overview

Firstly - it appears that we are all looking for a Object-Oriented approach, but I think older proposals diverged from this to a certain extent. To be O-O the author should probably be thinking something like ...

An Object has:

• A collection of geometry, data and properties - essentially the existing VRML plus some parts of the Interface proposal.
• A collection of behaviors or methods
• A set of event handlers that link actions on the objects to behaviors/methods on this object or others.

If our O-O model is essentially that a Separator is our Object (since any lower level of detail may have been thrown away by the browser), then this could get fairly easy. Basically we have to:

• Indicate the set of behaviors associated with the object (or class)
• Attach a collection of event handlers to it

The various Nodes in SDSC, Sony and my proposals composite some or all of three sections,

• A way to specify which behavior file to use - this specifies a filename and a scriptType.
• A way to specify the function to call - specifies the procedure, and parameters
• A way to specify an event - specifies eventType (optional parameters)

ScriptNode

    Script { 
        behavior  "http://foo.com/bar.class"  ; MFSTRING
        scriptType "JAVA" ; SFSTRING
        eventIn SFSTRING name	
        eventIn SFVEC3F lookfrom
        eventIn PICK selected
        eventOut SFSTRING lookto
        optimisation  SFBITMASK (CULLEVENTS | LODEVENTS | WORLDINOUT |  
                          UNKNOWNOUTPUTS | ALWAYSEVAL )                 
    }

• The behavior field specifies the URL of a script to use, e.g. "http://foo.com/mycode.java",
• The scriptType can be used to specify the Mime type of the script e.g. "application/java". If unspecified it will be inferred by the transport layer.
• The eventIn fields are used to specify field events that the script can handle, they have two uses
  1. They can be used to make the API of the script available to a visual authoring tool so that it can make connections between a field on another node, and a particular function call to the script.
  2. On some (OI) systems they will allow optimisation to be done, since if this Script's object is not visible, and it is set to allow AUTO culling, then the source of the eventIn may not need to be evaluated.
  Note that multiple outputs can be fanned into a single eventIn. Also note the example of specifing the type as PICK, which is a hint to the authoring system that it should only attach this eventIn to a ClickSensor.
• The eventOut fields are used to specify outputs from the script, if all the outputs are known then some optimisation can be done. (See Culling rules)
• The optimisation flags are used to help smarter browsers decide when to evaluate a script.
  • CULLEVENTS - says that the script should receive events when the browser would otherwise cull it, for example because it went out of view.
  • LODEVENTS - says that the script should receive events when the browser would change the LOD level of its object. These will only occur when the behavior is attached to a LOD node. Issue, would it be better to allow this to be independant of visual LOD, for example with a LODSensor
  • WORLDEVENTS - says that the script should receive events when the browser wants to load or unload the world. This can be used to save state for example.
  • UNKNOWNOUTPUTS - says that the script may effect other parts of the scene graph than its object and its outputs. See Culling rules.
  • ALWAYSEVAL - says that no matter what any other flags say, evaluate this script whenever its inputs change.
  There is a really tough balance to be struck between functionality, and the ability to optimize, it is our belief that these flags allow the author to get the functionality at the cost of optimisation only in the cases where this cost has to be born. Note that the defaults are chosen to support full optimisation when the author does nothing.

Examples

A cube that changes color when picked and again when released

    This example needs rewriting            

Turning the lights on when the switch is grabbed

    This example needs rewriting

Inline behaviors

Short scripts

Trivial Methods

EventHandler {
    eventType GRAB
   actionMethod "set SFBOOL on TRUE"
}

    set FIELDTYPE FIELDNAME CONSTANT

    load URL

Open issue

    EventHandler {
        eventType GRAB
        actionMethod "setOn"
        actionParameters "SFBOOL TRUE"
    } 

For each event we have to define the paramaters that are passed across to the script in the case that a actionMethod is not specified. All events, and messages, include the following information.

• EventOn: Where the event first occurred, e.g. the object which was picked. Note that in the case of Cascading, this might be different from where the event gets handled.
• EventSentBy: This is used if the event is sent from some other object than that which the script is attached too, for example if a switch sends an On message to a light.
• Time: Open: SGI will probably want a time added to each event

The first set are events sent by sensors. (The ... refers to the parameters sent with all events, and listed above).

• PICK pick( ... , SFNODE Picksensor)
  When the mouse is clicked the event generates a pointer to the PICK EventHandler, which can be interogated for the relevant Information.
• GRAB,RELEASE grab(..., SFNODE Grabsensor) and release (..., SFNODE Grabsensor),
  When the mouse is held down on the object, and when it is released.
  The OS will decide whether the mouse-click is short enough to send as a PICK event, or whether to send GRAB and RELEASE. Also note that DRAG events only occur inside GRAB,RELEASE pairs.
  Open issue: Some questions have been raised as to whether the Application should get the raw events (mouse clicks and times) or processed events (Pick, Grab, Drag). The advantage of the latter is that it would then work across different input devices, and be consistent with OS settings (such as the double-click time selected by the user). There are some questions as to whether it is possible to map events for all OS's and most input devices into this metaphor, or whether for example a Data Glove needs a whole different set of events.
• DRAG drag(..., SFNode DragSensor)
  Sent when a GRAB-ed object is moved, and where the pointing devise indicates it should be moved to, and - where relevant - the orientation it should be rotated by.
• ENTER, LEAVE enter(..., SFNode VolumeSensor) or leave(..., SFNode VolumeSensor)
  Sent when the viewpoint enters or leaves a sensed volume, or collides with a sensed object.
• COLLISION collision(..., SFNode CollisionSensor)
  Triggered when the user collides with the object, note this is independant of the changes being made to the Separator to incorporate collision detection. The Collision event is triggered when the collision detection logic prevents the user entering an object. It is never called in conjunction with Enter and Leave. It indicates what part of the user collided with what part of the object, and the orientation of the user (relative to the object?).
• TICK tick(..., SFNode Tick)
  Sent when from a TimerSensor.

• LODIN and LODOUT lodin(..., SFLONG range), lodout(..., SFLONG range)
  When the object is being signalled that it is not being drawn for LOD reasons. Typically a behavior may use these events to suspend behavior, or to reduce the CPU cycles this behavior absorbs - for example a mobile object might keep track of where it was, but suspend updating the geometry.
• WORLD_IN and WORLD_OUT worldin(...) and worldout(...)
  These are called when the world is loaded, and when it is unloaded. Note these are distinct from the Init and End methods which are called when the applet is loaded and unloaded.
• FRAME frame(...)
  Sent once per frame.
• CULLIN and CULLOUT cullin(...), cullout(...)
  Triggered when rendering is being culled, a behavior may suspend, or scale back, its activities. Note this is different from LODIN and LODOUT.

Field events

FieldEvent {
    eventOn   ""  ; SFSTRING Node name to watch, default to this Separator
    field     ""  ; SFSTRING field to watch

    actionOn, actionMethod, actionParameters  ; as for Sensors     
}

This could be extended to allow more complex tests - for example only trigger when this field goes to TRUE, however is is not clear that the added complexity is worth it, as compared to having the full power of a programming language available in the behavior. Typically the field being watched will be passed to the Applet in one of the actionParameters.

Open issue

     ROUTE foo.alpha -> ascript.timein

    FieldEvent {
        eventOn        foo
        field          alpha
        actionOn       ascript
        actionMethod   setTimein   # Or whatever naming convention is used.
        actionParameters  SFFLOAT alpha
    }

Timers

_Time

_Time {
    realTime    ; SFTIME - the time asserted by the machine's clock. 
                ; This might not be accurate, especially on low-end 
                ; machines.
    wallClock   ; SFTIME - in a simulation this is the time being 
                ; simulated in most circumstances this will be the 
                ; same as realTime.
    elapsedTime ; SFTIME - time since this simulation started, 
                ; or the world was loaded. (This might not be useful)
}

Culling rules

1. If a script has an output that is visible, it must be evaluated when it receives events.
2. If a script has the UNKNOWNOUPUTS flag set, meaning that it might effect other things in the scene graph, then it must be assumed to be connected to a visible object, so rule 1 applies.
3. If a script has the ALWAYSEVAL flag set, then it must be evaluated whenever it receives events. (this might be equivalent to UNKNOWNOUTPUTS, but I'm not sure it will be for all systems).
4. If a script would be evaluated, then any Node that it depends on will need evaluating.

Further Work

Nasties

• There is no way to have multiple scripts on the same object, or to specify which program file that an Sensor should use.

Consider

• Handling key input - e.g. grabbing focus, and directing key events?
• The prototype node needs extending to support local variables, scoped by the PROTO, or some other way for items inside a prototype to access each other without exposing themselves outside the prototype.

References