VRML Behaviors - an API

Overview and Goals

This proposal presents a simple "thin" API consisting of the minimum set of calls that enable a language interpreter and browser to interact, while allowing for appropriate extensability.

An API for VRML must be.

1. Implementable - to ensure support across a variety of browsers
2. Work across multiple platforms
3. Support a number of languages
4. Avoid a M platforms by N languages implementation burden.
5. Work with new nodes, without changing the interpreters or browsers.
6. Work well with networking, while not being dependant upon the network protocol.

Dependencies

API - Object Model

Overview - two parts

1. Browser
   Reads behavior files via HTTP or whatever, loads interpreters, and passes behavior files to them, handles callbacks from running Applets including those that manipulate the scene graph, and passes events to them. Passes messages between Applets.
2. Interpreter
   A library or DLL that implements the language and can manage one or more running Applets passed to it by the browser. It will also translate calls from the browser into calls to specific Applets, and calls from the Applets into calls to the Browser.
3. Applets
   Applications running in the interpreter, executing code supplied in the behavior files. Think of these as being equivalent to C++ objects or Java classes.

What is NOT specified

• Threading models - i.e. whether each applet is in the browser thread, or its own thread, or a common thread for the interpreter.
• How applets running in the SAME interpreter call each other - they are free to take back doors for efficiency reasons.
• How clever browsers integrate languages they are good at - for example an SGI browser will probably implement Open Inventor Engines directly, while Sun might implement Java directly.
• Execution Model - several different execution models are possible, this doesn't specify how behaviors should be implemented. Documentation of one way of doing it will probably get added as an appendix.
• Networking Model - this API supports many different networking models, in particular it supports the three existing approaches.
  • Server based approaches such as World's VRML+ or Sony's.
  • Multicasting or DIS based approaches.
  • Direct extension of the API over sockets, as in SDSC's VRBS.

Few simple calls

Starting and stopping the Applet

Interpreter* LoadInterpreter(char* LanguageName)

Loads an interpreter (see below for what might happen if the interpreter isn't found), returns a pointer to the interpreter. The implementation of this method starts off in the browser code, where in a machine dependant way it will locate and load an interpreter. For example on Windows it could have a rule to find "java.dll".

This call should then call "LoadInterpreter" on the interpreter, to allow it to initialise or whatever is appropriate, for example some languages will want all calls to go through the same interpreter-object, others will want a seperate interpreter-object or OS thread for each behavior. This is hidden inside this call.

Note that this call happens once for EVERY applet loaded, however it may be a no-op after the first call.

Applet* Interpreter::Load(Node*, char* Program)

Load the program, sanitize it (e.g. remove calls to "eval"). The Node is passed so that the interpreter can create the appropriate pointers to the Node.
This call returns a pointer to an Applet, which is used to send messages to the Applet later, it returns NULL if the interpreter cannot run the program.

Applet::Init(Node*)

This has the interpreter run the program for the first time, it calls the "Init" function in the script. The Node* gives the program a reference, so that for example the same program can be run for a number of different Nodes, and so that the Applet can call back to the browser and tell it fields etc to change.

void Applet::End(Node*)

Is used to signal to the Applet that it should clean up and stop, after this point the Browser should not send any more calls to the Applet.

void Browser::Ended(Applet*)

Is used to signal the Browser that the Applet has cleaned up and exited.

Resolution

Open: It is an open issue as to what an Applet should be able to resolve, opinions varied between "anything" and "only-fields" in the Script node. I believe consensus is emerging on allowing resolution of fields in your own Prototype, and on other named Prototypes and certain fields on certain other named nodes.

Field* ResolveField(Node* relativenode, char* name)

This call is used to turn a textual name for a node into a cookie for that node e.g.

ResolveField(NULL,"Windmill.Vane")

The relativenode field is used so that an applet can refer to a field in its own object without knowing its own name.

Changing Fields

int Browser::SetField(Field* field, SFRotation(0,1,0,3.1415))

int Browser::SetField(Field* field, SFBool(TRUE));

If it is desired to avoid overloaded functions then we'll need to have seperate functions for each type e.g.

int Browser::SetFieldSFRotation(Field* field, SFRotation(0,1,0,3.14.15))

int Browser::SetFieldSFBool(Field* field, SFBool(TRUE))

SFRotation Browser::GetFieldSFRotation(Field* field)

SFBool Browser::GetFieldSFBool(Field* field)

int GetFieldMFNumber(Field* field)

To return the (possibly 0) number of values in the field

long GetFieldMFLong(Field* field, int n)

To get the n-th value from an MFLong.

int AppendFieldMFLong(Field* field, Long newvalue)

To add a new value to a MFLong - and return its index

DeleteFieldMFLong(Field* field, int n)

To remove the n-th value from an MFLong.

int findFieldMFString(Field* field, char* testvalue)

Enables some optimisation of string handling, by testing if testvalue is a member of the MFString.

Although the need to optimise means that we need to seperate Resolution and the GetField call, for terseness of code, a set of calls will also be supported of the form:

char* GetNamedFieldSFString(char* name);

Adding and Deleting Nodes

Node* Browser::CompileVRML(char* VRML)

Compiles some valid VRML into a snippet of Scene Graph (whose structure is browser dependant), returns NULL if VRML is invalid.

int Browser::AppendFieldMFNode(Field* parent, Node* child)

This requests the browser to append a node, created with CompileVRML or otherwise, as a child of parent.

int Browser::DeleteNode(Field* parent, int n)

Removes the pointer to the n-th child from the parent.

int Browser::InsertNode(Field* parent, int n, Node* child)

Insert new child at position "n", or if there are less than (n-1) children, insert at the end.

Ref Counters

void Browser::FreeNode(Node* old)

void Browser::FreeField(Field* old)

Render pausing and synchronization

void Browser::RenderGroup(int start)

    Browser->RenderGroup(TRUE);
    Browser->DeleteNode(tablekids,1);
    Browser->AppendField(tablekids,newobj);
    Browser->RenderGroup(FALSE);

Event handling

Registering Events

    Node* Browser::AddEventHandler(Node* eventOn, int eventType, 
                    Node actionObject, char* actionMethod, 
                    Field** actionParameters);
    Node* Browser::AddTimerEvent(int type, time_t time, Node* actionObject, 
                    char* actionMethod, Field** actionParameters 
    Node* Browser::AddFieldEvent(Node* eventOn, Field* field, Node* actionObject, 
                    char* actionMethod, Field** actionParameters);     

Receiving Events

    Applet::pick(Node* EventOn,Node* EventSentBy,SFTIME Time, 
                 SFVEC3F Coord, Field** actionParameters)

    Applet::do(char* actionMethod, Node* eventSentBy, 
                Field** actionParameters)

    <actionMethod>(Node* eventSentBy, Field** actionParameters)

On a non-object oriented language, the callbacks would need a first paramater to be the object the action is taking place on, i.e.

    do(actionMethod, Node* actionOn, Node* eventSentBy, 
                Field** actionParameters)

Removing Events

    Browser::RemoveEvent(Node* event);

Sending Events

    Browser::SendEvent(Node* destination, Applet* me, char* actionMethod, Field** actionParameters)

Examples

For example, to put a cube on two tables.

  {
    Node* table1 = ResolveField("Table1.top");
    Node* table2 = ResolveField("Table2.top");
    Node* newcube = CompileVRML("Cube { width 2 height 2 depth 2 }");
    AddFieldMFNode (table1, newcube);
    AddFieldMFNode (table2, newcube);

    // Now decrement Ref counts on pointers we have.
    FreeField(table1);
    FreeField(table2); 
    FreeNode(newcube);
  }  

Open issues / Further Work

• More work is needed on control, authorisation and security issues. At the very least the appropriate hooks are needed to allow these to be added later.

Platform specific parts

Apple

Windows (3.1, 95 and NT)

Unix

Sockets

Language specific parts

Java

Extensability to new languages

Obviously this brings up security concerns which is why the site should be specified in a config file, rather than in the VRML. Behind a firewall the MIS people can then specify a local site where only trusted interpreters are made available.

Open Issues

• Address Behavior node and inheritance
• Define data structures for all basic types
• Define structure of Field**
• Define a structure to pass MFFields - and add calls to return the entire MFVec3f etc.
• Are any new calls required to deal with newly defined nodes, or will the new PROTYPE and EXTERNPROTO keywords be sufficient along with Behaviors?

Acknowledgements

• BESoft. "A Format for Representing Behavior" http://besoft.com/bef/index.html
• Berne Roehl. Some thoughts on Behavior in VR Systems http://sunee.uwaterloo.ca/~broehl/behav.html
• Tom Meyer and Brookshire Conner of Brown. Adding Behavior to VRML. http://www.cs.brown.edu/research/graphics/research/papers/vrmlbehaviors.html
• John Moreland, David Nadeau of SDSC. The Virtual Reality Behavior System (VRBS). A Behavior Language Protocol for VRML.

<Flame>: Please put URL's and email addresses on your papers - it makes it hard or impossible to cite them! </Flame>

Thanks also to

• Rob Meyers, Gavin Bell of SGI. informal discussions about the activation nodes,
• to Chee Yu of SGI for comments on efficiency
• the rest of the Worlds VRML+ team (Alan Steremberg, Kyle Hayes, Jeff Close) for tearing apart the earlier versions :-)