VRML Behaviors - an API

This document is part of a collection of VRML2.0 history at http://www.mitra.biz/vrml/vrml2/mw_history.html
This version was last updated on 9 October 95.
Feedback, and comments are welcome to: Mitra mitra@mitra.biz

Goals

The goal of this document, is to see if we can come up with a means to add arbritray behaviors to VRML objects, by means of a common API and/or Object model that meets the following criteria. Some of these goals may be hard to do together of course!

Overview

This proposal adds behaviors to VRML through two mechanisms.
  1. Two other proposals - Changes to Separator, and the addition of Sensors and Connectors, allow simple behaviors to be handled totally within the VRML file, and the browser.
  2. The addition of a behavior node, and a defined API allow more complex behaviors to be implemented as Applets in any supported language.

Dependencies

This proposal depends on a number of other proposals that are not yet set in concrete. These are extracted to seperate files because they are not dependant on this proposal, and are working their way through the standards process seperately.

Behaviors node

The Behaviors node allows an external program to be embedded. For example: 
    Behavior {
        fields [ SFString language SFBool READWRITE running 
                 SFString program MFString parameters ]
        language "java"                            # SFString READ the interpreter to use 
        running FALSE                              # SFBool READWRITE whether running currently
        lodState FALSE                             # SFBool READWRITE Set to true when 
                                                   # applet running but has had LodPause called  
        program "http://foo.com/myapplet.java"     # SFString READ the URL to the code
        parameters [ SFNode, "USE MyToaster.handleDown", SFLong, 5]  # MFString Paramaters to pass
    }
or 
    Behavior {
        language "foo"
        paramaters [" ... application written in foo ... "] # MFString 
    }
This is really simple, there is no difference between using this node for Java and using it for TCL, or Lifeforms. The second example exhibits a desirable goal, that of allowing trivial applications to be passed over as a string, rather than fetched seperately.

API - Object Model

Overview - two parts

The implementation of the API is split into two distinct parts, a Browser specific part supports initialising the language interpreter, passing it new behaviors to run, handling call backs from the interpreter to manipulate the scene graph or send messages, and passing messages to running applets.

The language specific part has to be able to load the interpreter, handle new behaviors - e.g. by spawning a thread or adding to a queue or whatever. A typical interpreter writer would also provide a library of callbacks that the applets could call to manipulate the scene graph. The interpreter also has (where supported by the language) to be able to receive messages to running applets.

What is NOT specified

In order to give language writers as much flexibility as possible this spec does not specify:

Few simple calls

The API breaks down into a very few calls. This document will assume C++ for defining the API although this does not need to be a requirement.

Starting and stopping the Applet

Starting the Applet is seperated into three stages, locating the interpreter, loading the program, and running the program.
Interpreter* LoadInterpreter(char* LanguageName)
Loads an interpreter (see below for what might happen if the interpreter isn't found), returns a pointer to the interpreter. Some languages will want all calls to go through the same interpreter-object, others will want a seperate interpreter-object or OS thread for each behavior. This is hidden inside this call.

Applet* Interpreter::Load(Node*, char* Program, Field** Parameters)
Load the program, sanitize it (e.g. remove calls to "eval"). The Node and Parameters are passed so that the interpreter can do variable substitution etc at this stage if it wants.
This call returns a pointer to an Applet, which can be used to send messages to the Applet later, it returns NULL if the interpreter cannot run the program.
Note how the parameters are passed as Field**, the browser should convert whatever parameters are given into temporary fields, so that they look the same to the Applet as a pointer to a field.

Applet::Run(Node*, Field** Parameters)
This has the interpreter run the program. The Node* gives the program a reference, so that for example the same program can be run for a number of different Nodes, and so that the Applet can call back to the browser and tell it fields etc to change.

void Applet::LODPause(bool paused)
This is sent to the Applet with a value of TRUE when the browser wants to pause the Applet, because it is no longer relevant to the scene, the applet may choose to stop completely, or to keep running in some "cheaper" mode, for example updating state, but not changing nodes, the browser should call the Applet again with a value of FALSE immediately before redrawing the node its responsible for, this enables the applet to update the scene graph if relevant.

void Applet::End()
Is used to signal to the Applet that it should clean up and stop, after this point the Browser should not send any more calls to the Applet.

void Browser::Ended(Applet*)
Is used to signal the Browser that the Applet has cleaned up and exited.

Resolution

Original this was all written using strings being passed back and forwards across the API, however as Chee Yu pointed out, this makes optimisation hard. So, in this draft, Strings are explicitly turned into Pointers by the browser allowing them to be cached by the Applet.
Field* ResolveNode(char* name)
This call is used to turn a textual name for a node into a pointer to that node e.g. 
ResolveNode(NULL,"Windmill.Vane")
Note that because of the way interfaces are designed, the only fields a behavior can access are those that are declared with READ or READWRITE in a Separator.

Changing Fields

It is desirable to be able to change arbritrary fields in the scene graph, as long as they are made public by declaring them with READWRITE. If we allow overloaded functions then this is easy. We have examples such as.
int Browser::SetField(Field* field, SFRotation(0,1,0,3.1415))
int Browser::SetField(Field* field, SFBool(TRUE));
In both these cases, field is obtained from the ResolveField call above.

If it is desired to avoid overloaded functions then we'll need to have seperate functions for each type e.g.

int Browser::SetFieldSFRotation(Field* field, SFRotation(0,1,0,3.14.15))
int Browser::SetFieldSFBool(Field* field, SFBool(TRUE))
Sometimes the Applet will need the value of a field, in which case we have for each SF* type.
SFRotation Browser::GetFieldSFRotation(Field* field)
SFBool Browser::GetFieldSFBool(Field* field)
MF* types are a little harder, I suggest.
int GetFieldMFNumber(Field* field)
To return the (possibly 0) number of values in the field

long GetFieldMFLong(Field* field, int n)
To get the n-th value from an MFLong.

int AppendFieldMFLong(Field* field, Long newvalue)
To add a new value to a MFLong - and return its index

DeleteFieldMFLong(Field* field, int n)
To remove the n-th value from an MFLong.

int findFieldMFString(Field* field, char* testvalue)
Enables some optimisation of string handling, by testing if testvalue is a member of the MFString.
Any approach that returns all the values with a single call may give us problems defining appropriate data structures that are cross-browser.

While being able to optimise means that we need to seperate Resolution and the GetField call, for terseness of code, a set of calls will also be supported of the form: 

char* GetNamedFieldSFString(char* name);

Adding and Deleting Nodes

With the new interface proposal, to be changed, nodes have to be pointed at by a field. Typically a node to be added will be compiled from the VRML first, and then a pointer to it will be added.
Node* Browser::CompileVRML(char* VRML)
Compiles some valid VRML into a snippet of Scene Graph (whose structure is browser dependant), returns NULL if VRML is invalid.

int Browser::AppendFieldMFNode(Field* parent, Node* child)
This requests the browser to append a node, created with CompileVRML or otherwise, as a child of parent.

int Browser::DeleteNode(Field* parent, int n)
Removes the pointer to the n-th child from the parent.

Message passing

Its essential that Applets be able to communicate with each other. If Applets are running within the same interpreter, then they can use any internal mechanism to do this. However message passing is required so that for example a TCL application simulating Holland can communicate with all the Java applications that turn the Windmills.

Messages are designed to be generic, to allow the passing of any data between two Applets, or between the browser and the Applet. The content of the data is defined by the receiving Applet, i.e. it can define its own external interface - this encapsulation should allow Applet's to be communicated with independantly of their internal structure.

int Browser::SendMessage(Node* destn, Applet* source, int MessageType, void* Data)
Passes a message to the browser to be passed along to the Applet associated with a specific Behavior Node. The return code is used to indicate an error, the Data may be destroyed after the call returns. The source is a "this" pointer which the Browser can turn into the Node* for the source.

int Applet::SendMessage(Node* source, int MessageType, void *Data)
Is called on the receiving Applet. The Applet can use the MessageType to determine what follows in the Data. The source is the sending Applet.

Ref Counters

It is assumed that some browser dependant mechanism exists for keeping a count of References to a data structure, because of the asynchronous nature of the browser and Applet, it is essential that the browser not destroy a data structure which the Applet still has a pointer to. For this reason any time the Browser resolves a string into a pointer to a data structure, it increments the RefCount on that data structure, allowing the Applet to be sure that it will stay in existance. This means that the Applet must explicitly free any pointer it has, so that the Browser can delete the data structure if nothing else points to it.
void Browser::FreeNode(Node* old)
void Browser::FreeField(Field* old)

Pseudo Nodes

Its useful for an Applet to be able to query the browser to be able to find out what version it is, and what extensions it supports. This is done by a set of standard named fields that the browser can read, so for example to read the name of the Browser, the behavior would call 
     GetNamedFieldSFString("_Browser.name");
These access a set of pseudo nodes in the browser. The definitions of these need fleshing out, but the following set are offered to start the thinking going.

Event handling

The Applet gets automatic notification of certain events, for example moving in an out of LOD, Starting and Stopping, but it also is going to need to depend on certain fields. To avoid the need for polling, we need a means for the Applet to register for particular events.

Most (possibly all) cases are handled by a general purpose call. 

    AddEventInterestField(Field*)
e.g 
    Field* temp = ResolveField("_Browser.position);
    AddEventInterest(temp);
This tells the browser that the Applet is interested in any changes of behavior of a particular field, in the example given, the Applet wants to know whenever the user's camera position changes.

When this field changes, the browser calls the Applet with, 

    SetFieldSFVec3f(Field*, SFVec3f newvalue);
When the Applet is no longer interested in changes to a field, for example because it has been called withLODPause(), it should declare disinterest with a call to: 
    RemoveEventInterest(Field*);

Examples

These examples are written in C, obviously they would be written in Java or TCL or whatever language was being interpreted.

For example, to put a cube on two tables. 

  {
    Node* table1 = ResolveField("Table1.top");
    Node* table2 = ResolveField("Table2.top");
    Node* newcube = CompileVRML("Cube { width 2 height 2 depth 2 }");
    AddFieldMFNode (table1, newcube);
    AddFieldMFNode (table2, newcube);

    // Now decrement Ref counts on pointers we have.
    FreeField(table1);
    FreeField(table2); 
    FreeNode(newcube);
  }

Platform specific parts

This section needs filling out for each platform. Its probably going to be very similar between different applications on the same platform.

Apple

Apple Events are an obvious, but probably wrong, choice - an Apples Guru is needed here.

Windows (3.1, 95 and NT)

OLE or DLL are obvious choices, but a windows guru is needed here!

Unix

In Unix, shared dynamically linked libraries are probably the easiest way to go.

Language specific parts

Different languages will have specific implementation concerns, here are some comments, but this section needs replacing by people with expertise in those languages.

Java

Java requires that it run as a seperate thread, and wants to poll for events. The threading model is not specified in this API, the suggestion is that the API is implemented as a small library which just queues events for interpretation by the regular Java interpreter running in a seperate thread.

TCL

BESoft

Physics

Open Inventor

Extensability to new languages

One goal is to be able to add languages, with any of the above approaches, the interpreter is a seperate piece of code. This enables dynamic, or at least user-initiated, downloading. As a simple (but not neccessarily sufficient) solution, its suggested to create a known location (or one that can be set up in a config file) from which an interpreter can be downloaded. For example, if running on Windows, and needing Java, then the interpreter could be obtained from: http://vrml.org/windows/java.dll.

Obviously this brings up security concerns which is why the site should be specified in a config file, rather than in the VRML. Behind a firewall the MIS people can then specify a local site where only trusted interpreters are made available.

Synchronization and triggering

Various synchronisation mechanisms will be required by the Applets, the Activation nodes (Sensors plus Connect and Trigger) provide a means to trigger when a behavior is run. Another mechanism has Applets reading and writing fields in a synchronisation object, (a Separator with some READWRITE fields). For example: 
    DEF MyDoor Door {
        isA Separator
        fields [ SFBool open ]
        Behavior { 
            language "java" 
            program "URL to some code calling SetFieldSFBool(MyDoor.open,TRUE)" }
    } 
    DEF MyLights Lights {
        isA Separator
        fields [ SFBool on ]
        Switch { 
            whichChild USE MyLights.on 
            { DirectionLight { 0 0 -1 } }
        } 
        DEF Code Behavior {
            language "java"
            program "URL to some code calling GetFieldSFBool(MyDoor.open) and
                then SetFieldSFLong(MyLights.on,1)"
        }
    }

Message passing from VRML

For simple messages, it would be nice to be able to pass them directly from the VRML, especially when all they require is something as simple as the Connect Node.

I recommend we create a simple syntax to pass a single string value to a method. 

   Trigger {
        condition Door.Open.Value       # when door value changes to 1
        action "send Light.Code,5,On"   # Send method 5 to Light.Code
                                        # with a data field of "On"
    }

Areas not covered

This proposal deliberately does not cover some of the areas that are crucial to VRML2.0, this is deliberate, since in some of the proposals we've had it is hard to seperate parts which are really orthogonal.

This proposal doesn't cover:

Open Issues

The following problems need sorting out before this can be finalised:

Acknowledgements

This paper condenses good ideas from a lot of other people's work. Thanks to the ideas from the papers from A document contrasting these papers is under construction - look at http://www.mitra.biz/vrml/vrml2/vrml-behavior-contrast.html

<Flame>: Please put URL's and email addresses on your papers - it makes it hard or impossible to cite them! </Flame>

Thanks also to