Moving worlds: behaviors for VRML

Mitra <mitra@mitra.biz>, WorldMaker
Yasuaki Honda <honda@csl.sony.co.jp>, Sony Computer Science Laboratory
Kouichi Matsuda <matsuda@csl.sony.co.jp>, Sony Computer Science Laboratory
Gavin Bell, <gavin@sgi.com>, Silicon Graphics
Chee Yu, Silicon Graphics
Chris Marrin, Silicon Graphics

This document can be found at http://www.mitra.biz/vrml/vrml2/behaviors/behaviors.html. It was last updated on Dec. 11, 1995.


Our goal is to design a set of extensions to VRML that give the user a much richer experience than is possible in VRML 1.0. We believe that the ability to interact with intelligent objects with simple behaviors, the ability to create animated 3D objects, and the addition of sound into 3D worlds will provide the rich, interactive experience that will enable a completely new set of applications of VRML.

Goals

Our design was guided by the following constraints (in rough order of priority):

Performance
We believe that speed is a key to a good interactive experience, and that it is important to design the system so that VRML browsers will be able to optimize the VRML world.
Scalability
Our goal is to allow the creation of very large virtual worlds. Any feature which limits scalability is unacceptable.
Composability
Related to scalability, we want to be able to compose VRML worlds or objects to create larger worlds. We assume that we will be able to compose worlds that are created by different people simply by creating a 'meta-world' that refers to the other worlds.
Authoring
We assume that sophisticated VRML authoring tools will be created, and wish to make it possible to perform most of the tasks necessary to create an interesting, interactive VRML world using a graphical user interface. We believe that VRML will not be successful until artists and creative people that are not interested in programming are able to create compelling, interactive VRML content.
Power
We must allow programmers to seamlessly extend VRML's functionality, by allowing them to create arbitrary scripts/applets/code that can then be easily re-used by the non-programmer.
Multi-user potential
We expect VRML to evolve into a multi-user shared experience in the near future, allowing complete collaboration and communication in interactive 3D worlds. We have attempted to anticipate the needs of multi-user VRML in our design, considering the possibility that VRML browsers might eventually need to support synchronization of changes to the world, locking, persistent distributed worlds, event rollback and dead reckoning in the future.

Overview

The essence of simple behaviors is changes to the world over time. With a single spectator interacting with the world, we identify three sources of change:

  1. User events from some device, translated into affects on the 3D world.
  2. Time.
  3. "Real-world" sources of input, accessed through some scripting/logic language, perhaps by a process running asynchronously to the VRML browser.

We are proposing a model for describing how changes are communicated between objects in the VRML scene. Objects are defined to send and/or receive "events" or "messages"; the names and types of events that can be sent or received is defined by each object. A generic object (the Script node) is defined to allow user-defined processing and generation of events.

This change communication model is combined with a new prototyping capability that allows the encapsulation and re-use of objects, behaviors, or both.

For some applications of VRML (such as database visualization, scientific visualization, and possibly shared, multi-user, distributed worlds) it is desirable to allow the creation and modification of VRML scene graphs. We address this need by allowing arbitrary changes to the well-defined parts of the world.

Description

The following documents describe in detail what is proposed:

This document attempts to describe the high-level concepts, not low-level details. This document also describes how the VRML browser must behave; just because that behavior is described in terms of a particular implementation do not assume that all VRML browsers must be implemented in that way.

Sensors, Routes, and Nodes

Sensors detect change. Several different types of sensors are proposed, to detect changes to time, user-input, changes to the position of the viewer with respect to the virtual world, etc.

When changes are detected, messages are produced. Those messages may be routed to other objects that need to know about those changes. Standard VRML objects may receive messages that change their fields; for example, a VRML Transform node may receive setTranslation messages to change its translation field. The setTranslation message will contain the new SFVec3f value for the translation.

In order to keep this specification simple, messages types are restricted to be the same as the existing VRML field types. For example, a Transform node may receive setTranslation messages of type SFVec3f, a PointLight node may receive setOn messages of type SFBool, etc. If it is necessary to communicate several pieces of information between two objects, then either several routes must be established (in which case several events will be received), or event types that contain several pieces of information must be defined.

Gavin: The message may contain a pointer to the object sending the message (in which case the receiving object will receive an event of type SFNode and will be responsible for extracting the information from the node), but I think that it will typically be much easier to just establish several routes and deal with multiple events.

For example, the ClickSensor will produce an isActive TRUE event when some geometry associated with it is pressed, and will produce an isActive FALSE event when released. Ignoring exactly how we specify on which geometry a ClickSensor should detect clicks, using a ClickSensor to turn on /off a light when the geometry is pressed/released would like like this in a VRML file:

DEF LIGHTCONTROL ClickSensor { }
   ...more scene stuff...
DEF LIGHT PointLight { }
   ...more scene stuff...
ROUTE LIGHTCONTROL.isActive -> LIGHT.setOn

Note that ROUTE creates a message path across the hierarchical scene graph structure; routes are not themselves in the scene hierarchy, even though they are part of the VRML file. In fact, the only requirement is that the ROUTE between two objects must appear after both objects.

Keeping ROUTE specification separate from the objects being connected has some important benefits. For example, to specify that two lights turn on when some geometry is pressed, that is trivial:

DEF LIGHTCONTROL ClickSensor { }
   ...more scene stuff...
DEF LIGHT_1 PointLight { }
   ...more scene stuff...
DEF LIGHT_2 PointLight { }
   ...more scene stuff...
ROUTE LIGHTCONTROL.isActive -> LIGHT_1.setOn
ROUTE LIGHTCONTROL.isActive -> LIGHT_2.setOn

Similarly, if a light should receive turn on/off messages from either of two sensors that is also trivial:

DEF LIGHTCONTROL_1 ClickSensor { }
   ...more scene stuff...
DEF LIGHTCONTROL_2 ClickSensor { }
   ...more scene stuff...
DEF LIGHT PointLight { }
   ...more scene stuff...
ROUTE LIGHTCONTROL_1.isActive -> LIGHT.setOn
ROUTE LIGHTCONTROL_2.isActive -> LIGHT.setOn

Gavin: If it was possible for two ClickSensors to send out contradictory isActive messages at the same time, then the above example might not generate predictable results. However, only one ClickSensor can possibly be active at any given time assuming a single user, so this won't be an issue, and some kind of resolution mechanism for ambiguities will be necessary anyway in a multi-user system.

Scripting

In general, changes from sensors will not be directly routed to a node. Usually some additional logic is necessary to decide what effect a change should have on the scene-- "if the vault is currently closed AND the correct combination is entered, THEN open the vault". These kinds of decisions are expressed as Script nodes that are inserted in between the Sensors and the VRML nodes in the scene. A Script node may receive messages, may produce messages, and may keep track of some information to remember what its internal state is based on change messages it has received.

For example, a Script node that decided whether or not to open a bank vault might receive vaultClosed and combinationEntered messages, produce openVault messages, and remember the correct combination and whether or not the vault is currently open. The VRML for this Script node would look like:

DEF OpenVault Script {
    # Declarations of what's in this Script node:
    eventIn SFBool vaultClosed
    eventIn SFString combinationEntered
    eventOut SFBool openVault
    field SFString correctCombination "43-22-9"
    field SFBool currentlyOpen FALSE

    # Implementation of the Script:
    scriptType "Java"
    script "processMessages(MessageList mlist) {
               for (int i = 0; i < mlist.getLength(); i++) {
                  Message m = MessageList[i];
                  if (m.getName() == "vaultClosed") {
                      currentlyOpen = FALSE;
                  }
                  if (m.getName() == "combinationEntered" &&
                         currentlyOpen == FALSE &&
                         m.getValue() == correctCombination) {

                         currentlyOpen = TRUE;
                         sendMessage("openVault", TRUE);
                   }
              }"

Gavin: The syntax of the script field will be different for each language supported. I'm also being fast-and-loose with the example; please consider it pseudo-code, and use it to get the broad ideas, not the specific syntax.

The API document describes a proposed Script API.

Script nodes and/or geometry may be gathered together using the proposed prototyping mechanism, creating geometries with encapsulated behaviors, re-usable behaviors, and parameterized geometries.

Open Issue: Independent Scripts, or Scripts associated with a particular node:

Mitra, Sony: Script nodes are usually associated with a particular node, and can be thought of as implementing the behavior of that Object. So routing a method to a Script Node can be considered as calling a method on that object.

Gavin, SGI: Like routes, Script nodes connect nodes across the scene graph hierarchy and should not be considered to be part of the scene graph hierarchy even though they are specified inside the VRML file.

Interpolators are equivalent to prototyped Script nodes, but are being proposed as built-in as part of the VRML standard because we believe that animations will be very common and performance of VRML systems will be better if animations are done with built-in objects instead of a general, probably interpreted, scripting language.

Changing the scene structure

Operations such as creating and deleting nodes in the scene, establishing or breaking routes between objects, and moving objects around in the scene hierarchy seem to be best handled by directly invoking methods on the objects involve instead of via "create/delete/route/etc" events; certainly in the case of "create" there is no object to send a "create" message to. These kinds of operations can be performed inside Script nodes, assuming that there are mechanisms by which a Script node can get handles or pointers to the nodes involved. The scripting specification discusses what types of operations may be used to access node pointers, and what types of operations might be allowed once a Script node has a node pointer.

Examples

Several examples are given in the examples document.