User Interface Customization

BS Contact offers a set of new nodes proposed for the VRML200x standard which allow the world builder to set up a navigation unit on VRML-scope. Find an introduction and a description of the nodes in our paper A Generic User Interface Framework. Please note that latest errata to the node definitions and the implementation status can be found on this page. Additionally some browser extensions allow to configure BS Contact over a VRML-based UI.

  • Simple Beamto Navigation
  • Standard Navigation Modes implemented with the Camera-node
  • Examine an object
  • A quake like navigation
  • Please use the latest version ( >= 5.015) of BS Contact.

     

    Concepts

    Driving the Camera

    The basic function of the Camera node is to drive the camera at a certain speed which is specified by two SFVec3f vectors. In contrast to a viewpoint animation collsion detection is performed. Additional fields allow to realize an examine and an jumpTo mode.

    Intercepting User Input

    Various nodes have been introduced for intercepting all kind of possible user input. Thereby the basic browser functions as there are the navigation module and the handler of the VRML-sensors are competing with the new nodes for the user input. For example if the user does a mouse drag over a PlaneSensor no navigation action may take place, if the mouse drag is in an empty area the action is handled by the navigation module, whereas a VRML author wants the navigation module be be inactive if he intercepts the user input with a MouseSensor. A event cascade model ensures operability. First all events to the raw input sensors (the new nodes). The eventsProcessed eventIn determines if any given event flows further down the event cascade. If the event was not stopped by explicitly setting eventProcessed to TRUE, the events flow to the VRML-sensor event handler. In the end the built in navigation module receives the event. The list of raw input sensors is unordered, so if the event is stopped it is undefined which sensors will not receive the event.

    For simple applications the MouseSensor and the KeySensor suffice all requirements whereas for more elaborate cases we see the DeviceSensor with its open architecture as the right choice. In its basic configuration the DeviceSensor acts similar to the MouseSensor and the KeySensor, but allows further filtering of the input and gives additional information about the device, e.g. a certain keyboard group can be selected. The DeviceSensor itself acts as a shell for the actual event captured in a dedicated Event-node. It initializes the device with the given filter information or loads external device modules in case of advanced inpute devices. An external device gets a pointer to the event-node which can be the built in Event-node or a instance of a device specific event-PROTO.

     

    Node Proposal - Extension Nodes for Customizable Input Handling and Navigation

    Camera
    Camera { 
      eventIn        SFBool     set_bind
      eventIn        SFVec3f    xyz
      eventIn        SFVec3f    ypr
      eventIn        SFVec2f    yp
      eventIn        SFVec3f    moveTo
      eventIn        SFVec3f    orientTo
      eventIn        SFVec3f    examineCenter
      eventIn        SFBool     straighten
      exposedField   SFTime     duration        0  
      exposedField   SFInt32    examineRadius   0
      exposedField   SFVec3f    offset          0 0 0       			
      exposedField   SFBool     collide         TRUE 
      exposedField   SFBool     gravity         TRUE
      field          SFString   description     ""
      field          SFVec3f    upVector        0 1 0
      eventOut       SFRotation orientation   
      eventOut       SFVec3f    position
      eventOut       SFTime     bindTime
      eventOut       SFBool     isBound
    }

    All standard modes like WALK, SLIDE, PAN, FLY are realized by applying different values to the ypr and the xyz field. The xyz field specifies the speed in m/s, the ypr field is the angular speed in rad/s (yaw around the y-axis, pitch around the x-axis, roll around the z-axis).

    The speed vectors are applied to the local coord-system of the bound viewpoint. Values of the NavigationInfo-node as there are the speed and type are exceeded by the Camera-node.

    With examineCenter and examineRadius a virtual sphere is defined on which the camera is moved with yp angular velocity.After the examineCenter is set the examineRadius is computed as the distance between viewer and examineCenter. If no examineCenter is set (examineCenter 0 0 0) the browser inplementation should take a suitable value like the centre of the scene bounding box. The camera target (look-at point) is not changed, i.e. the viewer moves on the virtual sphere but does not look to the center automatically. To change the camera target see orientTo.

    Setting a moveTo point the camera is animated to the point in duration seconds. With the orientTo eventIn the camera turns to the given point in duration seconds.

    offset is an additional offset applied to the current camera position and orientation. Further movements will always be applied to the actual position without the offset. The value is specified in spherical coords with the first component as radius, second and third as angles.

    If the gravity field is set to TRUE, the built-in ground detection is enabled. To detect the ground, a ray hit test is performed with a ray defined by the down vector. Thus velocity vectors in contrary direction of the down-vector are disregarded. The collision-field switches the collision detection of the browser.

     

    DeviceSensor

    The DeviceSensor-node is capable of observing arbitrary input devices such as a six degrees-of-freedom mouse or a speech recognition system. The device data is wrapped in an Event node which already covers a considerable amount of possible event types. Special-purpose devices can replace the default implementation with their own Event node, guaranteeing maximum flexibility for the support of all possible input devices.

    DeviceSensor {
      exposedField     SFBool      enabled     TRUE
      exposedField     SFString    device      "<device>[[<number>]]"
      exposedField     SFString    eventType   "all", "mouseup", "positional", "absolute", ... 
      exposedField     SFNode      event       NULL
      eventOut         SFBool      isActive
    }

    The device field specifies the hardware device which is observed by the node. An optional number can decide which instance of the device - if more than one are present - should be taken. By setting

    device = "JOYSTICK[2]"

    all events generated by the second joy stick are reported by the DeviceSensor.

    <device> ::= STANDARD | JOYSTICK | SPACEMOUSE | ...

    STANDARD makes the DeviceSensor report events from the key board, mouse and drag & drop. With the eventType field a sub set of reported events can be defined.

    The eventType field is device dependend and determines which event types to oberserve. For the STANDARD device the eventType accpets multiple type values that are specified in W3C-DOM-Level2 [14] (see Apendix 7.1). For other devices the interpretation of eventType varies.

    For devices not mentioned here a new device name can be created. The node uses this string to identify the device driver plugged into the event-dispatcher. By default all events of the specified device are reported by the node.
    The DeviceSensor SDK provides information on how to write such a plug-in.

     

    Event

    The Event node is used implicitely by the "STANDARD" device. It is modeled after the W3C-DOM Events [14].

    Event {
      eventIn  SFBool     cancelBubble
      eventIn  SFBool     returnValue
    
      eventOut SFString   type 
      eventOut SFVec2f    screen
      eventOut SFVec2f    client
      eventOut SFVec2f    position
      eventOut SFVec3f    xyz 
      eventOut SFVec3f    ypr
      eventOut SFBool     altKey
      eventOut SFBool     ctrlKey
      eventOut SFBool     shiftKey
      eventOut SFInt32    keyCode
      eventOut SFString   dataType
      eventOut SFString   data
      eventOut SFInt32    button
    }

    For a detailed description of the fields refer to the DOM-model [14], see also Appendix 7.1. Nevertheless some additional fields can be found in the definition.

    For drag-and drop-events the dataType- and data fields have been added. The dataType-field can have following values:

    <dataType> ::= File | URL | HTML | Text | Image

    The data field delivers the URL of the data.

    For advanced input devices the built in Event-node can be replaced by a device-specific Event-node. The DeviceSensor must be initialized with the PROTO instance and the corresponding device. An external device-driver is loaded which has the knowledge of its event-proto and fills it with data when polled by the DeviceSensor-implementation. The device field identifies the device handler which can be built in as the keyboard and mouse handler or can be an external module for advanced input devices. If an external device handler can not be loaded by the browser the isActive-field will always be FALSE.

      
    PROTO SixDof
    [
        eventOut SFVec3f      position
        eventOut SFVec3f      rotationYPR  # rotation in yaw pitch roll notation
        eventOut SFInt32      buttons      # all buttons as bit mask
    ]
    {}
    
    DEF ds DeviceSensor
    {
        device "SPACEMOUSE"
        event DEF Data SixDof {}
    }
    

    Data can be routed away from the event field. This field fires every time a field on the the PROTO instance has changed .

        ROUTE ds.event TO inputScript.onSpaceMouse
    

    Alternatievely each field on the PROTO instance can be individually routed away.

        ROUTE Data.position TO inputScript.onPosition
    

     

    MouseSensor

    Analogous to the KeyboardSensor we define a MouseSensor that reports the events of the mouse.

    MouseSensor {
      eventIn      SFBool      eventsProcessed
      exposedField SFBool      enabled     TRUE  
      eventOut     SFVec2f     client
      eventOut     SFVec2f     position
      eventOut     SFBool      lButton
      eventOut     SFBool      mButton
      eventOut     SFBool      rButton
      eventOut     SFFLoat     mouseWheel
      eventOut     SFBool      isActive 
     }

    The client-field indicates the coordinate at which the event occurred relative to browser’s client window. The position-field indicates the normalized coordinate at which the event occurred.

    lButton, mButton, rButton events are generated as the buttons are pressed and released.

    The mouseWheel field indicates the distance rotated. If the wheel is rotated forward the values are positive, in the other case they are negative. The size of the value depends on the resolution of the mouse-wheel. Typical values are mutiples of 120.

    If the eventsProcessed is set to TRUE no default action associated with the event is excuted by the browser. The flag signals the browser that all events are processed by the node while its value is TRUE.

     

    KeySensor

    As a shortcoming of the VRML200x proposed KeyboardSensor [15] (see Appendix 7.2) the node catches all keyboard events. Even if certain keys that are normally used by the browser are not processed in the scene, the events get lost for the browser. Following the W3C-DOM-proposal [14] we propose to add a eventsProcessed field.

    KeySensor {
      eventIn      SFBool     eventsProcessed
      exposedField SFBool     enabled          TRUE
      eventOut     SFInt32    keyPress
      eventOut     SFInt32    keyRelease
      eventOut     SFInt32    actionKeyPress
      eventOut     SFInt32    actionKeyRelease
      eventOut     SFBool     shiftKey_changed
      eventOut     SFBool     controlKey_changed
      eventOut     SFBool     altKey_changed
      eventOut     SFBool     isActive
    }
    

    A KeySensor node generates events when the user presses keys on the keyboard. The KeySensor supports the notion of "keyboard focus"; if there are multiple KeyboardSensors and/or StringSensors in a world, only one will generate events at any given time.

    If the eventsProcessed is set to TRUE no default action associated with the event is excuted by the browser. The flag signals the browser that all events are processed by the node while its value is TRUE.

    keyPress and keyRelease events are generated as keys which produce characters are pressed and released on the keyboard. The value of these events is an integer which is the UTF-8 character value for the key pressed. The set of UTF-8 characters that can be generated will vary between different keyboards and different implementations.

    actionKeyPress and actionKeyRelease events are generated as 'action' keys are pressed and released on the keyboard. The value of these events are in Table 7.9:

    Table 7.9 Action key values

    KEY

    VALUE

    KEY

    VALUE

    KEY

    VALUE

    HOME

    1000

    END

    1001

    PGUP

    1002

    PGDN

    1003

    UP

    1004

    DOWN

    1005

    LEFT

    1006

    RIGHT

    1007

    F1-F12

    1008-1019

     

    shiftKey_changed, controlKey_changed, and altKey_changed events are generated as the shift, alt and control keys on the keyboard are pressed and released. Their value is TRUE when the key is pressed and FALSE when the key is released.

    The KeySensor is not affected by its position in the transformation hierarchy.

     

    Some Useful Browser Extensions

    For implementing a customized navigation the author has to care of setting the proper cursor-mode. The browser call
    browser.setCursor(SFString mode);
    mode: walk, slide, fly, beamto, examine, pan
    allows the content author to switch the appearance of the cursor. Thereby the browser automatically changes the appearance if the cursor is over a sensor node. While the navigtion unit is active no sensor cursors are shown.

     

    Appendix

    Attributes
    type
    The type property represents the event name as a string property.
    cancelBubble
    The cancelBubble property is used to control the bubbling phase of event flow. If the property is set to true, the event will cease bubbling at the current level. If the property is set to false, the event will bubble up to its parent. The default value of this property is determined by the event type.
    returnValue
    If an event is cancellable, the returnValue property is checked by the DOM implementation after the event has been processed by its event handlers. If the returnValue is false, the DOM implementation does not execute any default actions associated with the event.
    screen
    screen.x indicates the horizontal coordinate at which the event occurred relative to the origin of the screen coordinate system. screen.y indicates the vertical coordinate at which the event occurred relative to the origin of the screen coordinate system.
    client
    client.x indicates the horizontal coordinate at which the event occurred relative to the DOM implementation's client area.
    client.y indicates the vertical coordinate at which the event occurred relative to the DOM implementation's client area.
    position
    position.x indicates the horizontal coordinate at which the event occurred relative to the DOM implementation's normalized client area.
    position.y indicates the vertical coordinate at which the event occurred relative to the DOM implementation's normalized client area.
     
    altKey
    altKey indicates whether the 'Alt' key was depressed during the firing of the event.
    ctrlKey
    ctrlKey indicates whether the 'Ctrl' key was depressed during the firing of the event.
    shiftKey
    shiftKey indicates whether the shift key was depressed during the firing of the event.
    keyCode
    The value of keyCode holds the virtual key code value of the key which was depressed if the event is a key event. Otherwise, the value is zero. Currently the raw Win32 keycode is reported.
    button
    During mouse events caused by the depression or release of a mouse button, button is used to indicate which mouse button changed state. This is a bitmask with the values 1 for left button, 2 for right and 4 for middle. During a buttonup event, the mask of the button causing the event is reported.

    This extension is modeled after the W3C DOM Events.

    Event types
    click
    The click event occurs when the pointing device button is clicked over an element. This attribute may be used with most elements.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: screen, client, position, altKey, ctrlKey, shiftKey, button
    dblclick
    The dblclick event occurs when the pointing device button is double-clicked over an element. This attribute may be used with most elements.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: screen, client, position, altKey, ctrlKey, shiftKey, button
    mousedown
    The mousedown event occurs when the pointing device button is pressed over an element.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: screen, client, position, altKey, ctrlKey, shiftKey, button
    mouseup
    The mouseup event occurs when the pointing device button is released over an element.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: screen, client, position, altKey, ctrlKey, shiftKey, button
    mouseover
    The mouseover event occurs when the pointing device is moved onto an element.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info:screen, client, position, altKey, ctrlKey, shiftKey
    mousemove
    The mousemove event occurs when the pointing device is moved while it is over an element.
    • Bubbles: Yes
    • Cancellable: No
    • Context Info:screen, client, position, altKey, ctrlKey, shiftKey
    mouseout
    The mouseout event occurs when the pointing device is moved away from an element.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: screen, client, position, altKey, ctrlKey, shiftKey, button
    keypress
    The keypress event occurs when a key is pressed and released.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: keyCode, charCode
    keydown
    The keydown event occurs when a key is pressed down.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: keyCode, charCode
    keyup
    The keyup event occurs when a key is released.
    • Bubbles: Yes
    • Cancellable: Yes
    • Context Info: keyCode, charCode
    resize
    The resize event occurs when a document is resized.
    • Bubbles: Yes
    • Cancellable: No
    • Context Info: None