FirstStage Help Centre
FirstStage Product Website
  • Welcome
  • Release Notes
    • Release 35
    • Release 34
    • Release 33
    • Release 32
    • Release 30
    • Release 28
    • Release 27
    • Release 26
    • Release 25
    • Release 24
    • Release 23
    • Release 22
    • Release 20
    • Release 19
    • Release 18
    • Release 17
    • Release 16
    • Release 15
    • Release 14
    • Release 13
    • Release 12
    • Release 11
    • Release 10
    • Release 9
    • Release 8
    • Release 7
    • Release 6
    • Release 5
    • Release 4
    • Release 3
    • Release 2
  • Basics
    • Getting Started
      • Setting up a Quest headset
      • Starting FirstStage for the first time
      • Controllers
      • Installing on Quest (Standalone)
    • The Office
    • Projects
      • Project Team
    • Scenes
    • Core Tools
      • Grabber
      • Navigator
      • Duplicator
      • Selector
      • Destroyer
      • Animator
      • Sketcher
      • Viewfinder
      • Tape Measure
    • WristPad
      • User Preferences
    • Moving Around
      • VR Comfort Preferences
    • Asset Library
    • Working with Things
      • Manipulating 3d Objects
      • Using Movement Handles
      • Moving with the Waldo
      • Attaching Things
      • Inspector
      • Groups and Assemblies
    • Working with Time
      • Timeline
      • Activities
      • Script
      • Motion Paths
    • Controls and Automation
      • Lights
      • Audio
      • Stage Machinery `
    • Undo / Redo
    • Collaboration
      • Video Streaming from VR
      • Running a Workshop
    • Using FirstStage on the Desktop
  • Crafting
    • Sketching
      • Cutting a Tunnel
      • Manipulating Edges & Vertices on 3d Objects
      • Cutting Faces
      • Standard Shapes
    • Painting with Materials
    • Painting with the Airbrush
    • Lathe
    • Parametric Shapes
    • Images
  • Animation
    • Rigid Animation
      • Realtime Animation
    • Posing Characters
    • Performance Capture
      • Motion Capture (VRIK)
      • Setting up Vive Trackers
      • Performing
      • Face Capture
        • Setting up Facial Capture
      • Mocap with Rokoko Smartsuit
    • Layering Animations
    • Using Physics
  • Camerawork
    • Cameras
    • Through The Lens
    • Camera Sequencer
  • Interactive Scenes
    • Player
    • Branching Scripts
    • Triggers
      • Trigger Volume
      • Click to Trigger
      • Place To Trigger
    • Multiple Choice Dialog
    • Task Checklist
  • Asset Management
    • FirstStage Repository
    • Set up Unity
    • Setup the FirstStage Asset Importer project in Unity
    • Unity Basics
    • Asset Import Process
      • Source assets and import
      • Prepare Assets for FirstStage
      • Export to FirstStage
        • Asset Tagging
      • Example - Unity Asset Import
      • Example - Unity Scene Import
      • Test in FirstStage
      • Fix common issues
      • Share in FirstStage (Cloud)
    • Importing from SketchUp
    • Creating a Character for Firstage using Character Creator
    • Importing from Reallusion Character Creator™
Powered by GitBook
On this page
  • Assign a Character
  • Filters
  • Record Voice
  • Recording the Performance
  1. Animation
  2. Performance Capture

Face Capture

PreviousPerformingNextSetting up Facial Capture

Last updated 2 years ago

Facial animation can really bring a character to life. FirstStage lets you use an iPhone X or later (must support face tracking) to capture facial animation. It works by running a companion app on your which connects to FirstStage and sends it the facial animation data.

Face Capture is controlled from the rather than VR as the input is coming from a phone or tablet. If the Motion Capture window is not showing go to the drop down menu (top right) and click on it to make it appear.

Facial animation will only work with puppets with suitable blendshapes on the face (see [Importing a Character] for details). Some puppets may only have a jaw bone to animate the face: for these you will at least get basic “mouth flap” which may be sufficient for background characters. Most of the standard FirstStage character assets work with facial animation as do characters from ReadyPlayerMe or Reallusion Character Creator.

It is usually helpful to set up a camera in close up on the character's face. Then show the Camera Sequencer as well and select that camera so you can monitor your character's face.

Assign a Character

The Motion Capture window should now show a source for your device in the left hand column. If you select it (if it’s the only one, it will be selected by default) the right hand pane will show the device name and a drop down menu which will allow you to assign the device to one of your characters. Initially it will be Unassigned. Choose the character for which you’d like to record some facial animation.

It is often helpful to create a camera targetting your character in close up. Then select that camera in the [Camera Sequencer] so you can see how your character is responding to your facial performance.

Filters

Every character responds to facial animation differently depending on how it is rigged and the quality of its facial blendshapes. You can tailor your performance to the puppet but to make it easier, each of the main areas of the face - Jaw, Mouth, Eyes, Brows, Cheeks and Tongue - has a filter applied.

For each facial channel, there is a slider where you can set the range of expression to be used: if your character is prone to extreme expressions, reduce the upper limit; if your character seems to mumble, raise the lower limit. Underneath each slider, there is also a dial that lets you set the gain on each channel: gain is applied after the range reduction.

Experiment to get the best match between your expressions and the character’s.

Record Voice

You can either record the performer’s voice at the same time or simply “mouth over” an existing dialogue track (or music track).

Recording the Performance

Simply press Record on the Transport Controls to record the performance and then press stop when done.

Afterwards, there will be a Face Mocap activity on the timeline for that character. You can move it about freely and you can also trim it to remove any start up or cool down.

The mocap device has a Record Arm button on it. This shows red when enabled. It is automatically disabled after each performance as otherwise you will not be able to replay your recent performance. Click on it to re-enable it if you want to retake your facial capture or record another section.

Setting up Facial Capture
desktop