First run (non-Docker)

On this page are the instructions on how to run the Python modules included in FACSvatar. For the quickstart with Docker, go here: Quickstart

  • Real-time: Modified OpenFace –> FACSvatar modules –> Blender / Unity3D

  • Offline: (Modified) OpenFace –> .csv –> FACSvatar modules –> Blender / Unity3D

We’re assuming here that the avatar being target is created with MB-Lab.

Prepare Python environment

At present, all the core modules work with Python, so let’s setup an environment. FACSvatar recommends using Anaconda for managing packages and virtual environments in Python, therefore code instructions assume Anaconda.

This project uses the Asyncio library for asynchronous code execution, hence we use Python 3.6+ (although some modules work with Python 3.5).

For the instructions below, replace / with \ if you’re on Windows and you’re not using WSL 2 (Windows Subsystem for Linux).

Choose a setup method

Follow one of the 3 setup instructions below.

1. Create a new FACSvatar environment with YAML

cd FACSvatar/modules
conda install -f environment.yml  # create a new env named 'facsvatar' with all required Python packages

2. Update existing conda environment with YAML

cd FACSvatar/modules
conda env update --name myenv environment.yml  # update existing env with all required Python packages

3. Manually install packages

Probably conda install .. can be replaced by pip install .. if you don’t want to use Anaconda.

Show instructions (click me)
conda create --name facsvatar python=3.7  # new virtual env and force python 3.x
conda activate facsvatar  # activate our environment

conda install pyzmq  # make sure it's for py3.6+
conda install pandas  # library for dataframes; used for .csv reading and JSON-to-Dataframe

# Basic environment setup finished, but ipykernel setup recommended for control panel GUI
conda install ipykernel  # allows the use of env kernels in jupyter notebook
conda install ipywidgets  # GUI elements in jupyter notebook

Check FACSvatar/modules/environment.yml for additional packages FACSvatar uses. E.g. building this documentation.

Optional setup steps

# enable our conda environment as kernel in jupyter notebook
python -m ipykernel install --user --name facsvatar --display-name "py3 facsvatar"

Start modules

Open 3 terminals

  1. Start Bridge module

    1. conda activate facsvatar

    2. cd *your_path*/FACSvatar/modules/process_bridge

    3. python main.py

  2. Start FACS to MB-Lab blend shapes module

    1. conda activate facsvatar

    2. cd *your_path*/FACSvatar/modules/process_facstoblend

    3. python main.py

  3. Start OpenFace offline module (input)

    1. conda activate facsvatar

    2. cd *your_path*/FACSvatar/modules/input_facsfromcsv

    3. python main.py

Congratulations, data should be flowing between the modules now! Next is a to data-drive your avatar.

Real-time (needs Windows PC)

You want to use your own facial expressions in real-time you say? Skip starting the OpenFace offline FACSvatar module (step 3) and instead start the modified OpenFace GUI: OpenFaceOffline.exe –> menu: File –> Open Webcam.

See for more detailed instructions: OpenFace

Data-drive your avatar

To move an avatar’s head, eyes and face, use one of these visualization engines: Blender / Unity3D / FACSHuman

Next

Every Python module has a help function, giving more information on what arguments you can give. Type: python module_x.py --help You can for example run modules on different PCs by changing the IP where the module is subscribed to for data with: python module_x.py --sub_ip 192.168.xxx.xxx

Use a GUI in Jupyter Notebook to manipulate the data flow in FACSvatar.

With FACSvatar, you can run modules on a different PC, you can create your own modules (in any programming language) to extend FACSvatar’s capabilities, and animate any number of (FACS-standard) avatar simultaneously, among others. So check out those other pages of this documentation!