GrooveNet: Real-time Music-Driven Dance Movement Generation using Artificial Neural Networks

Omid Alemi and Jules Françoise and Philippe Pasquier

Complementary Materials for the Submission to the Workshop on Machine Learning for Creativity

Training Data

Learning and Generating Movement Patterns

• FCRBM - Labeled Mocap Segments - No Audio
  Hidden Units: 150 | Factors: 400 | Order: 6 | Frame Rate: 30
  16-Dimensional, One-hot-encoded Labels
  
Pattern 4 • Pattern 5 • Pattern 6 • Pattern 7 • Pattern 8 • Pattern 9 • Pattern 10 • Pattern 11 • Pattern 12 • Pattern 13 • Pattern 14 • Pattern 15

* The rest of the labels (1, 2, 3, and 16) either represented non-moving portions of the mocap sequence, e.g., the beginnning, or did not cause the model to learn any patterns.

Dancing with Training Songs

• FCRBM - Cooked Features
  Hidden Units: 500 | Factors: 500 | Order: 30 | Frame Rate: 60
  Audio Features: 84-Dimensions:
  low-level features (RMS level, Bark bands)
  spectral features (energy in low/middle/high frequencies, spectral centroid, spectral spread, spectral skewness, spectral kurtosis, spectral rolloff, spectral crest, spectral flux, spectral complexity),
  timbral Features (Mel-Frequency Cepstral Coefficients, Tristimulus),
  melodic Features (pitch, pitch salience and confidence, inharmonicity, dissonance).
  
  Based on audio track 1: Output 1 • Output 2 • Output 3
  Based on audio track 2: Output 4 • Output 5 • Output 6
  Based on audio track 3: Output 7 • Output 8 • Output 9

Dancing with Unheard Songs

• FCRBM - Cooked Features
  Hidden Units: 500 | Factors: 500 | Order: 30 | Frame Rate: 60
  Audio Features: 84-Dimensions:
  low-level features (RMS level, Bark bands)
  spectral features (energy in low/middle/high frequencies, spectral centroid, spectral spread, spectral skewness, spectral kurtosis, spectral rolloff, spectral crest, spectral flux, spectral complexity),
  timbral Features (Mel-Frequency Cepstral Coefficients, Tristimulus),
  melodic Features (pitch, pitch salience and confidence, inharmonicity, dissonance).
  
  Output 1 • Output 2 • Output 3 • Output 4 • Output 5 • Output 6

Fun Material!