DDSP's RnnFcDecoder runs independent FC stacks per conditioning input, concatenates, then runs an RNN
The default decoder in DDSP autoencoders is RnnFcDecoder: for each input conditioning signal (loudness, f0, latent z), it applies a separate fully-connected stack (typically 3 layers of 512 units with layer normalization). The outputs are concatenated and fed to a GRU (or LSTM). The RNN output is concatenated again with the original stack outputs, passed through another FC stack, and split into synthesizer parameters (amplitudes and harmonic_distribution). The independent FC stacks allow each conditioning variable to be pre-processed into a shared embedding space before temporal modeling. The final split is configured via output_splits, e.g., (('amps', 1), ('harmonic_distribution', 40)).
Examples
decoder = ddsp.training.decoders.RnnFcDecoder(
rnn_channels=512,
rnn_type='gru',
ch=512,
layers_per_stack=3,
input_keys=('ld_scaled', 'f0_scaled', 'z'),
output_splits=(('amps', 1), ('harmonic_distribution', 40)))
Assessment
Why does RnnFcDecoder process each conditioning input (loudness, f0, z) through separate FC stacks rather than concatenating them immediately? What temporal information does the GRU provide that FC layers cannot?