July 21, 2006

Monte Carlo KL-d

In matlab, the Monte Carlo method of estimating the KL-divergence between GMMS is too slow, at least using the netlab implementation of sampling and computing gmm probability. Using 2000 samples, it takes over 1.5 seconds to compute a single model pair, so a 350x350 kernel would take over 2 days, and forget about a 1000x1000 kernel.

I implemented it using in C++ using Torch, which had most of the machinery I needed, and it takes 150 ms to compute one pair - two orders of magnitude faster. A 350x350 kernel will take about 2 hours. And that's without doing any of the obvious optimizations I can think of like not resampling from each model across each kernel row.

Results coming soon... Posted by madadam at July 21, 2006 10:55 AM