Entropy measuring algorithms

List of algorithms from NIST 800-90B, Schürmann and Grassberger, Hutter Prize entries and Gupta and Agarwal deep learning techniques for entropy measurement in the general (correlated/ non-IID) case:-

List (incomplete) of entropy measuring algorithms:-

  1. Most Common Value Estimate.

  2. Collision Estimate.

  3. Markov Estimate.

  4. Compression Estimate.

  5. t-Tuple Estimate.

  6. Longest Repeated Substring (LRS) Estimate.

  7. Multi Most Common in Window Prediction Estimate.

  8. Lag Prediction Estimate.

  9. MultiMMC Prediction Estimate.

  10. LZ78Y Prediction Estimate.

  11. Ziv - Lempel.

  12. Gambling & suffix trees.

  13. Bayesian probability estimation.

  14. Rissanen’s method.

  15. Superposition of probabilities.

  16. Global probability estimates.

  17. Hutter Prize entries.

  18. Ouija technique for asking Shannon himself.

  19. And deep learning/artificial intelligence techniques…

Deep learning compression methods for complex and long range correlations.

Deep learning compression methods for complex and long range correlations.

And there are others, with a significant proportion based around compression theory. So what to do?