What is Entropy(information theory)?

Home Blog Github

Tag: Machine Learning; Date: 02 January 2020

Information Theory:

What is Information?

How to measure the information?

\[h(x)=-\log_2P(x)\]

Why log?

Entropy

\[H[x] = - \sum_xP(x)\log_2P(x)\]

Reference