1/4

  • Compressed representations generalize better.
  • The Information Bottleneck (IB) balances compression and prediction:

​

​

Results:

  • 1. If is chosen improperly, learning will not happen:

Learnability for the Information Bottleneck

Tailin Wu ([email protected]), Ian Fischer ([email protected]),

Isaac L. Chuang ([email protected]), Max Tegmark ([email protected])

2/4

Results:

2. Prove sufficient conditions for IB Learnability:

​

​

​

​

​

Identify IB-learnability is determined by the conspicuous subset: the largest confident, typical, and

imbalanced subset

​

3. Provide a practical algorithm to estimate and discover the conspicuous subset

Learnability for the Information Bottleneck

Tailin Wu ([email protected]), Ian Fischer ([email protected]),

Isaac L. Chuang ([email protected]), Max Tegmark ([email protected])

3/4

Results:

4. Demonstrate our theoretical conditions and algorithm in synthetic, MNIST and CIFAR10 datasets:

​

Learnability for the Information Bottleneck

Tailin Wu ([email protected]), Ian Fischer ([email protected]),

Isaac L. Chuang ([email protected]), Max Tegmark ([email protected])

Synthetic

CIFAR10

4/4

MNIST:

Conspicuous subset

(all “1”):

Above the conspicuous threshold

(other digits):

OSZAR »