Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

On Information Divergence Measures and a Unified Typicality

Siu-Wai Ho, Raymond W. Yeung
2010 IEEE Transactions on Information Theory  
Strong typicality, which is more powerful for theorem proving than weak typicality, can be applied to finite alphabets only, while weak typicality can be applied to countable alphabets. In this paper, the relation between typicality and information divergence measures is discussed. The new definition of information divergence measure in this paper leads to the definition of a unified typicality for finite or countably infinite alphabets which is stronger than both weak typicality and strong
more » ... cality. Unified typicality retains the asymptotic equipartition property and the structural properties of strong typicality, and it can potentially be used to generalize those theorems which are previously established by strong typicality to countable alphabets. The applications in rate-distortion theory and multi-source network coding problems are discussed.
doi:10.1109/tit.2010.2080431 fatcat:axlm6c7yc5evpcu4w6w7pmjvni