66826

Mutual Information of Discrete Memoryless Channel (DMC)

For the given,

set of input messages: X = [x_t,...,x_M }, set of output messages: Y = {>•, ,...,y_N},

Discrete Memoryless Channel(DMC) is defined by ils probabilistic model (Fig.3.1), MN transition (conditional) probabilities:

p( >v /x_i); i = 1.....M; j = 1.....N

For the given channel and input probabilities, output probabilities can be calculated:

A/

Piy_J)^ssY_lP(x_l)p(yjłx_lyj=\.....N

1=1

For a noisy channel the following terms can be introduced:

Information loss: H(X/Y), misinformation: //(Y/X), mutual information:

/(X/Y) = 7(Y/X) = H(X)~ H(X/ Y)= H(Y)~ H(Y/X)

Mutual information between events (sources) X and Y is the information provided about the evcnt X by the occurrcncc of the cvcnt Y. Its unit is [bit].