Data compression and statistical inference

When mutually correlated data are observed at two different places, it may happen that data are compressed independently, losing the correlational information as a whole. Such cases are observed frequently in economical and environmental data. When the compressed data (statistics) are combined to perform a statistical inference concerning the probability structure governing the two observations, the loss of the statistical information due to the independent data compression should be noted. In other words, a problem arises of estimating Fisher information lost by the independent data compression in the sense of Shannon information at the two sites. It is another problem to determine the best ways of data compression and estimation. This paper introduced anew the notion of mutual Fisher information, and answers those questions by a new theory, combining the information and the statistical theories. The usefulness of the information geometry, which works on the geometrical structure of the probability space, is also indicated through the discussion.