Information Theory and Thermodynamics

Oded KAFRI

Abstract


 

Abstract. A communication theory for a transmitter broadcasting to many receivers presented.  In this case, energetic considerations cannot neglected as in Shannon theory.  It is shown that, when energy is assigned to the information bit, information theory complies with classical thermodynamic and is part of it. To provide a thermodynamic theory of communication it is necessary to define equilibrium for informatics systems that are not in thermal equilibrium and to calculate temperature, heat, and entropy with accordance to Clausius inequality.  It shown that for a binary file, the temperature is proportional to the bit energy and that information is thermodynamic entropy.  Equilibrium exists in random files that cannot compressed. Thermodynamic bounds on the computing power of a physical device, and the maximum information that an antenna can broadcast are calculated.

Keywords. Information theory, Thermodynamics, Entropy.

JEL. C62.


Full Text:


References


Bekenstein, J.D. (1973). Black holes and entropy, Physical Review D, 7(8), 2333. doi. 10.1103/PhysRevD.7.2333

Bennet, C.H. (2003). Notes on Landauer’s Principle, Reversible Computation, and Maxwell’s Demon, Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics, 34(3), 501-510. doi. 10.1016/S1355-2198(03)00039-X

Brillouin, L. (1962). Science and Information Theory, Academic Press NY.

Huang, K. (1987). Statistical Mechanics, John Wiely: New York.

Huffman, D. (1952). A method for the construction minimum redundancy codes. Proc. Institute of Radio Engineers, 40, 1098-1101.

Kestin, J., (Ed). (1976). The Second Law of Thermodynamics, Dowden, Hutchinson and RossStroudsburg.

Landau, L.D. & Lifshits, E.M. (1980). Statistical Physics, 3rd Edition, Pergamon: New York.

Li, M. & Vitanyi, P.M.B. (1997). An Introduction to Kolmogorov Complexity and its Applications, Springer-Verlag, New York.

Plischke, M., & Bergersen B. (2006). Equilibrium Statistical Physics, 2nd Edition, World Scientific: London.

Shannon, C.E. (1949). A Mathematical Theory of Communication, University of Illinois Press, Evanston, Ill.

Srednicki, M. (1993). Entropy and arez, Physical Review Letters, 71(5), 666. doi. 10.1103/PhysRevLett.71.666

Turing, A.M. (1936). On computable numbers with an application to the entscheidungsproblem, Proc. London Math. Soc. 42(1), 230-265. doi. 10.1112/plms/s2-42.1.230

Ziv, J., & Lempel, A. (1977). A Universal algorithm for sequential data compression, IEEE Transactions on Information Theory, 23(3), 337-343.




DOI: http://dx.doi.org/10.1453/jel.v4i1.1163

Refbacks

  • There are currently no refbacks.


.......................................................................................................................................................................................................................................................................................................................................

Journal of Economics Library - J. Econ. Lib. - JEL - www.kspjournals.org

ISSN: 2149-2379

Editor: jel@ksplibrary.org Secretarial: secretarial@ksplibrary.org   Istanbul - Turkey.

Copyright © KSP Library