Skip to main content

Entropy

I am reading two books at the same time: The Information by James Gleich and Cycles of Time by Roger Penrose.

By coincidence, both have Entropy at their core.

Roger Penrose develops Conformal Cyclic Cosmology (CCC), primarily from an analysis of the exceptionally low entropy at the the time of the Big Bang, and his argument that the theory of "inflation" in the early universe does not sync up with Entropy calculations.

Gleich, on the other hand, studies the rise of information, and while the whole book is interesting, I found the sections on Claude Shannon and the development of Information Theory to be the most interesting.  While I have used the outcome of Shannon's theory in some of my digital signal processing projects (years ago!), I did not know that his original insight was around Information Entropy.  That's pretty cool.

Why am I reading two books at once?  Well, the Penrose book is tough going!  The arguments are hard to follow, the writing is dense, and the diagrams are confusing.  I know where he is going (time exists before the Big Bang, and after the Big Crunch - rinse and repeat - and somehow Entropy increases all the way through this mess), but I am not following all of the connecting bits.

So, I am reading Gleick when I get stuck.  Amazingly, the high level descriptions of Entropy in The Information is giving me some insights into what Penrose is describing.  Lucky coincidence :-)


Popular posts from this blog

Decentralization, Democracy, and Well-Being

Those of us raised in Democratic societies take it for granted that those societies provide better well-being (for common individuals) than other forms of governance. At the heart of democracy is personal freedom and autonomy, backed by the rule of law. We also take for granted the interplay of decentralized versus centralized authority. Decentralization can mean many things, but here we refer to it in terms of power, authority, and decision making. The more authority individuals have, the more decentralized the power system in which they are operating.  Almost by definition the more democratic a system, the more decentralized it is, with the caveat that some agreed upon axioms exist, such as the rule of law and its enforcement. Of course, authority can be too decentralized leading to "every man for themselves", so we put limits on decentralization through that same rule of law. With the advent of decentralizing technologies , which make possible more decentraliz...
Read more

Echo vs Home

We love Alexa! We have had the Amazon Echo for well over a year.  Recently we also got a Google Home, to test it against our Alexa experience. The quick summary:  Interacting with Alexa is like interacting with a person.  Interacting with Home is like interacting with a computer.  Alexa is fun; Home is useful.  If you took away Alexa, I would be upset - I would be losing a friend.  If you took away Home, I wouldn't care too much.  It was very strange, but I actually felt like I might be offending Alexa when I purchased Home. Here are the two main differences: Wake-up words.  "Alexa" is friendly, easy to say, and evokes emotion.  Alex personifies the system - I am talking with someone.  "OK Google" is awkward, and constantly reminds you that you are talking to a machine - I am talking to something.  Of course, Google will update Home to allow us to customize the wake-up word, but the current out of box experience is less tha...
Read more