I attended a weekly meetup organized by NUS Hackers last Friday, and on that day, the speaker was a professor from my very own university itself, Dr. John L. Gustafson. He talked about the problems of the currently widely use (if not de facto standard) IEEE 754 floating point representation, which he said the usage of it is dangerous, calling it "weapons of math destruction". He touched on some very interesting points, such as why they designed it such that it has a huge number of exponent bits, the existence of too many representation of NaN, how it didn't fulfil even the most basic mathematical properties such as commutativity, associativity, and distributivity. A little note on commutativity, he mentioned that compilers nowadays "solved" the commutativity problem by always doing arithmetic operation of "smaller <operand> higher", whether you write the higher or smaller number as the first argument, which I think is funny but at least it works. He also went on to tell stories of when IEEE 754 caused some accidents and mishaps to happen due to its inaccuracie, some are plain funny but others are somewhat tragic.
He went on to explain to us his proposed float representation, which he calls "posits" and "valids". Posits are very similar in terms of purpose to IEEE 754; the difference is just their ways of representing numbers. Valids are for those mathematicians who need precise, accurate answers and want to be sure to know if the number the floating point representation represent is not actually accurate; something the currently available representations cannot do.
I found the representation system he proposed to be mindblowing. In a nutshell, it uses much fewer bits to represent the same number as compared to its IEEE 754 equivalent, but with a much higher precision. Another interesting point is that it can represent integers and low decimal points floats very precisely, so there's no way you can end up with 0.1+0.1 != 0.2; 1 + 0 = 1.0...0234... and so on. There is also only one way to represent 0, unlike +0 and -0 on IEEE 754. NaN is also eliminated, leaving much more bits to be used to allow higher precision. It may sound too good to be true to some, and to be honest, I haven't seen it in action, nor do I understand fully how it works, but from the high level overview I understand from the talk, it sounds really promising.
For those of you who are interested in his floating point representation proposal, NUS Hackers had very kindly shared the talk slides on Google Drive, so go ahead and take a look! Let me know if the link is down so I can request for the new link from them!
Also, you can get a more comprehensive explanation on the floating point representation he is working on in his book, The End of Error. However, note that the representation he talked about in the book is "outdated". The one in the slides I shared with you earlier is his work which was just completed a couple of months ago, so there might be some discrepancies with what I have said and what you read on the book itself.
I really hope his proposal will pass the IEEE committee and it will be in the mainstream soon. I shall end off this post using Dr. Gustafson's own words, taken from the slides itself. Let's "Make Single Precision Great Again".
No comments:
Post a Comment