Showing posts with label history. Show all posts
Showing posts with label history. Show all posts

Monday, February 22, 2010

Growth in computing power

I am running a fever, cough and cold that collaboratively make me tired at body and heart. The tiredness unfortunately keeps me away from blogging, as research, work, and study effectively occupy my reduced "active" time. When the going gets tough, I have to give priority to what I am paid for and what I pay for. Somehow I find sometime to write this, when my Python script is running in the background.

Recently I have read this 2002 NBER paper, titled The Progress of Computing by Yale's economic professor, W.D. Nordhaus. This is a good 61 page article that covers the 150 year history of computers. I am always a big fan of the evolution of computers, because it teaches a lot of lessons about what not to do. It's still a serious paper, not a set of anecdotes.

The important observation is the rate at which the power of computation has grown after the World War II. Believe it or not, over this period of the last 150 years, the growth in computational power is trillion times. I don't think any of technology or field can boast this kind of growth. Of course, medicine is a field that grows very fast, although it is not easily measurable like computers. But the growth in medicine had a huge acceleration when computers started playing a role in improving diagnostic and surgical ability of doctors, as with any other field.


The most interesting part of the essay is the graph shown above. This graph is plotted for a forty year period in which it shows the exponential decrease in the computer price (it's a semilog plot). As it can be observed, the price of computation from 1970 to 2000 has gone down by a factor of one million. I would like to see the increase in efficiency and productivity in various industries that has been brought about by this reduced cost of computation. This may arguably be the most important testimonial for man's achievements over the past half-a-century.

Wednesday, July 08, 2009

Connection Machines – Prelude to Parallel Processing

The computer architecture entered into a new phase with the stored program concept and programmable and general purpose computing architecture. The credit for this development goes to John Von Neumann, Grace Hopper, and Howard Aiken. Later it was relatively easy to build microprocessors and computers like ENIAC, since the general computing architecture was well established.

However there was a problem with this primordial architecture. Unlike human intelligence, it was massively relying on a single powerful processor that operated on the stored program in a sequential order. The first computer to depart from this view and behave more similar to human brain was Connection Machine(CM).

In early eighties, Danny Hillis a graduate student in MIT Artificial Intelligence Lab designed a highly parallel supercomputer that incorporated 65,536 processors. This design was commercially manufactured by Thinking Machines Corporation(TMC) created by Danny Hillis under the name CM-1. Thousands of processors which formed a part of CM-1 were extremely simple one bit processor connected together in a complex 20-dimensional hypercube. The routing mechanism between several processors in CM-1 was designed by Nobel Laureate Richard Feynman himself. In 1985, CM-1 was a dream SIMD machine for labs working on artificial intelligence(AI). But it had some practical problems. First it was too expensive a machinery to be purchased by budding AI labs. Second it did not have a FORTRAN compiler, which was the famous programming language among scientists at that time. Third it did not have floating point processing mechanism, a must for scientific analysis. So CM-1, although was a parallel processing marvel, looked more immature to face the market. Realizing the mistakes done in the design of CM-1, Thinking Machine released CM-2 which had a floating point processor and FORTRAN compiler. But still it did not fly. Evidently Danny Hillis was making a machine for future that present had no use with.

In early nineties, Thinking Machine introduced CM-5, which featured in the control station of Steven Speilburg’s Jurassic Park. It is considered to be not only a technological marvel, but also a totally sexy supercomputer (see figure). Instead of simple processors, CM-5 had a cluster of powerful SPARC processors. They also came out of the hypercube concept and built their data network as a binary fat tree. CM-5 is a synchronized MIMD machine combined with some best aspects of SIMD. The system can support up to 16,384 SPARC processors. The processing nodes and data networks interacted through 1-micron standard cell CMOS network interfaces with clock synchronization. The raw bandwidth for each processing node was 40 MBPS. But in a fat tree as it goes up the level, the cumulative bandwidth may reach up to several GBPS (if all these remind you of Beowulf, you are not alone!). TMC guaranteed that CM-5 was completely free of fetch-deadlock problem that occurs between multiple processors (using this).

Although it looks like a great architecture, from pure technical standpoint it is evident that TMC has toned down its idea of a complete parallel machine, simply because in eighties it did not sell partly due to the lack of market readiness and partly due to some blows in design. Secondly the failure of earlier CM series took a toll on TMC’s strategy. When CM-5 was introduced, the future of the company was dependent on that supercomputer sale. Los Alamos National Laboratory bought one. Jurassic Park set bought one. I cannot think of any other major customer. If CM-5 had survived, it would have had to fight with the likes of Intel Paragon, and Beowulf cluster for market space.

After the cold war, DARPA has cut down its funding on high performance computing which fell as a final blow on Thinking Machines. One fine morning in 1994, TMC filed Chapter 11 bankruptcy protection. The Inc. gives an alternative explanation for the failure of Thinking Machine. It is a good insight, but I cannot buy in its opinion about Danny Hillis. Perhaps The Inc. should restrict itself to its primary aim of advising budding entrepreneurs and refrain from measuring scientific minds. Nobody can deny that Danny Hillis was a genius, but the problem was that he was an out of the world freak who could not become a good businessman. Currently he is working on an ambitious project to build a monumental, mechanical clock that would run for multi-millennium.

The way that Thinking Machine took, was certainly not the way to build a successful enterprise. But the initial architecture that they introduced in early eighties was truly a stroke of a genius that every computer architect must study and understand.