The conclusion of that blog post:
- Break Amdahl's law - It is a correct attitude. You must not
- get into a deadlock by finding the algorithm is not much
- improvable because of Amdahl's law results.
Thanks, that's the title I chose for the article. :-)
- "Herb fought the law--Amdahl's Law, that is--and Herb won-DDJ"
- It is not a good title.
Right, the copyeditor added that tagline, and I don't like it either.
If you look at the print version of this article, you'll see I got
them to change the above back to "Here's a law you should break early
and often" there, but the change didn't stick online. (Yes, I could
ask them again to change it online too, but I already ask for enough
changes which they always kindly accept, and you have to prioritize
what's worth complaining about and pick your battles, like when the
wrong figures get used... even with good editors mistakes happen, and
these are quite good editors though a just little creative
sometimes. :-) )
- Herb didn't even try to fight with the Amdahl's law. Herb only
- proves it is better to take Gustafson's law into consideration
- while deciding on going for parallelization or not.
Partly true but incomplete. I also mentioned techniques to apply
parallelism to improve "s" (the sequential portion) by at least a
fixed amount. That doesn't transform s into "p" (the fully scalable
portion) but it still cheats Amdahl's Law as generally understood by
developers in the trenches.
Common statements of Amdahl's Law have got many developers into the
mindset of thinking a program is made up of only "stuff that's fully
parallelizable (e.g., divide-and-conquer on independent data)" and
"everything else is sequential and therefore can't benefit from
parallel hardware" -- and the latter conclusion is not correct. That
mindset ignores the third "middle ground" category consisting of
applying "non-fully" parallel techniques like pipelining that have
fixed limits but are still valuable to improve the performance of "s"
on a parallel machine.
I'm sure Amdahl realized that and had it in mind as ways of speeding
up "s", but most programmers I've heard citing him don't seem to
understand that, and they conclude that everything other than "p"
cannot benefit from parallel hardware -- and it's that last bit that
is wrong and I thought needed some illumination.