Newsgroups: comp.parallel From: bahendr@cs.sandia.gov (Bruce Hendrickson) Subject: comp.parallel posting (Re: Looking for Textbook Suggestions) Organization: Date: 2 Jul 1997 16:48:04 GMT Message-ID: <5pe0o4$1et@server1.ctc.com> Five weeks ago I posted a request for textbook suggestions for an applied course on parallel scientific computing. Several of you asked me to share the responses I received, so I enclose them below. My original posting read as follows. > I'll be teaching a course on parallel computing in the fall, > and I'm looking for suggestions for textbooks. This will be > a senior level course with a focus on scientific computing > applications and a fairly applied point of view. When I taught > a similar course a couple years back I used Kumar's book, which > was OK, but not great. Any suggestions for alternatives would > be appreciated. I was asked about my dissatisfaction with Kumar's book. I have two basic objections. First is the performance modeling and the emphasis on the isoefficiency metric. I've never found this metric to be very helpful, and my students didn't either. My second objection is the heavy focus on the topology of the underlying network. With current machines, I don't consider topology to be a particularly important issue. I was hoping that there was a more recent text with a similar focus on scientific computing which didn't have these shortcomings. Usage of MPI would be a definite plus. Unfortunately, it seems that no such book exists. After reviewing a large number of options we decided that there really wasn't a perfect option. In the end, we decided to use Peter Pacheco's book "Parallel Programming With MPI". (This despite Peter's opinion that the book is too low level.) We feel this text does an excellent job covering the first half of our course including parallel architectures, programming models, MPI, collective communications and performance modeling. For the remainder of the course, emphasizing scientific computing algorithms, we will supplement with research papers and web-sites. The latter are rather copious by now, and contain lots of good materials. Thanks again for all the responses. - Bruce Hendrickson _________________________________________________________________________ It is quite difficult to advise you on the selection of a single textbook on the subject of parallel computing. Parallel Computing Works! is a good survey of some success stories, but may not be appropriate as the main or sole text for such a course. Highly Parallel Computing 2nd. ed. (or something like that) by Almasi and Gotlieb is quite comprehensive, but does not really teach you how to program a parallel computer. In my opinion, what you need is to start off with a general introduction to the terms and techniques that have been used in the past. Then you need to move on to how the technology has evolved (e.g. SIMD machines are pretty much history, and most MIMD machines now have far fewer than 1000 processors). This leads into what I believe are the two main remaining paradigms: Message Passing (primarily MPI). Shared Memory loop level parallelism based on compiler directives (yes I know about attempts to use HPF to extend this to distributed memory environments, but I do not think highly of the concept). It is probably about this time a good idea to bring in the concept of tuning the code for good serial performance on machines with cache. In the O'Reilly Nutshell series there is a good book on this topic by Dowd. It is also a good idea to distinguish between those doing parallel computing to make a name for themselves and those trying to get a job done. The first category is almost always interested in huge jobs using large numbers of processors. Unfortunately, such efforts frequently fail and this type of effort may not produce any useful results unless it is a major success. The second category tends to have very specific and modest goals (e.g. increase the size problem that can be handled by a factor of 5, the speed by a factor of 2-10, and do all of this at an acceptable price). The importance of making this distinction is that people in the second group are more willing to adopt strategies that clearly don't scale to the TFLOPS level of performance, but will still meet their needs. Hope this helps, Daniel M. Pressel Computer Scientist U.S. Army Research Laboratory _________________________________________________________________________ One book that I would recommand for a senior or beginning Graduate level course in parallel computing is: S. Lakshmivarahan and Sudarshan K. Dhall, Analysis and Design of Parallel Algorithms - Arithmetic and Matrix Problems. ISBN 0-07-036139-8. -- Jay Liang jl@ou.edu _________________________________________________________________________ Hi, Have a look at some of our online course material. For a start all of it is freely available and may satisfy some (but probably not all) of your needs. See: http://www.epcc.ed.ac.uk/epcc-tec/documents.html Hope this is of some help, Mario -- ------------------------------------------------------------------------- | Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ. | | Tel:0131 650 5026:mario@epcc.ed.ac.uk:http://www.epcc.ed.ac.uk/~mario/| ------------------------------------------------------------------------- _________________________________________________________________________ Hi Bruce, Would you post a summary of the suggestions you get for textbooks? I'd also be interested in learning about why you're dissatisfied with Kumar. I've also used it and didn't find it perfect, but it seemed much superior to most other possibilities (e.g., Quinn). I've enjoyed reading Foster's book, "Designing and Building Parallel Programs," but I've never taught a class from it. I especially like his discussions of performance prediction and analysis. I haven't used it in a class mainly because its emphasis is on software engineering, and the class I teach places more emphasis on algorithms. I'd like to recommend my book, "Parallel Programming with MPI," ;-) but it sounds like it's too elementary for the course you have in mind. Best wishes, Peter Pacheco Department of Computer Science University of San Francisco San Francisco, CA 94117 (415) 422-6630 peter@usfca.edu _________________________________________________________________________ Dear Bruce, In response to your post of 27 May, please allow me to bring what I consider to be an appropriate textbook to your attention. the book is Practical Parallel Programming by Greg Wilson and I am the publisher. Information on the book can be found at: http://www-mitpress.mit.edu/mitp/recent-books/comp/wilfh.html If this looks like a likely candidate for your course, an examination copy can be requested from Margie Hardwick at . I hope you find the book appropriate for your students. Best, --Bob Prior ROBERT V. PRIOR Computer Science Editor prior@mit.edu The MIT Press (617) 253-1584 Five Cambridge Center Fax: (617) 258-6779 Cambridge, MA 02142 http://mitpress.mit.edu/ _________________________________________________________________________ -- Articles to parallel@ctc.com (Administrative: bigrigg@ctc.com) Archive: http://www.hensa.ac.uk/parallel/internet/usenet/comp.parallel