Abstract:

NVIDIA’s CUDA is a general-purpose architecture for writing highly parallel applications. CUDA provides several key abstractions—a hierarchy of thread blocks, shared memory, and barrier synchronization—for scalable high-performance parallel computing. Scientists throughout industry and academia use CUDA to achieve dramatic speedups on production and research codes. The CUDA architecture supports many languages, programming environments, and libraries including C, Fortran, OpenCL, DirectX Compute, Python, Matlab, FFT, LAPACK, etc.

In this tutorial NVIDIA engineers will partner with academic and industrial researchers to present CUDA and discuss its advanced use for science and engineering domains. The morning session will introduce CUDA programming, motivate its use with many brief examples from different HPC domains, and discuss tools and programming environments. The afternoon will discuss advanced issues such as optimization and sophisticated algorithms/data structures, closing with real-world case studies from domain scientists using CUDA for computational biophysics, fluid dynamics, seismic imaging, and theoretical physics.

Welcome to the course notes for the full-day SUPERCOMPUTING 2009 CUDA Tutorial!

(Note: the slides below are also available on the NVIDIA website.)

Abstract

NVIDIA’s CUDA is a general-purpose architecture for writing highly parallel applications. CUDA provides several key abstractions—a hierarchy of thread blocks, shared memory, and barrier synchronization—for scalable high-performance parallel computing. Scientists throughout industry and academia use CUDA to achieve dramatic speedups on production and research codes. The CUDA architecture supports many languages, programming environments, and libraries including C, Fortran, OpenCL, DirectX Compute, Python, Matlab, FFT, LAPACK, etc.

In this tutorial NVIDIA engineers will partner with academic and industrial researchers to present CUDA and discuss its advanced use for science and engineering domains. The morning session will introduce CUDA programming, motivate its use with many brief examples from different HPC domains, and discuss tools and programming environments. The afternoon will discuss advanced issues such as optimization and sophisticated algorithms/data structures, closing with real-world case studies from domain scientists using CUDA for computational biophysics, fluid dynamics, seismic imaging, and theoretical physics.

8:30 Introduction-Overview and CUDA Basics
David Luebke, NVIDIA
[Download PDF]

9:00 CUDA Programming Environments
Ian Buck, NVIDIA
[Download PDF]

10:30 CUDA Libraries & Tools
Jonathan Cohen, NVIDIA
[Download PDF]

11:15 Optimizing GPU Performance and CPU-GPU Performance
Paulius Micikevicius, NVIDIA
[Download PDF]

1:45 Irregular Algorithms & Data Structures
John Owens, University of California Davis
[Download PDF]

2:30 Molecular Modeling
John Stone, University of Illinois at Urbana-Champaign
[Download PDF]

3:30 Seismic Imaging
Scott Morton, Hess
[Download PDF]

4:00 Computational Fluid Dynamics
Jonathan Cohen, NVIDIA
[Download PDF]

5:00 Quantum Chromodynamics
Michael Clark, Harvard University
[Download PDF]

This course is presented in two parts, Beyond Programmable Shading I and Beyond Programmable Shading II.

There are strong indications that the future of interactive graphics programming is a more flexible model than today’s OpenGL/Direct3D pipelines. Graphics developers need a basic understanding of how to combine emerging parallel programming techniques and more flexible graphics processors with the traditional interactive rendering pipeline. The first half of the course introduces the trends and directions in this emerging field. Topics include: parallel graphics architectures, parallel programming models for graphics, and game-developer investigations of the use of these new capabilities in future rendering engines.

The second half of the course has leaders from graphics hardware vendors, game development, and academic research present case studies that show how general parallel computation is being combined with the traditional graphics pipeline to boost image quality and spur new graphics algorithm innovation. Each case study discusses the mix of parallel programming constructs used, details of the graphics algorithm, and how the rendering pipeline and computation interact to achieve the technical goals. The focus is on what currently can be done, how it is done, and near-future trends. Topics include volumetric and hair lighting, alternate rendering pipelines including ray tracing and micropolygon rendering, in-frame data structure construction, and complex image processing. The course concludes with a panel, moderated by the creator of OpenGL Kurt Akeley, on the future of interactive graphics programming models.

The course presenters are experts on advanced rendering, graphics hardware, and parallel computing for graphics from academia and industry, and have presented papers and tutorials on the topic at SIGGRAPH, High Performance Graphics, Supercomputing, IEEE Visualization, and elsewhere.

(Massimiliano Fatica, Timo Stich, and Graham Pullan.  High Performance Computing with CUDA.  Tutorial.  International Conference on Supercomputing 2009.  Hamburg, Germany.)

Welcome to the course notes for the full-day CUDA Tutorial from the 2009 International Conference on Supercomputing!

The tutorial was held at the International Conference on Supercomputing in Hamburg, Germany on Monday, June 22, 2009.

Course Organizers

Dr. Massimiliano Fatica, NVIDIA Corporation

Course Speakers

Dr. Timo Stich, NVIDIA Corporation
Dr. Graham Pullan, University of Cambridge, UK

Tutorial Slides

• Introduction to GPU Computing (PDF)
• Basic CUDA (PDF)
• CUDA Toolkit & Libraries (PDF)
• CUDA Optimization (PDF)
• Introduction to OpenCL (PDF)
• Case Study: Computational Fluid Dynamics (PDF)

Dense matrix inversion is a basic procedure in many linear algebra algorithms. A computationally arduous step in most dense matrix inversion methods is the inversion of triangular matrices as produced by factorization methods such as LU decomposition. In this paper, we demonstrate how triangular matrix inversion (TMI) can be accelerated considerably by using commercial Graphics Processing Units (GPU) in a standard PC. Our implementation is based on a divide and conquer type recursive TMI algorithm, efficiently adapted to the GPU architecture. Our implementation obtains a speedup of 34x versus a CPU-based LAPACK reference routine, and runs at up to 54 gigaflops/s on a GTX 280 in double precision. Limitations of the algorithm are discussed, and strategies to cope with them are introduced. In addition, we show how inversion of an L- and U-matrix can be performed concurrently on a GTX 295 based dual-GPU system at up to 90 gigaflops/s.

(Florian Ries, Tommaso De Marco, Matteo Zivieri and Roberto Guerrieri, Triangular Matrix Inversion on Graphics Processing Units, Supercomputing 2009, DOI 10.1145/1654059.1654069)

This session explored the art of performance tuning for CUDA using several case studies. Topics included profiling to identify bottlenecks, effective use of the GPU’s memory hierarchy and DRAM interface to maximize bandwidth, data versus task parallelism, and avoiding SIMD divergence.  Many of the lessons learned in the context of CUDA are likely to apply to other many-core architectures used in HPC applications.

ICS is the premier international forum for the presentation of research results in high-performance computing systems.  In 2010 the conference will be held at the Epochal Tsukuba (Tsukuba International Congress Center) in Tsukuba City, the largest high-tech and academic
city in Japan.

Papers are solicited on all aspects of research, development, and application of high-performance experimental and commercial systems. Special emphasis will be given to work that leads to better understanding of the implications of the new era of million-scale parallelism and Exa-scale performance; including (but not limited to):

• Computationally challenging scientific and commercial applications: studies and experiences to exploit ultra large scale parallelism, a large number of accelerators, and/or cloud computing paradigm.
• High-performance computational and programming models: studies and proposals of new models, paradigms and languages for scalable application development, seamless exploitation of accelerators, and grid/cloud computing.
• Architecture and hardware aspects: processor, accelerator, memory, interconnection network, storage and I/O architecture to make future systems scalable, reliable and power efficient.
• Software aspects: compilers and runtime systems, programming and development tools, middleware and operating systems to enable us to scale applications and systems easily, efficiently and reliably.
• Performance evaluation studies and theoretical underpinnings of any of the above topics, especially those giving us perspective toward future generation high-performance computing.
• Large scale installations in the Petaflop era: design, scaling, power, and reliability, including case studies and experience reports, to show the baselines for future systems.

In order to encourage open discussion on future directions, the program committee will provide higher priority for papers that present highly innovative and challenging ideas.

Papers should not exceed 6,000 words, and should be submitted electronically, in PDF format using the ICS’10 submission web site. Submissions should be blind.  The review process will include a rebuttal period. Please refer to the ICS’10 web site for detailed instructions.

Workshop and tutorial proposals are also be solicited and due by January 18, 2010.  For further information and future updates, refer to the ICS’10 web site at http://www.ics-conference.org or contact the General Chair (ics10-chair@hpcs.cs.tsukuba.ac.jp) or Program Co-Chairs (ics10-chairs@ac.upc.edu).

Important Dates

• Abstract submission:  January 11, 2010
• Paper submission:     January 18, 2010
• Author notification:  March 22, 2010
• Final papers:         April 15, 2010

For more information, please visit the conference web site at http://www.ics-conference.org

From the press release:

SANTA CLARA, Calif. -Sep. 30, 2009- NVIDIA Corp. today introduced its next generation CUDA™ GPU architecture, codenamed “Fermi”. An entirely new ground-up design, the “Fermi”™ architecture is the foundation for the world’s first computational graphics processing units (GPUs), delivering breakthroughs in both graphics and GPU computing.

“NVIDIA and the Fermi team have taken a giant step towards making GPUs attractive for a broader class of programs,” said Dave Patterson, director Parallel Computing Research Laboratory, U.C. Berkeley and co-author of Computer Architecture: A Quantitative Approach. “I believe history will record Fermi as a significant milestone.”

Presented at the company’s inaugural GPU Technology Conference, in San Jose, California, “Fermi” delivers a feature set that accelerates performance on a wider array of computational applications than ever before. Joining NVIDIA’s press conference was Oak Ridge National Laboratorywho announced plans for a new supercomputer that will use NVIDIA® GPUs based on the “Fermi” architecture. “Fermi” also garnered the support of leading organizations including Bloomberg, Cray, Dell, HP, IBM and Microsoft.

“It is completely clear that GPUs are now general purpose parallel computing processors with amazing graphics, and not just graphics chips anymore,” said Jen-Hsun Huang, co-founder and CEO of NVIDIA. “The Fermi architecture, the integrated tools, libraries and engines are the direct results of the insights we have gained from working with thousands of CUDA developers around the world. We will look back in the coming years and see that Fermi started the new GPU industry.”

As the foundation for NVIDIA’s family of next generation GPUs namely GeForce®, Quadro® and Tesla® – “Fermi” features a host of new technologies that are “must-have” features for the computing space, including:

• C++, complementing existing support for C, Fortran, Java, Python, OpenCL and DirectCompute.
• ECC, a critical requirement for datacenters and supercomputing centers deploying GPUs on a large scale
• 512 CUDA Cores™ featuring the new IEEE 754-2008 floating-point standard, surpassing even the most advanced CPUs
• 8x the peak double precision arithmetic performance over NVIDIA’s last generation GPU. Double precision is critical for high-performance computing (HPC) applications such as linear algebra, numerical simulation, and quantum chemistry
• NVIDIA Parallel DataCache™ – the world’s first true cache hierarchy in a GPU that speeds up algorithms such as physics solvers, raytracing, and sparse matrix multiplication where data addresses are not known beforehand
• NVIDIA GigaThread™ Engine with support for concurrent kernel execution, where different kernels of the same application context can execute on the GPU at the same time (eg: PhysX® fluid and rigid body solvers)
• Nexus – the world’s first fully integrated heterogeneous computing application development environment within Microsoft Visual Studio

Images, technical whitepapers, presentations, videos and more on “Fermi” can all be found at: www.nvidia.com/fermi.