Course Outline
Introduction to GPU-Accelerated Computing
- Heterogeneous computing and CPU-GPU architecture.
- CUDA execution and memory spaces.
- Compiling CUDA C++ using nvcc and CMake.
- Verifying the GPU development environment.
Parallel Algorithms with Thrust and CUB
- GPU-accelerated sorting, reduction, and transformation.
- Refactoring STL algorithms for GPU execution.
- Thrust device vectors and execution policies.
- CUB device-wide primitives for custom pipelines.
GPU Memory Architecture and Management
- Global, constant, and texture memory types.
- Explicit device memory allocation and transfers.
- Unified Memory for simplified data access.
- Memory coalescing and access pattern optimization.
Asynchronous Execution with CUDA Streams
- Creating and managing CUDA streams.
- Overlapping kernel execution with data transfers.
- Using CUDA events for dependency management.
- Stream priorities and concurrency tuning.
Writing Custom CUDA Kernels
- The SIMT programming model and warp execution.
- Kernel launch configuration and grid-stride loops.
- Thread indexing and multidimensional grids.
- Error handling and CUDA runtime API checks.
Thread Hierarchy and Execution Model
- Grids, blocks, and threads in device code.
- Warp-level primitives and ballot operations.
- Block-level synchronization and barriers.
- Occupancy and resource utilization analysis.
Cooperative Groups for Flexible Parallelism
- The cooperative_groups API and group types.
- Thread-block tiles and tiled_partition.
- Grid-level cooperative launches.
- Multi-grid synchronization patterns.
Shared Memory Optimization Techniques
- Shared memory banks and avoiding bank conflicts.
- Tiling strategies for matrix operations.
- Shared memory as user-managed cache.
- cuda::shared_memory_mdspan for multidimensional views.
Kernel Fusion and Advanced Parallel Patterns
- Fusing multiple kernels to reduce launch overhead.
- Scan, reduce-by-key, and segmented algorithms.
- Atomic operations and lock-free data structures.
- Warp-aggregated atomics for throughput.
Profiling and Optimization with Nsight Systems
- Timeline analysis of CPU and GPU activity.
- Identifying memory transfer bottlenecks.
- Kernel performance and occupancy profiling.
- Iterative optimization with Nsight Compute.
Modern C++ Features in CUDA Device Code
- Lambdas, constexpr, and auto in kernels.
- C++17 parallel algorithms and execution policies.
- C++20 concepts and ranges on device.
- C++23 support in nvcc and CCCL 3.x.
CUDA Graphs and Advanced Asynchrony
- Defining and launching CUDA graphs.
- Graph capture from stream execution.
- Graph update and conditional execution nodes.
- Reducing launch latency for iterative workloads.
Integration Patterns for Existing Applications
- Wrapping GPU code behind C++ interfaces.
- Managing multi-GPU and NUMA systems.
- Build system integration with CMake and CUDA.
- Debugging device code with cuda-gdb.
Summary and Best Practices
- Choosing between Thrust, CUB, and custom kernels.
- Performance portability across GPU architectures.
- Code organization and RAII for CUDA resources.
- Next steps and advanced CUDA learning paths.
Requirements
- Foundational C++ proficiency, including lambda expressions, templates, and the STL.
- Familiarity with standard algorithms, containers, and iterators.
- Comfort with loops, conditionals, and functions.
- No prior CUDA or GPU programming experience is required.
Audience
- C++ developers looking to accelerate compute-intensive applications using GPUs.
- Software engineers transitioning from CPU-only environments to heterogeneous parallel programming.
- Performance engineers and quantitative developers handling large datasets.
Testimonials (3)
Detailed explanation, re-iteration of points in a quite subtle way that really drove the knowledge home very well. Rod's willingness to double-check the odd obscure question that we rasied, to be sure that his answers were 100% right. Also, his interest in discussing the pros & cons of alternate coding styles so that we learnt not only how to use C++ in our intended way, but why it should be done that way.
Nick Dillon - cellxica Ltd
Course - Using C++ in Embedded Systems - Applying C++11/C++14
Experience sharing, it's teacher's know-how and valuable.
Carey Fan - Logitech
Course - C/C++ Secure Coding
The online-ness of it meant that we could save a lot of time. Much appreciated. Also, the fact that the trainer knew both c# and Cpp was a huge help since he could explain everything through knowledge we already possessed.
