ENCS5336 | PARALLEL COMPUTING

Parallel architectures: parallel computers taxonomy, examples of parallel computers, Distributed memory systems, Shared memory systems and cache coherence, Heterogeneous system architecture (GPU and Xeon Phi), Interconnection networks and routing, performance metrics. Parallel algorithms. Decomposition techniques, Characteristics of tasks and interactions, mapping techniques for load balancing, parallel algorithm models. Examples on sorting, matrix problems, and graph problems. Parallel programming: types of parallelism, Programming scalable systems. Programming heterogeneous systems. Analytical modelling of parallel program, Scalability of parallel systems, Sources of overhead in parallel programs, asymptotic analysis of parallel programs.