Title : 

Optimizing Compilers for High Performance Computing

Introduction: 

This course covers advanced compiler analysis and optimization techniques to improve the efficiency of programs on modern computer architectures. These optimizations are applied at the program source level (e.g., to C++/C code directly) to higher-level language constructs, e.g., loops, arrays, synchronization mechanisms, and abstract data containers. We will study program analysis techniques to automatically detect opportunities of advanced optimizations and how to coordinate the relevant optimizations to support high performance computing in scientific domains.

Topics : 

Dependence analysis, loop optimizations, automatic parallelization, specialization of compiler optimizations for different types of computations, analytical modeling of application efficiency, and effective coordination of compiler optimizations. 

Reference materials: 

Optimizing Compilers for Modern Architectures: A Dependence-based Approach by Ken Kennedy and Randy Allen, Morgan-Kauffman Publisheres Inc.

Optimizing And Tuning Scientific Codes, Qing Yi, SCALABLE COMPUTING AND COMMUNICATIONS: THEORY AND PRACTICE. 2013.

课程大纲

（一）Lecture 1: (July 7th) Introduction to loop dependence and Optimizations.

（1） Optimizing for multi-core architectures;

（2）categorization of compiler optimizations; 

（3）types of dependences;

（4）testing for dependence; 

（5）dependence-based analysis and optimization.

Reference : 

Chapter 2-3. Optimizing Compilers for Modern Architectures: A Dependence-based Approach  by Ken Kennedy and Randy Allen, Morgan-Kauffman Publisheres Inc.

（二）Lecture 2: (July 8th) The Memory Performance of Loops And Arrays.

（1） Optimizing and tuning scientific codes; 

（2）Categorization of reordering optimizations; 

（3）modeling of cache efficiency; 

（4）Loop Interchange, blocking, unroll&jam, unrolling, array copying, and scalar replacement.

Reference : 

Chapter 7-8. Optimizing Compilers for Modern Architectures: A Dependence-based Approach  by Ken Kennedy and Randy Allen, Morgan-Kauffman Publisheres Inc.

Optimizing And Tuning Scientific Codes.  by Qing Yi SCALABLE COMPUTING AND COMMUNICATIONS: THEORY AND PRACTICE. 2013.

（三）Lecture 3: (July 9th) Exploring Parallelism at Different Levels.

（1） Granularity of parallelism; 

（2）Loop parallelization;)

（3）Loop strip-mining and selection;)

（4）recognition of reductions and private variables;

（5）Pipelining;

（5）loop distribution and fusion.

Reference : 

Chapter 5-6. Optimizing Compilers for Modern Architectures: A Dependence-based Approach  by Ken Kennedy and Randy Allen, Morgan-Kauffman Publisheres Inc.

Optimizing And Tuning Scientific Codes.  by Qing Yi SCALABLE COMPUTING AND COMMUNICATIONS: THEORY AND PRACTICE. 2013.

（四）Lecture 4: (July 10th) Parallel Programming Patterns

（1）Specialization of compiler optimizations for different types of computations;

（2）effective coordination of compiler optimizations.

Reference : 

Structured Parallel Programming Patterns for Efficient Computation.   By Michael McCool, James Reinders, and Arch Robison. Morgan Kaufmann. 2012.

Automated Programmable Control and Parameterization of Compiler Optimizations.  Qing Yi. CGO'11: IEEE/ACM International Symposium on Code Generation and Optimization. Chamonix, France, Apr, 2011.

（五）Lecture 5: (July 11th) Programmable Optimization And Empirical Tuning

（1） How to develop your own compiler optimizations using POET, a scripting program transformation language developed to support programmable control of compiler optimizations and their empirical tuning. 

Reference : 

POET: A Scripting Language For Applying Parameterized Source-to-source Program Transformations.  Qing Yi. Software Practice & Experience. John Wiley&Sons. Vol 42, issue 6, pages 675-706. May, 2012.

Tutorial On Using POET To Build A Compiler Project (by Qing Yi)

Tutorial On Using POET To Optimize and Tune Scientific Codes [Slides by Qing Yi]