1-Advanced Systems Software

1 .CS 1651 Advanced Systems Software Jack Lange Assistant Professor University of Pittsburgh

2 .Course Objectives Understand basic OS principals Understand historical background of OS design Gain practical experience with current systems x 86 and Linux Learn how modern systems are designed and operated Gain some exposure to systems research

3 .Administrivia Instructor: Jack Lange Email: jacklange@cs.pitt.edu Office: Sennott Square #5407 Office Hours: Weds 2 -4PM

4 .Communication Course homepage http://www.cs.pitt.edu/~ jacklange/teaching/ cs1651-f13/ Announcements, clarifications, corrections Additional resources for projects Google Group http:// groups.google.com/group/pitt- cs1651-f13 pitt- cs1651-f13@ googlegroups.com Private discussion group Open venue for class discussions and questions Based on email (Pitt addresses) Email me if you want to use a different one

5 .Class Details Inverse of other graduate courses Lectures on Tuesdays Provide background Supplemental readings assigned (Check Schedule!) Presentations on Thursdays Student presents selected research paper Other students read and provide a review of the paper before hand

6 .Grading Grading Participation (20%) Final (30%) 4 Projects (50%) Late policy Submit by midnight of the due date 10% penalty for every day late

7 .Projects Work Individually C is required Highly Recommended: OS or having some familiarity with Unix systems programming, preferably in C or C++ All projects are in C BUILDING software is 50% of the grade of this class 1-9

8 .Textbooks None are “required” but highly recommended Linux Device Drivers Understanding the Linux Kernel

9 .Reading papers and evaluating systems What are the most important ideas: perhaps a combination of their motivations, observations, interesting parts of the design, or clever parts of their implementation. What are the largest flaws; maybe an experiment was poorly designed or the main idea had a narrow scope applicability . Being able to assess weaknesses as well as strengths is an important skill for this course and beyond . What is the relevance of the ideas today, potential future research suggested by the article , etc .

10 .What makes a good presentation What is the problem? Why should people care about the problem? How do you intend to address it (high level)? How did you address it (w/ lowlevel details of interesting components)? Why is your approach superior to others? What are the shortcomings of your approach? Did your approach work?

11 .What makes a bad presentation Lack of context Reading from slides Lots of text or data Only focusing on technical details Ignoring high level points

12 .Why learn about Operating Systems? Tangible reasons Build or modify a real operating system Administer and use system well Tune application performance Intangibles Intrinsic curiosity Understand how much of you computer system works Gain/apply knowledge in other areas of Computer Science Computer architecture and devices Synchronization in programming languages Data structures and algorithms Performance analysis Challenge of designing large, complex systems

13 .What is an operating System Not easy to define: Users Applications Operating System Hardware OS: Everything that isn’t an application or hardware OS: Software that abstract hardware into a useful form for applications Standard Library Resource Coordinator

14 .First Function: Standard Library Advantages of standard library Allow applications to reuse common facilities Make different devices look the same Provide higher level abstractions Challenges What are the right abstractions?

15 .Second Function: Resource Coordinator Resource: “Anything valuable” (e.g. CPU, memory, disk, network) Advantages of resource coordinator Virtualize resource so multiple users/applications can share Protect applications from one another Provide efficient and fair access to resources Challenges What mechanisms? What policies?

16 .What Functionality in OS? No single right answer Desired functionality depends on outside factors Users & Applications Operating System Computer Architecture Technology Changes Expectations OS must adapt Change abstractions provided to users Change algorithms to implement those abstractions Change low-level implementation to deal with hardware Current operating systems driven by its evolution Two distinct cases in history Case 1: Computers are expensive Case 2: Computers are cheap

17 .History of the OS Commercial systems (1950s) and HPC systems (early 1990s) Enormous and expensive Goal: Get the system working Single operator/programmer/user runs and debugs at a time OS functionality Standard library -> No coordination of resources Monitor that is always resident; transfer control to programs OS User Job Memory Problem: Inefficient use of hardware Performance Metrics Throughput and Utilization

18 .Batch Processing Batch: Group of jobs submitted to machine together Operator collects jobs; orders efficiently; runs one at a time Role of OS: Same as before Advantages Amortize setup costs over many jobs Keeps machine busy during a single user s idle time Disadvantage User must wait for results until batch collected and submitted If bug, receive memory and register dump; submit job again Improve system throughput and utilization, but lose interactivity Still the prevalent supercomputing model

19 .Multiprogrammed Batch Systems Spooling provides pool of ready jobs Keep multiple jobs resident in memory OS chooses which job to run When job waits for I/O, switch to another resident job New OS functionality Job scheduling policies Memory management and protection (virtual memory) Advantage: Improves throughput and utilization Disadvantage: Still not interactive OS User Job 1 Memory User Job 2 User Job 2

20 .History of the OS: Phase 2 Introduction of inexpensive, fast devices Keyboards and monitors  text editors and interactive debuggers New set of performance trade-offs Goal: True interactivity (via improved response time) Time-sharing: Switch between jobs to give appearance of dedicated machine Advantage Users easily submit jobs and get immediate feedback New OS functionality More complex job scheduling, memory management Concurrency control and synchronization

21 .Personal Computers Computers became cheap (late 70s, early 80s) Dedicated machine per user “Improved” functionality from OS Remove time-sharing of multiple jobs No protection No virtual memory OS becomes subroutine again Conclusion: OS functionality changes with hardware and users OS User Job Memory

22 .State of current systems Large 100k’s to millions of lines of code 100-1000 person-years of work Complex Performance is important Conflicting needs of different users Poorly understood System outlives any of its builders Cannot remove all bugs Behavior is hard to predict, tuning is done by guessing Current trends Multiprocessors Networked systems Cloud based systems

23 .Distributed System Architecture

24 .Distributed System Architecture

25 .Distributed Memory Architecture

26 .Message Passing Explicit communication operations Specify sender and receiver Serialize data into a message Transmit message over communication channel Communication Channels Sockets E.g. Ethernet RDMA (Remote direct memory access) E.g. Infiniband Shared memory Becoming more prevalent on many-core architectures