1.Isolation The confinement principle CS155: Computer Security

2.Running untrusted code We often need to run buggy/ unstrusted code: programs from untrusted Internet sites: apps, extensions, plug-ins, codecs for media player e xposed applications: pdf viewers, outlook legacy daemons: sendmail , bind honeypots Goal : if application “ misbehaves ” ⇒ kill it

3.Approach: confinement Confinement : ensure misbehaving app cannot harm rest of system Can be implemented at many levels : Hardware : run application on isolated hw (air gap) ⇒ difficult to manage a ir gap n etwork 1 Network 2 a pp 1 a pp 2

4.Approach: confinement Confinement : ensure misbehaving app cannot harm rest of system Can be implemented at many levels : Virtual machines : isolate OS’s on a single machine Virtual M achine Monitor (VMM) OS 1 OS 2 app1 app2

5.Approach: confinement Confinement : ensure misbehaving app cannot harm rest of system Can be implemented at many levels : Process: System Call Interposition Isolate a process in a single operating system Operating System p rocess 2 p rocess 1

6.Approach: confinement Confinement : ensure misbehaving app cannot harm rest of system Can be implemented at many levels : Threads: Software Fault Isolation (SFI ) Isolating threads sharing same address space Application : e.g. browser-based confinement

7.Implementing confinement Key component: reference monitor Mediates requests from applications Implements protection policy Enforces isolation and confinement Must always be invoked: Every application request must be mediated Tamperproof : Reference monitor cannot be killed … or if killed, then monitored process is killed too Small enough to be analyzed and validated

8.A old example : chroot Often used for “ guest ” accounts on ftp sites To use do: (must be root) chroot / tmp /guest root dir “ / ” is now “ / tmp /guest ” su guest EUID set to “ guest ” Now “ / tmp /guest ” is added to file system accesses for applications in jail open( “ / etc / passwd ” , “ r ” ) ⇒ open( “ / tmp /guest/etc/ passwd ” , “ r ” ) ⇒ application cannot access files outside of jail

9.Jailkit Problem: all utility progs ( ls , ps , vi) must live inside jail jailkit project: auto builds files, libs, and dirs needed in jail env jk_init : creates jail environment jk_check : checks jail env for security problems checks for any modified programs, checks for world writable directories, etc. jk_lsh : restricted shell to be used inside jail note: simple chroot jail does not limit network access

10.Escaping from jails Early escapes: relative paths open( “ ../../ etc / passwd ” , “ r ” ) ⇒ open( “ / tmp /guest/../../ etc / passwd ” , “ r ” ) chroot should only be executable by root. otherwise jailed app can do: create dummy file “ / aaa / etc / passwd ” run chroot “ / aaa ” run su root to become root (bug in Ultrix 4.0 )

11.Many ways to escape jail as root Create device that lets you access raw disk Send signals to non chrooted process Reboot system Bind to privileged ports

12.Freebsd jail Stronger mechanism than simple chroot To run : jail jail-path hostname IP- addr cmd calls hardened chroot (no “ ../../ ” escape) can only bind to sockets with specified IP address and authorized ports can only communicate with processes inside jail root is limited, e.g. cannot load kernel modules

13.Not all programs can run in a jail Programs that can run in jail: audio player web server Programs that cannot: web browser mail client

14.Problems with chroot and jail Coarse policies : All or nothing access to parts of file system Inappropriate for apps like a web browser Needs read access to files outside jail (e.g. for sending attachments in G mail ) Does not prevent malicious apps from: Accessing network and messing with other machines Trying to crash host OS

15.Isolation System Call Interposition

16.System call interposition Observation: to damage host system (e.g. persistent changes) app must make system calls: To delete/overwrite files: unlink , open, write To do network attacks: socket , bind, connect, send Idea: monitor app’s system calls and block unauthorized calls I mplementation options: Completely kernel space (e.g. GSWTK ) Completely user space (e.g. program shepherding) Hybrid (e.g. Systrace )

17.Initial implementation (Janus ) [GWTB’96] Linux ptrace : process tracing process calls: ptrace (… , pid_t pid , …) and wakes up when pid makes sys call. Monitor kills application if request is disallowed OS Kernel monitored application (browser) monitor user space open( “ / etc / passwd ” , “ r ” )

18.Complications If app forks, monitor must also fork f orked monitor monitors forked app If monitor crashes, app must be killed Monitor must maintain all OS state associated with app current-working- dir ( CWD ), UID, EUID, GID When app does “ cd path ” monitor must update its CWD otherwise: relative path requests interpreted incorrectly cd(“/ tmp ”) open(“ passwd ”, “r”) c d(“/ etc ”) open(“ passwd ”, “r ”)

19.Problems with ptrace Ptrace is not well suited for this application: Trace all system calls or none i nefficient: no need to trace “ close ” system call Monitor cannot abort sys-call without killing app Security problems: race conditions Example : symlink : me ⟶ mydata.dat proc 1: open( “ me ” ) monitor checks and authorizes proc 2: me ⟶ / etc / passwd OS executes open( “ me ” ) Classic TOCTOU bug : time-of-check / time-of-use time not atomic

20.Alternate design: systrace [P’02] systrace only forwards monitored sys-calls to monitor (efficiency) systrace resolves sym -links and replaces sys-call path arguments by full path to target When app calls execve , monitor loads new policy file OS Kernel monitored application (browser) monitor user space open( “ etc/passwd ” , “ r ” ) sys-call gateway systrace permit/deny policy file for app

21.Ostia: a delegation architecture [GPR’04] Previous designs use filtering: Filter examines sys-calls and decides whether to block Difficulty with syncing state between app and monitor (CWD, UID, ..) Incorrect syncing results in security vulnerabilities (e.g. disallowed file opened) A delegation architecture: OS Kernel m onitored a pplication agent user space policy file for app libc open( “ etc / passwd ” , “ r ” )

22.Ostia: a delegation architecture [GPR’04] Monitored app disallowed from making monitored sys calls Minimal kernel change (… but app can call close () itself ) Sys-call delegated to an agent that decides if call is allowed Can be done without changing app (requires an emulation layer in monitored process) Incorrect state syncing will not result in policy violation What should agent do when app calls execve ? Process can make the call directly. A gent loads new policy file.

23.Policy Sample policy file: path allow / tmp /* path deny / etc / passwd network deny all Manually specifying policy for an app can be difficult: Systrace can auto-generate policy by learning how app behaves on “ good ” inputs If policy does not cover a specific sys-call, ask user … but user has no way to decide Difficulty with choosing policy for specific apps (e.g. browser) is the main reason this approach is not widely used

24.NaCl : a modern day example game: untrusted x86 code Two sandboxes: outer sandbox: restricts capabilities using system call interposition Inner sandbox: uses x86 memory segmentation to isolate application memory among apps Browser HTML JavaScript NaCl runtime game

25.Isolation Isolation via Virtual Machines

26.Virtual Machines Virtual Machine Monitor (VMM) Guest OS 2 Apps Guest OS 1 Apps Hardware Host OS VM2 VM1 Example: NSA NetTop single HW platform used for both classified and unclassified data

27.Why so popular now? VMs in the 1960 ’ s : Few computers, lots of users VMs allow many users to shares a single computer VMs 1970 ’ s – 2000 : non-existent VMs since 2000 : Too many computers, too few users Print server, Mail server, Web server, File server, Database , … Wasteful to run each service on different hardware More generally: VMs heavily used in cloud computing

28.VMM security assumption VMM Security assumption : Malware can infect guest OS and guest apps But malware cannot escape from the infected V M Cannot infect host OS Cannot infect other VMs on the same hardware Requires that VMM protect itself and is not buggy VMM is much simpler than full OS … but device drivers run in Host OS

29.Problem: covert channels Covert channel : unintended communication channel between isolated components Can be used to leak classified data from secure component to public component Classified VM Public VM secret doc malware listener covert channel VMM