05InteractionTechnique--Desktop Metaphors, Icons, Window Manager

1 .1 Lecture 5: Deep Dive: Desktop Metaphors, Icons, Window Managers Brad Myers 05-440/05-640: Interaction Techniques Spring, 2016 © 2016 - Brad Myers

2 .Quiz 2 © 2016 - Brad Myers 2

3 .Announcements Homework turn-in on paper in class on Monday Due before lecture starts Remember guest lecturer next Monday © 2016 - Brad Myers 3

4 .Paned Windows were first Probably lots of systems? Bravo (Xerox PARC editor), 1974 Emacs , 1976 by Richard Stallman and Guy L. Steele, Jr., etc. Easy to implement, useful to see multiple documents at the same time Same document or different documents © 2016 - Brad Myers 4

5 .Smalltalk Alan Kay proposed the idea of overlapping windows in his 1969 doctoral thesis Overlapping windows first appeared in 1974 in the Smalltalk’74 system Also used popup windows, scroll bars, etc. I worked with Smalltalk in 1977 Did not update windows when covered – brought the window to the top Only one window could update at a time Top window is the “focus” or “listener” window Menu of window manager commands, including: Top, Bottom, Reframe, Resize, Move, Close, etc. Then use mouse for parameters © 2016 - Brad Myers 5

6 .InterLisp -D, Tajo (XDE), etc. Many other Xerox PARC systems quickly adopted covered windows, with various tweaks Tajo (XDE) was the programming environment in which Star was developed (1975) Also had simple icons (“tiny windows”) Different buttons on different parts of title bar did different actions Chording of 2 buttons = middle button Interlisp -D (1980) Windows without title bars Window groups (attachments) Shrink into “icons” © 2016 - Brad Myers 6

7 .Spatial Data Management System (SDMS) 1978 MIT “Architecture Machine Group” now MIT Media Lab Display everything you want on an infinite sheet, and scroll around One monitor for “world view”, big screen for area of current interest Semantic zooming First system to put calculators, address books, etc. on the screen Multi-media support: pictures, text, video, audio Required lots of expensive and special-purpose hardware Small touch screens, joysticks, 3D finger trackers, large rear-projected displays Redone as Pad (1993) and Pad++ (1994) “Multi-scale architectures” © 2016 - Brad Myers 7

8 .Pygmalion: A Computer Program to Model and Stimulate Creative Thought David Canfield Smith’s PhD thesis, 1977 First large system implemented in Smalltalk Invented the name “icon” Small graphic symbols that represent something else Have capability of doing something, name, contents, code Also drag and drop of icons See guest lecture from last time © 2016 - Brad Myers 8

9 .David Smith video, 2014 Drawing diagrams on a whiteboard until looked right Then code in a language This translation introduced errors Try to execute blackboard diagrams directly Implemented in Smalltalk on Alto 1975: Pygmalion – invented 2 things Icons Name based on Russian Orthodox religious symbols Embody some of the properties of what is portrayed So more than just a picture Programming-by-example Drag-and-drop icons to demonstrate the data flow © 2016 - Brad Myers 9

10 .Xerox Star 1982 First system to provide desktop metaphor 2 columns of 3 windows each Tiled! Big display so “didn’t need” overlapping “Viewpoint” – later version (1985) – overlapping optional © 2016 - Brad Myers 10

11 .Xerox Star – David C. Smith Goal to help “knowledge workers” use computers Applied idea of using icons Office icons – for things around his office: Folder, file cabinet, clock , etc. Distributed file syste m “Machine semantics” – attributes and methods for each object Attributes: name, creator, contents Data icons vs. function icons (documents, folders vs. printer , in-box/out-box) “ Envelop” for addressee , etc. for email Methods – create, delete, drop-on More “desktop metaphor” than currently David didn’t draw the icons! Metaphors – balance between familiarity & power Can add extra functionality, like sorting, searching Help make it seem familiar, non-threatening Claims that iPhone’s aren’t “icons” & are not metaphorical & no drag-and-drop for operations © 2016 - Brad Myers 11

12 .Xerox Star Keyboard Commands Very few “commands” for very rich functionality ~24 total, used everywhere Text, drawings Only 1 way to do things “Rule of 2” “Move” command was foundation of whole system Got rid of lots of commands Were modal Create by copy “Transfer sheets” of templates (more of the metaphor) Property sheets (David C. Smith invented these too!) Optimizations of common operations at top of keyboard © 2014 - Brad Myers 12

13 .Cedar Another Xerox PARC (research) system Influential tiled design, with icons 1982, 1983 Many commands to manipulate windows New windows put at bottom of columns © 2016 - Brad Myers 13

14 .Andrew System From CMU’s “Information Technology Center” (ITC) – where Cyert Hall is now Fully funded by IBM Jim Morris hired from Xerox to be the head 1982-1987 Key contributions: Distributed file system (AFS) Component model for operating systems Tiled window system Automatic algorithm for where and how much to grow No icons – shrink to title bar Elaborate popup menu system © 2016 - Brad Myers 14

15 .Lisa and Macintosh 1983, 1984 Larry Tesler’s talk from last time ( assigned with text editing ) Popularized the desktop metaphor Covered windows Windows that are covered can update (e.g., clock) Listener (focus) window always comes to the top (click-to-type) Could only grow a window from bottom right corner Icons for files, folders, trashcan Not other actions like printing, emailing, etc. Animations so actions more apparent Rounded corners © 2016 - Brad Myers 15

16 .Sapphire My window system for PERQ, 1984 S creen  A llocation  P ackage P roviding H elpful  I cons and R ectangular E nvironments No graphic designer, so I made the icons and cursors myself Press down to preview, release to operate, move before release to abort. Becomes a mode, with the cursor as feedback Grow and move handles All operations also from keyboard Listener window could be covered Icons for all windows, shows progress, etc. © 2016 - Brad Myers 16

17 .Microsoft Windows Windows 1.0 released in Nov, 1985 Tiled window manager Scott McGregor from PARC (Cedar)  Microsoft Interesting history: http://www.technologizer.com/2010/03/08/the-secret-origin-of-windows/ Windows 2.0 was overlapping 1987 Resize window from any side or corner, move from title bar Window menu from upper left icon All operations from keyboard Windows 3.0 in 1990, 3.1 in 1992 © 2016 - Brad Myers 17 Windows 1 from Wikipedia

18 .Rooms Henderson & Card, 1986 Influential research system from Xerox PARC Collections of groups of windows: “a suite of virtual workspaces” Same window could be in multiple groups Designed to support different tasks Different backgrounds so can tell them apart “Doors” to go from one to another Overview to see & go to all of the rooms Macintosh “spaces” © 2016 - Brad Myers 18

19 .Newer Window Features Windows 95 (1995) added task bar Open applications Separate part as launcher Also Start menu Macintosh OS X added Dock (1999) Both open and not open applications At some point ( when? ), zooming so more will fit Spotlight – quick search by name Tabs – code editors & browsers What else? © 2016 - Brad Myers 19

20 .PDAs and Smartphone s Palm, iPhone , Android: Only 1 window at a time Icons of applications to start them No files Palm – scroll to see the rest iPhone Pages Dock for 4 icons Folders of icons Newer: search for icons by name But doesn’t show where they are  © 2016 - Brad Myers 20

21 .PDAs and Smartphone s Palm, iPhone , Android: Only 1 window at a time Icons of applications to start them No files Palm – scroll to see the rest iPhone Pages Dock for 4 icons Folders of icons Newer: search for icons by name But doesn’t show where they are  © 2016 - Brad Myers 20

22 .PDAs and Smartphone s Palm, iPhone , Android: Only 1 window at a time Icons of applications to start them No files Palm – scroll to see the rest iPhone Pages Dock for 4 icons Folders of icons Newer: search for icons by name But doesn’t show where they are  © 2016 - Brad Myers 20

23 .New Research on Desktops, 2 © 2016 - Brad Myers 23 Stephen Fitchett , Andy Cockburn, and Carl Gutwin. 2013. Improving navigation-based file retrieval. CHI ‘2013 . ACM, http://dl.acm.org/citation.cfm?id=2481323 Stephen Fitchett, Andy Cockburn, Carl Gutwin , “Finder Highlights: Field Evaluation and Design of an Augmented File Browser”, CHI’2014 , http://dl.acm.org/citation.cfm?id=2557014 File browser navigation measured to be slow: “12 seconds per retrieval for Mac users and more than 17 seconds for Windows users [5].” Current approaches: recently used lists, full-text search of disk 3 augmentations: Highlighting the predicted icons Hover menus for most likely content items to be selected Combined search-based navigation: highlighting items that match search

24 .Research Ideas for the Future of the Desktop? Replacing application-centric view with data-centric view? Has never worked before I talked about an “ Open Data Model” in 1998 Speech as “meta control” of applications Multi- and cross-application operations Siri and Google Now © 2016 - Brad Myers 24