Automatic Vs Manual Placement

From ObjectBrowser

Application user interfaces usually have unconstrained manual placement of objects. On the other hand, games usually have constrained automatic placement of objects. There isn't an iota of coincidence in this; games have uniformly superior user interfaces and unconstrained/manual placement of objects is simply a Bad Idea.

The reasons why unconstrained/manual placement of objects is a Bad Idea are many and complex. They fall into two broad categories, quantitative reasons and qualitative reasons. The quantitative reasons will be examined first.

The key concerns affecting a user of an ObjectBrowser are: Even a crude cost/benefit analysis will show that manual/unconstrained placement is a definite loss in 3 out of 4 of these areas (pathfinding, locomotion, manipulation), and doesn't provide any meaningful gains in the fourth (interpretation).

Pathfinding is only aided by manual placement if the user remembers where they placed an object in relation to where they are. In a 10,000 object system, this is almost exactly never. Meanwhile, manual placement hinders pathfinding by forcing the user to manually invoke an autoplacement function (in Windows, auto-arrange).

Locomotion is only aided by unconstrained placement if the user can calculate vectors toward an object and between objects. Since normal humans aren't so capable, locomotion devolves into following hops between objects. Allowing unconstrained movement is only an invitation for users to lose their chosen path during locomotion. And unconstrained placement (even with constrained movement) only serves to confuse users by providing them with useless spatial cues. Cues they will automatically try and inevitably fail to successfully use.

Interpretation is only aided by manual placement if the user interface's visualization is exceedingly poor (broken in fact). The most common reason given for manual placement is to be able to group similar (for some definition of similar) objects together in such a way that you can see all groups simultaneously. This is not a concern for an ObjectBrowser that lets a user see, simultaneously, a chosen node, and all the nodes it links to, and all the nodes they link to. Such a UI lets the user group objects around the central node and see all of the groups' contents simultaneously. In other words, manual placement is not a gain for interpretation.

Manipulation loses greatly from manual/unconstrained placement because the mechanism used to place objects within the geometric space (typically DirectManipulation dragging and dropping of an object onto another) can't be used to link and unlink objects from each other. This leads to the usage of inferior and unesthetic solutions such as 1) modes, 2) context menus / invoked commands, or 3) reifying links as UI objects.

To recap, manual/unconstrained placement is a definite loss to 3 out of the 4 things that a general object-UI is for, and not a gain in the 4th. But in fact, it is much, much worse than this because manual/unconstrained placement provides not just a definite loss but a monumental loss. To see why, we have to understand that choice by itself is a cost and not a gain.

Learning, interpreting, deciding and making a choice from among several options 'all incur costs. If there is only one option, the decision cost is reduced to zero automatically. If there is only one option, the choice can be made automatically by the system on behalf of the user, thus reducing all costs to zero.

So why is it that people consider choice to be a good thing? It's because people mistakenly associate the outcome of a choice with the choice itself. People are sufficiently varied in their tasks and preferences that having an outcome closer to their preferences is a benefit that outweighs the cost of the choice. But this gain is only realized when #1 the different choices are meaningfully different, and #2 the different choices are actually used. With these facts in mind, we are ready to conduct a more powerful cost/benefit analysis of automatic placement versus manual placement.

Any sufficiently powerful automatic placement system will have at least half a dozen placement functions, including but not limited to: The first half-dozen are all very different options, meaningfully different options, and they are used extensively. In contrast, a UI that lets you put objects in an infinite space presents you with a continuum (an infinite number) of options which are very rarely meaningful (eg, "slightly to the left"), often are mirror images of each other, and are almost never used in practice. Note that they are almost never willingly used even with broken UIs which do not display the contents of subnodes. It's quite likely that they would never willingly be used with a decent UI.

In conclusion, manual/unconstrained placement incurs an enormous (theoretically infinite) cost and absolutely zero benefit. Now onto the qualitative reasons why manual/unconstrained placement is a Bad Idea.

The chief qualitative reason to avoid manual placement is that placement in a space is equivalent to an object name, thus moving an object is equivalent to continuously and repeatedly renaming it. This causes a great deal of writebacks, synchronization problems and message traffic.

In addition to this, there is a large morass of pointless issues and annoying details to resolve if one chooses manual/unconstrained placement.

For example, The last two are less of a concern if you display all links from/to on-screen objects all the time (ie, you're willing to live in a spider's web). But even then, it's possible to lose or hide objects unless you employ sophisticated collision detection to constrain placement in very annoying ways.

In addition, you could of course limit free placement of an object belonging to a directory node within a space circumscribed by that directory node, but if you do that, you end up with a rather baroque "unconstrained" placement system.

To recap, there are a lot of security, aesthetic and performance issues with free placement. And resolving the security issues raises additional aesthetic issues.

-- RichardKulisz

Does anyone find it odd that console games always have better interfaces than computer games? This is literally without any exceptions. In every console, for every game, the UI will be far superior. Is that coincidence? Its been annoying the hell out of me for years. I think windows apps could learn a lot from game UIs.

Games generally have better interfaces, and maybe console games are even better on average. I attribute this to the fact that games (and console games in particular) are single-task programs. With PC games, multitasking is sometimes possible, but the games are intended to be run full-screen. When you're playing the game, that is all you're doing. The game owns the computer, and doesn't have to worry about fitting in with the environment. Windows applications are different. It wouldn't make sense for each little windows application to pioneer some cool new user interface. That would be an incredible usability loss. When your application isn't the sovereign owner of the computer screen, the best bet is to fit in with the environment and not create further distractions for the user by providing some super-leet interface. The standard Windows interface is not ideal, but it is predictable. The user can easily figure out the mechanics of a dialog box, regardless of which application it belongs to. IMO, if you want to give the user a new way to interact, it should be system-wide. If the user decides to upgrade to the new form of interaction, it should take effect immediately for every bit of user interface the system has. Unfortunately, very few operating systems have this capability. Windows only has consistency through convention, and that is part of the problem. -- MichaelSparks

I must offer disagreement. The answer is simple: consoles have better UIs for two, and only two, reasons:

This is critical, because most people find keyboards quite intimidating (and yes, people do think mice are footswitches sometimes), and most people who purchase games for consoles haven't graduated from highschool yet. And, those same people generally find computers to be "Nerdy"(tm), so they won't be caught dead arguing for their favorite OS UI.


I'm a person who often feels intimidated by the extra buttons on my cellular phone and microwave. I use approximately eighteen buttons of the ninety or so distributed across three remote controls for my television, DVR, and sound system... and, of those, ten are numeric buttons 0-9. Yet I see children of age nine and ten casually using these tools in much greater depth. I feel confident in asserting that the 'intelligence of someone who hasn't completed highschool' isn't particularly relevant... in fact, I think the situation is reversed: us 'old people' (and I say that at age 26) are much more comfortable with just a few buttons. Fewer buttons means we have less to learn, which is good because we generally have both less time and less motivation to bother learning (given that we've gotten by so far without learning the interface to the foobar, why start?).

Anyhow, I'll agree with MichaelSparks on this matter. Despite the fact that pioneering applications tend to each add their own, cool little interfaces that sometimes propagate back to the OS, it makes more sense for the OS to have provided the cool interface in the first place and for the 'application' to simply provide whatever function is needed. I'd go a bit more extreme and say we should move to an entirely 'NoApplication' framework... such that the OS itself provides one massive, consistent interface to all utilities... much like a video-game. Or a WorldWideWeb

Samuel's two points don't really seem to be arguing the points I made. Skinned media players get used because they are pretty, not because they are usable. It's true that there are inconsistencies in the Windows interface. Those inconsistencies are there because Windows tries to achieve consistency through convention. It relies on the individual application programmers to enforce the consistency. As long as those individual programmers are fallible, there will be some inconsistencies like the ones mentioned. That doesn't mean that it is wrong to try to achieve consistency through some other means.

Computer keyboards are the way they are because of economics, not technology. Sure, they could be improved, but not in today's world. To do it right, you'd need much more vertical integration in the industry, or true separation-of-concerns layering for interface software. I prefer the latter, and I agree that NoApplication is the way to do it. I've advocated it in the past on this wiki.

-- MichaelSparks

Skinned media players get used because they are pretty, not because they are usable. Thank you for re-iterating my point. Think logically -- if they get used because they're pretty, and not because they're consistent, then therefore consistency has nothing at all to do with the issue. And since they're used so often, one must expect that use of said software must influence the user's expectations of software on that system. Those inconsistencies are there because Windows tries to achieve consistency through convention. This sentence makes no sense. When Windows 95 was released, not only were the conventions hardwired into the GDI, but a thick book was released by Microsoft on the subject too. What happens next? Microsoft rapes their own style guidelines in Office, with warm-n-fuzzy effects like smooth-dropping menus that not only behave differently from the system provided ones, but look different too. Then, they introduce the concept of hidden menu entries, using the excuse that, "menu items you don't use are not displayed." Oh, yeah, those menus look different too. Well, maybe I didn't use them because I didn't know they were there. It took me some time to figure out I had to click on a special item that is nigh invisible to show the whole menu. Windows 2000 was released with smooth-fade-in menus, though they DID make the "show all menu items" meta-item bigger, and even had a drop shadow to them. By this time, Microsoft had a pretty slick, and moderately consistent UI ... again. Then, Windows XP is released which combines the worst possible of WinME with Windows 2000, with the most butt-ugly windows ever known to man. So ugly are they (and the rest of the visuals), that it is distracting to my work. This is such a problem, in fact, that Microsoft left the old UI preferences in the OS, to be selected via the display properties for those of us who actually want to get stuff done. And, now, Windows Vista is even worse, with horrible color selection, the most bizarre possible menu placement, and a start menu that consumes some 75% the screen when displayed.

And, each and every release of Windows was preceded by a version of Office or Media Player that anticipated these UI changes (in the case of Vista, it was both Media Player and IE7 that anticipated the GUI changes). But it doesn't stop here. CD burners, non-Microsoft media players (I would like to point out that Microsoft is the follower here, not the leader; I forget who precisely, I think it was RealPlayer, that created their player to look like a component stereo system on the screen -- well, they ruined everything. Set the HCI industry back 35 years!), tax preparation software of all things. Ever use TurboTax? recently? Is there a reason they had to go and wholesale re-invent the ****ing GUI? Adobe PhotoShop is another example, where they try to half-*** the MacOS System 8 UI on Windows. Firefox is another, choosing to reimplement the GUI instead of using the host-native. Portability is no excuse here; they could just as easily used wxWindows for their cross-platform GUI and be done with it. Ahhh, but then you lose the themability. Yes, gotta have that.

All you need to do is open your eyes, and you'll see quite clearly that there never has been any consistency on the Windows platform. Never. Nobody is trying to be consistent with anyone else. It's interesting how Windows applications are tending towards chaos, while Linux is at least starting to converge on Gnome and KDE for some aspect of consistency. But, again, we still have retarded media players with retarded GUIs, so we're not completely free of this muck yet.


Skinned media players get used because they're prettier than the alternatives, none of which provide significant consistency. And your rant on Windows aside (we've all had such urges and sympathize deeply), Michael stated that Microsoft tries to obtain consistency through convention; he then went on to disparage their success. I think you'll find that MichaelSparks (feel free to follow the link) is no fan of any popular OS today; I expect he looks forward to the next true generation of operating systems... something more than a mere incremental step from the last. Discussions on AutomaticVsManualPlacement (please note the topic of this discussion, Samuel) and the related ObjectBrowser are associated with the next KillerOperatingSystem and NewOsFeatures (in particular, the study on UserInterface). Moving to a NoApplication framework means marginalizing the ability to create software that 'takes over' the screen and computer or even part of it (like RealPlayer and PhotoShop); instead, the goal is to provide something like the flexibility and consistency of the *nix standard utilities command-line and the associated shells, but in alternative HCI environments (including visual and audio components, and additional sensors of the human rather than just a keyboard).

It could be the complexity, but there are other candidates. I organize the items in my house, but not the items in the library. It makes sense that when finding objects in the library, I would not know where to look. When I find books in my personal "library" (admittedly small, but much larger than 20 items), I usually don't remember where each individual item is located, but I still find them quite easily. It could be that they have distinct sizes and colors, making them easy to spot. It could also be that I have a way of organizing them that I do remember. I've noticed that when revisiting old source code after a year or more, I may not remember where I put a particular function or piece of functionality. I can usually find it though, by thinking about where I would put it if I were adding it now. If that fails, then there is always the search feature.

When I compare all of that to how I work in "constrained placement" systems, I don't see any advantage it has. The library books are kept in sorted order, but nobody finds a book by binary-searching the entire library. At best, that works if you already know the name (or author, etc) of the book you are looking for. I can't go searching for "that one book... what was it called?". -- MichaelSparks

Library books are presented using a monocline grouping. There are sections but the items of each section are presented continuously with other sections. When looking for particular books, everyone looks them up by section and then they do indeed binary search through the section. Browsing is a completely different issue and people either ask the librarian or they do a linear search through a section.

One of the reasons why you can't compare your library to a computer monitor is because your library is much bigger physically. So you can afford to manually place a bit more than 7-20 items, and this says nothing about computer systems where you have 10,000 objects for a single user system and 10 billion objects for a multi-user system. -- RK

[Our sense of navigation is deeply rooted]

Using automatic placement as the only placement mechanism can work only if it is continuous, predictable, and reversible, exactly because of our spatial memory. (I bet part of how Michael finds his books is vague memories of where he left them). -- GunnarZarncke

What do you mean by graph layout?

Most usual layouts are variants of plain sequential lists or trees, but this usually captures hypertext or any networks rather badly. Say you are navigating thru the users data with an ObjectBrowser. I imagine, that you have an object personalBudget, which is located in you "directory" richard/personal/financial (say). But you use it very often, so it is referenced also on richard/desktop. This could be a symbolic link, but lets assume, that all references are the same and we have garbage collection, so this is already a graph. You can of course always display this as a tree by replicating (virtually) nodes as needed, but a graph browser would be better.

I plan to have highlighting provided by multiple levels of selections. So someone can select a bunch of objects and then work on a completely different selection. There's also the possibility of ContinuousTransition, making the objects fly around to their new placements in the sort order instead of teleporting.

One of the reasons manual placement is so bad is precisely because it doesn't preserve reversibility. Once you've used any automatic sort, it's generally impossible to revert back. And if you've sorted then manually placed a couple things aside, then it would be impossible to keep that second manual placement when you revert to the first. -- RK

I agree, that having manual placement as the only (or main) method has its deficiencies, but not having it at ones disposal at all is problematic too. I suggest, that you add manual placement as kind of a special user defined placement mechanism (maybe even with special purpose controls). -- .gz

What purpose could it serve? Currently I've got a reasonable space for temporary storage, a stack of sets 3 deep, and subgrouping in aggregate nodes (eg, directories) for permanent storage. If a user wants to place something somewhere, the proper thing to do is to rename it. Which means that all objects must be able to have multiple names / links, and that the system can easily find them so the UI presents them all, but I have that too.

Graph layouts are usually double-ended trees (one upward and one downward) centered around the currently viewed node. Seeing more than 2 generations around some node (5 levels of objects) isn't useful, or even possible really, and it's better to cut it to 4 levels of objects. That's the center node, 2 levels of the tree beyond the center, and 1 level of the tree that's blocking your view of the center.

I don't believe interconnections are going to be very common within any given view area, and the more interconnections are visible, the less useful they are. So I can deal with interconnects using special mechanisms. Currently, the UI chooses the link to display to the user, this is where the object will be placed, and displays a cord from the other placements to the object. Naturally the user gets to override the initial placement and an animation ensues. -- RK

I agree, that graph layout is often overkill for simple mainly treelike structure like file systems (though the cause of this may be due to the inherent restrictions of them) can do without them. But I think, that there are graph structures, that require them. My main example being real object graphs (e.g. during debugging), but also linking on web-sites or traffic plans (imagine underground stations). If you add a node, if the full graph is relayouted, nobody will be able to work with the plan anymore and has to adjust lengthly. -- .gz

Object reference graphs I use are tree-like, excluding mutual references of course.

I cannot visualize the last part. Can you be more detailed (if needed).

Sure. Imagine that you've got an aggregate node 'sex' with subnodes 'het', 'gay', 'whips' and 'plush toys'. You're looking at 'sex' so the UI displays it at the center of the screen and the four others spaced around a sub-ring from 12 o'clock to 4 o'clock. If you make a dozen other nodes under 'sex', the UI will squeeze the first four between 12 and 2 o'clock and then start a second ring beyond the first one. Okay, so on the screen you've got the first level which is 'sex' and a second level containing 4 nodes, and a third level which is whatever those nodes contain. Now let's say you've got the 'Barb Wire' movie which is in both 'plush toys' and 'whips' (makes perfect sense, right?). Since it's the first time you've visited the 'sex' node, the UI decides to arbitrarily display 'Barb Wire' as a subnode of 'plush toys'; 'whips' is an equal distance from 'sex' so that's not a basis for decision. Now, there's 4 items in 'whips' and going by the link to 'Barb Wire', it would be in position #2 starting at 12 o'clock. But an object can't be at two different places at once, that violates unity of identity, so what you have is a placeholder and something that looks like a rope to the real object which is still in 'plush toys'. At this point you can select 'Barb Wire' or perform any kind of action on it and as a pure side effect of doing so the object is dragged along the rope to the placeholder, leaving another placeholder where it was. But the usual course of events is that you move to 'whips' which brings that particular placement of 'Barb Wire' closer than the other one and so the UI repositions it. The only time the user will directly cause such a repositioning is if there's two links to an object in the same aggregate node. And in that case, they use a NOP to reposition the object. -- RK

I see, your main display "mode" seems to be centered around your WheelMenu idea. All navigation is inherently local (the mentioned 4-levels-at-most condition). And as far as I can see, it exploits this idea as far as possible (and usable). My only remaining question about these WheelMenus is about your "virtual siblings": Am I right, that this is about different views of the same object or do they serve another purpose? Another general issue: Do you think, that local search is always appropriate? What about navigation on a map (think Google Earth)? What about e.g. spreadsheets? Or is this somehow orthogonal to your navigation? -- GunnarZarncke

Ummm, wheel menus don't have much to do with display of nodes, though they have much to do with display of commands available on nodes. Aha, it took me a minute to understand what you were referring to. I don't actually have a name for the three tiers of concentric links-to-elsewhere mode of display, and yes that is the default, and main, mode of display. It's the way that aggregate nodes (eg, directories) are displayed and it's the way that any object (subgraph) whose type doesn't have a registered handler will be displayed. And it's a display mode that you can always revert to in order to fiddle with things at a low level. By the way, if you can think of a good name for this display mode, please suggest it.

As far as navigation being inherently local, you are correct. Navigation is overwhelmingly local and the only mechanism that breaks this can still be construed as preserving locality; and it is displayed in a local manner. The purpose of virtual siblings is precisely to have multiple spatially disconnected regions in visual proximity and make them appear to be connected. You can almost think of them as connected, that the system is simply dragging the connections along whenever you move on to a different node.

If A1 to A7 are virtual siblings with A1 at center, and B is a subnode of A1 then when you move to B, A2 to A7 will be virtual siblings of B. Now, if B had a real sibling B2 then B2 will be displayed as sibling along with A2 to A7. But when you move on to C, the subnode of B, A2 to A7 will be dragged along but B2 will not.

However, they're not really connected because the connections only exist in your ObjectBrowser and not in the underlying graph. What that means is that you see those connections but someone else on a different machine accessing the same graph won't see those connections, because they aren't real.

Different views of the same object, such as song and music sheet / lyrics views of a song, are provided by a different mechanism. It's bad policy to allow the same object to be viewable in different places so I don't allow that. Switching to a different view of an object would be presented as flipping the object and would be accessed using a navigation command. So that makes yet another navigation command. This discussion has turned out to be extremely useful since up to now I could only imagine flipping the view on an object using some kind of direct manipulation, which I knew was massively ugly and lame.

My definition of local search is appropriate to aggregate nodes and won't be appropriate to all object types. Usually, it will be overridden in a way that makes sense for that object type. So a map of the Earth or a spreadsheet would have its own search and sort methods. No command, whether an action or a navigation command, is hardwired into the system but rather all commands are provided by handlers (software that provides input and display methods) for specific object types. Until now I didn't realize that navigation commands would have to be redefined for each object type but only because I had so few navigation commands previously. Action commands were always meant to be highly overridden and both navigation and action commands were always meant to be symmetric, I just never figured out the conclusion.

I just reread WheelMenu and I'd earlier classified selection commands as navigations instead of actions. Well, that's annoying. I also forgot about autolocking the hand. By default the hand moves independently of your viewpoint in 3rd person mode, autolocked means 1st person mode. I'll have to list all the commands I do know and figure out how to classify them. Especially since I'm already up to 4 different meanings for the MouseKeys. There's their alphabetic meaning, and then there's camera motions (or a quasimode that changes the meaning of mouse movements), and then there's navigation and action menus. I'm not sure I have a free modifier key to add a selection menu, and I'm pretty sure I don't want to. Besides, it's not possible since certain pure selection commands (like popping the selection stack) look a lot like pure action commands (inserting / dropping the top selection into the current node). So I think that selection actions will be put in a submenu off the action menu, which makes it a two tier menu as it was always intended to be.

Going back to modifier keys, it would be useful to have a free general-purpose quasimode that repurposed the mouse in a view-specific manner so you could provide continuous information to an object. It's been pointed out on WinAmp that it would be extremely useful to provide a shuttle/jog control to navigate through a song or film. But aha! If it's acceptable to limit such input to a single object, then it's possible to use the 'zoomed into object' display mode as such a quasimode. So when you're zoomed into a song object then mouse movement has shuttle/jog functionality, perhaps even with an appropriate visual display of panning. I'd planned something similar for panning through documents. "I love it when a plan comes together." I love it when I make good plans. :) -- RK

Perhaps there is some confusion or a false dichotomy here about classification versus "default" placement. I pretty much agree with SeparateMeaningFromPresentation. However, there will always need to be a default presentation, and for the default I see no harm in letting the user customize that. If the meta information (classification attributes) are properly done, then multiple other factors can be used to find and group stuff in an ad-hoc needs-specific basis. I am a relativist (EverythingIsRelative) and don't believe in One Right Taxonomy. But I see no problem with a customized default. There are other topics about classification systems if you are interested in that. A given presentation does not have to be the end-all-be-all of classification. With computer multi-indexing, we can have our cake and eat it too. -- top

True. And I do provide multiple classification, oh how I provide it. But placement is something I'm limiting to a few options. I think 3 categories of placement, a dozen sort orders, and the ability to define custom sorts, ought to be enough for everyone. Famous Last Words I know. -- RK

Related: CoordinateVersusNestedGui, OnePileFilingSystem
CategoryInteractionDesign, CategoryUserInterface, CategorySolutions

View edit of October 3, 2012 or FindPage with title or text search