Author akallish

The Treachery of Representation

Recently, there was a shakeup inside Apple’s iOS software development group. Scott Forestall, its leader left Apple and industry watchers connected his departure with Jonathan Ive’s displeasure of Scott’s use of user interface elements such as leather, stitching and other adornments that mimicked historical objects in Calendar, Contacts and other Apple software.

Skeuomorphism is the use of iconic representations from the physical world to connect digital functions and details to what is familiar to users. For example, Apple used leather and stitching for their Calendar application, or the use of drop shadows for a curved page and multiple pages to represent dimension for contacts or for books. It is here that many users have been vocal about these faux details that seem to look backward and in the end make these software applications look quaint and kitchy.

This issue came into focus when Microsoft released Windows 8 using a more graphic and flat user interface language called “Metro”. If one compares Metro to Apple, there is a stark contrast between the two and Apple’s visual language looks dated and even quaint – especially in context to the industrial design of Apple products which are sleek and minimal:

” . . . Apple veterans, and industry insiders hostile towards Apple’s approach to software design. Equally eye opening was the number who genuinely praise Microsoft for its novel approach for Windows 8, the most radical redesign to date of the world’s most ubiquitous operating system. The criticism and controversy, much of it revolving around a trend called skeuomorphism, reveal chinks in Apple’s armor rarely visible to those outside One Infinite Loop.” Source

What I find intriguing is that even Metro uses iconic representations of extinct analog objects buried in its colored tiles. The industries use of visual metaphors is nothing new. Steve Jobs visited Xerox Parc in the late 1970s to see their Graphical User Interface. Xerox used windows, icons, and menus (including the first fixed drop-down menu) to support commands such as opening files, deleting files, moving files, etc. In 1975, Xerox engineers demonstrated a Graphical User Interface including icons and the first use of pop-up menus.

The first metaphor Xerox used was the term desktop to compare a computer screen to someone’s actual desk. A desk would have a working surface, a file drawer with folders (or directories) and a trash can. This was a common sense way to understand how a computer could be relevant to a general user. The die was cast for both the GUI (Graphical User Interface) through a WIMP (Windows, Icons, Mouse, Pointer) which we still use now and is considered by many users as natural. Source

As personal computers began to proliferate, the desktop and WIMP metaphors proliferated along with them through operating systems and the software that was designed to operate with them. In the 1990s icons began to proliferate where it seemed like every function needed to have an iconic or symbolic equivalent as a way to connect computers to everyday life.

Print = A Printer
Email = An Envelope
Save = A Floppy Disk which then became a paper clip, or a down arrow
Delete = A Trash Can

These icons have been in use for years, but there have been discussions about using a floppy disk icon for save as floppy disks have not been used for over a decade. If a user has never seen a floppy disk, then will they know that it means save? The larger question is when does a medium begin to have its own language and vocabulary, even if it replaced previous technologies? Every new medium emulates a medium that came before it:

Johannes Gutenberg created movable type in 1439, an innovation that speeded up book production and allowed for multiple copies of one book. However, Gutenberg emulated calligraphic letters, which propagated large books and continued to decorate books after they were printed as people’s expectation was to have the feeling of an illuminated manuscript. It was not until Nicholas Jensen in England and the development of typography that abstracted calligraphic writing styles into a more modern, and smaller scale alphabet that made books smaller, cheaper and more contemporary.

• Early photography emulated the rules of painting in terms of subject matter and composition. It was not until photographers like Alfred Stieglitz broke with the visual narratives of painting and started photographing every day objects and situations did photography become a medium in its own right.

• In recent developments the SteamPunk movement is the contemporary world in reverse. Here, there is a desire for a victorian world of gears, pumps, tubes and other mechanical representations that act as theatre for human interaction. They are not like modern day Mennonites with a victorian veneer but rather a niche group that wants texture back in a world of increasingly abstract digital experiences.

Communication tools are substitutions for primary experience which allow humans to extend knowledge by documenting the world through media. Pablo Picasso once stated that art was a lie that told a greater truth. Jean-Luc Godard, in Le Petit Soldat stated that The cinema is truth 24 frames per second which quickly became twisted into The cinema is a lie at 24 frames per second. Marshall McLuhan also rightly attributed media structuring human discourse and in many ways constricting a greater understanding of the world. In the ultimate conundrum, René Magritte in his painting The Treachery of Images which had an image of a pipe with text underneath it stating Ceci n’est pas une pipe, or This is not a pipe beautifully articulated the tension between secondary representation and primary experience. Media are essentially syntactic structural metaphors that point back to primary experience through semantic ideas contained within it.


When is a metaphor appropriate?
When does it become an embellishment?
When does it become cliche or kitsch?
It depends who you ask.

From my perspective, a drop shadow, a glassy reflection, or a brushed metal background or element could be an appropriate language for user interface elements. In other situations they can be considered an embellishment. When a visual effect is used too much without questioning its appropriateness, it can quickly become cliche or kitsch simply because elements call too much attention to themselves and get in the way of intended meaning.

It’s important to note that not all visual metaphors are bad. Rather, it’s the excessive UI adornments of these visual metaphors that many insiders I’ve spoken with find distasteful and inherently confusing. Source

In the case of Apple, using wooden bookshelves as an iconic reference in their bookstore feels cliche and kitschy. While I am sure there are many wooden bookshelves, why does it have to be in an application that is on millions of phones, tablets and laptop computers? Wood, leather, stitching, faux book pages and drop shadows and reflective elements are not extensible as a language to other applications and actually get in the way of using applications. They also become dated and highlight a past period, rather than feeling contemporary.

What are the limits of skeuomorphism? In many website, tablet and smartphone platforms, I have been sensing a move towards abstract graphical user interfaces that rely on color, typography, grids and rules as the main vocabulary accented by arrows and supporting icons. We still have icons, which in many cases need to be supported by textual equivalents. While an envelope is a historical representation of letter, it is a convenient and accepted metaphor for e-mail, which is in itself a representation of mail. So an icon, turns into an index, which then turns into a symbol. The overall effect is that these applications feel more contemporary, streamlined, and focused.

As a good friend of mine, Dave Zihlman stated:

In general, the desktop metaphors have been losing potency as operating systems such as iOS step back from exposing applications and file systems as separate elements. Opening applications, opening files, managing windows all are fading into the past as applications that are specific in tasks, rather than general as tools, become the standard. This task centric world is managed by the “app” with expectations for the app to gather information off the web, perform a task, manage the result, and socially engage a community, all managed within a tidy container. The Macintosh interface has slowly morphed towards these notions as typical actions such as save and save as have been confusingly moved toward the iOS paradigm.

As technology becomes more ubiquitous and convergent as more and more of our media is digitally integrated, the use of metaphorical representations will also change. As the internet has become more sophisticated and rich internet applications have become more frequent, a new series of behaviors began to appear : digital objects exhibit certain behaviors such as modals, dynamic expansion, and also trigger a series of events. A move to design patterns as a way of creating standardized libraries is a systems approach to modifying object containers that have:

- graphic representation
- object behavior
- events handling code
- underlying client/server code

These patterns can have infinite strains as the four layers can be modified for customized objects. So the visual will become less important and will be balanced by other sensory interactions that will need to be integrated into Apple’s Human Interface Guidelines:

After all, Apple’s Human Interaction Guidelines (HIG) has never been just about the aesthetics of icon shadows or button alignment, but also about the behavioral aspects of application design in general. A generation ago, especially prior to the ascendency of web design, HIG was far more respected and adhered to both by Apple itself and its developers. Loyal users also noticed deviations and complained. HIG debates on public forums were not uncommon . . . iOS ought to have much better inter-application management and navigation than users fiddling with tiny icons. Source

There has been a move to create digital equivalents to touch and the way physical objects behave in the real analog world called haptics. It’s purpose is to enhance the accuracy of how digital representations of physical objects behave and any tactile feedback to users that these objects may generate. Haptics is a type of communication and could be a variant of skeuomorphism by making iconic representations of touch and physics and doing for the sense of touch what computer graphics does for vision.

Skeuomorphism is a necessary concept to make digital applications more accessible and familiar. However, heavy reliance on visual iconic, indexic or symbolic representations are not enough with contemporary digital platforms. Not taking into account emerging sensorial alternatives or additions to visual graphical user interfaces that create more immersive experiences need to be explored. All sensorial functions speed up understanding, engagement and retention of users in their interactions with digital systems.

8512 is a Most Powerful Number

I recently attended Chicago Ideas Week and there was a presentation given by Toni Maraviglia, an educator who has lived in Kenya (East Africa) for over seven years. She was teaching in rural villages and observed that many students needed additional tutoring help through questions and answers to specific subjects. She began to explore how best to deliver this model and further observed that all families had basic cell phones and that this platform could be a gateway for students to receive tutoring help.

Finding Meaning Naturally : An Interview with Jeannette Hanna

Jeannette Hanna is one of the brightest brand strategists I have known in my professional career. We first met when I interviewed for a position at Spencer, Francey, Peters in Toronto. My impression of the company was a smart, laser-focused company integrating organizational design, brand strategy and identity systems. Jeannette had a lot to do with this impression presents herself as thoughtful, infusing intelligence into conversations about the purpose and focus of branded systems. She is also an American that has become a cross-cultural translator between two countries that often are seen as the same (Canada and the United States).

Since the first time we met, Jeannette is now part of Trajectory, a brand consultancy that continues to create sustainable brands in a world of commoditization. Strong intellectual scaffolding around purpose, goals, and actions is what Jeannette thinks and consults about. These efforts led her to co-author Ikonica, A Field Guide to Canada’s Brandscape, the first systematic look Canadian brands and their cultural distinctions. Hanna’s thesis in the book – that culture, commerce and community mores are highly inter-dependent – transcends the Canadian context.

Augmenting IKEA’s Augmented Reality

I recently received news that IKEA was releasing a smart phone application that would integrate “augmented reality” technology. The company currently releases over 211 million catalogues in the United States and estimates that they have about a two-week shelf life before being tossed into the recycling bin. Source

Of course being an advocate for IKEA I downloaded the new smartphone app developed in conjunction with the advertising agency McCann. It is interesting to understand how brick and mortar stores approach to anything digital and especially mobile technology.

Thinking with Your Gut

I recently attended an AIGA Chicago Design Thinking lecture series held at Morningstar with Julia Hoffmann, design director of the Museum of Modern Art.

Being interested, I attended this lecture to understand how Julia would frame design thinking. The first thing she prefaced about design thinking is that she did not use the term or concept and had to quickly understand what it was. She stated that from what she understood, design thinking focused on three attributes : empathy, creativity and rationality.

Are We Programmed to Work?

As human beings are also animals, to manage one million animals gives me a headache.
Terry Gou, Chairman, Foxconn

That startling quote was in an article from the New York Times about the proliferation of robots in manufacturing and distribution centers. This proliferation is accelerating due to the increase in sophistication, programmability, miniaturization, and integration of vision recognition capabilities.

Is Data Experience Design?

I recently attended a presentation by the City of Chicago’s Chief Technology Officer, Brett Goldstein who discussed the city’s desire to create a smarter city.  The desire is to create a more open and transparent municipal government.

All government business is built on data transactions. The goal for the City of Chicago is to create the best data standards using open source technologies that can then be clustered and contextualized for interpretation and action. This can be done by municipal governments creating better applications to become more efficient and responsive and for citizens to adapt data to local needs. This was not as much a discussion of analytics and the meaning of data, but how data should be structured that allows for the greatest flexibility of citizens to take data files and apply them to real-world issues facing neighborhoods or the city as a whole.

IBM has been using the term smarter planet to describe a world where our environment and its objects and actions are captured by IBM to find patterns and gain insights in order to increase the performance of the planet. To most, this sounds magical and even slightly disconcerting. However from an experience design and user experience standpoint, this concept will have far ranging implications on current assumptions and interaction models.

Experience Design and User Experience models have focused on cognitive and workload issues of users and how they interact with digital systems to ease their burdens and frictions which degrade the value of digital experiences. The focus is on human to computer interactions and indirectly how it affects human to human interactions. This includes the clustering and representation of features and functions and the cognitive processes that support their utility value.

However, the role of data and computer to computer interactions are shaping more and more human to computer interactions. This is important as search, personalization, and customization are being shaped by historical data patterns between one person and a digital system, or large numbers of users that are aggregated into classes and their sum total of patterns affect the choices and results that are served to one user.

A large part of the public internet is fed by a series of databases structured by organizations accessed through front-end websites and applications by their markets. Data streams slosh back and forth between users and servers. Based on several variables such as class, location, time, date, and social networks data is structured back to the user which is served up as a series of choices for the user to act upon through event handling. “Information from all the consumer devices, in addition to data from billions of sensors and Web-crawling robots, is crunched in these supercomputing clouds, creating a Big Data revolution full of business opportunities and dangers.” Source

  • Some examples of data that is used to shape human to computer interactions through computer to computer interactions are :
  • Unique event data that the customer shares about himself or herself
  • Data that is based on specific search, or customer queries
  • Aggregate data that the company has on a specific customer over time
  • Aggregate data that the company has on a class of customers (which can include social media as a class, or series of classes)
  • Purchased data that may include geolocation, US Census, zip code, or other specific criteria
  • Data that the company uses to market products and services based on business need
  • Data that a company uses to market specific products and services at particular times

Selected choices continually refine a databases ability to be more relevant. However, business systems create computational algorithms, or sequential unambiguous instructions to be followed by a computer to further make user experiences beneficial to the customer and the company. Based on aggregate patterns and the needs of a business, users will be offered choices, usually called intelligent offers like what is found on Amazon that displays Customers Who Bought This Item Also Bought or If you buy this, you may want to consider purchasing . . . So data automation and using algorithms for digital systems to self-learn based on aggregate interactions with a system continually affects the end user experience.

In contrast, Apple’s new release of their own mapping system which is a critical component to smartphone core functionality demonstrates how data does become the user experience. Apple’s desire to have their own mapping system is due to being at odds with Google’s mapping software because of the value of the data. David Pogue in his Bits blog outlined the problems facing Apple’s maps :

“Every time you use Google’s maps, you’re sending data from your phone to Google. That information — how you’re using maps, where you’re going, which roads actually exist — is extremely valuable; it can be used to improve both the maps and Google’s ability to deliver location-based offers and advertising.” He went on to say that ” . . . when the overall data set is that huge, even half a percent of faulty data means a lot of flaws. And the trouble is, you never know when you’re going to encounter one. One wild goose chase, and you’ll find it hard to trust the software again.” Source

The wrong data provides the wrong result, which then impacts the usefulness of the representations that a user relies on to make a decision.

Experience Design focuses on creating a holistic experience from a cognitive and cultural standpoint; from how anticipation is built, to the most important interactions users have that create the right perceptions from benefits derived from the experience. It is a form of cognitive design and engineering. User Experience focuses on how to take experience design goals and determine how best this can be achieved from users interactions with digital systems. Much of these activities are possible because of data and data that is bundled together to provide relevance to a user’s cognitive and workload performance can be viewed as a service.

Service Design is the activity of planning and organising people, infrastructure, communication and material components of a service in order to improve its quality, the interaction between service provider and customers and the customer’s experience. Source

One may ask do users of digital systems think that their online actions are being affected by data exchange and modeling? For most, their answer would be an abstract recognition that data plays some role in what is being served back to them. If you ask the same question to a user experience team, there will be recognition in the abstract that data does play a role, but not specific understanding of how data plays a role in elements and choices that are served up to a user to make decisions and complete specific tasks.

Collaborating as a senior information architect on large enterprise systems, many user experience groups only tangentially discuss data when developing user experience models, workflows and user interface elements through specific screen types – and only in specific tactical ways emphasizing the interaction. I have been only in a few meetings when data models are discussed, but not in a way that clearly connects data to how it will impact the user experience or vice versa. This has always intrigued me as my experience has informed me that if the data model is not understood or the dynamics of data interactions are not understood, how does one know that the desired user-experience is being achieved?

Returning to Brett Goldstein’s presentation, he stated that data structure and interpretation is multidisciplinary in order to come to a series of meanings that reinforce one another. The challenges to data is reducing the number of data islands that are stove-piped but continually transact; data warehousing architecture; performance management, data mining and prediction. The data mining and prediction is the most important goal in order to improve digital systems and the performance of user experience models.

The opportunity to experience design is engaging with project teams on defining and architecting data models that support a desired series of system epics, stories and use cases. Also the variability of data patterns that affect individual personalization as well as group personalization is important to understand how these will affect the choices a user will be offered – or will affect what a user will be able to do with a digital system.

A new Window to Windows?

“It is a risk to do something new, but its also a risk to sit where we are.”
Julie Larson-Green, Microsoft

Fast Company had an interesting article “Microsoft Wipes the Slate” and the UX team that developed the new metaphors for Window 8 in general and Windows Mobile in particular. With the introduction of the new Microsoft Tablet with the operating system and user experience has received a lot of press, much of it positive from communities that are not usually positive about anything Microsoft produces.

The company just redesigned its corporate identity to signal its transformation as something other than the historical engineering product based culture that brought such unmemorable products such as Zune, Microsoft Watch, Windows 7 and many other mediocre implementations. Microsoft’s long walk in the experience design desert and their recent stop at a more progressive XD/UX/UI oasis provides an interesting respite that either will point to greener pastures or simply be an anomoly to a company that has lost over half of its market value in ten years.

Windows 8 and the tip of the deployment spear of Windows Mobile has shown that genuinely re-imagining the experience design, user experience and corresponding user interface may leapfrog over Apple’s entrenched market perception as providing the best overall integrated digital ecosystem experience with Mountain Lion and iOS6. Windows 8 is built on what is called inside the company as “Metro” that espouses an “authentically digital” experience with a stripped down, flat user interface using tried and true user interface language of self-assured typography, color and screen compositions that support a focused user experience model to interact with digital content.
Sam Moreau, Director of User Experience for Microsoft called this effort “ . . .  the ultimate design challenge.” The use of tiles (or Skittles in Microsoft parlance) becomes a gateway to a diverse ecosystem of digital gateway content housed in a clean user interface. Moreau looked to the work of the Bauhaus, DeStijl, Swiss School and pre-WWII modernism to the most elemental forms and functions became their gateway to creating something very different than what Microsoft is associated with – mediocrity, function rich but experience poor products, services and content.

When the Microsoft Store opened to much fanfare where I live and was only 150 feet from the Apple Store, I was greeted by a self-assured space housing a cacophony of different OEM hardware that run Microsoft software. When I approached an HTC phone loaded with new windows mobile, the form factor of the phone with its large surface coupled with interacting with Metro was a pleasant experience. A Microsoft sales person approached and he informed me that Microsoft was providing integrated start-up experiences when users bought different OEM products. This included creating consistency in hardware form factors so certain buttons would always be in the same place regardless of manufacturer. He also discussed that the way products turn on and access functions would become more harmonized so users would associate the experience across the different OEM products.

Upon hearing this bit of news, I was both pleasantly surprised, and a bit glad that Microsoft was starting to understand the art of integrated merchandizing and product development (albeit 25 years later than it should have). Customers now expect integrated experiences that are packaged in a unified envelope. This is an extension of what is referred to as the Starbucks effect, where they transformed a simple product called a cup of coffee and created a larger narrative through unified environments, services, associations and expressions into a new experience that was priced at a premium. Microsoft, taking its cue from Apple and increased expectations of users in general stated that “[Microsoft} is placing an emphasis on design, because the dollars sit there. They're looking at Apple's market cap.

What is interesting is that Metro is questioning Apple’s current skeuomorphistic library of leather, glossy reflections, wood book shelves and drop shadows to imply three dimensional pages and as markers to past objects. Moreau calls these details ” . . . useless distraction . . .”. Metro has a unified syntactic symbology for all functions and is starting to migrate to other Microsoft products like XBox.

This points to a much larger trend of new technologies that are creating a much richer internet.  The world of gaming is impacting wider digital development of business systems through integration of game theory, multi-user environments, and social behaviors are driving new types of feature, functions and digital vessels to deliver these behaviors. The rise of data driven content, animation & responsive objects, sensors, mash-ups, haptics, gestures, and adaptable digital objects through augmented reality platforms. These trends have put stress on over ten years of user interface and user experience assumptions of the structure of what constitutes a web page, buttons, iconography, and navigation principles. The WIMP interface (windows, icons, mouse and pointer) that was developed at Xerox Parc and refined by Apple and other computer hardware and software manufacturers is becoming supplanted by new ways to search and interact with digital content.

Pattie Maes and the MIT Media Lab Fluid Interfaces Group is one example where they are exploring emerging technologies and behaviors to create new topologies, typologies and taxonomies for XD/UX/UI :

“Why do we still use a keyboard and mouse to interact with digital information? This mode of human-computer interaction, invented more than 40 years ago, severely constrains our ability to access and interact naturally with digital content. Computer systems lack the contextual knowledge to offer relevant information when and where we need it. Further, traditional screen-based interfaces divert our attention in mobile and social situations. They are designed for a single user, and not well suited to accommodate collaborative activities.”

New theoretical constructs are being proposed and pure research explorations into these concepts and technologies are producing new types of digital experiences where the collective memory of data interactions and actual user behavior modifies digital experiences on the fly creating user experiences that do not rely on static forms, lists of options, pointing, and pressing. It is still too soon to know how these explorations will impact mainstream implementations of digital platforms, but we are already seeing bits and pieces of changes and a clustering of trends around more common user interface elements.

Based on my experience in interacting with Microsoft personnel, design is still an optional add on that emphasizes downstream expressions, rather than the important upstream corporate ethos and values informed by design that drive behaviors and skills for great outcomes. But at least Microsoft is willing to consider the issue of design once again and not cling to outdated historical precedent. After many years of not embracing the internet, Microsoft through its release of Silverlight and Surface applications and platforms is providing the company with a second chance at user experience relevancy. The acceptance of Windows 8 and of their mobile platforms using Metro’s principles will be interesting evidence if the installed user base is willing to accept these principles.

What is it to be human? : Digital tools may provide a series of answers

What is it to be human? Fundamental questions such as these are both simple and complex because it is asking for the fundamental answer for an abstract concept. For many, this question conjures up the role of DNA and the millions of years of evolution that make us who we are today, or conversely the role of our environment and culture that shape us as one node of billions of nodes. To professionals who are involved with these debates, both evolution and environment interplay with one another to define what it is to be human.

Searching for Context : Interview with Sharon Poggenpohl

Sharon Poggenpohl has had a lifelong fascination with design. She is a dedicated educator that has focused her career on working with masters and PhD design candidates in the United States and internationally.

I first met Sharon while attending graduate school at Rhode Island School of Design where she became an advisor on my thesis activities. Her accessible style and genuine interest in collaboration was of great benefit to me, both during school and afterward when I had to collaborate with many disciplines in creating large digital systems.

She taught at Rhode Island School of Design, Illinois Institute of Technology’s Institute of Design, and until recently at Hong Kong Polytechnic University’s School of Design. Her experience in teaching and administering both masters and PhD programs gives Sharon a unique perspective on the challenges and potential of design education. Her desire to both understand and create the context for design and support that context through research and collaboration is rarely discussed. Addressing actual empirical behavior — rather than remaining at an abstract rhetorical level is Sharon’s focus. Sharon is co-editor of the recent book, Design Integrations: Research and Collaboration and publisher of Visible Language, one of a few high-standard journals in design.

What struck me about the interview with Sharon was how thoughtful and disciplined her answers were; they reinforced my direct experience with her at RISD and over the years. Design education has been at a continual crossroads for decades and the cacophony of design educational models, skills and outcomes at an undergraduate, graduate and doctoral level feel in many ways insurmountable bordering on a wicked problem. Sharon has very common sense foundations to address the continual problems that degrade the value of design education and how focusing, assembling, editing and delineating educational goals and methods can increase the level of rigor of design programs.