Samsung: Context Framework (2011-12)

I worked for Samsung UXCA in the Mobile Lab for two years. One of the areas I focused on was "context." Some of the questions I examined included: What does context mean? How can context be detected? And how can a mobile device with so many sensors and its embedded knowledge of the user enhance a user’s interactions with friends, family and the world around her?

My Role: Concept originator. Deliverables included the first drafts of the diagrams shown here, afterwards refined by colleagues.

Four Key Objects

My first breakthrough came in realizing that "context" is really about the user and where she is at any given moment. After all, the universe always revolves around "me," where "me" is every user. Family and friends far away are still important, of course, but at any given moment, the user is most likely interested in sharing data such as photos or contact details with people physically nearby, controlling or exchanging data with devices at hand, or learning more about the current location (or "venue" as a generic term). Thus the venue is also a natural grouping mechanism for people and devices, as well. In summary, these are the four key objects in the Context Framework: the User, a Person, a Venue, and a Device.

Further, important insights added social meaning to each of these objects:

• Me: I am the center of (my) universe. In order to power smart interactions with other people, places, and devices around me, they need to know something about me in an automated fashion, while also respecting my privacy and security.
• Person: Humans are fundamentally social creatures. The richest interactions are with people whom we love and/or trust the most. We must prioritize these interactions, while leaving space for contributions from others.
• Venue: The social meaning of a location provides critical context. People are creatures of habit. They frequent the same places again and again and often form loyalties to them -- or places that are like the places they like. But people also like to explore an occasionally try something new.
• Device: Smart devices are all around us and can act as on-demand accessories to provide services that make life easier and more enjoyable. Seamless, intuitive connectivity are key concepts as people struggle today connecting devices, such as PCs to LCD projectors or lose special dongles.

Six Levels of Data Richness

My second breakthrough was in realizing that there is a ton of data about any of these four key objects which can be identified, retrieved and filtered with the user's ever-present smartphone, often in association with the Cloud. If any of these objects can be uniquely identified and cross-referenced with the user, then rich data can be retrieved from social networks and other API services to highlight meaningful bits for the user. After all, people don't want thousands of pages of random search results. Instead, people want to learn information which enhances their interactions with loved ones, and the people, venues and devices around them.

The six levels of richness build upon one another, as follows:

• Level 1: Anonymous The most basic and pared down information of proximity, location, person, venue or device. For example, the phone may know the current latitude & longitude, but there is no resolution to higher order information (e.g., an address or venue name).
• Level 2: Basic The basic identification of a name, IP address, location, make and model, etc. To continue the previous example, now the latitude and longitude are resolved to an address and venue name (e.g., a bar in a given city), thanks to a geolocation lookup service like Google Maps.
• Level 3: Social "Dumb" social meta-data such as a real name, ownership, affiliations, social IDs, similar objects, evaluations, connectivity channels, etc. Continuing with the bar, now we can pull social data about the bar from APIs such as Foursquare, Yelp, and Facebook, including ratings and reviews.
• Level 4: Personal Personally relevant wisdom like history of likes, ratings, reviews, social shared content, presence. By cross-referencing friend lists from Facebook, for example, now we can highlight reviews of the bar by friends, friends' pictures taken at the bar, and whether any friends are currently at the bar.
• Level 5: Behavioral Time-based and activity-based relationships, like frequency, connection, and the activities involved. Over a period of time, we could deduce that the user goes to the bar most every Tuesday night after work and with a frequent set of friends.
• Level 6: Possessive Ownership-based relationship, like close relationships and family. Venues like Home and Work. Devices like Home TV, smart kitchen appliances, and personal computers. Continuing with the bar example, the user may be hired as the new manager. Now the bar becomes the user's Workplace, and the user gets access to all devices owned by the bar, such as the jukebox and TVs. Now, the user can easily connect his smartphone to the devices in the bar to push content, update settings, print invoices, check maintenance requirements on equipment, and so on.

Groups: "Hard," "Soft," and "Ephemeral"

The third significant breakthrough involved understanding different kinds of groupings of people, venues and devices -- specifically, what the user identified vs. what a smart device or service could infer. It's a fact that regular users don't organize their collections very well. They don't create or maintain contact groups very well, for example, let alone files and pictures on their computers. As a result, it's critical for a smart context-oriented system to infer how the user relates to groupings of people, venue and devices in order to best serve them.

The six levels of richness build upon one another, as follows:

• "Hard" Groups These are groups which the user manually defines, and may or may not maintain well. Examples of people include speed dial lists on the phone, Facebook lists, contact groups, and so on. Examples of venues include favorited restaurants, addresses of known family members, and the user's "Home" and "Work" addresses, such as those collected by Google Maps and Waze.
• "Soft" Groups These are collections which the phone or service infer, so there is a high level of confidence in the contents, though there may be some errors. For example, various social groupings could be inferred based on frequency and intensity of interaction across communication channels and in-person interactions. Venue groupings could be inferred based on patterns of frequency and recency of visits.
• "Ephemeral" Groups These collections are based on in-the-moment relevancy. For example, people groupings may be based on a shared activity or event, attendance at a professional meetup, sporting event, concert, etc. A device grouping might be inferred because of an event, activity or special occasion, such as an airport Wi-Fi or displays at a baseball stadium.