A lot of people seem to be using “big data” as a proxy for systems at scale and the data that comes with those systems. The general suggestion is that if you have a large system with lots of users, there must be patterns hidden in that data. And it follows that those hidden patterns must be worth something to somebody (right?)…so there’s gold in them there digital hills. (So many references to prospecting in the data world; mining, sharding, etc.)

I had the good fortune of hearing Cesar Hidalgo this week the Media Lab. He spends a lot of time thinking about networks and large data sets, and he had some great thoughts on the topic. In his talk, Hidalgo defined a nice framework to distinguish Big Data from a lot of data. He had three simple qualifying questions.

- Do you have size? – This is pretty relative to the problem you’re working on. But it’s usually in the hundreds of thousands/millions of records. You’ll need enough to provide some statistical significance across your population. But the greater the set of data the more edges you may be able to discover.

- Do you have resolution? – This brings some analysis to the data at hand. Just as all rock does not contain gold, all data does not contain (new) patterns. Low-fidelity data might be all customers transitions with order-level (total amount spent, etc). High-fidelity data would be all the customer transitions with item-level data, (the thing the customer purchased to make up the transaction.) Visa has the former and Amazon has the latter, and it’s no surprise Amazon knows you better. High-resolution data will illuminate new patterns, like Target’s recent misstep of identifying a pregnant teen before she could tell her father.

- Do you have scope? – This question starts to consider the reach of your data. Are you only gathering data against a very focused problem, or are you gathering data that will give you insight beyond your core business? Being able to understand patterns outside your immediate market will create new opportunities for understanding. As an example, Hidalgo spoke about telephone companies, who know your calling patterns, but also can also make determinations around mobility patterns because they know which cell towers you’ve used during your day.

So, though there’s a lot of noise around this space there’s a lot to be done here. And as the hardware, software, and services companies wind people up to capture more data, there will be more patterns to discover – this space is very self-fulfilling like that. Along those lines, this stat came up during the talk: 70% of all data captured about people it’s gathered by machines. So as we put more sensors in everything, we’ll push this ratio further.)

Getting beyond the hype, I’m excited to see what type of new patterns emerge from deeper analysis of data. There’s definitely space for data scientists to unearth new patterns that help designers create new experiences. But to be certain, the real opportunity isn’t in Big Data, it’s in gaining better resolution to the problems we’re trying to solve and the markets we’re trying to serve.

(If you’re into this sort of thing, here’s another talk by Cesar Hidalgo. It’s really nice, definitely worth your time.)

In the design of the app, the developers must have really thought hard about people’s hesitance to share their location because they designed a timeline into each notification. So, if we were meeting somewhere and I was running late, I could send you a link that would display my location on map and that link would only work for a configurable amount of time, (say 30 minutes). During that time, as I moved around you could see where I was on the map. After 30 minutes, the link goes dead.

I’m pretty excited about this little bit of functionality because I think we’ve entered into a new phase of how we deal with our connected life. We have so much data and so many connections, sometimes the data or the connection would be better if it wasn’t permanent.

What if the systems that carry more temporal data really started to reflect that data’s ephemerality? Twitter is decent example of a designed to decay system, tweets only hang around for a handful of weeks. What if restaurant reviews created a year ago carried less weight than the ones made last week? What if past-date promotional emails just disappeared from my inbox? I have loads of weak Facebook connections that I wouldn’t miss if they just expired? (No offense, but that let’s me focus on the people I have greater connection with).

Right now we live at the end of the digital firehouse, everything just lands in our lap and we have to decide what to do with it. Some of The most meaningful online interactions mirror their real world counterparts. For the moments that matter now but not later, we will begin to have to design for disappearance.

Yesterday I was interviewing with a colleague of mine, Joe. Without realizing it, I think he blurted out a pretty perfect description of the idea of business design. He simply said, “we hack business models.”

I really love that statement because of all the implications of the idea of “hacking”. For me hacking implies that you’re working with an existing system and pushing and pulling on its boundaries to see what will happen. Tools can be crude and fast, but there is an eye to understanding and evolve the larger system. Hacking implies that what you’re doing isn’t a science, but there’s probably a lot of underlying laws ad principles involved. There’s no certification to be a hacker, but not everyone can do it. And to be a good hacker, you have to be pretty curious, confident, and inspired.

As I frame Business Design loosely as a hacking exercise, it also starts to draws some boundaries for what is and what isn’t business design to me. Businesses tweak their model all the time, and not every change is a design. If you increase the price for your goods, that’s not really design. If you change your entire pricing structure to communicate a new type of value, that’s probably business design. Netflix raising rates isn’t business design. Netflix launching a streaming-only pricing option is definitely the result of a lot of hacking and some pretty smart business design.

All this hacking leads me back to the idea of a system. Businesses after all are systems that create/provide value. That’s very academic sounding, but thinking of a business as a system that must remain in balance is sort of the first step to being able to frame and solve problems differently. (And there’s a ton of companies who think of a business as a kit of parts.) These systems have many interrelated parts (and people). As you add or remove some element of the business, a different component will be affected. As you design the customer experience, you have to design the business model that supports it. As you design the business model, you have to think about what sort of experience you can provide.

It’s all about the system; it’s all about balance.

At first, I was glazing over a little bit. The idea of ‘leaders of tomorrow’ is one of those phrases that’s been so co-opted by the business schools and business press of the world, it almost doesn’t mean anything anymore. But as the conversation continued, I started to realize just how massive of a challenge she was talking about.

The idea of leadership is a weird animal. It’s mostly internal personality characteristics that manifest themselves in significant ways. Good leaders see the world from a unique perspective, they get things done, they make people feel valuable. It’s easier to reflect that someone is a good leader, rather than project that they will be a good leader.

After a lot of thinking and conversations, I believe that what makes a ‘leader’ has to do with their levels of curiosity, confidence and inspiration. Of course there are lots of other characteristics at play, but those elements seem to be the three traits I see over and over that define people and how they become these strong leaders. There are many talents good leaders learn over time, these three feel a little more innate.

Now the interesting thing about these characteristics is that they don’t have formal outputs, they’re personality components. They drive how we react internally to our external environment. You see evidence if these characteristics through storytelling. These personal stories help explain the energy that moves people between moments, the ‘why’ not the ‘how’ or the ‘what’. For example, you spotted an opportunity somewhere because you are curious about the world you live in, you were able to attack a problem because your had confidence you could solve it, or you designed/built something because you were inspired and were moved to action. Etc, etc.

So, as the conversation continues, I start to realize just how curiosity, confidence, and inspiration work like the physical engine in a car or a virtual engine in a video game. They need inputs to produce outputs, and, depending on their design, they deliver radically different things.

This is the point at which I started to think about how the idea of leadership has changed. People that are twenty years old today grew up with massively different inputs than people who are forty, (and in someways we’re culturally trying to fit them into this classic leadership definition.) These people may lead in the same style, they make some of the same decisions, but the way they find their confidence, inspiration, and curiosity to make those decisions is so different, we can’t even know how it works yet.

Today, anyone curious and inspired enough can have massive amounts of data from completely different industries to consider, they have global networks they can learn from, and possibly most importantly, they can operate in near-real time. Our connected world is probably the greatest force in upending the classical definition of a leader. The problems they will tackle and outcomes they will know will probably be drastically different, but I still think those three characteristics will survive (they’ll just function very differently).

I’m definitely not doing the conversation justice, but You can see how it’s easy to focus on a lot of misleading elements when we try to identify leadership. It’s easy to go for major achievements and heroic moments, but the ‘why’ behind the ‘what’ always will tell a deeper tale.

There’s much more to think and write about here…this is sort of a beginning.

What do you think?

@LenKendall – I’ve got Park Place for McDonald’s monopoly. If you have boardwalk I want to be your friend

This made me think about how times have changed. When McDonald’s Monopoly was originally designed, the world wasn’t connected. So the possibility of you finding the elusive Boardwalk piece to complete your set and win millions was extremely slim. Today, in the connected world, I wonder if you would have a better chance? (Think Lazlo Hollyfeld.)

Ionically, from McDonald’s perspective the probability of you winning today was the same as it was 10 years ago. McD’s releases 3 Boardwalk pieces into the world and those pieces divided by the total number of play pieces released is your probability. The underlying assumption is that the only pieces you can play are the pieces you earn through buying fries/drinks/burgers. It doesn’t work that way anymore, (and hasn’t for some time). In 2007 people were selling pieces on eBay. The contest just launch and nothing’s changed..

The best inspiration/example to explain the difference between yesterday and today is the DARPA balloon challenge this past December. DARPA wanted to know how fast a networked group of people could solve a large-scale, time critical task. To learn, they offered $40k in a challenge that involved releasing 10 8-foot balloons in secret locations across the US. It took a team from MIT less than nine hours because they created a pyramid scheme around the challenge. Before the challenge they issued the following message:

“We’re giving $2,000 per balloon to the first person to send us the correct coordinates, but that’s not all — we’re also giving $1,000 to the person who invited them. Then we’re giving $500 whoever invited the inviter, and $250 to whoever invited them, and so on…”

So imagine if a networked entity (like 4Chan) decided to unleash the same wrath on McDonald’s that they did on Time’s Time’s Person of the Year. There are already Facebook and MySpace groups with the same intention. There was a Andriod app in the UK for the last contest.

Let’s say only 5% of McDonald’s customers are capable of pulling of this sort of collusion, should McDonald’s design the game differently? (It would cost the same either way.) Should the game be built for the connected world? Individually, those consumers aren’t that significant, but collectively they could screw the game to the wall.

I’m sure McDonald’s has all sorts of rules to prevent this sort of collaborating, but maybe they shouldn’t? The whole contest is about marketing and buzz. maybe they should invite people collude. The cost to McDOnald’s would be the same, but the engagement from their consumers would be radically different. They would reframe their brand in an entirely different context, you would have stories celebrating how people were collaborating to win. The game would be about collaborating to win, not gobbling more McDonald’s. It feels like the publicity alone would reach new audiences and meet our culture where it already is.

According to the foreword in the report, the author ( Heather LeFevre ) found herself in a pretty normal predicament; she felt she was underpaid, but couldn’t prove it. So instead of sitting on her hands, she put together an anonymous survey and sent it out to her network inquiring about their skill level and pay scale. She promised to share out the results and she’s been conducting this experiment for a few years. So, with a cheap web survey and a decent address book, she completely turned an age old process on it’s head.

This is pretty inspiring for me for a few reasons. First, instead of wringing her hands that she didn’t have the information to figure out her problem, she just went after the data. Instead of reinventing the wheel, she used simple tools she had at her disposal- an anonymous survey and an email. The data we don’t have often seems to be the first roadblock to progress; we don’t start because we’re not sure. This is such a great example of how to keep it simple and get it going.

Second, she solved for her problem, not all the world’s problems. If she would have stepped back and thought to herself “this is a big idea, how can create a salary report for the entire industry” she probably would have failed. Even limiting to the industry, she probably wouldn’t have gotten enough responses to complete the first report. By keeping the effort small, she could actually engage her audience. There are salary comparison websites all over the web (Glassdoor.com, Salary.com). These sites promise to share salary data, but they never seem to get enough scale to be useful. The idea behind the concept is so big people don’t know where they fit in the process. I love how she used technology to amplify her effort and didn’t make building the tool the object of her project.

There’s a big idea here for me. It’s the same thing that drove the success of Facebook (and social media in general). How can you use technology to amplify the network, connect people and then get the hell out of the way. The Internet isn’t much different than a good house party- if you can set the stage for people to interact, the party will usually take care of itself.

Sort of funny, right? Well I LOL’d, but then I started to wonder, could this video be a fake? Without really realizing it, I found myself dissecting the clip looking for deeper meaning – it looks older, so maybe it predates Internet pratfalls…but the grainy flim technique wreaks of post-production…but the guy does start screaming quite quickly and somewhat naturally…but maybe it’s a comedian and this is a stunt.

Whether the video is real or not isn’t the point. I realized that I was trying to figure out if the video was fake because its authenticity would determine whether or not i would share that video. (Meanwhile determining the clips authenticity was also some sort of odd meta puzzle in itself. If the clip is real, i’ve found internet gold. If its a fake and i share it, I’ve been duped making me the fool.

Why, I feel this way i really don’t know. What interesting to me is that this is an evolved reaction. Had I seen this clip 5 years ago, I would have probably emailed it to half my friends, dishing out a cheap laugh over email. But since i couldn’t decide if it was real, I couldn’t send it.

It dawned on me that viral videos have changed how we consume online content. If we think someone has created something that only intends to trick us into spreading the idea to our network, we may actually not share it…which is the exact opposite effect the creators hoped for. Here’s the root of it. If we’re manipulated into sharing something, somehow we’re saying to our inner circle that we aren’t quick enough to recognize a fake. We care about authenticity because we don’t want to be tricked and we don’t want to trick our friends. This act of filtering/curating is a form of self-actualization – you don’t want to be associated with a fake.

All this made me wonder if it will become more difficult for media to be deemed share-worthy because authenticity and intention matter more and more as we build online identities. We’re in the early days of this movement, but this is playing out very rapidly. As our online identities continue to be more and more important to us, we will redefine what’s acceptable to share. Viral content as we know it now may become quickly become passé.

I started to wonder if this self-editing phenomenon effects how we create media. Sharing is still happening, but the tricking people into spreading seems to be dangerous territory for your brand. Take this Domino’s video from CP+B. The video starts as a full-on, unflinching Christopher Guest lampoon of a pizza photo shoot. It driving straight for prime 2005 viral territory. Then, about hallway through the video a protagonist chef emerges to lead us away from the lampoon and toward the core message – ‘Dominos is the real deal, we don’t need a fancy photo shoot.’ This seems like an evolution of the style; teasing with the lampoon, but pulling back curtain before the joke finishes. The spot’s well done, but I keep wondering if it were only five years ago, would the spot have just been one solid lampoon that wanted to ‘go viral’?

In a very short span of time, i think we’re getting really good at rapidly adapting to the media we consume. Much of this seems driven by our own online identity – we are what we share. So this all sort of begs the question; does culture create the media, or does media create the culture?

I started to ask myself this question: If you really wanted to beat Apple, who seems years a head of many phone providers, what would you do? I started to think about Apple’s main Achilles heal – planned obsolescence. Apple may do a lot of radical things, but there’s one thing you can set your watch to; they’ll refresh a product line about every 2-3 years. You can count on each new model to contain amazing new functionality, but that feature set is framed around one single release. (They’re beginning to break this pattern with phone OS updates, but those are rather rare.)

All this Android activity could hit Apple in their weak spot. By using an open platform Google has convinced many providers to constantly evolve a product platform. What this mean is that multiple companies are working to add more customers to the common platform, which will create more apps, and an overall better experience. This open platform is sort of working as an informal collusion amongst many of Apple’s competitors. The platform is their agreement on how the market will evolve. If you notice how rapidly this platform has begun to gain parity with the iPhone experience, you can also see how soon it could actually surpass it. Sure, the fact that the platform doesn’t have walled gardens creates some quality control problem on their apps, but it also allows for some pretty cool features, (mobile, ad hoc wifi networks.)

This made me realize that the products that a company creates can actually be a fleeting advantage. I had a strategy professor in school that used to talk about ‘economic time.’ His big thesis was that no competitive advantage could withstand the tests of time, and you could only create temporary inhibitors (patents will expire, technology will obsolesce, relationships will erode).

As Android (and other open platforms) gain ground, it sort of takes the wind out of planned obsoleteness for Apple. They’re going to have to start pushing more OS updates, they’re going to have to get better carrier partners. They created a solid game, but now a group of players is looking like they’re ready to play harder (which could be rocky for their customers). While working on the same platform will make it more difficult for any one player to surpass the others, that’s still bad news for Apple, who quickly becomes the odd man out.

Apple has a lot of other advantages that hurt the overall Android movement, but if you were Apple, how would you defend?

(Side note: I ran across this ridiculously extensive review of smart phone sales. If you’re interested, you can grab it here.)

It’s quite a funny experience because they’ve designed every little detail of the experience, yet the app just freezes and you have to hard-restart your device. From a readers standpoint, you would appreciate less flourish and more stability. It’s obvious that a team of interaction designers sweated every last thing, but you can also tell they outsourced the development portion. The app wasn’t developed and run through QA, it was built on spec.

This experience led me to think about capability and depth. There are people at VF that know the most arcane things about magazines; how layouts read, which fonts communicate subtle meaning, how something will look on paper vs. screen – they are extremely deep in the craft of print media. They don’t know anything about software; magazines don’t have bugs, they have errata.

The death of print conversation that drug on and on last year was a business case conversation – the cost of magazines would be too high, digital distribution would blow it to bits. No one really thought too much the capability of an organization, and how it helped delivery. We just sort of assume that you can build a capability if the business is big enough. This experience made me realize there a difference between capability and expertise, the difference between those to is how good you are at the details.

It’s true, those GPS manufacturers are in a pretty bad place. TomTom, Garmin, and other GPS manufacturers had their share price free fall on Google’s free GPS announcement. Why would anyone pay for a device, if they could the same functionality in their phone for free?

I think there’s more than just paid vs. free. I think the difference lies in the difference in perspective a device company has from a service company. If you’re a device company your customer buys a ‘thing’. Once they buy that thing, they’re a cost to retain until they buy again; customer service, upgrades, anything. Garmin has changed nothing about my GPS since I bought it. In the three years we’ve owned that GPS Garmin has never evolved the experience. Sure they’ve fixed bugs, but I will never get a better experience until I buy another device.

It would be a completely different relationship if Garmin was a service company. A service mindset realizes that you only have a customer if you serve them. So beyond the service being free, Google will actually interact with the customers differently that Garmin. If I were to use a Google phone as a GPS, I have assurance that Google will constantly upgrade the service. They’ll be adding ads I’m sure, but I’d also expect them to improve screen flows and consistently refine the experience.

So when people talk about the dark days for physical GPS manufacturers, I don’t really think it’s a free vs. paid argument, I think it’s about how you serve your customer. I have a lot of confidence that if Garmin or any of those players really turned out a significantly better in-car GPS experience, they could hold on to their market share. That doesn’t mean adding photo albums, or fitness feaures…do you honestly think I’m going to go running with my car’s GPS? It means you have to be brilliant at the basics, that’s what people pay you for. Garmin sells a simple touchscreen device. They could deliver a software upgrade and overwhelm their entire customer base and make a big deal about it. If you’re a device company, you can’t see that. Upgrading the interface would just be foolish, you’ve already earned those customers. Something like that would just be a sunk cost.

Good luck guys. It’s going to be a long year.