Blog

Can census data be fun?

At first glance, you might not think median home prices or transit data would be all that interesting. But Sitegeist, the location-based open data app from the Sunlight Foundation and famed design firm IDEO, is proving that good design can make public data cool.

The concept seems simple when you open the app, the app uses data and information pulled from a wide variety of sources. Sitegeist allows users to set their location and then flip through several screens to explore data tied to their specific location. The screens include: people (median age, age distribution, household income, gender, and political contributions), housing (median home value, average rent, percentage renting v. buying, and commuting patterns), local attractions (popular businesses, local movie theaters, and restaurants), environment (weather), and history (median home age and number of housing units.) The app is available for both iOS and Android.

The sheer volume of data that the Sitegeist app provides for each location is a reminder that between census data and public APIs from companies like Yelp, Foursquare, and Dark Sky, users can put together a relatively comprehensive profile of a geographic location. The Sunlight Foundation put together a “how we did it” blog post, explaining the technology behind the app and how they used different APIs to build a comprehensive product:

All of this wonderful data resides on our servers. When you select a location, the latitude and longitude are passed along with the ID of the pane you want to view. Of the various geographies we keep track of (census tracts, ZIP codes, etc.), the boundaries of any shape that contains your location is found. This uses a customized version of Chicago Tribune’sboundary service. We then match those geographic boundaries with any data we have, making calls to third-party APIs as needed. The collected data is rendered into templates and returned to you as the beautiful infographics you see in the app.

But presenting the information in an intuitive manner with help from IDEO was just crucial to the app’s success as the data that went into it. The Sunlight Foundation won a grant from the John and James L. Knight Foundation to hire external designers to work with them on the Sitegeist app, which was the third in a series of data apps the group worked on. The Sunlight Foundation is focused on using technology to promote openness and transparency in government, an idea we’ll be exploring at GigaOM’s Structure:Data conference in New York in March.

“These apps are designed to make the case for why open data is important to people,” said Tom Lee, director of Sunlight Labs. “And I think it’s really accessible for people. It’s visually striking, which is a credit to IDEO’s designers and Sunlight’s designers, and everyone has an investment in their neighborhood.”

I downloaded the app about two weeks ago, and I’ve found myself opening it in different neighborhoods around San Francisco to see what kinds of people live there. The app would be an ideal travel companion for exploring completely new areas, but even comparing two spots in SOMA (on the left) and Telegraph Hill (on the right), was eye-opening:

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How emerging technologies will influence collaboration
• Readers weigh in: future prospects for Twitter
• AWS Storage Gateway jolts cloud-storage ecosystem

The close we get to this month’s Mobile World Congress event, the more information about new Android devices gets leaked. First up is the third iteration of Samsung’s small slate. The Galaxy Tab 3 may be one of the bigger Samsung reveals at MWC as it’s more likely the Samsung Galaxy S 4 smartphone will warrant its own event later.

So what’s the word on the Galaxy Tab 3? Samsung has already created 7- and 7.7-inch tablets, and it appears the new slate will be in available in a similar size. It could be even be an 8-inch tablet; the same screen size as the expected Galaxy Note 8.0. A key difference, however, is that the Galaxy Tab 3 won’t have an active digitizer nor supporting digital S-Pen like the Note model. That should keep costs down for the new Tab 3; perhaps even in the sub-$200 range to compete against Google’s Nexus 7.

Alleged benchmarks for the Galaxy Tab 3 indicate a 1280 x 800 display running Android 4.2.1 and a CPU capable of up to 1.5 GHz. I’m willing to bet that Samsung keeps the price down by not using one of its new Exynos 5 processors, but a dual-core version of the prior Exynos generation. It’s more likely that Samsung uses the newer chip in the Galaxy Note 8.0.

HTC will also be at Mobile World Congress but it may debut its newest flagship phone even sooner. I’ll be attending an HTC press event in New York City later this month, where the company could out the HTC M7. There’s little doubt the phone exists now that a video and pictures of HTC’s CEO, Peter Chou, shouting out “M7!” and using the phone’s camera at an year-end company celebration.

Little information is known about the M7 other than it should have a new version of HTC’s Sense user interface based on some Android Police photos. That’s to be expected, given the company’s yearly iteration of its smartphone software. I suspect this to be a large-screened 1080p device — perhaps 5-inches diagonally — with one of the latest Qualcomm Snapdragon Pro chips running a version of Android 4.2. I wouldn’t be surprised for HTC to put it its highest resolution camera sensor to date in the M7 as well. We’ll see as soon as Feb. 19, which is when the HTC press event is planned.

Enough about phones and tablets that are coming soon: How about a phone that’s coming after being sold out for so long? Amid quick sell-outs, the Google Nexus 4 appeared back in stock on line and many folks are reporting shipments in under 48 hours, per Geek.com. My recent check of availability says both the 8 and 16  GB versions of the unlocked phone are now shipping in 1 to 2 weeks if you order now. That’s much better than the “coming soon” message we’ve seen for weeks on end; perhaps LG is kicking up the production pace as noted two weeks ago?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• CES 2012: a recap and analysis
• What the Google-Motorola deal means for Android, Microsoft and the mobile industry
• Mobile 2012 and beyond

In case you missed any of our thrilling podcasts this week, this is your chance to catch up! There’s something here for everyone:

Kickstarter CEO Perry Chen talks about creativity and crowdfunding.

(download)

Kevin Tofel answers your questions about gadgets. For instance, who should get a 128 GB iPad?

(download)

Our weekly news wrap up tackled the new Blackberry, Facebook’s mobile moves and Bob Sutton, author of The No Assholes Rule, talked about the continuing excellence of Netflix.

(download)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• GigaOM Research highs and lows from CES 2013
• How HR can make the case for workforce analytics
• The 2013 task management tools market

Santa Claus and UFOs may indeed have something in common. As Michael Shermer shared at TED2010, they are two things that engage the “belief engine” that is also known as our brains.

Michael Shermer: The pattern behind self-deception

In his talk, “The pattern behind self-deception,” Shermer debunks superstitions and urban legends and shares why we are prone to believe in them — because of our brain’s hardwiring for survival. He explores what is behind our shared desire to believe, even when it means actively deceiving ourselves — and potentially others.

Today, TED Weekends on the Huffington Post features Shermer’s compelling talk. Below, find some of our favorite essays that accompany it, taking a look at deceptive tendencies.

Michael Shermer: Why We Deceive Ourselves (Sometimes)

As the publisher of Skeptic magazine I am often asked what I mean by skepticism, and if I’m skeptical of everything or if I actually believe anything. Skepticism is not a position that you stake out ahead of time and stick to no matter what.

Consider global warming: Are you a global warming skeptic? Or are you skeptical of the global warming skeptics? In this case, I used to be a global warming skeptic, but now I’m skeptical of the global warming skeptics, which makes me a global warming believer based on the facts as I understand them at the moment. The “at the moment” part is what makes conclusions in science and skepticism provisional.

Thus, science and skepticism are synonymous, and in both cases it’s okay to change your mind if the evidence changes. It all comes down to this question: What are the facts in support or against a particular claim? Read the full essay » 

Laura Cococcia: Santa, Self-Deception, and the Survival Instinct

It takes almost no effort to make a child believe in Santa Claus. Step one? Put some presents under a Christmas tree every year. Step two? Tell the child that Santa Claus put them there. Result? You’ve got a Santa-believer on your hands, and you have evolution to thank for it.

Belief is the topic of science writer and historian Michael Shermer’s TEDTalk on the patterns and Darwinian instincts behind self-deception. Even as a noted skeptic — in fact, as editor-in-chief of Skeptic magazine — Shermer asserts that belief is the natural human state, and that it is science and reason that seem unnatural to us. During his time on the stage, he pulls away the curtain that divides us from our understanding of why we are prone to believe ultimately illogical, at times fantastical, things.

Santa Claus and, as we’ll see, his mystical compatriots the Easter Bunny and the Tooth Fairy, are cultural as well as psychological phenomena that serve as apt examples of our propensity for self-deception. However, they aren’t the examples Shermer employs in his explanation. After all, their creation is too recent to represent the development of belief. He must travel a long way back in time. Read the full essay » 

Laura Kray: What if…

“Jamal Malik is one question away from winning 20 million rupees. How did he do it? (A) He cheated, (B) He’s lucky, (C) He’s a genius, and (D) It is written.” — Slumdog Millionaire, 2008

In the opening scene of the critically acclaimed film, a title card appears on the screen to raise the possibility that fate has intervened to guarantee that an uneducated orphan from the slums of Mumbai will defy the odds and become a game show champion. Throughout the film the possibility that Jamalʼs quest would end in failure looms large. Ultimately, the sheer improbability of his success suggests the universe conspired to provide a set of questions that he was uniquely suited to answer. In so doing, Jamal was reunited with Latika, his first and only true love, and his romantic destiny was fulfilled. Viewers are left with the sense that something so improbable could not have happened by chance alone, and the underdogʼs fate must have been written in the stars. Indeed, the filmʼs immense popularity is most likely enhanced by its universally appealing storyline: seemingly random and disconnected events are, in some unfathomable sense, intertwined by fate.

As a researcher at UC Berkeleyʼs Haas School of Business, I study how people reflect on fateful experiences to construct the story of their lives. Let me ask you, have you ever considered how your life would be different if certain pivotal experiences, or turning points, hadnʼt occurred? The scientific term for reflecting about “what might have been” is counterfactual thinking. It turns out that, rather than immobilizing us with regret, actively “what iffing” can help us to see our destiny more clearly. Even painful experiences, like loss of loved ones, can come to be appreciated for the growth and learning that they brought about by imagining how your life would be different had these events not occurred. Read the full essay » 

If you check you’re favorite automotive sites on a daily basis then you’ve no doubt seen the group of jackasses that shut down a Northern, CA freeway so they could do donuts. Well it comes to our attention that they’re now not only wanted by the California authorities, but that a few internet bloodhounds are eagerly trying to track them down as well. It’s no secret that people like to have a little fun in their rides every now and then, but doing something as stupid and reckless as this is simply unacceptable.

Source: Youtube.com

In this week’s address, President Obama calls on Congress to work together on a balanced approach to reduce our deficit and promote economic growth and job creation. 

Transcript | Download mp4 | Download mp3

If you’ve just received an email from Twitter warning that as a precautionary security measure the micro-blogging site has reset your Twitter account password, and inviting you to create a new one, you should take it seriously. Very seriously.

According to Twitter the service recently “detected an attack on our systems in which the attackers may have had access to limited user information — specifically, your username, email address and an encrypted/salted version of your password (not the actual letters and numbers in your password)”.

Details of the attack can be found in a blog post in which Twitter explains that the attackers may have gained access to details for approximately 250,000 users. A relatively small proportion, seeing that Twitter has 200 million active monthly users.

Twitter goes to explain that “this attack was not the work of amateurs, and we do not believe it was an isolated incident. The attackers were extremely sophisticated, and we believe other companies and organizations have also been recently similarly attacked. For that reason we felt that it was important to publicize this attack while we still gather information, and we are helping government and federal law enforcement in their effort to find and prosecute these attackers to make the Internet safer for all users”.

I’m among the 250,000 users who were affected by this hack and this morning received two messages from Twitter. The first warning that my password had been reset, and the other containing a link to create a new one. If you receive the same messages and, like me, are adverse to clicking these kind of links (this is the perfect opportunity for phishers of course) you can just go to Twitter and try to log in. Enter your old email address, phone number, or username and Twitter will send you a fresh reset link.

The micro-blogging site says that while “only a very small percentage of our users were potentially affected by this attack, we encourage all users to take this opportunity to ensure that they are following good password hygiene, on Twitter and elsewhere on the Internet. Make sure you use a strong password — at least 10 (but more is better) characters and a mixture of upper- and lowercase letters, numbers, and symbols — that you are not using for any other accounts or sites”.

All good advice of course. So whether you received the emails or not, now is a very good time to change your Twitter password because you can never be too careful.

Photo Credit: Julien Tromeur/Shutterstock

Last summer, I wrote that when Arrested Development returns to Netflix this spring, I’d call in sick so that I could marathon the entire season. Today’s premiere of House of Cards, therefore, was an opportunity to rehearse that experience — a much darker, moodier, Kevin Spacey-er rehearsal.

House of Cards, created for Netflix by Beau Willimon and David Fincher, is an adaptation of the British novel and miniseries of the same name, with Kevin Spacey starring as the manipulative Francis Underwood. It has a solid cast, premium cable production quality, and plenty of political intrigue to play out over the first 13 episodes, which went live at midnight Friday.

And as soon as it went live, I started watching. I made it through the first two episodes last night, then fell asleep around 2:30 AM watching Chapter 3, which I then resumed watching around 7:30 AM. Over the following five and a half hours, I was able to get to the beginning of Chapter 9, then took a two-hour break to run some errands and remember what fresh air smelled like.

By 6:30 PM, nearly 18 hours later, I’d consumed the entire first season, all 13 episodes. It was a pretty long day.

Complicated, grim, nasty and engaging

I’ve marathoned television before — in fact, Netflix’s recent acquisition of The West Wing has been a major obstacle in my productivity lately — but usually I know what kind of show I’m in for. So the first part of the experience was discovering that House of Cards was complicated, grim and alternately nasty and engaging. Kevin Spacey goes full-on anti-hero, while Robin Wright, as his chilly wife Claire, proves to be his equal. Spending a day in their world was intense.

The big thing I feel about binge-viewing is that it makes you conscious of what parts of the show really engage you. Chapter 8 was an interesting episode from a number of perspectives: Just past the halfway point in the season, the episode focuses on Spacey paying a visit to his alma mater for a library dedication; most of the action is devoted to Underwood and his old school friends paling around.

It makes for a nice calm before the storm — however, Kate Mara’s reporter character doesn’t show up at all in Chapter 8, which is disappointing because the twisted relationship between her and Spacey, I felt, lead to some of the show’s most fascinating scenes. Chapter 8, in which they don’t interact at all, was disappointing from that respect.

Fortunately, instead of waiting a week for a new episode to address that, I just had to queue up the next episode. Which was convenient. And then, the back half the season really takes off: Chapters 9, 10, and 11 each end with solid cliffhangers, and the show settles into a nice momentum up to the finale.

Binging doesn’t give you time to mull things over

However, the consequence of that momentum might be a loss of nuance. There are many elements I feel I didn’t comprehend as well as I might have, because of marathoning the show: It makes you conscious of the fact that with conventional dramas like Homeland and Game of Thrones, waiting a week between episodes creates an opportunity to mull over storylines that might get drowned out by bigger events.

For example, there’s a subplot involving Robin Wright’s growing interest in origami that doesn’t really affect the major plot at all, except when serving as punctuation in a couple of key scenes.

The origami thing lead to some intriguing character moments, but the only reason I remembered that it happened, at the end of the day, was because I’d made a note about it. The big events of the show stand out clearly in my head, but if I were to go back later and rewatch at a slower pace, I’m sure I’d discover new details.

Watched all at once, also, meant that things like the heavy product placement for Apple products and Sony video games stood out distinctly.

A game-changer, or just too dark?

Not only did Netflix publicly announce at midnight that the show was live, but throughout the day it encouraged viewers to marathon the show — cheering them on via both the Netflix and House of Cards Twitter accounts.

It's okay if you called in sick today. #HouseOfCards—
House of Cards (@HouseofCards) February 01, 2013

People have called the series a game-changer for Netflix, the subscription service’s equivalent to AMC’s Mad Men or HBO’s The Sopranos. But I’m not convinced that substituting the buzz that those shows acquire over the course of a season for the buzz of binge-viewing will pay off. It’s hard to watch television this way. Especially a show as dark and serious as House of Cards is.

If it wasn’t for the challenge of the assignment, to be honest, I don’t think I’d have gulped down House of Cards in a day. It’s heavy stuff, without much levity, and as mentioned above I think there are elements of it that I would have enjoyed more with a little distance and time.

That said, as the credits rolled on Episode 13, this was my first thought: “When does Season 2 start?”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected consumer fourth-quarter 2012 analysis
• How to manage the signaling storm in 2013
• How consumer media will change in 2013

Twitter announced late Friday afternoon that sophisticated hackers might have gained access to data associated with about 250,000 accounts on the site, and that the company believes “this attack was not the work of amateurs.” While the hacks only potentially affected a small percentage of Twitter users, the company urged everyone to change their passwords regularly and keep track of their accounts.

In a blog post Friday, the company explained what happened:

This week, we detected unusual access patterns that led to us identifying unauthorized access attempts to Twitter user data. We discovered one live attack and were able to shut it down in process moments later. However, our investigation has thus far indicated that the attackers may have had access to limited user information – usernames, email addresses, session tokens and encrypted/salted versions of passwords – for approximately 250,000 users.

The company noted that there have been reports of several high-profile incidents of Chinese hackers targeting U.S. media companies this week including at The Wall Street Journal and The New York Times. Twitter noted that it wanted to publicize the incident because it doesn’t think it was an isolated or random event, but it stopped short of identifying any likely group or country behind the attacks:

This attack was not the work of amateurs, and we do not believe it was an isolated incident. The attackers were extremely sophisticated, and we believe other companies and organizations have also been recently similarly attacked. For that reason we felt that it was important to publicize this attack while we still gather information, and we are helping government and federal law enforcement in their effort to find and prosecute these attackers to make the Internet safer for all users.

While the hackers may have gained access to 250,000 accounts, that’s still only a small percentage of regular Twitter users. Twitter announced on Dec. 18 that it had more than 200 million active users. So the hacked accounts would make up less than one percent of monthly active users.

Back in 2010, Twitter settled with the FCC after hackers found they were able to access user accounts including President Barack Obama’s account by guessing passwords and were able to send fake tweets. The company downplayed the security risk at the time, although it wasn’t the first security lapse the company faced.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How to navigate the new world of digital advertising
• Facebook’s IPO filing: ideas and implications
• Finding the Value in Social Media Data

Some days the sorry state of news reporting really baffles me. Today I read numerous headlines claiming that Android tablet share surged past 50 percent in Q4, usurping iPad — all using numbers I wrote about a day earlier. The one on CNN — “IDC says Android is the new king of tablet market share” — got to me. Immediate reaction: “What did I miss?” But in looking over the numbers, nothing really jumped out that IDC said any such thing. Sure iPad shipment share fell to 43.6 percent from 51.7 percent annually and from 46.4 percent sequentially. I chose to ask the analysts rather than follow the feeding frenzy.

“Android actually passed the 50 percent mark in 3Q 12”, Tom Mainelli, IDC research director for tablets, says. Whoa, there’s no new king at all. Android took the crown last summer. Still, that’s a phenomenal achievement, setting me to write a story I couldn’t imagine a year ago.

Cloudy Crystal Balls

C`mon, analysts had forecast iPad share well above 50 percent through 2015. Example, Gartner, in April 2011, predicted iPad would fall from 83.9 percent in 2010 to 68.7 percent in 2011 to 63.7 percent in 2012 to 47.1 percent in 2015. By comparison, the analyst firm predicted that Android would go from 19.9 percent in 2011 to 38.6 percent in 2015.

The tablet market is so volatile, no one gets it right. IDC revised tablet operating system forecasts three times throughout 2012. In the most recent (now subsumed by actual data), iOS: 53.8 percent, down from 59.7 percent in September and 62.5 percent in June. Meanwhile, Android forecast increased to 42.7 percent share from 38.8 percent in September but down from 35.3 percent in June.

“I think the big picture is that for many years people insisted that there wasn’t a tablet market, just an iPad market”, Mainelli says. “The last few quarter have shown that there is indeed a market for tablets beyond the iPad, although that product continues to be quite dominant from a vendor market share perspective”.

Beyond Apple’s brand, Mainelli puts perspective on the current market:

The iPad has long benefited from the fact that there are many, many apps created specifically for that device, whereas Android tablet owners often had to make due with smartphone apps scaled up to fit their tablet screen. I think part of the reason Android has gained traction recently is the simple fact that many of the Android devices shipping these days are 7-inch models, and smartphone apps don’t look nearly as bad on a 7-inch screen as they do on a 10-inch screen.

Actually, NPD DisplaySearch sees a dramatic shift in size preference underway, forecasting that 9.7-inch models (majority iPad), will command just 17 percent share this year, while 7-7.9-inchers rise to 45 percent. But, again, given how rapidly this market changes, consider no forecast reliable.

Ultimately, Mainelli sees apps as turning point. “As the installed based of Android tablets grow, I think we’ll see more tablet-specific apps appearing, and that will in turn lead to more Android tablet shipments”.

Platform Purge

In October 2009, I asserted that “Apple cannot win the smartphone wars“, then declared Android winner four months ago. In February 2012, I asserted that “Apple is winning the mobile platform wars“, largely because of iPad gains combined with iPhone.

But much changed since. Weeks later, Android Market became Google Play, starting a major transformation of the store. The rebranding marked Google’s first of many steps taking more authority over Android. By summer, cumulative Android shipments surpassed iOS for the first time — 500 million to 400 million. Apple’s platform only caught up in December. Then came Android 4.1, Google Now and Nexus 7 tablet in mid summer. Apps got a lot better, and suddenly by release of the Nexus 10’s release, Android really looked like a viable all-around mobile platform for anyone.

I was right a year ago, but wrong today. Now Android is winning the mobile platform war — 1.3 million device activations per day, compared to 834,000 for iOS, based on fourth quarter shipments of 75 million. Android smartphone shipments, so not counting tablets, were higher — 80.6 million, according to Strategy Analytics. Based on analyst data, Android share exceeds iOS on both major mobile platforms. As the numbers rise, the apps get better and more piqued for tablets.

“Of course, Apple isn’t sitting still”, Mainelli says. “The iPad mini was a response to consumer demand for smaller tablets, and as you know they couldn’t make enough of them over the holidays (they’re still supply constrained). And, interestingly enough, iPad has become the go-to tablet for commercial entities who consider Android insecure, and schools buy almost exclusively iPads at present”.

Locally last year, here in San Diego, the school district bought 25,000 iPads, which replaced the fleet of netbooks.

But Android and iOS aren’t the only contenders for the crown. “you have Microsoft and its partners trying to make inroads with WinRT and Win8 tablets”, Mainelli says. “So 2013 should bring plenty of good products and great competition, which is always good for consumers”.

Photo Credit: Joe Wilcox

The potential benefits of “big data” have been well described, both by us and others: the ability to spot flu trends earlier and potentially save lives, for example, or to make it easier for companies to provide services in a more personalized way. But these same tools could also be used for more disturbing purposes that smack of Orwell’s Big Brother, and two prominent digital skeptics — Nicholas Carr and Evgeny Morozov — recently raised warning flags about that prospect. Which kind of future will we get?

Carr looked at a recent speech from PayPal co-founder Max Levchin at the DLD conference in Germany (one Om also attended, where he conducted an in-depth interview with Levchin) and clearly didn’t like what he saw. Levchin’s view of people, according to Carr, is that they are just resources that are not being utilized efficiently, and the technology of sensors and real-time information can be used to improve that, in much the same way that programmers try to optimize the clock cycles of a microprocessor. To take one example, Levchin said:

“How about dynamic pricing for brain cycles? We have been maximizing utilization of very high-value, very low-frequency specialists — today you can already rent the brain of a data-mining genius via Kaggle by the hour, tomorrow by brain-hour. Just like the SETI@Home screensaver “steals” CPU cycles to sift through cosmic radio noise for alien voices, your brain plug firmware will earn you a little extra cash while you sleep, by being remotely programmed to solve hard problems.”

More efficient for users, or just creepy?

If you are a geek, this might sound like something with a lot of potential, but Carr describes it as “Clay Shirky’s ‘cognitive surplus’ idea taken to its logical, fascistic extreme.” Levchin goes on to paint a picture of a future in which his insurance company learns — via sensors in his car — that he is taking his children to work, and boosts his insurance premium by a few dollars for the extra risk (Note: we’ll be talking more about the potential of big data at our Structure:Data conference in New York).

Levchin no doubt sees this as efficient, but Carr sees the looming shadow of Big Brother: What if those same sensors detected that you were overweight, or had eaten too much pizza, he asks — would they report that to your insurance company? Maybe the company would boost your rates a little, or maybe you would be “scheduled for a brief re-education session down at the local office of the Bureau for Internal Resource Optimization.” As he puts it:

“This is the nightmare world of Big Data, where the moment-by-moment behavior of human beings — analog resources — is tracked by sensors and engineered by central authorities to create optimal statistical outcomes. We might dismiss it as a warped science fiction fantasy if it weren’t also the utopian dream of the Max Levchins of the world. They have lots of money and they smell even more.”

In a recent piece for Slate, Carr’s fellow digital skeptic Evgeny Morozov looked at the potential implications of banks and other credit-issuing agencies using Big Data to determine who deserves a loan. Although he says the idea of big data is “mostly big hype,” Morozov talks about several companies that are trying to use data from all kinds of sources — including social networks such as Facebook and Twitter — to figure out who is credit-worthy.

Hong Kong-based Lenddo and U.S.-based LendUp look at an applicant’s connections on Facebook and Twitter, Morozov says, and “the key to getting a successful loan is having a handful of highly trusted individuals in your social networks.” A British payday-loan company called Wonga even considers the time of day and how a user clicks around a website in order to determine whether they deserve a loan (although Morozov doesn’t mention it, PayPal uses similar methods to gauge credit-worthiness).

The key is who controls the use of the information

Morozov also mentions ZestFinance, founded by former Google chief information officer Doug Merrill (who we had at our Structure: Data conference in New York last year), whose company looks at more than 70,000 signals and 10 different models to assess credit risk. And he draws a direct link between this and Big Brother, saying: “If only East Germany’s Stasi — the true pioneers of “big data” — had the same model for assessing potential dissidents!”

Despite that comment, however, in the end he (somewhat surprisingly) seems concerned mostly that these companies will use all this information to market things to people who don’t need them, rather than turning them in to the government or their insurance company:

“What happens once these firms, having figured out that all data are credit data, realize that all data are also marketing data? Given how much they know about their clients, it would be very hard for such lending companies not to use this information to sell their existing customers on yet another loan or, perhaps, encourage them to use the loan to take advantage of some unique online sales offer.”

The common thread in both of these dystopian visions is a world in which our data is transmitted without our knowledge, and/or used against us in some way. Where Levchin seems to see an efficient exchange of data between user and service, one with benefits for both — and presumably a level (and secure) playing field in terms of who has access to it — Carr and Morozov see companies and governments misusing this data for their own nefarious purposes, while we remain powerless.

What makes it difficult to argue with either one is that we’ve already seen the building blocks of this potential future emerge, whether it’s Facebook playing fast and loose with the privacy settings of a billion people, or companies aggregating information and creating profiles of us and our activities and desires. What happens when the sensor-filled future that Levchin imagines becomes a reality? Who will be in control of all that information?

Images courtesy of Shutterstock / Lightspring and Flickr / Thomas Leuthard

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Dissecting the data: 5 issues for our digital future
• Helix Nebula and the future of Europe’s cloud
• The importance of putting the U and I in visualization

Watch the West Wing Week here.

Here's a quick glimpse at what happened this week on WhiteHouse.gov:

Immigration Reform: On Tuesday, President Obama traveled to Las Vegas to present his plans for comprehensive immigration reform. The four parts of the White House proposal call for a strengthening of our borders, cracking down on companies that hire undocumented workers, creating a path to earned citizenship and streamlining our legal immigration system for all.

“Now is the time to do this so we can strengthen our economy and strengthen our country's future,” said President Obama, urging a bipartisan effort.

read more

They may still be reluctant, but three of the four major carriers are now willing to entertain the idea of sharing the same airwaves with government users. This week Verizon Wireless, AT&T and T-Mobile signed a memorandum of understanding agreeing to explore spectrum sharing possibilities on 95 MHz of frequencies currently used by the U.S. Department of Defense and other federal agencies.

Charged with identifying more airwaves for commercial use, the National Telecommunications and Information Administration (NTIA) recommended last year that the private and public sector split time on government airwaves. The idea is that the DOD and other users were only using their spectrum at certain times and in certain places, so why not let carriers access those frequencies whenever and wherever they weren’t occupied by the feds?

Carriers traditionally like licenses they can call their own, and the industry initially responded to the deal with skepticism. In a blog post, AT&T still said it would rather see airwaves cleared entirely of government users and auctioned for government use but was amenable to the idea of sharing if clearing the airwaves was not possible.

“I want to emphasize that we continue to believe that clearing and reallocating is the best approach to freeing up much needed spectrum for commercial mobile broadband use,” AT&T Assistant VP Stacey Black wrote in the policy blog. “The existing exclusive licensing regime has resulted in billions of dollars in wireless infrastructure investment, enabling the U.S. to lead the way in the global mobile broadband marketplace. While clearing spectrum for exclusive commercial licensing must remain the top priority, when that is neither time nor cost effective, AT&T supports exploring sharing arrangements.”

Black hints at another possible obstacle to a shared spectrum plan: Congress. Lawmakers have counted on the billions raised in spectrum auctions to fill government coffers. Depending on how spectrum sharing was implemented it would either raise no money or far less than previous auctions.

Over the next few months, the three carriers and the DOD conduct trials and simulations of sharing scenarios at government installations. The results of those tests will be released in March.

Sprint was the only carrier in the Big 4 not on the list, but a Sprint spokesman told FierceWireless that the company works closely with the NTIA and will be following the tests closely.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• 2012: Data, spectrum and the race to LTE
• The future of mobile: a segment analysis by GigaOM Pro
• A look back at mobile in the third quarter

On Sunday in New Orleans, the San Francisco 49ers will face off with the Baltimore Ravens in Super Bowl XLVII. For the first time in Super Bowl history, the head coaches of the two opposing football teams are brothers. John Harbaugh, the older of the brothers, has been the head coach for the Ravens since 2008. Meanwhile, Jim Harbaugh — younger brother and a former NFL quarterback himself — is the head coach of the 49ers. And to make the story even more complicated, Jay Harbaugh — the 23-year-old son of Jim — is a coaching intern for Baltimore, working alongside his uncle.

So how might this sibling relationship shape Sunday’s matchup? Here’s a TED Talk that gives some insight.

In this talk from TEDxAsheville, Jeffrey Kluger explores the power of the sibling bond. Being the favorite or the screw-up, the pretty one or the smart one — these are things that frame a person’s life. He talks about why siblings often go into the same field — because one sibling saw the other getting attention for a specific achievement, and direct competition was encouraged — leading to jockeying that can last a lifetime. Making this stew even worse: that 70% of fathers and 65% of mothers outwardly express a preference for one child over another.

So which brother will win on Sunday? That’s a question up there with, “What will Beyoncé sing during halftime?” and “Which Super Bowl ad will win TED’s third annual Ads Worth Spreading search?” A past winner was this classic: “Born of Fire.”

Yesterday, I had the pleasure of speaking at the Technology Inclusion Summit, hosted by Chief Technology Officer Todd Park, the Office of Science and Technology Policy, and the Level Playing Field Institute. It was an amazing gathering of private and public partners who are united in their efforts to expand opportunities for training, education and jobs in technology.

President Obama has always believed that technology is an essential part of growing our economy, creating jobs and remaining globally competitive. The President continues to be committed to encouraging the entrepreneurial spirit in our country, starting with setting a goal of 1 million Science Technology Engineering Math (STEM) graduates over the next decade. This effort also means that we need to collectively act to knock down any barriers that stand in the way.

Last August, during the Tech Inclusion Roundtable, participants came up with some great private-sector initiatives to help drive innovation across every community. Whether it was pioneering new educational tools for students of all ages, bringing technical training to underrepresented communities, or mobilizing tech company CEOs to establish mentoring programs for young people, every one of these initiatives and ideas has the potential to shape America’s future.

read more

Everyone snowed-in and sick of winter will be happy to learn that the yearly Groundhog Day prediction in Punxsutawney, Pennsylvania will be live-streamed online this year. The Pennsylvania tourism website will stream Punxsutawney Phil’s annual “prognostication” from this website. The prediction is scheduled to take place around 7:25 am EST.

According to the folk tale, six more weeks of winter are predicted by Phil if he sees his shadow. If not, an early spring is “just around the corner.”

The tradition of Punxsutawney’s Groundhog Day dates back to 1887. Since February 2 falls on a weekend this year, record crowds are expected to visit Gobbler’s Knob.

In addition to live-streaming the event, the Pennsylvania Tourism Office has again teamed up with Foursquare to offer a special Groundhog Day badge. The first 500 Foursquare users to check in from Gobbler’s Knob on February 2 will receive a Groundhog Day-branded Slinky toy. The Slinky was declared the official toy of Pennsylvania in 2001.

Last year, Phil predicted six more weeks of winter, as is usually the case. Still, there is always hope that the groundhog will predict an early spring, and that the prediction will happen to be correct.

For those who prefer to sleep in on their Saturdays, the movie Groundhog Day is currently streaming via Netflix Watch Instantly. It is, in every way, more entertaining than one of the real predictions. As proof, last year’s festivities (which consist largely of men in top hats making a spectacle of a terrified rodent) can be seen below.

You won’t see this advertisement for Adobe’s Marketing Cloud during the Super Bowl on Sunday. It features a monkey talking to a horse about how ridiculous it is to spend millions over dollars on a TV ad.

Once again, you won’t be seeing this ad on Super Bowl Sunday. But you can watch it now:

3D printers are proving to be revolutionary in medical fields, and now the technology is being used to revolutionize dentistry.

Objet has a new video up on how its 3D printers are helping to cut down on the time required to make models of teeth for those seeking restoration. Apex Dental Milling says that it was able to make maybe three or four models in an hour traditionally, but now it makes 20 or 30 with 3D printers.

Here’s how they do it:

Apex Dental Milling use the Objet Eden260V 3D Printer to turn digital impressions into physical models. The use of the Objet 3D printer allows Apex to reduce their cost per case significantly and to stay at the forefront of Digital Dentistry. The final models are sent to the dental technician who performs a final fit and quality check before a restoration is sent back to the doctor. Objet’s reputation for fine resolution 3D printing allow the dental technician to provide the most accurate restorations to his clients. Objet machines are reliable and robust and are typically left to run overnight and over weekends without attendence – further improving the cost/benefit to Apex.

Tom Chi, one of the guys behind Google Glass spoke about prototyping the device at TEDYouth. He discusses how the company was able to speed up the creation process via “rapid prototyping”. He says it only took them one day to come up with the prototype (this was done with clay, paper and modeling wire). They were able to prototype projections for the device in 45 minutes.

In actual Google Glass-related news, a headset with bone-conduction speakers was revealed in a recent FCC filing.

More on Google Glass

Harvey Weinstein, an Oscar winning producer and prolific proponent of Obama, told Deadline that he is going to push for legislation that would force websites to pay for linking to news articles. This legislation would require any news website or blog to pay a monitoring organization a fee for every link to an article written by a journalist.

Should news sites, bloggers and other sites like Facebook, Twitter and Google pay for the privilege of including snippets and links to news stories? Also, should YouTube or sites that include embedded videos of movie/TV clips pay every time somebody views them?

Give us your thoughts on this important topic that goes to the heart of the internet in the comments below.

Weinstein said, “Journalists don’t benefit when their stories are taken, and given a link. It would be like me launching a newspaper–call it Link—where I can have the greatest journalists in the world working for me without paying them. It’s inconceivable. If BMI and ASCAP can monitor the music business, we need a BMI and an ASCAP to monitor these businesses. This will be the one legislation for our industry that I’ll press.”

This would be part of a broader law that where a monitoring organization would also monitor the web for video clips and require websites like YouTube to pay this organization a fee for each view of a clip of a movie or television show.

As the publisher of WebProNews and a long–time advocate of the right to link, in my opinion Weinstein’s idea would destroy the internet as we know it today. The internet is based on the idea of linking, that’s why it was originally referred to as the World Wide Web! If you make publications, blogs, Google, Twitter and Facebook pay for linking to a news story, how many of them would still do it. The answer is none.

Weinstein may think he’s only talking about making news linking giants like Google News pay, but laws against free linking could not just apply to them. His proposed legislation would also have to apply to Reddit, Stumbleupon, Facebook, Twitter and news publishers and bloggers who routinely republish snippets of news articles with links to the original. Many of these sites also inbed video clips as well.

Weinstein challenges the assertion by publishers that linking and taking small snippets of articles is not stealing content but is actually promoting the content. Weinstein equates linking and publishing as one and the same. Weinstein also told Deadline, “When it comes to journalists and journalism, I’m with you. It is important they get paid for good work, and wrong that others just take it, with a link.”.

Since most articles have numerous social buttons encouraging “sharing” their articles via social media sites like Facebook and Twitter, you would think it would be obvious to Weinstein that publishers and journalists want their stories to be linked to. The definition of going viral is mass sharing on social media sites which pushes huge numbers of people to a journalist article if he is so lucky. Linking drives traffic to an article which theoretically can then be monitized by the publisher. If the publisher doesn’t want the traffic he can put up a firewall login and charge visitors to read the sites content.

If a news site like Deadline doesn’t want its articles linked to then it shouldn’t publish them on a linking platform called the Web. Weinstein may be surprised to learn that Deadline and most news sites are quite happy that their articles get free traffic driven by links!

Just like the music industry, which has in the past sued the parents of kids who downloaded music without paying for it, Weinstein proposes that those linking to content should also have to pay up. He wants to do it a bit more tactifully than the RIAA, but still wants to collect nonetheless. His idea I presume is to first change the definition of fair use which is permitted per U.S. and many international copyright laws, where a website can take snippets of content and reuse it to a certain extent.

Theoretically, considering Weinstein’s personal connection with Obama, he could persuade the President to tighten this definition via some minor changes in regulations and rules and bypass Congress. The definition of fair use as written in U.S. copyright laws is vague and could easily be redefined via regulation. This is a scary proposition considering that linking and discussing news articles is integral to free speech.

Once fair use is redefined to allow copyright holders the ability to charge a retroactive fee to websites for each view of snippets and links to news articles, that’s when a new organization similar to BMI would emerge to ensure that journalists are paid for their work. BMI has people going into businesses, such as bars and restaurants, all around the country looking to see if music is being played without their license. When it catches a business playing unauthorized music it forces them to pay based on a variety of factors such as number of seats in a restaurant and number of songs played.

If a bar doesn’t join BMI and agree to pay a monthly fee up front, then often BMI will sue for huge amounts. For instance, one restaurant in North Carolina was order by a court to pay the BMI $30,450 for playing just four unauthorized songs.

This is what Weinstein wants for publishers and writers of news content! If you are a blogger that makes a small amount of money from ads and you include a snippet from a news article in your story you could be sued if you didn’t already agree to a monthly payment.

For Facebook, Google and Twitter the ramifications of this kind of heavy handed legislation could be huge. They are the YouTube of written content since so many of us share snippets and links via them. If sites like these need to license links with a BMI type organization, it’s likely that they would just eliminate news links and snippets altogether which would change the web forever… don’t you think?