Author: Serkadis

Most LinkedIn users know “People You May Know” as one of that site’s flagship features — an onmipresent reminder of other LinkedIn users with whom you probably want to connect. Keeping it up to date and accurate requires some heady data science and impressive engineering to keep data constantly flowing between the various LinkedIn applications. When Jay Kreps started there five years ago, this wasn’t exactly the case.

“I was here essentially before we had any infrastructure,” Kreps, now principal staff engineer, told me during a recent visit to LinkedIn’s Mountain View, Calif., campus. He actually came LinkedIn to do data science, thinking the company would have some of the best data around, but it turned out the company had an infrastructure problem that needed his attention instead.

How big? The version of People You May Know in place then was running on a single Oracle database instance — a few scripts and heuristics provided intelligence — and it took six weeks to update (longer if the update job crashed and had to restart). And that’s only if it worked. At one point, Kreps said, the system wasn’t working for six months.

When the scale of data began to overload the server, the answer wasn’t to add more nodes but to cut out some of the matching heuristics that required too much compute power.

So, instead of writing algorithms to make People You Know Know more accurate, he worked on getting LinkedIn’s Hadoop infrastructure in place and built a distributed database called Voldemort.

Since then, he’s built Azkaban, an open source scheduler for batch processes such as Hadoop jobs, and Kafka, another open source tool that Kreps called “the big data equivalent of a message broker.” At a high level, Kafka is responsible for managing the company’s real-time data and getting those hundreds of feeds to the apps that subscribe to them with minimal latency.

Espresso, anyone?

But Kreps’s work is just a fraction of the new data infrastructure that LinkedIn has built since he came on board. It’s all part of a mission to create a data environment at LinkedIn that’s as innovative as that of any other web company around, and that means the company’s applications developers and data scientists can keep building whatever products they dream up.

Bhaskar Ghosh, LinkedIn’s senior director of data infrastructure engineering — who’ll be part of our guru panel at Structure: Data on March 20-21 — can’t help but find his way to the whiteboard when he gets to discussing what his team has built. It’s a three-phase data architecture comprised of online, offline and nearline systems, each designed for specific workloads. The online systems handle users’ real-time interactions; offline systems, primarily Hadoop and a Teradata warehouse, handle batch processing and analytic workloads; and nearline systems handle features such as People You May Know, search and the LinkedIn social graph, which update constantly but require slightly less than online latency.

One of the most-important things the company has built is a new database system called Espresso. Unlike Voldemort, which is an eventually consistent key-value store modeled after Amazon’s Dynamo database and used to serve certain data at high speeds, Espresso is a transactionally consistent document store that’s going to replace legacy Oracle databases across the company’s web operations. It was originally designed to provide a usability boost for LinkedIn’s InMail messaging service, and the company plans to open source Espresso later this year.

According to Director of Engineering Bob Schulman, Espresso came to be “because we had a problem that had to do with scaling and agility” in the mailbox feature. It needs to store lots of data and keep consistent with users’ activity. It also needs a functional search engine so users — even those with lots of messages — can find what they need in a hurry.

With the previous data layer in tact, he explained, the solution for developers to solve scalability and reliability issues was doing so in the application.

However, Principal Software Architect Shirshanka Das noted, “trying to scale [your] way out of a problem” with code isn’t necessarily a long-term strategy. “Those things tend to burn out teams and people very quickly,” he said, “and you’re never sure when you’re going to meet your next cliff.”

Schulman and Das have also worked together on technologies such as Helix — an open-source cluster management framework for distributed systems — and Databus. The latter, which has been around since 2007 and the company just open sourced, is a tool that pushes changes in what Das calls “source of truth” data environments like Espresso to downstream environments such as Hadoop so that everyone can ensure they’re working with the freshest data.

In an agile environment, Schulman said, it’s important to be able to change something without breaking something else. The alternative is to bring stuff down to make changes, he added, and “it’s never a good time to stop the world.”

Next up, Hadoop

Thus far, LinkedIn’s biggest push has been in improving its nearline and online systems (“Basically, we’ve hit the ball out of the park here,” Ghosh said), so its next big push is offline — Hadoop, in particular. The company already uses Hadoop for the usual gamut of workloads — ETL, model-building, exploratory analytics and pre-computing data for nearline applications — and Ghosh wants to take it even further.

He laid out a multipart vision, most of which centers around tight integration between the company’s Hadoop clusters and relational database systems. Among the goals: better ETL frameworks, ad-hoc queries, alternative storage formats and an integrated metadata framework — which Ghosh calls the holy grail — that will make it easier for various analytic systems to use each other’s data. He said LinkedIn has something half-built that should be finished this year.

“[SQL on Hadoop] is going to take two years to work,” he explained. “What do we do in the meanwhile? We cannot throw this out.”

Actually, the whole of LinkedIn’s data engineering efforts right now put a focus on building services that can work together easily, Das said. The Espresso API, for example, allows developers to connect a columnar storage engine and do some limited online analytics right from within the transactional database.

Good infrastructure makes for happy data scientists

Yael Garten, a senior data scientist at LinkedIn, said better infrastructure makes her job a lot easier. Like Kreps, she was drawn to LinkedIn (from her previous career doing bioinformatics research at Stanford) because the company has so much interesting data to work with, only she was fortunate enough to miss the early days of spotty infrastructure that couldn’t handle 10 million users, much less today’s more than 200 million users. To date, she said, she hasn’t come across a problem she couldn’t solve because the infrastructure couldn’t handle the scale.

The data science team embeds itself with the product team and they work together to either prove out product managers’ hunches or build products around data scientists’ findings. In 2013, Garten said, developers should expect infrastructure that lets them prototype applications and test ideas in near real time. And even business managers need to see analytics as close to real time as possible so they can monitor how new applications are performing.

And infrastructure isn’t just about making things faster, she noted: “Something things wouldn’t be possible.” She wouldn’t go into detail about what this magic piece of infrastructure is, but I’ll assume it’s the company’s top-secret distributed graph system. Ghosh was happy to go into detail about a lot things, but not that one.

A virtuous hamster wheel

Neither Ghosh nor Kreps sees LinkedIn — or any leading web company, for that matter — quitting the innovation game any time soon. Partially, this is a business decision. Ghosh, for example, cites the positive impact on company culture and talent recruitment, while Kreps points out the difficult total-cost-of-ownership math when comparing paying for software licenses or hiring open source committers versus just building something internally.

Kreps acknowledged that the constant cycle of building new systems is “kind of a hamster wheel,” but there’s always an opportunity to do new stuff and build products with their own unique needs. Initially, for example, he envisioned two targets use cases for Hadoop but now the company has about 300 individual workloads; it went from two real-time data feeds to 650.

“But companies are doing this for a reason,” he said. “There is some problem this solves.”

Ghosh, well, he shot down the idea of relying too heavily on commercial technologies or existing open source projects almost as soon as he suggests it’s a possibility. “We think very carefully about where we should do rocket science,” he told me, before quickly adding, “[but] you don’t want to become a systems integration shop.”

In fact, he said, there will be a lot more development and a lot more open source activity from LinkedIn this year: “[I’m already] thinking about the next two or three big hammers.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Infrastructure Q1: Cloud and big data woo enterprises
• Defining Hadoop: the Players, Technologies and Challenges of 2011

We understand, you’re busy. So you may not have had the chance to check out our podcasts this week. That’s why we’ve gathered them together in one handy place for you to spend your Sunday listening and learning about all the technology trends you need to know about.

In our weekly news wrap up we talked about Yahoo’s work from home ban, as well as our Internet of things meetup in San Francisco and the no news out of Mobile World Congress.

(Download the weekly wrap up)

Stacey Higginbotham went nutty for the Almond+ touchscreen router in our Internet of things podcast.

(Download the Internet of things podcast)

And Kevin Tofel is super excited about the Chromebook Pixel, and answers your questions about the device.

Download Kevin’s call-in show)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The state of cross-platform media measurement
• Social third-quarter 2012: analysis and outlook
• How social discovery is transforming entertainment

The stable build for Internet Explorer 10 for Windows 7 has been released this week after spending a few months as a preview for developers.

The new browser from Microsoft is offered through Automatic Updates system to users with local administrator accounts, being automatically downloaded and installed without any notification.

Non-administrat… (read more)

Google is a hardware company now, something that is too often overlooked. Last week, the search and information company launched its first computer, Chromebook Pixel, which I reviewed overnight yesterday. In August 2011, Google bought Motorola, one of the most iconic American brands and inventor of the cell phone, for $12.5 billion; the deal closed in May 2012. Moto makes handsets and tablets, among other things.

But Motorola is a brand in decline, and one Google should want to reinvigorate if the plan is to release more-compelling, more-innovative Androids. Why should ASUS, LG and Samsung manufacture Nexus devices when Google has a huge hardware subsidiary? Meanwhile, one of the three is a partner out of control. Samsung threatens to fracture the broader Android ecosystem — a problem I sternly warned about in April 2012 post “Google has lost control of Android“. A resurgent Motorola could bring unity back to the Force, as the saying goes. Google bets that Guy Kawasaki can help make that happen.

What a Guy

Kawasaki joined Moto this week, as a consultant/evangelist. In BetaNews group chat yesterday I described him as: “Smart. Articulate. Friendly. And your worst nightmare if marketing against you”. I know him, as many long-time journalists do, as a kind of devil incarnate. During Apple’s worst days, he joined as an evangelist. I summed up his role to colleagues yesterday: “Kawasaki is the reason Mac enthusiasts are crazy online. He organized the first attack squads, nurtured that whole culture of guerrilla evangelism”.

When Apple was a runt company, Kawasaki’s Mac attack squads clobbered people writing anything even slightly critical of Apple. Any long-time journalist knows the drill. You write X story about Apple and the innuendo-carting cultists swarm in accusing you of Windows bias and shilling for Microsoft. Or in this decade, Google. The accusations whack the writer’s credibility often with no substance (e.g., facts) to support them. The attack evangelism culture Kawasaki nurtured at Apple lives on long past his influence.

In this decade, Kawasaki is an Android convert — and quite vocal about it. He explains:

Every time I tell people that I don’t use any iOS products, people go a little nuts when they figure out I’m a pure Android guy. This is a list of my top ten reasons why I like Android more than iOS.

1. Selecting default applications to open files.
2. Making Chrome my default browser.
3. Viewing apps, no matter what folder they’re in, in an alphabetical list.
4. Installing different keyboards—for example, Swype.
5. Viewing windows containing live feed of appointments or emails (‘widgets’).
6. Using any micro-USB cable to charge an Android phone or tablet.
7. Receiving utomatic, unattended updates to the operating system and applications.
8. Making multiple aliases for apps, not simply moving apps around.
9. Using Google Now.
10. Viewing summaries of notification messages—for example, ’25 new messages’ instead of 25 individual notices.

Kawasaki is a good fit for Motorola. Colleague Alan Buckingham expressed in BN group chat yesterday: “I loved my Droid X, but I am certainly not a Motorola fanboy”. To which I responded: “Spend an hour with Guy and you would be”. Mihaita Bamburic chimed in: “Kawasaki is a genius. He’s incredibly skillful when it comes to speeches, I’ve seen him a couple of times on YouTube and he got me hooked”. I added: “He’s more effective, more believable than Steve Jobs, but just as charismatic”.

Android’s Motorola Debt

Google doesn’t just own Motorola, it owes something to the cellular technology pioneer — and also to Verizon. The carrier launched the Motorola Droid brand, backed by $100 million marketing budget, in autumn 2009. The move invigorated Android global growth, by lifting it in, at the time, the largest region adopting smartphones. Increased visibility, even more than sales, helped Android sales jump a stunning 888.8 percent in 2010, according to Gartner. Moto’s fortunes have since declined, but Verizon remains firmly committed to the Droid brand, even after getting iPhone in February 2011.

Globally, Motorola’s sales share has collapsed, just 1.7 percent in fourth quarter — that’s tenth place — according to Gartner. Remember Razr, which seemingly everyone who was anyone owned in the mid-Noughties? The original shipped in third quarter 2004, with shipments reaching 50 million less than two years later. Moto is a ghost of former glory, yet with solid engineering culture, enormous patent portfolio and Google’s backing.

Moto also has a core cult-like following of users devotedly committed to the brand. They remind me of Apple circa 1996, when closure looked likely but a small group remained loyal and vocal about the products. Kawasaki is among them: “I made the switch from iPhone to RAZR more than a year ago”. Verizon brought back to brand, combined with Droid, to smartphones, in October 2011 (sales started the next month).

A Galaxy Far, Far Away

Kawasaki’s enthusiasm for a Motorola smartphone is more significant than his Android embrace. The evangelist’s real mission, whether or not acknowledged, is to save Google from Samsung. The South Korean electronics giant is largely responsible for Android’s enormous market share gains, but in process inflicts great pains on the broader ecosystem.

Based on sales to end users, not shipments to carriers and dealers, Android captured 69.7 percent global smartphone share in fourth quarter, according to Gartner. Samsung, the world’s leader for all handsets and smartphones, accounted for a stunning 42.5 percent of all Androids sold. The next closest vendor: 6 percent. Samsung’s success is mixed for Android as a platform. The electronics company, and not Google, largely controls customers’ experience via TouchWiz UI and other features. Meanwhile, Android updates lag Samsung phones by many months. At least Samsung adopts the newest Android eventually, but TouchWiz skinned.

Eleven months ago, Forrester Research analyst Frank Gillette predicted that companies adapting Google’s open-source OS would “cause proprietary Android share to surpass the installed base of Google’s Android ecosystem in 2015. This further fragmentation will challenge Android developers, customers, and especially enterprises, and hamper the creation of a shared ecosystem”. Samsung, which is No. 2 to Apple on tablets, is front and center leading to a fractured — not fragmented — Android ecosystem.

Samsung’s priorities aren’t necessarily Google’s or to the benefit of a the broader Android ecosystem. The Galaxy brand is hugely successful, and a second gravity well pulling brand attention, and even app developers and content partners, to customized Android. Then there is the power that comes with success — massive dominance over the entire market of Android sales. Samsung is a partner Google obviously wants to keep, but also should keep from getting too powerful.

Nexus RAZR

Enter Guy Kawasaki and his task evangelizing and revitalizing the Motorola brand, which needs more commitment from Google. As slow as Samsung is sometimes to update Android, Moto often takes longer or leaves customers hanging with nothing. Google must also get behind the subsidiary and get it picking up the pace — in every way. The search and information giant is agile and quick to update products. Motorola is slow moving, like an abandoned ship. There needs to be good captain and crew to support Kawasaki’s efforts.

If he can succeed in them, and Google supports him, there’s no reason why a new class of Nexus devices — even Chromebooks — shouldn’t come from Motorola. Evangelism is a great starting point. This week, Kawasaki set up the Mobile Devices community on Google+. Discussion is excellent beginning.

Samsung isn’t a bad Android partner, but the company needs some serious competition. Google and Motorola can give it. Nexus devices show that the Android Army wants pure Android, without the fancy clothes draped over the green robot. Imagine something like Nexus RAZR. Competition can keep Samsung on course with Android, rather than diverting destinations. Otherwise, soon the company will have none — that’s the problem, the future.

I’d like to congratulate Guy Kawasaki on his new role and wish him well. That sentiment isn’t easily given from someone battling Mac guerrilla evangelists since the 1990s.

Photo Source: Guy Kawasaki’s Google+ Profile

This week saw the Mobile World Congress event wrap up in Barcelona, with a few new Android devices to look forward to. One confirmed an earlier rumor that HP was getting back in on the tablet market as the company introduced its Slate7 running on Android. The most appealing aspect of the product may be the $169 price tag because there’s not much to make this “me-too” tablet stand out from the crowd.

The Slate7 is another 7-inch tablet, competing against Google’s Nexus 7, the new Asus FonePad, Amazon’s Kindle Fire, Samsung’s Galaxy Tab 2 and others in this space. I can see why HP opted for a small slate as their comeback product: Some recent data indicates that smaller tablets will outsell larger ones in 2013.

HP used a fairly standard set of hardware in the Slate7. A 1.6 GHz dual-core chip powers the Android 4.1 device, which includes 1 GB of memory. The 7-inch touchscreen uses a 1024 x 600 resolution panel; the same res as my original Galaxy Tab back in 2010. Storage capacity is 8 GB of flash memory that can be expanded with a microSD card. A pair of cameras complete the product with the rear one offering a meager 3 megapixels. In short, this a low-priced product with old specs competing against similarly priced products with better specs. As I said when HP was rumored to re-enter the tablet market: good luck with that.

Also out of MWC are tablets that include cellular voice capabilities: The aforementioned Asus FonePad and new Samsung Galaxy Note 8.0 are two examples. I’ve said for some time that I think small tablets will replace smartphones, but I think we’re very early in that trend for two reasons.

Consumers can’t conceive of carrying a 7- or 8-inch tablet everywhere because the device is not as pocketable as a traditional smartphone. I certainly understand that situation. Yet, I carry a small tablet everywhere; in a pocket when I can and in the hand when I can’t. As I said on this week’s podcast, I think this is a situation that has to be experienced; not simply written off because it sounds like a bad idea.

The other issue, at least in the US, is how carriers control what devices actually connect to the cellular networks. My Samsung Galaxy Tab actually had voice capability in 2010, but US carriers stripped the functionality out of the device. In contract, international versions of the Tab worked just fine for voice calls. I’m not yet convinced that US carriers will support voice features in these new Android slates, but I hope I’m wrong.

Finally, I’ll be spending some time using Android on a completely different device this coming week: Google’s Chromebook Pixel. I’m finding that besides a superb web experience thanks to the high resolution display paired with an Intel Core i5 processor, the Pixel is a versatile laptop as well.

I’m already running a simultaneous instance of Linux alongside Chrome OS and thanks to the Android-x86 port, I can run Android on the Pixel as well. The touchscreen isn’t yet supported, so I’ll have to use the Pixel’s touchpad; not a big deal as it’s one of the best I’ve used on a laptop, rivaling that of my old MacBook Air.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Analyzing the wearable computing market
• Tablets wars: Apple is from Venus, Amazon is from Mars
• The connected planet: Smartphones aren’t the only player

This week I posted our 10,000th article since Data Center Knowledge was launched in 2005. As the industry has grown, the pace of the news has picked up as well. In 2012 alone DCK brought you more than 1,800 stories about the data center industry. There’s much more to come, as we continue to expand our coverage to track the data center’s central role in the Internet economy.

These 10,000 news stories provide a thorough history of the data center industry over the past seven years. It’s all accessible through our search box at the top right corner of every page. If you need to research a company, trend or market, you’ll likely find a reference in our archives.

You can also stay current on our latest headlines by subscribing to our new and improved daily e-mail updates or by RSS. Here are some other resources that might be of interest:

• Lists of The World’s Largest Data Centers and Who Has the Most Servers?
• Our Data Center FAQs offering detailed look at the Internet infrastructures driving Google, Facebook and Apple.
• Data centers built under mountains, and in former shopping malls, military bunkers and particle accelerators. One day perhaps there may be a data center the moon.
• For thought leadership, top industry executives and engineers are sharing their viewpoints in our Industry Perspectives section. Read recent columns or contribute your own perspective.
• The DCK White Paper Library is an increasingly popular resource for education about the latest trends and topics in the data center.
• Two key resources for industry professionals are the Data Center Jobs Board, and our listing of Data Center Conferences.
• Check out our guides to Data Center Energy Efficiency and Data Center Strategies.
• If you’re interested in publicly held companies in this sector, check out our Data Center Investor channel for performance updates on data center stocks. For tracking individual stocks, you can use the “Companies” link in our navigation bar, or just type the company’s ticker symbol into our search box.
• If cool videos of data center tours are what you seek, check out our DCK video archive and the Data Center Videos channel on YouTube.

You can also follow DCK on Twitter (@datacenter), our Facebook page, on Google+ or connect on LinkedIn.

For your weekend reading, here’s a recap of five noteworthy stories that appeared on Data Center Knowledge this past week. Enjoy!

Intel Enters the Hadoop Software Market – The market for Hadoop software continues to attract new players. Intel (INTC) announced the availability of its Distribution for Apache Hadoop, including new management tools. More than 20 partners announced support for Intel’s Hadoop offering, including Cisco, Red Hat, Cray and Supermicro.

Windows Azure Cloud Crashed by Expired SSL Certificate – So how did an expired SSL certificate crash the Windows Azure storage cloud computing platform Friday and Saturday? It’s an expensive question for Microsoft, which will be offering customer credits for the outage.

GM Plans $258 Million Data Center in Michigan – General Motors is hoping to build a $258 million data center at a research facility it owns in Milford, Michigan. The company is seeking tax abatements for the project at the Milford Proving Ground, which would feature a 100,000 square foot data center and employ about 20 workers.

Equinix to Sell $1.5 Billion in Notes to Fund Construction, Acquisitions – Colocation provider Equinix plans to sell up to $1.5 billion in senior notes, and will use some of the money to build new data centers and fund acquisitions, the company said Thursday. The offering shows that the data center industry’s strongest players continue to use their financial strength to enter new markets and boost their competitive position.

Vantage Lines Up 1 Megawatt Lease in Santa Clara – Vantage Data Centers has signed a new long-term customer lease for a 1 megawatt data hall on its campus in Santa Clara, California, the company said this week. The deal continues the busy pace of leasing in Santa Clara, the hub of data center activity in Silicon Valley.

Stay current on Data Center Knowledge’s data center news by subscribing to our RSS feed and daily e-mail updates, or by following us on Twitter or Facebook or join our LinkedIn Group – Data Center Knowledge.

After a relatively long break, the weekend newsletter returns. I was traveling — a lot and as a result was having a tough time trying to find and read stuff on the web. Those international data rates are a killer. Here is an assortment of links that I find worth reading. Hopefully you will like them as well.

Fauja Singh, the 101-year-old marathoner: He started running marathons at the age of 89. And at 101 he is the oldest marathoner. His amazing story told by ESPN’s Outside the Lines. If possible, read this on a big screen.

Why does the U.S. gets weather forecasts wrong? Every time I travel outside of the U.S., I realize that the weather forecasts are very accurate, so much so that I am surprised. After all, I live in the U.S., where weather people are rarely right. It wasn’t up until I read this piece by Dan Satterfield that I understood the accuracy disparity.

How Manhattan’s area code was influenced by the rotary dial. Yes kids, there was this thing called the rotary dial. Read and find out.

India’s rice revolution: No genetic modifications, no herbicides. Just plain old ingenuity and India’s poorest state, Bihar, is turning into a rice mine. I am being serious. Don’t believe me, read this.

The Rape of Petty Officer Blumer. Rolling Stone looks at the military’s culture of sexual abuse, denial and coverup. The New York Times‘ James Risen writes about Virginia Messick who decided to speak up about what happened to her when she was a young Air Force recruit.

The rise and fall of Alex Rodriguez: The man with the one of the richest contracts in baseball is living up to his tabloid name: A-Fraud. What went wrong? Joe Posnanski postulates.

His saving grace: The amazing story of Chef Curtis Duffy and his battle to escape his demons. Also, worth reading on a big screen or on your tablet.

Also, if you are a fan of the Om Says newsletter, check out the GigaOM Reader, a weekend column that looks back at some of the top stories in tech with my take and also curates some of the best tech writing from around the web. I try and showcase indie writers, mostly because their work needs our love.

Gamers around the world are excited for the holidays following Sony’s (SNE) recent PlayStation 4 unveiling, but not everyone will want to shell out an estimated $500-$600 on the company’s new console. As such, budget-minded gamers who aren’t afraid to adopt a current-generation system on the eve of a new launch will be happy to know that PlayStation 3 pricing is likely about to plummet.

Continue reading…

The entrepreneurs who are still willing to attempt large scale manufacturing of next-gen energy technologies — whether it’s solar materials, LEDs, futuristic batteries or advanced biofuels — are increasingly looking to using existing machines from other industries to make their products. Many of the executives, and investors at the ARPA-E Summit this week told me they are building this requirement into their original business models.

While the move might seem obvious, the trend is in contrast to high-profile companies from yesteryear like Solyndra, which built expensive custom machines to produce their solar panels and had to raise and spend hundreds of millions of dollars on manufacturing. The added expense and complexity of developing new machines and new products just added to Solyndra’s struggles and contributed to its bankruptcy.

The CEO of Alphabet Energy, Matthew Scullin, told me at the Summit this week that his goal from day one was to require all of Alphabet’s products to be made on existing toolsets. Alphabet Energy develops thermoelectric materials and devices, which convert heat into electricity, and the technology can be built on standard chip industry machines. “A startup needs to focus on developing one product in order to be successful, and developing a tool is like developing a second product in parallel. The risk is high,” said Scullin.

Scullin also pointed out that by using traditional semiconductor tools Alphabet can more easily find skilled operators and can also outsource manufacturing to chip foundries, if they choose to do so. For custom machines, “the lack of existing know-how, secondhand equipment, service people, and competition means the cost of doing business is high, adding to risk.”

Battery startup Seeo is using standard machines used to make traditional lithium ion batteries to make its batteries, including its secret sauce: its unique electrolyte. The company employs basic mixers, coaters, and assembly and testing machines at its pilot line factory in Hayward, Calif. and is also using battery cell, module and pack materials that are commonly used to make lithium ion batteries. Later this year the Seeo team hopes to build a larger fab with the same equipment somewhere in the U.S.

Startup Imprint Energy, which is making a zinc battery, uses off-the-shelf  printing equipment from prototyping to scaled production, says Imprint Energy CEO Devin MacKenzie. They haven’t done any customization of the equipment and MacKenzie tells me they do not anticipate requiring any large scale special equipment or significant modifications of commercially-available systems.

Many of the companies in its sustainability portfolio are embracing the practice of using standard plug and play manufacturing machines, Khosla Ventures’s partner Andrew Chung said at the Summit this week. Seeo has raised funds from Khosla Ventures.

Not all energy innovations, by their nature, can use existing machines. Tesla has invested significantly in its factory in Fremont, Calif. that is now churning out the Model S and using programmed robots to assemble the cars in an entirely new way. But Tesla has also long been smart about taking advantage of the cost savings and innovation of the traditional battery sector, as it uses basic Panasonic laptop batteries linked together to power its Model S.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The fourth quarter of 2012 in cleantech
• Cleantech and investment in 2013
• The next generation of battery technology

Apple’s patent case against Samsung took an unexpected turn today. Judge Lucy Koh cut the damages, citing jury errors, removing some devices previously found to infringe the fruit-logo company’s patents. Jurors had awarded about $1.05 billion in damages, which now are just $600 million (rounded up slightly). The South Korean electronics giant isn’t off the hook by any means. More than a dozen devices remain infringers.

Judge Koh’s order is a blow to Apple, but not one that invalidates any real part of its patent victory. The judge found that the jury had inappropriately calculated damages for some products, based in part on their acceptance of arguments made by Apple expert witness Terry Musika.

Samsung had disputed the date that Apple gave for when infringement started — Aug. 4, 2010, based on a meeting between the companies. But the judge chose different dates, based on more official notifications. She writes:

The correct notice date for the ’915 and D’677 Patents is April 15, 2011. the correct notice date for the ’163, D’305, D’889, and D’087 Patents is June 16, 2011. In sum, Samsung is entitled to judgment as a matter of law that the earliest notice dates supported by the evidence are: August 4, 2010 for the ’381 patent; April 15, 2011 for the ’915 and D’677 Patents; and June 16, 2011 for the’163, D’305, D’889, and D’087 Patents. The damages numbers Mr. Musika presented to the jury were based on the August 4, 2010 notice date for all patents.

Different dates shorten the infringement and so lesser damages for most of the devices.

“It is improper to award damages for sales made before the defendant had notice of the patent, and an award that includes damages for sales made before notice of any of the intellectual property (IP) infringed is excessive as a matter of law”, Judge Koh explains.

Her 27-page ruling is a shocker for what she says must happen in the future:

Because the Court has identified an impermissible legal theory on which the jury based its award, and cannot reasonably calculate the amount of excess while effectuating the intent of the jury, the Court hereby ORDERS a new trial on damages for the following products: Galaxy Prevail, Gem, Indulge, Infuse 4G, Galaxy SII AT&T, Captivate, Continuum, Droid Charge, Epic 4G, Exhibit 4G, Galaxy Tab, Nexus S 4G, Replenish, and Transform. This amounts to $450,514,650 being stricken from the jury’s award. The parties are encouraged to seek appellate review of this Order before any new trial.

Groklaw’s Pamela Jones gives straightforward reason for the new trial: The jury calaculated the judgment “based on wrong theories. In some cases, she can’t even figure out what they did”. She adds: “This jury goofed big time, more than Judge Koh has so far acknowledged, in my view, but this order absolutely states as clearly as words can achieve that their award was based on mistakes”.

Depending on the appeal’s outcome, Samsung still faces problems. “The jury’s award stands for the Galaxy Ace, Galaxy S (i9000), Galaxy S II i9100, Galaxy Tab 10.1 WiFi, Galaxy Tab 10.1 4G LTE, Intercept, Fascinate, Galaxy S 4G, Galaxy S II Showcase, Mesmerize, Vibrant, Galaxy S II Skyrocket, Galaxy S II Epic 4G Touch, and Galaxy S II T-Mobile”, Koh writes. “The total award for these 14 products is $598,908,892”.

Apple had asked the court to increase damages to about $2.5 billion. Judge Koh’s response: “Apple’s motion for an increase in the jury’s damages award is DENIED. The Court declines to determine the amount of prejudgment interest or supplemental damages until after the appeals in this case are resolved”.

The jury awarded $1.05 billion in damages to Apple in late August. Strangely, much of the news reporting during the trial favored Apple’s storyline and ignored Samsung’s. I’ll be curious to see how today’s news fans across blogs and news sites.

Photo Credit: VIPDesignUSA/Shutterstock

Google has a new mobile app download ad format aimed at making it easier for people to download advertisers’ apps. It works with iTunes or the Google Play Store.

The format is only available for those who have upgraded to enhanced campaign settings.

“With enhanced campaigns, click-to-download ads will no longer trigger the app icon, so once you upgrade to enhanced campaign settings we recommend also creating new app promotion ads to replace your click-to-download ads,” Google notes.

To create the ads, open the campaign in AdWords, select the Ads tab, click “new ad,” and pick “App / digital content ad” as your template. Then, select the “App promotion ad.”

From there, you can choose your platform (Android/iOS), and enter the app package name or app ID.

Google says that new apps might take up to 24 hours to appear in its search results, and that the app promotion ads will only show on the devices upon which the apps can be installed.

Larry Kim, CTO of Worstream, who was an early partner on Google’s Enhanced Capaigns, tells WebProNews of the new format, “It’s a huge improvement in terms of ease of use and ad intelligence over the previous way of doing this.”

“The app Market is huge,” he says. “A $30 billion market in 2012 that didn’t even exist 5 years ago.” He notes that there are over 700,000 apps in both iTunes/Google Play stores, 40 Billion app Downloads on iTunes, etc.

“This new ad format is smart,” says Kim. “It just works. If your app works only on iOS tablets, then it will only show up to users that are using iOS tablets. The advertiser doesn’t have to fiddle around with dumb settings or anything. As a result, the ROI is better. Driving downloads and reviews to your mobile app is the key to getting good rankings in the app marketplace, and this is a very easy/cost effective way to do that.”

Kim shares more of his thoughts about the offering in his own post.

Google has started rolling out two new accessibility improvements in Google+ Hangouts: a sign language interpreter app, and more keyboard shortcuts.

Google’s Anna Cavender announced the features in a Google+ post Thursday evening, saying that both are rolling out gradually.

“Using the Sign Language Interpreter app, deaf or hard of hearing users who prefer sign language can invite interpreters to speak and sign for them during a Hangout,” she says. “They’ll always see their interpreter at the top right of the window, and they’ll become the focus of the Hangout whenever their interpreter speaks for them.”

The app can be installed from here.

Keyboard shortcuts, she says, are great for those who can’t or don’t want to use a mouse during video chat. “For example: muting your microphone is now as simple as Ctrl+D (PC) or Command+D (Mac), and you can start chatting with Ctrl+B (PC) or Command+B (Mac),” she explains.

The full list of shortcuts can be found here.

Earlier this week, Google also announced some new keyboard navigation for Gmail.

Google will not have to pay to use snippets of news content in Germany, according to a copyright law (pdf) that was passed in the country.

We discussed the proposed law last year, which Google had spoken out about. At the time, it looked like, if passed, it would have required search engines and aggregators to pay to license content from publishers in order to display headlines (with links) and snippets of text.

At the time, A Google spokesperson told WebProNews, “We don’t have any sympathy for these plans, as an ancillary copyright lacks all factual, economic, and legal foundation. And we are not alone with this opinion: The Federation of German Industries (BDI) and 28 other associations vehemently oppose an ancillary copyright for publishers. The German parliament is divided on the issue as well. For a good reason: An ancillary copyright would mean a massive damage to the German economy. It’s a threat to the freedom of information. And it would leave Germany behind internationally as a place for business.”

“Publishers should be innovative in order to be successful,” the Googler added. “A compulsory levy for commercial internet users means cross-subsidizing publishers through other industries. This is not a sustainable solution.”

“In difficult economic times, the Internet is thriving, generating economic gains, creating jobs and giving struggling businesses a vital lifeline,” they said. “It is important that any legislation supports, rather than hinders innovation on the internet to encourage new jobs and economic growth.”

It appears that things went a lot better than they could have, but the law still leaves question about how much text can actually be used in a snippet. As TechDirt points out, the wording says that quotations will still have to be licensed unless they are “single words or the smallest excerpts,” without defining what “smallest excerpts” actually means.

I guess we’ll see where it goes from here.

Apple’s (AAPL) award from last summer’s landmark patent trial with Samsung (005930) has been reduced to $598.9 million, according to a new report. The Wall Street Journal issued the report on Friday afternoon, and it comes as Apple shares dropped to a new 52-week low during Friday afternoon’s session. Apple was originally awarded more than $1 billion when various Samsung devices were found to infringe on six Apple patents. Judge Lucy Koh, who presided over the trial and cut Apple’s award, plans to schedule a new damages trial once judgements have been issued for all outstanding appeals.

Google says on its new “How Search Works” site that it launched 665 “improvements to search” in 2012.

In a graphic looking at data from 2012, Google explains that it had 118,812 “precision evaluations,” which are described as “The first phase is to get feedback from evaluators, people who evaluate search quality based on our guidelines. We show evaluators search results and ask them to rate the usefulness of the results for a given search.”

From there, it had 10,391 side-by-side experiments.

“In a side-by-side experiment, we show evaluators two different sets of search results: one from the old algorithm and one from the new, and we ask them for details about which results they prefer,” Google explains.

You can see Google’s Search Quality Raters Guidelines (at least a “cliffs notes version“) here (pdf).

Google ran 7,018 live traffic experiments in 2012. “If the evaluators’ feedback looks good, we move forward with a ‘live traffic experiment,’” Google explains. “In these experiments, we change search for a small percentage of real Google users and see how it changes the way they interact with the results. We carefully analyze the results to understand whether the change is an improvement to the search results. For example, do searchers click the new first result more often? If so, that’s generally a good sign.”

From there, came the 665 actual launches.

“Finally, our most experienced search engineers carefully review the data from all the different experiments and decide if the change is approved to launch,” says Google. “It sounds like a lot, but the process is well refined, so an engineer can go from idea to live on Google for a percentage of users in 24 hours. Based on all of this experimentation, evaluation and analysis, we launched 665 improvements to search in 2012.”

Typically, Google has been providing monthly lists of “search quality highlights” showing some of the tweaks they’ve made, but they haven’t done it in months. Despite today’s effort in transparency, it remains to be seen whether we’ll see exactly what Google has been up to since October.

We’ve seen Google’s search quality raters referenced numerous times, but now Google has made available the whole set of guidelines in one giant PDF for your perusal. The document is called “Search Quality Rating Guidelines,” and interestingly, it’s labeled version 1.0, and is dated November 2012. It was released as part of Google’s new “How Search Works” site.

“Google relies on raters, working in countries and languages around the world, to help us measure the quality of our search results, ranking, and search experience,” Google explains. “These raters perform a variety of different kinds of “rating tasks” designed to give us information about the quality of different kinds of results in response to different kinds of queries. The data they generate is rolled up statistically to give us within the Google search team a view of the quality of our search results and search experience over time, as well as an ability to measure the effect of proposed changes to Google’s search algorithms. Raters’ judgments do not directly impact Google’s search result rankings. While a rater may give a particular URL a score, that score does not directly increase or decrease a given website’s ranking. Instead these scores are used in aggregate to evaluate search quality and make decisions about changes.”

In the preface of the document, Google notes that the document itself is not the entire version that raters actually use on a daily basis, but rather a “Cliffs Note” version.

“The raters’ version includes instruction on using the rating interface, additional rating examples, etc.,” Google explains. “These guidelines are used as rating specifications for search raters, and this document in particular focuses on a core type of rating task called ‘URL rating.’ In a URL rating task, a rater is shown a search query from their locale (country + language) and a URL that could be returned by a search engine for that query. The raters ‘rate’ the quality of that result for that query, on a scale described within the document. Sounds simple, right? As you’ll see, there are many cases to think through, and this document is used to guide raters on some of those cases and how to look at them.”

In a Webmaster Help video released this past October, Matt Cutts also discussed the quality raters’ “impact” on algorithms.

Here’s another one they put out in May talking about how Google uses the human raters:

BlackBerry (BBRY) has clearly seen some early success with its next-generation BlackBerry Z10 smartphone, but there are still a number of barriers on the road ahead. BlackBerry 10 targets consumers and business customers alike, but while many saw a certain level of enterprise success as a sure thing just 12 months ago, increased competition from the likes of Apple (AAPL) and Samsung (005930) may be disconcerting. Adding further fuel to the fire, Nokia (NOK) CEO Steven Elop recently confirmed that he too is gunning for BlackBerry’s business user base.

Continue reading…

Google has released a new section on its Inside Search site called “How Search Works,” which includes an animated graphic explaining just that. It also provides a “view into major search algorithms and features,” a 43-page document about the guidelines for search raters, a slideshow about spam removal, graphs about spam, and a list of policies that explain when Google will remove content.

The animation is only available in English,but there’s also a text version in 43 languages.

On the site, Goole lists its projects that it “constantly tunes and refines”. These are: Answers, Autocomplete, Books, Freshness, Google Instant, Images, Indexing, Knowledge Graph, Mobile, News, Query Understanding, Refinements, SafeSearch, Search Methods, Site & Page Quality, Snippets, Spelling, Synonyms, Translation and Internationalization, Universal Search, User Context, and Videos.

For “Site & Page Quality,” Google says, “Uses a set of signals to determine how trustworthy, reputable, or authoritative a source is. (One of these signals is PageRank, one of Google’s first algorithms, which looks at links between pages to determine their relevance.)”

User context is defined as, “Provides more relevant results based on geographic region, Web History, and other factors.”

The “Live Spam Screenshots” section is interesting. It will show you a page, for example, that was removed from search results 34 minutes ago. It’s currently allowing you to look at 56 examples (all removed 33-34 minutes ago) in slideshow format. When you click “next,” you’re presented with a warning that says, “These screenshots are generated automatically and are not manually filtered. While uncommon, you may see offensive, sexually explicit, or violent content.”

There is still a note under the slideshow, which says, “We’ve removed some pornographic content and malware from this demo, but otherwise this is an unfiltered stream of fresh English examples of ‘pure spam’ removals.”

Google then runs through the various types of spam: cloaking and/or sneaky redirects, hacked site, hidden text and/or keyword stuffing, parked domains, pure spam, spammy free hosts and dynamic DNS providers, thin content with little or no added value, unnatural links fro a site, unnatural links to a site, and user-generated spam. The site includes this graph showing the number of domains affected by a manual action over time, broken down by different spam types:

This is followed by the listing of spam-fighting milestones, and some other interesting graphs about webmaster notifications and reconsideration requests.

Finally, the site gives you a brief overview of Google’s policies on: access to information, algorithms over manual action, exceptions lists, fighting spam and malware, transparency for webmasters, preventing identity theft, legal removals, fighting child exploitation, shocking content, and SafeSearch.