SANTA CLARA, Calif. — Jeff Rothschild’s machines at Facebook had a problem he knew he had to solve immediately. They were about to melt.
The company had been packing a 40-by-60-foot rental space here with racks of computer servers that were needed to store and process information from members’ accounts. The electricity pouring into the computers was overheating Ethernet sockets and other crucial components.
Thinking fast, Mr. Rothschild, the company’s engineering chief, took some employees on an expedition to buy every fan they could find — “We cleaned out all of the Walgreens in the area,” he said — to blast cool air at the equipment and prevent the Web site from going down.
That was in early 2006, when Facebook had a quaint 10 million or so users and the one main server site. Today, the information generated by nearly one billion people requires outsize versions of these facilities, called data centers, with rows and rows of servers spread over hundreds of thousands of square feet, and all with industrial cooling systems.
They are a mere fraction of the tens of thousands of data centers that now exist to support the overall explosion of digital information. Stupendous amounts of data are set in motion each day as, with an innocuous click or tap, people download movies on iTunes, check credit card balances through Visa’s Web site, send Yahoo e-mail with files attached, buy products on Amazon, post on Twitter or read newspapers online.
A yearlong examination by The New York Times has revealed that this foundation of the information industry is sharply at odds with its image of sleek efficiency and environmental friendliness.
Most data centers, by design, consume vast amounts of energy in an incongruously wasteful manner, interviews and documents show. Online companies typically run their facilities at maximum capacity around the clock, whatever the demand. As a result, data centers can waste 90 percent or more of the electricity they pull off the grid, The Times found.
To guard against a power failure, they further rely on banks of generators that emit diesel exhaust. The pollution from data centers has increasingly been cited by the authorities for violating clean air regulations, documents show. In Silicon Valley, many data centers appear on the state government’s Toxic Air Contaminant Inventory, a roster of the area’s top stationary diesel polluters.
Worldwide, the digital warehouses use about 30 billion watts of electricity, roughly equivalent to the output of 30 nuclear power plants, according to estimates industry experts compiled for The Times. Data centers in the United States account for one-quarter to one-third of that load, the estimates show.
“It’s staggering for most people, even people in the industry, to understand the numbers, the sheer size of these systems,” said Peter Gross, who helped design hundreds of data centers. “A single data center can take more power than a medium-size town.”
Energy efficiency varies widely from company to company. But at the request of The Times, the consulting firm McKinsey & Company analyzed energy use by data centers and found that, on average, they were using only 6 percent to 12 percent of the electricity powering their servers to perform computations. The rest was essentially used to keep servers idling and ready in case of a surge in activity that could slow or crash their operations.
A server is a sort of bulked-up desktop computer, minus a screen and keyboard, that contains chips to process data. The study sampled about 20,000 servers in about 70 large data centers spanning the commercial gamut: drug companies, military contractors, banks, media companies and government agencies.
“This is an industry dirty secret, and no one wants to be the first to say mea culpa,” said a senior industry executive who asked not to be identified to protect his company’s reputation. “If we were a manufacturing industry, we’d be out of business straightaway.”
These physical realities of data are far from the mythology of the Internet: where lives are lived in the “virtual” world and all manner of memory is stored in “the cloud.”
The inefficient use of power is largely driven by a symbiotic relationship between users who demand an instantaneous response to the click of a mouse and companies that put their business at risk if they fail to meet that expectation.
Even running electricity at full throttle has not been enough to satisfy the industry. In addition to generators, most large data centers contain banks of huge, spinning flywheels or thousands of lead-acid batteries — many of them similar to automobile batteries — to power the computers in case of a grid failure as brief as a few hundredths of a second, an interruption that could crash the servers.
“It’s a waste,” said Dennis P. Symanski, a senior researcher at the Electric Power Research Institute, a nonprofit industry group. “It’s too many insurance policies.”
JP Rangaswami: Information is food
How do we consume data? At TED@SXSWi, technologist JP Rangaswami muses on our relationship to information, and offers a surprising and sharp insight: we treat it like food.
JP Rangaswami thinks deeply (and hilariously) about disruptive data. Full bio »
An old farmer named Clyde had a car accident. In court, the trucking company’s fancy lawyer was questioning Clyde. “Didn’t you say at the scene of the accident, ‘I’m fine,’?” asked the lawyer.
Clyde responded, “Well, I’ll tell you what happened. I had just loaded my favorite mule, Bessie, into the……”
“I didn’t ask for any details,” the lawyer interrupted. “Just answer the question. Did you not say, at the scene of the accident, “I’m fine!’?”
Clyde said, “Well, I had just got Bessie into the trailer and was driving down the road….”
The lawyer interrupted again and said, “Judge, I am trying to establish the fact that at the scene of the accident, this man told the Highway Patrolman on the scene that he was just fine. Now several weeks after the accident he is trying to sue my client. I believe he is a fraud. Please tell him to simply answer the question.”
By this time, the Judge was fairly interested in Clyde’s answer and said to the lawyer, “I’d like to hear what he has to say about his favorite mule, Bessie.”
Clyde thanked the Judge and proceeded, “Well… as I was sayin’, I had just loaded Bessie, my favorite mule, into the trailer and was drivin’ her down the highway when this huge semi ran the stop sign and smacked my truck right in the side. I was thrown into one ditch and Bessie was thrown into the other. I was hurtin’ real bad and didn’t want to move. However, I could hear ole Bessie moanin’ and groanin’. I knew she was in terrible shape just by her groans.
“Real soon a Highway Patrolman came on the scene. He could hear Bessie moanin’ and groanin’, too. So, he went over to her. After he looked at her, he took out his gun and shot her between the eyes. Then the Patrolman came across the road, gun in hand, looked at me, and said, ‘How are YOU feeling?’
“Now what the heck would you say?”
The hullabaloo surrounding this weekend’s 100th anniversary of the sinking of the Titanic means a whole generation of youngsters is slowly realizing that the disaster was more than a figment of James Cameron’s imagination.
A fascinating short film about how the now-iconic “Keep Calm and Carry On” WW2 propaganda poster went unseen by the public for decades before being discovered and distributed by a small secondhand bookshop in Alnwick, Northumberland, called Barter Books.
In the brutal world of online commerce, where a competing product is just a click away, retailers need all the juice they can get to close a sale.
Readers shared their thoughts on this article.
Some exalt themselves by anonymously posting their own laudatory reviews. Now there is an even simpler approach: offering a refund to customers in exchange for a write-up.
By the time VIP Deals ended its rebate on Amazon.com late last month, its leather case for the Kindle Fire was receiving the sort of acclaim once reserved for the likes of Kim Jong-il. Hundreds of reviewers proclaimed the case a marvel, a delight, exactly what they needed to achieve bliss. And definitely worth five stars.
As the collective wisdom of the crowd displaces traditional advertising, the roaring engines of e-commerce are being stoked by favorable reviews. The VIP deal reflects the importance merchants place on these evaluations — and the lengths to which they go to game the system.
Fake reviews are drawing the attention of regulators. They have cracked down on a few firms for deceitful hyping and suspect these are far from isolated instances. “Advertising disguised as editorial is an old problem, but it’s now presenting itself in different ways,” said Mary K. Engle, the Federal Trade Commission’s associate director for advertising practices. “We’re very concerned.”
Researchers like Bing Liu, a computer science professor at the University of Illinois at Chicago, are also taking notice, trying to devise mathematical models to systematically unmask the bogus endorsements. “More people are depending on reviews for what to buy and where to go, so the incentives for faking are getting bigger,” said Mr. Liu. “It’s a very cheap way of marketing.”
By last week, 310 out of 335 reviews of VIP Deals’ Vipertek brand premium slim black leather case folio cover were five stars and nearly all the rest were four stars. The acclaim seemed authentic, barring the occasional indiscretion. “I would have done 4 stars instead of 5 without the deal,” one man bluntly wrote.
VIP Deals, which specializes in leather tablet cases and stun guns, denied it was quietly offering the deals. “You are totally off base,” a representative named Monica wrote in an e-mail.
But three customers said in interviews that the offer was straightforward. Searching for a protective case for their new Kindle Fire, they came upon the VIP page selling a cover for under $10 plus shipping (the official list price was $59.99). When the package arrived it included a letter extending an invitation “to write a product review for the Amazon community.”
“In return for writing the review, we will refund your order so you will have received the product for free,” it said.
Anne Marie Logan, a Georgia pharmacist, was suspicious. “I was like, ‘Is this for real?’ ” she said. “But they credited my account. You think it’s unethical?”
While the letter did not specifically demand a five-star review, it broadly hinted. “We strive to earn 100 percent perfect ‘FIVE-STAR’ scores from you!” it said.
The merchant, which seems to have no Web site and uses a mailbox drop in suburban Los Angeles as a return address, did not respond to further requests for comment. As of last week, the company (as opposed to its products) had received 4,945 reviews on Amazon for a nearly perfect 4.9 rating out of five.
Amazon is expected to sell 20 million Kindle Fire tablets this year, making the market for cases potentially enormous. But it is also bitterly competitive, with dozens of models in Amazon’s Kindle showroom. With a modest investment, VIP pushed its product far above the competition, none of which had so much enthusiasm with so little dissent. Customers like Ms. Logan, who got something they had genuinely wanted for only a small shipping charge, were of course thrilled. And Amazon racked up more revenue.
Even a few grouches could not spoil the party. “This is an egregious violation of the ratings and review system used by Amazon,” a customer named Robert S. Pollock wrote in a review he titled “scam.”
He was promptly chastised by another customer. This fellow, himself a seller on Amazon, argued that he had both given and gotten free items in exchange for reviews. “It is not a scam but an incentive,” he wrote.
Under F.T.C. rules, when there is a connection between a merchant and someone promoting its product that affects the endorsement’s credibility, it must be fully disclosed. In one case, Legacy Learning Systems, which sells music instructional tapes, paid $250,000 last March to settle charges that it had hired affiliates to recommend the videos on Web sites.
Amazon, sent a copy of the VIP letter by The New York Times, said its guidelines prohibited compensation for customer reviews. A few days later, it deleted all the reviews for the case, which itself was listed as unavailable. Then it took down the product page itself.
Asked why Amazon did not seem to notice that at least a few consumers called into question the VIP deal on its own site, a spokeswoman declined to comment. Nor would she say exactly what happened to VIP’s other products, like the Vipertek VTS-880 mini stun gun, which also disappeared from the retailer.
The gun, like the Kindle case, received nearly all five-star reviews. “I bought one for my wife and decided to let her try it on me,” one man wrote in a typical display of the sort of effusiveness that VIP inspired. “We gave it a full charge and let me just say WOW! Boy do I regret that decision.”