The rise of smartphones and tablets has presented some new challenges to performance testers, but those devices have also created demand for more reviews. Mr. Shimpi believes he can continue to prosper by sticking to a simple mantra.
“What are they not telling me?” he regularly asks, referring to the companies whose devices he tests.
Mr. Shimpi recently demonstrated how he works, running scripted videogame sequences on a MacBook Air to test the performance of its graphics chip. That’s just one example of several tests he runs on each device he reviews. The Harry Potter movie playing over and over on a Google Nexus 7 tablet was part of a test to document its battery life.
Mr. Shimpi carries out measurements several times for each device, with the results feeding spreadsheets with thousands of data points. It’s a never-ending process as Mr. Shimpi adds new products to his database and runs new benchmarks on older ones.
Chip executives have embraced the most professional of the benchmark reviewers and ship them samples of their new products, often ahead of their release. In return, they get objective feedback.
“We literally go into every review site in the world we can find, and our teams read the reviews, and they decide internally whether it was a good review for us or a good review for the competition,” Jen-Hsun Huang, chief executive of chipmaker Nvidia, told investors at a conference in May.
To make sure his reviews are ready in time for product launches, Mr. Shimpi pulls all-nighters and lays out his testing gear in hotel rooms during his frequent travels.
“If you put in an honest seven days of work – I’m not saying eight hours a day or less, I’m saying if you don’t sleep for a couple of nights, and that’s all you live and breath and do – I think it’s possible to deliver a good review within that seven-day period,” Mr. Shimpi said.
“Anything less and you start making sacrifices.”
Evaluating PC processors is a matter of connecting them to one of the motherboards on Mr. Shimpi’s table and running standard tests established over a decade ago. Testing the components in a mobile device like an iPad is trickier because it cannot easily be opened up and tinkered with.
To adapt, reviewers are resorting to some decidedly low-tech tools like stopwatches and cameras to measure the quality of tablet displays, how quickly web pages load, and battery life.
Soon after his start in high school building PCs for students and faculty at Saint Augustine’s College in Raleigh, where his father taught computer science, Mr. Shimpi created a website and started writing about components. He quickly gained a following with a rapidly growing niche of PC enthusiasts.
“I would build the PC for free and then say I want to review this stuff before I give you your computer,” Mr. Shimpi said. “As I got popular, a couple of resellers wanted to put ads on my site. So I gave them ad spots in return for more hardware to review.”
As the website grew, Mr. Shimpi started getting invitations to visit with companies and attend trade shows. Self-conscious about his age, he wore suits to meetings.
AnandTech soon made the teenager financially independent. He went on to study computer engineering at North Carolina State University while continuing to build his business.
Today he stills wears a suit to meetings and trade shows – sometimes accompanied by sneakers. He deliberately maintains a distance between his personal life and the tech world, even if that means frequent, long flights to Silicon Valley to visit chip execs.
His sprawling house, which he had built, includes a storage room for the parts companies have sent him over the years. It also includes a professional-quality home theater, carefully designed with the help of a reader and controlled by a computer Mr. Shimpi cobbled together for the task.
Plastic guitars and drums – the virtual instruments of the Rock Band videogame – are strewn across a sofa but Mr. Shimpi complains that he and his girlfriend, a sculptor who lives with him, are too busy to play much.
He takes phone calls from investors who pay him for his advice and spends more and more time hunkered down with design engineers. But Mr. Shimpi says his main focus will remain AnandTech’s readers – the sort of tech fans who spend hours reading up on new products before deciding which to buy.
“I don’t care so much how this affects the companies,” Mr. Shimpi said. “They’re going to be okay. It’s the guy putting $200 down that he worked really hard for, and some guy he’s never met is telling him he should do that. They’re the reason I get to do this.”