Prediction Markets: Research and Limits

Fact: A story in Science Daily this week, “Election Forecasters Preparing For Historic Election,” relates the publication this month of the “assembled insights of prominent election forecasters in a special issue of the International Journal of Forecasting.” 

Analysis:  The journal articles are available here for download.  One of them, “Prediction Market Accuracy in the Long Run” (by Joyce E. Berg, Forrest D. Nelson, and Thomas A. Reitz from the University of Iowa’s Tippie College of Business), compares the presidential election forecasts produced from the granddaddy of them all, the Iowa Electronic Market (IEM), to “forecasts from an exhaustive body of opinion polls.”  Science Daily says they find that the IEM is “usually more accurate than the polls.”

If we extrapolate out, these election markets are special cases of prediction markets, and I’m always interested in those.

Continue reading

“The Largest Social Network Ever Analyzed”

FACT: According to ComScore data cited in a story in Monday’s FInancial Times, “Facebook, the fast-growing social network, has taken a significant lead over MySpace in visitor numbers for the first time… Facebook attracted more than 123 million unique visitors in May, an increase of 162 per cent over the same period last year… That compared with 114.6 million unique visitors at MySpace, Facebook’s leading rival, whose traffic grew just 5 per cent during the same period… The findings mark the first time that Facebook, launched in 2004, has taken a significant lead in unique visitors, [and] come at a time of change inside Facebook, as the one-time upstart attempts to transform itself into a leading media company.

ANALYSIS:  This week several members of the Microsoft Institute met in Redmond with a visiting friend from government, and among other talks we had a very interesting discussion with Eric Horvitz, a Microsoft Research principal researcher and manager.  Eric’s well known for his work in artificial intelligence and currently serves as president of the Association for the Advancement of Artificial Intelligence (AAAI).

We talked about one of Eric’s recent projects for quite a while: “Planetary-Scale Views on a Large Instant-Messaging Network,” a project which has been described by his co-author as “the largest social network ever analyzed.” 

Continue reading

Data Centers and Aircraft Carriers (and Google)

FACT: Information Week has a solid story today, “Inside Microsoft’s $550 Million Mega Data Centers,” with a tour of the new San Antonio data center under construction.  It’s of the “Quincy-class” (our term in homage to Navy lingo, meaning big, but not the biggest of aircraft carriers; that would be the Chicago-class data center, see below).  The reporter writes: “By September, it’ll be the newest star in Microsoft’s rapidly expanding collection of massive data centers, powering Microsoft’s forays into cloud computing like Live Mesh and Exchange Online, among plenty of other as-yet-unannounced services.” 

ANALYSIS: I get asked about “the new way to build data centers” more often than any other question but one by government technology professionals.  The most popular question, and it’s related, is about cloud computing.  They both came up today during a meeting with one of the National Labs.

Continue reading

How to Find Research: Here, There, Everywhere

FACT: The Washington Post today has a story in the Business section (“Intelligence Agency Joins U-Md. Research Center“) about the relationship between IARPA and the University of Maryland, the location of the planned new IARPA headquarters. 

ANALYSIS: UMd has a set of valuable relationships with the public- and private-sector national security community, and the IARPA startup is just the latest agency to benefit.   Proximity is key, for research and bureaucracy.  In Maryland’s case, IARPA Director Lisa Porter told an IEEE interviewer last month that “It’s nice not to be sitting right next to one particular agency. It’s also nice to be near a university because we’re sending a message that we want to bring in nontraditional partners: academia, industry. It sends a nice message that we’re embracing the broad community to help us solve these challenging problems.”

I lament sometimes that Charlottesville (home to my undergraduate alma mater) is a good two hours away from DC, as even that distance puts a frustrating limit on the amount of joint work that winds up being done with Virginia faculty and students. 

Continue reading

War is Virtual Hell

FACT: According to market research compiled by Microsoft, the global market for Modeling and Simulation (M&S) software/hardware platforms across all industries, including the defense industry, has hit $18 billion per year; the cumulative growth rate is estimated at 9.6% annually.

ANALYSIS:  The lovely wife and I have been lackadaisically house-hunting down in Virginia’s Northern Neck, the Athens of America and the cradle of our democracy. The Neck is the birthplace of George Washington, the Lee brothers (the revolutionary patriots about whom John Adams used the phrase, “This Band of Brothers,” among them Declaration of Independence signers Richard Henry Lee and Francis Lightfoot Lee, not to mention their later nephew Robert E. Lee), James Monroe, John Ballentine, etc. etc. 

Anyway, recently we toured the historic 1859 house at Braehead, an 18-acre estate actually located within the Civil War battlefield in Fredericksburg, Virginia. The house is for sale, and while it’s likely overpriced (like everything else on the market these days) [this observation has been energetically and somewhat persuasively disputed by the listing agent, who read the post], but we enjoyed the tour.  I’ve posted many (too many) photos of our little tour here.  My interest in the house is the historic angle: it’s actually where Robert E. Lee visited and took breakfast on the morning of the Battle of Fredericksburg in 1862, one of his successful efforts against U.S. forces.  Here’s an article about Braehead’s history and historic preservation.

It was in fact at Fredericksburg that Lee spoke the words which would sum up the entire war, nay all wars, as he witnessed thousands of Union soldiers falling in battle to Confederate guns on the hills above the Rappahannock River: “It is well that war is so terrible, lest we grow too fond of it.”  

Continue reading

Is It Even Possible to Connect the Dots?

FACT: Among the inspired ideas of polymath Danny Hillis (pioneer of parallel computing) was establishing the Long Now Foundation, whose projects include the millennial Long-Now Clock (“the world’s slowest computer”) and the notion of “Long Bets.”  A Long Bet is an “accountable prediction,” meaning one that has a specified end-date and a testable, wagerable, proposition.  One of the early Long Bets posted wagers $2,000 that “By 2020, no one will have won a Nobel Prize for work on superstring theory, membrane theory, or some other unified theory describing all the forces of nature.”  That particular bet is one of many signs of scientific skepticism about string theory.

ANALYSIS: Even without the ease of hyperlinks, old-fashioned newspapers foster serendipitous connections between articles, particularly if you’re reading a Sunday morning paper with lots of sections. Sunday the Washington Post did me a service by placing in different sections a couple of articles which I connected, about intelligence “failures” and about stock-market prediction, leading me to some web-surfing about the questionable validity of string theory and some related observations about the difficulty of predicting human behavior.

In the Outlook section, the Post has an opinionated and thought-provoking op-ed piece by Mark Lowenthal, one of the most “intelligent” individuals in the recent history of the U.S. intelligence community (after all, he was the 1988 Jeopardy grand champion, as well as a former assistant director of CIA).

Continue reading

IARPA and the Virtual Long Tail

FACT: This week, the Intelligence Advanced Research Projects Activity, an arm of the Director of National Intelligence (DNI), launched its new unclassified website.  What’s there is initially fairly minimal, but they’ll be adding to the public information posted there regularly.

ANALYSIS:  I spent the week in Orlando, as a Keynote speaker at the IARPA “Incisive Analysis Conference.”  I’ll be writing a little more about the conference in the near future, as I saw some great demo’s and spoke to the principal investigators on many excellent and far-sighted advanced research projects sponsored by IARPA.  It was great to be there and to see so many old friends from the intelligence community, the national labs (PNNL, Sandia, Oak Ridge, Livermore), DoD, and innovative commercial R&D outfits.  Also, as the first IARPA conference since the organization’s launch, it was an opportunity to hear new director Lisa Porter communicate her vision and principles, which she did well and I’ll discuss those soon as well.  (She also kidded me about my efforts to make her a cultural phenomenon, but I blamed it on WIRED magazine.)

Continue reading

Google Accelerates Hiring of Nobel Laureates

FACT:  Answering a question in this week’s Business Week about several recent high-profile departures of Google executives and engineers, CEO Eric Schmidt said: “What bothers me is that some people write: ‘So-and-so left the company.’ Well, they don’t also write that we hired 120 people that week, five of whom have Nobel prizes, three of whom have PhDs, and so on, who are beginning their career here now.”

ANALYSIS: There have only been some 700 Nobel Laureates awarded in the history of the program since 1901, according to the official Nobel site, and at least as of a 2001survey there were approximately 210 living Nobel prize-winners.

So, with some trepidation, I calculate that by Schmidt’s aggressive hiring of five Nobel laureates in a typical week, the entire roster of living prize-winners will be working for Google within a year.

Continue reading

Microsoft Research Reclaims Value of Pi

pi-techFACT: Educators in the state of Alabama are chafing as the state celebrates a dubious anniversary: today marks ten years since the Alabama state legislature voted to change the value of the mathematical constant pi from 3.14159… to the “Biblical value” of 3.0.  Ramifications were felt across the state. 
Now, a team of Microsoft Research computer scientists have announced success in a groundbreaking effort to refactor the Biblical value, using modern high-performance computing hardware and machine-translation technologies on the original Old Testament texts.
  
ANALYSIS:  Looking back, an April 1998 issue of Science and Reason newsletter written by physicist Mark Boslough recounts the political and cultural battles which went behind the Alabama legislative change. The legislature of the “Yellowhammer State” justified the change by citing biblical injunction. As one supporter put it: “the Bible very clearly says in I Kings 7:23 that the altar font of Solomon’s Temple was ten cubits across and thirty cubits in diameter, and that it was round in compass.”

The use of “3.0” as the value of pi led to problems in schools, businesses, and local scientific pursuits, including a group of frustrated engineers at the NASA research facility in Huntsville.  According to NASA/Huntsville’s director of special projects “Dr.” Jim Simon (doctorate pending), “We had strayed from using our Microsoft software and instead had been trying to figure out how to use an advanced Google search platform, which was sold to us as a powerful Cloud Computing system.”

Unfortunately, that effort proved frustrating for the “rocket scientists” any time they used calculations involving pi, based on the Alabama-standard value of 3.0, mostly because they were under the mistaken impression that they were contractually barred from using Cloud Computing on any sunny days. Given the hospitable local weather that left them unable to use their computers for an average of 290 days each year.

Continue reading

The Future of Enterprise Computing – and Social Computing

I wrote the other day about how highly ranked the University of Virginia’s undergrad business school is (a close second in BusinessWeek’s annual ranking), and mentioned that one reason is the creative research and programs they sponsor.

In fact, thanks to UVA’s McIntire School of Commerce, I enjoyed a great day recently exploring some of my favorite topics with leading experts.  I was an invited speaker at their one-day conference on The Future of Enterprise Computing on March 14th, presented by McIntire’s Center for the Management of Information Technology (CMIT).  It was a fascinating day….

Continue reading