NASA first readied for 32 million hits per day. Private industry helped it handle 46.9 million.
Pathfinder was about to land on Mars, and engineers at NASA suddenly discovered a
looming problem.
Sure, everything was going extremely well for the Mars Pathfinder mission itself, and had
been ever since NASA's Jet Propulsion Laboratory (JPL) launched the probe on Dec. 2,
1996.
The crisis emerged not in space, but on the Internet: Last year, when NASA's Web site
planners had looked forward to the July touch-down date, they knew that NASA's Web
server capacity would be miserably inadequate to serve the unprecedented demand for
Pathfinder's "Mars, Live" reports.
Kirk Goodall was the NASA Web engineer for the JPL's Mars Pathfinder site team. He
had to cobble together the necessary bandwidth and server capacity for what turned out to
be the most popular event in the history of the Internet.
From July 1 to Aug. 4, the various Mars Pathfinder sites combined received a stratospheric
566 million hits. On the busiest day, July 8, the Mars Pathfinder sites combined for almost
47 million hits, which was more than double the traffic of any other Web site event.
This year's Kasparov-Deep Blue chess site hosted by IBM had 21 million hits on its
busiest day. The 1996 Summer Olympic Games site rang up a peak of 18 million hits on a
single day.
The most Internet traffic NASA had ever handled prior to the Mars Pathfinder mission was
about 2 million hits per day for the Shoemaker-Levy and Hale-Bopp comets. NASA JPL
has two T-3 lines (45 Mbps each) coming into its Pasadena, Calif., facility. The planners
knew the Pasadena site alone wouldn't be able to handle the load, but had assumed it
would be easy enough to distribute it across the rest of NASA's Web servers.
Goodall remembered discussing the problem in August 1996 with David Dubov, the site's
Webmaster, and Bob Anderson, the Pathfinder mission's public outreach coordinator, who
oversaw the Web site. Estimating that they would need to serve 25 million hits per day at
most--which in hindsight turned out to be about half of the 46.9 million peak--they tallied
up the estimated capacity of all of NASA's Web sites.
And then panic kicked in.
Goodall's first step was to mirror the site on government and university servers that are
part of the Very High Bandwidth Network Service (vBNS), run by MCI under an
agreement with the National Science Foundation. That gave him the ability to handle an
estimated 32 million hits a day.
But it turned out that not every vBNS location had a high-speed connection to the outside
Internet in place, although the administrators reassured Goodall that everything would be
fine come July 4.
Goodall was beginning to sweat. "I wasn't feeling too comfortable," he said. "I know from
experience that when you're trying to set things up in just a couple of weeks, it's difficult.
People go on vacation and things fall through the cracks."
At that point, Anderson suggested to Goodall that he turn to the private sector, but that
would introduce a number of complexities because of the restrictions the federal
government places on its agencies in dealing with corporations.
Though he knew that with corporate Web servers at NASA's disposal the Mars Pathfinder
sites could handle about 80 million hits, Goodall still felt as though he was winging it. To
get the necessary approval, he drafted a memorandum of understanding based on text from
JPL's legal division. The agreement between NASA and the corporations essentially said
their mirrored Web sites could not alter the content in any way, and could contain no
advertising.
Goodall approached Silicon Graphics, Sun Microsystems, and Digital, all of which
immediately agreed to mirror the Pathfinder site and each of which said they could handle
10 million hits a day. "They all knew that being part of this would give them a lot of
visibility," he said.
Then Goodall approached the major Internet service providers and online services for
support. He succeeded only with AT&T WorldNet and CompuServe, though America
Online eventually added a mirrored site.
Still, the corporate partnerships just about fell through. With two weeks remaining before
the landing, Goodall had not received approval from the JPL legal department. The
corporations wanted to use their logos, and the legal department was resisting because it
implied endorsement.
"I finally said, 'Forget the logos; if you guys want logos, it's going to kill the whole
thing,'" Goodall said. "'Just dump the logos and use the American flag.'" The agreements
were signed and approved one week before the landing.
Finally came the day of reckoning. Even with all the capacity of 20 mirror sites in place and
ready on July 4, some things went wrong. A router that was improperly configured at the
NASA Ames site in Northern California caused a crippling bottleneck, rendering the
center's network practically useless. Several servers around the world ran out of disk space
and crashed.
Even the two main Web servers at JPL started to get bogged down, so Goodall and crew
quadrupled the memory in their two Sun SPARC Web servers to 256 Mbytes. "All of our
sites used this as an opportunity to tune their networks. They had never seen traffic like this
before," Goodall said, without a trace of irony.
NASA now has 46 mirrors for the Mars Pathfinder site, and Goodall said another 50
mirror sites will come online in September. NASA is getting ready for Mars Global
Surveyor, another probe that will arrive at Mars Sept. 11, orbit for 4 months, and map the
entire surface of the planet. Goodall thinks there will be an even bigger demand for the
Surveyor data.
Surveyor will send back a lot more data, too; much more than the 1.2 Gbits of data
Pathfinder had transmitted through Aug. 4. Whereas Pathfinder communicated at only 9.6
Kbps, Surveyor can transmit at 56 Kbps through NASA's Deep Space Network, because it
has a more powerful antenna and because it can send its signals from orbit rather than from
the surface of Mars.
These days, Goodall feels much better about NASA's Web capacity for Surveyor. "We're
ready," he said confidently.
