Webmastering Mars

by Chuck Toporek Asst. Managing Editor, Web Review		It all started one day last July when David Dubov, the Webmaster for the Mars Pathfinder site walked into a cramped office at NASA's Jet Propulsion Laboratory in Pasadena, California and approached Dr. Cheick M. Diarra with a potential problem that could bring the whole Internet crashing down on them. Dr. Diarra, who is the Educational and Public Outreach Manager for the Mars Exploration Program, was in charge of overseeing JPL's Mars Pathfinder presence on the Web, was instantly faced with a dilemma that few of us would be able to deal with. "The JPL site won't be able to handle the traffic when Pathfinder lands on Mars," explained Dubov, "what are we going to do about it?" It was at that time that Dr. Diarra authorized the Mars Pathfinder Web development team to set out and find secure mirror sites to help manage the load. The only problem was, nobody on the team had any experience in this. Having an interest in setting up real-time information over the Internet, Dr. Diarra called upon Kirk Goodall to save the day. His primary task would be to find and secure mirror sites in the United States and abroad, as well as overseeing the transition to a distributed Web site.
Kirk Goodall stands near a model of the Mars Pathfinder lander and rover.		Beyond the initial fears of placing information from a government-regulated site onto someone else's server, one of the main hurdles that needed to be crossed was cleaning up the HTML code for the pages that already existed. When they first began the process, the Mars Pathfinder site had hundreds of links, all of which had to be changed to relative links -- no small task when you're talking about a few dozen pages with limited manpower. Over the next few months, the Web development team worked to change all of the links and set up the first mirror site internally in late November at Ames Research Center. Although media interest in the launch seemed sparce, public interest in the mission was huge. Traffic on launch day reached close to 25 million hits. Based on that information, Kirk Goodall factored in typical file sizes to determine bandwidth requirements and the required number of mirror sites to handle the amount of traffic they might receive when the Pathfinder mission actually started the Entry, Descent and Landing (EDL) phase of the mission. The numbers were staggering: Traffic to the site had the potential of being in the 60-80 million hits/day range. Managing the Mirrors Knowing full-well that the Mars Pathfinder site at JPL and the smaller mirror couldn't handle that much traffic, Goodall set out immediately to secure more hosts to mirror the site. Right off the bat, Kirk secured mirrors with all five of the research centers under the umbrella of the National Science Foundation (NSF) (see the table below).

The combined capacity from these sites measured ~30 million hits per day, but that was not even close to the estimated 60-80 million hits they might receive. After some debate, it was decided that the team would start approaching corporations and international scientific institutions to help carry the load. The scientific institutions were one thing -- afterall, they shared a common interest -- however, securing mirror sites with corporations opened a whole new can of worms. The team had to consider the legal implications of mirroring with them so it didn't look like JPL or NASA were favoring one company over another. Over the next few weeks, Kirk played part-time lawyer while the rest of the MPF Web development team targeted specific companies that they could approach. After drafting his own MOU (Memoranda of Understanding), Kirk contacted Sun Microsystems, Silicon Graphics, Inc., and Digital -- all of which responded immediately to the request and signed on the dotted line. Enter the ISPs One thing that he hadn't considered was the major Internet Service Providers (ISPs), until an article appeared in the Wall Street Journal that said some of major ISPs, like AOL and CompuServe, were planning to cache the Mars Pathfinder Web pages on July 4 to save their servers from blowing due to increased traffic. This was not good. Since the plan was to have constant updates to the site throughout the EDL phase of the mission, and with only three weeks until the big day, Kirk set out to contact five of the major national ISPs: AOL, ATT WorldNet, CompuServe, MSN, and Prodigy. His email message to the heads of the ISPs was simple: "We're landing on Mars on July 4th and we want to take you with us." The only problem was that some of the ISPs ignored this message, probably thinking he was a crank, and ignored the message. In fact, he received one message back from MSN with words to the effect, "I didn't ask for anything from you. Don't email me again." So Goodall spent the next two days following up with subsequent, less intrusive, email and voicemail. In the end, he was only able to get commitments from CompuServe and ATT WorldNet. (CompuServe agreed to make their mirror open to the public, while ATT WorldNet served up their mirror as an exclusive site for their subscribers.) Although MSN dropped the ball, and AOL's president was out of town, both ISPs will soon have a Mars Pathfinder mirror of their very own. You see, NASA doesn't hold any grudges.

This map indicates the locations of the worldwide network of mirror sites for the Mars Pathfinder Web site.

		At present, there are 21 mirror sites around the world with a combined capability of handling over 87 million hits per day. The amazing thing is that all of the mirror sites are in it for the long-haul. Most have agreed to stay on until well past the point where there is no new data to report -- which, from the sound of it, could be a long time. And in the coming weeks, new mirror sites are sure to pop up as others start to fade out of the limelite. The team is presently working on arranging mirrors in Italy, Germany, Kazakhstan, Brazil, Canada, South Africa, and Spain.
Interested in finding out how the mirrors performed on their first day of heavy traffic? See, Traffic on Mars.		Showtime Taking it down to the wire, Kirk Goodall worked desperately to put the finishing touches on providing real-time data during the EDL stage of the mission. This task wasn't exactly simple to pull off as it required precise timing and careful planning. Because Goodall knew what was going to happen during EDL, the Web development team prepared splash screens that would describe what was going on at a particular time. These screens were pre-approved by mission controllers, so that when a certain thing happened, the pages were automatically FTP'd to all of the mirror sites. A META refresh tag refresh was incorporated into the HTML so that the pages would automatically update once every 30 seconds. This enabled live coverage of EDL on television and on the Internet to be in total synch. For instance, one researcher at JPL had his computer monitor sitting next to a television tuned to CNN, and the coverage from the Mars Pathfinder Web site matched the live coverage by the CNN anchors. Adding the Imagery The path that the image data takes is a long one -- 127,858,738.720 million miles to be exact. As the rover takes the images, they are beamed to the lander where the data is stored until the next transmission time. Once the data is sent from Mars, it takes 20 minutes to reach one of three Deep Space Network (DSN) stations on Earth, who then beam the data back to the Mission Image Processing Lab (or MIPL, pronounced "mipple") at JPL. Once there, it's the duty of the men and women at MIPL to lace the image data together -- which, by the way, is received in high-resolution VICR format. Once the images are cleared, they are converted from high-res format into low-res for easy dissemination over the Internet. The high-res images are also made available to the press on two secret media Web sites for use in print and on television. The reason the high-res images are not made available to the general public, at least not yet, is a simple matter of bandwidth. Since most corporate sites lease their connection to the Internet, they are only allotted so much bandwidth. If, for instance, half of the visitors to SGIs mirror tried to download a 70-Mbyte high-res full-color image, it would chew up all of their bandwidth and they would lose their connection. So for now, the high-res data stays in the hands of NASA and the media, but will be made available to the general public in the near future. So how are NASA scientists taking advantage of the high-res imagery? They're using it in ways we never thought possible. Telemetry data and images from the landing site are being used by MIPL, Ames Research Center and members at Silicon Graphics to create a VRML layout of the lander and terrain which surrounds the landing site. All of the images will be seamlessly laced to create a full-color panorama in 3D. These VRML images, which will be available in one to two weeks, will allow scientists and engineers to map out the terrain and understand how the lander came to rest. And currently available on the Mars Pathfinder site are QuickTimeVR sterographic images of the landing site which were created by Apple Computer Corporation. These images require you to use special red/blue 3D glasses and a high-resolution monitor, but they're well worth the effort to see the terrain and the lander in spectactular 3D. The Web development team also wanted to add streaming audio and video to the site using feeds from NASA Select TV from press conferences and the Mission Control Room, but they didn't have the manpower to pull it off. They approached one of their mirrors, Digital, to see if they'd be interested in helping them out. And without even blinking an eye, Digital stepped forward to give them a hand. JPL hooked up an ISDN line at Digital Entertainment to transfer the NASA Select TV broadcasts. Using streaming audio and video technology from Vosaic, they turned the information around in record time and put it up on their site, which has received over 750,000 hits to date. Although some people who've tried to access the NASA Select TV broadcasts haven't been particularly pleased with the use of Vosaic (this was Digital's choice) JPL has been taking the brunt of the complaints. However, considering the timeframe they were working within, JPL has been nothing but pleased with the efforts of Digitals' technicians and stand behind their decision to use Vosaic due to its ability to monitor and adapt to varying bandwidths. And after all, three-quarter of a million hits is nothing to scoff at.
"We know how to do things right, but there are only a few of us, so we had to prioritize and do what was essential first." -- Kirk Goodall		What the Future Holds It's fairly safe to say that the world has been amazed by the imagery that's been received from the Mars Pathfinder mission. The true test of the project is when all of the scientific and technical data can be sorted through and placed online. After all, it's easy for us to forget that this is a scientific mission. For instance, although the landing took place two weeks ago, engineering data of the EDL phase is just now being made available and should be online by press time. NASA scientists are just beginning to roll up their shirt sleeves and delve into the information that's being sent back via the DSN. It takes a while for teams of scientist and engineers to pour over the data and reach their conclusions. And as soon as they have something to report, you're sure to find it online at one of the Mars Pathfinder mirror sites. "Our hope, is to use the Mars Pathfinder as the world's outpost to Mars," says Goodall quite candidly. "Wouldn't it be a kick if, in addition to your local weather report on the Today Show, that you could also receive the current weather report on Mars? Now that would truly be an astounding use of this technology." And although this isn't quite yet possible with the technology on the Mars Pathfinder, it certainly illustrates the possibilites for future planetary missions. Dan Goldin, the Head of NASA, couldn't agree more. "We [NASA] want to create a virtual presence on Mars and a continued presence on the Web." Other articles in the Webmastering Mars series: Traffic on Mars An interview with Richard Pavlovsky of the MPF Web development team Design to Discourage Wandering The Internet Hour: Mars, the Red Planet

Web Review copyright © 1998 Songline Studios, Inc. Web Techniques and Web Design and Development copyright © 1998 Miller Freeman, Inc. ALL RIGHTS RESERVED