Systems Journal

Technology Strategists is implementing a small Linux server for network monitoring and mail filtering. It is, after all, just another computer -- after 30+ years they all start to look the same, at least share a broad range of common problems and issues. The intent of this journal is to record our experiences with this technology to share with others. Originally called 'Linux Journal', this document was renamed to 'Systems Journal' to include our experiences with using and integrating Linux and Windows technologies.

Source an older no-name PC to be converted into a Linux server. Machine had 32mb memory, 233mhz Pentium processor, 3.2gb disk.
Download Red Hat Linux 8.0 installation files, burn CDs for installation. This part of the process was pretty mindless -- just required lots of disk space on the NT workstation and a CD burner.
Installation went smooth, but 32mb not enough to run graphical installer. Text mode installation instead -- essentially undocumented. Also, 3.2gb of disk is very tight, wish we had more disk, but the controller has a non-standard connector so adding disk is out for now. By not installing games, graphics tools and such, the file system after install is 76% full -- pretty tight, but usable.
Network configuration fun -- the graphics installer during setup may do all the network configuration, but not the text installer. A pretty brutal learning experience followed on what files needed to be changed for the network to work. Most of the references assume that the low level stuff is already done, so fixing this part was not a lot of fun -- disappointed that vendors like Red Hat don't really document where all the pieces are buried, did not make it any easier.
Install X-windows software on NT4 PC, bought WinaXE Plus over the web. Worked great once the Linux box was configured to support remote X sessions -- lots of info available on the web about how to do it. Google got a real workout.
Setup SAMBA to access NT printers, share files. Once again, the GUI configuration tools stopped short -- built all the right components but in a non-standard location. Manual changes to the configuration files were needed to get the printer driver and Samba to play together -- more Google-ing. Discovered that NTFS file structures are tricky to access from Linux -- so file sharing is a bit complicated. Can read NTFS from Linux, but can read-write Linux from NT. A bit more complicated than I expected but still not too bad -- but only marginally documented.
X-windows follies - 32mb barely enough for remote desktop. But text mode operations really fly -- just like Windows, the overhead of a GUI desktop is awesome. So convenient but soooo expensive in system resources.
Upgrade memory, locate motherboard config on www.motherboard.org. Install another 128mb for a total of 160mb -- definitely not for the faint of heart. Almost more fun than building the hardware from scratch (done that). But the extra memory made all the difference -- can now bring up Linux GUI applications on an NT workstation (over X-windows), almost an extension.
Cannot decide which GUI desktop to use, so ended up switching between GNOME and KDE. Each have their points. Red Hat seems to prefer GNOME, most of the GUI tools end up here. KDE looks better for software development -- oh well.
Decided that until the disk space issue is addressed, the Linux box is probably most useful as a SPAM filter and network monitor. And probably a better test environment for web page updates -- web site is hosted externally on Apache. Validation of new web page changes on IIS is of limited utility.
Spamassassin comes with RH, working out how to patch it into the POP3/Exchange process will be interesting.
Decided to install Nagios (used to be called netsaint) -- it can monitor NT as well, but requires that MySQL be installed in addition to Apache. Not going to make the disk space issue any easier.
Just received the notice for RH9. Checked the RH website, found nothing but marketing happytalk about the new version. Read the discussion on slashdot, decided that the potential of redoing everything was too depressing. Elected to stay with 8.0 for the present, perhaps consider 8.1 since the discussion suggested it was a bit more stable. Still, in a production shop doing major upgrades every six months would be a challenge.
Phone call from my web host/mail provider that they were shutting down forced a change in plans. Located a new provider who included SPAM filtering as part of the mail service with the web hosting -- moved everything to the new location over the weekend with only minor problems. So the mail filtering requirement has gone away, the linux box remains a monitoring tool and general playground.
For no clear, rational reason, decided to do the RH9 upgrade anyhow. Major mistake -- the update insisted on redoing the partitions -- this was ok as the critical parts of the configuration were backed up elsewhere -- but other than floppies, there was no local backup device on this machine. Well, the install redid the partitions, which wipes the disk, then started to load the new software. After an hour or so, the install terminated abruptly with 'sig 11'. After four tries over much of a week, each failing in a different place with the same error, discovered a website on various signals. The main reason for a sig 11 (segmentation fault) is flaky memory, it said. So I ran memory diagnostics for 48 hours, nothing! On re-reading it, noticed that there was a note about Red Hat installers since v6 doing sig11 randomly. That certainly has been my experience, but did not realize it was a 'feature'. Untill RH9, trying again was enough to get through -- not this time, we have blown up every time, leaving an unusable box. Trolling various sources for possible alternatives -- general recommendation was SuSE, a german linux distribution (supposedly cleaner and more stable).
So we are starting over with SuSE. Tried their evaluation cd, found it was a bit faster and smoother -- SuSE prefers KDE as the desktop default rather than GNOME. Their system configuration tool YAST (probably for Yet Another System Tool) seems a bit more unified. Under the covers, their configs are done a bit differently than RH -- the settings that allowed remote X access with RH doesnt work with SuSE. Could get FTP working but not telnet or ssh -- so working on the system is slow and uncomfortable. The local retailers don't have the current release, so are doing the install/updates via FTP. The network install is a lot slicker than downloading and burning the RH cd's, but totally graceless about handling FTP errors in the middle of a four hour install. And of course, if there were any errors, you dont get a bootable system -- so starting over from scratch is mandatory. Resolved that once this is completed, will resist the pressure to mess with the system.
Problem resolved, in a way. SuSE appears to have standardized on VNC as the prefered means to push the desktop out, even says so in the release notes. The error encountered during the FTP install was insufficient disk space -- would never have known if had not tried to restart the install. Installer appears to load the components for the entire install, then build it piece by piece. Trouble is, this takes almost 3x the final space. Recovery was to build a minimum graphical system, then install individual pieces through the YAST interface to RPM. The system retains the install source information, so this approach works very smoothly. Also interesting, the installer understands SMB, so was able to download the entire install tree for 8.2 to a server, declare a network share, mount the share from the installer -- very slick indeed. Oh, and using VNC from www.vnc.org -- viewer on windows, server on linux provides a very simple and cheap way to do 'X-windows' to the new machine.
Basic VNC display was functional but spare. Decided to install KDE desktop for a more complete graphics experience. The bulk of the install was drawn from the local installation share, so setup was flawless. After install, have the usual graphics sluggishness compared with the minimal VNC configuration, but much easier to use -- less to remember. Using YAST to update patches from the network is very easy. Issue as always is to use the YAST option to cleanup sourcecode after the install. This approach has kept overall disk utilization under 50% -- so plenty of room for MySQL and Nagios.
Changed hardware strategy -- setup server virtualization environment using VmWare. First candidates for implementation are SuSE Linux workstation, copied from physical box.
Virtualization appears to be successful, although some problems have been encountered in resurrecting the graphical environment. But as a text mode environment it is blindlingly fast -- much like the text environment on the physical box. Will continue to use linux for software development, exploring management tools, but in general, have gone over to using Microsoft Small Business Server 2003 as the general purpose working server. Dont necessarily agree with all the marketing bs -- but getting to a working server was much quicker, simpler.
Now running two types of virtualization environments -- Vmware workstation for test environments and Microsoft Virtual Server for production evaluations. Vmware is used to host Windows XP, Suse Linux, Solaris and Windows 2003 server images for product evaluations, client testing (and general fooling around). Virtual server hosts a pilot of the new Microsoft Data Protection Manager that has taken over many of the routine backup tasks, and a test instance of Internet Accelerator. A distributed file system has been established that maintains replica shares for all critical business data. Portions of the DFS tree is replicated onto laptops (as offline file copies). Samba clients are used on the *n*x instances for access to windows shares. Gigabit Ethernet became a necessity when the collection of scanned photo images (at 25mb to 700mb each) needed more space and were moved to the file server (and replicated across two machines via DFS and VSS). Conventional backups are still being done, but to disk, as a separate backup [means 3 or 4 copies of everything]. Tape continues to be used, but more deliberately -- as it will to copy everything prior to the final move to Kingston.
Virtualization has become a part of daily operations, not just testing. Over the last year, have acquired several new applications that just don't play well with each other. Moving each to an independent virtual XP box using the VMWare Server [free software] has been the best solution. Virtualized network connections do impact the latency of the connection, already substantial due to the satellite link. While the Microsoft Virtual Server was quite nice, the web console image was not as convenient to use as the VMWare tabbed console. Also, MS locked all allocated memory (but only revealed part of its usage) whereas VMW appears to page -- in any event, multiple virtual machines are real memory pigs.
Backup is now purely disk to disk with multiple automated processes -- means terabyte external disks with multiple directories. Scheduled maintenance jobs and backup jobs that run overnight means better data protection with much less operational overhead. Am thinking about establishing a backup server in another building for mirrored backups.
Testing Vista in virtual machine, probably not the ideal environment but workable. Have installed Office 2007 (which seems much happier there than on XP). Vista has some nice user interface changes -- but does not seem to understand being part of a managed network. Complains about being unable to check for updates, but accepts patches from WSUS as it should. Similar issue with Defender -- keeps insisting that I need to update it, but then complains that there are no new updates available for it from Microsoft. Strange. The file explorer interface is very slick, wish it could be retrofitted onto XP. The ribbon interface in Office is interesting, definitely a learning curve.