tech

I had been using https://example.com/exchange to login to Outlook Web Access, but starting with Exchange 2007 this added numerous steps.  I had to accept an unsigned security certificate, fill out a pop-up login box, accept another certificate (because it transferred me to https://servername.example.com/owa), and then login to the actual application.  If you're like me and looking for a shortcut, just use https://example.com/owa.  Only one sign-in.

I just finished running a benchmark on the new server's RAID 5 array.  Nothing really surprising but decent performance I think.  There are 4 500GB Seagate ST3500630AS drives in a RAID 5 using the built-in NVRAID on a Asus M2N-E motherboard.  The OS is on a separate drive.

I know, I know, this may seem painfully obvious to some people, but it is something I hadn't thought of doing before.  Say you have a domain controller on one network, and you have your laptop on another network across the WAN.  As long as your domain controller is open to the Internet (DMZ or on a routable address or something) just set your primary DNS server to the IP address of your domain's DNS server.  I suppose your domain controller has to be on a routable address then too (meaning you can directly ping it from anywhere in the world).

Now you should be able to open up any MMC tool, like AD Users & Computers, and use it to remotely administer your domain.  It would make sense that you could even set a computer's DNS entry and even join it to a domain from a remote location.

One implication of doing this is that now all your name resolution (converting google.com into an IP address, etc.) is now relying on your DNS server staying up.  Which makes me wonder how Windows uses the secondary DNS server entry.  Does it wait for the first one to time out?  How long would it wait?

Nitevision: a name which strikes fear into the hearts of many a Medora call-center workers.

To be fair, it's not that bad of a program.  It's what Medora has used for motel reservations for many years.  Written by REMco Software out of Dickinson, ND, Nitevision is a client/server application which keeps tracks of motel reservations, who is checked in, which rooms are clean, etc.  As near as I can tell the client sends raw SQL queries to the server which then spits back some data for the client to display.  A workable strategy, I think.  However, there are some problems.

I started working in Medora's call center at the beginning of the summer of 2005, as a lowly Customer Service Representative (don't be fooled, I have really really really enjoyed all of my jobs in Medora).  I vaguely remember how the Nitevision server had to be restarted quite often because all the clients on the workstations would simply lock up.  I remember more closely how this also happened in the summer of 2006 when I was a team leader, supervisor of CRSs.  At one point I was even trained in as to how to restart the server because the IT guy wanted a day off.  I don't think I ended up having to restart it, but the Internet connection did go down at one point while he was gone, which is another story on its own (credit card processing requires an Internet connection).  Anyways, the restarts were so frequent that Nitevision got its own server so the ticketing system could stay up while it was rebooting.

Enter me (again), in the summer of 2007, as the IT assistant.  Now instead of crossing my fingers in hopes that it didn't crash, I had half of the workers asking me why Nitevision crashed on them all the time.  I didn't really know, but us two IT guys spent a lot of time thinking about it.  Many hours were spent on the phone with REMco support, and they even remoted into the server to delete some rows in a logging table that looked like they were taking a lot of space.  The problem went on though, with crashes becoming a daily occurrence, and often-times hourly during busy times of the day (early morning: lots of reservations, and mid afternoon: lots of checkins).  We poured over all the diagnostics we could find: CPU usage, RAM usage, HDD activity (which is actually difficult to monitor), network activity, and the Event Viewer.

Finally I cracked open the SQL Server logs.  I should have done this sooner, but SQL Server Management Studio wasn't installed on the server and I didn't have it on my desktop.  When I got it installed on my laptop though, I found the following error message repeated tens of times in the minutes leading up to each server crash: "This SQL Server has been optimized for 8 concurrent queries. This limit has been exceeded by xx queries and performance may be adversely affected."  xx would be a 1 or 2 for about 20 minutes (always spaced evenly exactly one minute apart) and then it would jump to 20 or 30 for the last few minutes before the crash.

Shortly thereafter we discovered that the server was running the Microsoft Data Engine, better known as MSDE, also well-known for being limited to 8 concurrent queries.  We have 10 call center computers, 6 front desk computers, call accounting, online reservations, accounting staff, and 3 group sales computers fighting for database access. REMco would not really acknowledge that this was the problem, and it's quite possible that they had no experience with this scenario, because judging by a list on their website of their customers, I suspect that we are their largest.  In the end though they did decide to help us move to a trial version of full-blown SQL Server 2000.

Migration day was quite exciting.  I arrived for work at 1:00pm to discover that they had taken down the server at 10:00am to start the migration.  And it still wasn't up.  I found a number of funny things going on:

• They had backed up the databases and were then restoring them.  One backup was corrupted, and they were going to restore to the backup made during the night, losing an entire morning of new reservations.  So I taught the REMco tech how to detach and attach a database.



• They were using "SQL Editor."  I had seen this tool before on the Nitevision server.  It seems like some watered down version of Management Studio.  I suspect the tool does not have functionality for attaching and detaching databases, which may be why they weren't doing that before.  I don't think it supports Windows authentication either, because they weren't able to connect to the new database engine...and that's because...




• They installed the new engine with only Windows authentication.  Yes, the entire Nitevision program runs using SQL authentication.  SQL Editor uses SQL authentication also.  Upon pointing this out, it seemed that it wasn't merely an oversight on their part.  Rather, I think they genuinely did not know the difference between the two authentication methods.

We finally got the thing running around 3:00pm.  Since then it has only required reboots every other week as it gradually begins to more frequently freeze up for 30 seconds at a time.  End result?  Nitevision humming along acceptably, except for some annoying accessory apps running on the server that are poorly setup.  I'll elaborate on them some other time...along with several other interesting stories as I remember them.

Well here we go; I'm going to detail the unpleasant experience of setting up our new server as best as I can remember it.

Problem 1:  Floppy disk with drivers needed for RAID functionality.  We actually had a floppy drive, and even a computer to connect it to, but no floppy disks could be found.  So we drove a couple miles to someone's house and found one floppy--and old Intel motherboard driver disk.  We fired up the ancient computer there, put the disk in, put the CD-ROM from Asus in the optical drive...and got stuck.  It wouldn't read the disk.  Closer examination revealed that it was actually a DVD disk, which the 5+ year old computer couldn't read.  We took the floppy home and made the disk there.

Problem 2:  Getting the computer to boot correctly.  Having not dealt with a floppy drive for several years, we were both unfamiliar with the cause of those cryptic "failure to find boot disk" messages, which were very vexing.  We initially blamed it on the RAID and how that fit into the boot order.

Problem 3:  Not having disk 2.  Server 2003 64-bit comes on two CDs.  We had 2 MSDN-iso burned disks, one labeled disk 1 and one labeled disk 2.  The second one was most certainly not disk 2.  Off to MSDN to download...and in the meantime we went ahead and installed updates and Service Pack 2.

Problem 4:  We ran disk 2, only to get a warning that Service Pack 2 had already been installed.  We proceeded on anyways.  Around this time we started getting random lockups.  Then a message popped up detailing that the RAID had entered a degraded state.  After messing around in the RAID software for a couple minutes, we decided that one of the drives was bad, and that we would have to reinstall on a RAID composed of the three remaining disks.

Problem 5:  Windows installed again, everything updated, RAID fails again.  So this time I backed up an image of everything we had setup to another computer, re-installed Windows on one of the SATA HDDs (not in a RAID) and restored from the backup.  This seemed to work alright, until we started to have a LOT of problems installing Exchange Server 2007.

So we re-installed again (fourth time if you're counting).  By now I figured that something was up and these disks weren't actually failing.  But we were also sick of the RAID idea so just installed Windows on a spare IDE HDD we had laying around.  In the meantime, we figured out that the disks probably hadn't been given adequate time to rebuild (although I'm still not sure why a new RAID with empty disks needs to be built).

This is the install we are currently running on, and it's working quite well.  After the RAID was given time to build (I went into the BIOS RAID control panel and told it to rebuild) it has been running fine.  We had quite a lot of trouble again with Exchange Server 2007, but that is another story altogether...

Yep, we finally did it.  After inching along for two years on 1.0 GHz servers (one or two or three as we saw fit at the time) we have finally invested in some big iron.  $800 got us the following:

• 2x 1GB DDR2 667MHz RAM
• 4x 500GB Seagate 7200.10 HDDs
• 400-watt Rosewill PSU
• 4U Rackmount Case
• 2.2GHz AMD Athlon 64 X2 ProcessorAsus M2N-E Motherboard

I managed to get the HDDs for $100 through some sort of error on Tigerdirect's website I think.  They had ones with 16MB cache listed there for $120 (and the same price was at Newegg), but 8MB ones for $100.  I didn't really care that much about that so I ordered them.  Then I read online that those drives aren't available with 16MB cache so I clicked on the link in the shipping confirmation e-mail they had given me and that lead me to a non-existant page.  Then 16MB cache drives showed up at the door, which made me happy.  But the HDDs still made up 1/2 of the total price.

Buying the RAM second-hand from my boss in Medora also helped stay under budget as I got it for $60.  The power supply was pretty cheap ($35) and I have some doubts about it (it's not very heavy) but it had lots of good reviews on Newegg.

I got that motherboard because it's an Asus and it has 6 SATA ports with RAID 5 support.  So the HDDs are in a RAID 5 with an end total capacity in Windows of 1.36TB.  Unfortunately due to some initial issues installing Windows I ended up not putting the OS in the 60GB partition on the RAID that I had planned.  I threw in an old 160GB drive for that, and we are now using the RAID solely for file storage.

I also threw in one out-of-budget expenditure: an 8-port gigabit switch which I picked up for ~$60.

Tomorrow I will detail all the headaches that we had installing everything.

Today I did some server maintenance.  That would be the reason you noticed the blog was down for 45 minutes or so.  (Who am I kidding, like anyone actually visits.)  The main idea was to add some RAM, but I also removed a HDD from one machine.

I have two servers, one of which runs the web stuff (all ASP.NET) and one which stores backups and other "files."  It also runs runs SQL Server.  This file/database server used to get really hot, since it was stuffed with 5 HDDs at one point.  I have since added a fan to blow across most of the HDDs and things have improved a lot.  It runs on a 1.2GHz Athlon and now has 384MB RAM.

The web server (and domain controller, and mail server, and anything else I throw at it I guess) is a 1.1GHz Celeron with 640MB RAM.  When I opened the case up on this beast to add some RAM, I noticed that this was now the hotter of the two computers.  In fact, nearly everything was so hot I could hardly hold my hands tight up against the parts (chassis, power supply, HDD, you name it).  This computer used to run fairly cool, so something was obviously up.

I added the RAM, fired the box back up, and promptly noticed that the power supply fan wasn't spinning.  There is no other fan in there other than the CPU fan (still chugging along), so there was little-to-none airflow through the case.  Amazingly it had been stuggling along for probably a month or two like this, in a very warm closet even!

I thought maybe the fan was just jammed and tried rotating it with a key, but it is very sluggish.  I imagine the bearing dried out and failed.  So now I have the computer up and running again, but this time with the case cover off.  I must say though, what a tough little machine.

The moral of the story: If the blog goes down again, you know what happened...

Well the title pretty much sums it up.  I just discovered that WMP11 can rip tracks from 2 CDs at once.  This pretty much maxes out an AMD X2 3800+, which just one CD didn't.  Curious, because iTunes will max out the processor on just one CD.  Oh well, maybe it's because iTunes was going to 128kbps MP3 and WMP is going to WMA Lossless.

In other news, I've been learning a lot about PCL-5, PCL-6, PCL-X, and lots of other printing goodies.  As with most occasions in which I learn a lot about computers, this means something is broken.  This time it's my damn HP Laserjet 1012 and a lack of reliable functionality in Vista.  I don't feel so bad though, because lots of people have the same problem with XP too, which is even "officially supported."  What a POS, and HP, what a POS for not fixing this problem (printer is easily thwarted by complicated PCL commands from what I could tell).

I'll admit it, I have some DivX files on my computer.  Anyways, I use WMP11 to view them, courtesy of the DivX codec.  This codec is external to WMP11, and comes with its own little nifty tray icon (quite annoying if you ask me).

So I was watching WMP11 generate thumbnails for the movies, and it was taking quite a while.  Further inspection revealed the DivX tray icon flashing in and out of existence every few seconds.  It seems that WMP11 starts and stops the codec for every movie that it processes.  Pretty dumb.

Oh Vista, what were you thinking?  When you wanted to restart after applying updates, I carefully asked you to remind me in 4 hours to restart.

Why did you restart on your own after 4 hours?  Did it ever occur to you that if I leave you doing something overnight I **might** want you to finish doing that without a reboot in the middle that stops everything?  Thanks a lot.