Iterative Insanity the Microsoft Way

A friend of mine likes to say that the definition of insanity is doing the same thing over and over, and expecting a different result each time.

By that definition, Microsoft is the quintessential model for insanity.

I made the mistake of taking on Windows systems administration a few years ago where I work, as I thought the challenge of learning the intricacies of a previously unfamiliar OS would give me a more rounded experience with systems administration in general and allow me to learn to appreciate why Windows admins seemed to love the environment they worked with. Coming from a solid Unix background, I figured that I would find similarities in function and implementation, to the point that course work would not be needed to learn the ropes. I was correct in that aspect and learned quickly via a couple of incidents where I was able to resolve issues that long time Windows admins could not, that having my previous systems administration experience was a boon in general and very much a keystone to ability that only required study of freely available documentation to attain.

What I was also to learn, however, was that many of the paradigms I was used to would have to be ignored and the replacements that Microsoft had implemented, just plain suck. Consistency of methodology is damn near non-existent. Graphical User Interface (GUI) tools got in the way more times than they helped and their interfaces were inconsistent from each other. To get at the real functionality of the GUI tools, half the time you have to right-click on unexpected places to pull up hidden options. Even the simple structure of system settings are inconsistent and stupidly designed.

A perfect example of the nonsense I ran into can be found in the local security settings of any Windows box. You can find sensible settings, such as, “Domain member: Require strong (Windows 2000 or later) session key” with the options of “Enabled” or “Disabled”.  Just below it, however, are “Interactive logon: Do not display last user name” and “Interactive logon: Do not require CTRL+ALT+DEL” with the same options, “Enabled” or “Disabled”. This double negative verbiage is simply ridiculous.

Or how about the fact that even though you’ve setup your Active Directory (AD) domain when you promoted the first Windows server to be a Domain Controller, with clients logging in on your domain and the forest setup with trusts to other domains, et cetera, the name of the Domain Name System (DNS) domain is still “default_domain_name” (or something like that, memory serving,) until you open up the Microsoft Management Console (MMC), run the Active Directory Sites and Services plugin and right click on the entry to chose “Rename” from the popup menu. This is in spite of the fact that you have to enter the DNS name as part of the AD promotion process. I discovered this when I refused to base our entire network’s DNS service to Microsoft’s implementation and had to copy all the SRV records in the netlogon.dns file to the Unix DNS server. After digging around for an hour or so, I finally found out what was up. Of course, if I had enabled DNS and auto-updates of DNS on the AD controller, the information would have been setup correctly then. Most Windows administrators would have simply setup DNS on the AD controller and been done with it. I’ve even read articles advising doing so, no matter what you’ve been using for DNS before, just in case something breaks later with a change suddenly instigated through an update from Microsoft!

Like I said, inconsistency reigns.

The task set before me this week was a new one. The primary Active Directory controller is old and needs to retire. New hardware was ready to go and was tested, so it was time to replace the old with the new. Yes, Microsoft claims that there is no such thing as a Primary Domain Controller (PDC) anymore, but it is only a half truth - as you still have Flexible Single Master Operations (FSMO) server roles, limiting edits of various services to specific systems. You can (limitedly) spread them out among multiple machines, but that doesn’t change the fact that the FSMO roles exist on specific systems and do not have an order of precedence to roll over to another server, should the FSMO server go down. So, even if you spread out your five FSMO roles among different machines, now you have multiple points of failure instead of one. Net gain: nothing.

In my case, our AD domain is tiny. We support about 30 Windows machines anymore, so we had two AD controllers, with the first one setup as the FSMO role server for all five rolls. (This happens automatically the first time an AD controller is setup in an AD forest.) The process to transfer the FSMO roles can be done in one of two ways: right-clicking on a bunch of clumsy GUI menus through three different MMC plugins or running the ntdsutil on the command line and suffering through what is the most abysmal modal command line interface I’ve ever seen.

ntdsutil sucks - it really, really sucks, but it was better than flopping around in three different MMC plugins. So, I started the process of transferring all five FSMO roles from the old server to the new with the command line tool. The PDC Emulator and RID Master roles transferred without a hitch. But try as I might, the Schema Master, Domain Naming Master and Infrastructure Master roles would not transfer, giving a generic error that multiple searches on Google could not elucidate.

So, I decided to make the new server seize the roles which would not transfer. This worked - to an extent. All five roles were reported by the new server to be handled by the new server, but the two old domain controllers now believed that the old FSMO server was still serving all five roles. How the two which had previously transferred correctly were now on the old machine again, was yet another mystery. At this point I didn’t want conflict, so I tried to transfer the roles back from the new machine to the old, all of which failed without even an error message to tell me something was amiss. I now had two AD controllers in the same AD domain of the same AD forest, who both thought that they were the FSMO role master for all five roles.

I left it this way over the weekend, just to see if things would work out or whether additional error messages might tell me something of what was going on. No change came. No new information was revealed.

I tried demoting the new server to stop acting as an AD controller, but it would not allow me to demote the system, giving yet another seemingly random numbered error message. That was enough for me. In desperation, I did what many Windows administrators do at times like these: start over from scratch and do that exact damn thing all over again. I powered the new machine off, re-installed Windows 2003 Enterprise Server, put on anti-virus software and updated with all patches, promoted it to an AD domain controller and kicked up ntdsutil on the old machine and transferred the FSMO roles in the same order I had during the first attempt. Everything worked perfectly. A quick check with the netdom command showed that all three machines now understood that the new server handled all FSMO roles and the whole process was done in just a few seconds time. I had done everything the second time around as I had the first, each step was in the exact order as I had written down. Nothing different was done and everything suddenly worked.

A friend of mine likes to say that the definition of insanity is doing the same thing over and over, and expecting a different result each time.

I understand now that this is why so many people cannot understand that computers are not supposed to crash or otherwise fail in stupid and unpredictable ways. They keep doing things the Microsoft way and insanity prevails and becomes the norm. They can’t understand that things should work the same way every time and that system up time can be measured in years instead of days on a stable operating system. I’m half convinced that Microsoft went to their once a month “Patch Tuesday” methodology for updates, just to make sure that all Windows machines would have to be rebooted once a month, in order to keep the systems appearing stable. I have also come to realize that many people have been fooled into believing that a boondoggled GUI is “more advanced” or otherwise “better” than editing simple text files for system settings - that somehow editing a text file is primitive in comparison - overlooking the fact that cumbersome GUI’s are often simply doing that very task.

If my varied and insane experiences over the last eight years with Windows has taught me anything, it is that whenever possible, no matter how difficult the transition may be at first - if you can run the service on Unix instead, do so. If you leave it up to Windows, you leave it up to sporadic behavior, inane tools and retarded, clumsy and often secretive GUI interfaces. You seemingly leave it up to pure chance.

To me, that is insane.