stacks_image_628D490E-AA39-46DD-94A2-843F662FBE03

One of Canada's leading medical supply companies, Medical Mart, provides quality medical products and reliable services aligned to each specific health care market segment (Hospitals, Long Term Care Nursing Homes, Physicians, Dental Clinics, Veterinarians, Home Healthcare & Laboratories).

Having a company which operates 24 X 7 is hard enough, but being one which also delivers critical product to patients who are in need 365 days are year is an even bigger task.

The company, which was founded over 30 years ago spans across Canada and is made up of 5 offices including there head office in Missisauga Ontario, their retail store in Missisauga, and offices in Dartmouth Nova Scotia, Windsor Ontario,
and Edmonton Alberta.

Tying these offices together is a network which spans the country provided by Advanced Knowledge Networks The team at AKN built a stable and scalable network which allows for VOIP and Data networking with built in redundancy in case of failure.

At the heart of this network lives an HPUX database which manages the warehouses management. This covers everything from the time an order is placed until the time it is delivered.

On the front end of the network users access the network via iMacs or MacBooks over wired Ethernet or wireless Airport (802.11g/n) network.

The back end of the infrastructure is made up of Apple Xserves and Open Directory, running on top of Mac OS X Server.

Recently, the servers were updated to 10.5 Server. In order to do this, we needed to export the existing Open Directory data base to ensure proper preservation of the existing passwords and MCX Policies.

Once the Open Directory data was exported, we were then ready to start the setup of the new domain architecture. The overall setup is:

  • 1 Open Directory Master
  • 1 Open Directory Relay
  • 4 Open Directory Replicas
  • 4 Open Directory Child Servers

Medical Mart-Current

To the end user, everything works the same as it did previously, the servers however have a new way of communicating amongst each other.

Since all servers in the remote office were to be setup after hours, we needed to ensure that the remote Leopard Boot DVD's were loaded in the servers. Once we were certain this was completed. we were able to reboot off the DVDs for installation after cloning the initial boot drives to external devices in case of failure and to retrieve data which exists on the existing disks. To do this we issued the following commands:

# asr -s / -t /Volumes/serverbak --erase --noprompt

After the boot systems were cloned, we then did the following:

# /usr/sbin/systemsetup -setstartupdisk /Volumes/Install\ Mac\ OS\ X\ Server/System/Library/CoreServices

# shutdown -r now


The command below allows us to find the remote server booted up from the DVD and ready for installation.

# /System/Library/ServerSetup/sa_srchr 224.0.0.1

# ssh root@remote.ip.address

We then login with the first 8 digits of the servers hardware serial number. Once logged in, we need to format the internal boot system, in this case disk1, to begin a clean installation of the Leopard Server Operating System.

# /usr/sbin/diskutil eraseDisk "Journaled HFS+" server disk1
Then we install the Operating System(S) by issuing the following commnad:

# /usr/sbin/installer -verboseR -pkg /System/Installation/Packages/OSIntall.pkg -lang English -target /Volumes/server

WIth the systems installed, we ran all software updates to get the servers patched to the latest release of Leopard Server.

Each offsite Replica server has an identical copy of the Open Directory domain which is provided to them via the Open Directory Relay server. With the base infrastructure done, it was time to get deeper into the setup by tearing down the Local KDC server on each server, including the Open Directory Master and only allowing for the main Kerberos Distribution Centre.

Since the remote office's now get their data from the new Replica and no longer point directly to the Open Directory Master, this allowed us the assurance that adequate bandwidth was sustained and that the main Open Directory Master, which holds the main KDC system was not overwhelmed with requests.

In order to do this, we needed to make sure some pre-requisites were in place. First, we needed to ensure that all existing server names were removed from Open Directory via Workgroup Manager and or alternatively via the dscl command. These were the previous names which pointed to SERVER.DOMAIN.COM as required by the Open Directory implementation.

Once the keys were removed, we needed to physically remove any and all entries for these servers from the newly imported KDC. This was easily removed with a simple shell modifier running through the kadmin.local command, provided by Bob Bosiljevac of AKN.

Once this was completed, it was time to tear down the LKDC system in order to build a network only based Kerberos Realm. This type of setup generally helps ensure that the child servers are properly bound to the Open Directory Master KDC Realm.

Since all servers, except for 4 child servers within the head office are either running in Replica or Relay mode, only a few other servers would be looking to the Open Directory Master for authentication and not acting as an Open Directory Server themselves.

In order to remove the Local KDC on all servers, we did the following based on the LKDC's UUID number:

# sso_util remove -k -a admin -p ***** -r LKDC:SHA1.1A9FC49F1E499BFD1AF68001C454977B1A8F0BD2

This removed the Local KDC and shut down the local kadmin process. Now all we have left to authenticate against is the Network KDC and Open Directory. Once we removed all the Local KDC's on the servers, we were then able to create the new Relay and Replica Servers.

The new Open Directory was now ready to go live. With the main authentication piece in place and completed, just the required services such as the slave DNS, add in the automounts to the appropriate servers, and ensure that authentication works as it did before.

With each replica server at the external offices, the users have the same basic interaction as they do at the head office. All servers provide auto-mounted home directories over Apple Filing Protocol (AFP) for login and logout, and basic general user interaction

With all the Open Directory Replicas and Relay servers setup, it was time to tackle the Open Directory Child Servers.

Once we updated the servers to 10.5, all auto-mounts, and NFS mounts needed to be re-added in order to work in the same way it did prior to the latest version.Along side the updates on the child servers, they were rebound to the new domain for good measure.

The backup scripts which run via rsync to a central storage unit were also re-implamented in order to ensure all servers are properly backed up. These backups are then rsynced weekly to an offsite co-lo facility in case of disaster recovery.

On top of the nightly and weekly backup scripts which run, the company is also implementing Tape Backup storage for longer term archiving and recovery in the form of an Exabyte T24 Fibre backup Library (LTO-4) and Tolis Groups BRU software.

With everything back in place as it was before, it was time to start testing some of the users logins and management policies that were in place previously. When we began logging in users, there were zero issues discovered. All home directories, mail, and MCX policies were back in place as they should be with no interruption to their services.

This meant that within every office across the country, users would be able to come in on monday morning and work as they expect to. While upgrades like this can be tricky, we were glad to report that out of the 100+ users across the country not a single issue arose on the first day after the upgrades and not a single call was placed to the IT support team with regards to the 10.4 to 10.5 server upgrade.