OpenDirectory upgrade path from 10.4 to 10.5

In EE we've migrated over from various AD and OpenLDAP installations to what we hope is a more manageable solution long term. Sadly, upgrading OpenDirectory (MacOSX OpenLDAP-based directory services) from 10.4 to 10.5 doesn't work as Apple states it would. Here's the complete recipe we used to keep our data, our passwords, and most importantly, our domain SID. Apple tends to not care about maintaining the SID in various replica-to-master promotion steps.

First, a reference to the cookbook  doing things the hardway.

As recommended in the above and from other postings, upgrades do not work. Rather, what needs to be done is this:

10.4 Server:

1) go to Server Admin, OpenDirectory, and under the Archive tab, generate an archive of the OpenDirectory DB. Place in admin home directory
2) For safe keeping, go to /var/db/samba and get the secrets.tdb file. Place in admin home directory (readable by all)
3) get the current SID by running as root/sudo "net getdomainsid EE" where EE is the domain we are supporting. Place in home directory
4) copy off to a 3rd party machine the above three files/directories

10.5 Server:

1) Install fresh, and use the exact same IP and name as the 10.4 Server. You'll likely need to have these are their own net. Also note that without a link on the primary interface, smb, dns, and opendirectory don't work. I suggest connecting to the third party machine listed above, in my case my laptop's physical connection which I assign to the private net
2) You'll need DNS setup temporarily, so create a DNS server for your domain (stanford.edu) and create a host entry for your self. Point local network settings to self as DNS server
3) copy over the files saved from 10.4 from the laptop/3rd party machine
4) Make an OpenDirectory Master, using the correct domain "dc=ee,dc=stanford,dc=edu" and correct KRB realm "EE.STANFORD.EDU"
5) import the archive of 10.4
6) run as root "mkpassdb -kerberize"
7) Create a new PDC config for Windows. Use the directoryadmin account/password to give samba correct access to the OpenDirectory DB
8) edit /var/db/smb.conf to fit the /etc/smb.conf entries you had on 10.4. Likely you'll want to make "local path = " and add "admin users = directoryadmin, domainjoin, @admin" or the like, where the first is the directory admin acct, the second is a PDC join account that can't login, but has directory admin rights. @admin works to include anyone in admin group
9) run as root "chflags uchg /var/db/smb.conf" to freeze your samba config. Recommend making a copy as well in the same dir.
10) run as root "net setdomainsid (SID)" where SID is the one you saved from 10.4
11) Go into Workgroup Manager. Change preferences to enable Inspector. Go into Inspector and select "Config" and then "CIFSServer". The two Value lines with "xml version.." need to have Edit run against them, and replace the SID line in each with the SID you just used.
12) restart Samba/Windows services. Check SID with, as root, "net getdomainsid" and "net getlocalsid EE" or the like. If anything didn't stick, do 10, 11 again.
13) before going live, one needs to remove reference to the local DNS in Network preferences, and optionally disable DNS service. This setup also was only tested with Wins service enabled as the WINS Server
14) test, test, test from Windows including domain logins, enumeration of groups in windows for adding domain users, etc. Logs may show if accounts are failing.

On Windows, the simple tests you can do involve the utility "nltest" which is in the free SUPPORT TOOLS (but may not be installed by default). nltest /? gives commands although OS-X samba only supports some of them.

..to list PDC and BDCs --- nltest /dclist:your_domain

nltest /dclist:ee
Domain 'ee' is pre Windows 2000 domain. (Using NetServerEnum).
List of DCs in Domain ee
\\EE-OD (PDC)
The command completed successfully

..to verify schannel --- nltest /sc_query:your_domain
C:\>nltest /sc_query:ee
Flags: 0
Trusted DC Name \\EE-OD
Trusted DC Connection Status Status = 0 0x0 NERR_Success
The command completed successfully

To do a more detailed check, you can open the Windows Manager and try to look at the members of the Administrator group for the machine. When we had trouble, it just showed raw SID numbers, even for EE\DomAdmins. Once it was fixed, then that showed correctly.

Error cheat sheet:

1. If smb logs show that directoryadmin or domainjoin and the like have the "wrong sid" in passdb, you'll need to demote/promote Windows Servers to workgroup and back to PDC. You'll need to run "chflags nouchg /var/db/smb.conf" first and copy back your copied version after repromotion as the file will be rewritten. Do step 9-12 again above

2. If kerberos isn't effectively working on clients, you may need to reimport the archive OpenDirectory, rerun "mkpassdb -kerberize" and follow the above demote/promote steps.

Have NAS, Want Shell

Now that anyone can grab Nexenta's NAS product, many will undoubtedly want to get under the hood, especially developers. First, a fair warning that although the management infrastructure is resilient to many changes done manually, modifying various service configurations outside of Nexenta's internal version control can lead to one or two headaches if you aren't careful. That said, give me a shell!

Well, that's simple. When you login via the console (ssh, for example), simply run "setup appliance nmc edit-seettings". You can tab your way through that command as well. Once there, go and edit expert_mode to be "1". Yes, you've enter the "vi" command zone, so save and exit with ':wq'

Once the changes are saved, you'll be asked to refresh the console settings, and now you can type "!bash" to get a nice usable shell, or bang escape any command. You'll be root, so, be aware and behave! Now you know what Nexenta Core was all about, as its all there at your fingertips, along with NMS, NMC, and NMV subsystems that are the foundation of the NAS product.

update:
I was told that an alternative way to set expert mode is

option expert_mode = 1 -s

as denoted in the "option -h" documentation for NMC. The "-s" flag updates the on-disk configuration.

Developers, developers, developers...

Ever wanted that NAS on your own hardware, for free? Nexenta has finally released their NexentaStor Developer Edition 1.0, which is free version of their commercial product with only a 1TB limit on used storage. All functionality otherwise is there, unlimited. This is a near final release for the commercial version, but is the first version the general public can get and install on their own hardware.

The release represents many things, but the Developer releases are focused on more than just tire kicking or a free NAS product for your home NAS needs. Rather, there is a lot of potential to extend and use Nexenta's SA-API for storage service-enabled solutions. Wish to modify your DB to wrap a transaction in a snapshot? Need to automate separate file system creation, quotas, etc for your users? Registered users on the web site can look at an overview of the architecture and sample SA-API components. I expect much more in the way of API details in the near future. However, the release of the product is here today.

A general support forum is also available

There are two other automation aspects to NexentaStor that I haven't given much love to here. Both utilize the batch nature of NMC, the Nexenta Management Console. One is the 'query' functionality, which allows various introspections on the NAS and can query across multiple appliances at once if they are grouped together (the group function). In a similar vein, there is the NMC recording facility, which is handled by the "record" command. Recording allows you to save and play back actions for various tasks, including over a network of NAS devices. All of these commands have ready examples available by invoking the command with the "-h" help argument in the console. There is also good stuff in the User Guide which is available for download.