Multi-Tier Storage -- The Commodity Approach

I've been working on some internal documentation explaining some of our long term plans regarding storage. Initially, I imagined two documents, one an executive overview, and another a complete documentation set. Well, as things are hard to write up the first time and maintain, I decided to make one document, moving all the technical and site specific details to appendices. The end result is that I can now post the primary document sans appendices and make it public.

This is the culmination of a multi-year project to move to reliable commodity based storage and get away from nightly tape backup scenarios that do not scale with today's storage growth. So go ahead, and check out my multi-tier storage whitepaper.

Converting LDAP netgroup entries back to flat file format

I am surprised that no where on the net, someone hasn't posted how to convert back to flat file a netgroup objectclass. This is important for loading this dynamic data back into systems that are themselves relying on static files. You'll need openldap-clients or similar packages (to get ldapsearch). Also, in the below script I expect anonymous read access, and no SASL auth obviously. Finally, the "grep net" part of the netgrouplist is to only grab netgroup names with "net" in them, which is what we have standardized on.


#!/bin/bash
BASE="dc=example,dc=com"
HOST="ldap.example.com"

netgrouplist=`ldapsearch -x -b "$BASE" -h $HOST objectclass=nisnetgroup cn | grep cn: | grep net | awk '{print $2}'`

for i in $netgrouplist
do
echo "$i \\"
ldapsearch -x -b "$BASE" -h $HOST cn="$i" > /tmp/netgrp.$$
dn=`cat /tmp/netgrp.$$ | grep dn`
cat /tmp/netgrp.$$ | grep nisNetgroupTriple | awk -F' ' '{print $2}' > /tmp/netgrp-hosts.$$
lastentry=`tail -1 /tmp/netgrp-hosts.$$`
for j in `cat /tmp/netgrp-hosts.$$`
  do
    if [ $j == $lastentry ]
      then echo -e "\t $j"
    else
      echo -e "\t $j \\"
    fi
  done
rm /tmp/netgrp-hosts.$$ /tmp/netgrp.$$
done