Wednesday, July 25, 2007

Scratch - Shell Script Part 1

#!/bin/sh
set -e

# Version 0.6c, 2007-06-21
# - Updated 2007-06-21 by Eric Lannert (eric.lannert@gmail.com)
# - added openldap, openssl, and bdb
# - changed flow to package by package - I found the need to run in pieces and wanted
# to make it easier for future add-on's
# - added switch to choose whether to delete source directory as this prevents incremental installs
# and recovering from errors
# - NOTE - I do not know where oracle's ftp is for the bdb, so the script assumes you
# have downloaded via http and uploaded by hand
# - abstracted wget_source to a reusable function
# - added LD_LIBRARY_PATH, CPPFLAGS, and LDFLAGS to chk_prepare
# - Updated 2007-06-08 by Chris Shymanik (chris@chipsncheese.com)
# - Will backup and re-install any old php.ini's found (see notes in wiki about this).
# - Source file detection.
# - Allows cgi-bin to be installed in a sub-directory (EXTDIR)
# - Created steps as functions, for easier troubleshooting/modification.
# - Updated various packages and did some additional code cleanup.
# - OSSP mm (Shared Memory Allocation) and bzip2 packages added.
# - Changed which binary to use, based on the new location in PHP 5.2.3.
# - Updated 2007-01-15 by Charles Wiltgen (charles@wiltgen.net)
# - Make "nicer" to help keep it from getting killed by DreamHost
# - Make less verbose to keep signal-to-noise level high
# - Updated 2006-12-25 by Carl McDade (hiveminds.co.uk)
# - Allow memory limit and freetype

#### User Configuration Options
## Domain & Directory Configuration
# Domain to install to
export DOMAIN="events.icstars.org"
# Temporary source directory
SRCDIR=${HOME}/source
# Download temporary DIST files to which directory?
DISTDIR=${HOME}/dist
# Delete contents of DISTDIR after installation? (Default: No)
DISTDEL="No"
# Delete contents of SRCDIR after installation? (Default: No)
SRCDEL="No"
# Backup your previous php.ini files to which directory (if any)?
# Please Note: You'll need to remove this directory manually if it's used.
# (!!You MUST use a trailing slash when specifying this directive!!)
BACKUPDIR=${HOME}/php5tmp/
# Install PHP5 to which directory?
INSTALLDIR=${HOME}/php5
# Install PHP5's cgi-bin files to a directory besides home?
# ie.: /home/username/mywebsite.com/sub/level/directory
EXTYESNO="No"
# Directory under home in which the cgi-bin files will be installed to
# ie.: /sub/level/directory
EXTDIR=main
# Nice Level for Processes. (Deprecated)
# Higher is nicer, lower is less nice and could get your install process killed!
NICE=19
# Wget options
WGETOPT="-t1 -T10 -w5 -q -c"

####Setup environment
chk_prepare
chkproc_paths

####Download and install each package
#LIBICONV
FOLDER="libiconv-1.11"
FILENAME=${FOLDER}.tar.gz
MIRROR1="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/distfiles/${FILENAME}"
MIRROR2="http://mirrors.usc.edu/pub/gnu/libiconv/${FILENAME}"
CONFIGURESWITCHES="--enable-extra-encodings --prefix=${INSTALLDIR}"
wget_source
cd ${SRCDIR}
echo "Extracting ${FILENAME}..."
tar xzf ${DISTDIR}/${FILENAME} > /dev/null
cd ${SRCDIR}/${FOLDER}
echo "Configuring ${FOLDER}..."
./configure ${CONFIGURESWITCHES}
echo "Making ${FOLDER}..."
nice -n ${NICE} make
echo "Installing ${FOLDER}..."
nice -n ${NICE} make install
echo "Done with ${FOLDER}!"

#LIBMCRYPT
FOLDER="libmcrypt-2.5.8"
FILENAME=${FOLDER}.tar.gz
MIRROR1="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/distfiles/${FILENAME}"
MIRROR2="http://umn.dl.sourceforge.net/sourceforge/mcrypt/${FILENAME}"
CONFIGURESWITCHES="--enable-extra-encodings --prefix=${INSTALLDIR}"
wget_source
cd ${SRCDIR}
echo "Extracting ${FILENAME}..."
tar xzf ${DISTDIR}/${FILENAME} > /dev/null
cd ${SRCDIR}/${FOLDER}
echo "Configuring ${FOLDER}..."
./configure ${CONFIGURESWITCHES}
echo "Making ${FOLDER}..."
nice -n ${NICE} make
echo "Installing ${FOLDER}..."
nice -n ${NICE} make install
echo "Done with ${FOLDER}!"

#LIBXML2
FOLDER="libxml2-2.6.28"
FILENAME=${FOLDER}.tar.gz
MIRROR1="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/distfiles/${FILENAME}"
MIRROR2="ftp://xmlsoft.org/libxml2/${FILENAME}"
CONFIGURESWITCHES="--enable-extra-encodings --prefix=${INSTALLDIR}"
wget_source
cd ${SRCDIR}
echo "Extracting ${FILENAME}..."
tar xzf ${DISTDIR}/${FILENAME} > /dev/null
cd ${SRCDIR}/${FOLDER}
echo "Configuring ${FOLDER}..."
./configure ${CONFIGURESWITCHES}
echo "Making ${FOLDER}..."
nice -n ${NICE} make
echo "Installing ${FOLDER}..."
nice -n ${NICE} make install
echo "Done with ${FOLDER}!"

#LIBXSLT
FOLDER="libxslt-1.1.20"
FILENAME=${FOLDER}.tar.gz
MIRROR1="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/distfiles/${FILENAME}"
MIRROR2="ftp://xmlsoft.org/libxml2/${FILENAME}"
CONFIGURESWITCHES=" \
--prefix=${INSTALLDIR} \
--with-libxml-prefix=${INSTALLDIR} \
--with-libxml-include-prefix=${INSTALLDIR}/include/ \
--with-libxml-libs-prefix=${INSTALLDIR}/lib/"
wget_source
cd ${SRCDIR}
echo "Extracting ${FILENAME}..."
tar xzf ${DISTDIR}/${FILENAME} > /dev/null
cd ${SRCDIR}/${FOLDER}
echo "Configuring ${FOLDER}..."
./configure ${CONFIGURESWITCHES}
echo "Making ${FOLDER}..."
nice -n ${NICE} make
echo "Installing ${FOLDER}..."
nice -n ${NICE} make install
echo "Done with ${FOLDER}!"

#MHASH
FOLDER="mhash-0.9.9"
FILENAME=${FOLDER}.tar.gz
MIRROR1="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/distfiles/${FILENAME}"
MIRROR2="http://umn.dl.sourceforge.net/sourceforge/mhash/${FILENAME}"
CONFIGURESWITCHES="\
--prefix=${INSTALLDIR}"
wget_source
cd ${SRCDIR}
echo "Extracting ${FILENAME}..."
tar xzf ${DISTDIR}/${FILENAME} > /dev/null
cd ${SRCDIR}/${FOLDER}
echo "Configuring ${FOLDER}..."
./configure ${CONFIGURESWITCHES}
echo "Making ${FOLDER}..."
nice -n ${NICE} make
echo "Installing ${FOLDER}..."
nice -n ${NICE} make install
echo "Done with ${FOLDER}!"

#ZLIB
FOLDER="zlib-1.2.3"
FILENAME=${FOLDER}.tar.bz2
MIRROR1="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/distfiles/${FILENAME}"
MIRROR2="http://www.zlib.net/${FILENAME}"
CONFIGURESWITCHES="\
--shared --prefix=${INSTALLDIR}"
wget_source
cd ${SRCDIR}
echo "Extracting ${FILENAME}..."
tar xzf ${DISTDIR}/${FILENAME} > /dev/null
cd ${SRCDIR}/${FOLDER}
echo "Configuring ${FOLDER}..."
./configure ${CONFIGURESWITCHES}
echo "Making ${FOLDER}..."
nice -n ${NICE} make
echo "Installing ${FOLDER}..."
nice -n ${NICE} make install
echo "Done with ${FOLDER}!"

###########################################
# SUPPORT FUNCTIONS

###########################################
## Function to check for an existing install and kill if exists.
function check_prepare()
{
# Push the install dir's bin directory into the path
export PATH=${INSTALLDIR}/bin:$PATH

# set the install dir's lib and include directories for LD_LIBRARY_PATH, CPPFLAGS and LDFLAGS
export LD_LIBRARY_PATH=${INSTALLDIR}/lib
export CPPFLAGS=-I${INSTALLDIR}/include
export LDFLAGS=-L${INSTALLDIR}/lib

# Indicate the time at which the install started.
echo "Installation commencing..." `date +%r`
echo ""

# Backup previous php.ini file(s) if exist.
if [ -d ${INSTALLDIR}/etc/php5 ]; then
echo "--- Backing up php.ini file..."
mkdir -p ${BACKUPDIR}
cp -R ${INSTALLDIR}/etc/php5/* ${BACKUPDIR}/
echo "Done."
else
echo "No php.ini will be backed-up during this update!"
fi
# Check for existing php5 install, kill any running processes from it, and remove.
# Note: This is only for fast-cgi processes. This should probably be modified
# to detect other custom php5 processes for a fully functional script.
if [ -d ${INSTALLDIR} ]
then
# Check for a running process
# Note: This is only a very basic detect & kill system.
if [ "$(ps aux | grep -q php5.fcgi)" == "php5.fcgi" ]
then
while [ "$(ps aux | grep -q php5.fcgi)" == "php5.fcgi" ]
do
kill -9 php5.fcgi
sleep 5s
done
fi
echo "Install directory exists! Killing active processes and clearing it..."
rm -rf ${INSTALLDIR}
else
echo "Install directory check complete."
fi
# Remove the existing cgi-bin directory.
if [ -d ${HOME}/${DOMAIN}/cgi-bin ]; then
echo "cgi-bin directory exists! Clearing contents..."
rm -rf ${HOME}/${DOMAIN}/cgi-bin
else
echo "cgi-bin directory check complete."
fi
}

###########################################
## Function to check for and setup the appropriate paths.
function chkproc_paths()
{
# Detect how many processors the system has (for more optimal compliation).
cores=2 # the number of cores/procs to use when building
if [ $cores -a $cores -gt 1 ]; then
j="-j$cores "
fi
OS=`uname -s`
if [ "Darwin" = $OS ]; then
sed=gnused
makefile=makefile.macosx
else
makefile=makefile.linux_x86_ppc_alpha
sed=sed
fi
for i in $sed wget; do
$i --version >/dev/null 2>&1
done

# Clear and/or create the source directory.
if [ -d ${SRCDIR} ]; then
echo ""; echo "Source directory already exists!"; echo "Clean it?"
if [ ${SRCDEL} == "Yes" ]
then
echo ""; echo "Yes!"; echo "Cleaning now..."; echo ""
rm -rf $SRCDIR/*
else
echo ""; echo "No!"; echo "Leaving the source directory intact."; echo ""
fi
else
echo "Creating source directory..."
mkdir -p ${SRCDIR}
fi
# Create the dist files directory if it doesn't exist
# optionally cleaning it if it does exist already.
if [ -d ${DISTDIR} ]; then
echo ""; echo "Distribution directory already exists!"; echo "Clean it?"
if [ ${DISTDEL} == "Yes" ]
then
echo ""; echo "Yes!"; echo "Cleaning now..."; echo ""
rm -rf $DISTDIR/*
else
echo ""; echo "No!"; echo "Leaving the distribution directory intact."; echo ""
fi
else
echo "Creating distribution directory..."
mkdir -p ${DISTDIR}
fi
}
###########################################
## Function to wget the current package
function wget_source()
{
echo "*************************"
if [ -a ${DISTDIR}/${FILENAME} ]; then
echo "Skipping wget of ${FILENAME}"
else
echo "Getting MIRROR1: ${MIRROR1}"
wget $WGETOPT $MIRROR1
# If primary mirror fails, use the alternative mirror.
if [ -a ${DISTDIR}/${FILENAME}]; then
echo "Got ${FILENAME}"
else
echo "Failed: Getting from MIRROR2: ${MIRROR2}"
wget $WGETOPT $MIRROR2
# Check to make sure the alternative mirror worked.
if [ -a ${DISTDIR}/${FILENAME} ]; then
echo "Got ${FILENAME}"
else
echo "Failed to get ${FILENAME}. Aborting install!"
exit 0
fi
fi
fi
}

Monday, July 16, 2007

Active Directory User Account Provisioning

Since the Drupal ldap auth module does not support new account creation in Active Directory, I will need other options.

Microsoft Identity Lifecycle Manager 2007 formerly Microsoft Identity Integration Server 2003
http://www.microsoft.com/technet/technetmag/issues/2007/05/Workflow/

Drupal LDAP Integration - Extracting groups from W2K Domain Controller Active Directory

The current setup is not extracting group information for users.

Tried this, but it did not work. Also seems counterintuitive to put values in boxes where the checkbox was left unchecked.
http://drupal.org/node/80020


http://drupal.org/node/147824 lead me to this:
msg_r($groups);
Which shows my groups are coming back from AD, but not propagating into drupal groups.

http://drupal.org/node/136303
Explains that the function in ldapgroups.conf.php needed to be commented out. Otherwise, the only groups that would get propagated were hard-coded ones it found.

That worked!!!

Ok, in the ldap groups screen, i only needed to do three things. everything else is blank:

Groups are specified by LDAP attributes - Checked - yes
Attribute names (one per line): memberOf

Attribute holding group members: memberUid

Unable to connect to Active Directory via LDP on port 636

I am able to connect with ldp via port 389. I have run netstat/netdiag/and dcdiag with no issues.

However, when I try to connect using LDP via 636 and using an IP address as the server, I receive the following error:
ld = ldap_open("M.Y.I.P", 636);
Error <0x51>: Fail to connect to M.Y.I.P.

This says that is by design
http://support.microsoft.com/kb/814662

This indicates the problem is likely name resolution:
http://forum.java.sun.com/thread.jspa?threadID=645000&messageID=3824603

That works from the server using windowsdomain.domain.org. Still cannot connect via remote client.

This link:
http://www.pgina.org/?page_id=6
indicates that if you can connect via 389 but not 636 then you have a certificate problem.

The server event logs showed:
Event Type: Error
Event Source: Schannel
Event Category: None
Event ID: 36869
Date: 7/16/2007
Time: 7:31:05 PM
User: N/A
Computer: SERVER
Description:
The SSL client credential's certificate does not have a private key information property attached to it. This most often occurs when a certificate is backed up incorrectly and then later restored. This message can also indicate a certificate enrollment failure.

I reinstalled the certificate from server/certsrv and still have the same unable to connect, but now i no longer get any event log messages.

THis post
http://www.eggheadcafe.com/aspnet_answers/windowsserveractive_directory/May2006/post26947443.asp

Suggests looking in the client machine event log, duh...
Event Type: Error
Event Source: Schannel
Event Category: None
Event ID: 36884
Date: 7/16/2007
Time: 8:03:44 PM
User: N/A
Computer: ME
Description:
The certificate received from the remote server does not contain the expected name. It is therefore not possible to determine whether we are connecting to the correct server. The server name we were expecting is windowsdomain.domain.org. The SSL connection request has failed. The attached data contains the server certificate.

This tells you how to enable events in schannel:
http://support.microsoft.com/?id=260729

However, none of this explains how to fix the actual problem. How do you verify/change the name on the certificate that the server issues? it seems that ldp wants it to be the same fqdn that you use to address the server. however, that cannot be how mine is named, as i get the same error message when i tell the remote ldp client to use windowsdomain.domain.org.

Got it - from a remote machine, you have to use servername.windowsdomain.domain.org aka fqdn - dimwit.

This is ugly, as that is not a dns A record i want floating around out there. How to change the msft certificate services to issue a friendlier name, e.g. ldap.domain.org....?

This says you cannot change the name:
http://articles.techrepublic.com.com/5100-6345_11-5643908.html

Web Huddle

wow.

http://groups.drupal.org/files/webhuddle_0.pdf

How to install SSL cert on Dreamhost for use in Drupal ldapauth against Active Directory

THIS POST IS A DEAD END - It contains links and attempts at getting Drupal's ldap integration module to work against a W2K domain controller for active directory. It appears W2K does not support TLS and therefore, I need to create a separate post for how to get the drupal module to use ldaps instead of StartTLS.

Following up on the prior sequence of posts, this will document the learning/attempt path to get my cert installed on my dreamhost account that will enable drupal ldapauth to use secure communication with our Active Directory server.

This shows how to use openSSL to generate a self-signed certficate. Not what I need right now, but if I end up having to abandon using the MSFT self-signed, I'll come back to this.
http://www.neilstuff.com/apache/apache2-ssl-windows.htm

This looks promising... googled "ldapauth client SSL certificate active directory"
http://www.muquit.com/muquit/software/mod_auth_ldap/ssl_tls.html:

Ok, that got me through exporting the cer file from Microsofts Certificate Authority. FTP's the file up to my dreamhost account, and ran the steps to create a .pem file. On to getting ldapauth to see and use that file...

Not sure where to set the path for the pem file in the ldapauth module setup. Within ldapinterface.php there is a secretKey = null attribute that goes along with tls setting on the next line. However, it may not go here at all and maybe part of the openldap or openssl configuration.

Here's the current error message when I check "Start TLS" in the ldap integration settings in drupal administration:
warning: ldap_start_tls() [function.ldap-start-tls]: Unable to start TLS: Decoding error in /home/snip/modules/ldap_integration/ldap_integration/LDAPInterface.php on line 128.

My php install directory is php5, which contains an etc folder which contains an openldap folder which contains an ldap.conf file which is where I believe the cert reference needs to exist. Err, maybe not. The script above indicates i need to create an ldaprc file and reference the pem file there.
http://edoceo.com/liber/network-openldap.php
The docs are not consistent on what the ldaprc file switch contains.
muquit says:
TLS_CACERT /usr/local/certs/cacert.pem
TLS_REQCERT allow

Which I changed to:
TLS_CACERT $HOME/.ssl/[domain].pem
TLS_REQCERT allow


Apache needs to know where to find the ldaprc... snipped from muquit:
Before starting apache set a env variable (probably in apachectl) like: LDAPCONF=/path_of/ldaprc
export LDAPCONF


How does that work on Dreamhost??
Do i put this in .htaccess?

The very last post in this thread indicates that TLS is NOT supported by Windows 2000 domain controllers!!!! If that's the case, where do i go from here? How do I get ldapauth module to use ssl instead of tls?
http://forums.gentoo.org/viewtopic-t-295330-highlight-modauthldap+ssl.html
http://forum.java.sun.com/thread.jspa?threadID=592611&tstart=240
And the final word??
http://support.microsoft.com/kb/321051
"Windows 2000 does not support the Start TLS extended-request functionality"

So do I need to hack the ldapauth module in drupal to use ldaps? will this automatically use ssl and the local cert instead of startTLS?
http://drupal.org/node/75645

Additional References
http://www.openldap.org/lists/openldap-software/200403/msg00034.html
http://www.thetipspool.com/freenode/ldap/27Apr2007/3

This one has a good thread about getting apache to see the ldap cert file.
http://forums.gentoo.org/viewtopic-t-295330-highlight-modauthldap+ssl.html