Marcel's blog

Context

For our Lab we are deploying a new service frontend (login.enterpriselab.ch) which allows elegible students, faculty members and partners with Switch AAI enabled accounts to create and manage an EnterpriseLab account. The frontend it self has been created as part of a Bachelor Diploma work and is now adapted by us to fit some changed requirements. As a backend we us ODSEE (Oracle Directory Server Enterprise Edition) 11g, the successor (or minor update) of Sun DSEE 7.

The actual stuff – LDAP schemas

To store the AAI attributes in LDAP Switch provides OpenLDAP schema files to extend the available LDAP attributes and objectclasses, but sadly the syntax which OpenLDAP uses is not supported in ODSEE. So conversions where in order. After a bit of searching around the web I found a few scripts which are supposed to do the task… some of them where better, some worse and others didn’t even work…
In the end I went with schema-convert.py by Ludovic Poitou, an ex- SUN/Oracle employee, now apparently working at ForgeRock. He wrote this script to convert OpenLDAP schemas to OpenDS schemas, which luckily are (almost) compatible with ODSEE.

The most important transformations required being:

• replacing keyword “attributetype” with “attributeTypes:”
• replacing keyword “objectClass” with “objectClasses:”
• “correct” indentation
• OID expansion

There only was one problem left after the conversion: ODSEE (in contrast to OpenDS, OpenLDAP and OpenDJ) does not support the full range of syntax definitions specified in RFC 2252 but only the ones defined as MUST, so it does not support SYNTAX 1.3.6.1.4.1.1466.115.121.1.36, probably better know as numericString which is used in the attributes “swissEduPersonMatriculationNumber” and “swissEduPersonDateOfBirth”. I decided to replace it with 1.3.6.1.4.1.1466.115.121.1.26 aka IA5String (the closest match available in this case) which allows not only numbers but also almost any ASCII characters to be stored.

Results

The results of the whole ODSEE installation and configuration procedure can be found in our wiki including some more links and references.

And here the resulting schema files:

• swissedu-1.5.ldif
• eduPerson_200806.ldif

EEEnet connectivity – update

As mentioned in my last post the hpc cluster was unable to use our mail relay and incoming proxy. Turns out our side was ready but there was a route entry in the cluster network gateway missing and  some work on the clusters sendmail configuration was required. In the end we got it to work, like most of the time

Today has been one of These days, nothing wanted to work on the first go, everything behaved differently than expected or information was just not where it was expected, but by the end of the day almost every thing was running or running again.

Services – Subversion, Redmine, WordPress, Git

A few have probably noticed performance issues with one of our services in past few days, being it the blog, redmine, git or svn – just to let you know upfront: these where all related. As many of you hopefully already know, most of our systems run Solaris and on these we use Solaris Zones (or containers if you prefer) to get dedicated environments for most of our services. So even though it might not look like it all the above services share (or better shared) the same physical host. Usually this setup works quite well for us, at least it has during the last 4-5 years but in the recent past trouble arose. It startet out with Subversion/Apache suddenly producing log files of sizes of hundreds of megabytes to a few gigabytes a day, already then it was quite clear that the new build-servers, used in class, were the main reason for this but as I couldn’t easily change the build-servers I just fixed the logging of Apache and the log rotation a bit and for a moment everything seemed fine again. Until yesterday… wow it’s already this late… so it was the day before yesterday , when some people started to notice long waiting times for one or the other of these services so I had a look again and noticed right a way that the system hosting these zones had I load average of between 30 and 40 – in my opinion in this case just way to much! So the decision was made do move some of the services to a separate host – dev on one and git, blog and pm on the other, but first I turned my attention to the build-servers. It turns out they where polling Subversion each minute for each build project, together with the students accesses that was just too much for our meager V210 system, so I changed this schedule to every 5 minutes… – here was one of the points where stuff just didn’t want to work out, but telling this story in detail too would go to far – and almost like magic the load on the system dropped and the responsiveness of the other services was also way better again. Still, to avoid SVN impacting the other services again I moved them off to another host and we’re making sure that the build-servers will start using svn post-commit-hooks instead of polling to further reduce the load.

EEEnet – the Experimental Enterprise Environment Network

A few weeks ago our Lab was one of the first to be connected the the new EEEnet, a network which in the near future should connect all the different laboratories of all the school departments with as few barriers as possible. A second “Lab” that was added around the same time, is the new HPC Cluster of the department of Mechanical Engineering. We’ve already been maintaining their “old” Cluster and there they have been using some of our services (proxy, mail server, ssh gw, etc.) so they would have liked to do the same on the new cluster. But it turns out it was not just a setting in our firewall which was needed to allow access to do so, no, it was routing troubles again and figuring out routing problems around firewalls for some of which you don’t know the settings and rule sets, I’m telling you, can be a pain in the a**. This one is still an open task and a clean solution would be much appreciated, as our network can be quite confusing as it is and definitely doesn’t need anymore quirks which make it harder to understand .

let’s see what the next week brings.

As most of you will have noticed – I’m not the biggest blogger but today I felt like I had to write again.

Every once in a while I have to try new stuff and today is such a day. For a few tests that I want to do I’ll need a complete lab environment, meaning:

• Intranets and DMZ(s)
• Master & Slave DNS Servers
• Firewalls
• a Proxy
• a LDAP Server
• NTP Server
• Hosts (for the things I actually want to test)
• and later maybe also a DHCP and a AI Server

For such a test setup I would usually setup a few virtual machines and configure them appropriately, but for various reasons not this time. For one I have to say I love VMware Fusion on my Mac but when it comes to handling multiple internal (host-only) networks there’s sadly just no easy way to manage that (yet)… it would be doable and to be frank dedicated networks are not strictly necessary for this setup anyways but I want them and I want to try something new. So I decided to give Solaris 11 Express a go and have a look at project Crossbow and the new zone features (as this stuff will come in handy for new Lab exercises anyway).

At the moment I’m planning to setup the whole environment in one Solaris 11 VM creating a dedicated zone for each service. The whole network shall be built by employing the new vnic and etherstub constructs, so it will resemble a physical setup as closely as possible.

So over the next few days I hope to be able to setup this infrastructure and keep you up-to-date with the progress, but for now that’s it.

It took some time but now our blog planet is up again too, and so is our feed on the main page.

hope you’ll enjoy it.

This was quite some work…. and the solution was quite simple in the end….

no matter where I increased log levels or enabled debug modes, WPMU simply didn’t want to tell me why it would not display any pages. Not even a fresh install onto a new server with only the database transferred could fix it.

More by chance than anything else I found a forum message which told me to disable all plug-ins for each blog on it’s own (not only globally), which meant manually modifying some database tables. After that and re-adding some themes all the blogs seam to work again…. hopefully it stays like this for a while.

And now I’m moving on to reinstall our planet … I guess that’s gonna be fun too