Preview: Erik Rose’s Plone 3 for Education

I just received a copy of Plone 3 for Education from Packt. I’m planning a review once I’ve put it through its paces, but in the meantime I wanted to send along my first impressions.

The general focus of the book is on course creation, but it appears to cover a lot of the “eventualies” of using Plone in an educational environment (“eventually, you’ll need a staff directory”… “eventually you’ll want to embed media”). Very promising.

The book is on the thin side. From skimming through, it doesn’t seem to be a problem though. The sections are arranged in a fairly logical order, content creation up front and skinning, then deployment at the end.

The book is comprised of (relatively) short, cookbook-style, “if you want to do this, here’s how” sections. Right to the point. This is very promising. It should make the book a good reference as well as a tutorial.

I an very interested in getting into the technical aspects of the book, but I’m especially anxious to see how well it works from an even less-than-anticipated level of technical ability. The book’s target audience includes “makers of school websites… tech-savvy teachers… and seasoned software developers” (from the back of the book), and “web masters… and enterprising faculties” (from the preface), but it shows real promise as being suitable as a next-step for people who already have some basic plone skills and want an easy-to-follow guide for doing something practical. Especially for those of us in an educational environment.

I’ve got my fingers crossed :)

I’ll post a full review soon. Erik has a good reputation in the community, and a knack for explaining things well, so I’ve got high hopes.

There’s a sample chapter available on calendaring, if you’d like to read more of the book than I have at this point ;)

One side note: the cover of the book has a yellow rose on it. Classy. I guess it’s there because of Erik’s last name. I wish Packt would take a cue from O’Reilly and explain the covers of their books in a colophon, but it’s a fair assumption.

This is cool, but one hopes we’re looking at a special case and not a standard practice.

The mind wonders to strange places when contemplating what they’d put on a book authored by someone with a last name like, say, Johnson. :P

JOB: Python/Plone Developer

One of the python/plone developer positions I posted about before is still open.

It’s a mid-tier, mid-level position, developing Plone/Python-based applications for a research center at the University of North Carolina School of Medicine. Specifically, the job is for the Lineberger Comprehensive Cancer Center. It’s located in Chapel Hill, North Carolina.

This means it’s a state job and has great benefits. Being a mid-tier, mid-level position means that there is a lot of room for advancement over time.

My original post explains the job in detail.

It’s essentially building applications to meet research needs in a team environment. Data management, bioinformatics, image analysis and more. The goals are short, productive development cycles, innovation, and ultimately publication of our work.

See the job posting for application details.

If you have any questions, I can answer them off the record in a purely unofficial way via e-mail at josh underscore johnson at unc dot edu.

Purely Python: LSB-Compliant Init Scripts

I’ve recently been writing a large number of Linux Sys-V style init scripts, to control how services are started upon system boot (or terminated upon shutdown).

I’ve chosen Python over shell script as my language of choice. I’ve dug into the tools, conventions and standards behind init scripts, and I’m documenting what I’ve found, and what I’ve done.

If you’re not interested in the background details, you can check out my google code repository for this article, or jump right to my init script skeleton, or a real-world example.

JOB: Looking for Two Programmers

NOTE: All of the information in this post is to be considered unofficial and informal in nature. Be sure to read the actual postings carefully and apply through the proper channels.

I’m building a Python-centric, Plone-adjacent, web application development team. To that end, I’ve been given two mid-tier Applications Analyst positions to fill. These have been opened, posted, and we are actively taking applications.

This is a “ground up” development team; we have some pre-existing Plone infrastructure and applications we want to integrate, but there’s opportunity for development with different web frameworks if they will meet our requirements more completely than Plone can (but we will still be running Plone for general web sites and will want to try to integrate as much as we can).

The infrastructure is being established right now (by Yours Truly), and is being designed to be highly available and low maintenance. It will be under the full control of the team.

My focus is on finding people who really enjoy application development, have strong communications and organizational skills, like working in a highly productive team environment, and don’t mind getting their hands dirty when it comes to hardware.

Another primary function of the team will be to produce publications. It will require us to be especially diligent in documenting everything we do. It also allows us to choose solutions with innovation in mind. This is a great opportunity to get very public recognition for your work, and forge new territory.

These positions are for the University of North Carolina at Chapel Hill, and will be administered through the Lineberger Comprehensive Cancer Center.

You can view the postings and apply for the positions on the UNC HR website:

https://s4.its.unc.edu/RAMS4/details.do?reqId=0911083&type=S
https://s4.its.unc.edu/RAMS4/details.do?reqId=0911084&type=S

If you’d like to apply for either of these positions, please use the “Apply for this position” link at the bottom of the page.

I’d be happy to answer any questions (in an unofficial manner) via e-mail at josh underscore johnson at unc dot edu.

Read on for more info about the jobs, working at UNC, and living in this part of North Carolina :)

About the jobs

These are permanent, full-time, mid-level positions. The primary focus will be on application development and publication, with secondary emphasis on user support/training, maintaining and scaling the hardware infrastructure, and keeping on top of innovations in software development, hardware management concepts, and data management techniques.

We’ll be building applications to assist with a few different facets of research, that, to over simplify, represent some form of pathology. This means dealing with the marshaling, processing, and analyzing of data collected from specimens of varying origin.

It’s essentially bioinformatics work, but there will be some specialization in areas like in silico experimentation (developing experiments that occur entirely within a computer system), image analysis, inventory management, financial tracking (purchasing), business process automation, animal colony management, and systems integration (much of the equipment used has some sort of API that can be tapped into).

For project management, we will be imploring a variant of the SCRUM development methodology. I want to see the team actively cultivate a highly productive methodology that works best for us (as opposed to just adopting something whole-hog).

The goal is short, productive development cycles that focus on functionality.

We will be forging new ground with much of our work, so research and preliminary development practices (prototyping, testing, evaluation) will be essential.

As I mentioned earlier, publication is a primary focus. Everything that we do is novel given the current state of tools used for this type of research. We want to be able to collaborate with other departments, other universities, and generate publications that showcase what we’re doing to the entire world.

Documentation is always important, but in the case of this team, we must be especially careful to explain and document what we do in not only a highly technical way (e.g. inline documentation, API references, user manuals), but also in a less technical, more narrative way. Something that can be adapted easily to various scientific and industry journal formats, and conveyed to investigators and department personnel that may not have a firm understanding of the technology we’re using.

The hardware infrastructure is virtual machine based, with a large AoE SAN. We will not have a dedicated systems administrator, so we must participate in the ongoing maintenance and development of the hardware infrastructure as a group. The work plan for the positions require 20% of each analyst’s time to be available for hardware tasks. My hope is that the actual amount of needed time will be much less, and I am designing the infrastructure so that it will require as little maintenance as possible. There’s a focus on high availability, so we don’t have to address problems right away.

On a work-culture level, the work environment will be very nurturing to a highly productive team. We will have our own work space free from most distractions (but not far from the labs we support or our hosting facilities), flexible work hours and location. Retreats, field trips, and other “lets get out of the office and think” activities are all part of our job.

We’re located right in the middle of the UNC medical campus, not far from many food options, cafes, and beautiful landscapes with free wi-fi :)

We’ll have a faithful simulation of our production environment for testing and development, will employ a VCS system (Mercurial at present) and some sort of issue tracking.

We’ll participate in community efforts (conferences, sprints, local user group activities) as much as possible.

Working at UNC

UNC is a state university. UNC itself provides many great benefits to its employees, and the state provides even more. We get generous leave accruals, retirement fund (pension as well as 401k), very good medical benefits, longevity pay, early retirement options, flexible spending account, tuition reimbursement, employee discounts and more.

More information (and real details) can be found on the UNC HR website http://hr.unc.edu

I’ve worked for the university for 6 years and it’s been great.

Living in the Triangle

If you’re not from around here, UNC-CH is located in what’s known as the “Triangle” of North Carolina, which consists of three major metropolitan areas, Durham, Raleigh, and Chapel Hill. It’s a great place to live and work.

There’s a wide range of places to live depending on your taste; everything from highly urban downtown areas to country farmland to suburban neighborhoods. Chapel Hill itself is sort of a combination of the three. Nearly any town in the Triangle area is about a 30 minute drive from UNC-CH.

Beyond UNC, the Triangle is host to a lot of technology companies (IBM, SAS, Red Hat), as well as a few other major universities (Duke, NCSU), and biotech companies (GlaxoSmithKline, RTI). (see http://www.gonetocarolina.net/employers for more)

Chapel Hill itself, where I live now, is great. It’s a somewhat small town, but because of the university it’s been built up quite substantially (yet it’s just minutes from the country). There are many, many trees :)

We have a free public transit system, and a wide variety of things to do, both cultural and culinary in nature. And if Chapel Hill doesn’t have it, chances are Durham or Raleigh will.

The Triangle is also the home of one of the best user groups ever, the Triangle Zope/Python User Group, better known as TriZPUG. It provides a multitude of opportunities to network with other Pythonistas doing very similar types of work.

We’ve also got the TriLUG, the Triangle Linux User Group, which has also been a great networking resource.

This part of North Carolina is sort of moderate when it comes to weather. Prior to moving here I lived in Florida and New England, and I feel like the Triangle is sort of right in the middle between the two. We’ve got oceans and mountains about three hours in either direction, it gets warm in the summer and cool in the winter but never too hot or too cold for too long. We’re far enough from the coast to avoid most hurricanes or nor-easters (although we are susceptible to gulf coast weather), far enough away from the mountains to avoid most flash flooding; we get some snow some times, leaves change in the fall, and there’s a lot of outdoorsy activities all around.

extending a buildout: taking out unwanted parts

I’m working on a suite of buildout configuration files to use for different circumstances. Having a special buildout for development in a zeo cluster, and another for a single monolithic zope instance (plus useful debug tools already installed) was easy to do and is working well.

I then get to the notion of staging vs production. We stage our plone sites on a separate server, often using the staging instance for content development, debugging, etc. The IP addresses everything is listening on, the ports, etc, may need to vary between staging and production, but everything else has to be identical.

The idea of just running bin/buildout -c staging.cfg on the staging server seemed very elegant.

The production buildout.cfg does a lot. I’ve got homemade recipes that set up process control, multiple facets of apache (load balancer, ssl), etc. My first idea was to just duplicate the production buildout.cfg, and then modify it as needed, but I figured that would be hard to maintain. There had to be a better way.

So I read through the zc.buildout docs, and the Adding And Removing Options section caught my eye.

So say my production buildout.cfg file looks like this:

[buildout]
parts = apache
            ssl
            client1
            client2
            zeoserver
... 

[client1]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = me:pass
http-address = 8080

[apache]
recipe = my.apache.recipe
listen = 243.25.21.2:80

[ssl]
recipe = my.ssl.recipe
cert_file = /blah/blah
key = /secure/server.key
httpd_part = apache

... etc, you get the idea

And my staging.cfg file looks like this:

[buildout]
extends = buildout.cfg

[client1]
http-address = 9090

[apache]
listen = 127.0.0.1:5500

So I’m able to selectively override options in arbitrary parts. This is great. I like the idea of making the config that represents reduced functionality subtract functionality out of the other. Worries of having to maintain two files, or worse, have a dissimilar staging environment are mitigated.

But there’s a problem. Let’s say that on staging, I don’t care about SSL. I need to disable this part somehow, and from what I can tell, at least from the docs (code spelunking pending) there’s no way to do this.

I tried using the -= syntax that’s useful for lists of option values:

[buildout]
extends = buildout.cfg
parts -= ssl

[client1]
http-address = 9090

[apache]
listen = 127.0.0.1:5500

If this had worked, it would have been EPIC. But alas, it did not (should it have worked?).

So I poked around some more and checked with some folks on IRC with no luck.

I sort of solved the problem, by just repeating all the parts and then commenting out the ones I didn’t want:

[buildout]
extends = buildout.cfg
parts = apache
#          ssl
            client1
            client2
            zeoserver

[client1]
http-address = 9090

[apache]
listen = 127.0.0.1:5500

This works, but I don’t like it :P

I wonder if this is intentional, a bug, or just an oversight in zc.buildout. I’ll dig further when I’ve got time.

Request For Comments: My Plone/Apache Stack

I was recently took up the task of engineering my own plone infrastructure. I want to share what I did, and get any feedback I can about the scalability and usefulness of what I’ve done.

Requirements

• SSL – It must be secure while logged-in users are on the site*.
  • It must NOT be secure for public pages (unless the user is logged in)
  • It must force SSL for login pages.
• Hide /Plone – the infrastructure must present the user with URLs that don’t contain the plone instance id.
• Admin access – the infrastructure must provide SSL-encrypted access to the root ZMI.

* I understand that in recent versions of plone, this isn’t important to protect the user’s session cookies, however, I’m concerned about SSL encryption after login due to the content that will be stored on the site.

Approach

I’m very familiar with the Apache httpd server, so I wanted to leverage that experience as much as possible. I’m using the Apache 2.2.9 package that comes with Debian Lenny.

I used Plone 3.3.1 as I was developing this infrastructure, but I don’t think there’s anything about it that wouldn’t work with any version of Plone (as long as it’s got a virtual host monster).

We’ve settled on using one IP address (an alias) per Plone stack to keep the sites autonomous. This makes deployment and host registration easier.

I used IP-based Virtual Hosting and mod_proxy in conjunction with mod_proxy_balancer.

I’m logging to the local disk, and will be setting up a log rotation scheme separately.

Here’s what it looks like:

Production Plone Infrastructure (Existing Setup)

So I’ve got 3 Zope instances, acting as Zeo clients, on ports 1040, 1041, and 1042. I’ve heard more is just wasteful and fewer tends to not be enough.

I then have three virtual hosts. One is the load balancer, and back-ends to the Zope instances. It’s listening on port 8080 (the traditional proxy port, it really could be anything).

The other two virtual hosts are for "public" traffic (port 80), and SSL (port 443). They both back-end via a ProxyPass directive to the balancer virtual host. What’s different is what they tell the Virtual Host Monster to rewrite the URLs to and the RewriteRules that nudge the browser to use the right virtual host when doing certain actions, like logging in and logging out. Of course, the SSL virtual host is set up to do SSL using my SSL certificate (self-signed for now).

Configuration

I’ve got the config files sort of spread out because I’m using some buildout recipes I wrote to configure all this stuff, but here’s what it looks like all mashed into one file, pretending that you’re not using my recipes:

# httpd.conf for Apache2 - inspired by:  http://www.links.org/?p=264

ServerRoot  /etc/apache2

ServerName  localhost

Listen 127.0.0.1:8080
Listen 127.0.0.1:443
Listen 127.0.0.1:80

PidFile  /var/run/apache2.pid
LockFile /var/lock/apache2/accept.lock

LoadModule alias_module            /usr/lib/apache2/modules/mod_alias.so
LoadModule auth_basic_module       /usr/lib/apache2/modules/mod_auth_basic.so
LoadModule authn_file_module       /usr/lib/apache2/modules/mod_authn_file.so
LoadModule auth_digest_module      /usr/lib/apache2/modules/mod_auth_digest.so
LoadModule authz_user_module       /usr/lib/apache2/modules/mod_authz_user.so
LoadModule authz_host_module       /usr/lib/apache2/modules/mod_authz_host.so
LoadModule authz_groupfile_module  /usr/lib/apache2/modules/mod_authz_groupfile.so
LoadModule dir_module              /usr/lib/apache2/modules/mod_dir.so
LoadModule include_module          /usr/lib/apache2/modules/mod_include.so
LoadModule mime_module             /usr/lib/apache2/modules/mod_mime.so
LoadModule ssl_module              /usr/lib/apache2/modules/mod_ssl.so
LoadModule cgi_module              /usr/lib/apache2/modules/mod_cgi.so
LoadModule rewrite_module          /usr/lib/apache2/modules/mod_rewrite.so
LoadModule headers_module          /usr/lib/apache2/modules/mod_headers.so
LoadModule proxy_module            /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module       /usr/lib/apache2/modules/mod_proxy_http.so
LoadModule proxy_balancer_module   /usr/lib/apache2/modules/mod_proxy_balancer.so
#[BUILT-IN] LoadModule log_config_module  modules/mod_log_config.so
#[BUILT-IN] LoadModule logio_module       modules/mod_logio.so

# worker MPM - tuned
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
StartServers           1
MaxClients            16
MinSpareThreads        2
MaxSpareThreads        8
ThreadsPerChild        8
MaxRequestsPerChild    0

User www-data
Group www-data
ServerAdmin me@somehost.com

DirectoryIndex index.html

# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
<FilesMatch "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>

DefaultType text/plain

<IfModule mime_module>
    TypesConfig /etc/mime.types
</IfModule>

# "combinedio" includes actual counts of actual bytes received (%I)
# and sent (%O) - requires the 'mod_logio' module

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

ErrorLog /var/log/apache2/error.log
CustomLog /var/log/apache2/access.log combinedio

### Zeo Cluster Virtual Host ###################################################
<VirtualHost 127.0.0.1:8080>
    ProxyPreserveHost On

    Header add Set-Cookie "balancer_zopes=route.%{BALANCER_WORKER_ROUTE}e; path=/;"

    ProxyPass / balancer://zeocluster/
    <Proxy balancer://zeocluster>
        BalancerMember http://127.0.0.1:1040 route=client1
        BalancerMember http://127.0.0.1:1041 route=client2
        BalancerMember http://127.0.0.1:1042 route=client3
    </Proxy>
</VirtualHost>

### SSL Virtual Host ###########################################################

# example from http://httpd.apache.org/docs/2.2/mod/mod_ssl.html#sslsessioncache
# I think this should be conservative
SSLSessionCache shm:/usr/local/apache/logs/ssl_gcache_data(512000)

<VirtualHost 127.0.0.1:443>
    ServerName localhost

    SSLEngine on

    # this didn't work...
    # SSLCACertificatePath ...

    SSLCertificateKeyFile /home/jj/localhost.key

    # SSLCertificateChainFile /home/jj/localhost.ca-bundle

    SSLCertificateFile /home/jj/localhost.crt

    # ProxyPassInterpolateEnv On
    RedirectMatch /.*logged_out$ http://localhost:80/

    ProxyPass /admin http://localhost:8080/VirtualHostBase/https/localhost:443/VirtualHostRoot/_vh_admin
    ProxyPass / http://localhost:8080/VirtualHostBase/https/localhost:443/plone/VirtualHostRoot/

</VirtualHost>

### Non-SSL Virtual Host #######################################################

<VirtualHost 127.0.0.1:80>
    ServerName localhost

    ProxyPass / http://localhost:8080/VirtualHostBase/http/localhost:80/plone/VirtualHostRoot/
    RewriteEngine On
    RewriteCond %{HTTP_COOKIE} "__ac="
    RewriteRule ^(.*) https://%{SERVER_NAME}:443$1 [L]
    RewriteRule (.*)/login_form https://%{SERVER_NAME}:443$1/login_form [R]
</VirtualHost>

I was able to start apache using this file on a fresh Debian Lenny VM (bear in mind you have to create a self-signed certificate and a private key first)

The command looks like this:

   $ sudo /usr/sbin/apache2 -f /path/to/above/httpd.conf &

This has to be done as root (using sudo here) since port 80 and 443 are "priveledged" ports.

I, of course, have a process management daemon (runit) that handles this for me (and buildout recipes to configure it), which I will blog about at some point

I initially had this set up for testing with SSH tunneling (ssh -L) for the ports, and it worked well. This did however affect the setup, primarily I had to change the VHM paths so the URLs would be re-written to the local side of the tunnels, so if I connected like this:

    $ ssh -L1400:localhost:80 -L1443:localhost:443 jj@vmhost

The ProxyPass directive for the SSL VH looks like this:

    ProxyPass /admin http://localhost:8080/VirtualHostBase/https/localhost:1443/VirtualHostRoot/_vh_admin
    ProxyPass / http://localhost:8080/VirtualHostBase/https/localhost:1443/plone/VirtualHostRoot/

And the ProxyPass directive for the other host:

    ProxyPass / http://localhost:8080/VirtualHostBase/http/localhost:1400/plone/VirtualHostRoot/

This way the urls are pointed back through the SSL tunnel.

Discussion

The general Apache configuration is an amalgam of the /etc/apache2.conf file that came with Debian and some website I found about optimization.

Cluster

The Zeo Cluster virtual host uses mod_proxy_balancer to round-robin the Zope clients.

To get "sticky" sessions, I set a cookie using mod_headers that holds the "route" of the client. The routs are defined as additional information passed to the BalancerMember, and manifest in an evnironmen variable called "BALANCER_WORKER_ROUTE".

This way once someone comes into the site, as long as the cookie persists, they will always be proxied to the same client. I opted to let the cookie expire when the user closes their browser since that’s how Plone is set up by default.

SSL

The SSL virtual host is a very straight forward SSL setup. Since I used a self-signed certificate, I didn’t need a "ca-bundle" file.

The interesting bit is when I use ProxyPass to route all traffic to the load balancer. Here I utilize the Virtual Host Monster (standard issue in Zope/Plone these days) to ensure urls are properly re-written to use HTTPS. And while I’m at it, I also "hide" the plone root (called /plone in my case) so that it looks like https://www.myhost.com/ is the root of the plone site, instead of https://www.myhost.com/plone. Nice.

There’s another ProxyPass directive that proxies any access to /admin to the load balancer, but also uses some VHM magic to give the user access to the root of Zope, and the core ZMI. This allows access since we’re proxying / to /plone As a bonus, since this directive only exists in the SSL virtual host, this also "forces" the user to use SSL to do root ZMI stuff, which helps protect the admin password (and gives the administrator warm fuzzies).

I also add a RedirectMatch directive to push the user over to the non-ssl virtual host when they log out. No sense in keeping them on SSL when there’s no need.

Non-SSL HTTP

Finally, the "public" virtual host. It consists of a ProxyPass directive similar to the one used in the SSL virtual host, except in this case, we’re rewriting to port 80, standard http. The VHM bits are the same.

The last few directives are rewrite rules to get the user to use SSL if they are logged in, and redirect the user to the SSL virtual host when they click the "log in" link. I’m sure there’s more to do to sew this up (like alter the template for the login portlet), but this covers most of the bases.

The Future

Production Plone Infrastructure (Future Plans)

Logging

I’d like to set up separate logging for each virtual host, possibly to syslogd (or some other collective logging solution).

Virtual Crash Cart Access

I may add a fourth Zope client. This fourth client wouldn’t be in the load balancer group and is solely responsible for handling ZMI requests. This was something Joel Burton mentioned at the Advanced Bootcamp, and I’ve always been keen on it. All the Zope clients can be busy and/or locked up, apache can be hosed, and I can still get to an unhosed client’s ZMI to try to shut things down or manually fix whatever caused the problem.

Caching

I haven’t included caching in this setup for two reasons:

1. I don’t like to preemptively optimize, and
2. Cache-fu does an extraordinarily good job just being there. :)

This infrastructure makes it easy to drop in a caching mechanism between the public virtual host and the load balancer virtual host if I need it.

I’m keen on mod_cache, since it can be loaded easily and keeps with my "do as much as we can in apache" mantra. However, my unscientific tests have shown little benefit from it’s use. Of course, that was before I came up with this scheme, and I was trying to keep everything in one apache host (which I’ve dubbed the "one host to unite them, one host to rule them" folly). Stay tuned for more on that.

Even if mod_cache doesn’t work out, there are still lots of other options, from Varnish to Squid to PigeonNet.

Summary

So here’s my attempt to do Plone Up Right^TM, and distill all of the many tutorials that are out there into what I think is one pretty good approach. Please, leave comments and let me know if I’m out of my mind, or setting myself up for some pain later. All criticism is welcome. If I’m doing it right, that might be good to hear too :)

Plone User Management: LDAP vs RDBMS vs ?????

I’ve been faced with a user requirement I’ve never had to deal with before, and I’m on the fence as to what would be the best solution. I’m looking for war stories, implementation strategies, other ideas, anything that might help me make this decision.

Here are my basic requirements:

• Multiple users will log into the following services with the same username/password:
  • Multiple Plone Sites
  • Samba
• Most will be affiliated with the university I work for, but some will be outside collaborators.
• We also need to store extra metadata about users:
  • Contact info
  • Application-specific info

Softer requirements:

• Group/Role Management
• Softer metadata requirements:
  • Store service info (which plone sites are they on, what Samba file shares do they have access to, etc)
  • Plone site-specific info

Our campus uses Active Directory. I’m able to authenticate against it in Plone and Samba. I have no direct access, however, to the AD or would ever be able to routinely modify it.

This situation is further complicated by the addition of users who are not affiliated with our university. For Plone this isn’t too big of a deal, but I have yet to figure out how to get Samba to authenticate against multiple sources (e.g. user not in AD? auth against local user).

So I’ve come up with some proposed scenarios, and have drawn up some diagrams to help convey the ideas.

Scenario 1

Here LDAP is used as the central storage for users, groups, roles, etc, and LDAP handles both authentication and authorization. The information is managed via an external management interface. There are many LDAP management tools out there, so this wouldn’t require any development to that end.

There’s an alternate way to handle authentication here using Kerberos directly in Plone instead of letting LDAP do it. I added this after talking to a colleague of mine who’s ultimately in the same boat, and had a lot of trouble getting OpenLDAP to auth over Kerberos.

Scinerio 2

Here the situation is almost the same, except all of the user and group management is handled in Plone. This centralizes the user management to one common web-fronted UI (including Samba). I’m a little apprehensive of this situation. I don’t know if I like filesystem access decisions being made by Plone, and I also am not sure how well Plone works with LDAP in a read-write relationship.

Scinerio 3

This option involves building a user management system, and storing all of our user info in a relational database. The user management system pushes out user and group info to the Plone sites via XML-RPC (or some sort of RESTful addon), and separately alters the local userbase and samba config files on the Samba server.

In this scenario, Plone does authentication via a PAS plugin that talks to the AD, and falls back to the RDBMS. Samba authenticates against PAM, which tries AD and falls back to the local system.

One caveat here, I haven’t researched PAM yet to be sure it can handle this.

So, I’m at an impasse. I don’t know what to do. Avoiding development effort would be nice, but I’m not afraid of building a solution if I need to. I’m apprehensive about cramming application-specific data into LDAP, but such things have been done before.

I don’t know what to do. Any input would be greatly appreciated. :)

jjmojojjmojo – LIVE! (Plone Symposium Talk Video Availble)

I gave a talk on the “Great Saga of Content Type Creation in Plone” (read: AGX sucks but you should still do UML) at the Plone Symposium East 2009 at Penn State a month or two ago.

The video is available online. Give it a look.

I was scheduled against Nate Aune’s talk on Deliverance, a very hot topic, so I didn’t get as many people as I’d hoped, but the audience was great and the whole thing was a lot of fun. I turned some people onto other ways of making content, and got a ton of really positive and constructive feedback.

This was my first “major” presentation, so it was a big learning experience for me, as such, comments and suggestions are highly encouraged.

I’ll even put up with mild flaming ;)

ZopeSkel: Time For a Rebirth?

If you’re skimming this, I’d like to direct your attention to the poll below

I’ve been using ZopeSkel and advocating it’s use for over a year now. I’ve participated in a major sprint on it, I’ve taught it to friends and colleagues, and I’ve come to a conclusion:

ZopeSkel SUCKS.

OK, so it doesn’t really suck. IMHO, it sucks a lot less than ArchGenXML. It’s still an essential part of my development tool kit, and I can’t see doing Plone development without it.

I’m giving a talk about ZopeSkel at the Plone Symposium East next month. I’m going to be advocating ZopeSkel over AGX for content type creation. It will surely be epic.

Back to the suckage: ZopeSkel is not without its flaws. It’s inherent architecture isn’t quite right. ZopeSkel took a somewhat specific tool (PasteScript) and abused it to do something outside the realm of the tools purpose. This is bad!

I hit upon some of the shortcomings of ZopeSkel in the Sprint Reportout I posted last year. Over the past few months I’ve been pondering those shortcomings and think it’s time for a major change.

So here are the major issues:

• ZopeSkel is out of PasteScript’s scope
• ZopeSkel is impossible to test accurately
• ZopeSkel interactive sessions are not easily reproducable
• ZopeSkel can’t respond to user input
• ZopeSkel’s documentation varies from overly adequate to non-existent
• ZopeSkel’s development is severely fragmented.
• People complain about ZopeSkel not having a graphical user interface

That said, ZopeSkel is still extremely useful and intuitive, it just needs some work by a few dedicated individuals. Ultimately, I’d like to see the product position itself as the defacto code generation tool in the Plone universe.

So here’s what I think needs to be done:

We need to re-write zopeskel from the ground up. I like the use of cheetah templates and context-aware commands. I’d like to see a robust framework for collecting information from the developer, saving it into an INI file (or something else, maybe YAML) and providing a framework where the user interface is decoupled from the data collection and template execution.

There’s even an opportunity here for an integration point with AGX.

It’s ambitious, yes, but I think it’s well within the realm of possibility.

Now we get to the reason why I’m posting this: I’m seriously contemplating sprinting on this at Penn State next month.

Due to funding problems I’m paying my own way to the symposium. I don’t want to plan a sprint and stay 2 extra nights if nobody is going to sprint with me. I’m also open to the idea of taking on a sponsor if someone wants to put up some money to see this work progress.

I need to know what you think. Please leave a comment, or use the poll below.

I Used Macros In My Browser Views And Saved A Bunch of Money on My Car Insurance!

I had a template with a handful of tables that basically just displayed the same tabular data, just different data sets depending on the table. In case your curious, my data is segregated by workflow state.

I’ve had this problem before but in the past I had a skin layer registered too (thanks to ArchGenXML).

Defining macros is pretty easy, the Zope Book does a good job of explaining how to define them. It’s actually easier than it looks. If you don’t have a need for slots, it basically just comes down to slapping a metal:define-macro attribute into any arbitrary block of code.

Here’s a quick example:

<html>
<body>
    <p metal:define-macro="title">
        <span tal:replace="view/context/Title" />
    </p>
</body>
</html>

I’ve left out most of the usual ZPT window-dressings (like a i18n domain declaration), but I’ll assume you already know how to do that :).

This is also of course, a pretty stupid example. But it does illustrate an important concept in METAL: when your macro is being expanded, it’s as if you wrote the code right where the metal:use-macro attribute is used. So any variables that are available to the template that’s calling the macro are available here.

You have to be careful of this though, because it does limit how much your macro can be re-used.

In the past, because of the skin layer, I was able to drop a ZPT file into my skins directory, and call them like this (assume I put the last macro into a file called “macros.pt” in the skins directory along side this template):

<html>
<body>
    <p metal:use-macro="here/macros/macros/title" />
</body>
</html>

I saw a lot of examples where the name of the file itself was used (e.g. here/macros.pt/macros/title), but I don’t recall doing it that way.

As far as I can tell, this doesn’t work with templates on the filesystem that aren’t registered in a skin layer. So when you aren’t using a skins folder, because you’ve had the forethought to put that sort of stuff in a policy or theme product, this sort of macro expansion doesn’t seem to work.

I found a solution to this by accident. I don’t have my Zope 3 book (the essential Web Component Development with Zope 3) with me today, and was poking around Google aimlessly to solve this problem. I stumbled upon a page from a hereto unidentified Zope 3 course that showed me what I needed to do.

Zope 3 registers page templates, much like Zope 2/Plone does with the skins concept. But in Zope 3 you do it with ZCML as opposed to the CMF/GenericSetup glue that you need to register a Plone skin layer.

So in a handy configure.zcml (I put mine in the browser directory of my product), you first need to register the template containing your macros as a browser page:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:browser="http://namespaces.zope.org/browser">
...
    <browser:page
      for="*"
      name="my.product.macros"
      template="macros.pt"
      permission="zope.Public"
    />

</configure>

Note that I since my macro file (macros.pt) is in the same directory as my configure.zcml, I didn’t have to specify any sort of relative path information in the template attribute.

Since the macro template is being registered like any other browser view, you can specify a particular interface that is only allowed to use it, using the for attribute. Likewise, you can set a specific permission that will restrict access to the macros using the permission attribute. I haven’t tested either attribute with a browser page that’s being used for macros, so I don’t know what the expected behavior is.

OK, so with the ZCML tag above added to configure.zcml and a quick restart of the server (I had trouble with plone.reload in this case, FYI), any macros in macros.pt are available, to any view on any content type that feels inclined to use them.

To access the macros within a browser view/page (or viewlet, etc), the syntax is similar to what it was in the “skins layer” scenario mentioned earlier, but it’s very Zope 3:

<html>
<body>
    <p metal:use-macro="view/context/@@my.product.macros/title" />
</body>
</html>

You access the macro by getting the browser page defined in the ZCML above applied to the current context. That’s it.

So to summarize: you can use a browser page like a sort of template/code snippet resource. And as a bonus, said resource can contain METAL macros. METAL macros are a nice way to clean up your code, enforce DRY, and reduce bugs.

Side Note:
I’ve also come to realize that the same @@viewname syntax works to apply a view to an object inside of a page template. This is a natural predecessor to the macro stuff I was just talking about, but I didn’t realize it was possible until now. Whoo-hoo!