Chad Granum's blog

  • Boosting performance on a permission system with nested roles.

    Dec 29, 2009

    We discovered a problem while QA'ing emC against a huge client database. The page which displays who has access to a client would take a full minute to load. This was deemed unacceptable, and a solution was found.

    The elementalclinic permissions system works on the idea that there are clients, personnel (staff members), roles to which personnel are members, and permissions for roles to access clients. Every staff member is given a 'primary role' that only they are a member of. Personnel become members of roles when their primary role is added as a member of a system role. This also allows for nested roles where a staff member is a member of system role a, which is a member of system role b.

    Nested roles are useful when you want to share permissions between roles. For instance there is an all_clients role which grants access to all clients. The Admin role, being superuser, should also have access to all clients. With nested roles you simply make admin a member of all_clients. You can then add admins to admins, and non-admins who need all_client access to all_clients.

    The problem with nested roles comes when you need to check if a staff member has permissions to view a specific client. You need to build a list of all roles the staff member is a direct or indirect member of. You then need to check if any of these roles grants access to the client. On the access page this needed to be done for each staff member.

    To solve this problem I decided to leverage postgres. In the initial revision of the permissions system the database simply held the roles, direct memberships, and permissions. Traversing the roles and assessing client permissions for specific users was left entirely to the code. In the new revision I decided the database would maintain a complete map of memberships, both direct and nested. In addition the database would provide several views to simplify obtaining the results we need.

    The focus of the changes is with the table that hold memberships. Initially it's schema was very simple (Old way):

     21 CREATE TABLE personnel_role_member(
 22     rec_id        SERIAL NOT NULL PRIMARY KEY,
 23     role_id       INTEGER REFERENCES personnel_role( rec_id ) NOT NULL,
 24     member_id INTEGER REFERENCES personnel_role( rec_id ) NOT NULL,
 25     -- Make sure duplicate memberships do not occur.
 26     UNIQUE( role_id, member_id ),
 27     -- Roles should not be members of themselves.
 28     CHECK( member_id != role_id )
 29 );

    Here is the new schema for tracking role memberships:

     21 CREATE TABLE role_membership(
 22     rec_id         INTEGER UNIQUE NOT NULL PRIMARY KEY DEFAULT nextval('role_membership_rec_id_seq'),
 23     role_id        INTEGER NOT NULL REFERENCES personnel_role( rec_id ) ON DELETE CASCADE,
 24     member_id      INTEGER NOT NULL REFERENCES personnel_role( rec_id ) ON DELETE CASCADE,
 25     direct_cause   INTEGER REFERENCES role_membership( rec_id ) ON DELETE CASCADE,
 26     indirect_cause INTEGER REFERENCES role_membership( rec_id ) ON DELETE CASCADE,
 27     -- Roles should not be members of themselves.
 28     CHECK( member_id != role_id ),
 29     CHECK( direct_cause != indirect_cause ),
 30     -- Duplicates are pointless.
 31     UNIQUE( role_id, member_id, direct_cause, indirect_cause )
 32 );

    The main thing to note in the new schema are the 'direct_cause' and 'indirect_cause' fields. These reference other entrees in the same table. The point of this change is to track both direct and indirect memberships in one table. When a membership is added a trigger fires which adds an entree for all resulting indirect memberships.

    The trigger code is fairly complicated, so I am posting a simplified psudo-code version. This trigger is fired whenever an insert occurs on the role_membership table. Note: In this psudo-code 'us' or 'we' refers to the role that is becoming a member.

      Check for recursive memberships, raise an exception if found.
 
  Add memberships to all the roles the role we just became a member of is a member of. Direct cause is the new membership, indirect cause is the existing membership
 
  Add memberships to the role we just became a member of to the roles that are members of us. Direct cause is the existing membership, indirect cause is the new membership.

    This trigger also fires off for any indirect memberships added by the trigger recursively. This trigger results in a table that always has a complete list of both direct and indirect memberships.

    From this table it is a simple matter to create views for just direct or indirect members. Views for which roles have access to which clients is also fairly trivial. We can even create a view that shows client permissions with the reason(membership) that grants them.

    Here are links to the original permissions schema, as well as the migration to the new system.
    Original
    Migration
    And here is some additional documentation
    README

  • Retrofitting a role and permissions system to a large application.

    Oct 12, 2009

    The project

    Recently I was given the task of overhauling the permissions system of eleMentalClinic. We needed a way to specify which clinicians could view which clients. We needed to ensure client security by preventing unauthorized clinicians from viewing certain clients.

    Background

    The original eMC permissions system was very simple. In the database there was a list of roles. There was also a table that listed client->role relations. Roles were identified by name. Roles were only used to verify a clinician had access to a given controller.

    The plan

    This refactor was a multi-step process. The first step was to write a new role+permissions system that worked with the controller-access logic, but also allowed for clinician+client associations. The second step was to retrofit security check logic to the application.

    The first step is fairly simple. Creating roles and associations like what we needed for eMC is a common problem with several common solutions. I ended up migrating to a new system where we had system roles, personnel roles, role memberships, and role-client associations. Memberships are all role-role associations.

    The problem

    The second step is significantly more difficult, specially when you consider the size and complexity of eMC. One thought is to have the dispatcher intercept the client_id parameter and run a check against the current_user. The problem with this solution comes on pages where multiple clients are displayed, many without a request.

    The next possibly solution is to modify the client object to require authorization in order to build. The problem here is that the current_user is known to the controller, but not globally. Making the current_user a global did not sound like a good idea, in addition this would break a majority of the tests in the eMC test suite, a suite that is 2.9mb. If we had to refactor the client object, and all the resulting test failures, this task could take months.

    The solution

    I began toying with the idea of code that could retrofit the permissions into eMC at runtime. A magical module that would be able to provide the security check with all the information it needs. A module that could ensure security without the need to rewrite any of eMC's existing object code. Enter Package-Watchdog. Package-Watchdog is a Perl module I wrote to do just that.

    With Package-Watchdog you can 'watch' subroutines within a specific package, and 'forbid' them from using subroutines in another package. The way it works is simple in concept, wrap code around the watched sub that rewrites the forbidden subs. During normal use the forbidden subs act as normal, However once inside a watched sub the forbidden ones are altered. This continues all the way down the stack, you could have a chain of subs as deep as you want, if any of them call the forbidden one the watchdog is triggered.

    By default Package-Watchdog will die when a forbidden sub is invoked within a watched sub. However Package-Watchdog is written so that you can provide a custom reaction subroutine. This custom reaction subroutine is passed all the information relevant to the watch. This includes the names of the subs called, the parameters they were called with, and the watchdog helper objects that manage them.

    Package-Watchdog in eMC

    In eMC a watchdog object is instantiated just before the controller. The watchdog is told to watch the controller's subroutines, and forbid the subroutines in the client object that are used to build it. The watchdog is provided a custom reaction sub. This custom reaction sub gets the controller via arguments, and the current user from the controller. The client_id being loaded is provided via the forbidden arguments. The forbidden sub can then take this information and run a security check. If the check passes the program continues, otherwise it throws a security exception.

    The main goal of the watchdog is to absolutely prevent displaying client information to an unauthorized clinician. The watchdog left the application badly broken, pages such as schedule would always throw security exceptions. This is still considered a win, it means no accidental displaying of client information.Now it becomes a task of fixing the security exceptions, and as an added bonus we can now see where they are so they don't slip by.

    As one might expect this could slow down an application considerably, and at first it did. I added caching to the permissions system, and also to the security check logic. The application now runs as fast as it always has in my experience.

    Links

    Package-Watchdog on CPAN
    Package-Watchdog on github

  • IE OnChange events

    Apr 08, 2009

    I have been working on a Drupal project here at OpenSourcery that involves a dynamic form which updates via ajax whenever a radio button or checkbox is modified. Drupal has built-in ajax/ahah, making this task much simpler. It lets you specify which type of event you want to trigger the ajax, as well as many other options.

    For this specifically the requirement was that it occur with the 'OnChange' event. This works great in firefox, but not so great in IE. IE does not register an 'OnChange' event until the mouse is clicked somewhere else on the page after having changed your radio/checkbox selection. This is of course not the desired behavior.

    Drupal offers other choices for events that trigger the ajax. The next obvious candidate was 'click'. Click provided consistant behavior between IE and FF. When you click the new button the form would update. However, it resulted in some undesirable behavior in both browsers. The radio button selection would not update. The correct option was sent to post for ajax, but the actual radio button selection did not change.

    At this point it became obvious that I was going to need to hand-code some javascript to work around this issue. At first I attempted to write javascript that would update the radio selection after the ajax fired. This was an exercise in futility. I encountered many difficulties, most of which were probably due to inexperience.

    Before pounding my head too long on my first attempted solution, a better solution reached my brain. Instead of using 'Click' and trying to re-write default radio and checkbox functionality, I would use 'OnChange' and work around IE's problem. The solution was simple, if the browser is IE, add an 'OnClick' event to all the ajax elements that triggers the 'OnChange' event.

    This solution worked perfectly, and was very easy to write, even for someone with little js experience. My first time through I failed to follow proper Drupal convention, and as such I had to implement my own logic to update all the element triggers whenever ajax was used. Fortunately Jonathan reviewed my JS and pointed out the proper way to achieve this (see note 2).

    Here is the final version of the javascript

    Drupal.behaviors.nexusForm = function (context) {
  if (jQuery.browser.msie) {
    trig_bind()
  }
  //Hide the botton we only want to see if the browser does not support js
  $("#edit-continue").hide()
}
 
//IE waits until another event to send the 'change' events on radios and checkboxes
//This bind a trigger for those events on click.
function trig_bind() {
  //unbind old events
  $("input[type='checkbox']").unbind( 'click' )
  $("input[type='radio']").unbind( 'click' )
  //bind the events
  $("input[type='checkbox']").bind( 'click', function() {
    $(this).trigger( 'change' )
  })
  $("input[type='radio']").bind( 'click', function() {
    $(this).trigger( 'change' )
  })
}

    Notes:

    1. when I refer to IE I am referring to IE 6.
    2. an even better solution would be to use 'live' from jquery 1.3, however Drupal still uses 1.2, and updating jquery was not an option.
  • Chad Granum

    Chad Granum

    OpenSourcery Alumnus