Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections

 

Introduction

The Bamboo Services Platform (BSP) Developer's Workbench is an integrated development environment for Project Bamboo service developers. The workbench facilitates the creation and maintenance of Bamboo services.

Characterizing the BSP 0.9 Developer's Workbench

The BSP Developer's Workbench is the local component of the Development layer, one of four layers which comprise the BSP Deployment Infrastructure. The workbench is composed of:

* An IDE within which the developer can create, read, update, or delete source code and / or unit test cases and data Integration with BSP core services.
* A dependency-based automated build framework capable of producing OSGI bundles deployable to a BSP instance
* A local instance of the BSP within which to perform integration testing with BSP Core and Utility services
* A database platform consisting of a runtime engine and an associated design tool

Supported Use Cases

The workbench supports the following use cases. Said use cases will be defined in a future service development guidelines document in BSP 1.0:


* Developer Creates Source
* Developer Unit Tests Source
* Developer Builds Local Project
* Developer Deploys Local Project
* Developer System Tests Local Project in BSP local instance

Installation

This document describes how a developer can build and configure a local, stand-alone BSP Developer's Workbench for use with BSP 0.9.

Requisite Software

Platform selection for workbench deployment is at the discretion of the developer, but limited to those operating system versions supported by Progress Fuse ESB as listed at Platforms Supported:

  • While not listed, Windows 7 is supported; please use JDK 1.6.x.
  • Note Java 1.5 is no longer supported
  • Note Java 1.7 u17 has been cursorily tested and found to support build and deploy of BSP v0.8 as well as the Tufts Syntactic Annotation v1.1.1 and Morphological Analysis v1.1.1 services.
  • Fuse ESB only supports JREs and JDKs distributed or OEMed from Sun Microsystems; Fuse ESB is not guaranteed to run with gcj. If you are using a Red Hat-derived Linux system (where, gcj is the default Java distribution) you must download and install a Sun JDK.

Given the fluidity of third-party software releases, it is not practical to define a canonical set of software down to the version level that is guaranteed to be available for installation over an extended period of time. As such, the table below reflects a point-in-time selection of the most stable versions of the available releases of the requisite software for use during BSP 0.9 development. If a particular version is no longer available, the current stable release should be used instead. Betas and snapshots should be avoided.

Java Language

JDK 1.6.0

u38

http://www.oracle.com/technetwork/java/javase/downloads/index.html

Integrated Development Environment

Eclipse

JEE Juno SR1

http://www.eclipse.org/downloads/packages/eclipse-ide-java-ee-developers/heliossr2\

Repository (Subversion / svn) clientSubclipseSubclipse 1.6.x (plugin version to match Bamboo code repository)http://subclipse.tigris.org/update_1.6.x (Eclipse plugin update URL)

Automated build process

Maven3

3.0.4

http://maven.apache.org/download.html

Eclipse integration to TestNG for unit testing

TestNG

6.8.0

http://beust.com/eclipse (Eclipse plugin update URL)

Eclipse integration Maven for local build

m2eclipse Core

0.12.1

http://m2eclipse.sonatype.org/sites/m2e (Eclipse plugin update URL)

Maven integration with java tool (e.g. CXF) dependency management

m2eclipse Extras

0.12.0

http://m2eclipse.sonatype.org/sites/m2e-extras (Eclipse plugin update URL)

Local Instance of BSPFuse ESB EnterpriseFuseSource/fuse-esb-full-7.0.0.fuse-061https://access.redhat.com/jbossnetwork/restricted/softwareDetail.html?softwareId=14353&product=fuse.esb.enterprise&version=&downloadType=distributions

Database

PostgreSQL

9.2.2

http://www.postgresql.org/download/

Directory Structure

The software that comprises the workbench will be deployed into a predefined directory structure, employing a naming scheme based on the BSP release and that of each software package:

  1. Select an install directory (do not use a directory that has spaces in its path name)
  2. Within the selected install directcory, create a BSP directory structure with a root directory of BSP_0.9
  3. Create a child directory to BSP_0.9 with the name BSP-local-store [however, if installing on a Unix machine, create BSP-local-store under the directory /srv]
  4. Within the directory BSP-local-store:
    1. Create a cache/BSP directory
    2. Create a logs/security directory
    3. Create a policies directory

 The remaining directories will be added while deploying the requisite software as described below

Setting Environment Variables

Using either the Windows Control Panel or a script (e.g. .bash_profile), set/export the following variables; note that your actual values may differ depending on the version of the requisite software used and the platform selected (e.g., Windows variables are referenced using percent-signs, e.g., %BSP_HOME%); this also applies to the different usage of the slash character in Linux / Mac vs. Windows:

Runtime variables must be set in the run-time FUSE ESB context. Build-time variables must be set in the build (Maven) context.

Run-timeBuild-time

Variable

Value

(tick) 

HOSTNAME

Set to the value of the fully qualified domain name of your host (if any; otherwise set to: localhost). Note that this FQDN will be the server name in URLs generated by BSP (e.g., http://localhost/...)

(tick) HOSTNAME_SERVICE_QUALIFIER

Set to one of these two choices:

  • :8181 mind the colon; use this value if Apache httpd is NOT used to intercept and forward calls to the BSP (e.g., a developer's machine w/o client auth configured, services generally addressed as http://localhost:8181/my/service/url)
  • /services mind the forward slash; use this value if Apache httpd IS used to intercept and forward calls to the BSP (e.g., as described in this documentation for use of client auth, where httpd forwards over an AJP connector)
(tick) BSP_READ_ONLY_SERVICE_USER_IDBSP_READ_ONLY_SERVICE_USER
(tick) BSP_READ_ONLY_SERVICE_USER_PASSWORDtacm0t-
(tick) BSP_GROUP_SERVICE_USER_IDBSP_GROUP_SERVICE_USER
(tick) BSP_GROUP_SERVICE_USER_PASSWORDtacm0t-
(tick) GROUPER_URL

${YourActualServerURL}/grouper-ws/servicesRest (e.g., https://bsp-dev.projectbamboo.org/grouper-ws/servicesRest)

If you are not installing Grouper do not set this environment variable. Bamboo services will interpret the absence of this environment variable to mean that Grouper is not available, and in this case will return messages that more clearly explain why certain functions are failing (e.g., Group service requests; and anything having to do with AuthZ). Absent Grouper, all access to BSP services is treated as anonymous persons accessing from unknown clients, and only "Permit All" policies will allow service requests to be fulfilled.

(tick) GROUPER_VERSIONv2_1_002 (use actual version installed)
 (tick)

BSP_HOME

${INSTALL_DIR}/BSP_0.9

(tick)(tick)

BSPLOCALSTORE_HOME

This value is OS dependent:

  • For Windows use: ${BSP_HOME}/BSP-local-store
  • For UNIX use: /srv/BSP-local-store
 (tick)

SERVICEMIX_HOME

${BSP_HOME}/fuse-esb-7.0.0.fuse-061

(tick)(tick)

JAVA_HOME

${BSP_HOME}/Java/jdk1.6.0_38

 (tick)

M2_HOME

${BSP_HOME}/apache-maven-3.0.4

 (tick)

MAVEN_OPTS [1]

'-Xmx512m -XX:MaxPermSize=512m'

 (tick)  
 (tick)BSP_DB_ NAMEBSP_V1_TEST
 (tick)

BSP_DB_PASSWORD

Bamboo_password_01

 (tick)

BSP_DB_ USER

postgres


Notes:

[1] MAVEN_OPTS sets the Maven memory settings. Mind the "-" in the value. For Windows, no quotation signs are used, e.g., MAVEN_OPTS value is:  -Xmx512m -XX:MaxPermSize=512m 

Setting PATH (command path) Environment Variable

Using either the Windows Control Panel or a script, add the following bin directories to the command path (PATH environment variable; noting that Windows variables are referenced using percent-signs, e.g., %JAVA_HOME%):

  • ${SERVICEMIX_HOME}/bin
  • ${JAVA_HOME}/bin
  • ${M2_HOME}/bin

Note: For Windows, the JDK bin directory should be placed in front of "c:\windows\system32" and "c:\windows"; this is because some Windows systems may have an out-dated copy of JDK/JRE in these directories.

These path elements are required to invoke, respectively, the ServiceMix console (karaf), Java, and Maven from a command-line.

 

Creating the SMX User

The following applies to Unix / Linux environments. Windows environments will use the default logged-in user.
# Create a new user with name smx. This user account should be used throughout the installation and all resources should belong to this user.

Deploying Requisite Software: Java and JVM-deployed

Depending on your platform and the approach taken by each software package provider, the requisite software may be deployed either by extracting a binary distribution or by running an installer program. In the case of a binary distribution, the software is packaged as either a .zip file or a .tar.gz file. When asked to extract the software in the steps below, please use an archive tool (such as Zip or Gunzip) appropriate to your platform to unpack the distribution:

  • Download the JDK and run the installation wizard, changing the destination directory to ${BSP_HOME}/Java
  • Download Maven, and extract the distribution to ${BSP_HOME}
  • Download FuseESB and extract the distribution to ${BSP_HOME}
  • Download Eclipse and extract the distribution to ${BSP_HOME}

Validating the Installation

To confirm that the installation has been successful, run the following commands, verifying that the results returned match those expected (note that the expected results below match the versions used in the example above):

Purpose

Command

Expected Result

Verify JDK

java -version

java version "1.6.0_38"
Java(TM) SE Runtime Environment (build 1.6.0_38-b05)
Java HotSpot(TM) 64-Bit Server VM (build 20.13-b02, mixed mode)

Verify Maven3

mvn -version

Apache Maven 3.0.4 (r1232337; 2012-01-17 00:44:56-0800)
Maven home: D:\bsp_0.9\apache-maven-3.0.4
Java version: 1.6.0_38, vendor: Sun Microsystems Inc.
Java home: D:\bsp_0.9\Java\jdk1.6.0_38\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"

Verify Eclipse

eclipse.exe

Eclipse should open and display the Java EE perspective

Note that the initial startup requires the user to choose a workspace directory. It is suggested that the  that the path of the chosen directory not include spaces.

Verify FuseESB

karaf

karaf "Servicemix" should appear with a prompt of "karaf@root>"

Configuring FuseESB

The following changes will take effect the next time FuseESB is started.

By default, the CXF servlet in FuseESB is assigned an alias of "cxf". To replace this with "bsp":

  1. Create a text file org.apache.cxf.osgi.cfg in ${SERVICEMIX_HOME}/etc
  2. Add entry: org.apache.cxf.servlet.context=/bsp
  3. add entry (substituting your server name): org.apache.cxf.servlet.base-address = http://${servername}:8181/bsp

Add support for JavaSE 1.6:

  1. Open ${SERVICEMIX_HOME}/etc/config.properties
  2. Edit the # OSGi Execution Environment setting to read: org.osgi.framework.executionenvironment=JavaSE-1.6,J2SE-1.5,J2SE-1.4,J2SE-1.3,J2SE-1.2

BSP uses several features which should be installed at boot:

  1. Open ${SERVICEMIX_HOME}/etc/org.apache.karaf.features.cfg
  2. Edit the featuresBoot setting by appending ,webconsole,jpa-hibernate

Enable Client Auth:

  1. Comment out all content in ${SERVICEMIX_HOME}/etc/org.ops4j.pax.web.cfg
  2. From the Karaf console:
    1. features:list | grep spring-dm-web
    2. If spring-dm-web is not installed, execute features:install spring-dm-web

Further Configuration and Installation (including IDE plugins)

The following steps will configure Maven, install PostgreSQL, and install and configure IDE plugins for use in the Developer's Workbench.

Configure Maven

The path to the local repository Maven will use to store artifacts defaults to ${HOME}/.m2/repository (where ${HOME} references the user's home directory for the platform-in-use). To ensure that all build artifacts are stored in the same repository, this setting must be changed to direct Maven to use the BSP-local-store directory:

  1. Open ${M2_HOME}/conf/settings.xml with a text or XML editor
  2. Search for "localRepository"
  3. Replace "/path/to/local/repo" with an absolute path to ${BSPLOCALSTORE_HOME}/mavenRepo/repository (note that you will have to uncomment the line)
  4. Save and close the file
  5. Run mvn install from a shell. Note that this command will report a "Build Failure" (i.e. we did not specify a POM file), but in the process of failing will initialize the Maven repository which is the intent. Note that this may take a while to execute.

See the attached Maven settings.xml file (stored as Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections) for additional Maven configuration of if the developer is deploying to a shared – rather than local – Maven repository using mvn deploy. The file is configured appropriately for deploying to the Collection Space instance of Artifactory from which built BSP bundles are available as of April 2013 (i.e., it matches the POMs in the tagged v0.9.0.-ALPHA release of the BSP codebase); YMMV.

 

Install and Configure Eclipse plug-ins: TestNG and m2eclipse

In order to use Eclipse in the context of the Developer Workbench, you will need to add the following plugins using the Eclipse Install New Software dialog, accessible from the IDE's Help menu:

Purpose

Plugin Name

Update Site

TestNG integration for Eclipse

TestNG

http://beust.com/eclipse

Maven integration for Eclipse

m2eclipse Core

http://m2eclipse.sonatype.org/sites/m2e

Maven Integration for WTP

m2eclipse Extras

http://m2eclipse.sonatype.org/sites/m2e-extras


For instructions on how to use the update manager, please search for the Installing New Software topic in Eclipse Help, or visit http://help.eclipse.org/juno/index.jsp?topic=%2Forg.eclipse.platform.doc.user%2Ftasks%2Ftasks-124.htm 

The plugins should be installed in the following order (be sure to check the boxes if the plugin is not already selected):

  1. TestNG
  2. m2eclipse Core
  3. m2eclipse Extras. Select "Maven Integration for Eclipse WTP" only

Note you will have to restart Eclipse after each plugin install. 
Configure the Eclipse m2eclipse plugin to use the Maven repository at ${BSPLOCALSTORE_HOME}:

  1. Open Window | Preferences | Maven | Installations
  2. Add a new installation using an absolute path pointing to ${M2_HOME}
  3. Make sure "Global settings' points to the absolute path of the settings.xml file modified during the "Configure Maven" step above before clicking OK.

Install and Configure Eclipse svn plug-in: Subclipse

  • Subclipse update site URL: http://subclipse.tigris.org/update_1.6.x
  • Choose/click alltop-level components:
    • Core SVNKit Library
    • Optional JNA Library (recommended)
    • Subclipse
  • Install and, when prompted, restart
  • Configure Subclipse
    • Go to Eclipse Preferences ( Window | Preferences)
    • Navigate to Team : SVN in the Preferences dialog
    • Choose the SVN interface: SVNKit (Pure Java) SVNKit v1.3.5.7406
    • Accept changes by clicking OK
  • Open the SVN Repositories view: (Window | Show View | Other | SVN | SVN Repositories)
  • Add a new repository (i.e., the Bamboo Technology Project svn repository):
  • The following steps assume you have been granted commit permissions in the repository https://source.projectbamboo.org/svn/btp (which is world-readable). Cf. BTP Subversion Repository - Guidelines and Info for more information. Still in the SVN Repositories view of Eclipse:
    • Check out a working copy of a repository directory to which you have commit permissions
      • Navigate to the desired repository directory, e.g., https://source.projectbamboo.org/svn/btp/_example/trunk
      • Right-click on the desired directory
      • Choose "Check Out"
      • Choose "Check out as a project in the workspace"
      • Choose and type an appropriate project name (probably not "trunk"!), e.g., "example-project"
      • If appropriate, choose a particular revision, set Depth, etc.; but in general these settings should be left as-is
      • Click "Next" and confirm that the chosen workspace location is appropriate (this is where your working copy will be stored on your local filesystem)
      • Click finish
      • The checked out project should appear as the project name typed above in your Project Explorer (or Resource) view
    • Create or modify a file or folder in order to confirm that you can check it into the repository
    • Commit the file or folder:
      • Right-click the file or folder you wish to commit. [Note:If you are committing multiple files/folders in one operation, you must right click a node that is an ancestor (parent or above) to the multiple files/folders you wish to commit.]
      • Enter a commit comment.
      • You will be presented with a dialog requesting your credentials (anonymous commits are not permitted on the BTP repository)
        • Your credentials are the same as those used to log into the Bamboo wiki (Confluence) or Issue Tracker (JIRA)
        • Enter both the UID and Password
        • Recommended: click "Save Password" so you need not enter credentials the next time you commit to the repo
    • When the icon next to the changed / committed file transforms from a black asterisk (changed locally) to a small yellow cylinder (in synch with repository), your commit has succeeded

Install and Configure PostgreSQL and pgAdmin III

Use of PostgreSQL and pgAdmin in the local database instance has been tested in both the Windows and Linux environments.

These components require a platform specific installation. For Windows the PostgreSQL installer will also install a Microsoft Visual C++ 2010 Redistributable module.


  To install in a Windows environment:

  1. Download PostgreSQL and run the installation wizard, changing the destination directory to ${BSP_HOME}\PostgreSQL\9.2
    1. if quick and dirty (rather than secure) installation is desired, make the PostgreSQL super-user and password the same as the environment variables BSP_DB_USER and BSP_DB_PASSWORD set above – e.g., user postgres with password Bamboo_password_01
  2. For Windows users to run PostgreSQL from a console (psql), it is advisable to first execute ${BSP_HOME}\PostgreSQL\9.2\pg_env.bat to set the PostgreSQL environment variables. These are not necessary if the bundled GUI administrative application ${BSP_HOME}\PostgreSQL\9.2\pgAdmin III is utilized for administration.

To install in a Linux environment:

  1. Install PostgrSQL using YUM
  2. Install PGAdmin using YUM
  3. Create a postgres user with password "Bamboo_password_01"

For all environments:

  1. From the  PGAdmin III application in Windows, from the command line, or using another administrative interface, create new database BSP_V1_TEST giving ownership to the user postgres (or, if different, the user set with the environment variable BSP_DB_USER) which has the password (or, if different, the password set with the environment variable BSP_DB_PASSWORD). [See above re: environment variable values.]
  2. Cf. create-BSP_V1.sql (DDL) in the codebase (${REPOSITORY_ROOT}/platform-services/bsp/trunk/bsp-ddl/src/main/resources/create-BSP_V1.sql)
  3. It is unnecessary to run the other DDL in the above-referenced directory; it is run on the initial build of BSP. However, if one wanted for some reason to manually re-create the database without performing a build, the DDL can be executed in this order (examples given using command-line syntax):
    1. psql -f create-BSP_V1.sql
    2. psql -f person-tables.sql BSP_V1
    3. psql -f notification-tables.sql BSP_V1
    4. psql -f cache-tables.sql BSP_V1
    5. psql -f utility.protected-resource.domain.sql BSP_V1

 

Deploying BSP

Deploying BSP from built features in a Maven repository

Once you have created a workbench, you can deploy the latest build of BSP directly from a Maven repository to which built bundles and features have been deployed. This option may be preferred to checking out the source code and building it locally (for local build and deploy, see below). As of April 2013, Project Bamboo's BSP bundles are archived at v0.9.0.-ALPHA in a Maven repository maintained by the Collection Space project, at http://nightly.collectionspace.org:8081/artifactory/. (During the active phase of the project, Bamboo maintained its own instance of an Artifactory maven repository, to which bundles were deployed by a continuous integration environment.)

You must have built BSP at least once in order for the test cases to have created the database. Alternately, execution of the DDL described above vis-a-vis PostgreSQL installation creates the necessary database and tables therein.

Initial Setup

Make the following configuration changes to FuseESB (assumes the April 2013 bundles are being deployed from the maven repo indicated immediately above):

  1. Edit ${SERVICEMIX_HOME}etc/org.apache.karaf.features.cfg:
    1. Add mvn:org.projectbamboo/bsp.features.build/0.9.0.-ALPHA/xml/features to the ".\" list of featuresRepositories
  2. Edit  ${SERVICEMIX_HOME}etc/org.ops4j.pax.url.mvn.cfg:
    1. Add http://nightly.collectionspace.org:8081/artifactory/repo@snapshots to the "\"-delimited list of org.ops4j.pax.url.mvn.repositories
  3. Place policy files in the directory ${BSPLOCALSTORE_HOME}/policies [for a dev machine, use the PermitAllServices.xml policy; cf. Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections for more info]
  4. Bounce smx (shut down, then restart)

Deploying BSP

  1. Confirm that the latest version of BSP does not require any configuration or database changes
  2. Delete any previously hot-deployed BSP jars from ${SERVICEMIX_HOME}/deploy
  3. Open a Karaf console and execute the following commands:
    1. features:refreshurl mvn:org.projectbamboo/bsp.features.build/0.9.0.-ALPHA/xml/features
    2. features:list | grep BSP (to confirm that the BSP feature is available)
    3. features:install BSP (to deploy BSP)

Example command console output, below, is taken from a Windows installation of FUSE ESB and shows deployment of the BSP v0.9.0.-ALPHA features:

D:\bsp_0.9\>karaf
[...]
FuseESB:karaf@root> features:refreshurl mvn:org.projectbamboo/bsp.features.build/0.9.0.-ALPHA/xml/features
FuseESB:karaf@root> features:list | grep BSP
[uninstalled] [0.9.0.-ALPHA        ] BSP                                   repo-0
FuseESB:karaf@root> features:install BSP
FuseESB:karaf@root>

 

Deploying BSP from local Maven repository

Once you have created a workbench, you can check out code, build, and deploy BSP directly from your local Maven repository. The assumption in the examples below is that code is checked out from trunk, where the version in Maven POM files is set to 0.9.0.-SNAPSHOT.

Initial Setup

Make the following configuration changes to FuseESB:

  1. Edit ${SERVICEMIX_HOME}etc/org.apache.karaf.features.cfg:
    1. Add mvn:org.projectbamboo/bsp.features.build/0.9.0.-SNAPSHOT/xml/features to the ".\" list of featuresRepositories
  2. Edit  ${SERVICEMIX_HOME}etc/org.ops4j.pax.url.mvn.cfg:
    1. Remove the repository http://nightly.collectionspace.org:8081/artifactory/repo@snapshots from the "\"-delimited list of org.ops4j.pax.url.mvn.repositories (assuming it was added per steps describing deployment from built bundles in a Maven repo, above)
  3. Place policy files in the directory ${BSPLOCALSTORE_HOME}/policies [for a dev machine, use the PermitAllServices.xml policy; cf. Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections for more info]
  4. Bounce smx

Build BSP

  1. Confirm that the latest version of BSP does not require any configuration or database changes
  2. Download appropriately-tagged codebase, e.g., trunk, into an Eclipse workspace project, e.g. workspace/${MyBSPdownload} (or other local location)
  3. Run the following Maven install targets from the IDE or command line. Command line given below:
    1. ${MyBSPdownload}/bsp>mvn install
    2. ${MyBSPdownload}/bsp-features/bsp-features>mvn install

Deploying BSP

  1. Delete any previously hot-deployed BSP jars from ${SERVICEMIX_HOME}/deploy
  2. Open a Karaf console and execute the following commands:
    1. features:refreshurl mvn:org.projectbamboo/bsp.features.build/0.9.0.-SNAPSHOT/xml/features
    2. features:list | grep BSP (to confirm that the BSP feature is available)
    3. features:install BSP (to deploy BSP)

 

Confirming deployment of BSP services

 

 

Available SOAP services:


Available RESTful services:
Endpoint address: http://localhost:8181/bsp/requests
WADL : http://localhost:8181/bsp/requests?_wadl

Endpoint address: http://localhost:8181/bsp/protectedresources
WADL : http://localhost:8181/bsp/protectedresources?_wadl

Endpoint address: http://localhost:8181/bsp/serviceprofiles
WADL : http://localhost:8181/bsp/serviceprofiles?_wadl

Endpoint address: http://localhost:8181/bsp/contacts
WADL : http://localhost:8181/bsp/contacts?_wadl

Endpoint address: http://localhost:8181/bsp/notifications
WADL : http://localhost:8181/bsp/notifications?_wadl

Endpoint address: http://localhost:8181/bsp/groups
WADL : http://localhost:8181/bsp/groups?_wadl

Endpoint address: http://localhost:8181/bsp/persons
WADL : http://localhost:8181/bsp/persons?_wadl

Endpoint address: http://localhost:8181/bsp/resultsetcaches
WADL : http://localhost:8181/bsp/resultsetcaches?_wadl
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<person:profileList xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:bsp="http://projectbamboo.org/bsp/resource" xmlns:contacts="http://projectbamboo.org/bsp/services/core/contact" xmlns:person="http://projectbamboo.org/bsp/BambooPerson" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <bsp:listMetadata>
        <dcterms:creator xsi:type="foaf:Agent" rdf:about="persons"/>
        <dcterms:created xsi:type="dcterms:W3CDTF">2013-04-10T15:59:05.403-07:00</dcterms:created>
        <bsp:orderedBy>ascending</bsp:orderedBy>
        <bsp:listLength>0</bsp:listLength>
        <bsp:pageNumber>1</bsp:pageNumber>
        <bsp:pageLength>20</bsp:pageLength>
    </bsp:listMetadata>
</person:profileList>

 

Additional Steps

Install and Configure Apache Web Server (httpd) to enforce Client Auth (trusted client access)

A secured BSP instance – one that enforces policy and accepts requests from trusted clients – requires that access be redirected through Apache Web Server (httpd). Apache Web Server enforces secure (HTTPS) connections over SSL; handles authentication of trusted application via pre-exchanged X.509 certificates; and directs traffic to the BSP over the AJP connector port through which FUSE ESB is configured to handle security.

Instructions for configuration of Apache Web Server on the FUSE ESB host server, and for generation and exchange of certificates with clients, can be found on the page Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections.

Install and Configure Grouper - as persistence store for Group and Trusted Application data

In phase one of the Bamboo Technology Project, Internet2's Grouper was selected for use as a persistence store for data about user-created groups, and for 'application catalog' data (records of applications registered in the Bamboo Trust Federation, cf. Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections).

A secured BSP instance – one that enforces policy and accepts requests from trusted clients – requires that Grouper be available (installed) and configured as described on the page Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections. In this case, the environment variable GROUPER_URL must be set in the context where FUSE ESB is running, and must point to the Grouper installation (cf. the Setting Environment Variables section of this page).

If Grouper is not installed (because the BSP instance is intended to be run as world-accessible, without security), it is essential that the environment variable GROUPER_URL not be set.

Important notes for service developers

Unit Tests require maintenance

Overview

A number of unit tests in the v0.9-alpha BSP code base will pass only if expected data is found in stores of Bamboo Person data, registered application data, and group data. These data are stored in:

  • a database backing the Person Service (and other BSP-deployed services); 
  • the Grouper instance that stores 'Application Catalog' data in the stem bamboo:projects; and,
  • the (same) Grouper instance that stores user-created groups in the bamboo:groups stem.

The 'expected data' in the unit tests as committed to the BSP codebase was extant in the shared instances of the BSP and Grouper that were in-use as of March 2013. These instances have been decommissioned. Thus, for unit tests to pass a developer adopting this codebase must assume certain responsibilities vis-a-vis reinstantiating backing data or modifying the unit test source code.

An alternative to building the BSP bundles is to deploy built bundles at their v0.9-ALPHA release. See Built bundles available for deployment, below.

Examples

The Group Service test in FindGroupTest.java assumes the existence of data hard-coded into the class in a set of static final strings:

//This data reflects the state of the Grouper instance on services-qa.projectbamboo.org 01.31.2013
  private static final String BSP_TEST_APP_UUID  = "2c38ef3775a948719388ebdac1cf3d78";
  private static final String BAMBOO_STEM = "458930f0-0716-11e2-892e-0800200c9a66";
  private static final String ADMIN_STEM = "41f062e2-156e-457e-8371-4131c60a6976";
  private static final String APPS_STEM = "020494e9-7e84-4442-bcbb-6dc299a2df45";
  private static final String BSP_TEST_APP_ID = "4e9d6f57-f68d-49eb-bcc4-200f56e7f4d2";

Similarly, the Group Service tests in GroupProxyClientMethodsTest.java assume existence of these data:

//Test data consistent with Grouper on services-qa circa 02.26.2013
  private static final String UNIT_TEST_GROUP_ID = "4e9d6f57-f68d-49eb-bcc4-200f56e7fbfb";
  private static final String UNIT_TEST_STEM_ID = "4e9d6f57-f68d-49eb-bcc4-200f56e7fcfc";
  private static final String FARKLE_SUBJECT_ID = "urn:uuid:c00f294d-651b-495b-9633-a900dd1a7418";
  private static final String ADAMS_SUBJECT_ID = "urn:uuid:04574969-e851-4fbf-8535-0b81b9b3f2f3";

Tests for the Request Manager service found in PatternsTest.java within that service's codebase assume existence of these data:

private static final String SOURCEDID_RESOURCE = "/bsp/persons/urn:uuid:a486e4fd-38e6-4915-8c11-c9dc00963f5f/sourcedids/urn:uuid:99999999-9999-9999-9999-999999999999";
private static final String PERSONID_RESOURCE = "/bsp/persons/urn:uuid:c00f294d-651b-495b-9633-a900dd1a7418";

Options

Developers who wish to build BSP in the future have several options:

  1. execute builds without executing unit tests
  2. manually re-create expected data in new instances of BSP databases and Grouper
  3. re-create a base of users and groups with which to execute tests, and appropriately modify values hard-coded in the unit test classes
  4. contact archivists@lists.projectbamboo.org to inquire about availability of PostgreSQL database dumps of March 2013 from which newly-instantiated instances of BSP and Grouper might be populated (cf. Resources for bootstrapping a fully-functional dev environment, below)

Any of these options entail work and/or risk. These are necessarily responsibilities that a developer assumes in adopting the Bamboo codebase. Also see Built bundles available for deployment, below.

Built bundles available for deployment

BSP bundles are built and available for deployment from a Maven repository as described in Deploying BSP, above. As of April 2013, these bundles are hosted by the Collection Space project's Maven repository.

Resources for bootstrapping a fully-functional dev environment

Overview

Anticipating future interest in setting up a shared developer deployment environment such as that used during the active period of the Bamboo Technology Project (October 2010 to March 2013), a number of database dumps and disk images were preserved for future use. Please contact archivists@lists.projectbamboo.org to inquire about availability of these data to projects and developers known to the Project Bamboo community.

Extant disk images and database dumps

The following data were archived in March 2013:

  • Grouper database dumps to reconstitute Grouper 2.1.2, including a Subject Authority database used as an interim solution in advance of development of a Grouper Subject Adapter to the Bamboo Person Service.
  • BSP database dump to reconstitute the set of Bamboo Persons, profiles, etc. that complement the Grouper data referenced in the prior bullet. There are some otherwise-useless artifacts of manual tests of Bamboo services in this database, so it is a superset of what's needed to complement Grouper data and successfully run unit tests.
  • Disk image of a Fedora 16 64-bit Linode VM on which FUSE ESB and Grouper were installed and populated with the data described in the prior bullets (~6.2 GB)
  • Disk image of a CentOS 6.2 64-bit Linode VM on which the PoC "Account Services Module" was deployed in a Drupal instance configured as a trusted client application in the Bamboo Trust Federation (~3.1 GB)

Notes on instantiating BSP on (Linode) virtual machines

Linode Usage Overview

Several generations of BSP installations and installations of other applications (Grouper, the Continuous Integration server Atlassian Bamboo, an Artifactory Maven repository) were deployed on Linode virtual machines during the active period of the Bamboo Technology Project. These VMs were built atop Fedora 16 in the latest instances, and were built to a standard recipe. That recipe would be idiosyncratic and out-of-date to any future deployer, and so is not included in this documentation set. Instead, a few key notes and artifacts are published here in the hope that they may prove useful to deployers using newer software.

Notes

  • Apache httpd (Apache web server) is required and must be set up to participate in client auth interactions, as described in the wiki page Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections 
  • Environment variables, PostgreSQL, and FUSE ESB must be installed and configured as described on this page (above), with appropriate OS-specific tweaks
  • Appropriate filesystem directories must be created and given appropriate permissions (e.g., to the unix user under which FUSE ESB will run; smx was the unix user used by the Project Bamboo team
  • Unix utilities sendmail and mailx were installed on the Linode instances to handle e-mail traffic initiated by the BSP's Notification Service
  • Firewall rules blocked ports other than 80 and 443, in order to protect against attacks; traffic to BSP, Grouper, Atlassian Bamboo, Artifactory, etc., was routed through the Apache Web Server and an AJP connector. (In unsecured deployments of the BSP, port 8181 was left open for direct, unencrypted traffic to BSP services.)
  • See note on FUSE ESB / Service Mix startup scripts, below
Startup Scripts for FUSE ESB (Service Mix)

On Project Bamboo Linode VMs, Service Mix was started as a service by utilizing scripts adapted from scripts created at UC Berkeley for that institution's local instantiation of FUSE ESB instances. Note that the file locations described below are particular to the Fedora 16 OS on which they were invoked. The scripts are attached in TXT files, as linked below.

Install fuse-smx scripts

Copies are attached to this wiki page of the two scripts

  1. /etc/rc.d/init.d/fuse-smxDeveloper Workbench Environment for BSP Service Developers -- Fall 2013 corrections
    • assure that script is executable (0755, executable by user, group, and other)
  2. /etc/sysconfig/fuse-smxDeveloper Workbench Environment for BSP Service Developers -- Fall 2013 corrections
    1. assure that the config file is readable by u, g, o (0644)

The file /etc/rc.d/init.d/fuse-smx was modified to comment-out (and in some cases replace) elements specific to the UC Berkeley VM installations for which the scripts were created. Cf. comments in the attached copy.

Enable fuse-smx startup script with chkconfig

sudo /sbin/chkconfig --add fuse-smx
sudo /sbin/chkconfig fuse-smx on