Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections
Introduction
The Bamboo Services Platform (BSP) Developer's Workbench is an integrated development environment for Project Bamboo service developers. The workbench facilitates the creation and maintenance of Bamboo services.
Characterizing the BSP 0.9 Developer's Workbench
The BSP Developer's Workbench is the local component of the Development layer, one of four layers which comprise the BSP Deployment Infrastructure. The workbench is composed of:
* An IDE within which the developer can create, read, update, or delete source code and / or unit test cases and data Integration with BSP core services.
* A dependency-based automated build framework capable of producing OSGI bundles deployable to a BSP instance
* A local instance of the BSP within which to perform integration testing with BSP Core and Utility services
* A database platform consisting of a runtime engine and an associated design tool
Supported Use Cases
The workbench supports the following use cases. Said use cases will be defined in a future service development guidelines document in BSP 1.0:
* Developer Creates Source
* Developer Unit Tests Source
* Developer Builds Local Project
* Developer Deploys Local Project
* Developer System Tests Local Project in BSP local instance
Installation
This document describes how a developer can build and configure a local, stand-alone BSP Developer's Workbench for use with BSP 0.9.
Requisite Software
Platform selection for workbench deployment is at the discretion of the developer, but limited to those operating system versions supported by Progress Fuse ESB as listed at Platforms Supported:
- While not listed, Windows 7 is supported; please use JDK 1.6.x.
- Note Java 1.5 is no longer supported
- Note Java 1.7 u17 has been cursorily tested and found to support build and deploy of BSP v0.8 as well as the Tufts Syntactic Annotation v1.1.1 and Morphological Analysis v1.1.1 services.
- Fuse ESB only supports JREs and JDKs distributed or OEMed from Sun Microsystems; Fuse ESB is not guaranteed to run with gcj. If you are using a Red Hat-derived Linux system (where, gcj is the default Java distribution) you must download and install a Sun JDK.
Given the fluidity of third-party software releases, it is not practical to define a canonical set of software down to the version level that is guaranteed to be available for installation over an extended period of time. As such, the table below reflects a point-in-time selection of the most stable versions of the available releases of the requisite software for use during BSP 0.9 development. If a particular version is no longer available, the current stable release should be used instead. Betas and snapshots should be avoided.
Java Language | JDK 1.6.0 | u38 | http://www.oracle.com/technetwork/java/javase/downloads/index.html |
Integrated Development Environment | Eclipse | JEE Juno SR1 | http://www.eclipse.org/downloads/packages/eclipse-ide-java-ee-developers/heliossr2\ |
Repository (Subversion / svn) client | Subclipse | Subclipse 1.6.x (plugin version to match Bamboo code repository) | http://subclipse.tigris.org/update_1.6.x (Eclipse plugin update URL) |
Automated build process | Maven3 | 3.0.4 | |
Eclipse integration to TestNG for unit testing | TestNG | 6.8.0 | http://beust.com/eclipse (Eclipse plugin update URL) |
Eclipse integration Maven for local build | m2eclipse Core | 0.12.1 | http://m2eclipse.sonatype.org/sites/m2e (Eclipse plugin update URL) |
Maven integration with java tool (e.g. CXF) dependency management | m2eclipse Extras | 0.12.0 | http://m2eclipse.sonatype.org/sites/m2e-extras (Eclipse plugin update URL) |
Local Instance of BSP | Fuse ESB Enterprise | FuseSource/fuse-esb-full-7.0.0.fuse-061 | https://access.redhat.com/jbossnetwork/restricted/softwareDetail.html?softwareId=14353&product=fuse.esb.enterprise&version=&downloadType=distributions |
Database | PostgreSQL | 9.2.2 |
Directory Structure
The software that comprises the workbench will be deployed into a predefined directory structure, employing a naming scheme based on the BSP release and that of each software package:
- Select an install directory (do not use a directory that has spaces in its path name)
- Within the selected install directcory, create a BSP directory structure with a root directory of BSP_0.9
- Create a child directory to BSP_0.9 with the name BSP-local-store [however, if installing on a Unix machine, create BSP-local-store under the directory /srv]
- Within the directory BSP-local-store:
- Create a cache/BSP directory
- Create a logs/security directory
- Create a policies directory
The remaining directories will be added while deploying the requisite software as described below
Setting Environment Variables
Using either the Windows Control Panel or a script (e.g. .bash_profile), set/export the following variables; note that your actual values may differ depending on the version of the requisite software used and the platform selected (e.g., Windows variables are referenced using percent-signs, e.g., %BSP_HOME%); this also applies to the different usage of the slash character in Linux / Mac vs. Windows:
Runtime variables must be set in the run-time FUSE ESB context. Build-time variables must be set in the build (Maven) context.
Run-time | Build-time | Variable | Value |
HOSTNAME | Set to the value of the fully qualified domain name of your host (if any; otherwise set to: localhost). Note that this FQDN will be the server name in URLs generated by BSP (e.g., http://localhost/...) | ||
HOSTNAME_SERVICE_QUALIFIER | Set to one of these two choices:
| ||
BSP_READ_ONLY_SERVICE_USER_ID | BSP_READ_ONLY_SERVICE_USER | ||
BSP_READ_ONLY_SERVICE_USER_PASSWORD | tacm0t- | ||
BSP_GROUP_SERVICE_USER_ID | BSP_GROUP_SERVICE_USER | ||
BSP_GROUP_SERVICE_USER_PASSWORD | tacm0t- | ||
GROUPER_URL | ${YourActualServerURL}/grouper-ws/servicesRest (e.g., https://bsp-dev.projectbamboo.org/grouper-ws/servicesRest) If you are not installing Grouper do not set this environment variable. Bamboo services will interpret the absence of this environment variable to mean that Grouper is not available, and in this case will return messages that more clearly explain why certain functions are failing (e.g., Group service requests; and anything having to do with AuthZ). Absent Grouper, all access to BSP services is treated as anonymous persons accessing from unknown clients, and only "Permit All" policies will allow service requests to be fulfilled. | ||
GROUPER_VERSION | v2_1_002 (use actual version installed) | ||
BSP_HOME | ${INSTALL_DIR}/BSP_0.9 | ||
BSPLOCALSTORE_HOME | This value is OS dependent:
| ||
SERVICEMIX_HOME | ${BSP_HOME}/fuse-esb-7.0.0.fuse-061 | ||
JAVA_HOME | ${BSP_HOME}/Java/jdk1.6.0_38 | ||
M2_HOME | ${BSP_HOME}/apache-maven-3.0.4 | ||
MAVEN_OPTS [1] | '-Xmx512m -XX:MaxPermSize=512m' | ||
BSP_DB_ NAME | BSP_V1_TEST | ||
BSP_DB_PASSWORD | Bamboo_password_01 | ||
BSP_DB_ USER | postgres |
Notes:
[1] MAVEN_OPTS sets the Maven memory settings. Mind the "-" in the value. For Windows, no quotation signs are used, e.g., MAVEN_OPTS value is: -Xmx512m -XX:MaxPermSize=512m
Setting PATH (command path) Environment Variable
Using either the Windows Control Panel or a script, add the following bin directories to the command path (PATH environment variable; noting that Windows variables are referenced using percent-signs, e.g., %JAVA_HOME%):
- ${SERVICEMIX_HOME}/bin
- ${JAVA_HOME}/bin
- ${M2_HOME}/bin
Note: For Windows, the JDK bin directory should be placed in front of "c:\windows\system32" and "c:\windows"; this is because some Windows systems may have an out-dated copy of JDK/JRE in these directories.
These path elements are required to invoke, respectively, the ServiceMix console (karaf), Java, and Maven from a command-line.
Creating the SMX User
The following applies to Unix / Linux environments. Windows environments will use the default logged-in user.
# Create a new user with name smx. This user account should be used throughout the installation and all resources should belong to this user.
Deploying Requisite Software: Java and JVM-deployed
Depending on your platform and the approach taken by each software package provider, the requisite software may be deployed either by extracting a binary distribution or by running an installer program. In the case of a binary distribution, the software is packaged as either a .zip file or a .tar.gz file. When asked to extract the software in the steps below, please use an archive tool (such as Zip or Gunzip) appropriate to your platform to unpack the distribution:
- Download the JDK and run the installation wizard, changing the destination directory to ${BSP_HOME}/Java
- Download Maven, and extract the distribution to ${BSP_HOME}
- Download FuseESB and extract the distribution to ${BSP_HOME}
- Download Eclipse and extract the distribution to ${BSP_HOME}
Validating the Installation
To confirm that the installation has been successful, run the following commands, verifying that the results returned match those expected (note that the expected results below match the versions used in the example above):
Purpose | Command | Expected Result |
Verify JDK | java -version | java version "1.6.0_38" |
Verify Maven3 | mvn -version | Apache Maven 3.0.4 (r1232337; 2012-01-17 00:44:56-0800) |
Verify Eclipse | eclipse.exe | Eclipse should open and display the Java EE perspective |
Verify FuseESB | karaf | karaf "Servicemix" should appear with a prompt of "karaf@root>" |
Configuring FuseESB
The following changes will take effect the next time FuseESB is started.
By default, the CXF servlet in FuseESB is assigned an alias of "cxf". To replace this with "bsp":
- Create a text file org.apache.cxf.osgi.cfg in ${SERVICEMIX_HOME}/etc
- Add entry: org.apache.cxf.servlet.context=/bsp
- add entry (substituting your server name): org.apache.cxf.servlet.base-address = http://${servername}:8181/bsp
Add support for JavaSE 1.6:
- Open ${SERVICEMIX_HOME}/etc/config.properties
- Edit the # OSGi Execution Environment setting to read: org.osgi.framework.executionenvironment=JavaSE-1.6,J2SE-1.5,J2SE-1.4,J2SE-1.3,J2SE-1.2
BSP uses several features which should be installed at boot:
- Open ${SERVICEMIX_HOME}/etc/org.apache.karaf.features.cfg
- Edit the featuresBoot setting by appending ,webconsole,jpa-hibernate
Enable Client Auth:
- Comment out all content in ${SERVICEMIX_HOME}/etc/org.ops4j.pax.web.cfg
- From the Karaf console:
- features:list | grep spring-dm-web
- If spring-dm-web is not installed, execute features:install spring-dm-web
Further Configuration and Installation (including IDE plugins)
The following steps will configure Maven, install PostgreSQL, and install and configure IDE plugins for use in the Developer's Workbench.
Configure Maven
The path to the local repository Maven will use to store artifacts defaults to ${HOME}/.m2/repository (where ${HOME} references the user's home directory for the platform-in-use). To ensure that all build artifacts are stored in the same repository, this setting must be changed to direct Maven to use the BSP-local-store directory:
- Open ${M2_HOME}/conf/settings.xml with a text or XML editor
- Search for "localRepository"
- Replace "/path/to/local/repo" with an absolute path to ${BSPLOCALSTORE_HOME}/mavenRepo/repository (note that you will have to uncomment the line)
- Save and close the file
- Run mvn install from a shell. Note that this command will report a "Build Failure" (i.e. we did not specify a POM file), but in the process of failing will initialize the Maven repository which is the intent. Note that this may take a while to execute.
See the attached Maven settings.xml file (stored as Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections) for additional Maven configuration of if the developer is deploying to a shared – rather than local – Maven repository using mvn deploy. The file is configured appropriately for deploying to the Collection Space instance of Artifactory from which built BSP bundles are available as of April 2013 (i.e., it matches the POMs in the tagged v0.9.0.-ALPHA release of the BSP codebase); YMMV.
Install and Configure Eclipse plug-ins: TestNG and m2eclipse
In order to use Eclipse in the context of the Developer Workbench, you will need to add the following plugins using the Eclipse Install New Software dialog, accessible from the IDE's Help menu:
Purpose | Plugin Name | Update Site |
TestNG integration for Eclipse | TestNG | |
Maven integration for Eclipse | m2eclipse Core | |
Maven Integration for WTP | m2eclipse Extras |
For instructions on how to use the update manager, please search for the Installing New Software topic in Eclipse Help, or visit http://help.eclipse.org/juno/index.jsp?topic=%2Forg.eclipse.platform.doc.user%2Ftasks%2Ftasks-124.htm
The plugins should be installed in the following order (be sure to check the boxes if the plugin is not already selected):
- TestNG
- m2eclipse Core
- m2eclipse Extras. Select "Maven Integration for Eclipse WTP" only
Note you will have to restart Eclipse after each plugin install.
Configure the Eclipse m2eclipse plugin to use the Maven repository at ${BSPLOCALSTORE_HOME}:
- Open Window | Preferences | Maven | Installations
- Add a new installation using an absolute path pointing to ${M2_HOME}
- Make sure "Global settings' points to the absolute path of the settings.xml file modified during the "Configure Maven" step above before clicking OK.
Install and Configure Eclipse svn plug-in: Subclipse
- Subclipse update site URL: http://subclipse.tigris.org/update_1.6.x
- Choose/click alltop-level components:
- Core SVNKit Library
- Optional JNA Library (recommended)
- Subclipse
- Install and, when prompted, restart
- Configure Subclipse
- Go to Eclipse Preferences ( Window | Preferences)
- Navigate to Team : SVN in the Preferences dialog
- Choose the SVN interface: SVNKit (Pure Java) SVNKit v1.3.5.7406
- Accept changes by clicking OK
- Open the SVN Repositories view: (Window | Show View | Other | SVN | SVN Repositories)
- Add a new repository (i.e., the Bamboo Technology Project svn repository):
- Right-click in the SVN Repositories pane
- Navigate to: New | Repository Location
- Enter repository location URL: https://source.projectbamboo.org/svn/btp
- Click Finish
- The following steps assume you have been granted commit permissions in the repository https://source.projectbamboo.org/svn/btp (which is world-readable). Cf. BTP Subversion Repository - Guidelines and Info for more information. Still in the SVN Repositories view of Eclipse:
- Check out a working copy of a repository directory to which you have commit permissions
- Navigate to the desired repository directory, e.g., https://source.projectbamboo.org/svn/btp/_example/trunk
- Right-click on the desired directory
- Choose "Check Out"
- Choose "Check out as a project in the workspace"
- Choose and type an appropriate project name (probably not "trunk"!), e.g., "example-project"
- If appropriate, choose a particular revision, set Depth, etc.; but in general these settings should be left as-is
- Click "Next" and confirm that the chosen workspace location is appropriate (this is where your working copy will be stored on your local filesystem)
- Click finish
- The checked out project should appear as the project name typed above in your Project Explorer (or Resource) view
- Create or modify a file or folder in order to confirm that you can check it into the repository
- Commit the file or folder:
- Right-click the file or folder you wish to commit. [Note:If you are committing multiple files/folders in one operation, you must right click a node that is an ancestor (parent or above) to the multiple files/folders you wish to commit.]
- Enter a commit comment.
- You will be presented with a dialog requesting your credentials (anonymous commits are not permitted on the BTP repository)
- Your credentials are the same as those used to log into the Bamboo wiki (Confluence) or Issue Tracker (JIRA)
- Enter both the UID and Password
- Recommended: click "Save Password" so you need not enter credentials the next time you commit to the repo
- When the icon next to the changed / committed file transforms from a black asterisk (changed locally) to a small yellow cylinder (in synch with repository), your commit has succeeded
- Check out a working copy of a repository directory to which you have commit permissions
Install and Configure PostgreSQL and pgAdmin III
Use of PostgreSQL and pgAdmin in the local database instance has been tested in both the Windows and Linux environments.
These components require a platform specific installation. For Windows the PostgreSQL installer will also install a Microsoft Visual C++ 2010 Redistributable module.
To install in a Windows environment:
- Download PostgreSQL and run the installation wizard, changing the destination directory to ${BSP_HOME}\PostgreSQL\9.2
- if quick and dirty (rather than secure) installation is desired, make the PostgreSQL super-user and password the same as the environment variables BSP_DB_USER and BSP_DB_PASSWORD set above – e.g., user postgres with password Bamboo_password_01
- For Windows users to run PostgreSQL from a console (psql), it is advisable to first execute ${BSP_HOME}\PostgreSQL\9.2\pg_env.bat to set the PostgreSQL environment variables. These are not necessary if the bundled GUI administrative application ${BSP_HOME}\PostgreSQL\9.2\pgAdmin III is utilized for administration.
To install in a Linux environment:
- Install PostgrSQL using YUM
- Install PGAdmin using YUM
- Create a postgres user with password "Bamboo_password_01"
For all environments:
- From the PGAdmin III application in Windows, from the command line, or using another administrative interface, create new database BSP_V1_TEST giving ownership to the user postgres (or, if different, the user set with the environment variable BSP_DB_USER) which has the password (or, if different, the password set with the environment variable BSP_DB_PASSWORD). [See above re: environment variable values.]
- Cf. create-BSP_V1.sql (DDL) in the codebase (${REPOSITORY_ROOT}/platform-services/bsp/trunk/bsp-ddl/src/main/resources/create-BSP_V1.sql)
- It is unnecessary to run the other DDL in the above-referenced directory; it is run on the initial build of BSP. However, if one wanted for some reason to manually re-create the database without performing a build, the DDL can be executed in this order (examples given using command-line syntax):
- psql -f create-BSP_V1.sql
- psql -f person-tables.sql BSP_V1
- psql -f notification-tables.sql BSP_V1
- psql -f cache-tables.sql BSP_V1
- psql -f utility.protected-resource.domain.sql BSP_V1
Deploying BSP
Deploying BSP from built features in a Maven repository
Once you have created a workbench, you can deploy the latest build of BSP directly from a Maven repository to which built bundles and features have been deployed. This option may be preferred to checking out the source code and building it locally (for local build and deploy, see below). As of April 2013, Project Bamboo's BSP bundles are archived at v0.9.0.-ALPHA in a Maven repository maintained by the Collection Space project, at http://nightly.collectionspace.org:8081/artifactory/. (During the active phase of the project, Bamboo maintained its own instance of an Artifactory maven repository, to which bundles were deployed by a continuous integration environment.)
You must have built BSP at least once in order for the test cases to have created the database. Alternately, execution of the DDL described above vis-a-vis PostgreSQL installation creates the necessary database and tables therein.
Initial Setup
Make the following configuration changes to FuseESB (assumes the April 2013 bundles are being deployed from the maven repo indicated immediately above):
- Edit ${SERVICEMIX_HOME}etc/org.apache.karaf.features.cfg:
- Add mvn:org.projectbamboo/bsp.features.build/0.9.0.-ALPHA/xml/features to the ".\" list of featuresRepositories
- Edit ${SERVICEMIX_HOME}etc/org.ops4j.pax.url.mvn.cfg:
- Add http://nightly.collectionspace.org:8081/artifactory/repo@snapshots to the "\"-delimited list of org.ops4j.pax.url.mvn.repositories
- Place policy files in the directory ${BSPLOCALSTORE_HOME}/policies [for a dev machine, use the PermitAllServices.xml policy; cf. Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections for more info]
- Bounce smx (shut down, then restart)
Deploying BSP
- Confirm that the latest version of BSP does not require any configuration or database changes
- Delete any previously hot-deployed BSP jars from ${SERVICEMIX_HOME}/deploy
- Open a Karaf console and execute the following commands:
- features:refreshurl mvn:org.projectbamboo/bsp.features.build/0.9.0.-ALPHA/xml/features
- features:list | grep BSP (to confirm that the BSP feature is available)
- features:install BSP (to deploy BSP)
Example command console output, below, is taken from a Windows installation of FUSE ESB and shows deployment of the BSP v0.9.0.-ALPHA features:
D:\bsp_0.9\>karaf [...] FuseESB:karaf@root> features:refreshurl mvn:org.projectbamboo/bsp.features.build/0.9.0.-ALPHA/xml/features FuseESB:karaf@root> features:list | grep BSP [uninstalled] [0.9.0.-ALPHA ] BSP repo-0 FuseESB:karaf@root> features:install BSP FuseESB:karaf@root>
Deploying BSP from local Maven repository
Once you have created a workbench, you can check out code, build, and deploy BSP directly from your local Maven repository. The assumption in the examples below is that code is checked out from trunk, where the version in Maven POM files is set to 0.9.0.-SNAPSHOT.
Initial Setup
Make the following configuration changes to FuseESB:
- Edit ${SERVICEMIX_HOME}etc/org.apache.karaf.features.cfg:
- Add mvn:org.projectbamboo/bsp.features.build/0.9.0.-SNAPSHOT/xml/features to the ".\" list of featuresRepositories
- Edit ${SERVICEMIX_HOME}etc/org.ops4j.pax.url.mvn.cfg:
- Remove the repository http://nightly.collectionspace.org:8081/artifactory/repo@snapshots from the "\"-delimited list of org.ops4j.pax.url.mvn.repositories (assuming it was added per steps describing deployment from built bundles in a Maven repo, above)
- Place policy files in the directory ${BSPLOCALSTORE_HOME}/policies [for a dev machine, use the PermitAllServices.xml policy; cf. Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections for more info]
- Bounce smx
Build BSP
- Confirm that the latest version of BSP does not require any configuration or database changes
- Download appropriately-tagged codebase, e.g., trunk, into an Eclipse workspace project, e.g. workspace/${MyBSPdownload} (or other local location)
- Run the following Maven install targets from the IDE or command line. Command line given below:
- ${MyBSPdownload}/bsp>mvn install
- ${MyBSPdownload}/bsp-features/bsp-features>mvn install
Deploying BSP
- Delete any previously hot-deployed BSP jars from ${SERVICEMIX_HOME}/deploy
- Open a Karaf console and execute the following commands:
- features:refreshurl mvn:org.projectbamboo/bsp.features.build/0.9.0.-SNAPSHOT/xml/features
- features:list | grep BSP (to confirm that the BSP feature is available)
- features:install BSP (to deploy BSP)
Confirming deployment of BSP services
- View services deployed in ServiceMix at http://localhost:8181/system/console/bundles (default user / password is smx / smx)
- View WADLs of deployed BSP services at http://localhost:8181/bsp [see first code block below]
- A fresh database will return an empty set of persons in response to the request http://localhost:8181/bsp/persons [see second code block below]
Available SOAP services: Available RESTful services: Endpoint address: http://localhost:8181/bsp/requests WADL : http://localhost:8181/bsp/requests?_wadl Endpoint address: http://localhost:8181/bsp/protectedresources WADL : http://localhost:8181/bsp/protectedresources?_wadl Endpoint address: http://localhost:8181/bsp/serviceprofiles WADL : http://localhost:8181/bsp/serviceprofiles?_wadl Endpoint address: http://localhost:8181/bsp/contacts WADL : http://localhost:8181/bsp/contacts?_wadl Endpoint address: http://localhost:8181/bsp/notifications WADL : http://localhost:8181/bsp/notifications?_wadl Endpoint address: http://localhost:8181/bsp/groups WADL : http://localhost:8181/bsp/groups?_wadl Endpoint address: http://localhost:8181/bsp/persons WADL : http://localhost:8181/bsp/persons?_wadl Endpoint address: http://localhost:8181/bsp/resultsetcaches WADL : http://localhost:8181/bsp/resultsetcaches?_wadl
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <person:profileList xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:bsp="http://projectbamboo.org/bsp/resource" xmlns:contacts="http://projectbamboo.org/bsp/services/core/contact" xmlns:person="http://projectbamboo.org/bsp/BambooPerson" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <bsp:listMetadata> <dcterms:creator xsi:type="foaf:Agent" rdf:about="persons"/> <dcterms:created xsi:type="dcterms:W3CDTF">2013-04-10T15:59:05.403-07:00</dcterms:created> <bsp:orderedBy>ascending</bsp:orderedBy> <bsp:listLength>0</bsp:listLength> <bsp:pageNumber>1</bsp:pageNumber> <bsp:pageLength>20</bsp:pageLength> </bsp:listMetadata> </person:profileList>
Additional Steps
Install and Configure Apache Web Server (httpd) to enforce Client Auth (trusted client access)
A secured BSP instance – one that enforces policy and accepts requests from trusted clients – requires that access be redirected through Apache Web Server (httpd). Apache Web Server enforces secure (HTTPS) connections over SSL; handles authentication of trusted application via pre-exchanged X.509 certificates; and directs traffic to the BSP over the AJP connector port through which FUSE ESB is configured to handle security.
Instructions for configuration of Apache Web Server on the FUSE ESB host server, and for generation and exchange of certificates with clients, can be found on the page Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections.
Install and Configure Grouper - as persistence store for Group and Trusted Application data
In phase one of the Bamboo Technology Project, Internet2's Grouper was selected for use as a persistence store for data about user-created groups, and for 'application catalog' data (records of applications registered in the Bamboo Trust Federation, cf. Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections).
A secured BSP instance – one that enforces policy and accepts requests from trusted clients – requires that Grouper be available (installed) and configured as described on the page Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections. In this case, the environment variable GROUPER_URL must be set in the context where FUSE ESB is running, and must point to the Grouper installation (cf. the Setting Environment Variables section of this page).
If Grouper is not installed (because the BSP instance is intended to be run as world-accessible, without security), it is essential that the environment variable GROUPER_URL not be set.
Important notes for service developers
Unit Tests require maintenance
Overview
A number of unit tests in the v0.9-alpha BSP code base will pass only if expected data is found in stores of Bamboo Person data, registered application data, and group data. These data are stored in:
- a database backing the Person Service (and other BSP-deployed services);
- the Grouper instance that stores 'Application Catalog' data in the stem bamboo:projects; and,
- the (same) Grouper instance that stores user-created groups in the bamboo:groups stem.
The 'expected data' in the unit tests as committed to the BSP codebase was extant in the shared instances of the BSP and Grouper that were in-use as of March 2013. These instances have been decommissioned. Thus, for unit tests to pass a developer adopting this codebase must assume certain responsibilities vis-a-vis reinstantiating backing data or modifying the unit test source code.
An alternative to building the BSP bundles is to deploy built bundles at their v0.9-ALPHA release. See Built bundles available for deployment, below.
Examples
The Group Service test in FindGroupTest.java assumes the existence of data hard-coded into the class in a set of static final strings:
//This data reflects the state of the Grouper instance on services-qa.projectbamboo.org 01.31.2013 private static final String BSP_TEST_APP_UUID = "2c38ef3775a948719388ebdac1cf3d78"; private static final String BAMBOO_STEM = "458930f0-0716-11e2-892e-0800200c9a66"; private static final String ADMIN_STEM = "41f062e2-156e-457e-8371-4131c60a6976"; private static final String APPS_STEM = "020494e9-7e84-4442-bcbb-6dc299a2df45"; private static final String BSP_TEST_APP_ID = "4e9d6f57-f68d-49eb-bcc4-200f56e7f4d2";
Similarly, the Group Service tests in GroupProxyClientMethodsTest.java assume existence of these data:
//Test data consistent with Grouper on services-qa circa 02.26.2013 private static final String UNIT_TEST_GROUP_ID = "4e9d6f57-f68d-49eb-bcc4-200f56e7fbfb"; private static final String UNIT_TEST_STEM_ID = "4e9d6f57-f68d-49eb-bcc4-200f56e7fcfc"; private static final String FARKLE_SUBJECT_ID = "urn:uuid:c00f294d-651b-495b-9633-a900dd1a7418"; private static final String ADAMS_SUBJECT_ID = "urn:uuid:04574969-e851-4fbf-8535-0b81b9b3f2f3";
Tests for the Request Manager service found in PatternsTest.java within that service's codebase assume existence of these data:
private static final String SOURCEDID_RESOURCE = "/bsp/persons/urn:uuid:a486e4fd-38e6-4915-8c11-c9dc00963f5f/sourcedids/urn:uuid:99999999-9999-9999-9999-999999999999"; private static final String PERSONID_RESOURCE = "/bsp/persons/urn:uuid:c00f294d-651b-495b-9633-a900dd1a7418";
Options
Developers who wish to build BSP in the future have several options:
- execute builds without executing unit tests
- manually re-create expected data in new instances of BSP databases and Grouper
- re-create a base of users and groups with which to execute tests, and appropriately modify values hard-coded in the unit test classes
- contact archivists@lists.projectbamboo.org to inquire about availability of PostgreSQL database dumps of March 2013 from which newly-instantiated instances of BSP and Grouper might be populated (cf. Resources for bootstrapping a fully-functional dev environment, below)
Any of these options entail work and/or risk. These are necessarily responsibilities that a developer assumes in adopting the Bamboo codebase. Also see Built bundles available for deployment, below.
Built bundles available for deployment
BSP bundles are built and available for deployment from a Maven repository as described in Deploying BSP, above. As of April 2013, these bundles are hosted by the Collection Space project's Maven repository.
Resources for bootstrapping a fully-functional dev environment
Overview
Anticipating future interest in setting up a shared developer deployment environment such as that used during the active period of the Bamboo Technology Project (October 2010 to March 2013), a number of database dumps and disk images were preserved for future use. Please contact archivists@lists.projectbamboo.org to inquire about availability of these data to projects and developers known to the Project Bamboo community.
Extant disk images and database dumps
The following data were archived in March 2013:
- Grouper database dumps to reconstitute Grouper 2.1.2, including a Subject Authority database used as an interim solution in advance of development of a Grouper Subject Adapter to the Bamboo Person Service.
- BSP database dump to reconstitute the set of Bamboo Persons, profiles, etc. that complement the Grouper data referenced in the prior bullet. There are some otherwise-useless artifacts of manual tests of Bamboo services in this database, so it is a superset of what's needed to complement Grouper data and successfully run unit tests.
- Disk image of a Fedora 16 64-bit Linode VM on which FUSE ESB and Grouper were installed and populated with the data described in the prior bullets (~6.2 GB)
- Disk image of a CentOS 6.2 64-bit Linode VM on which the PoC "Account Services Module" was deployed in a Drupal instance configured as a trusted client application in the Bamboo Trust Federation (~3.1 GB)
Notes on instantiating BSP on (Linode) virtual machines
Linode Usage Overview
Several generations of BSP installations and installations of other applications (Grouper, the Continuous Integration server Atlassian Bamboo, an Artifactory Maven repository) were deployed on Linode virtual machines during the active period of the Bamboo Technology Project. These VMs were built atop Fedora 16 in the latest instances, and were built to a standard recipe. That recipe would be idiosyncratic and out-of-date to any future deployer, and so is not included in this documentation set. Instead, a few key notes and artifacts are published here in the hope that they may prove useful to deployers using newer software.
Notes
- Apache httpd (Apache web server) is required and must be set up to participate in client auth interactions, as described in the wiki page Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections
- Environment variables, PostgreSQL, and FUSE ESB must be installed and configured as described on this page (above), with appropriate OS-specific tweaks
- Appropriate filesystem directories must be created and given appropriate permissions (e.g., to the unix user under which FUSE ESB will run; smx was the unix user used by the Project Bamboo team
- Unix utilities sendmail and mailx were installed on the Linode instances to handle e-mail traffic initiated by the BSP's Notification Service
- Firewall rules blocked ports other than 80 and 443, in order to protect against attacks; traffic to BSP, Grouper, Atlassian Bamboo, Artifactory, etc., was routed through the Apache Web Server and an AJP connector. (In unsecured deployments of the BSP, port 8181 was left open for direct, unencrypted traffic to BSP services.)
- See note on FUSE ESB / Service Mix startup scripts, below
Startup Scripts for FUSE ESB (Service Mix)
On Project Bamboo Linode VMs, Service Mix was started as a service by utilizing scripts adapted from scripts created at UC Berkeley for that institution's local instantiation of FUSE ESB instances. Note that the file locations described below are particular to the Fedora 16 OS on which they were invoked. The scripts are attached in TXT files, as linked below.
Install fuse-smx scripts
Copies are attached to this wiki page of the two scripts
- /etc/rc.d/init.d/fuse-smx: Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections
- assure that script is executable (0755, executable by user, group, and other)
- /etc/sysconfig/fuse-smx: Developer Workbench Environment for BSP Service Developers -- Fall 2013 corrections
- assure that the config file is readable by u, g, o (0644)
The file /etc/rc.d/init.d/fuse-smx was modified to comment-out (and in some cases replace) elements specific to the UC Berkeley VM installations for which the scripts were created. Cf. comments in the attached copy.
Enable fuse-smx startup script with chkconfig
sudo /sbin/chkconfig --add fuse-smx sudo /sbin/chkconfig fuse-smx on