Thursday, October 13, 2011

Platform Update - Consolidation & Cleanup

I wanted to take a few minutes to detail the progress in the platform projects. As most of you know, we have gone through a round of project consolidation in the platform. The new projects are as follows:

===================================================
api - unchanged

core

- bi-platform-engine-core
- bi-platform-engine-security
- bi-platform-engine-services
- bi-platform-test-foundation
- bi-platform-ui-foundation
- bi-platform-util
- bi-platform-security-userroledao

The engine-* projects had previously enjoyed a GPL license, this remains unchanged, however, the license the projects brought into the core is now GPL if not already.

repository - relatively unchanged (this is the JCR)

scheduler - we have dropped the old scheduler, scheduler2 takes over

extensions
- bi-platform-web
- bi-platform-web-servlet
- bi-platform-plugin-actions
- bi-platform-plugin-services

user-console - though future plans for mantle include making it a plugin, it is out of the old bi-platform-v2/trunk and renamed

assembly
- bi-platform-appserver
- bi-platform-assembly
- bi-platform-build
- bi-platform-sample-data
- bi-platform-sample-solution

We have also removed several projects, such as portlet, legacy, and test-solution.

The dependency order is also, as I have listed the projects:
api, core, repository, scheduler, extensions, user-console
===================================================

All of the new projects produce new ivy artifacts, as to not overwrite anything already in artifactory. We have incorporated changes to subfloor allowing the publishing of test jars. For example, the core project's test-src contains classes essential for testing in many downstream projects.

api - pentaho-platform-api.jar
core:
- pentaho-platform-core.jar
- pentaho-platform-core-test.jar (BaseTest and friends)
repository:
- pentaho-platform-repository.jar
- pentaho-platform-repository-test.jar
scheduler - pentaho-platform-scheduler.jar
extensions - pentaho-platform-extensions.jar
user-console - pentaho-user-console.jar (typically resolved is the package/zip)

With the introduction of the JCR the notion of "solution" "path" "name" goes away and we simply have a single path to the resource "/path/to/the/resource". I have modified the API of SolutionEngine so that it only accepts this single path element and fixed all (known) downstream implications.

Unit tests are in outstanding shape right now and cobertura coverage is being published. I spent 2 days doing nothing but getting unit tests working again. Many of the unit tests which are now working have not been working since 2009 (some even earlier than that). Plenty of unit tests were commented out, where possible, I have uncommented these and they are now working. The unit test summary:

core - 100% unit test pass
repository - 100% unit test pass
scheduler - 100% unit test pass
extensions - out of 406 unit tests, 5 are failing, these failures are the real deal, they warrant investigation (they might have caught something)

So for the entire BI platform, there are only 5 unit tests which are failing! We're going to watch these projects carefully and keep the unit tests passing (and squash any failures). Over time we'll be adding new tests, and increasing the code coverage.

The assembly has been dramatically cleaned up and simplified. The assembly project does not require the checkout of any other projects, it has all the resources required to lay down the solutions, data, appserver, etc as well as ivy dependencies on package archives of other projects. You can simply checkout the assembly project, run an ant build and a BI-SERVER build will be created.

CI is running jobs for each of the new projects with the new build order in place. These can be seen in the SUGAR grouping:
http://ci.pentaho.com/view/Sugar/

Lastly, I have officially "closed" bi-platform-v2/trunk the new location in SVN for these projects is:
svn://source.pentaho.org/svnroot/pentaho-platform/trunk
The bi-platform-v2/trunk has been renamed as trunk-closed to further discourage any checkins to the wrong location.

Any SUGAR development and 4.1/4.5 fixes should be incorporated in the new project structure (though the project names have remained the same, using the summary I listed above you can easily find where to checkin).

2 comments:

  1. I tried to get source from the URL specified svn://source.pentaho.org/svnroot/pentaho-platform/trunk which is not accessible. I want build and debug this new project.
    While i am doing i get error :
    Unable to connect to a repository at URL 'svn://source.pentaho.org/svnroot/pentaho-platform/trunk'
    Reason: Can't read from connection: An existing connection was forcibly closed by the remote host.

    ReplyDelete
  2. You have to use an SVN client to read the URL. You can switch the svn to http if you want to browse.

    ReplyDelete