HOWTO: Configure Farrago Repository Storage

The metadata in Farrago's system catalog is stored in an instance of an MDR repository. MDR supports a storage plugin system for extensibility; by default, Farrago uses a JDBC connection to an in-process HSQLDB engine.

This document describes how to configure other kinds of repository storage. You might well ask why Farrago relies on external storage for its metadata instead of providing the storage itself. Indeed, self-storage is the eventual goal, but at the moment Farrago still lacks some of the necessary SQL support. (Also, the bootstrapping challenge is non-trivial.) Even once self-storage is available, external storage is still a useful capability in cases such as those listed below.


When might you want to configure custom repository storage?

Storage Properties

The Farrago build script configures repository storage by reading the file dev/farrago/catalog/, which is in Java properties file format. If this file does not exist when the build starts, a default version is copied from dev/farrago/catalog/templates/ The file is also used at runtime to access the repository. Let's take a look at the default:

# /home/jvs/open/farrago/catalog/
# Repository storage properties for hsqldb

# Class name of MDR storage plugin factory

# JDBC driver for hsqldb\

# URL for database storage\

# user name

# password

# schema name (used as a table prefix)
What do these properties mean?

Build Configuration

Now, let's suppose you want to customize Farrago to store its catalog in a PostgreSQL server. Before starting, make sure that you can access your PostgreSQL server via JDBC, and record information such as the URL, username, and password. Here is an example using sqlline:

jvs@jackalope:~/open/test$ java -cp \
/home/jvs/postgresql-7.4.6/src/interfaces/jdbc/jars/postgresql.jar \
sqlline.SqlLine \
-u jdbc:postgresql://localhost/test \
-d org.postgresql.Driver \
-n postgres \
Connecting to jdbc:postgresql:test
Connected to: PostgreSQL (version 7.4.6)
Driver: PostgreSQL Native Driver (version PostgreSQL 7.4.6 JDBC3 with SSL (build 215))
Autocommit status: true
sqlline version 1.0.0-jvs-1 by Marc Prud'hommeaux
0: jdbc:postgresql:test> select * from information_schema.schemata;
| catalog_name  |     schema_name     | schema_owner  | default_character_set_ |
| test          | pg_toast            | postgres      |                        |
| test          | pg_temp_1           | postgres      |                        |
| test          | pg_catalog          | postgres      |                        |
| test          | public              | postgres      |                        |
| test          | information_schema  | postgres      |                        |
5 rows selected (0.068 seconds)

The next step is to check the dev/farrago/catalog/templates directory for a suitable template. Not surprisingly, there's already one defined for PostgreSQL, and it just happens to match the connection information above. Copy it into the parent directory (dev/farrago/catalog) and rename it to If a file with that name already exists because you previously built Farrago with the default HSQLDB storage, you can overwrite it (it was copied verbatim from the templates directory by the build).

Now, edit to match your configuration. At this point, it's a good idea to verify your settings. The Farrago build script provides an ant target named verifyReposSqlStorage for just this purpose. Let's try it now:

jvs@jackalope:~/open/farrago$ ant verifyReposSqlStorage
Buildfile: build.xml


/home/jvs/open/farrago/build.xml:1411: Class Not Found: JDBC driver org.postgresql.Driver could not be loaded

Total time: 3 seconds
Ooops! We forgot to tell the system the location of the jar containing the driver, and it's not one of the default thirdparty components. So how do we do this? The Farrago build system supports custom properties via an optional file dev/farrago/ And the specific property we need to set is named farrago.custom.classpath. So, create that file now, with contents like:

# /home/jvs/open/farrago/

# Tell Farrago where to find the PostgreSQL JDBC driver

# Tell Farrago to preserve our customizations
Now try again:

jvs@jackalope:~/open/farrago$ ant verifyReposSqlStorage
Buildfile: build.xml



     [echo] Successfully connected to jdbc:postgresql://localhost/test

Total time: 3 seconds
Passing this test does not guarantee a good configuration, because it just connects without trying to execute any SQL statements. However, it can save time in eliminating basic connectivity problems.

Now, run the command ant createCatalog. This should rebuild the Farrago catalog, storing it in PostgreSQL as requested. If you get a build error, it probably means you have a deeper configuration problem; see the debugging section below for help. Otherwise, you're good to go. Runtime scripts such as farragoServer should work automatically, because ant createCatalog emits the customized classpath into dev/farrago/classpath.gen for use by these scripts.

Let's take a look at what happened behind the scenes. Returning to sqlline:

0: jdbc:postgresql://localhost/test> select * from information_schema.schemata;
| catalog_name  |     schema_name     | schema_owner  | default_character_set_ |
| test          | pg_toast            | postgres      |                        |
| test          | pg_temp_1           | postgres      |                        |
| test          | pg_catalog          | postgres      |                        |
| test          | public              | postgres      |                        |
| test          | information_schema  | postgres      |                        |
| test          | FarragoCatalog      | postgres      |                        |
6 rows selected (0.013 seconds)
Aha, there's our FarragoCatalog schema. What's inside?

0: jdbc:postgresql://localhost/test> select * from information_schema.tables where table_schema='FarragoCatalog';
| table_catalog  |  table_schema   |    table_name     | table_type  | self_re |
| test           | FarragoCatalog  | MOFID_SEQ         | BASE TABLE  |         |
| test           | FarragoCatalog  | PRIMARY_INDEX     | BASE TABLE  |         |
| test           | FarragoCatalog  | Contexts26        | BASE TABLE  |         |
| test           | FarragoCatalog  | ObjectsByClasses  | BASE TABLE  |         |
| test           | FarragoCatalog  | Properties        | BASE TABLE  |         |
| test           | FarragoCatalog  | aibn_1057         | BASE TABLE  |         |
| test           | FarragoCatalog  | aicp_1057         | BASE TABLE  |         |
| test           | FarragoCatalog  | ae_1057_1086_1    | BASE TABLE  |         |
| test           | FarragoCatalog  | ae_1057_1086_2    | BASE TABLE  |         |
| test           | FarragoCatalog  | ae_1057_1087_1    | BASE TABLE  |         |
| test           | FarragoCatalog  | ae_1057_1087_2    | BASE TABLE  |         |
Blech. If you were hoping to be able to query this schema directly for metadata, forget about it. The MDR model-to-table mapping is very low level as you'll find out if you try querying some of those tables. For SQL access to metadata, use Farrago.

Debugging Catalog Creation

When catalog creation fails, MDR usually suppresses the real error information and gives back a generic unhelpful message such as "Failed accessing storage factory." To dig out the real error message, add the following line to your file:
Then retry the failed catalog creation. Buried in the output should be some useful details, e.g.

     [java]       [mdr] org.postgresql.util.PSQLException: Backend start-up failed: org.postgresql.util.PSQLException: FATAL: user "digres" does not exist

     [java]       [mdr]         at org.postgresql.jdbc1.AbstractJdbc1Connection.openConnectionV3(
     [java]       [mdr]         at org.postgresql.jdbc1.AbstractJdbc1Connection.openConnection(
     [java]       [mdr]         at org.postgresql.Driver.connect(
     [java]       [mdr]         at java.sql.DriverManager.getConnection(
     [java]       [mdr]         at java.sql.DriverManager.getConnection(
     [java]       [mdr]         at org.netbeans.mdr.persistence.jdbcimpl.JdbcStorage.<init>(
     [java]       [mdr]         at org.netbeans.mdr.persistence.jdbcimpl.JdbcStorageFactory.createStorage(
     [java]       [mdr]         at org.netbeans.mdr.storagemodel.MdrStorage.<init>(
     [java]       [mdr]         at org.netbeans.mdr.NBMDRepositoryImpl.initCheck(
     [java]       [mdr]         at org.netbeans.mdr.NBMDRepositoryImpl.beginTrans(
     [java]       [mdr]         at org.netbeans.mdrant.MdrTask.execute(MdrTask.j
Once you have fixed the problem, delete the debug setting to avoid spurious printing on standard error at runtime.


As a developer convenience, the Farrago build system provides the ant targets backupCatalog and restoreCatalog to perform offline backup of the catalog and database. However, it only works on files stored in the dev/farrago/catalog directory, which is where the btree and HSQLDB files are stored. So if your repository is stored off in another database, these commands won't work. Instead, you need to use the backup/restore facilities of your DBMS.

If you try to re-run ant createCatalog again after a successful installation, you will probably get an error such as "Package extent named 'FarragoMetamodel' already exists". Normally, ant createCatalog avoids this by blowing away the repository storage in the catalog directory first, but since the repository is now stored elsewhere, this doesn't work. You have to do it manually:

0: jdbc:postgresql://localhost/test> drop schema "FarragoCatalog" cascade;
1 row affected (1.221 seconds)


So far, this document has assumed the context of a Farrago developer build. How does all of this get packaged up and delivered to an end user? At the moment, there's no easy path. A manual process would be to perform a database export from the external server after a complete build and packaging the export file together with Then an installation program would import the repository database and tweak with site-specifics.

What's missing is a way to recapitulate the build process as part of Farrago installation, so that nothing storage-specific needs to be prepackaged. This should be fairly trivial, since the ant tasks used by the build are just thin wrappers around MDR API calls. This should be taken into consideration when developing the currently non-existent Farrago installer.