Documentum High Volume Server and Lightweight SysObjects – Lessons Learned

We just helped complete a major infrastructure effort for one of our clients leveraging the High Volume Server offering from Documentum .  For this post, we will share lessons learned.

Background

High Volume Server (HVS) has three major components

  • Batch Operations and Batch Operations and Currency Scoping – provides an efficient mechanism for ingesting and working with large numbers of documents in batches
  • Data Partitioning – allows Documentum to leverage the database’s partitioning features, in particular allowing for objects to be loaded into an “offline” partition, then swapped into Documentum very quickly.
  • Lightweight SysObjects – allows the database to represent several documents with similar metadata in a very efficient format

Our client opted not to use Batch Operations and Currency Scoping as it didn’t really apply to their implementation.  The remainder of this article will focus on Data Partitioning and Lightweight SysObjects.

Data Partitioning

For those unfamiliar with the concept, partitioning is the process of breaking a large database into smaller chunks.  Given that smaller chunks are more efficient than large chunks, typical expensive database functions, like loading data, can be separated into multiple smaller jobs and ingested into the partitioned database quicker with less overhead.

Data Partitioning in HVS allows client to load a large number of objects into an “offline” table, then perform a “partition exchange” to bring them into Documentum instantly.   Most of the functionality is actually provided by the underlying database, which must have its own partitioning enabled.  Documentum provides a method to “reserve” a range of r_object_id values, so that the loading program can assign them as it prepares the offline tables.  Documentum provides fairly detailed documentation on exactly which tables need to be handled, including the super-secret DMI_OBJECT_TYPE table.

One thing we learned from our client experience was that the Oracle scripts Documentum generates to put Data Partitioning in place are often incomplete.  Our client chose to use the supplied scripts as a starting point, then have a DBA write his own scripts.

TSG would definitely recommend a number of testing iterations before trying partitioning on any production system; while a Documentum team can normally ignore the database-level operations, in this case an intimate knowledge of how Documentum represents its objects in the database is essential to making sure the partitioning is implemented correctly.    Though HVS Partitioning is new to D6, database partitioning has been around for some time and is a “known quantity”.  Since most of the actual functionality of HVS DP is provided by the database, TSG recommends clients implement this if their use case suits it and the client feels comfortable enough with their DBA support of partitioning.

Lightweight SysObjects

Lightweight Sysobjects (or LWSO)  use a “shared parent” object, which is a “regular” object in the docbase; and 0ne or more “lightweight child” objects, which don’t have their own metadata but instead refer to their parents.  As an example, if documents of a certain group, say invoices, will all have the same security, LWSO could be used to save database space by not assigning an ACL for every object in the database.  LWSO reduces the storage requirements for the child objects in the database.  The use case for these is really limited to certain types of business documents: many documents with shared metadata; and documents that will never me modified once ingested.

If a child object changes (checked-out for an example), it is “materialized”, undoing its savings.  LWSOs cannot be versioned.  It is important to understand that LWSO-specific API calls are needed to manipulate LWSOs

  • Store – IDfSession has a new method for creating a LWSO: newLightObject
  • Update – care must be taken when operating on child objects, as certain “normal” API calls will cause the objects to be materialized (e.g., setStorageType and setContentType, link and unlink, checkout and checkin)
  • View – LWSOs are not visible in folders using standard client tools like Webtop
  • Process – LWSOs cannot be routed or forwarded, nor can they have lifecycles attached

Many client tools, including Webtop and DA, cannot properly display LWSOs and, as of the date of this post, Captiva does not support LWSO either.

As LWSO is deployed at fairly few Documentum customers, there is not yet a large knowledge base to draw from in order to debug and diagnose issues that may arise.

If you would like any additional information about our experience, please comment or send us an email with your thoughts.