New OpenMigrate Tutorials Available in TSG Learning Zone

TSG’s open source migration framework, OpenMigrate, offers extensive configuration and customization features.  However, with this flexibility comes some complexity.  OpenMigrate’s distribution includes nearly 20 sample migration projects, intended to get folks up and running quickly.  But sometimes even getting the environment configured properly can be a nuisance.

To address this issue, we’ve created several new tutorial walk-throughs in the TSG Learning Zone.  Each takes the user through the process of installing OpenMigrate, modifying the necessary batch and configuration files, and executing a sample migration.  We’ve include both Documentum and SharePoint samples.

Are you having trouble getting started with OpenMigrate?  Let us know in the comments, or via email at openmigrate@tsgrp.com.

OpenMigrate 1.7 Released: Enhanced Support for Documentum, Alfresco and SharePoint Migrations

With the new year comes exciting new product updates, and the  release of TSG’s migration framework, OpenMigrate version 1.7, leads the way. This release of OpenMigrate includes some new features as well as enhancements to existing features, as described after the jump.

Continue reading

Documentum 5 vs. 6, Databases and Dates: Does Anybody Really Know What Time It Is?

When EMC revealed Documentum 6 a few years ago, they made a subtle—yet important—change to the way Documentum would store dates and times in the database. While most clients would never even notice the change, some of us would. In particular, when it becomes necessary to access the underlying database tables, like OpenMigrate can when preserving modification dates, confusion can be the order of the day.
Continue reading

Migrating From Documentum with OpenMigrate: Best Practices

OpenMigrate, TSG’s open source migration framework, supports migration to and from a number of ECM repositories, including Documentum, Alfresco, FileNet and SharePoint (and many others).

We’ve designed OpenMigrate to perform several different types of migrations:

  • Between different instances of the same ECM product (e.g., during an upgrade)
  • Out of one ECM technology into a “neutral zone” (usually a file system and database or text files for metadata)
  • From a neutral zone into an ECM technology
  • Out of one ECM technology directly into another (e.g., Documentum to Alfresco)

While the general pattern of migrating content and metadata is consistent across all of these migration types,

Continue reading

Migrating From FileNet to Documentum: Could OpenMigrate Possibly Do That?

Frequent readers of the TSG Blog know OpenMigrate is our flexible open source migration framework; that it can be configured or extended to move content and data between different types of repositories; and that it’s been used in numerous successful migrations with Documentum, Alfresco, Qumas, Hummingbird and many others.

But perhaps you’re wondering: “There’s no way it could migrate from an old FileNet IDM system, could it?  I mean, that system was built for old optical disk storage devices!  I heard there was a C API but that people had mixed results trying to use it for high-volume use cases.  It’s practically older than Java and the web; and certainly much older than web services.  Surely you’ll tell me it can’t be done?”

Well, I’m happy to report that TSG has two successful FileNet IDM migrations under our belts, both using OpenMigrate.  For the first, we migrated 1M files and data and made minor customizations to the framework; for the second, we migrated just over 2M, and ran OpenMigrate out of the box.

Migration Features

Some features of both migrations include:

  1. Selection of documents and their metadata using specific rules for each FileNet document class
  2. For picklists (FileNet menus): inclusion of codes, descriptions or both
  3. Translations of FileNet dates
  4. Concatenation of multi-page image files into single PDF files
  5. Binary concatenation of very large files (e.g., SAP Archive files)
  6. Migrations executed in “platter order,” in order to minimize optical disk swapping
  7. Two migration phases: a “pre-cutover” bulk migration in the weeks prior to cutover; and a final “delta” migration extracting the remaining documents and metadata
  8. Robust error recovery

In both instances, we executed OpenMigrate with 6 threads during business hours and 15 threads overnight in order to minimize the impact to interactive users.

Technical Approach

After studying the underlying FileNet database structure and the system tools available for retrieving content, TSG determined the most cost-effective approach for implementing a migration from FileNet.  Rather than attempting to integrate the Java-based OpenMigrate with the dated C API, we configured OpenMigrate to query the underlying database tables directly.  And to retrieve the content, we used a few key system tools installed on the FileNet server, scripted and controlled by a single OpenMigrate component.

The details….

  1. The JDBC Queue Populator builds the migration’s “To-Do List” from FileNet’s underlying DOCTABA table (joining against the menu item table if appropriate for the doc classes being migrated).  It includes built-in and business metadata in the query.
  2. Each OpenMigrate Source thread uses the FileNetCsmContentLoader component to retrieve the document from FileNet.  It uses a variety of FileNet system tools to extract the image or images, determines whether to concatenate the images using iText, and translates FileNet dates to Java dates.
  3. The Mapping layer performs any metadata translation in order to prepare the document for import into the target system (e.g., set the r_object_type in Documentum based on the FileNet doc class).
  4. The Target threads write the content and its metadata to the target repository.

The resulting migration approach ended up being refreshingly straightforward and repeatable; as I noted above, our second migration was able to use the custom component out of the box.

OpenMigrate: Bulk Load Interface Available for Download

In the content management world, users often require an “all-in-one” interface to help them assemble batches of related documents, tag them with metadata , then import them into the underlying Content Management System.  Traditional web-based interfaces, such as Webtop or Web Publisher from Documentum, Explorer or Share from Alfresco, don’t offer this functionality out-of-the-box.

Could TSG’s OpenMigrate open source migration framework fit this need?

OpenMigrate is typically used in a batch mode, especially in high-volume situations.  And while the browser-based Administrator does allow for interactive configuration and execution of migrations, the actual tagging of documents with metadata is outside of its realm.

To bridge this divide, TSG is pleased to announce the availability of the new OpenMigrate Bulk Load Interface for download.

OpenMigrate Bulk Load Interface

The OpenMigrate Bulk Load Interface provides a highly configurable user interface which enables the user to easily manage documents and import them into a variety of targets.  The interface is built on top of the Spring framework making it extremely easily maintainable, customizable, and configurable.

To learn more about this versatile new entry in the OpenMigrate toolset, please visit www.tsgrp.com.