We have recently completed a project to create a single search interface to retrieve both Documentum and SharePoint content. This post will present a high-level overview of other SharePoint search interface alternatives and why we chose the integration approach.
Attended another roadmap session focused on next 18 months. This post will try to summarize and present TSG thoughts.
The agenda for Momentum, the Documentum portion of EMC World, has recently been posted (http://www.emcworld.com/images/Momentum_2011_Agenda-master4webpage.pdf). For this post, we thought we would highlight things we (TSG) or our typical clients attend. Overall, EMC has divided the sessions into “tracks” focused on Architecture/Best Practices, ECM, Software Developer, Information Governance, Case Management, User Group, Capture and Labs. Continue reading
Often times Documentum users, frustrated with Webtop Search or Advanced Search will request “Can we just have a Google Search?”. This post will provide input to Documentum developers on how to best address this ongoing request. While this post is specifically focused on Documentum developers, lessons learned about interface design apply to our Alfresco and SharePoint readers as well.
Typically Documentum users will have heard Ed speak at Momentum or EMC World in the past in regards to system performance and Documentum. For this post, we are going to take a different approach and talk about Documentum Search APPLICATION Tuning for system and USER performance. All of the concepts relate to our Alfresco and SharePoint clients as well as the Documentum users. The experience is based on a recent client but we will try to bring in best practices from other clients as well. Continue reading
One last reminder – Midwest Documentum User Group (MWDUG) is coming up this Friday, October 29th, in downtown Chicago – presentations will include American Electric Power, United Airlines, Documentum Roadmap as well as others. There are still spots open if you are interested in attending. http://www.mwdug.com
As we were talking to users outside of the Chicago area about MWDUG, it isn’t hard to see that travel is still being restricted during the downturn (or post-downturn). Obviously, not many of us could travel to Lisbon, Portugal for Momentum 2010 this month. One thing unique this year is that
EMC/Documentum has opened up the ability to attend Momentum virtually. Slide decks and presentations will be posted next week and available virtually for only $200 – signup is here – Virtual Momentum.
Every couple of months we like to step back and offer a “What’s next” post in regards to our thoughts on what Documentum customers should be considering with their implementations. For this post, we will highlight our thoughts based on a thorough review of EMC World and our client briefing discussions. Continue reading
Way back at Momentum 2001 in Chicago, I remember having an in-depth conversation with a Documentum architect in integrating Autonomy into the Documentum platform. TSG was implementing Autonomy at the time and Documentum was looking to build a pluggable architecture into Documentum in which any search engine could be integrated. The 5.3 platform helped usher in that pluggable architecture with the replacement of Verity (now owned by Autonomy) with FAST. 9 years later at EMC World 2010, Documentum is getting closer to releasing Documentum Search Services, which is essentially an integration between Lucene and xDB.
Ed Buche and Aamir Farooq both presented at EMC World, providing a good technical overview of DSS and lessons learned from how FAST currently interacts with the Content Server. I’ve always looked forward to Ed Buche’s presentations, and glad he has been very involved in the architecture of DSS. A couple of items to highlight:
Using an XML database like xDB in conjunction with Lucene makes a lot of sense in regards to performance and scalability. All metadata for content is being converted to an XML file and stored within xDB. This is very similar to how FAST ingests metadata today. However, with DSS, an XML representation of the ACL will also be created and stored in xDB, allowing security to be evaluated by the search engine, not at the Documentum level. Replication of ACLs from the Content Server to DSS will be asynchronous, not necessarily transaction based.
A new full text admin interface will also be available, providing much more detailed reports on indexing status, errors, graphs, etc.
Performance and Scalability
Queries that may have taken minutes in FAST, will take seconds in DSS. Documentum has taken a number of lessons learned from the FAST integration and has addressed a number of performance issues that have caused angst in the past. Querying inside folders with a large number of subfolders has been optimized. Additionally, underprivileged users belonging to a small subset of content but searching a wide range of content should see a significant increase in performance. This is a specific issue we’ve run into with our clients and looking forward to comparing the performance difference.
Facets provide the ability to display your search results and drill down further by a set of pre-defined categories. If you have a large results set, you can further drill down by date, format, etc. to refine your search. CenterStage will support this out of the box. I will be curious how or if this will be integrated into Webtop Search Results or how custom search applications will be able to make use of the capability.
Cost / Upgrading to DSS
DSS will remain part of the Content Server and will not be licensed separately.
Microsoft/FAST and Documentum have agreed on extended support for customers until the end of 2011. Therefore, customers making use of full text indexing must upgrade to at least 6.5 SP2 and migrate to DSS by then. DSS will become standard starting with the D6.7 Release. This may be a key driver for customers to start planning you upgrades based on the 2011 date.
Customers who are currently deployed on 6.5 SP2 or later will be able to upgrade to DSS. To evaluate and test DSS compared to FAST, a new docbase may be created using DSS. Both FAST and DSS can therefore be running at the same time and provide a seamless transition from one search platform to another.
In addition to the enhancements to HPI disscussed last week, we are also enhancing the search capability in HPI for a large Documentum pharmaceutical client. Features will include:
- Lucene Integration: enhanced compatibility of HPI and Lucene through TSG’s OpenContent web services layer
- Save Search: save, recall, run, delete and edit a saved search
- OpenSearch Integration: execute a Lucene search (full text and metadata) and return it in an OpenSearch compliant RSS feed that includes a custom namespace to allow for exporting custom metadata in XML format
- Enhanced Search Controls: new multi-select control that includes a type-ahead feature
- Direct Linking: enable direct linking to content via a URL
- Doc Management: one-click links to view versions from the search results
- Single Sign On: leverage Kerberos to perform automatic application authentication
Core components of these enhancements will be rolled into our HPI 1.4 feature set and will be available later in 2010. For more information and to download HPI or view recorded demos. Please visit http://www.tsgrp.com/ and our Learning Zone.
The last post discussed the results of an HPI Lucene Search test compared to a Webtop FAST Search as part of a proof of concept for a client looking to provide a consumer interface. As we have often mentioned on this forum, we continually see clients looking for a better search interface than Webtop, as well as some content cached outside of Documentum for business continuity, performance, and licensing.
One accurate comment raised by the post was that our comparison of HPI/Lucene against a Webtop/FAST search wasn’t really comparing apples to apples as the Webtop search was running against Documentum with security, while the Lucene search was not. While the client’s goals were to show the benefits of the cached repository and Lucene against Documentum, many Documentum users would like to know how Lucene would perform directly against a Documentum repository (as with upcoming DSS).
For this post, we will discuss TSG’s strategy and initial proof of concept results in leveraging Lucene for a Documentum full text search engine.