Using the KM Indexmanagement APIs for searching with TREX

Table of Contents

• Prerequisites and Relevant Knowledge Management APIs
• KM Indexmanagement Request Flow
• Coding in Detail
• Using the SearchQueryListBuilder factory class
• Running the Sample Coding
• Known limitations
• Sample Servlet using KM Indexmanagement API for searching with TREX
• Recommended Literature

Applies To:

Knowledge Management 6.0

Summary

Searching in the SAP NetWeaver Enterprise Portal is a key feature of the product and is a requirement in most of the customer project scenarios. Search functionality within the Enterprise Portal is exposed through the KM Indexmanagement service which offers a Java API for accessing the search capability in custom developed components.

Prerequisites and Relevant Knowledge Management APIs

The following prerequisites must be fulfilled to get started with the Java API for searching the SAP Enterprise Portal:

• Fully installed and configured TREX (see Installation Guide)

• SAP NetWeaver Enterprise Portal including Knowledge Management

• A KM repository containing some documents for indexing and searching

• At least one index created for the KM repository containing the documents (see Administration Guide)

• Basic understanding of developing web applications using KMC functionality (see Using the KMC Platform in Standalone Web Applications [KMW])

• Basic understanding of the User Management Engine (UME).

Developers starting to use KM Indexmanagement APIs need an understanding of the concepts described in Repository Framework Concepts [RCO]. This tutorial describes the implementation of a simple Servlet extending HTTPServlet which uses the search functionality of KM Indexmanagement. The following Knowledge Management API packages are used in the sample implementation:

• com.sapportals.wcm.service.indexmanagement.*: Provides a service for managing indexes.

• com.sapportals.wcm.service.indexmanagement.retrieval.search.*: Provides functions for searching and text mining.

• com.sapportals.wcm.repository.*: Contains the basic repository framework interfaces and classes.

The following User Management API package is used in this sample also:

• com.sap.security.api.*: UME functionality for user authentication in web applications

KM Indexmanagement Request Flow

The KM Indexmanagement API is the central search service in the Knowledge Management application of SAP NetWeaver Enterprise Portal. It can manage several search engine' integrations through its index service concept. (See Integrating 3rd party search engines into KM indexmanagement [IMS]). The standard index service provided by SAP is the TREX index service, which enables the SAP NetWeaver Enterprise Portal to search in KM repositories.

For using the search functionality from a client point of view the KM Indexmanagement API can be used in UI components like PortalComponents, WebDynpro UIs or Servlets. A typical flow of a single request will look like the following diagram:

The required APIs for executing a search with the Indexmanagement Service is the IFederatedSearch interface which combines a search with all activated index services implementations. The IFederatedSearch interface offers the possibility to execute the search either via search() or searchWithSession() method (recommended by SAP). The searchWithSession() method offers a paging mechanism which allows you to retrieve search results from the search engine in a limited request list if there are too many hits in the search engine, e.g. results from start count to end count.

Coding in Detail

This example shows the implementation of a simple Servlet using the search functionality of KM Indexmanagement API.

Authentication and transformation of user objects

Due to the fact that this sample is built out of a standard servlet technology, you need to provide valid user credentials to KM Indexmanagement to execute the search properly. User credentials can be gathered from the Usermanagement Engine by forcing the user to log on with a valid user ID.

com.sap.security.api.IUser nwUser = 

    UMFactory.getAuthenticator().forceLoggedInUser(

            request, 

            response

        ); 

if (nwUser == null)

To force a User to be authenticated, the servlet request and response has to be passed through the UserManagement Factory’s authenticator component.

A user authentication will be passed as valid if the user object returned by the authenticator object is not null.

Due to the use of the old EP 5.0 Usermanagement Engine for KM specific user objects, the users from the authentication process have to be converted into EP 5.0 UME compliant user objects.

    IUser user = WPUMFactory.getUserFactory().getEP5User(nwUser);

This transformed user object will then be used to build up a valid KM resource context, which is required for executing searches.

    ResourceContext c = new ResourceContext(user);

Using the search interfaces

The search functionality in KM Indexmanagement is accessible through the IFederatedSearch interface, which can be retrieved out of the Indexmanagement service instance.

    IIndexService indexService = (IIndexService) ResourceFactory.

        getInstance().getServiceFactory().getService(

        IServiceTypesConst.INDEX_SERVICE

    );

    IFederatedSearch federatedSearch = 

        (IFederatedSearch)indexService.getObjectInstance(

        IWcmIndexConst.FEDERATED_SEARCH_INSTANCE

    );

The IFederatedSearch interface allows you to search over all active index services of the KM Indexmanagement. By default the only index service which is available is TREX. An index service itself can manage several indexes which can also be included in the search execution. To get all active indexes of a index service you can simply call

    indexService.getActiveIndexes();

In this sample you have the possibility to specify either your own index within the servlet URL or use all active indexes for searching.

In the next step the query which is passed through the Indexmanagement must be built up. Therefore you can use the simple factory class SearchQueryListBuilder of standard KM.

    SearchQueryListBuilder sqb = new SearchQueryListBuilder();

    sqb.setSearchTerm("SAP AND SDN");

    IQueryEntryList qel = sqb.buildSearchQueryList();

This query list builder class takes care of the syntax validity of the created query entry objects. The queries constructed by the SearchQueryListBuilder are KM Indexmanagement compliant queries which are later passed to every index service implementation. A more detailed usage of the SearchQueryListBuilder is described in the following chapter.

After the successful construction of the query entry list, the execution of the search can be done by simply calling the method:

    federatedSearch.searchWithSession(IQueryEntryList, 

        List<indexes>, IResourceContext);

The searchWithSession() method is recommended to execute a search on the KM indexmanagement. This method ensures that only the requested results instead of all results are retrieved by the index service implementation later on.

You can get a list of the search result items out of the search session object. The requested results can be limited by a count between 1(!) and session.getTotalNumberResultKeys() which is the maximum number of results retrieved for the specified query entry list.

    ISearchResultList results = session.getSearchResults(

        1, session.getTotalNumberResultKeys()

    );

For performance reasons, we recommend you retrieve only a subset of the search result items instead of all, also a paging mechanism should be introduced on an appropriate user interface.

To get each result out of the search results list, you can simply iterate through the number of search results and gather the resource object.

    ISearchResultListIterator iter = results.listIterator();

    while (iter.hasNext()) {

        ISearchResult result = iter.next();

        IResource resource = result.getResource();

        :

    }

Besides the resource object, additional search-relevant information can be retrieved out of a single result item like rank value, a content snippet or search-specific properties.

    result.getRankValue();

    result.getContentSnippet();

    result.getLocalProperties();

As a result of the iteration you can build up a valid user interface structure for each search result item. In this sample the result is rendered in a simple HTML table.

Optional: Processing the “Did you mean?” term

If the index service implementation supports the “Did you mean?” functionality, like the TREX implementation does, you can create an additional query entry list for searching this term. A “Did you mean?” term can only be generated after a search was executed successfully.

After receiving a vaild result list object you can get the “Did you mean?” term out of the SearchQueryListBuilder class by calling the static method getDidYouMeanTerm() with the last searched query entry list and the corresponding search term:

    String didYouMeanTerm = SearchQueryListBuilder.getDidYouMeanTerm(

        qel, "SAP AND SDN"

    );

If a “Did you mean?” term is set by the index service, it will be returned by the method otherwise an empty string is returned.

For building a valid corresponding query entry list for the “Did you mean?” term, use the method buildDidYouMeanQueryEntries():

    IQueryEntryList didYouMeanQEL = sqb.buildDidYouMeanQueryEntries(

        qel, "SAP AND SDN", didYouMeanTerm

    );

The newly created query entry list object can be used for an additional search execution.

Using the SearchQueryListBuilder factory class

The SearchQueryListBuilder factory is responsible for the creation of Indexmanagement-compliant query entry list objects. Within the query entry list you have to specify the search relevant information. Therefore, the SearchQueryListBuilder offers either different setter methods or a constructor which uses a Hashtable containing all the search parameters. The following set of search parameter keys are currently supported by the Hashtable searchParameters:

Samples for using the SearchQueryListBuilder

For detailed method descriptions please refer to the corresponding JavaDoc for the class com.sapportals.wcm.control.released.search.SearchQueryListBuilder.

The following example shows the usage of the SearchQueryListBuilder on a detailed level. It generated a query entry list for a search term excluding some of the options like description and displayname:

SearchQueryListBuilder sqb = new SearchQueryListBuilder();

// set the search term, can contain logical operators in capital letters

sqb.setSearchTerm("banana AND pine");

// only search in documents 

sqb.setSelectedItems(SearchParamConst.ITEM_ONLY_DOCS);

// include the property embedded-keywords for search

sqb.setSearchAddProps("embedded-keywords");

// do not search in the description property

sqb.setDescriptionIncluded(false); 

// do not search in the display name property

sqb.setDisplaynameIncluded(false); 

// only search in documents modified/created in the last month

sqb.setSelectedNotOlderRange(SearchParamConst.NOT_OLDER_MONTH);

IQueryEntryList qel = sqb.buildSearchQueryList();

A query only for documents containing the words banana or pine in the document’s displayname property could look like the following sample:

SearchQueryListBuilder sqb = new SearchQueryListBuilder();

sqb.setSelectedTitle("banana,pine");

IQueryEntryList qel = sqb.buildSearchQueryList();

Building queries for certain custom properties is also possible with this factory class. This sample builds a query for the modelled custom property “fruits” in the namespace “custom” (see PropertyMetadataService) linked by a logical OR operation:

SearchQueryListBuilder sqb = new SearchQueryListBuilder();

sqb.setSelectedCustomProps(

    "custom:fruits(value=banana),custom:fruits(value=pine)"

);

IQueryEntryList qel = sqb.buildSearchQueryList();

A fast way for building queries is the usage of the Hashtable for setting multiple query options directly:

Hashtable searchParameters = new Hashtable();

searchParameters.put(

    SearchParamConst.PARAM_SEARCH_TERM, "banana AND pine");

searchParameters.put(

    SearchParamConst.PARAM_SEARCH_ACTION, SearchParamConst.ACTION_EXACT

);

SearchQueryListBuilder sqb = new SearchQueryListBuilder(searchParameters);

IQueryEntryList qel = sqb.buildSearchQueryList();

The generated IQueryEntryList object can be used directly for Indexmanagement searches.

Connecting several query entry lists

If you want to connect several query entry lists object, you can use the buildQueryEntryConnector() method for creating a valid connected query entry list object:

IQueryEntry queryEntryConnector =

    sqb.buildQueryEntryConnector(IQueryEntry.OPERATOR_AND);

The connection of different connection lists can be done by simple adding them to the first query entry list:

// building the first query entry list

IQueryEntryList q1 = sqb.buildSearchQueryList();

// building the second query entry list

IQueryEntryList q2 = sqb.buildSearchQueryList();

// creating an AND connector

IQueryEntry queryEntryConnector =

    sqb.buildQueryEntryConnector(IQueryEntry.OPERATOR_AND);

// adding the connector to the first query entry list

q1.add(queryEntryConnector);

// adding the second query entry list to the first one

q1.addAll(q2);

Now you have an AND connected list of queries which can be used also for searching. Due to the fact, that the SearchQueryListBuilder is linking property queries for the same property name with an logical OR, you can create you own property queries by using the connector builder method and the list linkage mechanisme and connect them with other logical operations like AND or NOT.

Running the Sample Coding

The sample is based on a simple servlet which has to be deployed on the Web Application Server. To start the servlet on the deployed Portal installation you have to call the following URL:

    http://<server>:<port>/SimpleSearchApplication/Search?query=<a valid query>

The URL parameter query contains the query string which is executed by the search, e.g. query=SDN. Make sure that you have configured the Indexmanagement service correctly and at least one index is available for searching.

After you log in with a valid user and password, the result page of the servlet would look like this:

Known limitations

If you provide access to SAP NetWeaver's search and classification capabilities (TREX) from a non-SAP application as described in this article, the users that do have access to the search functionality via the non-SAP application must be licensed to use SAP NetWeaver. Please contact your SAP sales representative accordingly.

Sample Servlet using KM Indexmanagement API for searching with TREX

The following archive contains a valid and deployable SAP NetWeaver Developer Studio project based on KMC API of NW04 SPS 11, which shows the steps described above in a whole example.

Download example com.sap.netweaver.kmc.searchservlet.zip