com.sapportals.wcm.service.xcrawler
Interface IXCrawlerService

[contained in: com.sap.km.cm.service.base.par - km.shared.service.xcrawler_api.jar]

public interface IXCrawlerService

Global service for crawling repositories.

Copyright (c) SAP AG 2004

Field Summary

static int	MAX_TASK_DISPLAY_NAME_LENGTH
static int	MAX_TASK_ID_LENGTH
static int	MAX_USER_DATA_LENGTH

Method Summary

IXCrawlerParameters	createCrawlerParameters(int maxDepth, int retrieverCount, int providerCount, boolean useETag, boolean useChecksum, boolean followLinks, boolean crawlVersions, boolean crawlHidden, boolean crawlSystem, long requestDelayInMilliseconds, IXCrawlerParameters.ModificationCheckMode modificationCheckMode, boolean findAllDocsInDepth, boolean respectRobots, boolean test, com.sapportals.wcm.service.resourcefilter.IResourceFilter[] scopeFilters, com.sapportals.wcm.service.resourcefilter.IResourceFilter[] resultFilters, long maxLogFileSizeInBytes, int maxBacklogFiles, java.lang.String logFilePath, IXCrawlerParameters.LogLevel maxLogLevel, long documentTimeoutInSeconds) Create crawler parameters.
IXCrawlerParameters	createCrawlerParameters(java.lang.String parameterName) Create crawler parameters from a configurable in the configuration plugin /cm/services/xcrawlers.
void	deleteCrawlerTask(java.lang.String taskID) Delete a crawler task. The task is stopped before deletion. The task is deleted in the database.
java.lang.String[]	getCrawlerParameterNames() Get the names of the available crawler parameters. These are the configurables in the configuration plugin /cm/services/xcrawlers of the config class XCrawler.
IXCrawlerTaskSummary[]	getCrawlerTaskSummaries() Get the state summaries of all crawler tasks.
IXCrawlerTaskSummary	getCrawlerTaskSummary(java.lang.String taskID) Get the state summary of a crawler task.
java.lang.String	getDefaultCrawlerParameterName() Get the name of the default crawler parameters.
boolean	isRunning(java.lang.String taskID) Check, if a crawler task is running for the specified taskID.
boolean	isScheduled(java.lang.String taskID) Check, if a crawler task is scheduled for the specified taskID (and will run if any running or suspended crawler tasks for the same taskID are finished).
boolean	isSuspended(java.lang.String taskID) Check, if a crawler task is suspended for the specified taskID.
void	resumeCrawlerTask(java.lang.String taskID) Resume a crawler task. The task must be suspended.
void	runCrawlerTask(java.lang.String taskID, java.lang.String taskDisplayName, IRidList[] startResources, IXCrawlerParameters[] parameters, java.lang.String resultReceiverFactoryClassName, java.lang.String userDataForFactory, boolean survivesRestart, boolean delta, ISystem node, boolean deleteAfterCompletion) Run a crawler task. Multiple lists of start resources can be specified.
void	stopCrawlerTask(java.lang.String taskID) Stop a crawler task. The method returns after the task is stopped.
void	suspendCrawlerTask(java.lang.String taskID) Suspend a crawler task. The task must be running.

Field Detail

MAX_TASK_ID_LENGTH

public static final int MAX_TASK_ID_LENGTH

MAX_TASK_DISPLAY_NAME_LENGTH

public static final int MAX_TASK_DISPLAY_NAME_LENGTH

MAX_USER_DATA_LENGTH

public static final int MAX_USER_DATA_LENGTH

Method Detail

getCrawlerParameterNames

public java.lang.String[] getCrawlerParameterNames()
                                            throws XCrawlerException

Get the names of the available crawler parameters.
These are the configurables in the configuration plugin /cm/services/xcrawlers of the config class XCrawler.

Returns:

the names of the available crawler parameters

getDefaultCrawlerParameterName

public java.lang.String getDefaultCrawlerParameterName()
                                                throws XCrawlerException

Get the name of the default crawler parameters.

Returns:

the name of the default crawler parameters

createCrawlerParameters

public IXCrawlerParameters createCrawlerParameters(java.lang.String parameterName)
                                            throws XCrawlerException

Create crawler parameters from a configurable in the configuration plugin /cm/services/xcrawlers.

Parameters:

parameterName - name of the configurable

Returns:

the created crawler parameters

createCrawlerParameters

public IXCrawlerParameters createCrawlerParameters(int maxDepth,
                                                   int retrieverCount,
                                                   int providerCount,
                                                   boolean useETag,
                                                   boolean useChecksum,
                                                   boolean followLinks,
                                                   boolean crawlVersions,
                                                   boolean crawlHidden,
                                                   boolean crawlSystem,
                                                   long requestDelayInMilliseconds,
                                                   IXCrawlerParameters.ModificationCheckMode modificationCheckMode,
                                                   boolean findAllDocsInDepth,
                                                   boolean respectRobots,
                                                   boolean test,
                                                   com.sapportals.wcm.service.resourcefilter.IResourceFilter[] scopeFilters,
                                                   com.sapportals.wcm.service.resourcefilter.IResourceFilter[] resultFilters,
                                                   long maxLogFileSizeInBytes,
                                                   int maxBacklogFiles,
                                                   java.lang.String logFilePath,
                                                   IXCrawlerParameters.LogLevel maxLogLevel,
                                                   long documentTimeoutInSeconds)
                                            throws XCrawlerException

Create crawler parameters.

Parameters:

maxDepth - maximum depth of the crawl (0 is unlimited)

retrieverCount - number of threads which retrieve the resources from the repositories

providerCount - number of threads which provide the found resources to the result receivers

useETag - true if the ETag of a resource should be used to detect modification

useChecksum - true if the checksum of the resource content should be used to detect modification

followLinks - true if links should be followed during the crawl

crawlVersions - true if versions of resources should be included in the crawl

crawlHidden - true if hidden resources should be included in the crawl

crawlSystem - true if system resources should be included in the crawl

requestDelayInMilliseconds - number of milliseconds between two consecutive resources retrievals (to limit repository load)

modificationCheckMode - mode of resource modification detection (ETag AND checksum, ETag OR checksum)

findAllDocsInDepth - true if resources should be found on the shortest possible path

respectRobots - true if robot-rules of web-servers should be respected

test - true if no resources should be provided to the result receiver

scopeFilters - resource filters narrowing the scope of the crawl

resultFilters - resource filters which are applied to the result of the crawl but do not narrow the scope

maxLogFileSizeInBytes - maximum size of the crawler log file in bytes (0 is unlimited)

maxBacklogFiles - maximum number of old crawler log files

logFilePath - path to the crawler log file (if null the current system path is used)

maxLogLevel - maximum log level

documentTimeoutInSeconds - the document retrieval timeout in seconds

Returns:

the created crawler parameters

runCrawlerTask

public void runCrawlerTask(java.lang.String taskID,
                           java.lang.String taskDisplayName,
                           IRidList[] startResources,
                           IXCrawlerParameters[] parameters,
                           java.lang.String resultReceiverFactoryClassName,
                           java.lang.String userDataForFactory,
                           boolean survivesRestart,
                           boolean delta,
                           ISystem node,
                           boolean deleteAfterCompletion)
                    throws XCrawlerException

Run a crawler task.
Multiple lists of start resources can be specified. Each list has its own crawler parameters. The number of crawler parameters must match the number of lists of start resources.
The crawler task is started asynchronously.
Tasks with the same ID are started sequentially.

Parameters:

taskID - ID of the new task (maximum length is MAX_TASK_ID_LENGTH)

taskDisplayName - display name of the new task (maximum length is MAX_TASK_DISPLAY_NAME_LENGTH, may be null)

startResources - lists of start resources

parameters - crawler parameters for the lists of start resources

resultReceiverFactoryClassName - class which created result receivers; the name of the class is persisted in the database and reused via reflection when the crawler task is resumed; the class must implement IXCrawlerResultReceiverFactory

userDataForFactory - this string is passed to the createResultReceiver() method of the resultReceiverFactory; here the result receiving application can store any data up to MAX_USER_DATA_LENGTH characters in length (may be null)

survivesRestart - if true the crawler can be resumed even after a restart of CM

delta - true if an incremental update should be performed

node - cluster node on which the task should be executed

deleteAfterCompletion - true if the crawler should be deleted after it is complete

Throws:

XCrawlerException -

suspendCrawlerTask

public void suspendCrawlerTask(java.lang.String taskID)
                        throws XCrawlerException

Suspend a crawler task.
The task must be running.

Parameters:

taskID - ID of the task

Throws:

XCrawlerException -

resumeCrawlerTask

public void resumeCrawlerTask(java.lang.String taskID)
                       throws XCrawlerException

Resume a crawler task.
The task must be suspended.

Parameters:

taskID - ID of the task

Throws:

XCrawlerException -

stopCrawlerTask

public void stopCrawlerTask(java.lang.String taskID)
                     throws XCrawlerException

Stop a crawler task.
The method returns after the task is stopped.

Parameters:

taskID - ID of the task

Throws:

XCrawlerException -

deleteCrawlerTask

public void deleteCrawlerTask(java.lang.String taskID)
                       throws XCrawlerException

Delete a crawler task.
The task is stopped before deletion.
The task is deleted in the database.

Parameters:

taskID - ID of the task

Throws:

XCrawlerException -

getCrawlerTaskSummaries

public IXCrawlerTaskSummary[] getCrawlerTaskSummaries()
                                               throws XCrawlerException

Get the state summaries of all crawler tasks.

Returns:

the state summaries of all crawler tasks

Throws:

XCrawlerException -

getCrawlerTaskSummary

public IXCrawlerTaskSummary getCrawlerTaskSummary(java.lang.String taskID)
                                           throws XCrawlerException

Get the state summary of a crawler task.

Returns:

the state summary of a crawler task (or null if no summary exists for this task)

Throws:

XCrawlerException -

isRunning

public boolean isRunning(java.lang.String taskID)
                  throws XCrawlerException

Check, if a crawler task is running for the specified taskID.

Returns:

true if a crawler task is running for the specified taskID

Throws:

XCrawlerException -

isSuspended

public boolean isSuspended(java.lang.String taskID)
                    throws XCrawlerException

Check, if a crawler task is suspended for the specified taskID.

Returns:

true if a crawler task is suspended for the specified taskID

Throws:

XCrawlerException -

isScheduled

public boolean isScheduled(java.lang.String taskID)
                    throws XCrawlerException

Check, if a crawler task is scheduled for the specified taskID
(and will run if any running or suspended crawler tasks for the
same taskID are finished).

Returns:

true if a crawler task is scheduled for the specified taskID

Throws:

XCrawlerException -