Wednesday, June 23, 2010

Indexer tasks: Update and Updall

The Update and Updall tasks keep view indexes and full-text indexes up-to-date.

Update

Update is loaded at server startup by default and runs continually, checking its work queue for views and folders that require updating. The indexer uses modest system resources by waiting five seconds between each database update operation that is performs.

The Update task performs three different updating tasks:

  • Updates views in the Domino Directory.
  • Updates views in all other databases. When a request is made to update a view, the view is only updated if there are at least 20 note changes since the last update and if the view has been accessed in the last 7 days. The view update service increases the speed of the view access time when a view is opened in the Notes client. If views are not updated often, the only effect on users or applications is a slow view open time because views are automatically updated when opened.
  • Updates full-text indexes. Full-text indexing provides the ability to search for notes that have been recently added. If a note is added after the most recent full-text indexing, that note will not be found by a full text search.


The Update task uses a separate thread for full-text indexing which makes view updates more timely than in releases prior to Domino 7.

Update maintains two queues of work -- an immediate queue and a deferrred queue. Other server components, such as the router and replicator, post requests to the updater when changes are made to databases. Some requests are posted as deferred and some as immediate.

This table lists how full-text index updates are performed according the update frequency:

Update frequency

Description

Daily

Performed by the nightly Updall task. If this nightly task is not run, the daily updating is not performed.

Scheduled

Performed by a Program document which runs Updall. You need to set the frequency to Scheduled and create the proper Program document. You can also use this method to update different databases at different times.

Hourly

Triggered by the chronos task and performed by the update task if the update task is running. If the update task is not running, chronos performs the update. If the chronos task is not running, the update is not performed.

Immediate

Performed by the Update task. If Update is not running, the update is not performed. All immediate requests are processed as they are received.

Deferred

Deferred requests are held for 15 minutes before they are processed. Requests to update the same database that occur in that time are ignored as duplicate requests.

When a view or folder change is recorded in the queue, Update waits approximately 15 minutes before updating all view indexes in the database so that the update can include any other database changes made during the 15-minute period. After updating view indexes in a database, it then updates all databases that have full-text search indexes set for immediate or hourly updates.

When Update encounters a corrupted view index or full-text index, it rebuilds the view index or full-text index in an attempt to correct the problem. Update deletes the view index or full-text index and rebuilds it.

Note The Update task spawns a directory indexer thread. The directory indexer runs at one-minute intervals and is dedicated to keeping Domino Directory view indexes up-to-date so that any changes to the directory are usable as soon as possible. The directory indexer runs against any local or remote Domino Directory or Extended Directory Catalog that a server uses for directory services. The task of updating the Domino Directory view indexes does not lock the views, and you should be able to create new server sessions while this task is running.

To improve view-indexing performance, you can run multiple Update tasks if your server has adequate CPU power.

Managing the update task and its use of system resources

The indexer is able to keep up with the update rate in the server's default configuration if the server has a low update rate, that is, if few changes are made to databases on the server. If a server has a high update rate due to heavy application database use, a large number of mail users, or a large volume of mail, the default resource usage configuration can cause the updater queues to become large. To determine whether the updater queues are large, examine the queue length statistics that are available in Lotus Domino 7. If you determine that the update queues are too large, determine a methodology for performing updates on that server. Long queues typically indicate that views and full-text indexes are not up-to-date.

Here are some sample scenarios and practices that you may want to use as well as the steps to implement them.

  • Scenario one -- The queues are usually short, unless a full-text index starts for a large update volume database. When this occurs, the view updating requests wait for the full-text index. This causes the queues to increase until the full-text indexing is complete. To use slightly more system resources to keep the queues short, perform view updates and full-text index updates in separate threads. To do so, enter this variable, UPDATE_FULLTEXT_THREAD=1, in your server's NOTES.INI file.
  • Scenario two -- The queues grow slowly over time and become too long because the Updater task is not getting sufficient system resources to keep the queues short. To use additional resources to keep the queues short, set a delay between each Update operation. To set the delay, enter these variables, UPDATE_IDLE_TIME (and FTUPDATE_IDLE_TIME if two threads are used) in the server's NOTES.INI file. By default, the delay is 5 seconds. To allow the Update task to use additional system resources, set the delay to less than 5 seconds. Finer precision may be required on a large server. In that case, you can set the delay in milliseconds (Domino 7 only) by adding these variables, UPDATE_IDLE_TIME_MS (and FTUPDATE_IDLE_TIME_MS if two threads are used), to the server's NOTES.INI file.
  • Scenario -- Servers that have high update rates often require too many system resources to keep the queues small. In this case, you can decide not to perform view updates at all, and just allow view opens to perform the updates automatically. Disable the view updates by adding this variable, UPDATE_DISABLE_VIEWS=1, to the server's NOTES.INI file. Another option is to limit the number of immediate updates for full-text databases. You change the update frequency for databases to hourly, daily, or to a specific schedule. You can also delete extraneous full-text indexes.

To allow frequent full-text indexing on only a small number of databases, and to prevent other databases from being full-text indexed, disable full -text indexing in the Updater and then add Program documents to schedule Updall to run, for example, every half hour (30 minutes). To disable full-text indexing in the Updater, enter this variable, UPDATE_DISABLE_FULLTEXT=1, in the server's NOTES.INI file.

You can prevent performing any updates at all, and just allow view opens to perform the view updates automatically. To prevent updates, edit the NOTES.INI variable by remoinge the update string.

If a system has adequate system resources to perform updates, you can run multiple Update tasks. To do so, edit the variable, ServerTasks, in the NOTES.INI file and add a second Update task.

You can adjust the controls that determine whether a modified view is actually updated or not. The database and view must still be opened, but if these thresholds are not reached, the view is not updated.


For more information, see UPDATE_ACCESS_FREQUENCY and UPDATE_NOTE_MINIMUM as well as other NOTES.INI settings.

Updall

Updall is similar to Update, but it doesn't run continually or work from a queue; instead you run Updall as needed. You can specify options when you run Updall, but without them Updall updates any view indexes or full-text search indexes on the server that need updating. To save disk space, Updall also purges deletion stubs from databases and discards view indexes for views that have been unused for 45 days, unless the database designer has specified different criteria for discarding view indexes. Use the NOTES.INI setting Default_Index_Lifetime_Days to change when Updall discards unused view indexes.

Like Update, Updall rebuilds all corrupted view indexes and full-text search indexes that it encounters.

By default Updall is included in the NOTES.INI setting ServerTasksAt2, so it runs daily at 2 AM. Running Updall daily helps save disk space by purging deletion stubs and discarding unused view indexes. It also ensures that all full-text search indexes that are set for daily updates are updated.

Note When views are being rebuilt - either through the Designer or Updall tasks - all new server sessions that are attempted once the rebuild process has started are locked out. Therefore, it is recommended that changes to master templates, as well as complete view rebuilds, be scheduled for late at night, when users are far less likely to require access to the server.

The following table compares the characteristics of Update and Updall. For Updall, the table describes default characteristics. For information on options you can use to modify some of these characteristics, see the topic Updall options.

Characteristic

Update

Updall

When it runs

Continually after server startup

2 AM and when you run it

Runs on all databases?

No. Runs only on databases that have changed.

Yes

Refreshes views indexes?

Yes

Yes

Updates full-text indexes?

Yes. Updates full-text indexes set for immediate and hourly updates.

Yes. Updates all full-text indexes.

Detects and attempts to rebuild corrupted view indexes?

Yes

Yes

Detects and attempts to rebuild corrupted full-text indexes?

Yes

Yes

Purges deletion stubs?

No

Yes

Discards unused view indexes?

No

Yes (after a view is unused for 45 days or according to a view discard option specified by a designer)

Ignores "Refresh index" view property?

Yes

Yes

Can customize with options?

No

Yes

How the Router allocates threads on a Domino server

Problem According to the Domino Release 5.x Administration Guide, if the "Maximum Transfer Threads" and / or "Maximum Delivery Threads" fields in the Server Configuration document are left blank...

"The Router sets a default maximum number of [transfer and delivery] threads based on server memory. Letting the Router select the maximum number is usually best."

Exactly what dictates how many transfer and delivery threads are used?

Solution: When these fields are set to blank, the router uses the size of the NSF Buffer Pool as a guideline for determining the maximum amount of threads to be used. This value can be altered by using the NSF_Buffer_Pool_Size NOTES.INI parameter*. If the parameter is not used, Domino will use the following amount of physical RAM for the NSF Buffer Pool (the amount allocated for the NSF Buffer Pool varies depending on the total amount of physical RAM on the machine).

*NOTE: Modifying the NSF Buffer Pool size should only be done on recommendation of Lotus Customer Support. This is especially important in Domino R5 because there are new algorithms in place that determine how much memory is allocated to the buffer pool.

The router sets an initial maximum of three (3) threads, and will use one (1) additional thread per 32MB of NSF Buffer Pool, with a maximum of twenty-five (25) threads. This calculation applies for both transfer and delivery threads, but it is done independently for each.

For example, an R5 Domino server, with 256MB of physical memory, and all default settings, will have the following resource allocation:

RAM = 256 MB
=> NSF Buffer Pool = RAM / 4 = 64 MB
Transfer Threads = Minimum Threads + Additional Threads
Minimum Threads = 3
Additional Threads = 64 / 32 = 2
=> Transfer Threads = 3 + 2 = 5

Hence, the router will use a maximum of five (5) Transfer Threads, and, following the same logic, five (5) Delivery Threads.

NOTE: The transfer threads on the Server Configuration document can be overridden by using the NOTES.INI setting MailMaxThreads=. The order for which the transfer threads are set: first the INI files are searched for the MailMaxThreads parameter, then the Server Configuration document, and if still not defined, will be calculated using the resource allocation formula.

Examples using the resource allocation formula:

Physical Memory

Buffer Pool

Delivery Threads

64MB

16MB

3

128MB

32MB

3+1=4

256MB

64MB

3+2=5

512MB

128MB

3+4=7

1GB

256MB

3+8=11

2GB

512MB

3+16=19

>2.8GB

25


The maximum number of transfer or delivery threads is configured using the above resource allocation formula and will be between 3 and 25. It is possible to increase the number of threads past the maximum default of 25 manually. However, increasing these threads past 25 could cause some performance issues if the hardware resources are not available to handle the configured number of threads.

How the Router allocates threads on a Domino server

Problem According to the Domino Release 5.x Administration Guide, if the "Maximum Transfer Threads" and / or "Maximum Delivery Threads" fields in the Server Configuration document are left blank...

"The Router sets a default maximum number of [transfer and delivery] threads based on server memory. Letting the Router select the maximum number is usually best."

Exactly what dictates how many transfer and delivery threads are used? Solution When these fields are set to blank, the router uses the size of the NSF Buffer Pool as a guideline for determining the maximum amount of threads to be used. This value can be altered by using the NSF_Buffer_Pool_Size NOTES.INI parameter*. If the parameter is not used, Domino will use the following amount of physical RAM for the NSF Buffer Pool (the amount allocated for the NSF Buffer Pool varies depending on the total amount of physical RAM on the machine).

*NOTE: Modifying the NSF Buffer Pool size should only be done on recommendation of Lotus Customer Support. This is especially important in Domino R5 because there are new algorithms in place that determine how much memory is allocated to the buffer pool.

The router sets an initial maximum of three (3) threads, and will use one (1) additional thread per 32MB of NSF Buffer Pool, with a maximum of twenty-five (25) threads. This calculation applies for both transfer and delivery threads, but it is done independently for each.

For example, an R5 Domino server, with 256MB of physical memory, and all default settings, will have the following resource allocation:

RAM = 256 MB
=> NSF Buffer Pool = RAM / 4 = 64 MB
Transfer Threads = Minimum Threads + Additional Threads
Minimum Threads = 3
Additional Threads = 64 / 32 = 2
=> Transfer Threads = 3 + 2 = 5

Hence, the router will use a maximum of five (5) Transfer Threads, and, following the same logic, five (5) Delivery Threads.

NOTE: The transfer threads on the Server Configuration document can be overridden by using the NOTES.INI setting MailMaxThreads=. The order for which the transfer threads are set: first the INI files are searched for the MailMaxThreads parameter, then the Server Configuration document, and if still not defined, will be calculated using the resource allocation formula.

Examples using the resource allocation formula:

Physical Memory

Buffer Pool

Delivery Threads

64MB

16MB

3

128MB

32MB

3+1=4

256MB

64MB

3+2=5

512MB

128MB

3+4=7

1GB

256MB

3+8=11

2GB

512MB

3+16=19

>2.8GB

25


The maximum number of transfer or delivery threads is configured using the above resource allocation formula and will be between 3 and 25. It is possible to increase the number of threads past the maximum default of 25 manually. However, increasing these threads past 25 could cause some performance issues if the hardware resources are not available to handle the configured number of threads.