Spectre cache Options

The Spectre command line cache sub-command displays and manipulates the cache files. It has the following syntax:

spectre cache [options]

For example:

spectre cache

Enter spectre cache -h to display the syntax and options as shown in the following table.

Spectre cache Sub-command Options

Option Description
--dataroot arg

Set the path to the DiveLine server dataroot.

For example:

spectre cache --dataroot c:\di\solution\dl-dataroot

Instead of using this option, you can set the DI_HOME environment variable. See Setting Environment Variables.
--dry-run

Use with --sweep, --max-age, --max-idle, or --max-size to describe what the command would do but without modifying the cache.

For example:

spectre cache --dry-run --sweep

--info

Print a histogram with information about the cache state (default).

For example:

spectre cache --info

spectre cache

--list

Print a tab-delimited table of the contents of the cache.

For example:

spectre cache --list

Columns in this output include ID, Last Access, Created, Updated, Access Count, Update Count, Size, and Initial User.
--list-advanced

Print a tab-delimited table of the contents of the cache including the request scripts and cache tokens.

For example:

spectre cache --list-advanced

Columns in this output include ID, Last Access, Created, Updated, Access Count, Update Count, Size, Initial User, Request, Token, and Milliseconds.
--max-age arg

Remove entries from the cache that have not been updated for this many hours.

For example:

spectre cache --max-age 12

Removes entries last updated over 12 hours ago.
--max-idle arg

Remove entries from the cache that have not been read for this many hours.

For example:

spectre cache --max-idle 6

Removes entries that have not been read in the last 6 hours.
--max-size arg

Remove the most idle entries until the entire cache is at most the specified size on disk in MB.

For example:

spectre cache --max-size 25

Removes idle entries one by one until the on-disk cache size is at most 25 MB.
--remove-all

Remove all cache entries.

For example:

spectre cache --remove all

Clears the cache.
--q [ --quiet ]

Print only warnings and errors to the console.

For example:

spectre cache --quiet --list

When the command runs with this option, no timestamps or individual processes display; only warnings and errors are output to the console. If successful, only the command prompt displays.
--sweep

Do a full scan for stale cache entries and remove them. A stale cache entry is one that is no longer valid because the source data has changed. For example:

spectre cache --sweep

This option removes the stale cache entries, which are not used for their data. However, stale entries can be useful for a refresh. This option also checks for and removes temporary cache entries that were unable to be moved into place.
--refresh

Refresh all stale cache entries. Refresh means to take a stale entry and re-run it to update the data. For example:

spectre cache --refresh

This is best used after a sequence of runs with the --max-age, --max-size, or --max-idle options. Run these options first, then --refresh all remaining entries. This option is also useful after rebuilding a cBase to ensure that cache entries are updated for the end users.
--refresh-with-predicate arg

Take an expression and evaluate it; if it is true, refresh the cache entries. This option is useful if the criteria are more complex.

The expression runs over a virtual table with the following columns:

  • age—The number of hours since this cache entry was last updated
  • idle—The number of hours since this cache entry was last read
  • num_hits—The number of queries that were satisfied by reading this cache entry
  • num_updated—The number of times this cache entry has been updated
  • user—The current user name
  • size—The size of this cache entry in bytes
  • time_to_generate—The time to generate this cache in seconds
  • project—The project the cache entry is attached to

You can write expressions that refer to these columns. For example, if user Bob likes his queries cached but only if they took more than a second:

spectre cache --refresh-with-predicate "value(\"user\") = \"Bob\"

and value(\"time_to_generate\") > 1"

TIP: Doing a --sweep after a --refresh-with-predicate removes all the remaining stale entries, which use up disk space.

NOTE:  Starting with Spectre 7.1(18), --refresh-with-predicate removes stale compiler cache entry (ccache requests) regardless of the predicate.

--remove-with-predicate arg

Take an expression and evaluate it; if it is true, remove the cache entries. This option is useful if the criteria are more complex.

The expression runs over a virtual table with the following columns:

  • age—The number of hours since this cache entry was last updated
  • idle—The number of hours since this cache entry was last read
  • num_hits—The number of queries that were satisfied by reading this cache entry
  • num_updated—The number of times this cache entry has been updated
  • user—The current user name
  • size—The size of this cache entry in bytes
  • time_to_generate—The time to generate this cache in seconds
  • project—The project the cache entry is attached to

You can write expressions that refer to these columns. For example, if you know that cache entries over a certain size are really mistakes (for example, the initial dive was not a good one):

spectre cache --remove-with-predicate "value(\"size\") > 1024 * 1024 * 1024

and value(\"user\") != \"the boss\""

-j [ --jobs ] arg

With --refresh with predicate, specifies the maximum number of cache files to refresh in parallel.

spectre cache --refresh-with-predicate "value(\"age\") <= 36" -j 3

Up to 3 cache refreshes run in parallel. Use arg to set the number of core processors (likely between 2 and 8). Start with a lower number as this depends on the I/O capabilities of the host system.
  • Virus scanning should be turned off for Spectre cache directories.
  • You can make a refresh of the entire cache part of your regular ETL that builds cBases. See the Spectre Cache Refresh Process Node.
  • It is a best practice to use spectre cache with --refresh to refresh items you want to refresh, then use --sweep to remove the remaining stale entries. These steps avoid accumulation of stale entries that use up disk space.
  • You can use spectre cache with --refresh and --sweep to re-run recently made queries after a cBase build. Use an Execute process node in a Production Script. See Execute Process Node.
  • You can automate opening Markers with DIAL to force caching of those resources, looping over different users. Use a DIAL process node in a Production script. See DIAL Process Node.
  • The default location for cache files is dl-dataroot\cache\spectre. An alternative location can be specified by using the spectre_cache_path variable in the atlcfg.cfg file, which you can set using Workbench. See Miscellaneous DiveLine Settings.
  • Use of spectre cache with no options prints a histogram to the console showing graphically what is stale and what is fresh.

  • DiveLine server logs include the cache ID—the name used for the cache file. For example, a cache file could have the filename: 4d/4dd4eb7485fb8fc7fd2da0d16c958dd4, and an extension of meta, request, or token.

  • Caching is on by default. To turn it off during development, change the permissions on the cache directory so that the DiveLine server does not have write access.

The following commands assume that the environment variable PATH includes the location of the Spectre executable file.

spectre cache --max-age 24 --dataroot C:\di\solution\dl-dataroot

This command indicates how many cache entries are present, removes entries older than 24 hours, and indicates how many were removed and how many are left.

spectre cache --refresh-with-predicate "value(\"age\") <= 24" 
     --dataroot c:\di\solution\dl-dataroot

This command updates only those entries written in the last 24 hours.

spectre cache --refresh-with-predicate "value(\"user\") = \"admin\""
     --dataroot c:\di\solution\dl-dataroot

This command scans the cache directory, tests the entries against the predicate, finds those entries that belong to user admin, and refreshes those entries.

./spectre cache --remove-with-predicate "value(\"time_to_generate\")<3 or (value(\"num_hits\")=0 and value(\"num_updated\")>31)"

On a Linux system with DI_HOME and --dataroot set, this command checks that the time to create the cache is under 3 seconds or the cache has not been used but it has been updated over 31 times.

./spectre cache --remove-with-predicate 'value("user")=""'

On a Linux system with DI_HOME and --dataroot set, run after a --refresh-with-predicate, removes the ccache compiler cache entries manually. Prior to Spectre 7.1(18), compiler cache entries could accumulate and take up disk space.

Mentioned in: