We’re very pleased to introduce a new service partition on Sherlock, specially designed to run non-computational management and administrative tasks.

Jobs like data transfer tasks, backups, CI/CD pipelines, workflow managers, or lightweight database instances are often necessary to manage scientific workloads, but typically don’t require significant amounts of computing resources. Running those tasks on regular compute nodes takes precious computing cycles and resources away from actual computing jobs, which is often a trade-off that researchers have to make. This new service partition solves that problem by offering a dedicated place to run those lightweight, non-computational tasks.

Easier scheduling with scrontab

To make it easier to schedule repetitive tasks, and in addition to the usual recurring and persistent jobs, we also support the Slurm scrontab feature (more details available here) on the service partition.

Oversubscribed resources

One of the most important difference with the other partitions on Sherlock is that resources in the service partition are heavily oversubscribed. This means that multiple jobs may share the same CPU and memory resources, leading to minimal compute performance. The focus is on maximizing throughput for many small, low-impact jobs, not on delivering fast or isolated execution.

Complete details about the new service partition (and the other public partitions on Sherlock) are available in the documentation at https://www.sherlock.stanford.edu/docs/user-guide/running-jobs


As usual, please don’t hesitate to reach out if you have any questions, and happy computing!