Sherlock changelog

www.sherlock.stanford.edu

A new tool to help optimize job resource requirements

by Kilian Cavalotti, Technical Lead & Architect, HPC

It’s not always easy to determine the right amount of resources to request for a computing job. Making sure that the application will have enough resources to run properly, but avoiding over-requests that would make the jobs spend too much

Tracking NFS problems down to the SFP level

by Kilian Cavalotti

When NFS problems turn out to be... not NFS problems at all.

Tracking NFS problems down to the SFP level

Sherlock is hard at work against COVID-19

by Kilian Cavalotti

About a month ago, we announced that we were dedicating a portion of Sherlock's computing resources to research projects around COVID-19. Since then, more than 15 PIs and research groups have reached out to share their projects, and...

Adventures in storage

by Kilian Cavalotti

_This is part of our blog series about behind-the-scenes things we do on a regular basis on Sherlock, to keep it up and running in the best possible conditions for our users. Now that Sherlock's old storage system has been retired, we...

When setting an environment variable gives you a 40x speedup

by Kilian Cavalotti

Today, we'd like to share some of our recent work on Sherlock that allowed a pretty significant speedup when listing files in directories with a lot of entries. > Unlike our usual announcements, this post is more of a behind-the-scenes...

When setting an environment variable gives you a 40x speedup

Persistent processes on Sherlock

by Kilian Cavalotti

There are many cases where having a persistent process (or service) running alongside computing jobs is of great benefit. For instance, when data is stored in a database format to allow for highly efficient queries, jobs that want to...

Running containers on Sherlock

by Kilian Cavalotti

Have you ever wanted to run Docker containers on Sherlock, and skip the hassle of figuring out your application dependencies, installing them, compiling the code, resolving compilation errors, finding missing libraries, or making sure...

Running containers on Sherlock

MySQL on Sherlock

by Kilian Cavalotti

Have you ever wondered how you could use a database server on Sherlock? Or wanted to run jobs that query data from a SQL database? Or even better, multiple jobs that would interact with the same database? Have you ever wanted to store...

Spark on Sherlock

by Kilian Cavalotti

Apache Spark is now available on Sherlock, via the spark module: $ ml spark We also added some new documentation and example job scripts, so if you ever needed to run Spark jobs on Sherlock, please take a look at those and let us know...

Newer