Compute

SciServer Compute Icon
Go to Compute

Overview

SciServer Compute uses Jupyter Notebooks running within server-side containers attached to big data collections to bring advanced analysis to big data “in the cloud.” Compute extends the popular CasJobs and SkyServer systems with server-side computational capabilities and very large scratch storage space, and further extends their functions to a range of other scientific disciplines.

Interactive Compute

CasJobs has revolutionized database querying of big datasets like SDSS catalog data. Yet until now users had to download their data from the cloud to perform additional computational analysis close to the data, in a networking sense. SciServer Compute is a web app that runs in any modern browser, allowing users to process their data in a cloud-based research environment.

Data Stores and Access

Compute uses the SciServer Web Services API to access CasJobs datasets and MyDB to access and update relational databases. There are also APIs to access file storage systems of which there are 3 broad types:

  1. User Volumes: Each user has a file storage Quota, like MyDB
  2. Scratch Volumes: Each user can access temporary file space, like MyScratch
  3. Data Volumes: Public or private domain data sets made available to users on an access controlled basis

Features

Compute supports a number of features that make it widely applicable across science domains:

  1. Language Support: Python, R, Julia and Matlab are supported
  2. Software Support: Many custom utility and science libraries are pre-packaged up to provide a rich set of processing tools
  3. Development Tools: Tools such as gcc compilers and debuggers are available
  4. Filesystem Support: Filesystems can be mounted during the execution of a job allowing direct and optimized access to files
  5. API Support: SciScript is a sub component of Compute that provides APIs in python and R for all SciServer components – including Compute itself

SciScript

SciScript is the main programmatic access point for SciServer resources. SciScript technology allows users to write scripts in Python, R, or Matlab that offer access to all tools within existing programs.

Back to Tools page