SciServer package¶

This python package provides functions for quick access of SciServer APIs (web services) and tools. SciServer (http://www.sciserver.org) provides a new online framework for data-intensive scientifc computing in the cloud, where the motto is to bring the computation close where the data is stored, and allow seamless access and sharing of big data sets within the scientific community.

Some SciServer tools you can access with this package:

Login Portal: Single sign-on portal to all SciServer applications.
Although some tools accept anonymous access, you can use Authentication.login to login and access the tools and your own data and environment (after registering in the Login Portal). If you are running this package in a Jupyter Notebook in the SciServer-Compute environment, the use of Authentication.login is not necessary since it’s done automatically.
CasJobs: Database storage and querying.
You can have access big databases, as well as saving your data tables in your own database called ‘MyDB’. The user can run synchronous or asynchronous SQL queries and get the result back as a Pandas data-frame (using CasJobs.executeQuery or CasJobs.submitJob, respectively). Uploading of CSV files or data-frames into a database table can be done using CasJobs.uploadCSVToTable and CasJobs.uploadDataFrameToTable, respectively.
SciQuery: Advanced database storage and querying.
SciQuery expands the capabilities of CasJobs, as it is designed to work with databases loaded in several database management systems beyond MS SQL Server, such as PostgreSQL and MySQL. Query result tables can also be directly written as JSON or CSV files in the SciServer filesystem.
SciDrive: Drag-and-drop file storage and sharing.
You can create directories in SciDrive using SciDrive.createContainer, upload a file to SciDrive using SciDrive.upload, and share its URL with your collaborators by using SciDrive.publicUrl.
SkyServer: Access to the SDSS astronomical survey.
You can query the SDSS database using SkyServer.sqlSearch, run cone searches using SkyServer.radialSearch, or get cutout images from the sky using SkyServer.getJpegImgCutout, between other tasks.
SkyQuery: Cross-match of astronomical source catalogs.
You can use this scalable database system for uploading your own catalogs and cross-matching them against huge astronomical source catalogs, or even cross-matching huge catalogs against each other!. Use SkyQuery.submitJob to run the cross-match, and use SkyQuery.listAllDatasets, SkyQuery.listDatasetTables and SkyQuery.listTableColumns to browse the catalogs and database schema.
Compute Jobs: Submission of Jupyter notebooks or shell commands as jobs
You can execute whole Jupyter notebooks and shell commands as asynchronous batch jobs, as well as synchronous jobs.
SciQuery Jobs: Submission of SQL queries.
You can execute synchronous or s synchronous SQL queries against Postgres and other database backends.
Files: Interaction with the SciServer file system.
You can upload, and download data into your own file space in SciServer, as well as share data with your collaborators, between other things.

References

SciServer-Python repository: http://www.github.com/sciserver/SciScript-Python
SciServer: http://www.sciserver.org

Maintainer: Manuchehr Taghizadeh-Popp <mtaghiza@jhu.edu>

Authors: Gerard Lemson <glemson1@jhu.edu>, Manuchehr Taghizadeh-Popp <mtaghiza@jhu.edu>

Version: sciserver-v2.2.0

SciServer.CasJobs module¶

class SciServer.CasJobs.Task[source]¶

Bases: object

The class TaskName stores the name of the task that executes the API call.

name = None¶

SciServer.CasJobs.cancelJob(jobId)[source]¶

Cancels a job already submitted.

Parameters:: jobId – id of job (integer)
Returns:: Returns True if the job was canceled successfully.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.
Example:: response = CasJobs.cancelJob(CasJobs.submitJob(“select 1”))

See also

CasJobs.submitJob, CasJobs.waitForJob.

SciServer.CasJobs.executeQuery(sql, context='MyDB', format='pandas')[source]¶

Executes a synchronous SQL query in a CasJobs database context.

Parameters:

sql – sql query (string)
context – database context (string)
format –
parameter (string) that specifies the return type:

’pandas’: pandas.DataFrame.

’json’: a JSON string containing the query results.

’dict’: a dictionary created from the JSON string containing the query results.

’csv’: a csv string.

’readable’: an object of type io.StringIO, which has the .read() method and wraps a csv string that can be passed into pandas.read_csv for example.

’StringIO’: an object of type io.StringIO, which has the .read() method and wraps a csv string that can be passed into pandas.read_csv for example.

’fits’: an object of type io.BytesIO, which has the .read() method and wraps the result in fits format.

’BytesIO’: an object of type io.BytesIO, which has the .read() method and wraps the result in fits format.

Returns:

the query result table, in a format defined by the ‘format’ input parameter.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error. Throws an exception if parameter ‘format’ is not correctly specified.

Example:

table = CasJobs.executeQuery(sql=”select 1 as foo, 2 as bar”,format=”pandas”, context=”MyDB”)

See also

CasJobs.submitJob, CasJobs.getTables, SkyServer.sqlSearch

SciServer.CasJobs.getJobStatus(jobId)[source]¶

Shows the status of a job submitted to CasJobs.

Parameters:: jobId – id of job (integer)
Returns:: Returns a dictionary object containing the job status and related metadata. The “Status” field can be equal to 0 (Ready), 1 (Started), 2 (Canceling), 3(Canceled), 4 (Failed) or 5 (Finished). If jobId is the empty string, then returns a list with the statuses of all previous jobs.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.
Example:: status = CasJobs.getJobStatus(CasJobs.submitJob(“select 1”))

See also

CasJobs.submitJob, CasJobs.waitForJob, CasJobs.cancelJob.

SciServer.CasJobs.getNumpyArrayFromQuery(queryString, context='MyDB')[source]¶

Executes a casjobs query and returns the results table as a Numpy array (http://docs.scipy.org/doc/numpy/).

Parameters:

queryString – sql query (string)
context – database context (string)

Returns:

Returns a Numpy array storing the results table.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.

Example:

array = CasJobs.getNumpyArrayFromQuery(“select 1 as foo”, context=”MyDB”)

See also

CasJobs.submitJob, CasJobs.getJobStatus, CasJobs.executeQuery, CasJobs.writeFitsFileFromQuery, CasJobs.getPandasDataFrameFromQuery

SciServer.CasJobs.getPandasDataFrameFromQuery(queryString, context='MyDB')[source]¶

Executes a casjobs quick query and returns the result as a pandas dataframe object with an index (http://pandas.pydata.org/pandas-docs/stable/).

Parameters:

queryString – sql query (string)
context – database context (string)

Returns:

Returns a Pandas dataframe containing the results table.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.

Example:

df = CasJobs.getPandasDataFrameFromQuery(“select 1 as foo”, context=”MyDB”)

See also

CasJobs.submitJob, CasJobs.getJobStatus, CasJobs.executeQuery, CasJobs.writeFitsFileFromQuery, CasJobs.getNumpyArrayFromQuery

SciServer.CasJobs.getSchemaName()[source]¶

Returns the WebServiceID that identifies the schema for a user in MyScratch database with CasJobs.

Returns:: WebServiceID of the user (string).
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.
Example:: wsid = CasJobs.getSchemaName()

See also

CasJobs.getTables.

SciServer.CasJobs.getTables(context='MyDB')[source]¶

Gets the names, size and creation date of all tables in a database context that the user has access to.

Parameters:: context – database context (string)
Returns:: The result is a json object with format [{“Date”:seconds,”Name”:”TableName”,”Rows”:int,”Size”,int},..]
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.
Example:: tables = CasJobs.getTables(“MyDB”)

See also

CasJobs.getSchemaName

SciServer.CasJobs.submitJob(sql, context='MyDB')[source]¶

Submits an asynchronous SQL query to the CasJobs queue.

Parameters:

sql – sql query (string)
context – database context (string)

Returns:

Returns the CasJobs jobID (integer).

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.

Example:

jobid = CasJobs.submitJob(“select 1 as foo”,”MyDB”)

See also

CasJobs.executeQuery, CasJobs.getJobStatus, CasJobs.waitForJob, CasJobs.cancelJob.

SciServer.CasJobs.uploadCSVDataToTable(csvData, tableName, context='MyDB')[source]¶

Uploads CSV data into a CasJobs table.

Parameters:

csvData – a CSV table in string format.
tableName – name of CasJobs table to be created.
context – database context (string)

Returns:

Returns True if the csv data was uploaded successfully.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.

Example:

csv = CasJobs.getPandasDataFrameFromQuery(“select 1 as foo”, context=”MyDB”).to_csv().encode(“utf8”); response = CasJobs.uploadCSVDataToTable(csv, “NewTableFromDataFrame”)

See also

CasJobs.uploadPandasDataFrameToTable

SciServer.CasJobs.uploadPandasDataFrameToTable(dataFrame, tableName, context='MyDB')[source]¶

Uploads a pandas dataframe object into a CasJobs table. If the dataframe contains a named index, then the index will be uploaded as a column as well.

Parameters:

dataFrame – Pandas data frame containg the data (pandas.core.frame.DataFrame)
tableName – name of CasJobs table to be created.
context – database context (string)

Returns:

Returns True if the dataframe was uploaded successfully.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.

Example:

response = CasJobs.uploadPandasDataFrameToTable(CasJobs.getPandasDataFrameFromQuery(“select 1 as foo”, context=”MyDB”), “NewTableFromDataFrame”)

See also

CasJobs.uploadCSVDataToTable

SciServer.CasJobs.waitForJob(jobId, verbose=False, pollTime=5)[source]¶

Queries regularly the job status and waits until the job is completed.

Parameters:

jobId – id of job (integer)
verbose – if True, will print “wait” messages on the screen while the job is still running. If False, will suppress the printing of messages on the screen.
pollTime – idle time interval (integer, in seconds) before querying again for the job status. Minimum value allowed is 5 seconds.

Returns:

After the job is finished, returns a dictionary object containing the job status and related metadata. The “Status” field can be equal to 0 (Ready), 1 (Started), 2 (Canceling), 3(Canceled), 4 (Failed) or 5 (Finished).

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.

Example:

CasJobs.waitForJob(CasJobs.submitJob(“select 1”))

See also

CasJobs.submitJob, CasJobs.getJobStatus, CasJobs.cancelJob.

SciServer.CasJobs.writeFitsFileFromQuery(fileName, queryString, context='MyDB')[source]¶

Executes a quick CasJobs query and writes the result to a local Fits file (http://www.stsci.edu/institute/software_hardware/pyfits).

Parameters:

fileName – path to the local Fits file to be created (string)
queryString – sql query (string)
context – database context (string)

Returns:

Returns True if the fits file was created successfully.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the CasJobs API returns an error.

Example:

CasJobs.writeFitsFileFromQuery(“/home/user/myFile.fits”,”select 1 as foo”)

See also

CasJobs.submitJob, CasJobs.getJobStatus, CasJobs.executeQuery, CasJobs.getPandasDataFrameFromQuery, CasJobs.getNumpyArrayFromQuery

SciServer.SciQuery module¶

class SciServer.SciQuery.Database(rdb_compute_domain: Union[str, int, dict], database: Union[str, int, dict])[source]¶

Bases: object

Defines a database context where users can run sql queries.

Parameters:

rdb_compute_domain – Parameter that identifies the relation database domain or environment that contains the database. Could be either its name (string), ID (integer), or a dictionary containing the attributes of the domain.
database – defines the database. Can be either the database name (string), ID (integer), or a dictionary containing all the attributes of an object of class Database.

get_metadata() → DataFrame[source]¶: Gets Database metadata.

class SciServer.SciQuery.DatabaseTableOutput(table: str = 'resultTable', database: str = None, statement_indexes: Union[int, List[int]] = 1, rdb_compute_domain: str = None, schema: str = '')[source]¶

Bases: Output

Defines the output of a database query into a database table.

Parameters:

table – name of the output database table (string), such as “resultTable”
database – name of the database (string) where the output table in created. If it is owned explicitly by a user, then it should follow the pattern “mydb:<username>”
statement_indexes – list of integers or integer. Each integer value denotes the index or position (>=1)
rdb_compute_domain – name (string) of the relational database (RDB) compute domain that contains the database, or object of class RDBComputeDomain corresponding to it. Name of such domains available to the user is returned by the function Jobs.getRDBComputeDomainNames().
schema – database schema (string) of the sql statements within the input query, whose resultset is going to be written into this OutputTarget

classmethod get_default()[source]¶: Gets a OutputTarget object filled with default values: JSON output file where only the 1st SQL statement of the query is written in it.

class SciServer.SciQuery.FileOutput(name: str = 'output.json', output_type: str = 'FILE_JSON', statement_indexes: Union[int, List[int]] = 1, file_service: str = None)[source]¶

Bases: Output

Defines the output of a database query into a file.

Parameters:

name – name of the file (string), such as “result.json”
output_type – type (string) of the file containing the query result(s) (e.g., “FILE_JSON”). As set of possible values is given by the static members of class ‘SciQuery.OutputTargetType’
statement_indexes – list of integers or integer. Each integer value denotes the index or position (>=1) of the sql statements within the input query, whose resultset is going to be written into this OutputTarget
file_service – string denoting name or identifier of file service where the output file is written into.

static build_file_base_path(top_volume: str = 'Temporary', user_volume: str = 'scratch', user_volume_owner_name: str = '', relative_path: str = 'sciqueryjobs', add_date_ending: bool = False) → str[source]¶

Builds the base path for an output file located in the SciServer filesystem.

Parameters:

top_volume – name (str) of top volume in SciServer’s filesystem, such as “Temporary” or “Storage”.
user_volume – name (str) of user volume in SciServer’s filesystem.
user_volume_owner_name – name (str) of user volume owner in SciServer’s filesystem.
relative_path – relative path (str) after <top_volume>/<user_volume_owner_name>/<user_volume>/
add_date_ending – if True, then the relative path is built from the current date.

Returns:

path string

static find_file_service(file_service: Union[str, dict] = None) → dict[source]¶: Gets FileService info

classmethod get_default()[source]¶: Gets a feault FileOutput object filled with default values: JSON output file where only the 1st SQL statement of the query is written in it.

get_path() → str[source]¶: Get file output path on SciServer’s FileSystem

class SciServer.SciQuery.Output(name: str = 'output.json', output_type: str = 'FILE_JSON', statement_indexes: Union[int, List[int]] = 1)[source]¶

Bases: object

Base class for output objects, including files or database tables.

set_statement_indexes(statement_indexes: Union[int, List[int]] = 1)[source]¶

Sets the index(es) of the sql statement(s) within the input query, whose result-set(s) is(are) going to be written into this Output.

Parameters:: statement_indexes – integer or list of integers, which are the indices (starting with 1) of the sql statements within the input query, whose resultsets are going to be written into this Output.

class SciServer.SciQuery.OutputType[source]¶

Bases: object

Contains a set of allowed database output types.

DATABASE_TABLE = 'TABLE'¶

FILE_CSV = 'FILE_CSV'¶

FILE_JSON = 'FILE_JSON'¶

class SciServer.SciQuery.Outputs(*outputs)[source]¶

Bases: list

Contains a list of output objects, defining database query result outputs.

append(obj)[source]¶: Appends an Output object to this list.

static get_default()[source]¶: Gets an OutputList with one element, consisting of a JSON FileOutput object.

get_target_list(file_base_path: str = None, file_service: str = None)[source]¶: Gets list of output targets.

class SciServer.SciQuery.RDBComputeDomain(rdb_compute_domain: Union[str, int, dict])[source]¶

Bases: object

Creates an instance of an RDBComputeDomain, which defines a domain or environment with databases that users. are able to query.

Parameters:: rdb_compute_domain – Parameter that identifies the domain. Could be either its name (string), ID (integer), or a dictionary containing all the attributes of the domain.

get_database(database: Union[str, int, dict, Database]) → Database[source]¶: Gets Database

get_database_names() → list[source]¶

Gets a list of the names of databases in an RDBComputeDomain.

Returns:: list of database names (strings)
Example:: dbnames = SciQuery.get_database_names(rdbComputeDomainName);

See also

SciQuery.get_databases_metadata

get_databases_metadata() → DataFrame[source]¶

Gets metadata of the databases in this RDBComputeDomain.

Returns:: pandas dataframe with associated metadata.

See also

SciQuery.get_database_names

get_default_database() → Database[source]¶: Gets default database.

get_metadata(do_include_databases: bool = False) → DataFrame[source]¶: Gets metadata of this RDBComputeDomain.

class SciServer.SciQuery.RDBComputeDomains(rdb_compute_domains: Union[Iterable, RDBComputeDomain])[source]¶

Bases: list

Defines a list of RDBComputeDomains, which are domains or environments with databases that users are able to query.

Parameters:: rdb_compute_domains – Parameter that identifies a list of RDBComputeDomain objects. Could be either single RDBComputeDomain object, or an iterable containing multiple RDBComputeDomain objects.

get_default_rdb_compute_domain() → RDBComputeDomain[source]¶: Gets default RDBComputeDomain from this list.

get_rdb_compute_domain(rdb_compute_domain: Union[str, int, dict, RDBComputeDomain]) → RDBComputeDomain[source]¶: Gets RDBComputeDomain from this list

class SciServer.SciQuery.RDBJob(job)[source]¶

Bases: object

Contains the definition of a job consisting on a query run in a Relational Database (RDB).

Parameters:: job – can be the job ID (string), or a dictionary containing all the attributes of an RDBJob object.

cancel()[source]¶: Cancels this RDBJob.

get_dataframe_from_output(output: Union[Output, int] = 0, result_index: int = 0) → DataFrame[source]¶: Gets query output as a Pandas DatFrame.

static get_job(job_id: int)[source]¶: Gets RDBJob object from its Id.

static get_job_status(status: int) → str[source]¶: Gets job status string from its integer representation.

get_json_output(output: Union[Output, int, str] = 0) → dict[source]¶: Gets content of output file in SciServer’s filesystem as a dictionary.

get_metadata(result_format='pandas') → DataFrame[source]¶: Gets this RDBJob’s metadata as a Pandas DataFrame.

get_output_as_string(output: Union[Output, int, str] = None)[source]¶: Gets content of output file in SciServer’s filesystem as a string.

get_output_path(output: Union[Output, int] = 0) → str[source]¶: Gets output path on SciServer’s filesystem, if output type is a file.

refresh()[source]¶: Refreshes metadata and info of this RDBJob.

class SciServer.SciQuery.SciQuery(rdb_compute_domain: Union[str, int, dict, RDBComputeDomain] = None, database: Union[str, int, dict, Database] = None, file_service: Union[str, dict] = None, results_base_path: str = None, outputs: Outputs = None, verbose: bool = True, hard_fail: bool = False, poll_time: float = 1.0)[source]¶

Bases: object

Created an instance of the SciQuery app for querying relational databases.

Parameters:

rdb_compute_domain – defines a domain or environment of multiple databases where users can run queries. Can be either the domain’s name (string), ID (integer), an object of class RDBComputeDomain, or a dictionary containing all the attributes of an object of class RDBComputeDomain. If set to None, a default value will be assigned to it.
database – defines the database where the queries are executed in. Can be either the database name (string), ID (integer), an object of class Database, or a dictionary containing all the attributes of an object of class Database. If set to None, a default value will be assigned to it.
file_service – a File Service defines an available file system where query result sets can be written into. This parameter can be it name or identifier (string), or a dictionary defining a file service. If set to None, a default value will be assigned to it.
results_base_path – base path (string) of the directory where the query results are written into. Can be constructed by using FileOutput.build_file_base_path(). If set to None, a default value will be assigned to it at the moment of running a sql query.
outputs – Defines the query(ies) output(s). Can be an object derived from the Output base class (such as FileOutput or DatabaseTableOutput), or a list of those. If set to None, a default value (json file output) will be assigned to it.
verbose – Boolean parameter. If True, warning messages will be written in case of errors, in the case when the hard_fail parameter is set to False. If False, nothing will be written.
hard_fail – Boolean parameter. If True, exceptions will be raised in case of errors during instantiation. If False, then no exceptions are raised, and warnings might be showed instead (depending on the value of the verbose parameter).
poll_time – time (float) in seconds between consecutive requests for updates in the jobs status.

static cancel_job(job)[source]¶

Cancels the execution of a job.

Parameters:: job – id of the job (string), or object of class RDBJob.

property database: Database¶: Property defining the selected database in this SciQuery instance.

execute_query(sql_query, database: Union[str, int, dict, Database] = None, results_base_path: str = None, rdb_compute_domain: Union[str, int, dict, RDBComputeDomain] = None, job_alias: str = '', file_service: str = None, write_job_id=True) → DataFrame[source]¶

Returns the query result (as a Pandas data frame) of a sql query submitted as a job to a relational database (RDB) compute domain.

Parameters:

sql_query – sql query (string)
database – defines the database where the sql query is executed in. Can be either the database name (string), ID (integer), an object of class Database, or a dictionary containing all the attributes of an object of class Database. If set to None, then the current value of the database field in this SciQuery instance will be used.
results_base_path – full path to results folder (string) where query output tables are written into. E.g.: /home/idies/workspace/rootVOlume/username/userVolume/jobsFolder . If set to None, then its current value in this SciQuery instance will be used. If that value is None, then a default folder will be set automatically.
rdb_compute_domain – defines a domain or environment of multiple databases where users can run queries, and that contains the database. Can be either the domain’s name (string), ID (integer), an object of class RDBComputeDomain, or a dictionary containing all the attributes of an object of class RDBComputeDomain. If set to None, then the currently set value of rdb_compute_domain in the SciQuery object is internally used.
job_alias – alias (string) of job, defined by the user.
file_service – a File Service defines an available file system where query result sets can be written into. This parameter can be its name or identifier (string), or a dictionary defining a file service. If set to None, then the currently set value of file_service in the SciQuery object is internally used.
write_job_id – if True, the job id will be written on the screen, just before returning the result. The job id won;t be written if write_job_id = False. into. This parameter can be its name or identifier (string), or a dictionary defining a file service. If set to None, then the currently set value of file_service in the SciQuery object is internally used.

Returns:

Pandas data frame containing the result of the query.

property file_service: dict¶: Property defining the selected FileService available in this SciQuery instance.

get_column_names(table_name, database=None, rdb_compute_domain=None)[source]¶: Gets a list of the names of table columns in a particular database belonging to an RDBComputeDomain.

get_columns_metadata(table_name, database=None, rdb_compute_domain=None)[source]¶: Gets metadata related to columns in a particular database table belonging to an RDBComputeDomain.

get_constraint_names(table_name, database=None, rdb_compute_domain=None)[source]¶: Gets a list of the names of table constraints in a particular database belonging to an RDBComputeDomain.

get_constraints_metadata(table_name, database=None, rdb_compute_domain=None)[source]¶: Gets metadata related to table constraints in a particular database table belonging to an RDBComputeDomain.

get_database(database: Union[str, int, dict, Database] = None, rdb_compute_domain: Union[str, int, dict, RDBComputeDomain] = None) → Database[source]¶

Returns an object of class Database, either defined by the input name or identifiers, or that which is set in the SciQuery instance.

Parameters:

database – identifies the database, which this function returns as an object of class Database. Can be either the database name (string), ID (integer), an object of class Database, or a dictionary containing all the attributes of an object of class Database. If set to None, then the currently set value of database in the SciQuery object is returned.
rdb_compute_domain – defines a domain or environment of multiple databases where users can run queries, and that contains the database. Can be either the domain’s name (string), ID (integer), an object of class RDBComputeDomain, or a dictionary containing all the attributes of an object of class RDBComputeDomain. If set to None, then the currently set value of rdb_compute_domain in the SciQuery object is internally used.

Returns:

Object of class Database

get_database_metadata(database=None, rdb_compute_domain=None)[source]¶: Gets database metadata.

get_database_names(rdb_compute_domain=None)[source]¶

Gets a list of the names of the databases in this RDBComputeDomain.

Returns:: array of database names (strings)

get_databases_metadata(rdb_compute_domain=None)[source]¶: Gets metadata (name and description) of databases in an RDBComputeDomain.

get_default_database(rdb_compute_domain: Union[str, int, dict, RDBComputeDomain] = None) → Database[source]¶: Gets default database.

get_default_file_service() → dict[source]¶: Gets default FiLeService

get_default_outputs() → Outputs[source]¶: Gets default query outputs.

get_default_rdb_compute_domain()[source]¶: Gets default RDBComputeDomain.

get_default_results_base_path(add_date_ending=True) → str[source]¶: Gets default results base path on SciServer’s filesystem.

get_file_service(file_service: Union[str, dict] = None) → dict[source]¶

Returns the definition of a file service as a dictionary, either defined by the input name or identifiers,: or that which is set in the SciQuery instance.

Parameters:: file_service – name or identifier (string) of a file service, or the dictionary with its definition. If set to None, then the currently set value of file_service in the SciQuery object is returned.
Returns:: dictionary with the definition of a file service.

static get_job(job_id)[source]¶

Gets the definition of the job as a RDBJob object.

Parameters:: job_id – Id of job
Returns:: RDBJob object containing the description or definition of the job.

static get_job_status(job_id)[source]¶

Gets a dictionary with the job status as an integer value, together with its semantic meaning. The integer value is a power of 2, that is, 1:PENDING, 2:QUEUED, 4:ACCEPTED, 8:STARTED, 16:FINISHED, 32:SUCCESS, 64:ERROR, 128:CANCELED

Parameters:: job_id – Id of job (string).
Returns:: dictionary with the integer value of the job status, as well as its semantic meaning.

static get_jobs_list(top=5, open=None, start=None, end=None, result_format='pandas') → Union[DataFrame, list][source]¶

Gets the list of SciQuery Jobs submitted by the user.

Parameters:

top – top number of jobs (integer) returned. If top=None, then all jobs are returned.
open – If set to ‘True’, then only returns jobs that have not finished executing and wrapped up (status <= FINISHED). If set to ‘False’ then only returns jobs that are still running. If set to ‘None’, then returns both finished and unfinished jobs.
start – The earliest date (inclusive) to search for jobs, in string format yyyy-MM-dd hh:mm:ss.SSS. If set to ‘None’, then there is no lower bound on date.
end – The latest date (inclusive) to search for jobs, in string format yyyy-MM-dd hh:mm:ss.SSS. If set to ‘None’, then there is no upper bound on date.
result_format – string defining the return format. “pandas” for a pandas dataframe and “list” for a list of RDBJob objects.

Returns:

pandas dataframe, or list of RDBJob objects or, each containing the definition of a submitted job.

static get_mydb_name(owner_name: str = None) → str[source]¶: Returns name of mydb based on the owner’s name.

get_outputs(outputs: Union[Outputs, Output] = None) → Outputs[source]¶

Returns an object of class Outputs, either defined by the inputs parameters, or that which is set in the SciQuery instance.

Parameters:: outputs – object of class Outputs, or iterable of output objects. If set to None, then the currently set value of outputs in the SciQuery object is returned.
Returns:: object of class Outputs.

get_rdb_compute_domain(rdb_compute_domain: Union[str, int, dict, RDBComputeDomain] = None) → RDBComputeDomain[source]¶

Returns an object of class RDBComputeDomain, either defined by the input name or identifiers, or that which is set in the SciQuery instance.

Parameters:: rdb_compute_domain – defines a domain or environment of multiple databases where users can run queries. Can be either the domain’s name (string), ID (integer), an object of class RDBComputeDomain, or a dictionary containing all the attributes of an object of class RDBComputeDomain. If set to None, then the currently set value of rdb_compute_domain in the SciQuery object is returned.
Returns:: Object of class RDBComputeDomain.

get_rdb_compute_domain_metadata(rdb_compute_domain=None, do_include_databases=False)[source]¶: Gets metadata of an rdb_compute_domain.

get_rdb_compute_domain_names()[source]¶

Returns the names of the RDB compute domains available to the user.

Returns:: an array of strings, each being the name of a rdb compute domain available to the user.

static get_rdb_compute_domains(result_format: str = 'class') → RDBComputeDomains[source]¶

Gets a list of all registered Relational Database (RDB) compute domains that the user has access to.

Parameters:: result_format – If set to “class”, then the returned value will be of class RDBComputeDomains. If set to “dict”, then the return value will be a list of dictionaries, each of them containing the attributes of an RDBComputeDomain object.
Returns:: an object of class RDBComputeDomains, or a list of dictionaries, each of them containing the attributes of an RDBComputeDomain object.

get_rdb_compute_domains_metadata(do_include_databases=False)[source]¶

Gets metadata related to all relational database (RDB) compute domains (RDBComputeDomains) available.

Parameters:: do_include_databases – Boolean parameter. If True, it will return metadata related to all available databases in each RDBComputeDomain as well.
Returns:: pandas dataframe containing associated metadata.

get_results_base_path() → str[source]¶: Gets results base path on SciServer’s filesystem.

get_routine_names(database=None, rdb_compute_domain=None)[source]¶: Gets a list of the names of routines or functions in a particular database belonging to an RDBComputeDomain.

get_routine_parameter_names(routine_name, database=None, rdb_compute_domain=None)[source]¶: Gets a list of the names of routine parameters in a particular database belonging to an RDBComputeDomain.

get_routine_parameters_metadata(routine_name, database=None, rdb_compute_domain=None)[source]¶: Gets metadata related to routine parameters in a particular database belonging to an RDBComputeDomain.

get_routines_metadata(database=None, rdb_compute_domain=None)[source]¶: Gets metadata related to routines or functions in a particular database belonging to an RDBComputeDomain.

get_table_names(database=None, rdb_compute_domain=None)[source]¶: Gets a list of the names of tables in a particular database belonging to an RDBComputeDomain.

get_tables_metadata(database=None, rdb_compute_domain=None)[source]¶: Gets metadata related to tables in a particular database belonging to an RDBComputeDomain.

static get_token() → str[source]¶: Gets user’s auth token.

static get_user() → KeystoneUser[source]¶: Gets logged Keystone user info.

get_view_names(database=None, rdb_compute_domain=None)[source]¶: Gets a list of the names of views in a particular database belonging to an RDBComputeDomain.

get_views_metadata(database=None, rdb_compute_domain=None)[source]¶: Gets metadata related to views in a particular database belonging to an RDBComputeDomain.

property outputs: Outputs¶: Property defining a list of query result Output objects.

property rdb_compute_domain: RDBComputeDomain¶: Property defining the selected RDBComputeDomain in this SciQuery instance.

property rdb_compute_domains: RDBComputeDomains¶: Property defining the RDBComputeDomains available in this SciQuery instance.

refresh()[source]¶: Refreshes SciQuery instance.

property results_base_path: str¶: Property defining the file output base path in SciServer’s filesystem for this SciQuery instance.

set(rdb_compute_domain: Union[str, int, dict, RDBComputeDomain] = None, database: Union[str, int, dict, Database] = None, file_service: Union[str, dict] = None, results_base_path: str = None, outputs: Outputs = None, verbose: bool = None, hard_fail: bool = None, poll_time: float = None)[source]¶

Sets or refreshes the parameters in the SciQuery object, all at once.

Parameters:

rdb_compute_domain – defines a domain or environment of multiple databases where users can run queries. Can be either the domain’s name (string), ID (integer), an object of class RDBComputeDomain, or a dictionary containing all the attributes of an object of class RDBComputeDomain. If set to None, the current value is refreshed.
database – defines the database where the queries are executed in. Can be either the database name (string), ID (integer), an object of class Database, or a dictionary containing all the attributes of an object of class Database. If set to None, the current value is refreshed.
file_service – a File Service defines an available file system where query result sets can be written into. This parameter can be it name or identifier (string), or a dictionary defining a file service. If set to None, the current value is refreshed.
results_base_path – base path (string) of the directory where the query results are written into. Can be constructed by using FileOutput.build_file_base_path().
outputs – Defines the query(ies) output(s). Can be a list of Output objects, or a single object of class Outputs. If set to None, a default value (json file output) will be assigned to it.
verbose – Boolean parameter. If True, warning messages will be written in case of errors, in the case when the hard_fail parameter is set to False. If False, nothing will be written.
hard_fail – Boolean parameter. If True, exceptions will be raised in case of errors during instantiation. If False, then no exceptions are raised, and warnings might be showed instead (depending on the value of the verbose parameter).
poll_time – time (float) in seconds between consecutive requests for updates in the jobs status.

submit_query_job(sql_query: str, database: Union[str, int, dict, Database] = None, outputs: Union[Outputs, Output] = None, results_base_path: str = None, rdb_compute_domain: Union[str, int, dict, RDBComputeDomain] = None, file_service: str = None, job_alias: str = '') → int[source]¶

Submits a sql query for execution (as an asynchronous job) inside a relational database (RDB) compute domain.

Parameters:

sql_query – sql query (string)
database – defines the database where the sql query is executed in. Can be either the database name (string), ID (integer), an object of class Database, or a dictionary containing all the attributes of an object of class Database. If set to None, then the current value of the database field in this SciQuery instance will be used.
outputs – Defines the query(ies) output(s). Can be an object derived from the Output base class (such as FileOutput or DatabaseTableOutput), or a list of those. If set to None, then the current value of the outputs field in this SciQuery instance will be used.
results_base_path – full path to results folder (string) where query output tables are written into. E.g.: /home/idies/workspace/rootVolume/username/userVolume/jobsFolder . If set to None, then its current value in this SciQuery instance will be used. If that value is None, then a default folder will be set automatically.
rdb_compute_domain – defines a domain or environment of multiple databases where users can run queries, and that contains the database. Can be either the domain’s name (string), ID (integer), an object of class RDBComputeDomain, or a dictionary containing all the attributes of an object of class RDBComputeDomain. If set to None, then the currently set value of rdb_compute_domain in the SciQuery object is internally used.
file_service – a File Service defines an available file system where query result sets can be written into. This parameter can be its name or identifier (string), or a dictionary defining a file service. If set to None, then the currently set value of file_service in the SciQuery object is internally used.
job_alias – alias (string) of job, defined by the user.

Returns:

the ID (string) that labels the job.

wait_for_job(job_id, verbose=False)[source]¶

Queries the job status regularly and waits until the job is completed.

Parameters:

job_id – id of job (integer)
verbose – if True, will print “wait” messages on the screen while the job is still running. If False, it will suppress the printing of messages on the screen.

Returns:

After the job is finished, returns an object of class RDBJob, containing the job definition.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the JOBM API returns an error.

SciServer.Config module¶

SciServer.Config.isSciServerComputeEnvironment()[source]¶

Checks whether the library is being run within the SciServer-Compute environment.

Returns:: Returns True if the library is being run within the SciServer-Compute environment, and False if not.

SciServer.Authentication module¶

class SciServer.Authentication.KeystoneUser[source]¶

Bases: object

The class KeystoneUser stores the ‘id’ and ‘name’ of the user.

id = None¶

token = None¶

userName = None¶

class SciServer.Authentication.Token[source]¶

Bases: object

The class token stores the authentication token of the user in a particular session.

value = None¶

SciServer.Authentication.getKeystoneToken()[source]¶

Warning

Deprecated. Use Authentication.getToken instead.

Returns the users keystone token passed into the python instance with the –ident argument.

Returns:: authentication token (string)
Example:: token = Authentication.getKeystoneToken()

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.setToken, Authentication.token, Authentication.getToken.

SciServer.Authentication.getKeystoneUserWithToken(token)[source]¶

Returns the name and Keystone id of the user corresponding to the specified token.

Parameters:: token – Sciserver’s authentication token (string) for the user.
Returns:: Returns a KeystoneUser object, which stores the name and id of the user.
Raises:: Throws an exception if the HTTP request to the Authentication URL returns an error.
Example:: token = Authentication.getKeystoneUserWithToken(Authentication.getToken())

See also

Authentication.getToken, Authentication.login, Authentication.setToken.

SciServer.Authentication.getToken()[source]¶

Returns the SciServer authentication token of the user. First, will try to return Authentication.token.value. If Authentication.token.value is not set, Authentication.getToken will try to return the token value in the python instance argument variable “–ident”. If this variable does not exist, will try to return the token stored in Config.KeystoneTokenFilePath. Will return a None value if all previous steps fail.

Returns:: authentication token (string)
Example:: token = Authentication.getToken()

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.setToken, Authentication.token.

SciServer.Authentication.identArgIdentifier()[source]¶

Returns the name of the python instance argument variable where the user token is stored.

Returns:: name (string) of the python instance argument variable where the user token is stored.
Example:: name = Authentication.identArgIdentifier()

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.getToken, Authentication.token.

SciServer.Authentication.login(UserName=None, Password=None)[source]¶

Logs the user into SciServer and returns the authentication token. This function is useful when SciScript-Python library methods are executed outside the SciServer-Compute environment. In this case, the session authentication token does not exist (and therefore can’t be automatically recognized), so the user has to use Authentication.login in order to log into SciServer manually and get the authentication token. Authentication.login also sets the token value in the python instance argument variable “–ident”, and as the local object Authentication.token (of class Authentication.Token).

Parameters:

UserName – name of the user (string). If not set, then a user name prompt will show.
Password – password of the user (string). If not set, then a password prompt will show.

Returns:

authentication token (string)

Raises:

Throws an exception if the HTTP request to the Authentication URL returns an error.

Example:

token1 = Authentication.login(); token2 = Authentication.login(‘loginName’,’loginPassword’);

See also

Authentication.getKeystoneUserWithToken, Authentication.getToken, Authentication.setToken, Authentication.token.

SciServer.Authentication.setKeystoneToken(_token)[source]¶

Warning

Deprecated. Use Authentication.setToken instead.

Sets the token as the –ident argument

Parameters:: _token – authentication token (string)
Example:: Authentication.setKeystoneToken(“myToken”)

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.setToken, Authentication.token, Authentication.getToken.

SciServer.Authentication.setToken(_token)[source]¶

Sets the SciServer authentication token of the user in the variable Authentication.token.value, as well as in the python instance argument variable “–ident”.

Parameters:: _token – Sciserver’s authentication token for the user (string)
Example:: Authentication.setToken(‘myToken’)

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.getToken, Authentication.token.

SciServer.LoginPortal module¶

class SciServer.LoginPortal.KeystoneUser[source]¶

Bases: object

Warning

Deprecated. Use SciServer.Authentication.KeystoneUser instead.

The class KeystoneUser stores the ‘id’ and ‘name’ of the user.

id = 'KeystoneID'¶

userName = 'User Name'¶

SciServer.LoginPortal.getKeystoneToken()[source]¶

Warning

Deprecated. Use Authentication.getToken instead.

Returns the users keystone token passed into the python instance with the –ident argument.

Returns:: authentication token (string)
Example:: token = Authentication.getKeystoneToken()

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.setToken, Authentication.token, Authentication.getToken.

SciServer.LoginPortal.getKeystoneUserWithToken(token)[source]¶

Warning

Deprecated. Use SciServer.Authentication.getKeystoneUserWithToken instead.

Returns the name and Keystone id of the user corresponding to the specified token.

Parameters:: token – Sciserver’s authentication token (string) for the user.
Returns:: Returns a KeystoneUser object, which stores the name and id of the user.
Raises:: Throws an exception if the HTTP request to the Authentication URL returns an error.
Example:: token = Authentication.getKeystoneUserWithToken(Authentication.getToken())

See also

Authentication.getToken, Authentication.login, Authentication.setToken.

SciServer.LoginPortal.getToken()[source]¶

Warning

Deprecated. Use SciServer.Authentication.getToken instead.

Returns the SciServer authentication token of the user. First, will try to return Authentication.token.value. If Authentication.token.value is not set, Authentication.getToken will try to return the token value in the python instance argument variable “–ident”. If this variable does not exist, will try to return the token stored in Config.KeystoneTokenFilePath. Will return a None value if all previous steps fail.

Returns:: authentication token (string)
Example:: token = Authentication.getToken()

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.setToken, Authentication.token.

SciServer.LoginPortal.identArgIdentifier()[source]¶

Warning

Deprecated. Use SciServer.Authentication.identArgIdentifier instead.

Returns the name of the python instance argument variable where the user token is stored.

Returns:: name (string) of the python instance argument variable where the user token is stored.
Example:: name = Authentication.identArgIdentifier()

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.getToken, Authentication.token.

SciServer.LoginPortal.login(UserName, Password)[source]¶

Warning

Deprecated. Use SciServer.Authentication.login instead.

Logs the user into SciServer and returns the authentication token. This function is useful when SciScript-Python library methods are executed outside the SciServer-Compute environment. In this case, the session authentication token does not exist (and therefore can’t be automatically recognized), so the user has to use Authentication.login in order to log into SciServer manually and get the authentication token. Authentication.login also sets the token value in the python instance argument variable “–ident”, and as the local object Authentication.token (of class Authentication.Token).

Parameters:

UserName – name of the user (string)
Password – password of the user (string)

Returns:

authentication token (string)

Raises:

Throws an exception if the HTTP request to the Authentication URL returns an error.

Example:

token = Authentication.login(‘loginName’,’loginPassword’)

See also

Authentication.getKeystoneUserWithToken, Authentication.getToken, Authentication.setToken, Authentication.token.

SciServer.LoginPortal.setKeystoneToken(token)[source]¶

Warning

Deprecated. Use Authentication.setToken instead.

Sets the token as the –ident argument

Parameters:: _token – authentication token (string)
Example:: Authentication.setKeystoneToken(“myToken”)

See also

Authentication.getKeystoneUserWithToken, Authentication.login, Authentication.setToken, Authentication.token, Authentication.getToken.

SciServer.SciDrive module¶

class SciServer.SciDrive.Task[source]¶

Bases: object

The class TaskName stores the name of the task that executes the API call.

name = None¶

SciServer.SciDrive.createContainer(path)[source]¶

Creates a container (directory) in SciDrive

Parameters:: path – path of the directory in SciDrive.
Returns:: Returns True if the container (directory) was created successfully.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SciDrive API returns an error.
Example:: response = SciDrive.createContainer(“MyDirectory”)

See also

SciDrive.upload.

SciServer.SciDrive.delete(path)[source]¶

Deletes a file or container (directory) in SciDrive.

Parameters:: path – path of the file or container (directory) in SciDrive.
Returns:: Returns True if the file or container (directory) was deleted successfully.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SciDrive API returns an error. :example: response = SciDrive.delete(“path/to/SciDrive/file.csv”)

See also

SciDrive.upload.

SciServer.SciDrive.directoryList(path='')[source]¶

Gets the contents and metadata of a SciDrive directory (or file).

Parameters:: path – path of the directory (or file ) in SciDrive.
Returns:: a dictionary containing info and metadata of the directory (or file).
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SciDrive API returns an error.
Example:: dirList = SciDrive.directoryList(“path/to/SciDrive/directory”)

See also

SciDrive.upload, SciDrive.download

SciServer.SciDrive.download(path, format='text', localFilePath='')[source]¶

Downloads a file (directory) from SciDrive into the local file system, or returns the file conetent as an object in several formats.

Parameters:

path – path of the file (or directory) in SciDrive.
format – type of the returned object. Can be “StringIO” (io.StringIO object containing readable text), “BytesIO” (io.BytesIO object containing readable binary data), “response” ( the HTTP response as an object of class requests.Response) or “text” (a text string). If the parameter ‘localFilePath’ is defined, then the ‘format’ parameter is not used and the file is downloaded to the local file system instead.
localFilePath – local path of the file to be downloaded. If ‘localFilePath’ is defined, then the ‘format’ parameter is not used.

Returns:

If the ‘localFilePath’ parameter is defined, then it will return True when the file is downloaded successfully in the local file system. If the ‘localFilePath’ is not defined, then the type of the returned object depends on the value of the ‘format’ parameter (either io.StringIO, io.BytesIO, requests.Response or string).

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SciDrive API returns an error.

Example:

csvString = SciDrive.download(“path/to/SciDrive/file.csv”, format=”text”);

See also

SciDrive.upload

SciServer.SciDrive.publicUrl(path)[source]¶

Gets the public URL of a file (or directory) in SciDrive.

Parameters:: path – path of the file (or directory) in SciDrive.
Returns:: URL of a file in SciDrive (string).
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SciDrive API returns an error.
Example:: url = SciDrive.publicUrl(“path/to/SciDrive/file.csv”)

See also

SciDrive.upload

SciServer.SciDrive.upload(path, data='', localFilePath='')[source]¶

Uploads data or a local file into a SciDrive directory.

Parameters:

path – desired file path in SciDrive (string).
data – data to be uploaded into SciDrive. If the ‘localFilePath’ parameter is set, then the local file will be uploaded instead.
localFilePath – path to the local file to be uploaded (string).

Returns:

Returns an object with the attributes of the uploaded file.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SciDrive API returns an error.

Example:

response = SciDrive.upload(“/SciDrive/path/to/file.csv”, localFilePath=”/local/path/to/file.csv”)

See also

SciDrive.createContainer

SciServer.SkyServer module¶

SciServer.SkyServer.getJpegImgCutout(ra, dec, scale=0.7, width=512, height=512, opt='', query='', dataRelease=None)[source]¶

Gets a rectangular image cutout from a region of the sky in SDSS, centered at (ra,dec). Return type is numpy.ndarray.

Parameters:

ra – Right Ascension of the image’s center in floating point ICRS J2000 coordinates.
dec – Declination of the image’s center in floating point ICRS J2000 coordinates.
scale – Scale of the image, measured in [arcsec/pix]
width – Width of image, measured in pixels (width in arcsec = width * scale). If larger than 2048, silently truncated to 2048.
height – Height of image, measured in pixels (width in arcsec = width * scale). If larger than 2048, silently truncated to 2048.
opt –
Optional drawing options, expressed as concatenation of letters (string). The letters options are

”G”: Grid. Draw a N-S E-W grid through the center

”L”: Label. Draw the name, scale, ra, and dec on image.

”P PhotoObj. Draw a small cicle around each primary photoObj.

”S: SpecObj. Draw a small square around each specObj.

”O”: Outline. Draw the outline of each photoObj.

”B”: Bounding Box. Draw the bounding box of each photoObj.

”F”: Fields. Draw the outline of each field.

”M”: Masks. Draw the outline of each mask considered to be important.

”Q”: Plates. Draw the outline of each plate.

”I”: Invert. Invert the image (B on W).

(see http://skyserver.sdss.org/public/en/tools/chart/chartinfo.aspx)
query –
Optional string. Marks with inverted triangles on the image the position of user defined objects. The (RA,Dec) coordinates of these object can be given by three means:
1. query is a SQL command of format “SELECT Id, RA, Dec, FROM Table”.
2. query is list of objects. A header with RA and DEC columns must be included. Columns must be separated by tabs, spaces, commas or semicolons. The list may contain as many columns as wished.
3. query is a string following the pattern: ObjType Band (low_mag, high_mag).
  ObjType: S | G | P marks Stars, Galaxies or PhotoPrimary objects.
  
  Band: U | G | R | I | Z | A restricts marks to objects with Band BETWEEN low_mag AND high_mag Band ‘A’ will mark all objects within the specified magnitude range in any band (ORs composition).
Examples:

S

S R (0.0, 23.5)

G A (20, 30)

(see http://skyserver.sdss.org/public/en/tools/chart/chartinfo.aspx)
dataRelease – SDSS data release string. Example: dataRelease=’DR13’. Default value already set in SciServer.Config.DataRelease

Returns:

Returns the image as a numpy.ndarray object.

Raises:

Throws an exception if the HTTP request to the SkyServer API returns an error.

Example:

img = SkyServer.getJpegImgCutout(ra=197.614455642896, dec=18.438168853724, width=512, height=512, scale=0.4, opt=”OG”, query=”SELECT TOP 100 p.objID, p.ra, p.dec, p.r FROM fGetObjFromRectEq(197.6,18.4,197.7,18.5) n, PhotoPrimary p WHERE n.objID=p.objID”)

SciServer.SkyServer.objectSearch(objId=None, specObjId=None, apogee_id=None, apstar_id=None, ra=None, dec=None, plate=None, mjd=None, fiber=None, run=None, rerun=None, camcol=None, field=None, obj=None, dataRelease=None)[source]¶

Gets the properties of the the object that is being searched for. Search parameters:

Parameters:

objId – SDSS ObjId.
specObjId – SDSS SpecObjId.
apogee_id – ID idetifying Apogee target object.
apstar_id – unique ID for combined apogee star spectrum.
ra – right ascention.
dec – declination.
plate – SDSS plate number.
mjd – Modified Julian Date of observation.
fiber – SDSS fiber number.
run – SDSS run number.
rerun – SDSS rerun number.
camcol – SDSS camera column.
field – SDSS field number.
obj – The object id within a field.
dataRelease – SDSS data release string. Example: dataRelease=’DR13’. Default value already set in SciServer.Config.DataRelease

Returns:

Returns a list containing the properties and metadata of the astronomical object found.

Raises:

Throws an exception if the HTTP request to the SkyServer API returns an error.

Example:

object = SkyServer.objectSearch(ra=258.25, dec=64.05)

See also

SkyServer.sqlSearch, SkyServer.rectangularSearch, SkyServer.radialSearch.

SciServer.SkyServer.radialSearch(ra, dec, radius=1, coordType='equatorial', whichPhotometry='optical', limit='10', dataRelease=None)[source]¶

Runs a query in the SDSS database that searches for all objects within a certain radius from a point in the sky, and retrieves the result table as a Panda’s dataframe.

Parameters:

ra – Right Ascension of the image’s center.
dec – Declination of the image’s center.
radius – Search radius around the (ra,dec) coordinate in the sky. Measured in arcminutes.
coordType – Type of celestial coordinate system. Can be set to “equatorial” or “galactic”.
whichPhotometry – Type of retrieved data. Can be set to “optical” or “infrared”.
limit – Maximum number of rows in the result table (string). If set to “0”, then the function will return all rows.
dataRelease – SDSS data release string. Example: dataRelease=’DR13’. Default value already set in SciServer.Config.DataRelease

Returns:

Returns the results table as a Pandas data frame.

Raises:

Throws an exception if the HTTP request to the SkyServer API returns an error.

Example:

df = SkyServer.radialSearch(ra=258.25, dec=64.05, radius=3)

See also

SkyServer.sqlSearch, SkyServer.rectangularSearch.

SciServer.SkyServer.rectangularSearch(min_ra, max_ra, min_dec, max_dec, coordType='equatorial', whichPhotometry='optical', limit='10', dataRelease=None)[source]¶

Runs a query in the SDSS database that searches for all objects within a certain rectangular box defined on the the sky, and retrieves the result table as a Panda’s dataframe.

Parameters:

min_ra – Minimum value of Right Ascension coordinate that defines the box boundaries on the sky.
max_ra – Maximum value of Right Ascension coordinate that defines the box boundaries on the sky.
min_dec – Minimum value of Declination coordinate that defines the box boundaries on the sky.
max_dec – Maximum value of Declination coordinate that defines the box boundaries on the sky.
coordType – Type of celestial coordinate system. Can be set to “equatorial” or “galactic”.
whichPhotometry – Type of retrieved data. Can be set to “optical” or “infrared”.
limit – Maximum number of rows in the result table (string). If set to “0”, then the function will return all rows.
dataRelease – SDSS data release string. Example: dataRelease=’DR13’. Default value already set in SciServer.Config.DataRelease

Returns:

Returns the results table as a Pandas data frame.

Raises:

Throws an exception if the HTTP request to the SkyServer API returns an error.

Example:

df = SkyServer.rectangularSearch(min_ra=258.2, max_ra=258.3, min_dec=64,max_dec=64.1)

See also

SkyServer.sqlSearch, SkyServer.radialSearch.

SciServer.SkyServer.sqlSearch(sql, dataRelease=None)[source]¶

Executes a SQL query to the SDSS database, and retrieves the result table as a dataframe. Maximum number of rows retrieved is set currently to 500,000.

Parameters:

sql – a string containing the sql query
dataRelease – SDSS data release (string). E.g, ‘DR13’. Default value already set in SciServer.Config.DataRelease

Returns:

Returns the results table as a Pandas data frame.

Raises:

Throws an exception if the HTTP request to the SkyServer API returns an error.

Example:

df = SkyServer.sqlSearch(sql=”select 1”)

See also

CasJobs.executeQuery, CasJobs.submitJob.

SciServer.SkyQuery module¶

SciServer.SkyQuery.cancelJob(jobId)[source]¶

Cancels a single job (more info in http://www.voservices.net/skyquery).

Parameters:: jobId – the ID of the job, which is obtained at the moment of submitting the job.
Returns:: Returns True if the job was cancelled successfully.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: isCanceled = SkyQuery.cancelJob(SkyQuery.submitJob(“select 1 as foo”))

See also

SkyQuery.submitJob, SkyQuery.getJobStatus

SciServer.SkyQuery.dropTable(tableName, datasetName='MyDB')[source]¶

Drops (deletes) a table from the user database (more info in http://www.voservices.net/skyquery).

Parameters:

tableName – name of table (string) within dataset.
datasetName – name of dataset or database context (string).

Returns:

returns True if the table was deleted successfully.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.

Example:

result = SkyQuery.dropTable(“myTable”, datasetName=”MyDB”)

See also

SkyQuery.listQueues, SkyQuery.listAllDatasets, SkyQuery.getDatasetInfo, SkyQuery.listDatasetTables, SkyQuery.getTableInfo, SkyQuery.getTable, SkyQuery.submitJob

SciServer.SkyQuery.getDatasetInfo(datasetName='MyDB')[source]¶

Gets information related to a particular dataset (more info in http://www.voservices.net/skyquery).

Parameters:: datasetName – name of dataset (string).
Returns:: returns a dictionary containing the dataset information.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: info = SkyQuery.getDatasetInfo(“MyDB”)

See also

SkyQuery.listQueues, SkyQuery.listAllDatasets, SkyQuery.listDatasetTables, SkyQuery.getTableInfo, SkyQuery.listTableColumns, SkyQuery.getTable, SkyQuery.dropTable

SciServer.SkyQuery.getJobStatus(jobId)[source]¶

Gets the status of a job, as well as other related metadata (more info in http://www.voservices.net/skyquery).

Parameters:: jobId – the ID of the job (string), which is obtained at the moment of submitting the job.
Returns:: a dictionary with the job status and other related metadata.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: status = SkyQuery.getJobStatus(SkyQuery.submitJob(“select 1 as foo”))

See also

SkyQuery.submitJob, SkyQuery.cancelJob

SciServer.SkyQuery.getQueueInfo(queue)[source]¶

Returns information about a particular job queue (more info in http://www.voservices.net/skyquery).

Parameters:: queue – queue name (string)
Returns:: a dictionary containing information associated to the queue.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: queueInfo = SkyQuery.getQueueInfo(‘quick’)

See also

SkyQuery.listQueues, SkyQuery.submitJob, SkyQuery.getJobStatus

SciServer.SkyQuery.getTable(tableName, datasetName='MyDB', top=None)[source]¶

Returns a dataset table as a pandas DataFrame (more info in http://www.voservices.net/skyquery).

Parameters:

tableName – name of table (string) within dataset.
datasetName – name of dataset or database context (string).
top – number of top rows retrieved (integer).

Returns:

returns the table as a Pandas dataframe.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.

Example:

table = SkyQuery.getTable(“myTable”, datasetName=”MyDB”, top=10)

See also

SkyQuery.listQueues, SkyQuery.listAllDatasets, SkyQuery.getDatasetInfo, SkyQuery.listDatasetTables, SkyQuery.getTableInfo, SkyQuery.dropTable, SkyQuery.submitJob

SciServer.SkyQuery.getTableInfo(tableName, datasetName='MyDB')[source]¶

Returns info about a particular table belonging to a dataset (more info in http://www.voservices.net/skyquery).

Parameters:

tableName – name of table (string) within dataset.
datasetName – name of dataset (string).

Returns:

returns a dictionary containing the table properties and associated info/metadata.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.

Example:

info = SkyQuery.getTableInfo(“myTable”, datasetName=”MyDB”)

See also

SkyQuery.listQueues, SkyQuery.listAllDatasets, SkyQuery.getDatasetInfo, SkyQuery.listDatasetTables, SkyQuery.listTableColumns, SkyQuery.getTable, SkyQuery.dropTable

SciServer.SkyQuery.listAllDatasets()[source]¶

Lists all available datasets (more info in http://www.voservices.net/skyquery).

Returns:: returns dataset definitions as a list object.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: datasets = SkyQuery.listAllDatasets()

See also

SkyQuery.listQueues, SkyQuery.getDatasetInfo, SkyQuery.listDatasetTables, SkyQuery.getTableInfo, SkyQuery.listTableColumns, SkyQuery.getTable, SkyQuery.dropTable

SciServer.SkyQuery.listDatasetTables(datasetName='MyDB')[source]¶

Returns a list of all tables within a dataset (more info in http://www.voservices.net/skyquery).

Parameters:: datasetName – name of dataset (string).
Returns:: returns a list containing the tables and associated descriptions/metadata.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: tables = SkyQuery.listDatasetTables(“MyDB”)

See also

SkyQuery.listQueues, SkyQuery.listAllDatasets, SkyQuery.getDatasetInfo, SkyQuery.getTableInfo, SkyQuery.listTableColumns, SkyQuery.getTable, SkyQuery.dropTable

SciServer.SkyQuery.listJobs(queue='quick')[source]¶

Lists the jobs in the queue in descending order by submission time. Only jobs of the authenticated user are listed (more info in http://www.voservices.net/skyquery).

Parameters:: queue – queue name (string). Can be set to ‘quick’ for a quick job, or ‘long’ for a long job.
Returns:: returns job definitions as a list object.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: jobsList = SkyQuery.listJobs(‘quick’)

See also

SkyQuery.getJobStatus, SkyQuery.listQueues

SciServer.SkyQuery.listQueues()[source]¶

Returns a list of all available job queues and related metadata (more info in http://www.voservices.net/skyquery).

Returns:: a list of all available job queues and related metadata.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.
Example:: queueList = SkyQuery.listQueues()

See also

SkyQuery.getQueueInfo, SkyQuery.submitJob, SkyQuery.getJobStatus

SciServer.SkyQuery.listTableColumns(tableName, datasetName='MyDB')[source]¶

Returns a list of all columns in a table belonging to a particular dataset (more info in http://www.voservices.net/skyquery).

Parameters:

tableName – name of table (string) within dataset.
datasetName – name of dataset (string).

Returns:

returns a list containing the columns and associated descriptions.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.

Example:

columns = SkyQuery.listTableColumns(“myTable”, datasetName=”MyDB”)

See also

SkyQuery.listQueues, SkyQuery.listAllDatasets, SkyQuery.getDatasetInfo, SkyQuery.listDatasetTables, SkyQuery.getTableInfo, SkyQuery.getTable, SkyQuery.dropTable

SciServer.SkyQuery.submitJob(query, queue='quick')[source]¶

Submits a new job (more info in http://www.voservices.net/skyquery).

Parameters:

query – sql query (string)
queue – queue name (string). Can be set to ‘quick’ for a quick job, or ‘long’ for a long job.

Returns:

returns the jobId (string), unique identifier of the job.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.

Example:

jobId = SkyQuery.submitJob(‘select 1 as foo’, “quick”)

See also

SkyQuery.getJobStatus, SkyQuery.listQueues

SciServer.SkyQuery.uploadTable(uploadData, tableName, datasetName='MyDB', format='csv')[source]¶

Uploads a data table into a database (more info in http://www.voservices.net/skyquery).

Parameters:

uploadData – data table, for now accepted in CSV string format.
tableName – name of table (string) within dataset.
datasetName – name of dataset or database context (string).
format – format of the ‘data’ parameter. Set to ‘csv’ for now.

Returns:

returns True if the table was uploaded successfully.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.

Example:

result = SkyQuery.uploadTable(“Column1,Column2n4.5,5.5n”, tableName=”myTable”, datasetName=”MyDB”, format=”csv”)

See also

SkyQuery.listQueues, SkyQuery.listAllDatasets, SkyQuery.getDatasetInfo, SkyQuery.listDatasetTables, SkyQuery.getTableInfo, SkyQuery.getTable, SkyQuery.submitJob

SciServer.SkyQuery.waitForJob(jobId, verbose=False, pollTime=5)[source]¶

Queries regularly the job status and waits until the job is completed.

Parameters:

jobId – id of job (integer)
verbose – if True, will print “wait” messages on the screen while the job is still running. If False, will suppress the printing of messages on the screen.
pollTime – idle time interval (integer, in seconds) before querying again for the job status. Minimum value allowed is 5 seconds.

Returns:

After the job is finished, returns a dictionary object containing the job status and related metadata.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the SkyQuery API returns an error.

Example:

SkyQuery.waitForJob(SkyQuery.submitJob(“select 1”))

See also

SkyQuery.submitJob, SkyQuery.getJobStatus.

SciServer.Files module¶

SciServer.Files.createDir(fileService, path, quiet=True)[source]¶

Create a directory.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – path (in the remote file service) to the directory (string), starting from the root volume level or data volume level. Examples: rootVolume/userVolumeOwner/userVolume/directory or dataVolume/directory
quiet – If set to False, it will throw an error if the directory already exists. If set to True. it will not throw an error.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.createDir(fileServices[0], “Storage/myUserName/persistent/myNewDir”);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.delete, Files.upload, Files.download, Files.dirList

SciServer.Files.createUserVolume(fileService, path, quiet=True)[source]¶

Create a user volume.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines the file service that contains the user volume. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – path (in the remote file service) to the user volume (string), starting from the root volume level. Example: rootVolume/userVolumeOwner/userVolume
quiet – if set to False, will throw an error if the User Volume already exists. If True, won’t throw an error.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.createUserVolume(fileServices[0], “volumes”,”newUserVolume”);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.delete, Files.upload, Files.download, Files.dirList

SciServer.Files.delete(fileService, path, quiet=True)[source]¶

Deletes a directory or file in the File System.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – String defining the path (in the remote fileService) of the file or directory to be deleted, starting from the root volume level or data volume level. Examples: rootVolume/userVolumeOwner/userVolume/fileToBeDeleted.txt or dataVolume/fileToBeDeleted.txt
quiet – If set to False, it will throw an error if the file does not exist. If set to True. it will not throw an error.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.delete(fileServices[0], “Storage/myUserName/persistent/myUselessFile.txt”);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.createDir, Files.upload, Files.download, Files.dirList

SciServer.Files.deleteUserVolume(fileService, path, quiet=True)[source]¶

Delete a user volume.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines the file service that contains the root volume. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – path (in the remote file service) to the user volume (string), starting from the root volume level. Example: rootVolume/userVolumeOwner/userVolume
quiet – If set to False, it will throw an error if user volume does not exists. If set to True. it will not throw an error.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.deleteUserVolume(“volumes”,”newUserVolume”,fileServices[0]);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.delete, Files.upload, Files.download, Files.dirList

SciServer.Files.dirList(fileService, path, level=1, options='')[source]¶

Lists the contents of a directory.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – String defining the path (in the remote file service) of the directory to be listed, starting from the root volume level or data volume level. Examples: rootVolume/userVolumeOwner/userVolume/directoryToBeListed or dataVolume/directoryToBeListed
level – amount (int) of listed directory levels that are below or at the same level to that of the relativePath.
options – string of file filtering options.

Returns:

dictionary containing the directory listing.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); dirs = Files.dirList(fileServices[0], “Storage/myUserName/persistent/”, level=2);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.delete, Files.upload, Files.download, Files.createDir

SciServer.Files.download(fileService, path, localFilePath=None, format='txt', quiet=True)[source]¶

Downloads a file from the remote file system into the local file system, or returns the file content as an object in several formats.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – String defining the path (in the remote file service) of the file to be downloaded, starting from the root volume level or data volume level. Examples: rootVolume/userVolumeOwner/userVolume/fileToBeDownloaded.txt or dataVolume/fileToBeDownloaded.txt
localFilePath – local destination path of the file to be downloaded. If set to None, then an object of format ‘format’ will be returned.
format – name (string) of the returned object’s type (if localFilePath is not defined). This parameter can be “StringIO” (io.StringIO object containing readable text), “BytesIO” (io.BytesIO object containing readable binary data), “response” ( the HTTP response as an object of class requests.Response) or “txt” (a text string). If the parameter ‘localFilePath’ is defined, then the ‘format’ parameter is not used and the file is downloaded to the local file system instead.
userVolumeOwner – name (string) of owner of the volume. Can be left undefined if requester is the owner of the volume.
quiet – If set to False, it will throw an error if the file already exists. If set to True. it will not throw an error.

Returns:

If the ‘localFilePath’ parameter is defined, then it will return True when the file is downloaded successfully in the local file system. If the ‘localFilePath’ is not defined, then the type of the returned object depends on the value of the ‘format’ parameter (either io.StringIO, io.BytesIO, requests.Response or string).

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.upload(fileServices[0], “Storage/myUserName/persistent/fileToBeDownloaded.txt”, localFilePath=”/fileToBeDownloaded.txt”);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.createDir, Files.delete, Files.upload, Files.dirList

SciServer.Files.getDataVolumesInfo(fileService, verbose=True)[source]¶

Gets the names and descriptions of data volumes available to the user in a particular FileService.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
verbose – boolean parameter defining whether warnings will be printed (set to True) or not (set to False).

Returns:

list of dictionaries, where each dictionary contains the name and description of a data volume.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the RACM API returns an error.

Example:

fileServices = Files.getFileServices(); dataVolumesInfo = getDataVolumesInfo(fileServices[0])

See also

Files.getRootVolumesInfo

SciServer.Files.getFileServiceFromName(fileServiceName, fileServices=None, verbose=True)[source]¶

Returns a FileService object, given its registered name.

Parameters:

fileServiceName – name of the FileService, as shown within the results of Files.getFileServices()
fileServices – a list of FileService objects (dictionaries), as returned by Files.getFileServices(). If not set, then an extra internal call to Jobs.getFileServices() is made.
verbose – boolean parameter defining whether warnings will be printed (set to True) or not (set to False).

Returns:

a FileService object (dictionary) that defines a FileService. A list of these kind of objects available to the user is returned by the function Jobs.getFileServices(). If no fileService can be found, then returns None.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the RACM API returns an error.

Example:

fileService = Files.getFileServiceFromName(‘FileServiceAtJHU’);

See also

Files.getFileServices

SciServer.Files.getFileServices(verbose=True)[source]¶

Gets the definitions of file services that a user is able to access. A FileService represents a file system that contains root volumes accessible to the user for public/private data storage. Within each rootVolume, users can create sharable userVolumes for storing files.

Parameters:: verbose – boolean parameter defining whether warnings will be printed (set to True) or not (set to False).
Returns:: list of dictionaries, where each dictionary represents the description of a FileService that the user is able to access.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the RACM API returns an error.
Example:: fileServices = Files.getFileServices();

See also

Files.getFileServiceFromName

SciServer.Files.getFileServicesNames(fileServices=None, verbose=True)[source]¶

Returns the names and description of the fileServices available to the user.

Parameters:

fileServices – a list of FileService objects (dictionaries), as returned by Files.getFileServices(). If not set, then an extra internal call to Jobs.getFileServices() is made.
verbose – boolean parameter defining whether warnings will be printed (set to True) or not (set to False).

Returns:

an array of dicts, where each dict has the name and description of a file service available to the user.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the RACM API returns an error.

Example:

fileServiceNames = Files.getFileServicesNames();

See also

Files.getFileServices

SciServer.Files.getRootVolumesInfo(fileService, verbose=True)[source]¶

Gets the names and descriptions of root volumes available to the user in a particular FileService.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
verbose – boolean parameter defining whether warnings will be printed (set to True) or not (set to False).

Returns:

list of dictionaries, where each dictionary contains the name and description of a root volume.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the RACM API returns an error.

Example:

fileServices = Files.getFileServices(); rootVolumesInfo = getRootVolumesInfo(fileServices[0])

See also

Files.getUserVolumesInfo

SciServer.Files.getUserVolumesInfo(fileService, rootVolumeName=None, verbose=True)[source]¶

Gets the names definitions the RootVolumes available in a particular FileService, and of the file services that a user is able to access.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
rootVolumeName – name of root Volume (string) for which the user volumes are fetched. If set to None, then user volumes in all root folders are fetched.
verbose – boolean parameter defining whether warnings will be printed (set to True) or not (set to False).

Returns:

list of dictionaries, where each dictionary contains the name and description of a root volumes that a user is able to access.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the RACM API returns an error.

Example:

fileServices = Files.getFileServices(); rootVolumeNames = Files.getUserVolumesInfo(fileServices[0])

See also

Files.getRootVolumes,

SciServer.Files.move(fileService, path, destinationFileService, destinationPath, replaceExisting=True, doCopy=True)[source]¶

Moves or copies a file or folder.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – String defining the origin path (in the remote fileService) of the file or directory to be copied/moved, starting from the root volume level or data volume level. Examples: rootVolume/userVolumeOwner/userVolume/fileToBeMoved.txt or dataVolume/fileToBeMoved.txt
destinationFileService – name of fileService (string), or object (dictionary) that defines a destination file service (where the file is moved/copied into). A list of these kind of objects available to the user is returned by the function Files.getFileServices().
destinationRelativePath – String defining the destination path (in the remote destinationFileService) of the file or directory to be copied/moved, starting from the root volume level or data volume level. Examples: rootVolume/userVolumeOwner/userVolume/recentlyMovedFile.txt or dataVolume/recentlyMovedFile.txt}
replaceExisting – If set to False, it will throw an error if the file already exists, If set to True, it will not throw and eeror in that case.
doCopy – if set to True, then it will copy the file or folder. If set to False, then the file or folder will be moved.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.move(fileServices[0], “Storage/myUserName/persistent/myFile.txt”, fileServices[0], “Storage/myUserName/persistent/myFile2.txt”)

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.createDir, Files.delete, Files.upload, Files.dirList

SciServer.Files.shareUserVolume(fileService, path, sharedWith, allowedActions, sharee_type='USER')[source]¶

Shares a user volume with another user or group

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – String defining the path (in the remote fileService) of the user volume to be shared, starting from the root volume level. Example: rootVolume/userVolumeOwner/userVolume.
sharedWith – name (string) of user or group that the user volume is shared with.
allowedActions – array of strings defining actions the user or group is allowed to do with respect to the shared user volume. E.g.: [“read”,”write”,”grant”,”delete”]. The “grant” action means that the user or group can also share the user volume with another user or group. The “delete” action meand ability to delete the user volume (use with care).
sharee_type – type (string) of the entity defined by the “sharedWith” parameter. Can be set to “USER” or “GROUP”.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.shareUserVolume(getFileServices[0], “Storage/myUserName/myUSerVolume”, “userName”, [“read”,”write”], type=”USER”);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.createDir, Files.upload, Files.download, Files.dirList

SciServer.Files.splitPath(path, fileService)[source]¶

If the topVolume in the ‘path’ is a rootVolume, then this function splits the path of the form rootVolume/userVolumeOwner/userVolume/relativePath and returns the tuple (rootVolume, userVolumeOwner, userVolume, relativePath, True). On the other hand, if the topVolume in the ‘path’ is a dataVolume, then this function splits the path of the form dataVolume/relativePath and returns the tuple (dataVolume, None, None, relativePath, False).

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – file system path (string), starting from the root volume level or data volume level. Example: rootVolume/userVolumeOwner/userVolume/relativePath… or dataVolume/relativePath…

Returns:

a tuple containing the components: (topVolume, userVolumeOwner, userVolume, relativePath, isTopVolumeADataVolume)

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.splitPath(“Storage/myUserName/persistent/my/Relative/Path”, fileServices[0]);

See also

Files.getFileServices(), Files.getFileServiceFromName

SciServer.Files.upload(fileService, path, data='', localFilePath=None, quiet=True)[source]¶

Uploads data or a local file into a path defined in the file system.

Parameters:

fileService – name of fileService (string), or object (dictionary) that defines a file service. A list of these kind of objects available to the user is returned by the function Files.getFileServices().
path – path (in the remote file service) to the destination file (string), starting from the root volume level or data volume level. Examples: rootVolume/userVolumeOwner/userVolume/destinationFile.txt or dataVolume/destinationFile.txt
data – string containing data to be uploaded, in case localFilePath is not set.
localFilePath – path to a local file to be uploaded (string),
userVolumeOwner – name (string) of owner of the userVolume. Can be left undefined if requester is the owner of the user volume.
quiet – If set to False, it will throw an error if the file already exists. If set to True. it will not throw an error.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the FileService API returns an error.

Example:

fileServices = Files.getFileServices(); Files.upload(fileServices[0], “myRootVolume/myUserName/myUserVolume/myUploadedFile.txt”, None, localFilePath=”/myFile.txt”);

See also

Files.getFileServices(), Files.getFileServiceFromName, Files.createDir, Files.delete, Files.download, Files.dirList

SciServer.Jobs module¶

SciServer.Jobs.cancelJob(jobId)[source]¶

Cancels the execution of a job.

Parameters:: jobId – Id of the job (integer)
Raises:: Throws an exception if the HTTP request to the Authentication URL returns an error. Throws an exception if the HTTP request to the JOBM API returns an error.
Example:: job = Jobs.submitShellCommandJob(Jobs.getDockerComputeDomains()[0],’pwd’, ‘Python (astro)’); isCanceled = Jobs.cancelJob(job.get(‘id’));

See also

Jobs.submitNotebookJob, Jobs.getJobStatus, Jobs.getDockerComputeDomains.

SciServer.Jobs.getDockerComputeDomainFromName(dockerComputeDomainName, dockerComputeDomains=None)[source]¶

Returns a DockerComputeDomain object, given its registered name.

Parameters:

dockerComputeDomainName – name of the DockerComputeDomainName, as shown within the results of Jobs.getDockerComputeDomains()
dockerComputeDomains – a list of dockerComputeDomain objects (dictionaries), as returned by Jobs.getDockerComputeDomains(). If not set, then an internal call to Jobs.getDockerComputeDomains() is made.

Returns:

a DockerComputeDomain object (dictionary) that defines a Docker compute domain. A list of these kind of objects available to the user is returned by the function Jobs.getDockerComputeDomains().

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the JOBM API returns an error.

Example:

dockerComputeDomain = Jobs.getDockerComputeDomainFromName(‘dockerComputeDomainAtJHU’);

See also

Jobs.getDockerComputeDomains

SciServer.Jobs.getDockerComputeDomains()[source]¶

Gets a list of all registered Docker compute domains that the user has access to.

Returns:: a list of dictionaries, each one containing the definition of a Docker compute domain.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the JOBM API returns an error.
Example:: dockerComputeDomains = Jobs.getDockerComputeDomains();

See also

Jobs.submitShellCommandJob, Jobs.getJobStatus, Jobs.cancelJob

SciServer.Jobs.getDockerComputeDomainsNames(dockerComputeDomains=None)[source]¶

Returns the names of the docker compute domains available to the user.

Parameters:: dockerComputeDomains – a list of dockerComputeDomain objects (dictionaries), as returned by Jobs.getDockerComputeDomains(). If not set, then an extra internal call to Jobs.getDockerComputeDomains() is made.
Returns:: an array of strings, each being the name of a docker compute domain available to the user.
Raises:: Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the RACM API returns an error.
Example:: dockerComputeDomainsNames = Files.getDockerComputeDomainsNames();

See also

Files.getDockerComputeDomains

SciServer.Jobs.getJobDescription(jobId)[source]¶

Gets the definition of the job,

Parameters:: jobId – Id of job
Returns:: dictionary containing the description or definition of the job.
Raises:: Throws an exception if the HTTP request to the Authentication URL returns an error, and if the HTTP request to the JOBM API returns an error.
Example:: job1 = Jobs.submitShellCommandJob(Jobs.getDockerComputeDomains()[0],’pwd’, ‘Python (astro)’); job2 = Jobs.getJobDescription(job1.get(‘id’));

See also

Jobs.submitShellCommandJob, Jobs.getJobStatus, Jobs.getDockerComputeDomains, Jobs.cancelJob

SciServer.Jobs.getJobStatus(jobId)[source]¶

Gets a dictionary with the job status as an integer value, together with its semantic meaning. The integer value is a power of 2, that is, 1:PENDING, 2:QUEUED, 4:ACCEPTED, 8:STARTED, 16:FINISHED, 32:SUCCESS, 64:ERROR and 128:CANCELED

Parameters:: jobId – Id of job (integer).
Returns:: dictionary with the integer value of the job status, as well as its semantic meaning.
Raises:: Throws an exception if the HTTP request to the Authentication URL returns an error, and if the HTTP request to the JOBM API returns an error.
Example:: jobId = Jobs.submitShellCommandJob(Jobs.getDockerComputeDomains()[0],’pwd’, ‘Python (astro)’); status = Jobs.getJobStatus(jobId);

See also

Jobs.submitShellCommandJob, Jobs.getJobStatus, Jobs.getDockerComputeDomains, Jobs.cancelJob

SciServer.Jobs.getJobsList(top=10, open=None, start=None, end=None, type='all')[source]¶

Gets the list of Jobs submitted by the user.

Parameters:

top – top number of jobs (integer) returned. If top=None, then all jobs are returned.
open – If set to ‘True’, then only returns jobs that have not finished executing and wrapped up (status <= FINISHED). If set to ‘False’ then only returnes jobs that are still running. If set to ‘None’, then returns both finished and unfinished jobs.
start – The earliest date (inclusive) to search for jobs, in string format yyyy-MM-dd hh:mm:ss.SSS. If set to ‘None’, then there is no lower bound on date.
end – The latest date (inclusive) to search for jobs, in string format yyyy-MM-dd hh:mm:ss.SSS. If set to ‘None’, then there is no upper bound on date.
type – type (string) of jobs returned. Can take values of ‘rdb’ (for returning only relational database jobs), ‘docker’ (for returning only Docker jobs) and ‘all’ (all job types are returned).

Returns:

a list of dictionaries, each one containing the definition of a submitted job.

Raises:

Throws an exception if the HTTP request to the Authentication URL returns an error, and if the HTTP request to the JOBM API returns an error.

Example:

jobs = Jobs.getJobsList(top=2);

See also

Jobs.submitNotebookJob, Jobs.submitShellCommandJob, Jobs.getJobStatus, Jobs.getDockerComputeDomains, Jobs.cancelJob

SciServer.Jobs.submitNotebookJob(notebookPath, dockerComputeDomain=None, dockerImageName=None, userVolumes=None, dataVolumes=None, resultsFolderPath='', parameters='', jobAlias='')[source]¶

Submits a Jupyter Notebook for execution (as an asynchronous job) inside a Docker compute domain.

Parameters:

notebookPath – path of the notebook within the filesystem mounted in SciServer-Compute (string). Example: notebookPath = ‘/home/idies/worskpace/persistent/JupyterNotebook.ipynb’
dockerComputeDomain – object (dictionary) that defines a Docker compute domain. A list of these kind of objects available to the user is returned by the function Jobs.getDockerComputeDomains().
dockerImageName – name (string) of the Docker image for executing the notebook. E.g., dockerImageName=”Python (astro)”. An array of available Docker images is defined as the ‘images’ property in the dockerComputeDomain object.
userVolumes – a list with the names of user volumes (with optional write permissions) that will be mounted to the docker Image. E.g.: userVolumes = [{‘name’:’persistent’, ‘needsWriteAccess’:False},{‘name’:’scratch’, , ‘needsWriteAccess’:True}] . A list of available user volumes can be found as the ‘userVolumes’ property in the dockerComputeDomain object. If userVolumes=None, then all available user volumes are mounted, with ‘needsWriteAccess’ = True if the user has Write permissions on the volume.
dataVolumes – a list with the names of data volumes that will be mounted to the docker Image. E.g.: dataVolumes=[{“name”:”SDSS_DAS”}, {“name”:”Recount”}]. A list of available data volumes can be found as the ‘volumes’ property in the dockerComputeDomain object. If dataVolumes=None, then all available data volumes are mounted.
resultsFolderPath – full path to results folder (string) where the original notebook is copied to and executed. E.g.: /home/idies/workspace/rootVolume/username/userVolume/jobsFolder. If not set, then a default folder will be set automatically.
parameters – string containing parameters that the notebook might need during its execution. This string is written in the ‘parameters.txt’ file in the same directory level where the notebook is being executed.
jobAlias – alias (string) of job, defined by the user.

Returns:

the job ID (int)

Raises:

Throws an exception if the HTTP request to the Authentication URL returns an error. Throws an exception if the HTTP request to the JOBM API returns an error, or if the volumes defined by the user are not available in the Docker compute domain.

Example:

dockerComputeDomain = Jobs.getDockerComputeDomains()[0]; job = Jobs.submitNotebookJob(‘/home/idies/workspace/persistent/Notebook.ipynb’, dockerComputeDomain, ‘Python (astro)’, [{‘name’:’persistent’},{‘name’:’scratch’, ‘needsWriteAccess’:True}], [{‘name’:’SDSS_DAS’}], ‘param1=1nparam2=2nparam3=3’,’myNewJob’)

See also

Jobs.submitShellCommandJob, Jobs.getJobStatus, Jobs.getDockerComputeDomains, Jobs.cancelJob

SciServer.Jobs.submitShellCommandJob(shellCommand, dockerComputeDomain=None, dockerImageName=None, userVolumes=None, dataVolumes=None, resultsFolderPath='', jobAlias='')[source]¶

Submits a shell command for execution (as an asynchronous job) inside a Docker compute domain.

Parameters:

shellCommand – shell command (string) defined by the user.
dockerComputeDomain – object (dictionary) that defines a Docker compute domain. A list of these kind of objects available to the user is returned by the function Jobs.getDockerComputeDomains().
dockerImageName – name (string) of the Docker image for executing the notebook. E.g., dockerImageName=”Python (astro)”. An array of available Docker images is defined as the ‘images’ property in the dockerComputeDomain object.
userVolumes – a list with the names of user volumes (with optional write permissions) that will be mounted to the docker Image. E.g., userVolumes = [{‘name’:’persistent’, ‘needsWriteAccess’:False},{‘name’:’scratch’, , ‘needsWriteAccess’:True}] A list of available user volumes can be found as the ‘userVolumes’ property in the dockerComputeDomain object. If userVolumes=None, then all available user volumes are mounted, with ‘needsWriteAccess’ = True if the user has Write permissions on the volume.
dataVolumes – a list with the names of data volumes that will be mounted to the docker Image. E.g., dataVolumes=[{“name”:”SDSS_DAS”}, {“name”:”Recount”}]. A list of available data volumes can be found as the ‘volumes’ property in the dockerComputeDomain object. If dataVolumes=None, then all available data volumes are mounted.
resultsFolderPath – full path to results folder (string) where the shell command is executed. E.g.: /home/idies/workspace/rootVolume/username/userVolume/jobsFolder. If not set, then a default folder will be set automatically.
jobAlias – alias (string) of job, defined by the user.

Returns:

the job ID (int)

Raises:

Throws an exception if the HTTP request to the Authentication URL returns an error. Throws an exception if the HTTP request to the JOBM API returns an error, or if the volumes defined by the user are not available in the Docker compute domain.

Example:

dockerComputeDomain = Jobs.getDockerComputeDomains()[0]; job = Jobs.submitShellCommandJob(‘pwd’, dockerComputeDomain, ‘Python (astro)’, [{‘name’:’persistent’},{‘name’:’scratch’, ‘needsWriteAccess’:True}], [{‘name’:’SDSS_DAS’}], ‘myNewJob’)

See also

Jobs.submitNotebookJob, Jobs.getJobStatus, Jobs.getDockerComputeDomains, Jobs.cancelJob

SciServer.Jobs.waitForJob(jobId, verbose=False, pollTime=5)[source]¶

Queries regularly the job status and waits until the job is completed.

Parameters:

jobId – id of job (integer)
verbose – if True, will print “wait” messages on the screen while the job is still running. If False, will suppress the printing of messages on the screen.
pollTime – idle time interval (integer, in seconds) before querying again for the job status. Minimum value allowed is 5 seconds.

Returns:

After the job is finished, returns a dictionary object containing the job definition.

Raises:

Throws an exception if the user is not logged into SciServer (use Authentication.login for that purpose). Throws an exception if the HTTP request to the JOBM API returns an error.

Example:

dockerComputeDomain = Jobs.getDockerComputeDomains()[0]; job = Jobs.submitShellCommandJob(dockerComputeDomain,’pwd’, ‘Python (astro)’);Jobs.waitForJob(job.get(‘id’))

See also

Jobs.getJobStatus, Jobs.getDockerComputeDomains, Jobs.submitNotebookJob, Jobs.submitShellCommandJob

SciServer.Dask module¶

SciServer.Dask.getClient(ref_id=None)[source]¶

Creates a new client for the specified Dask cluster.

If ref_id is not set, cluster connection properties will be read from the ~/dask-cluster.json file injected into new SciServer Compute containers automatically when they are created with an attached Dask cluster.

Parameters:: ref_id – Dask cluster reference ID in SciServer Compute, defaults to None
Returns:: Dask client object
Return type:: dask.distributed.Client

SciServer package¶

SciServer.CasJobs module¶

SciServer.SciQuery module¶

SciServer.Config module¶

SciServer.Authentication module¶

SciServer.LoginPortal module¶

SciServer.SciDrive module¶

SciServer.SkyServer module¶

SciServer.SkyQuery module¶

SciServer.Files module¶

SciServer.Jobs module¶

SciServer.Dask module¶

Table of Contents

Previous topic

This Page