Online Data Resources available for UBC faculty & Students

Online Data storage Resources

As research methods grow ever more capable, they also have a tendency to produce larger and larger data sets in short periods of time.  Storing, Backing up, and sharing this data among lab members who all need access it can become a considerable challenge.  Additionally, provincal law requires that data be stored on secure servers located within Canada, meaning that some convenient services (e.g. Dropbox) are not actually approved for research data generated at UBC.  What other services are available to help with this challenge?  Here we review many services that may help with storage and other computational issues at UBC:

ONLINE DATA HOSTING FOR PUBLISHED DATA:

The UBC Dataverse is a Data repositiory run on campus at UBC.  This service is optimized for storing data from specific studies and should meet the requirements of journals that require data to be hosted online for a specific publication.  Data stored on the Dataverse can be assigned a particular DOI number, and is stored in a manner consistent wtih BC data policy. 

http://dvn.library.ubc.ca/dvn/

High capacity storage and High Performance Computing through Compute Canada:

Compute Canada Rapid Access Service (RAS)

Any faculty member at a Canadian university can register an account at the Computer Canada Database, and once registered can sponsor accounts for students, staff, and collaborators.

    https://ccdb.computecanada.ca/security/login

 

These accounts allow users access to Rapid Access Service (RAS), limited online storage and “fair share” access to HPC resources.  

RAS allows users access to  2 TB of storage per person, up to 10 TB per lab.  This storage is online accessible and well backed up, located on Canadian servers.

Compute Canada RAS also allow opportunistic use of High Performance Computing (HPC) resources.

Default accounts come with default priority, which means you can submit jobs to the scheduler, which will run them on HPC cores when they are available and return the results to you when the operations are completed.  

These nodes are high powered and have huge memory resources.  Submitting a job to the nodes in Compute Canada is akin to submitting a process on a linux computer, so most anything that can be run through the command line can be run on these resources.  

Details on RAS, what it provides, and how to access it are available here:

     https://www.computecanada.ca/research-portal/accessing-resources/rapid-access-service/

Getting Started with Compute Canada:

For BC, Compute Canada servers and resources are accessed through WestGrid.  WestGrid has excellent training resourses for getting started and using these online resources for storage and High Performance Computing.  For an overview with links to specific instructions, see here.

For more info on Westgrid, see here:  https://www.westgrid.ca

More Resources:  Resource Allocation Competitions (RAC)

If your lab needs more resources than can be provided by the RAS system, you can apply for more storage and higher priority access to computing resources through the Resource Allocation Competition (RAC).  Essentially RAC is a grant request for specific resources from Compute Canada. Allocation of resources is based on the score of your proposal and available resources for allocation. In general RAC applications are due in early October, and are divided into Resources for Research Groups (RRGs) and Research Platforms and Portals (RPPs).  RRGs are for dedicated allocations of processing power for HPC and/or dedicated online storage. RPPs are more for cloud computing services, mostly involving hosting data and research portals such as websites or online databases. More information about Compute Canada RACs can be found here:

https://www.computecanada.ca/research-portal/accessing-resources/resource-allocation-competitions/

 

The major advantage of the Compute Canada Resources is that they are free, and if you have a long term need for a large amount of storage and HPC they may be available through a RRG RAC.

UBC's Advanced Research Computing (ARC) team works closely with Compute Canada and can advise and advocate for UBC researchers working on this platform.  They are available to help individual labs utilize these resources.  More info about ARC is available here:  https://arc.ubc.ca

Or contact ARC at arc.support@ubc.ca

ARC is also available to provide advice on many of these systems as well as other high performance computing needs for your lab if you have questions about what might be useful in your particular case.  Advice and consultation from ARC is free of charge!


Alternatives to COmpute Canada

UBC-IT resources:

If your needs aren’t as great, or if you need resources very quickly, some on Campus UBC resources may be the answer:

Teamshare:

Teamshare services for UBC are very quick to implement and allow for convenient and rapid sharing of relatively small datasets for a reasonable cost.  Teamshare data is also well backed up and archived. The downside of TeamShare is that it is not intended for large data sharing, but as an on-campus only UBC owned model equivalent to Dropbox, it is a quite effective system.

https://it.ubc.ca/services/web-servers-storage/teamshare-storage-service


Educloud

Equivalent to Amazon Cloud storage, but much cheaper and can be used for content website administration, hosting content, etc.  Prices vary but are ~ $150/TB per year for standard performance storage and ~$400 /TB per year for high performance storage. This storage is backed up offsite daily and allows multiple daily backups.  But educloud also offers full cloud services should you require more extensive features to host web access to share a database or other cloud computing needs. Educloud is similar to Amazon Web Services, but compliant with BC data storage and privacy regulations and usually a lower cost solution.

    https://it.ubc.ca/services/web-servers-storage/educloud-server-service

Dropbox-like Resources:

WestGrid OwnCloud:

 Westgrid, a regional partner of Compute Canada, offers the OwnCloud service as a dropbox like alternative for WestGrid Users.  Data submitted to OwnCloud are backed up daily, and 50 GB are offered free for users. In most cases, data can be stored in a desktop file synced across multiple computers like Dropbox.  For more information, see here:

    https://www.westgrid.ca/resources_services/data_storage/cloud_storage  

 

Sync:

    Sync is a private company that offers Dropbox like services with servers that are located in Canada and a high degree of data security for stored data.  They provide 6 GB of data for a free trial account, and up to 2TB for individual accounts billed at $8 per month. More information about Sync is available here:

    https://www.sync.com/secure-dropbox-alternative/