Storage

Everybody needs data and produces results. But where should all these different data go to?

Limits

System	Path	Quota	Feature	Note
AURORA	`/srvfs/home`	200 GB / 1000k files	daily Backup	other than staff get 100 GB/ 100k files
JET	`/jetfs/home`	100 GB / 500k files	daily Backup
JET	`/jetfs/scratch`	no	no Backup

Where can data be?

Data Type	Location	Note
source code	HOME	use git repo for source control
personal info	HOME	nobody but you should have access. perm: `drwx------.`
model output	SCRATCH	small and large files do not need backup
important results	HOME	within your quota limits
input data	SCRATCH	if this is only your input data, otherwise
input data	SHARED	`/jetfs/shared-data` or `/srvfs/shared`
important data	DATA	`/srvfs/data` is backed up, daily.
collaboration data	WEBDATA	`/srvfs/webdata`, accessible via webdata.wolke

Remember: All data needs to be evaluated after some time and removed.

Long term storage

The ZID of the University of Vienna offers an archive system, where data can be stored for at least 3 years. If you have data that needs to be stored for some time, but not easily accessible, you can request the data to be sent to the archive:

Request data to be archived
# Only the admin can issue the transfer, but you can create the request 
# and add some documentation.
# You can add a notification if the data can be deleted after the 3 years 
# or should be downloaded again.
userservices archive -h

# Create an archive request

Publishing data

There are various data hubs, that can store your data following the FAIR principles and based on your data management plan.

External Hubs: - Zenodo (up to 50-200 GB/100 files) -

The University of Vienna offers not yet a comparable service that can host large data sets on longer time scales.

The department of Meteorology and Geophysics has established a collaboration, Cloud4Geo, to allow such long term storage of research data and share it with the scientific community. The data is made available via the Earth Observation Data Centre EODC.

Last update: December 6, 2024
Created: December 6, 2024