Wiki source code of Data storage

Last modified by mmolin@helsinki_fi on 2024/01/24 07:07
Show last authors
1 These pages describe [[Light Microscopy Unit>>url:http://www.biocenter.helsinki.fi/bi/lmu/||shape="rect"]] data storage systems and policies.
2
3
4
5 {{children/}}
6
7 **
8 **
9
10 = GENERAL USAGE POLICY =
11
12 * Data must be stored in folders named by your university username (and grouped alphabetically), e.g. L:\lmu_active1\users\h\hajaalin. See [[doc:LMU.Light Microscopy Unit.Data storage.Organizing data on LMU network drives.WebHome]].
13 * You may store microscopy data on lmu_active disk for two years. After that the data is moved into another space for a half year.
14 * Data from users without active project is also removed to separate space for a half year.
15 * We will move older data to L:\lmu-archive. Data transfer will be done semi automatically.
16 * Transfer is based on **modification time**. Please note that this may [[break your datasets>>url:https://wiki.helsinki.fi/display/LMU/Notes+about+automated+archival||shape="rect"]].
17 * The user is responsible for real archival of the data.
18
19 = ACCESS =
20
21 **NOTE: If you have problems connecting to LMU network disks, please contact [[lmu-staff@helsinki.fi.>>mailto:lmu-staff@helsinki.fi.||rel="nofollow" style="text-decoration: none;" shape="rect" class="external-link"]] **Helpdesk can help you with general connection issues, but we should first check that you have been granted access to the disks.
22
23 If you log in from a centrally managed workstation at the University of Helsinki the disks are mapped for you and you'll find them in windows:
24
25 * Dept (L:) \lmu_active1
26 * Dept (L:) \lmu_active2
27 * Dept (L:) \lmu_archive
28
29 These disk spaces should be visible in all centrally managed Windows 10 computers in the university network but for some reasons this is not always working. In that case you will have to tell to the system where the disk is:
30
31 **Windows 10:**
32
33 * Open File Explorer
34 * Choose This PC
35 * Open Computer ribbon
36 ** Map network drive
37
38 [[image:attach:ThisPC-ComputerRibbon.png]]
39
40 * Choose a free Drive letter (for example Y:)
41 * To mount Active1, at Folder box, write exactly: ~\~\dept1.ad.helsinki.fi\lmu_active1
42 * Or to mount Active2, write exactly: ~\~\dept1.ad.helsinki.fi\lmu_active2
43 * Click "Connect using different Credentials"
44
45 You should put your credentials in the form "atkk\username" 
46
47 This instruction shows connection to group disk (P:) which is quite similar as connection to (L:) disks except the address.
48 [[https:~~/~~/helpdesk.it.helsinki.fi/en/instructions/computer-and-printing/connecting-group-disk-space-home-computer>>url:https://helpdesk.it.helsinki.fi/en/instructions/computer-and-printing/connecting-group-disk-space-home-computer||rel="nofollow" style="text-decoration: none;" shape="rect" class="external-link"]]\\
49
50 **If you are connecting from home computer, you have to use VPN connection. Look more information from IT Helpdesk: [[https:~~/~~/helpdesk.it.helsinki.fi/en/logging-and-connections/networks/connections-outside-university>>url:https://helpdesk.it.helsinki.fi/en/logging-and-connections/networks/connections-outside-university||rel="nofollow" style="text-decoration: none;" shape="rect"]]**
51
52 Most instrument PCs have lmu_active1.bat and lmu_active2.bat scripts on desktops. Notice that the drive letter V: and U: respectively have to be free if you use them. The mapping instruction above should always work.
53
54 \\
55
56 In principle the mapping in Mac and Linux should go by smb. In practice there might be some problems but usually it works.
57
58 **Mac:**
59
60 * Finder → Go → Connect to server...
61 * Server smb:~/~/dept1.ad.helsinki.fi/lmu_active1
62 * connect
63 * when asked, enter personal user account & password at University of Helsinki
64
65 **Linux:**
66
67 (% class="auto-cursor-target" %)
68 \\
69
70 {{code language="bash"}}
71 # create mount point
72 mkdir -p /mnt/lmu_active1
73
74 # mount network drive
75 sudo mount -vvv -t cifs -o user=`whoami`,uid=`id -u`,file_mode=0600,dir_mode=0700,vers=3,iocharset=utf8 //dept1.ad.helsinki.fi/lmu_active1 /mnt/lmu_active1
76
77
78 {{/code}}
79
80 **Linux on** [[GE Deltavision Ultra>>url:https://wiki.helsinki.fi/display/LMU/Linux+command+line+%28Konsole%29+tricks+on+GE||style="text-decoration: none;" shape="rect"]] **PC:**
81
82 (% class="auto-cursor-target" %)
83 \\
84
85 {{code language="bash"}}
86 lmu_active1.sh
87 {{/code}}
88
89 = Overview of the LMU disk spaces in the University network =
90
91 == (% style="color: rgb(77,77,77);" %)lmu_active disks(%%) ==
92
93 lmu_active disk spaces are meant for **primary research data**, some intermediate archives and so on. Policy for light sheet data is not yet ready and so far that is worked case by case as needed. If you will have a LightSheet project that will create huge amount of data (several terabytes), discuss the data management and storage with LMU preferably beforehand.
94
95 * lmu_active1 70 TB, primary research data
96 * lmu_active2 70 TB, primary research data
97 * lmu_archive 70 TB plus online compression, file space that is used to archive older inactive data
98 * Not backed up, has limited ability to restore previous file versions, (% style="color: rgb(68,68,68);" %)(see (%%)[[https:~~/~~/helpdesk.it.helsinki.fi/en/instructions/other-instructions/previous-versions-files-and-folders>>url:https://helpdesk.it.helsinki.fi/en/instructions/other-instructions/previous-versions-files-and-folders||rel="nofollow" style="text-decoration: none;" shape="rect"]](% style="color: rgb(68,68,68);" %)).
99 * 10 GB network connection
100
101 \\
102
103 **
104 **
105
106 **
107 **
108
109 \\