IMECC dataflow 2009-2011

Last modified by pkolari@helsinki_fi on 2024/02/07 06:32

Timo.hyytiala.helsinki.fi (windows server) copies all the relevant data to hippunen.atm.helsinki.fi (windows server) every hour. The command file is d:\timedjobs\rsh.dat. At the moment 2010-10 the files are tdr100, mastrad, mast, sun, rain, pws, eddy233, soil, snow, canpar, (impactor). The data is copied to hippunen directories under d:\hyde

Hippunen runs a batch file c:\lab\hydeplot\plotmatd.bat every hour, five minutes past exact hour. The batch file starts matlab with m-file plothyde.m. This program does a lot of plotting, but also run program mast.m. Mast.m script runs atmosphere_data.m script (hosted by Petri K.) in directory c:\lab\hydeplot\atmosphere. Atmosphere_data script is calculating half hour averages from mast, sun (radiation with rain) and present weather  sensor and places the to directory c:\lab\hydeplot\atmosphere\data. The mast script calls also programs canparave, soilave, tdrave, fluxave and radmave, which are saving the half hour results for the variable outside the scope of atmosphere_data. Radmave is calculating the NDVI and PRI. The results are saved to the same directory as atmosphere_data is saving.

After the calculation of the half hour files impdatabase.bat is called, matlab c:\lab\hydeplot\impdatabase.m. This is calling at first script atmoinport, which is reading all the half hour files except flux. It is also calculating absolute humidity from dew point sensor and co2 storage. Script is deleting the data of the current day from the database table meta and inserting new data. Impdatabase is calling function flxinport, which is deleting the data of current day an inseting new data to table eddy233. Finally impdatabase is calling script imecc_export, which is downloading data from tables meta and edd233 and saving it to imecc daily datafiles C:\lab\hydeplot\imecc.

Also the phenologia camera pictures are submitted to imecc. Phenologia camera is taking pictures now (2010-10) every second hour and sending them to daily directories http://www.atm.helsinki.fi/~satrail/hydekamera/2010-10-28/. Batch file C:\lab\hydek\digicp.bat is copying the eight picture of the day to the imecc data directory C:\lab\hydeplot\imecc.

The data and and the picture files are synced to gaia.agraria.unitus.it/imeccreal/fi-hyy from C:\lab\hydeplot\imecc every day with C:\lab\imeccftp.bat and winscp script C:\lab\imeccftp.script.

With the data a text file 00readme2.txt is copied to gaia. This file describes the content of the data files.

You can access hippunen with remote desktop. You can use user mittaaja or ask Pasi A. to make you an own account. The directory c:\lab is also shared to windows network \\hippunen.atm.helsinki.fi\lab.

IMECC is getting obsolete by the end of 2011 as the project has ended. Data flow into the database continues under ICOS . The ICOS near-real-time demo dataflow is similar to IMECC dataflow, the scripts are located in hippunen (c:\lab\icos_demo).