User Tools

Site Tools


en:data-uploader

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Next revisionBoth sides next revision
en:data-uploader [2019/02/28 07:49] – created fluktuaciaen:data-uploader [2019/05/01 06:01] fluktuacia
Line 1: Line 1:
 FIXME **This page is not fully translated, yet. Please help completing the translation.**\\ //(remove this paragraph once the translation is finished)// FIXME **This page is not fully translated, yet. Please help completing the translation.**\\ //(remove this paragraph once the translation is finished)//
 +??
  
-====== Upload system for measured data from stations ======+====== Upload system for data measured by stations ======
  
 Measured data upload from stations uses python software [[https://github.com/bolidozor/data-uploader|data-uploader]]. It can be used to upload data of Bolidozor and [[http://wiki.ionozor.eu|Ionozor]] projects.  Measured data upload from stations uses python software [[https://github.com/bolidozor/data-uploader|data-uploader]]. It can be used to upload data of Bolidozor and [[http://wiki.ionozor.eu|Ionozor]] projects. 
Line 39: Line 40:
 ==== Configuration ==== ==== Configuration ====
  
-In order to data-uploader function properly, you have to add the following parameters in its configuration file:+In order for data-uploader to function properly, you have to add the following parameters into its configuration file:
  
   "project": "bolidozor",   "project": "bolidozor",
Line 50: Line 51:
 ==== Limiting the data flow ==== ==== Limiting the data flow ====
  
-In some cases, usually observatories with slow data connection, it is suitable to limit the upload speed from station. You can use trickle programme to limit the bandwidth of data-uploader.+In some cases, usually observatories with slow data connection, it is advantageous to limit the upload speed from station. You can use trickle programme to limit the bandwidth of data-uploader.
  
   trickle -u 200 ./dataUpload.py ~/bolidozor/station/Bolidozor.json   trickle -u 200 ./dataUpload.py ~/bolidozor/station/Bolidozor.json
  
-To check the outcome, you can use for example bmon or iftop programmes.+To check the outcome, you can usefor examplebmon or iftop software.
  
   sudo apt-get install bmon   sudo apt-get install bmon
  
-==== Actualisation of data-uploader from RMDS-data-uploaderu====+==== Actualisation of data-uploader from RMDS-data ====
      
   cd ~/repos/   cd ~/repos/
Line 70: Line 71:
   sudo apt install python-paramiko   sudo apt install python-paramiko
  
-Add the following parameters to configuration file before ‘’”frontend": "tcp_raw",'':+Add the following parameters to configuration file in front of ‘’”frontend": "tcp_raw",'':
  
   "project": "bolidozor",   "project": "bolidozor",
Line 92: Line 93:
   ],   ],
  
-==== Sorting the data to folders ====+==== Sorting data to folders ====
  
 The sorting takes place based on the name of the original file. The resulting structure looks as follows: The sorting takes place based on the name of the original file. The resulting structure looks as follows:
  
-  /station/typ dat/rok/měsíc/den/[hodina]/soubor.záznamu+  /station/data_type/year/month/day/[hour]/file.record
  
 The structure acts in accordance with [[en:convention|Bolidozor network standards]]. The structure acts in accordance with [[en:convention|Bolidozor network standards]].
Line 108: Line 109:
  
 The script checks the internet connection (availability of space.astro.cz) every 2 minutes and in case of authentication it begins the data synchronisation. The synchronisation takes place in four steps.  The script checks the internet connection (availability of space.astro.cz) every 2 minutes and in case of authentication it begins the data synchronisation. The synchronisation takes place in four steps. 
-Skript každé 2 minuty zkontroluje připojeni k internetu (dostupnost space.astro.cz) a v případě autentizace se spustí synchronizace dat. Synchronizace probíhá postupně ve čtyřech krocích: 
  
   - Synchronising the spectrum snapshots    - Synchronising the spectrum snapshots 
en/data-uploader.txt · Last modified: 2019/07/13 12:38 by fluktuacia