[obspy-users] obspy-sds-report

Tobias Megies megies at geophysik.uni-muenchen.de
Thu Jun 18 10:52:08 CEST 2020


There is certainly room for improvement, discussion about enhancements
we should take to github, I would propose.

The way I use it is do a "full" report only every now and then
(depending on how fast the filesystem is, how big the SDS archive etc),
e.g. once per day, so that new stations get eventually picked up.
Otherwise do "update" much more frequently, e.g. every 5 mins, depending
on how up-to-date it has to be (e.g. when reenabling a station in the
field and checking if data come back in).
If you do it like that (e.g. with cronjobs) you need to make sure that a
new full run output (which is much longer running) is not overwritten
again by an update run that was based on an old output file (I hope it's
clear what I mean). If I remember correctly, I kill all running "update"
processes immediately after completing a "full" run in my cronjob.

best,
Tobias



On 18/06/2020 10:26, F. Massin wrote:
> Hello Tobias and Mickael,
> 
> Could this be improved at least by specifying year(s), location and
> channel codes while making things like glob.glob on the SDS root directory ?
> 
> Cheers
> 
> Fred 
> 
>> Le 18 juin 2020 à 09:57, Tobias Megies
>> <megies at geophysik.uni-muenchen.de
>> <mailto:megies at geophysik.uni-muenchen.de>> a écrit :
>>
>> Hello Mickael,
>>
>> if my previous response wasn't sufficiently clear: in the "full"
>> operation mode (without using "--update"), the SDS filesystem might be
>> traversed fully (at least the folder structure) to identify what NSLC
>> (channel SEED Ids) are in the SDS overall and then it will check each
>> channel of the whole SDS for files according to the "backwards days"
>> option (and show them as grey if no data in that span).
>>
>> Subsequent "update" runs, will not search for NSLC again, but rather
>> look at the saved output txt file and then check the NSLC listed there.
>>
>> So if you want to only scan specific NSLC / SEED IDs, set up such an
>> output file fram scratch or modify the one you already got and then ever
>> only use the "update" option on that initial output file.
>>
>> No idea what is is about gaps not being checked, they should be checked
>> and reported in both the "full" and "update" mode i believe.
>>
>> best,
>> Tobias
>>
>>
>> On 17/06/2020 16:04, Mickaël Langlais wrote:
>>> Hello,
>>> Thanks for your answers.
>>> I already tested the update option, but it only works for latencies nor
>>> for the gaps...
>>> And when I run it without this option the script scans all my SDS
>>> (containing data since 2013)...the backward option does not seems to be
>>> considered in the scan, only in the results.
>>>
>>> The version i am using is obspy-sds-report 1.0.3.
>>>
>>> Maybe I should upgrade to a newer version ?
>>>
>>> Regards
>>>
>>> *Mickaël LANGLAIS*
>>> Responsable Service Instrumentations Géophysiques
>>> Institut des Sciences de la Terre
>>> Université Grenoble Alpes - ISTerre - CS40700 - 38058 Grenoble Cedex 9
>>> ✆ +33 4 76 63 51 13 
>>> ✆[MOB] +33 6 14 83 16 90 
>>>  mickael.langlais at univ-grenoble-alpes.fr
>>> <mailto:mickael.langlais at univ-grenoble-alpes.fr>
>>> <mailto:mickael.langlais at univ-grenoble.alpes.fr>
>>>  Site Internet : ISTerre <https://isterre.fr/>
>>>
>>> On 16/06/2020 10:11, F. Massin wrote:
>>>> Hello,
>>>>
>>>> I think this script relies en SDS client get_latency method (alias of
>>>> _get_current_endtime) ?
>>>>
>>>> It should not scan the whole SDS. It should read miniseed headers of
>>>> the most recent files, going backward from now to 15 days in the past,
>>>> and stop there anyway:
>>>> https://github.com/obspy/obspy/blob/88687a146a7c3ca8f35c608db2243cf5fed6813c/obspy/clients/filesystem/sds.py#L395
>>>>
>>>> Cheers
>>>>
>>>> Fred
>>>>
>>>>> Le 15 juin 2020 à 14:46, Mickaël Langlais
>>>>> <mickael.langlais at univ-grenoble-alpes.fr
>>>>> <mailto:mickael.langlais at univ-grenoble-alpes.fr>> a écrit :
>>>>>
>>>>> Hello,
>>>>>
>>>>> I am using obspy-sds-report script to generate html report of my SDS
>>>>> archive.
>>>>> But I would like to limit the depth of the scan to the last month or
>>>>> current year to avoid very long process and "no data within xx days"
>>>>> in the HTML figure for stations that are not any more used/available.
>>>>>
>>>>> The options that I am using are :  /--check-backwards-days 15
>>>>> --check-quality-days 15/, but the scan seems to check all the SDS
>>>>> archive...
>>>>>
>>>>> Is there a way to force the scan to a limited time span ?
>>>>>
>>>>> Regards
>>>>> Mickael.
>>>>> -- 
>>>>> *Mickaël LANGLAIS*
>>>>> Responsable Service Instrumentations Géophysiques
>>>>> Institut des Sciences de la Terre
>>>>> Université Grenoble Alpes - ISTerre - CS40700 - 38058 Grenoble Cedex 9
>>>>> ✆ +33 4 76 63 51 13 
>>>>> ✆[MOB] +33 6 14 83 16 90 
>>>>>  mickael.langlais at univ-grenoble-alpes.fr
>>>>> <mailto:mickael.langlais at univ-grenoble-alpes.fr>
>>>>> <mailto:mickael.langlais at univ-grenoble.alpes.fr>
>>>>>  Site Internet : ISTerre <https://isterre.fr/>
>>>>>
>>>>>



More information about the obspy-users mailing list