[obspy-users] FDSN downloader spamming new processes

Florian Fuchs florian.fuchs at univie.ac.at
Tue Feb 26 15:11:21 CET 2019


Dear ObsPy team,

I am facing a potential issue with the ObsPy FDSN client. I am using 
the EIDA routing client to download waveform data and metadata for many 
stations. The way I do it is a simple loop over all stations, using the 
get_waveforms() function in the loop. In principle it looks like this:

from obspy.clients.fdsn import RoutingClient as Client
client = Client('eida-routing')
for station in long_list :
	data = client.get_waveforms(...)
	data.write(filename)

The problem is, that each time data is actually downloaded, ObsPy seems 
to start several new processes in the background, which are not being 
closed until the end of the python script. Thus, for each station in 
the for-loop new additional processes are started but not closed again. 
Since I have a very long stationlist and many days that I loop over, 
basically I end up spawning more processes than the systen can handle, 
ultimately resulting in the message: "Can't start new thread"

This problem only occurs with the fdsn routing client, but e.g. not 
with the ArcLink client. I did not try the non-routed fdsn client. Same 
behaviour with ObsPy versions 1.1.0 and 1.1.1 ...

Any suggestions how to avoid this? Or did I just code it in a stupid 
way?

Thanks in advance and best regards,
Florian

_____________________________________

Dr. Florian Fuchs
Department of Meteorology and Geophysics
University of Vienna
UZA II, 2D508
Althanstraße 14
1090 Vienna
Austria
Phone: +43 1 4277 53726
Mobile: +43 664 60277 53726
www: http://imgw.univie.ac.at
_____________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.swapbytes.de/pipermail/obspy-users/attachments/20190226/ed043b7b/attachment.html>


More information about the obspy-users mailing list