XD blog

blog page

download, internet, python, requests


2014-06-17 Download data after redirection

I was looking for a way to download a file from http://sourceforge.net/ with the following code:

url = "something on source forge"
req = urllib.request.Request(url)
u = urllib.request.urlopen(req)
bin = u.read()
u.close()
with open("something.zip","wb") as f : f.write(bin)

It failed due to redirections. Doing it myself or trying to look for some module doing the job. Every query I tried on a search engine about getting a file from SourceForge in Python gave me links to Python projects on hosted on SourceForge. Not exactly what I was looking for. So...

I tried some search queries including redirection and after a couple of them I found the following module requests which made it a lot easier:

req = requests.get(url, allow_redirects = True, stream=True)
bin = req.raw.read()
with open("something.zip","wb") as f : f.write(bin)

I guess the module urllib3 could also do the trick but I did not try it.


<-- -->

Xavier Dupré