Tuesday, November 10, 2015

Creating A Pypi Mirror

One of the things I have read a lot of people like to do, is to create a local pypi mirror.  That way, when the real pypi is on the fritz or you don't have internet access, you can still install modules and work on your pet projects.

I worked through a bunch of different modules for creating a pypi mirror, and most of them seemed to make a mirror of the most recent versions of modules.  Which would be fine, except I am anal and wanted as complete of a mirror as I could get. 

So, after playing around with a number of different modules I discovered Bandersnatch.  At first it looked promising (claiming that the mirror would be about 120 Gb).  Considering that the module documentation was probably written (and not necessarily updated) a couple of years ago, I could only imagine what that number is now. 

I followed the installation from the above link and installed the module (in a virtual environment) and got it running and let it run until completion (which was about 3-4 hours later).  I did a df of the directory and BLAM!, a little over 180Gb of moudules.  Just WOW!!  Now that is the mirror I was looking for. 

I still want to play with some of the other methods (as having a mirror of the most recent set of modules is also handy), but this is definitely what I was looking for.  Hopefully this information allows others to create their own Pypi mirror as well.  I would say quickly, but that will depend on the internet connection you are using.

No comments:

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.