Using Souceforge.net to Host large files
A lot of times my GitHub repositories will depend on some external large binary files. These files can range anywhere from trained deep learning model parameters, or resources for a demo/test. Because GitHub will by default track files, and not easily accessible by a URL, making it not really ideal for hosting large files.
- 1. Create a new project in Sourceforge.net
- 2. Use
scpto upload to your project
- 3. Download using static url
Hence what I do is to use Sourceforge.net to host external files, and keep GitHub for just development purposes. Sourceforge also provides a download statistics counter and analyzer, which is a lot more convenient than the GitHub API for releases since the last time I checked.
There are a few simple steps to use
scp to upload files to Sourceforge.net.
1. Create a new project in Sourceforge.net
Remember to check “Unlimited File Downloads and Stats”, which should be checked by default.
Note: One of the very few things I don’t like about Sourceforge is that all project names are unique IDs. Hence if there exists a registered project that matches a your project name, you would need to re-name your project:(
Suppose everything is set up; let’s say, your user name is “user”, and your project’s name is “foo_proj”; the target folder we want to upload to is “foobar”
scp to upload to your project
scp data.h5 email@example.com:/home/frs/p/foo_proj/foobar/
Upon finished, the
data.h5 file should be accessible online.
Note: Remember to check the
md5sumof the online copy to see if that matches your local file!
3. Download using static url
There will a few mirrors for hosting your file. The general file download link pattern is:
Now you can use this static link in your GitHub code with ease. Bravo!