IDM gives me the best speed by maxing out my connection, no other application, FTP clients, Download managers alike give me the same speed. But I noticed that with HTTP access to files, I can only download a single file at a time using IDM or any download manager. IDM's site grabber has been a hit or miss for me. So, I made a simple python script which add all files in a folder and it's subfolders to IDM Download queue. I've seen many posts on reddit and other communities on people searching for ways to do this so I'm sharing this here hoping that it might be of any help to some of those. How do I use it?
I'm assuming you already have IDM installed.
The first step would be to add your IDM application to system's path so that it can be accessed by commandline quickly. Here's how to do it:
Follow these steps or google "How to add an application to System's path on windows" https://pastebin.com/jG8RHT3Q
Now You have these options : A pre-compiled executable(.exe) file or the python script:
(I have not looked at the html structure of other seedboxes' Http file access links so I can't tell if they're at all similar)
If you're using Ultraseedbox
and you don't want to bother about the extra details and the code, this is the recommended option. You can just Plug and play. Run the .exe file and you will be good to go.
Your username and password are stored in plain text if you are to use this method so, use it at your own risk. I've done this for convenience sake.
You can find the latest .exe file under the releases tab https://github.com/kruzchy/ftp-folder-idm/releases
- download ftp-idm.exe.
- Run the .exe file
- It will say "Settings.txt file has been generated! Please change settings and run again to proceed." on the first run. Close the .exe.
- Open the settings.txt which has been generated in the same directory as the .exe
- Edit the text with your username and password. The default path for download is your desktop, paste a new path if you want to change it.https://i.imgur.com/vEyK2WQ.png
- Run the .exe again, now it asks for the folder link. Paste the http link to the folder you want to download and press enter. The script should now find the links for the directories and add it to the IDM Queue.https://i.imgur.com/ahev9Jc.png
- Open IDM and go to scheduler, tweak the settings in the schedule tab and files in the queue tab to your liking. Apply the settings and click on start now to process the queue.
This is the source for the executable if you want to make an executable yourself https://github.com/kruzchy/ftp-folder-idm/blob/masteexecutable_ftp-idm.py
A quick google search on how to create an exe file from a python script will yield you many results. I used pyinstaller to create the executable. For other Seedboxes:
If you know some python and prefer to tailor the code to your needs, here's the source. It's a simple scraping job. If you aren't into coding python and think some functionality like this would be helpful to you, PM me so I can see if it's possible to slightly modify the code to fit your usage https://github.com/kruzchy/ftp-folder-idm/blob/masteftp-idm.py
I used the keyring module to store credentials which uses Windows Credential Locker as backend.
Basically, I took advantage of the HTTP access to the files and scraped the html for file links recursively while preserving the directory structure. Then I passed in those links combined with idm commandline options to add those files to the download queue.
I'm aware that it was not a really smart script, I just hacked my way into making it work real quick. I'm posting it here because I've once scoured the entire internet for a way to be able to do this few years ago and I'm hoping that maybe this might help someone in the near future.
PM me if you need any help regarding this.
Edit: Cleaned up the post. Sorry for the lazy writeup.
Edit2: Updated the code. Credentials will no longer be stored in plain text in settings.txt. Keyring module(which uses windows credentials locker as backend) is used to store credentials.