Dima Gerasimov
03c1b0009c
Fix 'Too many open files' error.
...
Happened before after continuous archiving of few hundreds links.
Fix by:
* setting process object to `None` to trigger GC finalizer cleanup of pipe descriptors
* protecting against double cleanup
2018-11-26 02:29:56 +00:00
Nick Sweeting
062d2ddc98
fix archive_url broken on first run
2018-06-17 18:32:52 -04:00
Nick Sweeting
aa5a674a17
add new migrate_data step to move old folder
2018-06-10 23:01:56 -04:00
Nick Sweeting
755845c69a
use latest instead of deriving wget path
2018-06-10 22:43:01 -04:00
Nick Sweeting
c90f4bfd5b
cleanup ARCHIVE_DIR paths
2018-06-10 21:26:11 -04:00
Nick Sweeting
a287900345
fix template locations
2018-06-10 21:12:55 -04:00
Nick Sweeting
d0f2e693b3
re-arrange and cleanup directory structure
2018-06-10 20:52:15 -04:00