1
0
Fork 0
mirror of synced 2024-08-05 13:21:35 +12:00
Commit graph

2036 commits

Author SHA1 Message Date
Nick Sweeting
2c76f29b78
better issue template 2019-01-22 19:16:09 -05:00
Nick Sweeting
74e925da4b update docs version 2019-01-20 14:08:49 -05:00
Nick Sweeting
f25be8bc24 add chrome user agent example in config 2019-01-20 14:08:33 -05:00
Nick Sweeting
071b39b944 nicer error formats 2019-01-20 14:08:33 -05:00
Nick Sweeting
de6eb649e0 disable wget timestamping if using warcs 2019-01-20 14:08:33 -05:00
Nick Sweeting
ce07a6c738 patch subprocess.run to have better timeout handling 2019-01-20 14:08:33 -05:00
Nick Sweeting
e620722bb9 fix timeouts hanging the progressbar 2019-01-20 14:08:33 -05:00
Nick Sweeting
7ea36c4adb bump timeouts and improve curl archive method 2019-01-20 14:08:33 -05:00
Nick Sweeting
370adb56bf
Update README.md 2019-01-18 17:45:48 -05:00
Nick Sweeting
ea905d68bb fix etc nginx and Docker config 2019-01-16 09:20:15 -05:00
Nick Sweeting
d24724ed79
Update Dockerfile 2019-01-16 08:05:01 -05:00
Nick Sweeting
0cb915cd24
Update README.md 2019-01-15 18:35:11 -05:00
Nick Sweeting
cc8611de83
Update config.py 2019-01-14 22:46:35 -05:00
Nick Sweeting
c42fcd42d7 fetch warc file inline with wget instead of as separate step 2019-01-14 22:43:14 -05:00
Nick Sweeting
2d86d4dfe0 update docs 2019-01-14 22:43:14 -05:00
Nick Sweeting
3186f24867
Update README.md 2019-01-14 19:12:08 -05:00
Nick Sweeting
45c499dc9e
Update README.md 2019-01-14 18:19:30 -05:00
Nick Sweeting
300b5c6182 put ArchiveBox and wget version in user agent 2019-01-14 18:17:30 -05:00
Nick Sweeting
39f5e12364 allow passing import list via stdin 2019-01-14 18:11:48 -05:00
Nick Sweeting
cb60bad1d7 disable WARC by default 2019-01-12 20:19:17 -05:00
Nick Sweeting
321cdeda11 add mirror to git clone 2019-01-12 20:13:13 -05:00
Nick Sweeting
c5dcb68a6d install youtubedl in setup 2019-01-12 03:14:07 -05:00
Nick Sweeting
6189b1a8af fix invalid old config vars 2019-01-12 03:13:55 -05:00
Nick Sweeting
b650c663a0 re-enable checking SSL by default 2019-01-11 22:48:09 -05:00
Nick Sweeting
f83750c545 cleanup options and make cli flags better for chrome headless timeouts 2019-01-11 22:38:50 -05:00
Nick Sweeting
911b84a79b make wget cli options easier to read 2019-01-11 22:13:51 -05:00
Nick Sweeting
b77476cb83 fix user agent quoting 2019-01-11 19:54:43 -05:00
Nick Sweeting
33942d37cb
Update README.md 2019-01-11 07:57:46 -05:00
Nick Sweeting
4609b11d15
Update README.md 2019-01-11 07:56:24 -05:00
Nick Sweeting
5ea8a1ad9c
Update README.md 2019-01-11 07:21:30 -05:00
Nick Sweeting
e8808b0a1f add WARC downloading 2019-01-11 07:02:49 -05:00
Nick Sweeting
329b06ec5a make output dir in script 2019-01-11 06:53:34 -05:00
Nick Sweeting
74189966bf add link to media 2019-01-11 06:50:42 -05:00
Nick Sweeting
a15a331798 fix media download with longer timeout 2019-01-11 06:33:35 -05:00
Nick Sweeting
c33f7ba91c add ability to fetch media 2019-01-11 05:52:29 -05:00
Nick Sweeting
0df098717a make git domains configurable 2019-01-11 05:27:25 -05:00
Nick Sweeting
d35c6cf8b5 fix fetch page title default 2019-01-11 05:19:22 -05:00
Nick Sweeting
67d103a293 dont float round timestamps 2019-01-11 05:19:07 -05:00
Nick Sweeting
827e15b31a add git downloading 2019-01-11 05:18:49 -05:00
Nick Sweeting
5b6c768a47 autodetect common chrome binary locations 2019-01-11 04:34:16 -05:00
Nick Sweeting
8a13ca449d
Update README.md 2019-01-11 04:12:34 -05:00
Nick Sweeting
cf9d1875c7 add plain text link parsing 2019-01-11 04:09:39 -05:00
Nick Sweeting
7a9487fad9
Update README.md 2019-01-10 14:06:34 -05:00
Nick Sweeting
3c236a044b
Update README.md 2019-01-10 14:03:41 -05:00
Nick Sweeting
c0f46b2728 Set theme jekyll-theme-merlot 2019-01-10 14:00:41 -05:00
Nick Sweeting
b43a239a2e
Update README.md 2019-01-10 13:59:37 -05:00
Nick Sweeting
45e7bd5927
Update README.md 2019-01-10 13:51:14 -05:00
Nick Sweeting
0e72addcda remove requests dependency 2019-01-09 04:05:34 -05:00
Nick Sweeting
84d4118e3c Set theme jekyll-theme-minimal 2019-01-09 03:59:42 -05:00
Nick Sweeting
8cf1b04fed
install requests with pip 2019-01-05 15:26:49 -05:00