Nick Sweeting
e4823cc5d7
add back CUSTOM_TEMPLATES_DIR config option
2021-04-06 00:57:20 -04:00
Nick Sweeting
5a9f27204a
dont use chrome when its not available on windows systems
2021-04-05 23:33:08 -04:00
Nick Sweeting
bbebc33b7a
better autodetection of node modules dependencies
2021-04-05 21:14:06 -04:00
Nick Sweeting
ef76f7c490
require running setup in existing data dir
2021-04-05 19:59:33 -04:00
Nick Sweeting
3449bac5bc
add new setup command to autoinstall dependencies
2021-04-05 19:51:57 -04:00
Nick Sweeting
0e39a2098d
split migrations and data dir check
2021-04-01 14:00:07 -04:00
Nick Sweeting
7e7c134ec0
use exc
2021-03-27 05:32:07 -04:00
Nick Sweeting
bd6d9c165b
enforce utf8 on literally all file operations because windows sucks
2021-03-27 01:16:29 -04:00
Nick Sweeting
185d2f9f9b
check for JSON1 extesnion in Sqlite on startup
2021-03-27 00:27:58 -04:00
Nick Sweeting
62089fdb22
close sqlite3 connections if unused
2021-02-28 22:53:34 -05:00
Nick Sweeting
e3b69c450c
move db checks under check_db if statement
2021-02-18 02:37:33 -05:00
Nick Sweeting
c877d67e0c
check if pragma is already wal before setting
2021-02-18 02:31:42 -05:00
Nick Sweeting
7b7aa239fd
autocreate db cache table when needed and reenable WAL mode
2021-02-17 18:26:04 -05:00
Nick Sweeting
6f0eec92eb
fix lint errors
2021-02-16 16:26:48 -05:00
Nick Sweeting
8cde6bc676
add gist.github.com to git domains
2021-02-16 07:36:09 -05:00
Nick Sweeting
46a7b972ee
disable WAL and cache table for now
2021-02-16 06:13:50 -05:00
Nick Sweeting
fb8e6cabcb
fix error log location
2021-02-16 04:15:09 -05:00
Nick Sweeting
19f7c907e0
exec archivebox from docker entrypoint script to avoid nesting pid under bash
2021-02-16 01:22:36 -05:00
Nick Sweeting
10d687c55c
add new MEDIA_MAX_SIZE=750m size option for limiting youtubedl downloads
2021-02-16 01:21:19 -05:00
Nick Sweeting
0c9db1c554
remove symbols from random secret key for easier copy pastin
2021-02-15 20:45:42 -05:00
Nick Sweeting
78463c243a
remove unused GIT_SHA config option
2021-02-15 20:42:33 -05:00
Nick Sweeting
9cd4ba38f0
add new SNAPSHOTS_PER_PAGE pagination limit config
2021-02-15 20:42:00 -05:00
Nick Sweeting
e61e12c889
use setup.py to determine dependencies in Dockerfile instead of egg-info requires.txt
2021-02-15 14:51:32 -05:00
Nick Sweeting
683a08772b
change wording of db not found error
2021-02-08 23:27:46 -05:00
Nick Sweeting
7d0f5653c3
fix lgtm alerts
2021-02-01 02:27:24 -05:00
Nick Sweeting
d072f1d413
hide ssl warnings when checking SSL is disabled
2021-01-30 20:44:49 -05:00
Nick Sweeting
9d24bfd0dc
disable progress bars on mac again
2021-01-30 20:44:49 -05:00
Nick Sweeting
ed13ec7655
remove active theme
2021-01-30 05:47:55 -05:00
Nick Sweeting
3227f54b52
limit youtubedl download size to 750m and stop splitting out audio files
2021-01-29 00:15:32 -05:00
Dan Arnfield
5c7842ffb3
Fix dependency dict entries
2021-01-20 09:24:34 -06:00
Nick Sweeting
9784dcb816
better config comments and docstrings
2020-12-20 03:11:19 +02:00
jdcaballerov
50df108863
Update archivebox/config.py
...
Co-authored-by: Nick Sweeting <git@sweeting.me>
2020-12-12 08:34:00 -05:00
jdcaballerov
24d4c44624
Add ripgrep configs
2020-12-12 07:36:31 -05:00
Nick Sweeting
335732649b
tweak node dependency version detection order
2020-12-11 21:03:17 +02:00
Nick Sweeting
2db5e51b54
fix windows shutil not able to handle pathlib
2020-12-11 19:33:18 +02:00
Nick Sweeting
6623497f18
fix MERCURY_PATH in version output when missing
2020-12-11 16:21:33 +02:00
Nick Sweeting
c084e70ea8
fix TEMPLATES_DIR location
2020-12-11 16:21:09 +02:00
Cristian
9745a5ac56
fix: Migrations should be silent when running in setup_django
2020-12-08 18:42:25 -05:00
Cristian
9aac09a5e1
feat: Patch setup_django so we can use an inmemory db in specific commands
2020-12-08 18:42:25 -05:00
JDC
95382b3812
Add ripgrep rg search backend and set as default
2020-12-06 01:13:39 +02:00
JDC
47daa038eb
Implement flush for search backend after remove command
2020-12-06 01:12:47 +02:00
JDC
c2c01af3ad
Add config for search backend
2020-12-06 01:12:47 +02:00
mAAdhaTTah
ac7ad9e942
Add parser for Pocket API
...
Pass a url like `pocket://Username` to import that username's archived Pocket
library. Tokens need to be stored in ArchveBox.conf with the following keys:
```
POCKET_CONSUMER_KEY = key-from-custom-pocket-app
POCKET_ACCESS_TOKENS = {"YourUsername": "pocket-token-for-app"}
```
`POCKET_ACCESS_TOKENS` MUST be on a single line, or the JSON will be
misinterpreted by the parser as a new key/value pair.
2020-12-04 22:54:39 -05:00
Nick Sweeting
7f2c834ea3
fix check_data_folder mypy types
2020-11-28 01:05:35 -05:00
Nick Sweeting
5e7c2d0ab8
show archivebox and node versions in version cmd output
2020-11-23 20:24:44 -05:00
Nick Sweeting
0e2ccbc10d
update urls to new repo path
2020-11-23 02:06:46 -05:00
Nick Sweeting
3f160eab8e
correctly handle WGET_AUTO_COMPRESSION failing when wget is missing
2020-11-12 14:28:43 -05:00
Nick Sweeting
22fb9c2ad7
tweak icons
2020-10-31 19:32:43 -04:00
Nick Sweeting
b8bbb75f9c
logarithmic progress bars woohoo
2020-10-31 07:57:11 -04:00
Nick Sweeting
ac9e0e356d
config fixes
2020-10-31 07:57:11 -04:00
Nick Sweeting
18355dc2c6
clean up config loading in settings and config file layout
2020-10-31 03:08:03 -04:00
Nick Sweeting
27708152d2
wip initial django setup
2019-04-02 16:36:41 -04:00
Nick Sweeting
f4e018ba0c
fix a bunch of mypy errors
2019-03-30 20:49:45 -04:00
Nick Sweeting
8283b353f4
fix wget_auto_compression check prining stderr to console
2019-03-30 17:43:44 -04:00
Nick Sweeting
5d0185b6dd
Merge branch 'master' into dev
2019-03-30 15:36:54 -04:00
Nick Sweeting
35c05c321f
minor bin version checking changes
2019-03-30 15:03:31 -04:00
Nick Sweeting
066b36b6a9
make URL_BLACKLIST empty by default
2019-03-30 14:56:19 -04:00
Nick Sweeting
a26c2fe467
show full version info using flag
2019-03-27 16:44:00 -04:00
Nick Sweeting
93216a3c3e
new version handling and absolute imports
2019-03-27 15:35:13 -04:00
Nick Sweeting
b9839500b2
make archivebox use current directory as OUTPUT_DIR by default
2019-03-27 15:15:51 -04:00
mlazana
a3705e31c6
Merge remote-tracking branch 'upstream/master'
2019-03-27 20:03:07 +02:00
Nick Sweeting
7ae680a461
Merge branch 'master' into dev
2019-03-27 10:36:29 -04:00
Nick Sweeting
6bec1709f6
Update config.py
2019-03-27 04:48:38 -04:00
Nick Sweeting
c9c5b04df0
full type-hinting coverage
2019-03-26 23:25:07 -04:00
Starbeamrainbowlabs
51859731d3
config.py: add WGET_AUTO_COMPRESSION
2019-03-26 23:34:45 +00:00
Nick Sweeting
25a107df43
switch to dataclasses, working Link type hints everywhere
2019-03-26 19:21:34 -04:00
Nick Sweeting
52871f42e7
cleaner config with type hints
2019-03-26 05:31:27 -04:00
Nick Sweeting
4f8c99011a
fix terminal resizing making progress bar go crazy
2019-03-26 05:30:23 -04:00
mlazana
4d10568477
exclude links that are in blacklist
2019-03-24 14:40:26 +02:00
mlazana
417ee9e302
add env variable URL_BLACKLIST
2019-03-23 21:27:41 +02:00
Nick Sweeting
f7a0568a6c
better chrome options loading
2019-03-22 23:00:53 -04:00
Nick Sweeting
4c499d77b6
move dependency checking into config file
2019-03-22 22:05:45 -04:00
Nick Sweeting
69f837bbf6
simplify chrome_user_data_dir default
2019-03-22 21:37:02 -04:00
Nick Sweeting
8f73fdbe09
fix chrome profile precedence order to be equal
2019-03-22 21:31:55 -04:00
noncetonic
28758cf16c
Adds CHROME_USER_AGENT
2019-03-19 10:15:52 -07:00
Nick Sweeting
1c1bc76ac1
add chrome headless option and improve default data dir finding
2019-03-12 17:50:10 -04:00
Nick Sweeting
8319ccf064
add docs link to config.py
2019-03-12 12:48:46 -04:00
Nick Sweeting
32c39d0fd0
cleaner output dir spec in config
2019-03-08 17:51:49 -05:00
Nick Sweeting
2e10f57f6e
fix relative links from index files
2019-03-08 17:46:14 -05:00
Nick Sweeting
a74d8410f4
also check for macOS binary defaults
2019-03-08 16:25:42 -05:00
Nick Sweeting
d689264365
add new config and dependency options
2019-02-21 15:47:15 -05:00
Nick Sweeting
d52c9c5304
allow passing COOKIES_FILE to wget
2019-02-21 12:58:51 -05:00
Nick Sweeting
5a7d00a639
fetch page title during archiving process
2019-02-19 01:44:54 -05:00
Nick Sweeting
9eb79258bb
check chrome version on startup if using chrome
2019-01-29 17:08:15 -08:00
Nick Sweeting
e60070dbb2
add youtubedl to help str
2019-01-25 17:38:47 -08:00
Nick Sweeting
20de451515
add linux config example
2019-01-23 01:48:04 -05:00
Nick Sweeting
95301c9306
better default config
2019-01-23 01:42:55 -05:00
Nick Sweeting
e1be96e597
working docker-compose with google chrome
2019-01-23 01:08:23 -05:00
Nick Sweeting
f25be8bc24
add chrome user agent example in config
2019-01-20 14:08:33 -05:00
Nick Sweeting
cc8611de83
Update config.py
2019-01-14 22:46:35 -05:00
Nick Sweeting
c42fcd42d7
fetch warc file inline with wget instead of as separate step
2019-01-14 22:43:14 -05:00
Nick Sweeting
300b5c6182
put ArchiveBox and wget version in user agent
2019-01-14 18:17:30 -05:00
Nick Sweeting
cb60bad1d7
disable WARC by default
2019-01-12 20:19:17 -05:00
Nick Sweeting
b650c663a0
re-enable checking SSL by default
2019-01-11 22:48:09 -05:00
Nick Sweeting
f83750c545
cleanup options and make cli flags better for chrome headless timeouts
2019-01-11 22:38:50 -05:00
Nick Sweeting
e8808b0a1f
add WARC downloading
2019-01-11 07:02:49 -05:00
Nick Sweeting
a15a331798
fix media download with longer timeout
2019-01-11 06:33:35 -05:00
Nick Sweeting
c33f7ba91c
add ability to fetch media
2019-01-11 05:52:29 -05:00
Nick Sweeting
0df098717a
make git domains configurable
2019-01-11 05:27:25 -05:00
Nick Sweeting
827e15b31a
add git downloading
2019-01-11 05:18:49 -05:00