Nick Sweeting
|
b3a89172ab
|
add original src url to dosctring
|
2021-04-06 01:38:24 -04:00 |
|
|
Nick Sweeting
|
e7c7a8f71c
|
force kill forked child processes after each extractor command
|
2021-04-06 01:35:21 -04:00 |
|
|
Nick Sweeting
|
af6561b527
|
remove CDN jquery and bootstrap by inlining them instead
|
2021-04-06 01:13:31 -04:00 |
|
|
Nick Sweeting
|
e4823cc5d7
|
add back CUSTOM_TEMPLATES_DIR config option
|
2021-04-06 00:57:20 -04:00 |
|
|
Nick Sweeting
|
4b098bea15
|
fix navbar showing user info even when not logged in
|
2021-04-06 00:44:01 -04:00 |
|
|
Nick Sweeting
|
68a506e1a7
|
fix Snapshot admin actions buttons submitting on Safari
|
2021-04-06 00:43:49 -04:00 |
|
|
Nick Sweeting
|
5a9f27204a
|
dont use chrome when its not available on windows systems
|
2021-04-05 23:33:08 -04:00 |
|
|
Nick Sweeting
|
1485e4eb90
|
use older docker compose version for more features
|
2021-04-05 23:22:05 -04:00 |
|
|
Nick Sweeting
|
cadac4838e
|
fix searching by url contents
|
2021-04-05 23:21:07 -04:00 |
|
|
Nick Sweeting
|
2fc9f1cfa3
|
dont depend on README in Dockerfile install step
|
2021-04-05 23:17:07 -04:00 |
|
|
Nick Sweeting
|
8b4f84959a
|
new setup flag on init command to autosetup on init
|
2021-04-05 21:15:32 -04:00 |
|
|
Nick Sweeting
|
5c181532b5
|
only install dependencies in setup if they are missing
|
2021-04-05 21:15:10 -04:00 |
|
|
Nick Sweeting
|
21820acecd
|
import User model into core models for convenience
|
2021-04-05 21:14:22 -04:00 |
|
|
Nick Sweeting
|
bbebc33b7a
|
better autodetection of node modules dependencies
|
2021-04-05 21:14:06 -04:00 |
|
|
Nick Sweeting
|
211bf86f4a
|
fix bug checking value in subcommand_args when None
|
2021-04-05 21:13:49 -04:00 |
|
|
Nick Sweeting
|
79fb60a47a
|
dont allow caching when installing packages
|
2021-04-05 20:08:48 -04:00 |
|
|
Nick Sweeting
|
05c17af4dc
|
fix lint errors
|
2021-04-05 20:00:16 -04:00 |
|
|
Nick Sweeting
|
ef76f7c490
|
require running setup in existing data dir
|
2021-04-05 19:59:33 -04:00 |
|
|
Nick Sweeting
|
8c1d12020f
|
fix lint errors
|
2021-04-05 19:53:42 -04:00 |
|
|
Nick Sweeting
|
3449bac5bc
|
add new setup command to autoinstall dependencies
|
2021-04-05 19:51:57 -04:00 |
|
|
Nick Sweeting
|
24f7dd9c05
|
dont raise systemexit on succesful config
|
2021-04-05 19:51:40 -04:00 |
|
|
Nick Sweeting
|
ffd119c66c
|
fix linter
|
2021-04-03 15:51:27 -04:00 |
|
|
Nick Sweeting
|
7162649b03
|
fix search fields on admin snapshots list
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
5492c204b8
|
tweak add page copy
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
f59b6d4189
|
only add url-list lines that are real urls
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
d73f7d7d96
|
add db_index on url field
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
c438e49871
|
fix clicking on tags on snapshot list
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
0e39a2098d
|
split migrations and data dir check
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
5d7ad9b1b4
|
add form parser option
|
2021-04-01 02:34:16 -04:00 |
|
|
Nick Sweeting
|
59d5423483
|
fix snapshot icon caching and ordering
|
2021-04-01 02:22:15 -04:00 |
|
|
Nick Sweeting
|
fc5d99b4b4
|
fix missing package.json repo
|
2021-04-01 01:40:02 -04:00 |
|
|
Nick Sweeting
|
ff9445648b
|
fix navbar and tags display
|
2021-04-01 01:39:34 -04:00 |
|
|
Nick Sweeting
|
7ee7a94e67
|
fix str split using old overwrite pipe instead of append pipe
|
2021-03-31 11:30:15 -04:00 |
|
|
Nick Sweeting
|
005da31f8f
|
add overwrite flag option to schedule cmd
|
2021-03-31 11:29:51 -04:00 |
|
|
Nick Sweeting
|
534ae60ab0
|
dont overwrite the log on each run, and use dedicated log for scheduled archiving
|
2021-03-31 11:20:42 -04:00 |
|
|
Nick Sweeting
|
5d3a03b299
|
use stderr and hint in case of parser returning no urls instead of bare exception
|
2021-03-31 01:39:01 -04:00 |
|
|
Nick Sweeting
|
8ce93ff787
|
use KEY, NAME, and PARSER to define parsers instead of hardcoding in init
|
2021-03-31 01:05:49 -04:00 |
|
|
Nick Sweeting
|
36f0646501
|
Merge pull request #669 from FliegendeWurst/fix-issue-235
add command: --parser option (fixes #235)
|
2021-03-31 00:53:47 -04:00 |
|
|
Nick Sweeting
|
2656e59215
|
change list style
|
2021-03-31 00:47:42 -04:00 |
|
|
Nick Sweeting
|
bd8a01e677
|
sigh vendor yet another dependency
|
2021-03-30 14:49:42 -04:00 |
|
FliegendeWurst
|
60bd9a902e
|
add command: --parser option
|
2021-03-28 10:09:11 +02:00 |
|
|
Nick Sweeting
|
a58ad5b272
|
allow larger tags
|
2021-03-27 05:52:42 -04:00 |
|
|
Nick Sweeting
|
7e7c134ec0
|
use exc
|
2021-03-27 05:32:07 -04:00 |
|
|
Nick Sweeting
|
5568b3b612
|
fix adding and removing tags via admin snapshot list
|
2021-03-27 05:27:02 -04:00 |
|
|
Nick Sweeting
|
49b77b91bb
|
fix unused var
|
2021-03-27 05:12:54 -04:00 |
|
|
Nick Sweeting
|
3e26ae4a66
|
support finding multiple urls as substrings in text
|
2021-03-27 04:30:40 -04:00 |
|
|
Nick Sweeting
|
f3a3d76439
|
support adding urls with tags directly via CLI and add page
|
2021-03-27 04:30:15 -04:00 |
|
|
Nick Sweeting
|
5fb9ca389f
|
check more url parsing invariants on startup
|
2021-03-27 03:57:22 -04:00 |
|
|
Nick Sweeting
|
fea0b89dbe
|
add tag cli option
|
2021-03-27 03:57:05 -04:00 |
|
|
Nick Sweeting
|
d486f8c162
|
use tempfile when logs dir is not available
|
2021-03-27 02:17:12 -04:00 |
|
|
Nick Sweeting
|
3c2f4c95cf
|
fix unused import
|
2021-03-27 01:50:26 -04:00 |
|
|
Nick Sweeting
|
56c6b11b14
|
add comment
|
2021-03-27 01:19:32 -04:00 |
|
|
Nick Sweeting
|
bd6d9c165b
|
enforce utf8 on literally all file operations because windows sucks
|
2021-03-27 01:16:29 -04:00 |
|
|
Nick Sweeting
|
185d2f9f9b
|
check for JSON1 extesnion in Sqlite on startup
|
2021-03-27 00:27:58 -04:00 |
|
|
Nick Sweeting
|
177d22da50
|
fix clicking add tag when no tag is selected throwing 500
|
2021-03-01 01:39:06 -05:00 |
|
|
Nick Sweeting
|
76ff789164
|
fix tags page rendering when snapshot has no updated timestamp
|
2021-03-01 00:40:41 -05:00 |
|
|
Nick Sweeting
|
0036e9cce2
|
add migration
|
2021-02-28 22:55:12 -05:00 |
|
|
Nick Sweeting
|
9fc9f52cfb
|
increase sqlite timeout and dont check for same thread condition
|
2021-02-28 22:55:00 -05:00 |
|
|
Nick Sweeting
|
1cabde3ccd
|
remove atomic transactions
|
2021-02-28 22:54:40 -05:00 |
|
|
Nick Sweeting
|
844b5c5e20
|
minor db fix
|
2021-02-28 22:54:04 -05:00 |
|
|
Nick Sweeting
|
62089fdb22
|
close sqlite3 connections if unused
|
2021-02-28 22:53:34 -05:00 |
|
|
Nick Sweeting
|
6aef1dd98d
|
dont check db if init is pending
|
2021-02-28 22:53:23 -05:00 |
|
|
Nick Sweeting
|
acbce25201
|
missing migrations
|
2021-02-18 08:05:05 -05:00 |
|
|
Nick Sweeting
|
9ead76f9f8
|
cross link snapshot index, admin, and actions pages using uuids
|
2021-02-18 08:04:50 -05:00 |
|
|
Nick Sweeting
|
c257ae240e
|
only enable debug toolbar if nothreading is on
|
2021-02-18 08:04:26 -05:00 |
|
|
Nick Sweeting
|
ca2bb673ea
|
improve inline admin forms and add tags autocomplete
|
2021-02-18 08:04:14 -05:00 |
|
|
Nick Sweeting
|
33df9c1ebe
|
fix after and before in remove tests
|
2021-02-18 06:21:44 -05:00 |
|
|
Nick Sweeting
|
9c07fbdc0b
|
unused import fix
|
2021-02-18 05:56:00 -05:00 |
|
|
Nick Sweeting
|
106f6adc59
|
fix lgtm alerts
|
2021-02-18 04:29:24 -05:00 |
|
|
Nick Sweeting
|
46a4197514
|
fix tests
|
2021-02-18 04:26:56 -05:00 |
|
|
Nick Sweeting
|
13c2b7c8eb
|
fix url used by gridview
|
2021-02-18 02:38:30 -05:00 |
|
|
Nick Sweeting
|
e3b69c450c
|
move db checks under check_db if statement
|
2021-02-18 02:37:33 -05:00 |
|
|
Nick Sweeting
|
74a9dd8880
|
add missing migrations
|
2021-02-18 02:36:21 -05:00 |
|
|
Nick Sweeting
|
7282f4a42a
|
add favicon and robots to internal static dir
|
2021-02-18 02:35:51 -05:00 |
|
|
Nick Sweeting
|
dcc812b9f5
|
make output link easier to see on archiveresult admin list
|
2021-02-18 02:35:33 -05:00 |
|
|
Nick Sweeting
|
df47486d31
|
make admin link easier to find on snapshot detail page
|
2021-02-18 02:35:11 -05:00 |
|
|
Nick Sweeting
|
8369ca0b87
|
fix snapshot detail page titles garbled due to double escaping
|
2021-02-18 02:34:56 -05:00 |
|
|
Nick Sweeting
|
ba2afa0672
|
allow cancelling init partway through and show hints to resume
|
2021-02-18 02:34:42 -05:00 |
|
|
Nick Sweeting
|
75e1bfd0a9
|
create_or_update ArchiveResults from history instead of get_or_create
|
2021-02-18 02:34:20 -05:00 |
|
|
Nick Sweeting
|
69e0302c2a
|
only add https to url if needed when prompting to save new snapshot
|
2021-02-18 02:34:03 -05:00 |
|
|
Nick Sweeting
|
9918bddb7e
|
remove emptystring tags before saving
|
2021-02-18 02:33:42 -05:00 |
|
|
Nick Sweeting
|
f1823381d0
|
improve latest title logic to take longest title
|
2021-02-18 02:33:28 -05:00 |
|
|
Nick Sweeting
|
ad4bd49163
|
increase max title length to 512
|
2021-02-18 02:33:08 -05:00 |
|
|
Nick Sweeting
|
4e5671dda9
|
add ability to add and remove tags directly from snapshot list
|
2021-02-18 02:32:58 -05:00 |
|
|
Nick Sweeting
|
c877d67e0c
|
check if pragma is already wal before setting
|
2021-02-18 02:31:42 -05:00 |
|
|
Nick Sweeting
|
7b7aa239fd
|
autocreate db cache table when needed and reenable WAL mode
|
2021-02-17 18:26:04 -05:00 |
|
|
Nick Sweeting
|
d60d73754d
|
fix favicon.ico and robots.txt served by runserver
|
2021-02-17 18:25:47 -05:00 |
|
|
Nick Sweeting
|
a8a6752b06
|
add CACHE_BACKEND options to settings for easier dev
|
2021-02-17 18:25:23 -05:00 |
|
|
Nick Sweeting
|
3457773988
|
fix ArchiveResult extractor showing up on top of dropdown in admin inline form
|
2021-02-17 18:25:01 -05:00 |
|
|
Nick Sweeting
|
084cf7ff51
|
add more explanation about snapshot.save timestamp bump
|
2021-02-17 13:34:46 -05:00 |
|
|
Nick Sweeting
|
265bcc0264
|
fix lint errors2
|
2021-02-16 16:29:41 -05:00 |
|
|
Nick Sweeting
|
6f0eec92eb
|
fix lint errors
|
2021-02-16 16:26:48 -05:00 |
|
|
Nick Sweeting
|
d89034dcde
|
disable debug_toolbar by default
|
2021-02-16 16:23:09 -05:00 |
|
|
Nick Sweeting
|
71cf8d5224
|
add migrations
|
2021-02-16 15:57:13 -05:00 |
|
|
Nick Sweeting
|
118bddb41a
|
fix admin title and url truncation and add edit/admin link and ID to Snapshot detail page
|
2021-02-16 15:55:07 -05:00 |
|
|
Nick Sweeting
|
05e891632c
|
add snapshot_id to Link and uuid to ArchiveResult
|
2021-02-16 15:54:27 -05:00 |
|
|
Nick Sweeting
|
0b8bdd8cc5
|
show dir size after each snapshot archiving process in cli
|
2021-02-16 15:53:52 -05:00 |
|
|
Nick Sweeting
|
acb932ba12
|
improve readability and mercury error handling and fix output path to be relative
|
2021-02-16 15:53:11 -05:00 |
|
|
Nick Sweeting
|
c95698e608
|
bump Snapshot.updated time after each extractor, change extractor order
|
2021-02-16 15:52:18 -05:00 |
|
|
Nick Sweeting
|
28e50c5e49
|
tweak snapshot asset serving logic to show multiple choices in case of conflict
|
2021-02-16 15:51:56 -05:00 |
|
|
Nick Sweeting
|
a6c64f2560
|
only get archive_size if index.html is present within dir
|
2021-02-16 15:51:17 -05:00 |
|
|
Nick Sweeting
|
c24e4bf11d
|
tweak django debug toolbar panels and add djdt_flamegraph
|
2021-02-16 15:51:02 -05:00 |
|
|
Nick Sweeting
|
1e2188517b
|
speed up latest_title by preferring db title if present
|
2021-02-16 15:49:53 -05:00 |
|
|
Nick Sweeting
|
8b236b9367
|
cache dir size, snapshot icons, tags str, and title in django cache
|
2021-02-16 15:49:29 -05:00 |
|
|
Nick Sweeting
|
51440ede3a
|
show extra fields on ArchiveResult admin list and detail page
|
2021-02-16 15:47:49 -05:00 |
|
|
Nick Sweeting
|
74c882ef16
|
add nothreading arg to archivebox server for perf profiling
|
2021-02-16 15:46:58 -05:00 |
|
|
Nick Sweeting
|
8cde6bc676
|
add gist.github.com to git domains
|
2021-02-16 07:36:09 -05:00 |
|
|
Nick Sweeting
|
bdf1b102be
|
load ArchiveResults from orphaned links history during init
|
2021-02-16 06:20:05 -05:00 |
|
|
Nick Sweeting
|
d0f8a5e710
|
change mercury atomic_write output order
|
2021-02-16 06:19:16 -05:00 |
|
|
Nick Sweeting
|
e62ecffeb0
|
disable django caches explicitly
|
2021-02-16 06:18:39 -05:00 |
|
|
Nick Sweeting
|
988a10a9f6
|
fix warc path in snapshot_icons
|
2021-02-16 06:18:05 -05:00 |
|
|
Nick Sweeting
|
48eddadc9b
|
inline archive_size and archive_path snapshot methods
|
2021-02-16 06:16:02 -05:00 |
|
|
Nick Sweeting
|
46a7b972ee
|
disable WAL and cache table for now
|
2021-02-16 06:13:50 -05:00 |
|
|
Nick Sweeting
|
24ae01cdbe
|
add ArchiveResult admin
|
2021-02-16 06:13:27 -05:00 |
|
|
Nick Sweeting
|
87069dd41e
|
fix wsgi.py
|
2021-02-16 04:16:51 -05:00 |
|
|
Nick Sweeting
|
8c2d81c041
|
always serve static files
|
2021-02-16 04:16:24 -05:00 |
|
|
Nick Sweeting
|
fb8e6cabcb
|
fix error log location
|
2021-02-16 04:15:09 -05:00 |
|
|
Nick Sweeting
|
26fa63749d
|
add debug toolbar
|
2021-02-16 02:50:05 -05:00 |
|
|
Nick Sweeting
|
fad2620c62
|
minor tweaks
|
2021-02-16 02:49:31 -05:00 |
|
|
Nick Sweeting
|
ae43d35e65
|
fix color option in docker
|
2021-02-16 01:26:26 -05:00 |
|
|
Nick Sweeting
|
4eac9d0ec1
|
show extra system setup info in version output
|
2021-02-16 01:23:58 -05:00 |
|
|
Nick Sweeting
|
82de67db34
|
fix missing/outdated template variables
|
2021-02-16 01:23:31 -05:00 |
|
|
Nick Sweeting
|
22da885148
|
log every archivebox command run to the errors.log
|
2021-02-16 01:23:01 -05:00 |
|
|
Nick Sweeting
|
19f7c907e0
|
exec archivebox from docker entrypoint script to avoid nesting pid under bash
|
2021-02-16 01:22:36 -05:00 |
|
|
Nick Sweeting
|
10d687c55c
|
add new MEDIA_MAX_SIZE=750m size option for limiting youtubedl downloads
|
2021-02-16 01:21:19 -05:00 |
|
|
Nick Sweeting
|
49939f3eaa
|
only accept stdin if args are not passed, fix stdin hang in docker
|
2021-02-16 01:20:47 -05:00 |
|
|
Nick Sweeting
|
de1a939df4
|
fix public index pagination links and total count
|
2021-02-15 21:07:12 -05:00 |
|
|
Nick Sweeting
|
108cb100b3
|
hotlink direclty to specific archivebox release from footer info
|
2021-02-15 21:06:56 -05:00 |
|
|
Nick Sweeting
|
56ec19fb29
|
allow hotlinking directly to a specific extractor output using hash on Snapshot detail page
|
2021-02-15 20:55:22 -05:00 |
|
|
Nick Sweeting
|
a089136fea
|
fix missing stdout template var on /add when not rendering success page
|
2021-02-15 20:54:47 -05:00 |
|
|
Nick Sweeting
|
b6d7c74680
|
speed up the Snapshot handling view and show index page when extractor output is missing or multiple snapshots returned
|
2021-02-15 20:52:08 -05:00 |
|
|
Nick Sweeting
|
0375853683
|
log error tracebacks to logs/errors.log file and filter noisy 404s and 200s from log output
|
2021-02-15 20:51:23 -05:00 |
|
|
Nick Sweeting
|
0ec9bfb971
|
fix dead missing template variables
|
2021-02-15 20:50:12 -05:00 |
|
|
Nick Sweeting
|
b3a50a2c10
|
fix server quick-init param not being passed properly to subcommand
|
2021-02-15 20:49:40 -05:00 |
|
|
Nick Sweeting
|
b06e256ad9
|
fix add command not updating snapshot detail index pages when passed index-only and overwrite flags together
|
2021-02-15 20:49:23 -05:00 |
|
|
Nick Sweeting
|
8e98cef7ad
|
fix after and before args flipped when filtering
|
2021-02-15 20:48:51 -05:00 |
|
|
Nick Sweeting
|
33d180afe7
|
allow filtering snapshots by timestamp in list, update, and remove cmds
|
2021-02-15 20:48:35 -05:00 |
|
|
Nick Sweeting
|
0c9db1c554
|
remove symbols from random secret key for easier copy pastin
|
2021-02-15 20:45:42 -05:00 |
|
|
Nick Sweeting
|
4faef03ba3
|
compute snapshot properties directly without loading whole Link
|
2021-02-15 20:44:08 -05:00 |
|
|
Nick Sweeting
|
9ce3bd5bdc
|
use index.LINK_FILTERS to validate filter-type args instead of hardocding them twice
|
2021-02-15 20:43:36 -05:00 |
|
|
Nick Sweeting
|
c28ad8bd1b
|
fix AddLinkForm widget complaining about missing template var class
|
2021-02-15 20:42:59 -05:00 |
|
|
Nick Sweeting
|
78463c243a
|
remove unused GIT_SHA config option
|
2021-02-15 20:42:33 -05:00 |
|
|
Nick Sweeting
|
9cd4ba38f0
|
add new SNAPSHOTS_PER_PAGE pagination limit config
|
2021-02-15 20:42:00 -05:00 |
|
|
Nick Sweeting
|
00ae1f15a7
|
ignore shm db file and config files in archivebox data dir on init
|
2021-02-15 14:52:37 -05:00 |
|
|
Nick Sweeting
|
3c3bae02d2
|
add quick-init option to skip reimporting all snapshot dirs on init
|
2021-02-15 14:52:10 -05:00 |
|
|
Nick Sweeting
|
e61e12c889
|
use setup.py to determine dependencies in Dockerfile instead of egg-info requires.txt
|
2021-02-15 14:51:32 -05:00 |
|
|
Nick Sweeting
|
0407d03b6b
|
add cli tests file back
|
2021-02-15 13:39:49 -05:00 |
|
|
Nick Sweeting
|
611216765d
|
switch sqlite to use WAL mode by default to prevent database locked errors
|
2021-02-15 13:39:03 -05:00 |
|
|
Nick Sweeting
|
683a08772b
|
change wording of db not found error
|
2021-02-08 23:27:46 -05:00 |
|
|
Nick Sweeting
|
6705354e57
|
fix assertion
|
2021-02-08 23:24:48 -05:00 |
|