Nick Sweeting
|
217952c8d0
|
sigh linters
|
2021-04-11 12:51:50 -04:00 |
|
|
Nick Sweeting
|
bfefe042bf
|
fix noqa
|
2021-04-11 03:57:04 -04:00 |
|
|
Nick Sweeting
|
335e4658c8
|
keep grid header on one line
|
2021-04-11 00:49:33 -04:00 |
|
|
Nick Sweeting
|
52429b8d15
|
dont prevent search when backend throws an exception or times out
|
2021-04-10 08:18:13 -04:00 |
|
|
Nick Sweeting
|
9f05cf8283
|
virtual-time-budget doesnt work with some chrome stuff
|
2021-04-10 08:04:59 -04:00 |
|
|
Nick Sweeting
|
62078a77f8
|
show run duration after each archived link in cli output
|
2021-04-10 07:52:01 -04:00 |
|
|
Nick Sweeting
|
6a6ef63775
|
enforce utf8 on more file reads
|
2021-04-10 07:22:32 -04:00 |
|
|
Nick Sweeting
|
193df5c8d3
|
add video subtitles and description to full-text index
|
2021-04-10 07:22:20 -04:00 |
|
|
Nick Sweeting
|
1e3c1e3c83
|
also write autogenerated subtitles
|
2021-04-10 06:56:19 -04:00 |
|
|
Nick Sweeting
|
e7473c4e3e
|
add some more padding to public index columns
|
2021-04-10 06:38:33 -04:00 |
|
|
Nick Sweeting
|
6949803395
|
enforce new models to use uuid keys
|
2021-04-10 06:32:45 -04:00 |
|
|
Nick Sweeting
|
1b67dd5c45
|
fix issues with django 3.2
|
2021-04-10 06:22:07 -04:00 |
|
|
Nick Sweeting
|
0c321a06d0
|
hide scrollbars in screenshots
|
2021-04-10 05:45:19 -04:00 |
|
|
Nick Sweeting
|
c5a9b77f4f
|
update user agents to mimic latest chrome
|
2021-04-10 05:45:02 -04:00 |
|
|
Nick Sweeting
|
bf0da664be
|
fix favicons on public index
|
2021-04-10 05:23:02 -04:00 |
|
|
Nick Sweeting
|
3093057675
|
bail out on sonic indexing after 5 errors
|
2021-04-10 05:18:03 -04:00 |
|
|
Nick Sweeting
|
89158d5dc7
|
allow full-text search from public index
|
2021-04-10 05:14:02 -04:00 |
|
|
Nick Sweeting
|
d1051e40bd
|
remove tzlocal
|
2021-04-10 04:42:39 -04:00 |
|
|
Nick Sweeting
|
a9986f1f05
|
add timezone support, tons of CSS and layout improvements, more detailed snapshot admin form info, ability to sort by recently updated, better grid view styling, better table layouts, better dark mode support
|
2021-04-10 04:21:36 -04:00 |
|
|
Nick Sweeting
|
cf7d7e4990
|
add new timezone autosetting and cache header setting middlewares
|
2021-04-10 04:21:36 -04:00 |
|
|
Nick Sweeting
|
1977ae8962
|
add more helper props to snapshot for getting latest headers, favicon, etc
|
2021-04-10 04:21:36 -04:00 |
|
|
Nick Sweeting
|
8d68f1744e
|
tweak columns in private snapshots list and shorten action names
|
2021-04-10 04:21:36 -04:00 |
|
|
Nick Sweeting
|
722f530c80
|
support adding and removing multiple tags at once using autocomplete multiselect field
|
2021-04-10 04:21:36 -04:00 |
|
|
Nick Sweeting
|
06c58ac42a
|
fix unreachable config backup clear
|
2021-04-09 12:15:47 -04:00 |
|
|
Nick Sweeting
|
32764347ce
|
add new SEARCH_BACKEND_TIMEOUT config option defaulted to 90sec
|
2021-04-09 00:27:12 -04:00 |
|
|
Nick Sweeting
|
2e57df917e
|
handle BaseExceptions properly
|
2021-04-08 06:08:17 -04:00 |
|
|
Nick Sweeting
|
2c07d8f1f3
|
fix lgtm alerts
|
2021-04-06 13:47:26 -04:00 |
|
|
Nick Sweeting
|
21bb84f627
|
fix lint errors
|
2021-04-06 02:50:51 -04:00 |
|
|
Nick Sweeting
|
e92db03f5f
|
fallback to using bin_path to find archivebox binary if sys.argv is empty
|
2021-04-06 02:45:57 -04:00 |
|
|
Nick Sweeting
|
c98053a8ca
|
allow filtering by archiveresult status
|
2021-04-06 02:07:22 -04:00 |
|
|
Nick Sweeting
|
f67a5a215a
|
fix readability indexing process and implement a max total character length on indexed content
|
2021-04-06 02:01:38 -04:00 |
|
|
Nick Sweeting
|
b3a89172ab
|
add original src url to dosctring
|
2021-04-06 01:38:24 -04:00 |
|
|
Nick Sweeting
|
e7c7a8f71c
|
force kill forked child processes after each extractor command
|
2021-04-06 01:35:21 -04:00 |
|
|
Nick Sweeting
|
af6561b527
|
remove CDN jquery and bootstrap by inlining them instead
|
2021-04-06 01:13:31 -04:00 |
|
|
Nick Sweeting
|
e4823cc5d7
|
add back CUSTOM_TEMPLATES_DIR config option
|
2021-04-06 00:57:20 -04:00 |
|
|
Nick Sweeting
|
4b098bea15
|
fix navbar showing user info even when not logged in
|
2021-04-06 00:44:01 -04:00 |
|
|
Nick Sweeting
|
68a506e1a7
|
fix Snapshot admin actions buttons submitting on Safari
|
2021-04-06 00:43:49 -04:00 |
|
|
Nick Sweeting
|
5a9f27204a
|
dont use chrome when its not available on windows systems
|
2021-04-05 23:33:08 -04:00 |
|
|
Nick Sweeting
|
1485e4eb90
|
use older docker compose version for more features
|
2021-04-05 23:22:05 -04:00 |
|
|
Nick Sweeting
|
cadac4838e
|
fix searching by url contents
|
2021-04-05 23:21:07 -04:00 |
|
|
Nick Sweeting
|
2fc9f1cfa3
|
dont depend on README in Dockerfile install step
|
2021-04-05 23:17:07 -04:00 |
|
|
Nick Sweeting
|
8b4f84959a
|
new setup flag on init command to autosetup on init
|
2021-04-05 21:15:32 -04:00 |
|
|
Nick Sweeting
|
5c181532b5
|
only install dependencies in setup if they are missing
|
2021-04-05 21:15:10 -04:00 |
|
|
Nick Sweeting
|
21820acecd
|
import User model into core models for convenience
|
2021-04-05 21:14:22 -04:00 |
|
|
Nick Sweeting
|
bbebc33b7a
|
better autodetection of node modules dependencies
|
2021-04-05 21:14:06 -04:00 |
|
|
Nick Sweeting
|
211bf86f4a
|
fix bug checking value in subcommand_args when None
|
2021-04-05 21:13:49 -04:00 |
|
|
Nick Sweeting
|
79fb60a47a
|
dont allow caching when installing packages
|
2021-04-05 20:08:48 -04:00 |
|
|
Nick Sweeting
|
05c17af4dc
|
fix lint errors
|
2021-04-05 20:00:16 -04:00 |
|
|
Nick Sweeting
|
ef76f7c490
|
require running setup in existing data dir
|
2021-04-05 19:59:33 -04:00 |
|
|
Nick Sweeting
|
8c1d12020f
|
fix lint errors
|
2021-04-05 19:53:42 -04:00 |
|
|
Nick Sweeting
|
3449bac5bc
|
add new setup command to autoinstall dependencies
|
2021-04-05 19:51:57 -04:00 |
|
|
Nick Sweeting
|
24f7dd9c05
|
dont raise systemexit on succesful config
|
2021-04-05 19:51:40 -04:00 |
|
|
Nick Sweeting
|
ffd119c66c
|
fix linter
|
2021-04-03 15:51:27 -04:00 |
|
|
Nick Sweeting
|
7162649b03
|
fix search fields on admin snapshots list
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
5492c204b8
|
tweak add page copy
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
f59b6d4189
|
only add url-list lines that are real urls
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
d73f7d7d96
|
add db_index on url field
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
c438e49871
|
fix clicking on tags on snapshot list
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
0e39a2098d
|
split migrations and data dir check
|
2021-04-01 14:00:07 -04:00 |
|
|
Nick Sweeting
|
5d7ad9b1b4
|
add form parser option
|
2021-04-01 02:34:16 -04:00 |
|
|
Nick Sweeting
|
59d5423483
|
fix snapshot icon caching and ordering
|
2021-04-01 02:22:15 -04:00 |
|
|
Nick Sweeting
|
fc5d99b4b4
|
fix missing package.json repo
|
2021-04-01 01:40:02 -04:00 |
|
|
Nick Sweeting
|
ff9445648b
|
fix navbar and tags display
|
2021-04-01 01:39:34 -04:00 |
|
|
Nick Sweeting
|
7ee7a94e67
|
fix str split using old overwrite pipe instead of append pipe
|
2021-03-31 11:30:15 -04:00 |
|
|
Nick Sweeting
|
005da31f8f
|
add overwrite flag option to schedule cmd
|
2021-03-31 11:29:51 -04:00 |
|
|
Nick Sweeting
|
534ae60ab0
|
dont overwrite the log on each run, and use dedicated log for scheduled archiving
|
2021-03-31 11:20:42 -04:00 |
|
|
Nick Sweeting
|
5d3a03b299
|
use stderr and hint in case of parser returning no urls instead of bare exception
|
2021-03-31 01:39:01 -04:00 |
|
|
Nick Sweeting
|
8ce93ff787
|
use KEY, NAME, and PARSER to define parsers instead of hardcoding in init
|
2021-03-31 01:05:49 -04:00 |
|
|
Nick Sweeting
|
36f0646501
|
Merge pull request #669 from FliegendeWurst/fix-issue-235
add command: --parser option (fixes #235)
|
2021-03-31 00:53:47 -04:00 |
|
|
Nick Sweeting
|
2656e59215
|
change list style
|
2021-03-31 00:47:42 -04:00 |
|
|
Nick Sweeting
|
bd8a01e677
|
sigh vendor yet another dependency
|
2021-03-30 14:49:42 -04:00 |
|
FliegendeWurst
|
60bd9a902e
|
add command: --parser option
|
2021-03-28 10:09:11 +02:00 |
|
|
Nick Sweeting
|
a58ad5b272
|
allow larger tags
|
2021-03-27 05:52:42 -04:00 |
|
|
Nick Sweeting
|
7e7c134ec0
|
use exc
|
2021-03-27 05:32:07 -04:00 |
|
|
Nick Sweeting
|
5568b3b612
|
fix adding and removing tags via admin snapshot list
|
2021-03-27 05:27:02 -04:00 |
|
|
Nick Sweeting
|
49b77b91bb
|
fix unused var
|
2021-03-27 05:12:54 -04:00 |
|
|
Nick Sweeting
|
3e26ae4a66
|
support finding multiple urls as substrings in text
|
2021-03-27 04:30:40 -04:00 |
|
|
Nick Sweeting
|
f3a3d76439
|
support adding urls with tags directly via CLI and add page
|
2021-03-27 04:30:15 -04:00 |
|
|
Nick Sweeting
|
5fb9ca389f
|
check more url parsing invariants on startup
|
2021-03-27 03:57:22 -04:00 |
|
|
Nick Sweeting
|
fea0b89dbe
|
add tag cli option
|
2021-03-27 03:57:05 -04:00 |
|
|
Nick Sweeting
|
d486f8c162
|
use tempfile when logs dir is not available
|
2021-03-27 02:17:12 -04:00 |
|
|
Nick Sweeting
|
3c2f4c95cf
|
fix unused import
|
2021-03-27 01:50:26 -04:00 |
|
|
Nick Sweeting
|
56c6b11b14
|
add comment
|
2021-03-27 01:19:32 -04:00 |
|
|
Nick Sweeting
|
bd6d9c165b
|
enforce utf8 on literally all file operations because windows sucks
|
2021-03-27 01:16:29 -04:00 |
|
|
Nick Sweeting
|
185d2f9f9b
|
check for JSON1 extesnion in Sqlite on startup
|
2021-03-27 00:27:58 -04:00 |
|
|
Nick Sweeting
|
177d22da50
|
fix clicking add tag when no tag is selected throwing 500
|
2021-03-01 01:39:06 -05:00 |
|
|
Nick Sweeting
|
76ff789164
|
fix tags page rendering when snapshot has no updated timestamp
|
2021-03-01 00:40:41 -05:00 |
|
|
Nick Sweeting
|
0036e9cce2
|
add migration
|
2021-02-28 22:55:12 -05:00 |
|
|
Nick Sweeting
|
9fc9f52cfb
|
increase sqlite timeout and dont check for same thread condition
|
2021-02-28 22:55:00 -05:00 |
|
|
Nick Sweeting
|
1cabde3ccd
|
remove atomic transactions
|
2021-02-28 22:54:40 -05:00 |
|
|
Nick Sweeting
|
844b5c5e20
|
minor db fix
|
2021-02-28 22:54:04 -05:00 |
|
|
Nick Sweeting
|
62089fdb22
|
close sqlite3 connections if unused
|
2021-02-28 22:53:34 -05:00 |
|
|
Nick Sweeting
|
6aef1dd98d
|
dont check db if init is pending
|
2021-02-28 22:53:23 -05:00 |
|
|
Nick Sweeting
|
acbce25201
|
missing migrations
|
2021-02-18 08:05:05 -05:00 |
|
|
Nick Sweeting
|
9ead76f9f8
|
cross link snapshot index, admin, and actions pages using uuids
|
2021-02-18 08:04:50 -05:00 |
|
|
Nick Sweeting
|
c257ae240e
|
only enable debug toolbar if nothreading is on
|
2021-02-18 08:04:26 -05:00 |
|
|
Nick Sweeting
|
ca2bb673ea
|
improve inline admin forms and add tags autocomplete
|
2021-02-18 08:04:14 -05:00 |
|
|
Nick Sweeting
|
33df9c1ebe
|
fix after and before in remove tests
|
2021-02-18 06:21:44 -05:00 |
|
|
Nick Sweeting
|
9c07fbdc0b
|
unused import fix
|
2021-02-18 05:56:00 -05:00 |
|
|
Nick Sweeting
|
106f6adc59
|
fix lgtm alerts
|
2021-02-18 04:29:24 -05:00 |
|
|
Nick Sweeting
|
46a4197514
|
fix tests
|
2021-02-18 04:26:56 -05:00 |
|
|
Nick Sweeting
|
13c2b7c8eb
|
fix url used by gridview
|
2021-02-18 02:38:30 -05:00 |
|
|
Nick Sweeting
|
e3b69c450c
|
move db checks under check_db if statement
|
2021-02-18 02:37:33 -05:00 |
|
|
Nick Sweeting
|
74a9dd8880
|
add missing migrations
|
2021-02-18 02:36:21 -05:00 |
|
|
Nick Sweeting
|
7282f4a42a
|
add favicon and robots to internal static dir
|
2021-02-18 02:35:51 -05:00 |
|
|
Nick Sweeting
|
dcc812b9f5
|
make output link easier to see on archiveresult admin list
|
2021-02-18 02:35:33 -05:00 |
|
|
Nick Sweeting
|
df47486d31
|
make admin link easier to find on snapshot detail page
|
2021-02-18 02:35:11 -05:00 |
|
|
Nick Sweeting
|
8369ca0b87
|
fix snapshot detail page titles garbled due to double escaping
|
2021-02-18 02:34:56 -05:00 |
|
|
Nick Sweeting
|
ba2afa0672
|
allow cancelling init partway through and show hints to resume
|
2021-02-18 02:34:42 -05:00 |
|
|
Nick Sweeting
|
75e1bfd0a9
|
create_or_update ArchiveResults from history instead of get_or_create
|
2021-02-18 02:34:20 -05:00 |
|
|
Nick Sweeting
|
69e0302c2a
|
only add https to url if needed when prompting to save new snapshot
|
2021-02-18 02:34:03 -05:00 |
|
|
Nick Sweeting
|
9918bddb7e
|
remove emptystring tags before saving
|
2021-02-18 02:33:42 -05:00 |
|
|
Nick Sweeting
|
f1823381d0
|
improve latest title logic to take longest title
|
2021-02-18 02:33:28 -05:00 |
|
|
Nick Sweeting
|
ad4bd49163
|
increase max title length to 512
|
2021-02-18 02:33:08 -05:00 |
|
|
Nick Sweeting
|
4e5671dda9
|
add ability to add and remove tags directly from snapshot list
|
2021-02-18 02:32:58 -05:00 |
|
|
Nick Sweeting
|
c877d67e0c
|
check if pragma is already wal before setting
|
2021-02-18 02:31:42 -05:00 |
|
|
Nick Sweeting
|
7b7aa239fd
|
autocreate db cache table when needed and reenable WAL mode
|
2021-02-17 18:26:04 -05:00 |
|
|
Nick Sweeting
|
d60d73754d
|
fix favicon.ico and robots.txt served by runserver
|
2021-02-17 18:25:47 -05:00 |
|
|
Nick Sweeting
|
a8a6752b06
|
add CACHE_BACKEND options to settings for easier dev
|
2021-02-17 18:25:23 -05:00 |
|
|
Nick Sweeting
|
3457773988
|
fix ArchiveResult extractor showing up on top of dropdown in admin inline form
|
2021-02-17 18:25:01 -05:00 |
|
|
Nick Sweeting
|
084cf7ff51
|
add more explanation about snapshot.save timestamp bump
|
2021-02-17 13:34:46 -05:00 |
|
|
Nick Sweeting
|
265bcc0264
|
fix lint errors2
|
2021-02-16 16:29:41 -05:00 |
|
|
Nick Sweeting
|
6f0eec92eb
|
fix lint errors
|
2021-02-16 16:26:48 -05:00 |
|
|
Nick Sweeting
|
d89034dcde
|
disable debug_toolbar by default
|
2021-02-16 16:23:09 -05:00 |
|
|
Nick Sweeting
|
71cf8d5224
|
add migrations
|
2021-02-16 15:57:13 -05:00 |
|
|
Nick Sweeting
|
118bddb41a
|
fix admin title and url truncation and add edit/admin link and ID to Snapshot detail page
|
2021-02-16 15:55:07 -05:00 |
|
|
Nick Sweeting
|
05e891632c
|
add snapshot_id to Link and uuid to ArchiveResult
|
2021-02-16 15:54:27 -05:00 |
|
|
Nick Sweeting
|
0b8bdd8cc5
|
show dir size after each snapshot archiving process in cli
|
2021-02-16 15:53:52 -05:00 |
|
|
Nick Sweeting
|
acb932ba12
|
improve readability and mercury error handling and fix output path to be relative
|
2021-02-16 15:53:11 -05:00 |
|
|
Nick Sweeting
|
c95698e608
|
bump Snapshot.updated time after each extractor, change extractor order
|
2021-02-16 15:52:18 -05:00 |
|
|
Nick Sweeting
|
28e50c5e49
|
tweak snapshot asset serving logic to show multiple choices in case of conflict
|
2021-02-16 15:51:56 -05:00 |
|
|
Nick Sweeting
|
a6c64f2560
|
only get archive_size if index.html is present within dir
|
2021-02-16 15:51:17 -05:00 |
|
|
Nick Sweeting
|
c24e4bf11d
|
tweak django debug toolbar panels and add djdt_flamegraph
|
2021-02-16 15:51:02 -05:00 |
|
|
Nick Sweeting
|
1e2188517b
|
speed up latest_title by preferring db title if present
|
2021-02-16 15:49:53 -05:00 |
|
|
Nick Sweeting
|
8b236b9367
|
cache dir size, snapshot icons, tags str, and title in django cache
|
2021-02-16 15:49:29 -05:00 |
|
|
Nick Sweeting
|
51440ede3a
|
show extra fields on ArchiveResult admin list and detail page
|
2021-02-16 15:47:49 -05:00 |
|
|
Nick Sweeting
|
74c882ef16
|
add nothreading arg to archivebox server for perf profiling
|
2021-02-16 15:46:58 -05:00 |
|
|
Nick Sweeting
|
8cde6bc676
|
add gist.github.com to git domains
|
2021-02-16 07:36:09 -05:00 |
|
|
Nick Sweeting
|
bdf1b102be
|
load ArchiveResults from orphaned links history during init
|
2021-02-16 06:20:05 -05:00 |
|
|
Nick Sweeting
|
d0f8a5e710
|
change mercury atomic_write output order
|
2021-02-16 06:19:16 -05:00 |
|
|
Nick Sweeting
|
e62ecffeb0
|
disable django caches explicitly
|
2021-02-16 06:18:39 -05:00 |
|
|
Nick Sweeting
|
988a10a9f6
|
fix warc path in snapshot_icons
|
2021-02-16 06:18:05 -05:00 |
|
|
Nick Sweeting
|
48eddadc9b
|
inline archive_size and archive_path snapshot methods
|
2021-02-16 06:16:02 -05:00 |
|
|
Nick Sweeting
|
46a7b972ee
|
disable WAL and cache table for now
|
2021-02-16 06:13:50 -05:00 |
|
|
Nick Sweeting
|
24ae01cdbe
|
add ArchiveResult admin
|
2021-02-16 06:13:27 -05:00 |
|
|
Nick Sweeting
|
87069dd41e
|
fix wsgi.py
|
2021-02-16 04:16:51 -05:00 |
|
|
Nick Sweeting
|
8c2d81c041
|
always serve static files
|
2021-02-16 04:16:24 -05:00 |
|
|
Nick Sweeting
|
fb8e6cabcb
|
fix error log location
|
2021-02-16 04:15:09 -05:00 |
|
|
Nick Sweeting
|
26fa63749d
|
add debug toolbar
|
2021-02-16 02:50:05 -05:00 |
|
|
Nick Sweeting
|
fad2620c62
|
minor tweaks
|
2021-02-16 02:49:31 -05:00 |
|