Pellaeon Lin
5e9d05483e
Fix bin_version: set LANG=C when calling executables to avoid parsing
...
localized output.
2022-02-24 17:01:00 +08:00
prnake
011bd104cb
remove unused import
2022-02-09 10:48:51 +08:00
papersnake
de8e22efb7
improve title extractor
2022-02-08 23:17:52 +08:00
Nick Sweeting
666ab20df5
Update archivebox/config.py
2022-01-10 20:42:09 -05:00
hannah98
fc3d2bb4dc
rename TAG_SEPARATORS to TAG_SEPARATOR_PATTERN
2022-01-06 14:14:41 +00:00
hannah98
049f88def9
Added TAG_SEPARATORS option to supply a regex of characters to use when splitting tags
2021-12-30 20:19:48 +00:00
Nick Sweeting
d7f01922f3
fix direct assignment of tags to many-to-many set
2021-12-23 12:29:17 -05:00
Nick Sweeting
b1b7ee2b85
Update sql.py
2021-12-23 12:17:55 -05:00
hannah98
4b8962b60b
Fix #725 - correctly parse tags on json import
2021-12-20 08:58:58 -06:00
Mika Tuupola
f14a861605
Change logfile open to write mode only
2021-12-19 23:17:33 +02:00
TheCakeIsNaOH
decab91ea2
( #847 ) Decode error output hints to string if needed
2021-12-16 16:46:12 -06:00
Nick Sweeting
44f5338470
fix typo in pocket_api articl variable name
2021-11-12 19:23:47 -05:00
Nick Sweeting
8878dcc5e8
Merge pull request #843 from bltavares/patch-1
2021-11-12 15:57:19 -08:00
Nick Sweeting
d5240f1a1d
Merge pull request #885 from adamwolf/safari-admin-actions
2021-11-12 08:56:31 -08:00
Adam Wolf
18e1fb0d96
Fixes Add button behavior on Safari
...
Previously, when you clicked the Add button, the page wouldn't change.
It looked like it wasn't doing anything, as noted by @rcarmo
(https://github.com/ArchiveBox/ArchiveBox/issues/658#issuecomment-948300055 )
I didn't track it down the exact reason why. It may be that Safari
didn't like the two opening <h3>s, but I was able to find a bunch of
people complaining about Safari being very finicky with innerHTML
and actually repainting the page, enough that I decided to try just
extending the block hide/show behavior already done with the
delay-warning, and it works for me now in both Chrome and Safari.
For #658 .
2021-10-28 22:31:54 -05:00
Adam Wolf
83731f5a68
Tweak JS so Safari can choose admin actions
...
I noticed that Safari was submitting both the empty option and the
selected options back to the server.
Digging into it, I was able to get Safari to deselect the ---------
option by using '[selected]' as the selector.
For #658
2021-10-28 22:22:46 -05:00
Igor Rzegocki
05de1c9fe6
healthcheck endpoint
2021-10-03 19:12:03 +02:00
Bruno Tavares
bb2a2e758a
Avoid KeyError on Pocket API parser
...
When trying to import my pocket library I got a lot of ` KeyError` on Python. Pocket API has a few idiosyncrasies, such as sometimes returning the keys on json, sometimes not.
` ` ` sh
archivebox add --parser pocket_api pocket://my_username
` ` `
Gave me this errors
` ` `
File "/app/archivebox/parsers/pocket_api.py", line 54, in link_from_article
title = article['resolved_title'] or article['given_title'] or url
KeyError: 'resolved_title'
` ` `
This commit are the patches I've changed to successfully import my library
2021-09-07 21:53:36 -03:00
Ross Williams
f6cf35a45d
Fix Pinboard RSS parsing valid links as None
...
`item.find(p)` returns either an `ElementTree.Element` or `None`. The
[lambda on line 24][lambda] coerces the return value to a bool, which is
`False` if the `<link>` element has no children (see
[`ElementTree.py` line 207][etbooldef]), so the lambda returns `None`.
Further, returning a `Link` with `url=None` violates
[an assertion in `index/schema.py`][assertion], which crashes
the `archivebox add` command.
[lambda]: 3d54b1321b/archivebox/parsers/pinboard_rss.py (L24)3d8993a744/Lib/xml/etree/ElementTree.py (L207)3d54b1321b/archivebox/index/schema.py (L165)
2021-08-04 10:13:37 -04:00
Inndy
0e81a0722e
Discard Referer header from iframe and link to original URL
2021-07-19 21:48:01 +08:00
Nick Sweeting
5a2c78e14b
add proper support for URL_WHITELIST instead of using negation regexes
2021-07-06 23:42:00 -04:00
Nick Sweeting
e4974d3536
support negation patterns by checking both re.search and re.match
2021-07-06 23:17:05 -04:00
TJ Horner
cdcfb7fa44
Exempt /add route from CSRF
2021-07-01 20:55:51 -04:00
Nick Sweeting
e0a2b2e252
ominous warnings
2021-06-01 03:03:42 -04:00
Nick Sweeting
aa53fe653c
fix use of uneeded perms arg
2021-06-01 02:58:36 -04:00
Nick Sweeting
c2d1a57581
fix umask dir permissions
2021-06-01 00:50:18 -04:00
Nick Sweeting
4715ace7dd
ignore BaseException lgtm errors
2021-05-31 20:59:05 -04:00
Nick Sweeting
9f1470cf03
fix output permissions tests
2021-05-31 20:57:46 -04:00
Nick Sweeting
8230f88d80
change default OUTPUT_PERMISSIONS to disallow execution except on dirs
2021-05-31 19:31:51 -04:00
Nick Sweeting
1112526543
add option ENFORCE_ATOMIC_WRITES to allow disabling forced FSYNC writes on network drives
2021-05-31 19:31:51 -04:00
Nick Sweeting
49faec8f6d
add no-zygote and single-process args to try and prevent orphan chrome processes after exit
2021-05-13 05:04:23 -04:00
Nick Sweeting
eb4d3bca9d
Update readability.py
2021-05-13 00:13:32 -04:00
Nick Sweeting
c3d009e4ec
fix python file execution checking
2021-04-24 04:43:02 -04:00
Nick Sweeting
79e19ecd47
shield pwd import for windows
2021-04-24 03:51:38 -04:00
Nick Sweeting
3db77fd1a2
fix lint errors
2021-04-24 03:49:01 -04:00
Nick Sweeting
3992e0fee3
auto prepend python binary to args when running system command with python file as first argument
2021-04-24 03:29:22 -04:00
Nick Sweeting
fa84f749ff
run archivebox version using python binary
2021-04-24 03:29:22 -04:00
Nick Sweeting
226e26852c
dont try to autodetect whether node, chrome, etc are needed during setup
2021-04-24 03:29:22 -04:00
Nick Sweeting
f129b9f443
allow executing main
2021-04-24 03:11:06 -04:00
Nick Sweeting
208f866fc4
fix user detection on freebsd always returning root
2021-04-24 02:52:09 -04:00
Nick Sweeting
06f6084e3b
ignore stdin when passed instead of throwing an error
2021-04-24 00:09:52 -04:00
Nick Sweeting
eb80dc26a1
cleanup config files
2021-04-23 22:58:44 -04:00
Nick Sweeting
4f91af0ce9
TODO create admin user from env vars
2021-04-23 15:50:34 -04:00
Nick Sweeting
9563e74f7c
enforce minimum django version
2021-04-23 15:02:26 -04:00
Kunwoo Ahn
b6ee7d291e
fix bookmarklet script
...
Fix '&' bug in parameter
2021-04-16 21:24:51 +09:00
Nick Sweeting
63552be0e3
whoops
2021-04-13 04:03:47 -04:00
Nick Sweeting
7d1f8cb496
no longer recommend snap chromium, just use playwright
2021-04-13 03:59:17 -04:00
Nick Sweeting
2c6f0a96bf
fix extra arg
2021-04-13 02:21:51 -04:00
Nick Sweeting
54cab113f3
allow static_index.json in dir root from old static exports
2021-04-12 17:06:32 -04:00
Nick Sweeting
50b341baab
bail out if old index.json is found during init but doesnt contain links
2021-04-12 16:51:45 -04:00
Nick Sweeting
217952c8d0
sigh linters
2021-04-11 12:51:50 -04:00
Nick Sweeting
bfefe042bf
fix noqa
2021-04-11 03:57:04 -04:00
Nick Sweeting
335e4658c8
keep grid header on one line
2021-04-11 00:49:33 -04:00
Nick Sweeting
52429b8d15
dont prevent search when backend throws an exception or times out
2021-04-10 08:18:13 -04:00
Nick Sweeting
9f05cf8283
virtual-time-budget doesnt work with some chrome stuff
2021-04-10 08:04:59 -04:00
Nick Sweeting
62078a77f8
show run duration after each archived link in cli output
2021-04-10 07:52:01 -04:00
Nick Sweeting
6a6ef63775
enforce utf8 on more file reads
2021-04-10 07:22:32 -04:00
Nick Sweeting
193df5c8d3
add video subtitles and description to full-text index
2021-04-10 07:22:20 -04:00
Nick Sweeting
1e3c1e3c83
also write autogenerated subtitles
2021-04-10 06:56:19 -04:00
Nick Sweeting
e7473c4e3e
add some more padding to public index columns
2021-04-10 06:38:33 -04:00
Nick Sweeting
6949803395
enforce new models to use uuid keys
2021-04-10 06:32:45 -04:00
Nick Sweeting
1b67dd5c45
fix issues with django 3.2
2021-04-10 06:22:07 -04:00
Nick Sweeting
0c321a06d0
hide scrollbars in screenshots
2021-04-10 05:45:19 -04:00
Nick Sweeting
c5a9b77f4f
update user agents to mimic latest chrome
2021-04-10 05:45:02 -04:00
Nick Sweeting
bf0da664be
fix favicons on public index
2021-04-10 05:23:02 -04:00
Nick Sweeting
3093057675
bail out on sonic indexing after 5 errors
2021-04-10 05:18:03 -04:00
Nick Sweeting
89158d5dc7
allow full-text search from public index
2021-04-10 05:14:02 -04:00
Nick Sweeting
d1051e40bd
remove tzlocal
2021-04-10 04:42:39 -04:00
Nick Sweeting
a9986f1f05
add timezone support, tons of CSS and layout improvements, more detailed snapshot admin form info, ability to sort by recently updated, better grid view styling, better table layouts, better dark mode support
2021-04-10 04:21:36 -04:00
Nick Sweeting
cf7d7e4990
add new timezone autosetting and cache header setting middlewares
2021-04-10 04:21:36 -04:00
Nick Sweeting
1977ae8962
add more helper props to snapshot for getting latest headers, favicon, etc
2021-04-10 04:21:36 -04:00
Nick Sweeting
8d68f1744e
tweak columns in private snapshots list and shorten action names
2021-04-10 04:21:36 -04:00
Nick Sweeting
722f530c80
support adding and removing multiple tags at once using autocomplete multiselect field
2021-04-10 04:21:36 -04:00
Nick Sweeting
06c58ac42a
fix unreachable config backup clear
2021-04-09 12:15:47 -04:00
Nick Sweeting
32764347ce
add new SEARCH_BACKEND_TIMEOUT config option defaulted to 90sec
2021-04-09 00:27:12 -04:00
Nick Sweeting
2e57df917e
handle BaseExceptions properly
2021-04-08 06:08:17 -04:00
Nick Sweeting
2c07d8f1f3
fix lgtm alerts
2021-04-06 13:47:26 -04:00
Nick Sweeting
21bb84f627
fix lint errors
2021-04-06 02:50:51 -04:00
Nick Sweeting
e92db03f5f
fallback to using bin_path to find archivebox binary if sys.argv is empty
2021-04-06 02:45:57 -04:00
Nick Sweeting
c98053a8ca
allow filtering by archiveresult status
2021-04-06 02:07:22 -04:00
Nick Sweeting
f67a5a215a
fix readability indexing process and implement a max total character length on indexed content
2021-04-06 02:01:38 -04:00
Nick Sweeting
b3a89172ab
add original src url to dosctring
2021-04-06 01:38:24 -04:00
Nick Sweeting
e7c7a8f71c
force kill forked child processes after each extractor command
2021-04-06 01:35:21 -04:00
Nick Sweeting
af6561b527
remove CDN jquery and bootstrap by inlining them instead
2021-04-06 01:13:31 -04:00
Nick Sweeting
e4823cc5d7
add back CUSTOM_TEMPLATES_DIR config option
2021-04-06 00:57:20 -04:00
Nick Sweeting
4b098bea15
fix navbar showing user info even when not logged in
2021-04-06 00:44:01 -04:00
Nick Sweeting
68a506e1a7
fix Snapshot admin actions buttons submitting on Safari
2021-04-06 00:43:49 -04:00
Nick Sweeting
5a9f27204a
dont use chrome when its not available on windows systems
2021-04-05 23:33:08 -04:00
Nick Sweeting
1485e4eb90
use older docker compose version for more features
2021-04-05 23:22:05 -04:00
Nick Sweeting
cadac4838e
fix searching by url contents
2021-04-05 23:21:07 -04:00
Nick Sweeting
2fc9f1cfa3
dont depend on README in Dockerfile install step
2021-04-05 23:17:07 -04:00
Nick Sweeting
8b4f84959a
new setup flag on init command to autosetup on init
2021-04-05 21:15:32 -04:00
Nick Sweeting
5c181532b5
only install dependencies in setup if they are missing
2021-04-05 21:15:10 -04:00
Nick Sweeting
21820acecd
import User model into core models for convenience
2021-04-05 21:14:22 -04:00
Nick Sweeting
bbebc33b7a
better autodetection of node modules dependencies
2021-04-05 21:14:06 -04:00
Nick Sweeting
211bf86f4a
fix bug checking value in subcommand_args when None
2021-04-05 21:13:49 -04:00
Nick Sweeting
79fb60a47a
dont allow caching when installing packages
2021-04-05 20:08:48 -04:00
Nick Sweeting
05c17af4dc
fix lint errors
2021-04-05 20:00:16 -04:00
Nick Sweeting
ef76f7c490
require running setup in existing data dir
2021-04-05 19:59:33 -04:00
Nick Sweeting
8c1d12020f
fix lint errors
2021-04-05 19:53:42 -04:00
