CodeMirror/ArchiveBox
1
0
Fork 0
mirror of synced 2024-07-01 20:40:41 +12:00
Code Issues Projects Releases Wiki Activity
3058 commits 5 branches 31 tags 9.4 MiB
Commit graph

1246 commits

Author SHA1 Message Date
Nick Sweeting 33ec2117e9
Update main.py 2022-06-08 19:50:45 -07:00
Nick Sweeting dd29e1bf78
clean up first line of CLI version output for easier downstream parsing 2022-06-08 19:46:09 -07:00
Nick Sweeting 0c6d4c82c3
Update config.py 2022-06-08 19:11:02 -07:00
Nick Sweeting f9c5808940
Update config.py 2022-06-08 19:09:11 -07:00
Nick Sweeting 5509b5cd8b
Update main.py 2022-06-08 19:08:33 -07:00
Nick Sweeting 19b88d30b2
fix missing brace 2022-06-08 19:06:03 -07:00
Nick Sweeting 31d5fbbf17
Update config.py 2022-06-08 19:04:06 -07:00
Nick Sweeting 6b019da3e9
Update config.py 2022-06-08 19:01:55 -07:00
Nick Sweeting c752c7053d
Update main.py 2022-06-08 18:59:08 -07:00
Nick Sweeting f9c82841ad
fix sqlite option detection 2022-06-08 18:58:15 -07:00
Nick Sweeting 1fd5830f58
enforce UTC timezone on server 2022-06-08 18:41:22 -07:00
Nick Sweeting 3e3c011f86
enforce UTC timezone on server 2022-06-08 18:40:48 -07:00
Nick Sweeting e06717419c
fix sqlite3 version detection 2022-06-08 18:35:31 -07:00
Nick Sweeting d0f129295f
move sqlite3 checks up a level 2022-06-08 18:29:53 -07:00
Nick Sweeting 0c7d7deb32
add missing brace 2022-06-08 18:26:42 -07:00
Nick Sweeting ca16c88a3d
show PUID, PGID, ENFORCE_ATOMIC_WRITES, and OUTPUT_PERMISSIONS in version output header 2022-06-08 18:24:58 -07:00
Nick Sweeting 89175ccb22
check SQLite3 version and enabled extensions on startup 2022-06-08 18:24:17 -07:00
Nick Sweeting c245d36e44
add PUID and PGID as config options in archivebox 2022-06-08 17:42:52 -07:00
Nick Sweeting c5fc3e1e65 --ammend 2022-05-09 23:59:27 -07:00
Nick Sweeting 0b4df768ba hack to check for generator type cause too lazy to import 2022-05-09 23:50:56 -07:00
Nick Sweeting 5e4ddbbf25 fix mercury bin parsing back 2022-05-09 21:58:17 -07:00
Nick Sweeting e96c1bcf13 bump mercury parser to git head version 2022-05-09 21:48:41 -07:00
Nick Sweeting d581a5081f correctly handle bytes strings in hints 2022-05-09 21:29:37 -07:00
Nick Sweeting a6767671fb append content of referenced files to imports 2022-05-09 21:21:39 -07:00
Nick Sweeting f6d6a06c78 always show all totals in log output 2022-05-09 21:21:26 -07:00
Nick Sweeting d05510f844 fix version string parsing on macOS in some cases where LANG C is not supported 2022-05-09 21:21:08 -07:00
Nick Sweeting 4b8b17e788 add update flag support to archivebox schedule 2022-05-09 20:18:43 -07:00
Nick Sweeting 8cfe6f4afb cleanup update flag handling and show better logging to clarify when its working 2022-05-09 20:15:55 -07:00
Nick Sweeting 38e54b93fe allow parsing to continue even when fetching URL contents fails 2022-05-09 19:56:24 -07:00
Nick Sweeting ecbcb6a1b3 fix bracing in template tag for PREVIEW_ORIGINALS 2022-05-09 19:56:08 -07:00
Nick Sweeting 8ebf3e2f93 add config option PREVIEW_ORIGINALS to hide original iframes in snapshot detail pages 2022-05-09 19:31:41 -07:00
Nick Sweeting acd53c854d handle new wallabag export format with newlines mid-tag attributes 2022-05-09 19:07:48 -07:00
Nick Sweeting 950b5cbbb6
Merge pull request #924 from prnake/dev 
improve title extractor
 2022-05-09 18:38:12 -07:00
Nick Sweeting 6e66863871 add max 5s writing delay for concurrent writers and flush WAL slower 2022-05-09 18:36:40 -07:00
Nick Sweeting 57df65f28f use yt-dlp for media archiving instead of youtube-dl 2022-04-21 07:11:35 -07:00
Nick Sweeting eb81d41f84 bump Dockerfile base image version and install yt-dlp 2022-04-21 07:11:35 -07:00
Ross c63822a5e5
Fix missing input redirection in a hint text 2022-04-19 22:25:49 +01:00
Igor Rzegocki d4f534e612
add LOGOUT_REDIRECT_URL 2022-03-31 21:40:14 +02:00
Pellaeon Lin 5e9d05483e Fix bin_version: set LANG=C when calling executables to avoid parsing 
localized output.
 2022-02-24 17:01:00 +08:00
prnake 011bd104cb
remove unused import 2022-02-09 10:48:51 +08:00
papersnake de8e22efb7 improve title extractor 2022-02-08 23:17:52 +08:00
Nick Sweeting 666ab20df5
Update archivebox/config.py 2022-01-10 20:42:09 -05:00
hannah98 fc3d2bb4dc rename TAG_SEPARATORS to TAG_SEPARATOR_PATTERN 2022-01-06 14:14:41 +00:00
hannah98 049f88def9 Added TAG_SEPARATORS option to supply a regex of characters to use when splitting tags 2021-12-30 20:19:48 +00:00
Nick Sweeting d7f01922f3
fix direct assignment of tags to many-to-many set 2021-12-23 12:29:17 -05:00
Nick Sweeting b1b7ee2b85
Update sql.py 2021-12-23 12:17:55 -05:00
hannah98 4b8962b60b Fix #725 - correctly parse tags on json import 2021-12-20 08:58:58 -06:00
Mika Tuupola f14a861605 Change logfile open to write mode only 2021-12-19 23:17:33 +02:00
TheCakeIsNaOH decab91ea2
(#847) Decode error output hints to string if needed 2021-12-16 16:46:12 -06:00
Nick Sweeting 44f5338470
fix typo in pocket_api articl variable name 2021-11-12 19:23:47 -05:00
Nick Sweeting 8878dcc5e8
Merge pull request #843 from bltavares/patch-1 2021-11-12 15:57:19 -08:00
Nick Sweeting d5240f1a1d
Merge pull request #885 from adamwolf/safari-admin-actions 2021-11-12 08:56:31 -08:00
Adam Wolf 18e1fb0d96 Fixes Add button behavior on Safari 
Previously, when you clicked the Add button, the page wouldn't change.

It looked like it wasn't doing anything, as noted by @rcarmo
(https://github.com/ArchiveBox/ArchiveBox/issues/658#issuecomment-948300055)

I didn't track it down the exact reason why. It may be that Safari
didn't like the two opening <h3>s, but I was able to find a bunch of
people complaining about Safari being very finicky with innerHTML
and actually repainting the page, enough that I decided to try just
extending the block hide/show behavior already done with the
delay-warning, and it works for me now in both Chrome and Safari.

For #658.
 2021-10-28 22:31:54 -05:00
Adam Wolf 83731f5a68 Tweak JS so Safari can choose admin actions 
I noticed that Safari was submitting both the empty option and the
selected options back to the server.

Digging into it, I was able to get Safari to deselect the ---------
option by using '[selected]' as the selector.

For #658
 2021-10-28 22:22:46 -05:00
Igor Rzegocki 05de1c9fe6
healthcheck endpoint 2021-10-03 19:12:03 +02:00
Igor Rzegocki 95cf85f8cf
Support for Reverse Proxy authentication backends (like authelia) 2021-09-30 17:40:13 +02:00
Bruno Tavares bb2a2e758a
Avoid KeyError on Pocket API parser 
When trying to import my pocket library I got a lot of ` KeyError`  on Python. Pocket API has a few idiosyncrasies, such as sometimes returning the keys on json, sometimes not.

` ` ` sh
archivebox add --parser pocket_api pocket://my_username
` ` ` 

Gave me this errors
` ` ` 
  File "/app/archivebox/parsers/pocket_api.py", line 54, in link_from_article
    title = article['resolved_title'] or article['given_title'] or url
KeyError: 'resolved_title'
` ` ` 

This commit are the patches I've changed to successfully import my library
 2021-09-07 21:53:36 -03:00
Ross Williams f6cf35a45d Fix Pinboard RSS parsing valid links as None 
`item.find(p)` returns either an `ElementTree.Element` or `None`.  The
[lambda on line 24][lambda] coerces the return value to a bool, which is
`False` if the `<link>` element has no children (see
[`ElementTree.py` line 207][etbooldef]), so the lambda returns `None`.

Further, returning a `Link` with `url=None` violates
[an assertion in `index/schema.py`][assertion], which crashes
the `archivebox add` command.

[lambda]: 3d54b1321b/archivebox/parsers/pinboard_rss.py (L24)
[etbooldef]: 3d8993a744/Lib/xml/etree/ElementTree.py (L207)
[assertion]: 3d54b1321b/archivebox/index/schema.py (L165)
 2021-08-04 10:13:37 -04:00
Inndy 0e81a0722e Discard Referer header from iframe and link to original URL 2021-07-19 21:48:01 +08:00
Nick Sweeting 5a2c78e14b add proper support for URL_WHITELIST instead of using negation regexes 2021-07-06 23:42:00 -04:00
Nick Sweeting e4974d3536 support negation patterns by checking both re.search and re.match 2021-07-06 23:17:05 -04:00
TJ Horner cdcfb7fa44 Exempt /add route from CSRF 2021-07-01 20:55:51 -04:00
Nick Sweeting e0a2b2e252 ominous warnings 2021-06-01 03:03:42 -04:00
Nick Sweeting aa53fe653c fix use of uneeded perms arg 2021-06-01 02:58:36 -04:00
Nick Sweeting c2d1a57581 fix umask dir permissions 2021-06-01 00:50:18 -04:00
Nick Sweeting 4715ace7dd ignore BaseException lgtm errors 2021-05-31 20:59:05 -04:00
Nick Sweeting 9f1470cf03 fix output permissions tests 2021-05-31 20:57:46 -04:00
Nick Sweeting 8230f88d80 change default OUTPUT_PERMISSIONS to disallow execution except on dirs 2021-05-31 19:31:51 -04:00
Nick Sweeting 1112526543 add option ENFORCE_ATOMIC_WRITES to allow disabling forced FSYNC writes on network drives 2021-05-31 19:31:51 -04:00
Nick Sweeting 49faec8f6d
add no-zygote and single-process args to try and prevent orphan chrome processes after exit 2021-05-13 05:04:23 -04:00
Nick Sweeting eb4d3bca9d
Update readability.py 2021-05-13 00:13:32 -04:00
Nick Sweeting c3d009e4ec fix python file execution checking 2021-04-24 04:43:02 -04:00
Nick Sweeting 79e19ecd47 shield pwd import for windows 2021-04-24 03:51:38 -04:00
Nick Sweeting 3db77fd1a2 fix lint errors 2021-04-24 03:49:01 -04:00
Nick Sweeting 3992e0fee3 auto prepend python binary to args when running system command with python file as first argument 2021-04-24 03:29:22 -04:00
Nick Sweeting fa84f749ff run archivebox version using python binary 2021-04-24 03:29:22 -04:00
Nick Sweeting 226e26852c dont try to autodetect whether node, chrome, etc are needed during setup 2021-04-24 03:29:22 -04:00
Nick Sweeting f129b9f443 allow executing main 2021-04-24 03:11:06 -04:00
Nick Sweeting 208f866fc4 fix user detection on freebsd always returning root 2021-04-24 02:52:09 -04:00
Nick Sweeting 06f6084e3b ignore stdin when passed instead of throwing an error 2021-04-24 00:09:52 -04:00
Nick Sweeting eb80dc26a1 cleanup config files 2021-04-23 22:58:44 -04:00
Nick Sweeting 4f91af0ce9 TODO create admin user from env vars 2021-04-23 15:50:34 -04:00
Nick Sweeting 9563e74f7c enforce minimum django version 2021-04-23 15:02:26 -04:00
Kunwoo Ahn b6ee7d291e
fix bookmarklet script 
Fix '&' bug in parameter
 2021-04-16 21:24:51 +09:00
Nick Sweeting 63552be0e3 whoops 2021-04-13 04:03:47 -04:00
Nick Sweeting 7d1f8cb496 no longer recommend snap chromium, just use playwright 2021-04-13 03:59:17 -04:00
Nick Sweeting 2c6f0a96bf fix extra arg 2021-04-13 02:21:51 -04:00
Nick Sweeting 54cab113f3 allow static_index.json in dir root from old static exports 2021-04-12 17:06:32 -04:00
Nick Sweeting 50b341baab bail out if old index.json is found during init but doesnt contain links 2021-04-12 16:51:45 -04:00
Nick Sweeting 217952c8d0 sigh linters 2021-04-11 12:51:50 -04:00
Nick Sweeting bfefe042bf fix noqa 2021-04-11 03:57:04 -04:00
Nick Sweeting 335e4658c8 keep grid header on one line 2021-04-11 00:49:33 -04:00
Nick Sweeting 52429b8d15 dont prevent search when backend throws an exception or times out 2021-04-10 08:18:13 -04:00
Nick Sweeting 9f05cf8283 virtual-time-budget doesnt work with some chrome stuff 2021-04-10 08:04:59 -04:00
Nick Sweeting 62078a77f8 show run duration after each archived link in cli output 2021-04-10 07:52:01 -04:00
Nick Sweeting 6a6ef63775 enforce utf8 on more file reads 2021-04-10 07:22:32 -04:00
Nick Sweeting 193df5c8d3 add video subtitles and description to full-text index 2021-04-10 07:22:20 -04:00
Nick Sweeting 1e3c1e3c83 also write autogenerated subtitles 2021-04-10 06:56:19 -04:00
Nick Sweeting e7473c4e3e add some more padding to public index columns 2021-04-10 06:38:33 -04:00
Nick Sweeting 6949803395 enforce new models to use uuid keys 2021-04-10 06:32:45 -04:00