Nick Sweeting
|
12f3977da5
|
fix source filenames missing domain
|
2020-08-18 04:39:29 -04:00 |
|
|
Nick Sweeting
|
ec1cce283e
|
include failed link updates in total count stats
|
2020-08-18 04:39:19 -04:00 |
|
|
Nick Sweeting
|
6e142e3531
|
match existing snapshots on url only, ignoring timestamp
|
2020-08-18 04:39:04 -04:00 |
|
|
Nick Sweeting
|
f371032b71
|
show warning when killing archivebox during index writing
|
2020-08-18 04:38:29 -04:00 |
|
|
Nick Sweeting
|
92de20af15
|
better detect missing dependencies on startup
|
2020-08-18 04:38:13 -04:00 |
|
|
Nick Sweeting
|
b681a477ae
|
add overwrite flag to add command to force re-archiving
|
2020-08-18 04:37:54 -04:00 |
|
|
Nick Sweeting
|
da671532a4
|
fix schedule cmd defaulting to day and remove week
|
2020-08-18 04:35:59 -04:00 |
|
|
Nick Sweeting
|
84a8b0d713
|
hide progress bars on mac
|
2020-08-18 03:27:52 -04:00 |
|
|
Nick Sweeting
|
29c8da83d6
|
0.4.16 release
|
2020-08-18 02:08:52 -04:00 |
|
|
Nick Sweeting
|
7c16944a44
|
Merge pull request #446 from cdvv7788/hotfix/#445
|
2020-08-18 02:06:32 -04:00 |
|
|
Nick Sweeting
|
7638dc45ea
|
0.4.15 release
|
2020-08-18 01:59:50 -04:00 |
|
|
Nick Sweeting
|
235eb20dbd
|
support cron in docker
|
2020-08-18 01:59:04 -04:00 |
|
|
Nick Sweeting
|
494be09bc2
|
add depth flag to schedule cmd
|
2020-08-18 01:58:54 -04:00 |
|
Cristian
|
05c71fc302
|
fix: Organize readability extractor so a timeout does not break the whole process
|
2020-08-17 08:34:40 -05:00 |
|
|
Nick Sweeting
|
225b63b732
|
skip invalid urls at all stages
|
2020-08-17 03:12:17 -04:00 |
|
|
Nick Sweeting
|
429f39dec1
|
0.4.14 release
|
2020-08-14 13:13:50 -04:00 |
|
|
Nick Sweeting
|
58e928520a
|
tweak log output for skipped methods
|
2020-08-14 13:12:50 -04:00 |
|
|
Nick Sweeting
|
03b73bfe77
|
Update archivebox/extractors/readability.py
|
2020-08-14 12:55:22 -04:00 |
|
|
Nick Sweeting
|
050b717bb9
|
Merge branch 'master' into readability-extractor
|
2020-08-14 12:35:35 -04:00 |
|
|
Nick Sweeting
|
0ef2b17678
|
only show data locations in version output when in a data dir
|
2020-08-13 23:21:57 -04:00 |
|
|
Nick Sweeting
|
a0901ba474
|
use BIND_ADDR config default for runserver
|
2020-08-13 23:21:37 -04:00 |
|
|
Cristian
|
b7aa3df8d2
|
feat: Disable singlefile and readability by default
|
2020-08-12 14:42:21 -05:00 |
|
|
Cristian
|
eb3528fa9f
|
feat: Add readability output to legacy index.html
|
2020-08-11 12:14:13 -05:00 |
|
|
Cristian
|
5dc7e63792
|
feat: Update dockerfile to support readability
|
2020-08-11 11:52:43 -05:00 |
|
|
Cristian
|
2a68af1b94
|
tests: Add readability tests
|
2020-08-11 11:15:15 -05:00 |
|
|
Cristian
|
8aa7b34de7
|
tests: Add readability to ignored methods in tests
|
2020-08-11 08:58:49 -05:00 |
|
|
Cristian
|
dc87d8b68c
|
tests: Update failing tests
|
2020-08-11 08:48:13 -05:00 |
|
|
Cristian
|
0ec747f64e
|
feat: Look in wget, singlefile or dom outputs before attempting to download the information again
|
2020-08-11 08:37:12 -05:00 |
|
|
Cristian
|
a14762640e
|
feat: Avoid running readability when the target is a file
|
2020-08-11 08:37:12 -05:00 |
|
|
Cristian
|
61e08a7c43
|
docs: Update docs link
|
2020-08-11 08:37:12 -05:00 |
|
|
Cristian
|
b33c66a9f7
|
feat: Split output of readability into multiple files
|
2020-08-11 08:37:12 -05:00 |
|
|
Cristian
|
7e2b249388
|
feat: Initial version of readability extractor
|
2020-08-11 08:37:12 -05:00 |
|
apkallum
|
50069d1eb3
|
set tz variable globally as UTC
|
2020-08-10 23:21:02 -04:00 |
|
|
apkallum
|
e9bd0b122e
|
fix: utc timing for initial command log as well
|
2020-08-10 19:17:17 -04:00 |
|
|
Nick Sweeting
|
cd09d1b077
|
0.4.13 release
|
2020-08-10 14:39:06 -04:00 |
|
|
Nick Sweeting
|
fcbc61917e
|
0.4.12 release
|
2020-08-10 14:26:32 -04:00 |
|
|
Nick Sweeting
|
33ab7fd4ec
|
autodetect when running inside docker and provide hints
|
2020-08-10 14:18:04 -04:00 |
|
|
Nick Sweeting
|
f24cb3dcbe
|
add docker help text
|
2020-08-10 13:42:31 -04:00 |
|
|
Nick Sweeting
|
430be7bc68
|
add missing staticfile check to singlefile
|
2020-08-10 13:42:20 -04:00 |
|
|
Cristian
|
76846d18a0
|
docs: Improve message for missing singlefile binary
|
2020-08-10 09:00:10 -05:00 |
|
|
Cristian
|
e358634f81
|
fix: Add missing configuration that breaks on edge case where only single file is being used
|
2020-08-08 09:12:14 -05:00 |
|
|
Nick Sweeting
|
87ba82ad39
|
0.4.11 release
|
2020-08-06 23:10:59 -04:00 |
|
|
Nick Sweeting
|
5b8abb2dce
|
bump version
|
2020-08-06 23:10:37 -04:00 |
|
|
Nick Sweeting
|
19aa5c3e94
|
fix SAVE_SINGLEFILE setting to depend on chrome
|
2020-08-06 23:07:25 -04:00 |
|
|
Cristian
|
3c5c6a689e
|
fix: Add missing configuration variable to be able to disable singlefile
|
2020-08-04 07:35:58 -05:00 |
|
|
Cristian
|
06d0e9de6c
|
feat: Add support for singlefile in docker
|
2020-08-03 13:23:05 -05:00 |
|
|
Nick Sweeting
|
5b6eb5e4ad
|
make filenames consistent with program name
|
2020-08-03 13:23:05 -05:00 |
|
|
Cristian
|
91f63635e8
|
feat: Add singlefile in a couple more places
|
2020-08-03 13:22:06 -05:00 |
|
|
Cristian
|
b325c0dd9f
|
feat: Add singlefile to latest outputs
|
2020-08-03 13:22:06 -05:00 |
|
|
Cristian
|
a40e337280
|
feat: Add link to admin list of files
|
2020-08-03 13:22:06 -05:00 |
|
|
Cristian
|
42b0c80465
|
feat: Add singlefile to link_details
|
2020-08-03 13:22:06 -05:00 |
|
|
Nick Sweeting
|
3d22da39fe
|
Update archivebox/config/__init__.py
|
2020-08-03 13:22:06 -05:00 |
|
|
Cristian
|
787a5ad43e
|
fix: Commit code review suggestions
|
2020-08-03 13:22:06 -05:00 |
|
|
Cristian
|
853685668c
|
feat: Add initial support for singlefile extractor
|
2020-08-03 13:22:06 -05:00 |
|
|
Nick Sweeting
|
dd916e91d0
|
Merge pull request #396 from cdvv7788/oneshot-command
|
2020-08-01 13:44:51 -04:00 |
|
|
Cristian
|
d0d2991c69
|
fix: Change import that was not working
|
2020-07-31 12:15:00 -05:00 |
|
Cristian Vargas
|
b2a318c5eb
|
fix: Update error message for oneshot command
Co-authored-by: Nick Sweeting <git@sweeting.me>
|
2020-07-31 10:51:54 -05:00 |
|
|
Cristian
|
a8c74730f8
|
docs: Add docstring to oneshot method
|
2020-07-31 10:28:30 -05:00 |
|
|
Cristian
|
e6c571beb2
|
fix: Remove title from extractors for oneshot
|
2020-07-31 10:24:58 -05:00 |
|
|
Cristian
|
8bcb171e74
|
fix: Remove support for multiple urls in oneshot command
|
2020-07-31 09:05:40 -05:00 |
|
|
Nick Sweeting
|
5707ffe657
|
fix old config name FETCH_TITLE
|
2020-07-30 16:55:24 -04:00 |
|
|
Nick Sweeting
|
a160e6bf20
|
fix None canon output to be emptystring
|
2020-07-29 23:54:50 -04:00 |
|
|
Nick Sweeting
|
9dedcdd577
|
remove inaccurate updated ts from main index UI
|
2020-07-29 23:54:50 -04:00 |
|
|
Cristian
|
3afb2401bc
|
fix: Add condition to avoid breaking the add command
|
2020-07-29 11:53:49 -05:00 |
|
|
Cristian
|
c073ea141d
|
feat: Initial oneshot command proposal
|
2020-07-29 11:19:06 -05:00 |
|
|
Nick Sweeting
|
c1f21880f3
|
0.4.9 release
|
2020-07-28 08:25:01 -04:00 |
|
|
Nick Sweeting
|
3c7966c13a
|
dont get bin path when bin is missing
|
2020-07-28 07:20:57 -04:00 |
|
|
Nick Sweeting
|
1b96c582a7
|
fix lint and improve docker-compose instructions
|
2020-07-28 07:18:10 -04:00 |
|
|
Nick Sweeting
|
9248ff5890
|
0.4.8 release
|
2020-07-28 06:52:44 -04:00 |
|
|
Nick Sweeting
|
acc697e73c
|
0.4.7 release
|
2020-07-28 06:51:18 -04:00 |
|
|
Nick Sweeting
|
9806ed8d8c
|
fix circular import
|
2020-07-28 06:50:03 -04:00 |
|
|
Nick Sweeting
|
301e220c53
|
v0.4.6
|
2020-07-28 06:22:24 -04:00 |
|
|
Nick Sweeting
|
b8c93889c1
|
hide prints and tweak url text in titlebar
|
2020-07-28 06:03:52 -04:00 |
|
|
Nick Sweeting
|
b1082cfbaa
|
ui and css improvements
|
2020-07-28 06:00:09 -04:00 |
|
|
Nick Sweeting
|
5a30e03778
|
rearrange tags column and improve files icons
|
2020-07-28 05:59:54 -04:00 |
|
|
Nick Sweeting
|
2e0b751376
|
accept methods argument to filder archive_link
|
2020-07-28 05:58:38 -04:00 |
|
|
Nick Sweeting
|
032c2458de
|
add missing setup_django import
|
2020-07-28 05:58:13 -04:00 |
|
|
Nick Sweeting
|
9e7330cc14
|
add init flag to server and fix SHOW_PROGRESS config being ignored
|
2020-07-28 05:57:34 -04:00 |
|
|
Nick Sweeting
|
55a237a435
|
also set snapshot title inside of fetch_title directly
|
2020-07-28 05:56:34 -04:00 |
|
|
Nick Sweeting
|
273059f054
|
accept gzipped responses when using curl
|
2020-07-28 05:55:54 -04:00 |
|
|
Nick Sweeting
|
af9084ee95
|
update Snapshot.title to latest_title after fetching
|
2020-07-28 05:55:09 -04:00 |
|
|
Nick Sweeting
|
943453a9a8
|
pass overwrite properly
|
2020-07-28 05:54:42 -04:00 |
|
|
Nick Sweeting
|
d6030e15c7
|
allow passing links to remove method
|
2020-07-28 05:52:15 -04:00 |
|
|
Nick Sweeting
|
313fcd0501
|
change defalt date format to ISO
|
2020-07-28 05:51:18 -04:00 |
|
|
Nick Sweeting
|
ece6d43078
|
hide builtin delete button
|
2020-07-28 05:51:02 -04:00 |
|
|
Nick Sweeting
|
d70bb7980e
|
use proper url naming instead of hardcoding paths
|
2020-07-27 23:56:35 -04:00 |
|
|
Nick Sweeting
|
ea1ff7b6bc
|
fix linter
|
2020-07-27 23:34:30 -04:00 |
|
|
Nick Sweeting
|
3aeca0e450
|
fix pending titles and favicons, improve add page, custom admin
|
2020-07-27 23:26:45 -04:00 |
|
|
Nick Sweeting
|
022231b362
|
fix favicon url and show size in separate column
|
2020-07-27 19:30:40 -04:00 |
|
|
Nick Sweeting
|
fd0d0563d1
|
bump version number
|
2020-07-27 18:52:57 -04:00 |
|
|
Nick Sweeting
|
3fe7a9b70c
|
also parse and archive sub-urls in generic_txt input
|
2020-07-27 18:52:57 -04:00 |
|
|
Nick Sweeting
|
6652982856
|
fix crazy progress bar wrappping when shrinking terminal window size
|
2020-07-27 18:52:57 -04:00 |
|
|
Nick Sweeting
|
904f728785
|
fix binary hash func when binary is missing
|
2020-07-27 18:52:57 -04:00 |
|
|
Cristian
|
d04c9b3281
|
fix: if cmd in Link parsing is found to be a string, put it inside a list
|
2020-07-24 14:36:08 -05:00 |
|
|
Nick Sweeting
|
74ad79f1fc
|
Merge pull request #391 from apkallum/origin/archived-item-permissions
ensure correct permissions for archived items
|
2020-07-24 14:33:43 -04:00 |
|
|
Nick Sweeting
|
fa17e20f8e
|
Update archivebox/system.py
|
2020-07-24 14:33:06 -04:00 |
|
|
apkallum
|
9cb0be183b
|
ensure correct permissions for archived items
|
2020-07-24 14:03:12 -04:00 |
|
|
Cristian
|
6006b4f93b
|
refactor: Organize code to remove flake8 issues
|
2020-07-24 12:25:25 -05:00 |
|
|
Cristian
|
82f8f8b661
|
fix: Use config information for path instead of hardcoded values
|
2020-07-24 10:34:47 -05:00 |
|
|
Cristian
|
100fa5d1f5
|
fix: Guess timestamps and add placeholders to support older indices
|
2020-07-24 09:24:52 -05:00 |
|
|
Cristian
|
fe0884f1ec
|
fix: Remove link from sql index on remove command
|
2020-07-23 15:07:00 -05:00 |
|
|
Cristian
|
030013899d
|
feat: change COLOR_DICT to a default dict to prevent future issues
|
2020-07-23 12:02:17 -05:00 |
|
|
Cristian
|
42a549833b
|
fix: Add missing colors to dict
|
2020-07-23 11:47:01 -05:00 |
|
|
Nick Sweeting
|
02a2fefbba
|
Merge pull request #385 from apkallum/origin/output-permissions
|
2020-07-23 11:52:31 -04:00 |
|
|
apkallum
|
b854884c56
|
move umask to init/__config__
|
2020-07-23 11:50:42 -04:00 |
|
|
Cristian Vargas
|
51716bbf74
|
Update warning message on detail index error
Co-authored-by: Nick Sweeting <git@sweeting.me>
|
2020-07-23 10:23:41 -05:00 |
|
|
Cristian
|
5ca7121fd8
|
refactor: Change path calculation to use pathlib in a better way
|
2020-07-23 10:22:36 -05:00 |
|
|
apkallum
|
0ed2a23670
|
ensure correct permissions for output folder
|
2020-07-23 10:28:10 -04:00 |
|
|
Cristian
|
71f5f03a20
|
fix: Add notice for issues with index detail
|
2020-07-22 17:08:32 -05:00 |
|
|
Cristian Vargas
|
e58c3deb05
|
feat: Update path generation in detail index fallback
Co-authored-by: Nick Sweeting <git@sweeting.me>
|
2020-07-22 14:46:03 -05:00 |
|
|
Cristian
|
263eb4e372
|
fix: Change path to use ARCHIVE_DIR_NAME
|
2020-07-22 14:37:10 -05:00 |
|
|
Cristian
|
9815241b78
|
feat: Fallback to link detail when there is an issue loading a link from main index
|
2020-07-22 14:22:00 -05:00 |
|
|
Nick Sweeting
|
0aa3ee06a9
|
Merge pull request #379 from cdvv7788/hotfix/#372-b
#372 Rename logging module to avoid conflicts
|
2020-07-22 12:13:59 -04:00 |
|
|
Cristian
|
a5550b2105
|
fix: Rename logging folder to avoid naming conflicts (and circular import issues)
|
2020-07-22 11:02:13 -05:00 |
|
|
Cristian
|
949f78aa65
|
fix: Use w3lib to improve the encoding extraction
|
2020-07-22 10:24:08 -05:00 |
|
|
Nick Sweeting
|
0965031d8f
|
fix archive_org header rename
|
2020-07-22 01:46:38 -04:00 |
|
|
Nick Sweeting
|
25e0cba0cc
|
check system config later in startup process to allow version to run during docker build
|
2020-07-22 01:31:23 -04:00 |
|
|
Nick Sweeting
|
8cb530230c
|
fix docker SHM limited to 64mb chrome crash
|
2020-07-21 23:39:21 -04:00 |
|
|
Cristian
|
834b33e6a8
|
fix: Re-add typings with conditional import to avoid circular import issue
|
2020-07-20 11:20:08 -05:00 |
|
|
Cristian
|
75e5a6fcdc
|
fix: Add missing change to refactor related to circular imports
|
2020-07-20 09:11:17 -05:00 |
|
|
Cristian
|
53dede8e16
|
fix: Remove imports causing circular import issues
|
2020-07-20 08:39:46 -05:00 |
|
|
Nick Sweeting
|
848977e7be
|
Merge pull request #371 from cdvv7788/circular_import
refactor: Move logging.py to main module to avoid circular import issues
|
2020-07-17 19:27:21 -04:00 |
|
|
Nick Sweeting
|
6d15b5cb42
|
Merge pull request #368 from apkallum/apkallum/date-fix
|
2020-07-17 19:24:47 -04:00 |
|
|
Cristian
|
f4d1b5121e
|
refactor: Move logging.py to main module to avoid circular import issues
|
2020-07-17 18:00:04 -05:00 |
|
|
Cristian
|
23e6803f02
|
fix: Add change to calculate wget folder when there is a port present
|
2020-07-17 16:55:56 -05:00 |
|
|
Apkallum
|
1f91f5b102
|
remove commented lines
|
2020-07-16 19:42:20 -04:00 |
|
|
apkallum
|
b7785c4138
|
use dateparser for parsing, let it handle error
|
2020-07-16 19:38:38 -04:00 |
|
|
Nick Sweeting
|
58ac44c867
|
Merge pull request #365 from cdvv7788/hotfix/#330
fix: htmlencode titles before rendering the static html index and detail
|
2020-07-16 14:45:00 -04:00 |
|
|
Cristian
|
83e5b019e4
|
feat: Add canonical link http header to the static response
|
2020-07-16 12:49:26 -05:00 |
|
|
Cristian
|
f845224d6f
|
fix: htmlencode titles before rendering the static html index and detail
|
2020-07-16 09:20:33 -05:00 |
|
|
apkallum
|
98dda68897
|
fix: timestamp comparison in to_json function
|
2020-07-14 10:26:33 -04:00 |
|
|
Cristian
|
5e2bf73f04
|
fix: Bugs related to add() refactor
|
2020-07-13 14:48:25 -05:00 |
|
|
Nick Sweeting
|
a79dd4685a
|
make snapshots unique again
|
2020-07-13 12:21:52 -04:00 |
|
|
Nick Sweeting
|
ae208435c9
|
fix the add links form
|
2020-07-13 12:21:37 -04:00 |
|
|
Nick Sweeting
|
215d5eae32
|
normal git clone instead of mirror
|
2020-07-13 11:41:37 -04:00 |
|
|
Nick Sweeting
|
b4ce20cbe5
|
write link details json before and after archiving
|
2020-07-13 11:41:27 -04:00 |
|
|
Nick Sweeting
|
d159e674e1
|
write stderr instead of stdout for version info
|
2020-07-13 11:41:18 -04:00 |
|
|
Nick Sweeting
|
4c4b1e6a4b
|
fix link creation
|
2020-07-13 11:33:38 -04:00 |
|
|
Nick Sweeting
|
d3bfa98a91
|
fix depth flag and tweak logging
|
2020-07-13 11:26:34 -04:00 |
|
|
Nick Sweeting
|
354a63ccd4
|
dont dedupe snapshots in sqlite on every run
|
2020-07-13 11:25:43 -04:00 |
|
|
Nick Sweeting
|
dfb83b4f27
|
add AttributeDict
|
2020-07-13 11:24:49 -04:00 |
|
|
Nick Sweeting
|
16f3746712
|
check source dir at the end of checking data dir
|
2020-07-13 11:24:36 -04:00 |
|
|
Nick Sweeting
|
96b1e4a8ec
|
accept local paths as valid link URLs when parsing
|
2020-07-13 11:22:58 -04:00 |
|
|
Nick Sweeting
|
7cbd068c95
|
add flake8
|
2020-07-13 11:22:23 -04:00 |
|
|
Nick Sweeting
|
5b571aa166
|
Merge pull request #356 from cdvv7788/depth-flag
|
2020-07-13 05:05:36 -04:00 |
|
|
Nick Sweeting
|
26e97f242a
|
Merge pull request #360 from apkallum/django
fix legacy index.html
|
2020-07-08 19:51:37 -04:00 |
|
|
Apkallum
|
09b4438c9f
|
fix legacy index.html
|
2020-07-08 17:54:01 -04:00 |
|
|
Cristian
|
d476b13007
|
fix: Add missing permission to add view (post)
|
2020-07-08 14:46:31 -05:00 |
|
|
Cristian
|
4ebf929606
|
refactor: Change wording on CLI help
|
2020-07-08 08:30:07 -05:00 |
|
|
Cristian
|
f12bfeb322
|
refactor: Change add() to receive url and depth instead of import_str and import_path
|
2020-07-08 08:17:47 -05:00 |
|