1
0
Fork 0
mirror of synced 2024-06-22 16:10:54 +12:00
Commit graph

124 commits

Author SHA1 Message Date
Nick Sweeting 62078a77f8 show run duration after each archived link in cli output 2021-04-10 07:52:01 -04:00
Nick Sweeting 193df5c8d3 add video subtitles and description to full-text index 2021-04-10 07:22:20 -04:00
Nick Sweeting a9986f1f05 add timezone support, tons of CSS and layout improvements, more detailed snapshot admin form info, ability to sort by recently updated, better grid view styling, better table layouts, better dark mode support 2021-04-10 04:21:36 -04:00
Nick Sweeting bd6d9c165b enforce utf8 on literally all file operations because windows sucks 2021-03-27 01:16:29 -04:00
Nick Sweeting 084cf7ff51 add more explanation about snapshot.save timestamp bump 2021-02-17 13:34:46 -05:00
Nick Sweeting acb932ba12 improve readability and mercury error handling and fix output path to be relative 2021-02-16 15:53:11 -05:00
Nick Sweeting c95698e608 bump Snapshot.updated time after each extractor, change extractor order 2021-02-16 15:52:18 -05:00
Nick Sweeting d0f8a5e710 change mercury atomic_write output order 2021-02-16 06:19:16 -05:00
Nick Sweeting 7d0f5653c3 fix lgtm alerts 2021-02-01 02:27:24 -05:00
Nick Sweeting 04c951cdd5 fix alerts 2021-02-01 02:22:02 -05:00
Nick Sweeting 846c966c4d use globbing to find wget output path 2021-01-30 22:02:39 -05:00
Nick Sweeting e6fa16e13a only chmod wget output if it exists 2021-01-30 22:02:11 -05:00
Nick Sweeting 385daf9af8 save the url as title for staticfiles or non html files 2021-01-30 22:01:49 -05:00
Nick Sweeting b9b1c3d9e8 fix singlefile output path not relative 2021-01-30 20:44:49 -05:00
Nick Sweeting d6de04a83a fix lgtm errors 2021-01-30 06:07:35 -05:00
Nick Sweeting c2aaa41c76 fix missing str path 2021-01-30 01:25:08 -05:00
Nick Sweeting 15e58bd366 fix using os.path calls on pathlib paths 2021-01-27 11:27:40 -05:00
Nick Sweeting 9764a8ed9b check for non html files from wget 2021-01-25 18:15:16 -05:00
Dan Arnfield 5420903102 Refactor should_save_extractor methods to accept overwrite parameter 2021-01-21 15:56:32 -06:00
Nick Sweeting ef7711ffa0 fix cookies file arg is path 2021-01-20 19:13:53 -05:00
Cristian 6031ffa3b2 fix: Mercury extractor error was incorrectly initialized 2021-01-07 09:22:46 -05:00
Cristian e9e4adfc34 fix: wget_output_path failing on some extractors. Add a new condition 2021-01-07 09:07:29 -05:00
Cristian 81d766aba1 refactor: Remove setup_django from title.py 2020-12-11 16:03:50 -05:00
Cristian 275ad22db7 refactor: Remove skip_index from archive related functions 2020-12-08 18:42:25 -05:00
Cristian f6c73f9aeb fix: Issue with oneshot command 2020-12-08 18:42:25 -05:00
JDC 7903db6dfb Add ArchiveResult Manager and sorted indexable filter 2020-12-06 01:13:39 +02:00
JDC b1f70b2197 Initial implementation 2020-12-06 01:12:45 +02:00
Cristian 33182fd53c fix: Add missing assignation 2020-11-04 15:07:45 -05:00
Cristian d064a3eeff fix: Handle case when update tries to re-add a link that is not in the sql index 2020-11-04 15:02:54 -05:00
Cristian f292cface2 fix: Add condition for oneshot when archiving links 2020-11-04 14:40:44 -05:00
Cristian 4484491fb7 feat: Create ArchiveResult after finishing an extractor process 2020-11-04 11:22:55 -05:00
Cristian ac0ec160d1 lint: Fix warnings in master branch 2020-11-02 08:51:48 -05:00
Nick Sweeting ac9e0e356d config fixes 2020-10-31 07:57:11 -04:00
Nick Sweeting 18355dc2c6 clean up config loading in settings and config file layout 2020-10-31 03:08:03 -04:00
Cristian e7e33ea7a5 tests: Add tests for several different ways to extract the title 2020-10-30 08:04:26 -05:00
Nick Sweeting f727ece7b3 add regex fallback back to title parser 2020-10-30 04:57:31 -04:00
Nick Sweeting 79bef1384e
Merge pull request #493 from ttimasdf/feat-ogtitle
Feature: add og:title metadata as alternative title
2020-10-30 04:51:14 -04:00
Cristian c12fe0e3d7 feat: Use CURL_ARGS on title extractor 2020-10-22 08:46:16 -05:00
Cristian 563d0f94ec feat: Use CURL_ARGS in favicon extractor 2020-10-22 08:46:16 -05:00
Cristian 2e1cdca789 feat: Use CURL_ARGS on header extractor 2020-10-22 08:46:16 -05:00
Cristian 972d57bd08 feat: Add CURL_ARGS to control curl arguments 2020-10-22 08:46:16 -05:00
Cristian 24e7a74855 feat: Add WGET_ARGS to control wget arguments 2020-10-22 08:46:16 -05:00
Cristian bc02e0ffe3 feat: Add config for youtubedl (YOUTUBEDL_ARGS) 2020-10-22 08:46:16 -05:00
Angel Rey ce71747538 replaced os.path in init extractors 2020-10-02 15:46:39 -05:00
Angel Rey 3fb410a604 Replaced os.path in favicon.py 2020-10-02 15:46:39 -05:00
ttimasdf eda3836dee feat: add og:title metadata as alternative title 2020-09-27 12:54:52 +08:00
Cristian abde871a3c fix: Wget absolute path generating issues 2020-09-25 08:24:06 -05:00
Cristian 7d3767b882 fix: oneshot command not running extractors 2020-09-24 12:56:16 -05:00
Cristian 62ed11a5ca fix: Improve headers handling 2020-09-24 12:55:51 -05:00
Angel Rey a40af98ced removed static file check 2020-09-24 12:55:51 -05:00