CodeMirror/bulk-downloader-for-reddit
1
0
Fork 0
mirror of synced 2024-05-19 19:52:41 +12:00
Code Issues Releases Wiki Activity
1137 commits 2 branches 45 tags 15 MiB
Commit graph

235 commits

Author SHA1 Message Date
OMEGA_RAZER 0a586425d0
Merge branch 'aliparlakci:development' into DelayForReddit 2022-11-11 17:38:16 -05:00
Serene 25fdd28037
Merge pull request #692 from OMEGARAZER/hardlink_to 2022-11-09 09:55:54 +10:00
OMEGARAZER 3c7f85725e
Narrow except 
Narrow except to AttributeError
 2022-11-08 12:06:20 -05:00
OMEGARAZER dfc21295e3
Add Delay for Reddit support 
Adds support for delayforreddit.com non-direct links.
 2022-11-05 10:51:33 -04:00
Soulsuck24 0e90a2e900
Switch Redgifs to temporary tokens 
Initial switch to temporary tokens for Redgifs. Gets a new auth token for every API request.
 2022-10-24 12:45:26 -04:00
Soulsuck24 df30a3a3ac
Temp fix for Redgifs 
TEMPORARY FIX.

I can't stress enough this is is temporary and will likely stop working at some point. It works for now though.
 2022-10-18 15:38:53 -04:00
OMEGARAZER dc5a9ef497
link_to depreciation coverage 
Futureproof for link_to depreciation.
https://bugs.python.org/issue39950
 2022-10-14 18:15:49 -04:00
OMEGARAZER 02b6e66941
Imgur edge case coverage 
Covers edge case of additional arguments on extension.

Also removed duplicate or redundant tests.
 2022-09-28 00:55:10 -04:00
Serene-Arc 1dff7500e7 Remove duplicate entries 2022-09-20 17:33:44 +10:00
Serene 06816098dc
Merge branch 'development' into imgur 2022-09-20 17:28:48 +10:00
SoulSuck24 106d7596b1 Imgur updates 
Update Imgur logic to cover malformed links that cause a redirect leading to the html of the page being saved as an image.
 2022-09-18 23:27:17 -04:00
SoulSuck24 7bd957aafa Redo edge case coverage for Redgifs 
Cover edge cases that shouldn't ever happen but probably will sometime.
 2022-09-18 14:32:12 -04:00
SoulSuck24 d4f7deaa68 Revert "Edge case coverage" 
This reverts commit 2f2b5b749c.
 2022-09-18 14:30:43 -04:00
SoulSuck24 2f2b5b749c Edge case coverage 
Cover edge cases that shouldn't ever happen but probably will sometime.

Also included Imgur changes to cover similar situations of malformed/redirected links.
 2022-09-18 13:24:42 -04:00
SoulSuck24 95749584ec Redgifs fixed? 
If this doesn't work then I give up...
 2022-09-16 20:41:17 -04:00
SoulSuck24 0a9ecac410 Redgif image fixes 2022-09-16 14:47:55 -04:00
SoulSuck24 e0a36f4eab Re-fix Redgifs 
API seems to return incorrect signature value when sending header. Other fixes seems to have worked temporarily but have stopped working so they're removed.
 2022-09-12 22:26:02 -04:00
Serene-Arc d60b4e7fdd Fix Redgifs module 2022-09-01 11:19:07 +10:00
Serene-Arc 2bbf1b644e Change logging message 2022-07-23 17:24:05 +10:00
Serene-Arc f22a8aec4d Fix line length 2022-07-23 17:24:05 +10:00
Jacob Chapman 5d76fcd5aa Update downloader.py 2022-07-23 17:24:05 +10:00
Jacob Chapman 7eb2ab6d7d Update configuration.py 2022-07-23 17:24:05 +10:00
Jacob Chapman 9545407896 Update __main__.py 2022-07-23 17:24:05 +10:00
Jacob Chapman 4fc0d5dc1d Add score filtering 2022-07-23 17:24:05 +10:00
Serene-Arc af3f98f59c Change logger message level 2022-07-22 15:45:38 +10:00
Serene-Arc 23e20e6ddc Rename variable 2022-07-22 15:45:09 +10:00
Serene-Arc cb3415c62f Extract YAML function 2022-07-22 15:44:33 +10:00
Piotr Migdal 5f443fddff a better check for opts 2022-07-22 15:38:46 +10:00
Piotr Migdal 395bf9180a explicit warnings for non-exisitng args 2022-07-22 15:38:46 +10:00
Piotr Migdal 798ed728f5 yaml for options 2022-07-22 15:38:46 +10:00
Serene 8ab13b4480
Merge pull request #633 from chapmanjacobd/patch-3 
fix: Redirect to /subreddits/search
 2022-07-17 11:17:52 +10:00
Serene-Arc 9277903308 Base VReddit class off of Youtube class 2022-07-15 15:18:28 +10:00
Serene-Arc 7d4eb47643 Rename class 2022-07-15 15:18:28 +10:00
Serene-Arc 8c59329ffa Add exclusion options to archiver 2022-07-15 15:18:28 +10:00
Serene-Arc 9f3dcece4d Strip any newline characters from names 2022-07-15 15:18:28 +10:00
Serene-Arc 12982c00cd Switch redgifs to dynamic file extensions 2022-07-15 15:18:28 +10:00
Serene-Arc f49a1d7a2d Fix gfycat after redgifs changes 2022-07-15 15:18:28 +10:00
Serene-Arc e8d767050f Add file scheme naming for archiver 2022-07-15 15:18:28 +10:00
Serene-Arc 90a2eac90d Add support for Redgifs images and galleries 2022-07-15 15:18:28 +10:00
Serene-Arc a620ae91a1 Add --subscribed option 2022-07-15 15:18:28 +10:00
Jacob Chapman 1ad2b68e03
fix: Redirect to /subreddits/search 
```
  File "/home/xk/github/o/bulk-downloader-for-reddit/bdfr/connector.py", line 413, in check_subreddit_status
    assert subreddit.id
  File "/home/xk/.local/share/virtualenvs/bulk-downloader-for-reddit-dCAFmVJi/lib/python3.10/site-packages/praw/models/reddit/base.py", line 34, in __getattr__
    self._fetch()
  File "/home/xk/.local/share/virtualenvs/bulk-downloader-for-reddit-dCAFmVJi/lib/python3.10/site-packages/praw/models/reddit/subreddit.py", line 584, in _fetch
    data = self._fetch_data()
  File "/home/xk/.local/share/virtualenvs/bulk-downloader-for-reddit-dCAFmVJi/lib/python3.10/site-packages/praw/models/reddit/subreddit.py", line 581, in _fetch_data
    return self._reddit.request("GET", path, params)
  File "/home/xk/.local/share/virtualenvs/bulk-downloader-for-reddit-dCAFmVJi/lib/python3.10/site-packages/praw/reddit.py", line 885, in request
    return self._core.request(
  File "/home/xk/.local/share/virtualenvs/bulk-downloader-for-reddit-dCAFmVJi/lib/python3.10/site-packages/prawcore/sessions.py", line 330, in request
    return self._request_with_retries(
  File "/home/xk/.local/share/virtualenvs/bulk-downloader-for-reddit-dCAFmVJi/lib/python3.10/site-packages/prawcore/sessions.py", line 266, in _request_with_retries
    raise self.STATUS_EXCEPTIONS[response.status_code](response)
prawcore.exceptions.Redirect: Redirect to /subreddits/search
```
 2022-04-28 19:44:17 -05:00
BlipRanger d64acc25f5 Add tests, fix style. 2022-04-25 12:53:59 -04:00
BlipRanger dbd0c6cd42 Add support for v.reddit links. 2022-04-25 12:09:09 -04:00
Serene-Arc 806bd76f87 Strip any newline characters from names 2022-03-25 10:50:52 +10:00
Serene-Arc 06988c40b3 Switch redgifs to dynamic file extensions 2022-02-20 15:48:02 +10:00
Serene-Arc 7645319510 Fix gfycat after redgifs changes 2022-02-18 12:49:46 +10:00
Serene-Arc 6e0c642652 Add file scheme naming for archiver 2022-02-18 12:30:38 +10:00
Serene-Arc 9deef63fdd Add support for Redgifs images and galleries 2022-02-18 12:04:37 +10:00
Serene-Arc 0177b434c2 Add --subscribed option 2022-02-18 10:21:52 +10:00
Serene-Arc af0a545c16 Catch additional error in Gallery 2021-12-20 20:43:09 +10:00
Serene-Arc 36ff95de6b Add Patreon image support 2021-12-19 13:44:24 +10:00
dbanon87 1530456cf7
Update downloader.py 2021-11-29 09:23:04 -05:00
dbanon87 9ccc9e6863
Update archiver.py 2021-11-29 09:22:21 -05:00
Serene-Arc f670b347ae Add integration test for archiver option 2021-11-24 12:49:11 +10:00
Serene-Arc d0d72c8229 Add integration test for downloader option 2021-11-24 12:49:11 +10:00
Jay R. Wren 2b50ee0724 add test. fix typos. 2021-11-24 12:49:11 +10:00
Jay R. Wren dd8d74ee25 Add --ignore to ignore user 2021-11-24 12:49:11 +10:00
Serene-Arc 8925643331 Rename module to reflect backend change 2021-11-24 10:40:18 +10:00
Serene-Arc 2dd446a402 Fix max path length calculations 2021-11-22 14:37:21 +10:00
Serene-Arc 17939fe47c Fix bug with youtube class and children 2021-11-22 14:37:21 +10:00
Serene-Arc 53562f4873 Fix regex 2021-11-16 17:05:46 +03:00
OMEGARAZER f05e909008 Stop videos from being downloaded as images 
Erroneous .gifv extensions such as .giff or .gift resolve to a static image and are downloaded by the direct downloader. (ex: https://i.imgur.com/OGeVuAe.giff  )
 2021-11-16 17:05:46 +03:00
Serene-Arc 4be0f5ec19 Add more tests for file length checking 2021-11-15 11:57:54 +10:00
Serene-Arc 801784c46d Fix a crash when downloading a disabled pornhub video 2021-11-05 13:23:55 +10:00
Serene-Arc e493ab048a Fix bug with period not separating file extension 2021-11-05 12:47:46 +10:00
Serene-Arc c6c6002ab2 Update Erome module 2021-10-02 17:50:20 +10:00
Serene-Arc 9b23f273fc Separate function out 2021-10-02 17:50:20 +10:00
Serene-Arc eeb2054606 Switch to yt-dlp 2021-10-02 17:50:20 +10:00
Serene e004ccd148
Merge pull request #521 from Serene-Arc/bug_fix_518 
Fix bug with different Vidble links
 2021-09-14 13:48:26 +10:00
Serene-Arc 80baab8de7 Fix bug with different Vidble links 2021-09-14 13:47:46 +10:00
Eli Lipsitz 33312687ac imgur: download videos as mp4 instead of gif 
Some imgur URLS have the extension ".gifv" and show up as a gif,
even though they're actually supposed to be mp4 videos. Imgur
serves all videos/gifs as both .gif and .mp4. The image dict has
a key "prefer_video" to distinguish the two. This commit
overrides the .gif extension if "prefer_video" is true to ensure
we download the submission as originally intended.
 2021-09-12 17:30:25 -05:00
Ali Parlakçı 483f179ccc
Merge pull request #482 from Serene-Arc/enhancement_481 
Add ability to read IDs from files
 2021-09-12 20:07:17 +03:00
Serene-Arc aee6f4add9 Add Vidble to download factory 2021-09-11 12:15:35 +10:00
Serene-Arc 940d646d30 Add Vidble module 2021-09-11 12:13:21 +10:00
Serene 3040a35306
Merge pull request #512 from Serene-Arc/bug_fix_510 2021-09-03 19:30:30 +10:00
Serene-Arc 87f283cc98 Fix backup config location 2021-09-03 19:24:28 +10:00
Serene-Arc 7bca303b1b Add in downloader parameters 2021-07-29 19:10:10 +10:00
Serene-Arc dbe8733fd4 Refactor method to remove max wait time 2021-07-27 14:02:30 +10:00
Serene-Arc 3cdae99490 Implement callbacks for downloading 2021-07-27 13:39:49 +10:00
Serene-Arc 1a4ff07f78 Add ability to read IDs from files 2021-07-21 17:32:38 +10:00
Serene-Arc 77aaee96f3 Fix bug with deleted galleries 2021-07-19 18:44:54 +10:00
Serene-Arc 2f8ca766c6 Update regex 2021-07-04 11:00:02 +10:00
Serene-Arc d03a5e556e Stop writing new value to config 2021-07-04 10:59:35 +10:00
Serene-Arc 7f1c929a08 Add fallback scope 2021-07-03 13:54:26 +10:00
Serene-Arc d5ef991b3a Catch additional error in galleries 2021-07-02 15:11:09 +10:00
Serene-Arc aa55a92791 Remove unused local variables 2021-07-02 14:58:56 +10:00
Serene-Arc 6efcf1ce7e Remove unused imports 2021-07-02 14:58:20 +10:00
Serene-Arc 1319eeb6da Fix error with crossposted Reddit galleries 2021-07-02 14:53:02 +10:00
Serene-Arc 8db9d0bcc4 Add test for unauthenticated instances 2021-07-02 14:29:39 +10:00
Serene-Arc bd34c37052 Add exception for special friends subreddit 2021-07-02 14:09:44 +10:00
Serene-Arc edfeb653a4 Record user flair in comment archive entries 2021-07-02 14:01:24 +10:00
Serene-Arc d53b3b7274 Update gallery code to work with NSFW galleries 2021-06-25 17:52:11 +10:00
Serene-Arc e8998da2f0 Catch some Imgur errors with weird links 2021-06-25 17:52:11 +10:00
Serene-Arc 1a52dfdcbc Add PornHub module 2021-06-25 17:47:49 +10:00
Serene-Arc e5be624f1e Check submission URL against filter before factory 2021-06-23 14:30:39 +10:00
Ali Parlakçı 71930e06a8
Merge pull request #457 from Serene-Arc/enhancement_322 
Add option for archiver full context
 2021-06-19 13:56:04 +03:00
Serene-Arc 7c27b7bf12 Update logging message 2021-06-13 09:49:42 +10:00
Serene-Arc c5c010bce0 Rename option 2021-06-12 10:35:31 +10:00
Serene-Arc 6eeadc8821 Add option for archiver full context 2021-06-11 15:31:11 +10:00
Serene-Arc 8ba2d0bb55 Add missing return statement 2021-06-10 18:59:22 +10:00