CodeMirror/bulk-downloader-for-reddit
1
0
Fork 0
mirror of synced 2024-06-29 11:30:30 +12:00
Code Issues Releases Wiki Activity
1090 commits 2 branches 45 tags 14 MiB
Commit graph

47 commits

Author SHA1 Message Date
OMEGARAZER 83f45e7f60
Standardize shebang and coding declaration 
Standardizes shebang and coding declarations.

Coding matches what's used by install tools such as pip(x).

Removes a few init files that were not needed.
 2022-12-19 18:32:37 -05:00
OMEGARAZER 3aa740e979
Add soft fail on 5xx Prawcore errors. 2022-12-10 12:36:54 -05:00
Serene-Arc 0873a4a2b2 Format according to the black standard 2022-12-03 15:11:17 +10:00
OMEGARAZER 831f49daa6
Refurb linting 
Lint with [refurb](https://github.com/dosisod/refurb) using `--disable 126 --python-version 3.9`
 2022-11-30 17:19:02 -05:00
Serene-Arc 5341d6f12c Add catch for per-submission praw errors 2022-11-20 18:54:56 +10:00
OMEGARAZER 3c7f85725e
Narrow except 
Narrow except to AttributeError
 2022-11-08 12:06:20 -05:00
OMEGARAZER dc5a9ef497
link_to depreciation coverage 
Futureproof for link_to depreciation.
https://bugs.python.org/issue39950
 2022-10-14 18:15:49 -04:00
Serene-Arc 2bbf1b644e Change logging message 2022-07-23 17:24:05 +10:00
Serene-Arc f22a8aec4d Fix line length 2022-07-23 17:24:05 +10:00
Jacob Chapman 5d76fcd5aa Update downloader.py 2022-07-23 17:24:05 +10:00
Jacob Chapman 4fc0d5dc1d Add score filtering 2022-07-23 17:24:05 +10:00
dbanon87 1530456cf7
Update downloader.py 2021-11-29 09:23:04 -05:00
Serene-Arc d0d72c8229 Add integration test for downloader option 2021-11-24 12:49:11 +10:00
Jay R. Wren 2b50ee0724 add test. fix typos. 2021-11-24 12:49:11 +10:00
Serene-Arc 7bca303b1b Add in downloader parameters 2021-07-29 19:10:10 +10:00
Serene-Arc 3cdae99490 Implement callbacks for downloading 2021-07-27 13:39:49 +10:00
Serene-Arc e5be624f1e Check submission URL against filter before factory 2021-06-23 14:30:39 +10:00
Serene-Arc 7c27b7bf12 Update logging message 2021-06-13 09:49:42 +10:00
Serene-Arc 8ba2d0bb55 Add missing return statement 2021-06-10 18:59:22 +10:00
Serene 6dcef83666
Add ability to disable modules (#434) 
* Fix test name to match standard

* Rename file

* Add ability to disable modules

* Update README

* Fix missing comma

* Fix more missing commas. sigh...

Co-authored-by: Ali Parlakçı <parlakciali@gmail.com>
 2021-06-06 13:47:56 +03:00
Serene 434aeb8feb
Add a combined command for the archiver and downloader: clone (#433) 
* Simplify downloader function

* Add basic scraper class

* Add "scrape" command

* Rename "scrape" command to "clone"

* Add integration tests for clone command

* Update README

* Fix failing test
 2021-06-06 13:29:09 +03:00
Serene-Arc 79fba4ac4a Fix indent 2021-05-31 13:42:41 +10:00
Serene-Arc e2582ecb3e Catch error with MacOS writing per issue #407 2021-05-23 12:17:14 +10:00
Serene-Arc 47a4951279 Rename variable 2021-05-23 12:13:44 +10:00
Serene-Arc 4395dd4646 Update logging messages to include submission IDs 2021-05-22 11:53:44 +10:00
Serene-Arc a104a154fc Simplify method structure 2021-05-22 11:53:44 +10:00
Ali Parlakci da8c64ec51 Read files in chunks instead when hashing (#416) 2021-05-22 08:46:39 +10:00
Ali Parlakci 7c401b1461
Merge branch 'reddit_connector_refactor' of https://github.com/Serene-Arc/bulk-downloader-for-reddit into Serene-Arc-reddit_connector_refactor 2021-05-17 13:53:48 +03:00
Serene c581bef790
Set file creation times to the post creation time (#391) 2021-05-17 13:49:35 +03:00
Serene-Arc 7016603763 Refactor out super class RedditConnector 2021-05-17 11:50:17 +10:00
Ali Parlakci 200916a150 Rename --exclude-id(-file) to --skip-id(-file) 2021-05-17 10:30:55 +10:00
Ali Parlakci f768a7d61c Rename --skip to --skip-format 2021-05-17 10:30:55 +10:00
Serene-Arc 7e70175e4c Change logging message to include submission ID 2021-05-10 19:03:20 +10:00
Serene-Arc afa3e2548f Add customisable time formatting 2021-05-03 14:05:05 +10:00
Serene-Arc eda12e5274 Make downloadfilter apply itself to Resources 2021-05-03 14:02:03 +10:00
Serene-Arc 711f8b0c76 Add exception for r/all in subreddit check 2021-05-02 14:00:23 +10:00
Serene-Arc 14195157de Catch errors for banned or private subreddits 2021-05-01 13:36:38 +10:00
Serene-Arc 17499baf61 Add informative error when testing user existence 2021-04-28 12:03:28 +10:00
Serene-Arc e6551bb797 Return banned users as not existing 2021-04-28 12:03:28 +10:00
Serene-Arc db46676dec Catch error when logfile accessed concurrently 2021-04-28 12:03:28 +10:00
Serene-Arc cb41d4749a Add option to specify logfile location 2021-04-28 12:03:28 +10:00
Serene-Arc 7438543f49 Remove unused variable 2021-04-28 12:03:28 +10:00
Serene-Arc ca495a6677 Add missing typing declaration 2021-04-28 12:03:28 +10:00
Serene-Arc 214c883a10 Simplify regex string slightly 2021-04-28 12:03:28 +10:00
Serene b37ff0714f Fix time filters (#279) 2021-04-18 16:44:52 +03:00
Serene d8752b15fa Add option to skip specified subreddits (#268) 
* Rename variables

* Add option to skip specific subreddits

* Update README
 2021-04-18 16:44:52 +03:00
Serene-Arc bd9f276acc Rename module 2021-04-18 16:44:52 +03:00
Renamed from bulkredditdownloader/downloader.py (Browse further)