OMEGARAZER
0c4cfd8b44
ANN202
...
ANN202 fixes
2023-03-10 15:28:22 -05:00
OMEGARAZER
a5f5df7ab3
ANN201
...
ANN201 fixes
2023-03-10 15:28:22 -05:00
OMEGARAZER
caa4421c78
ANN204
...
ANN204 fixes
2023-03-10 15:28:22 -05:00
OMEGARAZER
005454a5c2
Don't create directory if not needed
...
Moves creation of parent directories after dupe check so directories are not made if not needed.
2023-02-26 16:52:39 -05:00
OMEGARAZER
98aa3d7cb6
Quote cleanup
...
cleanup of some strings to prioritize outer double quotes when both are present or switch to double quotes when only single is present.
2023-02-18 16:06:32 -05:00
Serene
0051877e01
Merge pull request #769 from OMEGARAZER/gfycat-api
2023-02-12 11:45:25 +10:00
Serene
e5b184ef9a
Merge pull request #762 from OMEGARAZER/pyupgrade-lint
2023-02-12 11:45:00 +10:00
OMEGARAZER
0e28c7ed7c
Gfycat API
...
Moves Gfycat to use API via site access key.
Adds cachetools as dependency to reuse API keys for Gfycat/Redgifs at 95% of their TTL. Include tests to verify caching.
Updates versions of requests/yt-dlp/black/isort/pytest.
Added default timeout to requests calls.
Adds validate-pyproject and blacken-docs to pre-commit as well as updates versions.
2023-02-11 15:23:08 -05:00
OMEGARAZER
a535fee574
Black update
...
Black version 23.1.0 updates
2023-02-02 11:50:47 -05:00
OMEGARAZER
95c8c72271
UP008
2023-01-28 23:58:36 -05:00
OMEGARAZER
247ea5ddd0
UP009
2023-01-28 23:56:23 -05:00
OMEGARAZER
63b0607f58
bugbear B007
2023-01-28 02:04:06 -05:00
OMEGARAZER
cf5f7bfd16
pep585 and pathlib updates
2023-01-25 22:23:59 -05:00
Serene-Arc
8f17bcf43b
Modify code to accept pluggable logging handlers
2023-01-08 15:21:57 +10:00
OMEGARAZER
83f45e7f60
Standardize shebang and coding declaration
...
Standardizes shebang and coding declarations.
Coding matches what's used by install tools such as pip(x).
Removes a few init files that were not needed.
2022-12-19 18:32:37 -05:00
OMEGARAZER
3aa740e979
Add soft fail on 5xx Prawcore errors.
2022-12-10 12:36:54 -05:00
Serene-Arc
0873a4a2b2
Format according to the black standard
2022-12-03 15:11:17 +10:00
OMEGARAZER
831f49daa6
Refurb linting
...
Lint with [refurb](https://github.com/dosisod/refurb ) using `--disable 126 --python-version 3.9`
2022-11-30 17:19:02 -05:00
Serene-Arc
5341d6f12c
Add catch for per-submission praw errors
2022-11-20 18:54:56 +10:00
OMEGARAZER
3c7f85725e
Narrow except
...
Narrow except to AttributeError
2022-11-08 12:06:20 -05:00
OMEGARAZER
dc5a9ef497
link_to depreciation coverage
...
Futureproof for link_to depreciation.
https://bugs.python.org/issue39950
2022-10-14 18:15:49 -04:00
Serene-Arc
2bbf1b644e
Change logging message
2022-07-23 17:24:05 +10:00
Serene-Arc
f22a8aec4d
Fix line length
2022-07-23 17:24:05 +10:00
Jacob Chapman
5d76fcd5aa
Update downloader.py
2022-07-23 17:24:05 +10:00
Jacob Chapman
4fc0d5dc1d
Add score filtering
2022-07-23 17:24:05 +10:00
dbanon87
1530456cf7
Update downloader.py
2021-11-29 09:23:04 -05:00
Serene-Arc
d0d72c8229
Add integration test for downloader option
2021-11-24 12:49:11 +10:00
Jay R. Wren
2b50ee0724
add test. fix typos.
2021-11-24 12:49:11 +10:00
Serene-Arc
7bca303b1b
Add in downloader parameters
2021-07-29 19:10:10 +10:00
Serene-Arc
3cdae99490
Implement callbacks for downloading
2021-07-27 13:39:49 +10:00
Serene-Arc
e5be624f1e
Check submission URL against filter before factory
2021-06-23 14:30:39 +10:00
Serene-Arc
7c27b7bf12
Update logging message
2021-06-13 09:49:42 +10:00
Serene-Arc
8ba2d0bb55
Add missing return statement
2021-06-10 18:59:22 +10:00
Serene
6dcef83666
Add ability to disable modules ( #434 )
...
* Fix test name to match standard
* Rename file
* Add ability to disable modules
* Update README
* Fix missing comma
* Fix more missing commas. sigh...
Co-authored-by: Ali Parlakçı <parlakciali@gmail.com>
2021-06-06 13:47:56 +03:00
Serene
434aeb8feb
Add a combined command for the archiver and downloader: clone ( #433 )
...
* Simplify downloader function
* Add basic scraper class
* Add "scrape" command
* Rename "scrape" command to "clone"
* Add integration tests for clone command
* Update README
* Fix failing test
2021-06-06 13:29:09 +03:00
Serene-Arc
79fba4ac4a
Fix indent
2021-05-31 13:42:41 +10:00
Serene-Arc
e2582ecb3e
Catch error with MacOS writing per issue #407
2021-05-23 12:17:14 +10:00
Serene-Arc
47a4951279
Rename variable
2021-05-23 12:13:44 +10:00
Serene-Arc
4395dd4646
Update logging messages to include submission IDs
2021-05-22 11:53:44 +10:00
Serene-Arc
a104a154fc
Simplify method structure
2021-05-22 11:53:44 +10:00
Ali Parlakci
da8c64ec51
Read files in chunks instead when hashing ( #416 )
2021-05-22 08:46:39 +10:00
Ali Parlakci
7c401b1461
Merge branch 'reddit_connector_refactor' of https://github.com/Serene-Arc/bulk-downloader-for-reddit into Serene-Arc-reddit_connector_refactor
2021-05-17 13:53:48 +03:00
Serene
c581bef790
Set file creation times to the post creation time ( #391 )
2021-05-17 13:49:35 +03:00
Serene-Arc
7016603763
Refactor out super class RedditConnector
2021-05-17 11:50:17 +10:00
Ali Parlakci
200916a150
Rename --exclude-id(-file) to --skip-id(-file)
2021-05-17 10:30:55 +10:00
Ali Parlakci
f768a7d61c
Rename --skip to --skip-format
2021-05-17 10:30:55 +10:00
Serene-Arc
7e70175e4c
Change logging message to include submission ID
2021-05-10 19:03:20 +10:00
Serene-Arc
afa3e2548f
Add customisable time formatting
2021-05-03 14:05:05 +10:00
Serene-Arc
eda12e5274
Make downloadfilter apply itself to Resources
2021-05-03 14:02:03 +10:00
Serene-Arc
711f8b0c76
Add exception for r/all in subreddit check
2021-05-02 14:00:23 +10:00
