Serene-Arc
7bca303b1b
Add in downloader parameters
2021-07-29 19:10:10 +10:00
Serene-Arc
3cdae99490
Implement callbacks for downloading
2021-07-27 13:39:49 +10:00
Serene-Arc
e5be624f1e
Check submission URL against filter before factory
2021-06-23 14:30:39 +10:00
Serene-Arc
7c27b7bf12
Update logging message
2021-06-13 09:49:42 +10:00
Serene-Arc
8ba2d0bb55
Add missing return statement
2021-06-10 18:59:22 +10:00
Serene
6dcef83666
Add ability to disable modules ( #434 )
...
* Fix test name to match standard
* Rename file
* Add ability to disable modules
* Update README
* Fix missing comma
* Fix more missing commas. sigh...
Co-authored-by: Ali Parlakçı <parlakciali@gmail.com>
2021-06-06 13:47:56 +03:00
Serene
434aeb8feb
Add a combined command for the archiver and downloader: clone
( #433 )
...
* Simplify downloader function
* Add basic scraper class
* Add "scrape" command
* Rename "scrape" command to "clone"
* Add integration tests for clone command
* Update README
* Fix failing test
2021-06-06 13:29:09 +03:00
Serene-Arc
79fba4ac4a
Fix indent
2021-05-31 13:42:41 +10:00
Serene-Arc
e2582ecb3e
Catch error with MacOS writing per issue #407
2021-05-23 12:17:14 +10:00
Serene-Arc
47a4951279
Rename variable
2021-05-23 12:13:44 +10:00
Serene-Arc
4395dd4646
Update logging messages to include submission IDs
2021-05-22 11:53:44 +10:00
Serene-Arc
a104a154fc
Simplify method structure
2021-05-22 11:53:44 +10:00
Ali Parlakci
da8c64ec51
Read files in chunks instead when hashing ( #416 )
2021-05-22 08:46:39 +10:00
Ali Parlakci
7c401b1461
Merge branch 'reddit_connector_refactor' of https://github.com/Serene-Arc/bulk-downloader-for-reddit into Serene-Arc-reddit_connector_refactor
2021-05-17 13:53:48 +03:00
Serene
c581bef790
Set file creation times to the post creation time ( #391 )
2021-05-17 13:49:35 +03:00
Serene-Arc
7016603763
Refactor out super class RedditConnector
2021-05-17 11:50:17 +10:00
Ali Parlakci
200916a150
Rename --exclude-id(-file) to --skip-id(-file)
2021-05-17 10:30:55 +10:00
Ali Parlakci
f768a7d61c
Rename --skip to --skip-format
2021-05-17 10:30:55 +10:00
Serene-Arc
7e70175e4c
Change logging message to include submission ID
2021-05-10 19:03:20 +10:00
Serene-Arc
afa3e2548f
Add customisable time formatting
2021-05-03 14:05:05 +10:00
Serene-Arc
eda12e5274
Make downloadfilter apply itself to Resources
2021-05-03 14:02:03 +10:00
Serene-Arc
711f8b0c76
Add exception for r/all in subreddit check
2021-05-02 14:00:23 +10:00
Serene-Arc
14195157de
Catch errors for banned or private subreddits
2021-05-01 13:36:38 +10:00
Serene-Arc
17499baf61
Add informative error when testing user existence
2021-04-28 12:03:28 +10:00
Serene-Arc
e6551bb797
Return banned users as not existing
2021-04-28 12:03:28 +10:00
Serene-Arc
db46676dec
Catch error when logfile accessed concurrently
2021-04-28 12:03:28 +10:00
Serene-Arc
cb41d4749a
Add option to specify logfile location
2021-04-28 12:03:28 +10:00
Serene-Arc
7438543f49
Remove unused variable
2021-04-28 12:03:28 +10:00
Serene-Arc
ca495a6677
Add missing typing declaration
2021-04-28 12:03:28 +10:00
Serene-Arc
214c883a10
Simplify regex string slightly
2021-04-28 12:03:28 +10:00
Serene
b37ff0714f
Fix time filters ( #279 )
2021-04-18 16:44:52 +03:00
Serene
d8752b15fa
Add option to skip specified subreddits ( #268 )
...
* Rename variables
* Add option to skip specific subreddits
* Update README
2021-04-18 16:44:52 +03:00
Serene-Arc
bd9f276acc
Rename module
2021-04-18 16:44:52 +03:00