Merge pull request #721 from ZapperDJ/ZapperDJ-patch-1
This commit is contained in:
commit
74fb9e0ad7
|
@ -438,6 +438,10 @@ The best option would be to always download to a filesystem that is as permissio
|
||||||
|
|
||||||
The logfiles that the BDFR outputs are consistent and quite detailed and in a format that is amenable to regex. To this end, a number of bash scripts have been [included here](./scripts). They show examples for how to extract successfully downloaded IDs, failed IDs, and more besides.
|
The logfiles that the BDFR outputs are consistent and quite detailed and in a format that is amenable to regex. To this end, a number of bash scripts have been [included here](./scripts). They show examples for how to extract successfully downloaded IDs, failed IDs, and more besides.
|
||||||
|
|
||||||
|
## Unsaving posts
|
||||||
|
|
||||||
|
Back in v1 there was an option to unsave posts from your account when downloading, but it was removed from the core BDFR on v2 as it is considered a read-only tool. However, for those missing this functionality, a script was created that uses the log files to achieve this. There is info on how to use this on the README.md file on the scripts subdirectory.
|
||||||
|
|
||||||
## List of currently supported sources
|
## List of currently supported sources
|
||||||
|
|
||||||
- Direct links (links leading to a file)
|
- Direct links (links leading to a file)
|
||||||
|
|
|
@ -2,10 +2,11 @@
|
||||||
|
|
||||||
Due to the verboseness of the logs, a great deal of information can be gathered quite easily from the BDFR's logfiles. In this folder, there is a selection of scripts that parse these logs, scraping useful bits of information. Since the logfiles are recurring patterns of strings, it is a fairly simple matter to write scripts that utilise tools included on most Linux systems.
|
Due to the verboseness of the logs, a great deal of information can be gathered quite easily from the BDFR's logfiles. In this folder, there is a selection of scripts that parse these logs, scraping useful bits of information. Since the logfiles are recurring patterns of strings, it is a fairly simple matter to write scripts that utilise tools included on most Linux systems.
|
||||||
|
|
||||||
- [Script to extract all successfully downloaded IDs](#extract-all-successfully-downloaded-ids)
|
- [Script to extract all successfully downloaded IDs](#extract-all-successfully-downloaded-ids)
|
||||||
- [Script to extract all failed download IDs](#extract-all-failed-ids)
|
- [Script to extract all failed download IDs](#extract-all-failed-ids)
|
||||||
- [Timestamp conversion](#converting-bdfrv1-timestamps-to-bdfrv2-timestamps)
|
- [Timestamp conversion](#converting-bdfrv1-timestamps-to-bdfrv2-timestamps)
|
||||||
- [Printing summary statistics for a run](#printing-summary-statistics)
|
- [Printing summary statistics for a run](#printing-summary-statistics)
|
||||||
|
- [Unsaving posts from your account after downloading](#unsave-posts-after-downloading)
|
||||||
|
|
||||||
## Extract all Successfully Downloaded IDs
|
## Extract all Successfully Downloaded IDs
|
||||||
|
|
||||||
|
@ -67,3 +68,23 @@ Excluded submissions: 1146
|
||||||
Files with existing hash skipped: 0
|
Files with existing hash skipped: 0
|
||||||
Submissions from excluded subreddits: 0
|
Submissions from excluded subreddits: 0
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Unsave Posts After Downloading
|
||||||
|
|
||||||
|
[This script](unsaveposts.py) takes a list of submission IDs from a file named `successfulids` created with the `extract_successful_ids.sh` script and unsaves them from your account. To make it work you will need to make a user script in your reddit profile like this:
|
||||||
|
- Fill in the username and password fields in the script. Make sure you keep the quotes around the fields.
|
||||||
|
- Go to https://old.reddit.com/prefs/apps/
|
||||||
|
- Click on `Develop an app` at the bottom.
|
||||||
|
- Make sure you select a `script` not a `web app`.
|
||||||
|
- Name it `Unsave Posts`.
|
||||||
|
- Fill in the `Redirect URI` field with `127.0.0.0`.
|
||||||
|
- Save it.
|
||||||
|
- Fill in the `client_id` and `client_secret` fields on the script. The client ID is the 14 character string under the name you gave your script. .It'll look like a bunch of random characters like this: pspYLwDoci9z_A. The client secret is the longer string next to "secret". Again keep the quotes around the fields.
|
||||||
|
|
||||||
|
Now the script is ready tu run. Just execute it like this:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3.9 -m bdfr download DOWNLOAD_DIR --authenticate --user me --saved --log LOGFILE_LOCATION
|
||||||
|
./extract_successful_ids.sh LOGFILE_LOCATION > successfulids
|
||||||
|
./unsaveposts.py
|
||||||
|
```
|
||||||
|
|
40
scripts/unsaveposts.py
Normal file
40
scripts/unsaveposts.py
Normal file
|
@ -0,0 +1,40 @@
|
||||||
|
#! /usr/bin/env python3.9
|
||||||
|
'''
|
||||||
|
This script takes a list of submission IDs from a file named "successfulids" created with the
|
||||||
|
"extract_successful_ids.sh" script and unsaves them from your account. To make it work you must
|
||||||
|
fill in the username and password fields below. Make sure you keep the quotes around the fields.
|
||||||
|
You'll need to make a "user script" in your reddit profile to run this.
|
||||||
|
Go to https://old.reddit.com/prefs/apps/
|
||||||
|
Click on "Develop an app" at the bottom.
|
||||||
|
Make sure you select a "script" not a "web app."
|
||||||
|
Give it a random name. Doesn't matter.
|
||||||
|
You need to fill in the "Redirect URI" field with something so go ahead and put 127.0.0.0 in there.
|
||||||
|
Save it.
|
||||||
|
The client ID is the 14 character string under the name you gave your script.
|
||||||
|
It'll look like a bunch of random characters like this: pspYLwDoci9z_A
|
||||||
|
The client secret is the longer string next to "secret".
|
||||||
|
Replace those two fields below. Again keep the quotes around the fields.
|
||||||
|
'''
|
||||||
|
|
||||||
|
import praw
|
||||||
|
|
||||||
|
try:
|
||||||
|
r= praw.Reddit(
|
||||||
|
client_id="CLIENTID",
|
||||||
|
client_secret="CLIENTSECRET",
|
||||||
|
password="USERPASSWORD",
|
||||||
|
user_agent="Unsave Posts",
|
||||||
|
username="USERNAME",
|
||||||
|
)
|
||||||
|
|
||||||
|
with open("successfulids", "r") as f:
|
||||||
|
for item in f:
|
||||||
|
r.submission(id = item.strip()).unsave()
|
||||||
|
|
||||||
|
except:
|
||||||
|
print("Something went wrong. Did you install PRAW? Did you change the user login fields?")
|
||||||
|
|
||||||
|
|
||||||
|
else:
|
||||||
|
print("Done! Thanks for playing!")
|
||||||
|
|
Loading…
Reference in a new issue