CodeMirror/bulk-downloader-for-reddit
1
0
Fork 0
mirror of synced 2024-05-19 11:42:40 +12:00
Code Issues Releases Wiki Activity

Format supporting documents correctly

Browse source
This commit is contained in:
Serene-Arc 2022-12-03 16:46:45 +10:00
parent bfd2d31b7b
commit 8feb6517f1
2 changed files with 26 additions and 34 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
docs/ARCHITECTURE.md
View file

@ -19,21 +19,13 @@ Another major part of the ethos of the design is DOTADIW, Do One Thing And Do It
The BDFR is organised around a central object, the RedditDownloader class. The Archiver object extends and inherits from this class.
1. The RedditDownloader parses all the arguments and configuration options, held in the Configuration object, and creates a variety of internal objects for use, such as the file name formatter, download filter, etc.
2. The RedditDownloader scrapes raw submissions from Reddit via several methods relating to different sources. A source is defined as a single stream of submissions from a subreddit, multireddit, or user list.
3. These raw submissions are passed to the DownloaderFactory class to select the specialised downloader class to use. Each of these are for a specific website or link type, with some catch-all classes like Direct.
4. The BaseDownloader child, spawned by DownloaderFactory, takes the link and does any necessary processing to find the direct link to the actual resource.
5. This is returned to the RedditDownloader in the form of a Resource object. This holds the URL and some other information for the final resource.
6. The Resource is passed through the DownloadFilter instantiated in step 1.
7. The destination file name for the Resource is calculated. If it already exists, then the Resource will be discarded.
8. Here the actual data is downloaded to the Resource and a hash calculated which is used to find duplicates.
9. Only then is the Resource written to the disk.
This is the step-by-step process that the BDFR goes through to download a Reddit post.