2021-03-16 13:01:51 +13:00
|
|
|
# Contributing
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
When making a contribution to the BDFR project, please open an issue beforehand so that the maintainers can weigh in on
|
|
|
|
it. This helps create a trail on GitHub and keeps things organised.
|
2021-03-16 13:01:51 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
**Please don't open an issue on GitHub** unless you are reporting a bug or proposing a feature. For questions, there is
|
|
|
|
a discussion tab on the repository's GitHub page where you can interact with the developers and ask questions. If you
|
|
|
|
believe that something is a bug, or that a feature should be added, then by all means open an issue.
|
2021-03-16 13:01:51 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
All communication on GitHub, Discord, email, or any other medium must conform to the [Code of
|
|
|
|
Conduct](CODE_OF_CONDUCT.md). It's not that hard to stay respectful.
|
2021-03-16 13:01:51 +13:00
|
|
|
|
2021-04-14 12:06:43 +12:00
|
|
|
## Opening an Issue
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
**Before opening a new issue**, be sure that no issues regarding your problem already exist. If a similar issue exists,
|
|
|
|
try to contribute to the issue.
|
2021-04-18 12:26:07 +12:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
**If you are asking a question** about the functioning of the BDFR or the interface, please use the discussions page.
|
|
|
|
Bug reports are not the right medium for asking and answering questions, and the discussions page makes it much easier
|
|
|
|
to discuss, answer, and save questions and responses for others going forwards.
|
2023-02-14 17:37:55 +13:00
|
|
|
|
2021-04-18 12:26:07 +12:00
|
|
|
### Bugs
|
2021-04-14 12:06:43 +12:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
When opening an issue about a bug, **please provide the full log file for the run in which the bug occurred**. This log
|
|
|
|
file is named `log_output.txt` in the configuration folder. Check the [README](../README.md) for information on where
|
|
|
|
this is. This log file will contain all the information required for the developers to recreate the bug.
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
If you do not have or cannot find the log file, then at minimum please provide the **Reddit ID for the submission** or
|
|
|
|
comment which caused the issue. Also copy in the command that you used to run the BDFR from the command line, as that
|
|
|
|
will also provide helpful information when trying to find and fix the bug. If needed, more information will be asked in
|
|
|
|
the thread of the bug.
|
2021-04-14 12:06:43 +12:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
Adding this information is **not optional**. If a bug report is opened without this information, it cannot be replicated
|
|
|
|
by developers. The logs will be asked for once and if they are not supplied, the issue will be closed due to lack of
|
|
|
|
information.
|
2023-02-14 17:37:55 +13:00
|
|
|
|
2021-04-18 12:26:07 +12:00
|
|
|
### Feature requests
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
In the case of requesting a feature or an enhancement, there are fewer requirements. However, please be clear in what
|
|
|
|
you would like the BDFR to do and also how the feature/enhancement would be used or would be useful to more people. It
|
|
|
|
is crucial that the feature is justified. Any feature request without a concrete reason for it to be implemented has
|
|
|
|
a very small chance to get accepted. Be aware that proposed enhancements may be rejected for multiple reasons, or no
|
|
|
|
reason, at the discretion of the developers.
|
2021-04-14 12:06:43 +12:00
|
|
|
|
2021-03-16 13:01:51 +13:00
|
|
|
## Pull Requests
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
Before creating a pull request (PR), check out [ARCHITECTURE](ARCHITECTURE.md) for a short introduction to the way that
|
|
|
|
the BDFR is coded and how the code is organised. Also read the [Style Guide](#style-guide) section below before actually
|
|
|
|
writing any code.
|
2021-03-16 13:01:51 +13:00
|
|
|
|
|
|
|
Once you have done both of these, the below list shows the path that should be followed when writing a PR.
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2022-12-03 19:46:45 +13:00
|
|
|
1. If an issue does not already exist, open one that will relate to the PR.
|
|
|
|
2. Ensure that any changes fit into the architecture specified above.
|
|
|
|
3. Ensure that you have written tests that cover the new code.
|
|
|
|
4. Ensure that no existing tests fail, unless there is a good reason for them to do so.
|
|
|
|
5. If needed, update any documentation with changes.
|
|
|
|
6. Open a pull request that references the relevant issue.
|
|
|
|
7. Expect changes or suggestions and heed the Code of Conduct. We're all volunteers here.
|
2021-03-16 13:01:51 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
Someone will review your pull request as soon as possible, but remember that all maintainers are volunteers and this
|
|
|
|
won't happen immediately. Once it is approved, congratulations! Your code is now part of the BDFR.
|
2021-04-18 12:26:07 +12:00
|
|
|
|
|
|
|
## Preparing the environment for development
|
|
|
|
|
2022-11-30 05:48:24 +13:00
|
|
|
Bulk Downloader for Reddit requires Python 3.9 at minimum. First, ensure that your Python installation satisfies this.
|
2021-04-18 12:26:07 +12:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
BDfR is built in a way that it can be packaged and installed via `pip`. This places BDfR next to other Python packages
|
|
|
|
and enables you to run the program from any directory. Since it is managed by pip, you can also uninstall it.
|
2021-04-18 12:26:07 +12:00
|
|
|
|
|
|
|
To install the program, clone the repository and run pip inside the project's root directory:
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2021-04-18 12:26:07 +12:00
|
|
|
```bash
|
2022-11-30 05:48:24 +13:00
|
|
|
git clone https://github.com/aliparlakci/bulk-downloader-for-reddit.git
|
|
|
|
cd ./bulk-downloader-for-reddit
|
|
|
|
python3 -m pip install -e .
|
2021-04-18 12:26:07 +12:00
|
|
|
```
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
**`-e`** parameter creates a link to that folder. That is, any change inside the folder affects the package immidiately.
|
|
|
|
So, when developing, you can be sure that the package is not stale and Python is always running your latest changes.
|
|
|
|
(Due to this linking, moving/removing/renaming the folder might break it)
|
2021-04-18 12:26:07 +12:00
|
|
|
|
|
|
|
Then, you can run the program from anywhere in your disk as such:
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2021-04-18 12:26:07 +12:00
|
|
|
```bash
|
2022-11-30 16:38:45 +13:00
|
|
|
bdfr
|
2021-04-18 12:26:07 +12:00
|
|
|
```
|
2021-03-16 13:01:51 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
There are additional Python packages that are required to develop the BDFR. These can be installed with the following
|
|
|
|
command:
|
2022-12-08 18:46:58 +13:00
|
|
|
|
|
|
|
```bash
|
2022-12-17 17:45:36 +13:00
|
|
|
python3 -m pip install -e .[dev]
|
2022-12-08 18:46:58 +13:00
|
|
|
```
|
2021-03-16 13:01:51 +13:00
|
|
|
|
2022-12-08 18:46:58 +13:00
|
|
|
### Tools
|
2021-03-16 13:01:51 +13:00
|
|
|
|
2022-12-08 18:46:58 +13:00
|
|
|
The BDFR project uses several tools to manage the code of the project. These include:
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2022-12-08 18:46:58 +13:00
|
|
|
- [black](https://github.com/psf/black)
|
|
|
|
- [markdownlint (mdl)](https://github.com/markdownlint/markdownlint)
|
2023-02-25 09:10:17 +13:00
|
|
|
- [ruff](https://github.com/charliermarsh/ruff)
|
2022-12-08 18:46:58 +13:00
|
|
|
- [tox](https://tox.wiki/en/latest/)
|
|
|
|
- [pre-commit](https://github.com/pre-commit/pre-commit)
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
The first three tools are formatters. These change the code to the standards expected for the BDFR project. The
|
|
|
|
configuration details for these tools are contained in the [pyproject.toml](../pyproject.toml) file for the project.
|
2022-12-08 18:46:58 +13:00
|
|
|
|
|
|
|
The tool `tox` is used to run tests and tools on demand and has the following environments:
|
|
|
|
|
|
|
|
- `format`
|
|
|
|
- `format_check`
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
The tool `pre-commit` is optional, and runs the three formatting tools automatically when a commit is made. This is
|
|
|
|
**highly recommended** to ensure that all code submitted for this project is formatted acceptably. Note that any PR that
|
|
|
|
does not follow the formatting guide will not be accepted. For information on how to use pre-commit to avoid this, see
|
|
|
|
[the pre-commit documentation](https://pre-commit.com/).
|
2022-12-08 18:46:58 +13:00
|
|
|
|
|
|
|
## Style Guide
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
The BDFR uses the Black formatting standard and enforces this with the tool by the same name. Additionally, the tool
|
|
|
|
isort is used as well to format imports.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
See [Preparing the Environment for Development](#preparing-the-environment-for-development) for how to setup these tools
|
|
|
|
to run automatically.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
|
|
|
## Tests
|
|
|
|
|
|
|
|
### Running Tests
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
There are a lot of tests in the BDFR. In fact, there are more tests than lines of functional code. This is one of the
|
|
|
|
strengths of the BDFR in that it is fully tested. The codebase uses the package pytest to create the tests, which is
|
|
|
|
a third-party package that provides many functions and objects useful for testing Python code.
|
2021-04-14 12:06:43 +12:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
When submitting a PR, it is required that you run **all** possible tests to ensure that any new commits haven't broken
|
|
|
|
anything. Otherwise, while writing the request, it can be helpful (and much quicker) to run only a subset of the tests.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
This is accomplished with marks, a system that pytest uses to categorise tests. There are currently the current marks in
|
|
|
|
use in the BDFR test suite.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
|
|
|
- `slow`
|
2022-12-03 19:46:45 +13:00
|
|
|
- This marks a test that may take a long time to complete
|
|
|
|
- Usually marks a test that downloads many submissions or downloads a particularly large resource
|
2021-04-03 20:25:22 +13:00
|
|
|
- `online`
|
2022-12-03 19:46:45 +13:00
|
|
|
- This marks a test that requires an internet connection and uses online resources
|
2021-04-03 20:25:22 +13:00
|
|
|
- `reddit`
|
2022-12-03 19:46:45 +13:00
|
|
|
- This marks a test that accesses online Reddit specifically
|
2021-04-03 20:25:22 +13:00
|
|
|
- `authenticated`
|
2022-12-03 19:46:45 +13:00
|
|
|
- This marks a test that requires a test configuration file with a valid OAuth2 token
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
These tests can be run either all at once, or excluding certain marks. The tests that require online resources, such as
|
|
|
|
those marked `reddit` or `online`, will naturally require more time to run than tests that are entirely offline. To run
|
|
|
|
tests, you must be in the root directory of the project and can use the following command.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
|
|
|
```bash
|
|
|
|
pytest
|
|
|
|
```
|
|
|
|
|
|
|
|
To exclude one or more marks, the following command can be used, substituting the unwanted mark.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
pytest -m "not online"
|
2021-04-14 12:06:43 +12:00
|
|
|
pytest -m "not reddit and not authenticated"
|
2021-04-03 20:25:22 +13:00
|
|
|
```
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2021-04-21 08:26:45 +12:00
|
|
|
### Configuration for authenticated tests
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
There should be configuration file `test_config.cfg` in the project's root directory to be able to run the integration
|
|
|
|
tests with reddit authentication. See how to create such files [here](../README.md#configuration). The easiest way of
|
|
|
|
creating this file is copying your existing `default_config.cfg` file from the path stated in the previous link and
|
|
|
|
renaming it to `test_config.cfg` Be sure that user_token key exists in test_config.cfg.
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2021-04-21 08:26:45 +12:00
|
|
|
---
|
2022-11-30 05:48:24 +13:00
|
|
|
|
2021-04-14 12:06:43 +12:00
|
|
|
For more details, review the pytest documentation that is freely available online.
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
Many IDEs also provide integrated functionality to run and display the results from tests, and almost all of them
|
|
|
|
support pytest in some capacity. This would be the recommended method due to the additional debugging and general
|
|
|
|
capabilities.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
|
|
|
### Writing Tests
|
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
When writing tests, ensure that they follow the style guide. The BDFR uses pytest to run tests. Wherever possible,
|
|
|
|
parameterise tests, even if you only have one test case. This makes it easier to expand in the future, as the ultimate
|
|
|
|
goal is to have multiple test cases for every test, instead of just one.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
If required, use of mocks is expected to simplify tests and reduce the resources or complexity required. Tests should be
|
|
|
|
as small as possible and test as small a part of the code as possible. Comprehensive or integration tests are run with
|
|
|
|
the `click` framework and are located in their own file.
|
2021-04-03 20:25:22 +13:00
|
|
|
|
2023-06-25 16:26:12 +12:00
|
|
|
It is also expected that new tests be classified correctly with the marks described above i.e. if a test accesses Reddit
|
|
|
|
through a `reddit_instance` object, it must be given the `reddit` mark. If it requires an authenticated Reddit instance,
|
|
|
|
then it must have the `authenticated` mark.
|