Merge pull request #8088 from appwrite/feat-create-new-metric-md
Feat create new metric explanation
This commit is contained in:
commit
54508c95a8
1 changed files with 137 additions and 0 deletions
137
CONTRIBUTING.md
137
CONTRIBUTING.md
|
@ -301,6 +301,143 @@ This will allow the Appwrite community to sufficiently discuss the new feature v
|
||||||
|
|
||||||
This is also important for the Appwrite lead developers to be able to provide technical input and potentially a different emphasis regarding the feature design and architecture. Some bigger features might need to go through our [RFC process](https://github.com/appwrite/rfc).
|
This is also important for the Appwrite lead developers to be able to provide technical input and potentially a different emphasis regarding the feature design and architecture. Some bigger features might need to go through our [RFC process](https://github.com/appwrite/rfc).
|
||||||
|
|
||||||
|
## Adding New Usage Metrics
|
||||||
|
|
||||||
|
These are the current metrics we collect usage stats for:
|
||||||
|
|
||||||
|
| Metric | Description |
|
||||||
|
|--------|-------------------------------------------------|
|
||||||
|
| teams | Total number of teams per project |
|
||||||
|
| users | Total number of users per project|
|
||||||
|
| executions | Total number of executions per project |
|
||||||
|
| databases | Total number of databases per project |
|
||||||
|
| collections | Total number of collections per project |
|
||||||
|
| {databaseInternalId}.collections | Total number of collections per database|
|
||||||
|
| documents | Total number of documents per project |
|
||||||
|
| {databaseInternalId}.{collectionInternalId}.documents | Total number of documents per collection |
|
||||||
|
| buckets | Total number of buckets per project |
|
||||||
|
| files | Total number of files per project |
|
||||||
|
| {bucketInternalId}.files.storage | Sum of files.storage per bucket (in bytes) |
|
||||||
|
| functions | Total number of functions per project |
|
||||||
|
| deployments | Total number of deployments per project |
|
||||||
|
| builds | Total number of builds per project |
|
||||||
|
| {resourceType}.{resourceInternalId}.deployments | Total number of deployments per function |
|
||||||
|
| executions | Total number of executions per project |
|
||||||
|
| {functionInternalId}.executions | Total number of executions per function |
|
||||||
|
| files.storage | Sum of files storage per project (in bytes) |
|
||||||
|
| deployments.storage | Sum of deployments storage per project (in bytes) |
|
||||||
|
| {resourceType}.{resourceInternalId}.deployments.storage | Sum of deployments storage per function (in bytes) |
|
||||||
|
| builds.storage | Sum of builds storage per project (in bytes) |
|
||||||
|
| builds.compute | Sum of compute duration per project (in seconds) |
|
||||||
|
| {functionInternalId}.builds.storage | Sum of builds storage per function (in bytes) |
|
||||||
|
| {functionInternalId}.builds.compute | Sum of compute duration per function (in seconds) |
|
||||||
|
| network.requests | Total number of network requests per project |
|
||||||
|
| executions.compute | Sum of compute duration per project (in seconds) |
|
||||||
|
| network.inbound | Sum of network inbound traffic per project (in bytes)|
|
||||||
|
| network.outbound | Sum of network outbound traffic per project (in bytes)|
|
||||||
|
|
||||||
|
> Note: The curly brackets in the metric name represents a template and is replaced with a value when the metric is processed.
|
||||||
|
|
||||||
|
Metrics are collected within 3 scopes Daily, monthly, an infinity. Adding new usage metric in order to aggregate usage stats is very simple, but very much dependent on where do you want to collect
|
||||||
|
statistics ,via API or via background worker. For both cases you will need to add a `const` variable in `app/init.php` under the usage metrics list using the naming convention `METRIC_<RESOURCE_NAME>` as shown below.
|
||||||
|
|
||||||
|
```php
|
||||||
|
// Usage metrics
|
||||||
|
const METRIC_FUNCTIONS = 'functions';
|
||||||
|
const METRIC_DEPLOYMENTS = 'deployments';
|
||||||
|
const METRIC_DEPLOYMENTS_STORAGE = 'deployments.storage';
|
||||||
|
```
|
||||||
|
|
||||||
|
Next follow the appropriate steps below depending on whether you're adding the metric to the API or the worker.
|
||||||
|
|
||||||
|
**API**
|
||||||
|
|
||||||
|
In file `app/controllers/shared/api.php` On the database listener, add to an existing or create a new switch case. Add a call to the usage worker with your new metric const like so:
|
||||||
|
|
||||||
|
```php
|
||||||
|
case $document->getCollection() === 'teams':
|
||||||
|
$queueForUsage
|
||||||
|
->addMetric(METRIC_TEAMS, $value); // per project
|
||||||
|
break;
|
||||||
|
```
|
||||||
|
There are cases when you need to handle metric that has a parent entity, like buckets.
|
||||||
|
Files are linked to a parent bucket, you should verify you remove the files stats when you delete a bucket.
|
||||||
|
|
||||||
|
In that case you need also to handle children removal using addReduce() method call.
|
||||||
|
|
||||||
|
```php
|
||||||
|
|
||||||
|
case $document->getCollection() === 'buckets': //buckets
|
||||||
|
$queueForUsage
|
||||||
|
->addMetric(METRIC_BUCKETS, $value); // per project
|
||||||
|
if ($event === Database::EVENT_DOCUMENT_DELETE) {
|
||||||
|
$queueForUsage
|
||||||
|
->addReduce($document);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
In addition, you will also need to add some logic to the `reduce()` method of the Usage worker located in `/src/Appwrite/Platform/Workers/Usage.php`, like so:
|
||||||
|
|
||||||
|
```php
|
||||||
|
case $document->getCollection() === 'buckets':
|
||||||
|
$files = $dbForProject->getDocument('stats', md5(self::INFINITY_PERIOD . str_replace('{bucketInternalId}', $document->getInternalId(), METRIC_BUCKET_ID_FILES)));
|
||||||
|
$storage = $dbForProject->getDocument('stats', md5(self::INFINITY_PERIOD . str_replace('{bucketInternalId}', $document->getInternalId(), METRIC_BUCKET_ID_FILES_STORAGE)));
|
||||||
|
|
||||||
|
if (!empty($files['value'])) {
|
||||||
|
$metrics[] = [
|
||||||
|
'key' => METRIC_FILES,
|
||||||
|
'value' => ($files['value'] * -1),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!empty($storage['value'])) {
|
||||||
|
$metrics[] = [
|
||||||
|
'key' => METRIC_FILES_STORAGE,
|
||||||
|
'value' => ($storage['value'] * -1),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
```
|
||||||
|
|
||||||
|
**Background worker**
|
||||||
|
|
||||||
|
You need to inject the usage queue in the desired worker on the constructor method
|
||||||
|
```php
|
||||||
|
/**
|
||||||
|
* @throws Exception
|
||||||
|
*/
|
||||||
|
public function __construct()
|
||||||
|
{
|
||||||
|
$this
|
||||||
|
->desc('Functions worker')
|
||||||
|
->groups(['functions'])
|
||||||
|
->inject('message')
|
||||||
|
->inject('dbForProject')
|
||||||
|
->inject('queueForFunctions')
|
||||||
|
->inject('queueForEvents')
|
||||||
|
->inject('queueForUsage')
|
||||||
|
->inject('log')
|
||||||
|
->callback(fn (Message $message, Database $dbForProject, Func $queueForFunctions, Event $queueForEvents, Usage $queueForUsage, Log $log) => $this->action($message, $dbForProject, $queueForFunctions, $queueForEvents, $queueForUsage, $log));
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
and then trigger the queue with the new metric like so:
|
||||||
|
|
||||||
|
```php
|
||||||
|
$queueForUsage
|
||||||
|
->addMetric(METRIC_BUILDS, 1)
|
||||||
|
->addMetric(METRIC_BUILDS_STORAGE, $build->getAttribute('size', 0))
|
||||||
|
->addMetric(METRIC_BUILDS_COMPUTE, (int)$build->getAttribute('duration', 0) * 1000)
|
||||||
|
->addMetric(str_replace('{functionInternalId}', $function->getInternalId(), METRIC_FUNCTION_ID_BUILDS), 1)
|
||||||
|
->addMetric(str_replace('{functionInternalId}', $function->getInternalId(), METRIC_FUNCTION_ID_BUILDS_STORAGE), $build->getAttribute('size', 0))
|
||||||
|
->addMetric(str_replace('{functionInternalId}', $function->getInternalId(), METRIC_FUNCTION_ID_BUILDS_COMPUTE), (int)$build->getAttribute('duration', 0) * 1000)
|
||||||
|
->setProject($project)
|
||||||
|
->trigger();
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
## Build
|
## Build
|
||||||
|
|
||||||
To build a new version of the Appwrite server, all you need to do is run the build.sh file like this:
|
To build a new version of the Appwrite server, all you need to do is run the build.sh file like this:
|
||||||
|
|
Loading…
Reference in a new issue