1
0
Fork 0
mirror of synced 2024-09-19 18:59:06 +12:00
Commit graph

128 commits

Author SHA1 Message Date
Adria Navarro
73fd1f66c5 Fix 2023-11-30 18:39:33 +01:00
Adria Navarro
4cd76ea0fc Add tests 2023-11-30 18:39:33 +01:00
Adria Navarro
a32582eb8a Use autoextend as locktype 2023-11-30 18:39:33 +01:00
Adria Navarro
26a77298ac Use timers 2023-11-30 18:39:33 +01:00
Adria Navarro
5a7dbb0076 Async 2023-11-30 18:39:33 +01:00
Adria Navarro
3073397800 Fix ttl 2023-11-30 18:39:33 +01:00
Adria Navarro
f71e1ac03a Allow locks without TTL 2023-11-30 18:39:33 +01:00
mike12345567
0df315c478 There has been quite a few redlock errors the last while - the message does not provide much information about what is going wrong - trying to rectify this to see if this is an error that could be impacting performance. 2023-11-21 10:52:50 +00:00
mike12345567
8fdf378a57 Fix an issue with account portal redis connection, if no Redis port is supplied in URL we should assume the default port number 6379. 2023-11-20 15:34:16 +00:00
mike12345567
258ea803bc Further PR comments. 2023-11-17 16:37:00 +00:00
mike12345567
7fb6c0927a Moving things around to get rid of cyclics created by moving invite/passwordReset into backend-core - also updating currentapp test case to mock a bit better. 2023-11-17 16:20:10 +00:00
Sam Rose
f2406889df
Merge branch 'master' into bug/budi-7008-i-was-able-to-send-two-invitations-to-the-same-user-email-2 2023-11-10 16:24:51 +00:00
Sam Rose
4c7c10b121
Set Redis initialisation back to how it was before I started messing with it. 2023-11-10 16:17:18 +00:00
mike12345567
a427d990a1 Quick addition - if the object has been deleted but the key is still known, then CouchDB will alert us to the fact that it is deleted, leaving the response in a weird state. 2023-11-10 11:58:07 +00:00
Sam Rose
94983c289f
Hook redis init flow into overall worker init flow. 2023-11-10 11:39:26 +00:00
Sam Rose
dd2f68d099
Hook new Redis clients into init/shutdown flow. 2023-11-10 11:24:55 +00:00
Sam Rose
d98e217c6c
Fix backend-core redis imports. 2023-11-10 11:21:36 +00:00
Sam Rose
b29cfc600c
Move Invite and PasswordReset code into backend-core. 2023-11-09 14:51:07 +00:00
mike12345567
08c4ba0097 Updating redis option functions usage, as it is no longer returned as a part of the getRedisOptions response. 2023-11-03 18:06:12 +00:00
mike12345567
7bf307b0c2 Further updates to typing. 2023-11-03 18:03:11 +00:00
mike12345567
001cf01303 Updating public API rate limiting functionality to be better typed as well. 2023-11-03 18:00:13 +00:00
mike12345567
f8f1ec4ce9 Removing old redisProtocol string - it is causing confusion and should not be necessary. 2023-11-03 17:17:20 +00:00
Sam Rose
767900aec6
Remove type information from JSDoc in .ts files, it's redundant. 2023-10-17 16:46:32 +01:00
Adria Navarro
9cd7ef7827 Typing redis 2023-09-19 14:32:05 +02:00
Adria Navarro
10fe86e30d Type 2023-09-18 12:44:58 +02:00
mike12345567
c13530ef32 Stop attempt at bundling ioredis-mock, only need to check at runtime. 2023-06-02 13:59:33 +01:00
mike12345567
ddeb25b68d Updating mock redis to only import if it is needed. 2023-06-02 11:52:50 +01:00
Michael Drury
a3f9759a47 Reverting ioredis-mock removal for test cases - will come back to this in future as it currently breaks pro tests. 2023-06-01 00:02:55 +01:00
mike12345567
fba3ec987e Merge branch 'develop' of github.com:Budibase/budibase into fix/datasource-info-endpoint 2023-05-31 17:37:17 +01:00
mike12345567
249625ad1a Updating backend tests to actually utilise the redis container which is spun up as part of the tests, doing away with the somewhat problematic ioredis-mock which was breaking some tests due to the addition of the changes for multi-dev collab. 2023-05-31 17:18:01 +01:00
Andrew Kingston
8a2d4b7052 Merge branch 'develop' of github.com:Budibase/budibase into websocket-redis-integration 2023-05-31 16:28:14 +01:00
Andrew Kingston
e3b5d711df Disable redis select command in tests 2023-05-31 16:27:49 +01:00
Andrew Kingston
ffbbf04e86 Lint 2023-05-31 16:08:35 +01:00
Andrew Kingston
05e34076f7 Fully rewrite websocket redis integration to use individual keys per session, enable TTLs on sesisons, prune sessions when users connect and add a heartbeat to sockets 2023-05-31 15:13:22 +01:00
Andrew Kingston
d926650232 Add back in functionality to select different redis DB's per client 2023-05-31 10:52:39 +01:00
Michael Drury
99607ca06e Reverting try twice change to writethrough. 2023-05-30 20:45:10 +01:00
mike12345567
7c7bd4d5cb Fix for debugging with webstorm the old way (if desired), updating the builder middleware to be more multi-dev capable, ignoring 409s when attempting to update the last updated at for apps (if multiple devs hit at same time, only use one) also updating writethrough cache to retry once, with the extended TTL on locks, plus the multi-dev collab it can take a minute to update usage quota doc when a lot of updates occur at once. 2023-05-30 17:41:20 +01:00
Andrew Kingston
5e480a1527 Use socker.io redis adapter to broadcast events to all server instances 2023-05-25 08:48:56 +01:00
Rory Powell
595db7d346 Merge pull request #10495 from Budibase/fix/custom-locks
Fix usage of custom locks
2023-05-05 15:21:58 +01:00
Rory Powell
1ba5eda149 Fix usage of custom locks 2023-05-05 14:42:21 +01:00
Martin McKeaveney
50de32a8a0 Merge pull request #10452 from Budibase/feature/clustering-support
attaching clustered mode to an env var
2023-05-04 14:25:23 +01:00
Martin McKeaveney
167dfd39d7 fix ts build 2023-05-04 13:32:17 +01:00
Martin McKeaveney
e75e91e248 control all redis clustering through env var 2023-04-28 19:53:15 +01:00
Martin McKeaveney
f73af919a7 attaching clustered mode to an env var 2023-04-28 19:09:22 +01:00
Rory Powell
ec06f13aa6 Per user pricing (#10378)
* Update pro version to 2.4.44-alpha.9 (#10231)

Co-authored-by: Budibase Staging Release Bot <>

* Track installation and unique tenant id on licence activate (#10146)

* changes and exports

* removing the extend

* Lint + tidy

* Update account.ts

---------

Co-authored-by: Rory Powell <rory.codes@gmail.com>
Co-authored-by: mike12345567 <me@michaeldrury.co.uk>

* Type updates for loading new plans (#10245)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS` (#10247)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete + migration (#10250)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete

* Always sync user count from view total_rows value for accuracy

* Add migration to sync users

* Add syncUsers.spec.ts

* Lint

* Types and structures for user subscription quantity sync (#10280)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete

* Always sync user count from view total_rows value for accuracy

* Add migration to sync users

* Add syncUsers.spec.ts

* Prevent old installs from activating, track install info via get license request instead of on activation.

* Add usesInvoicing to PurchasedPlan

* Add min/max users to PurchasedPlan

* Additional test structures for generating a license, remove maxUsers from PurchasedPlan - this is already present in the license quotas

* Stripe integration for monthly prorations on annual plans

* Integrate annual prorations with test clocks

* Updated types, test utils and date processing for licensing (#10346)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete

* Always sync user count from view total_rows value for accuracy

* Add migration to sync users

* Add syncUsers.spec.ts

* Prevent old installs from activating, track install info via get license request instead of on activation.

* Add usesInvoicing to PurchasedPlan

* Add min/max users to PurchasedPlan

* Additional test structures for generating a license, remove maxUsers from PurchasedPlan - this is already present in the license quotas

* Stripe integration for monthly prorations on annual plans

* Integrate annual prorations with test clocks

* Updated types, test utils and date processing

* Lint

* Pricing/billing page (#10353)

* bbui updates for billing page

* Require all PlanTypes in PlanMinimums for compile time safety

* fix test package utils

* Incoming user limits warnings (#10379)

* incoming user limits warning

* fix inlinealert button

* add corretc button link and text to user alert

* pr comments

* simplify limit check

* Types and test updates for subscription quantity changes in account-portal (#10372)

* Add chance extensions for `arrayOf`. Update events spies with license events

* Add generics to doInTenant response

* Update account structure with quota usage

* User count limits (#10385)

* incoming user limits warning

* fix inlinealert button

* add corretc button link and text to user alert

* pr comments

* simplify limit check

* user limit messaging on add users modal

* user limit messaging on import users modal

* update licensing store to be more generic

* some styling updates

* remove console log

* Store tweaks

* Add startDate to Quota type

---------

Co-authored-by: Rory Powell <rory.codes@gmail.com>

* Lint

* Support custom lock options

* Reactivity fixes for add user modals

* Update ethereal email creds

* Add warn for getting invite from code error

* Extract disabling user import condition

* Handling unlimited users in modals logic and adding start date processing to store

* Lint

* Integration testing fixes (#10389)

* lint

---------

Co-authored-by: Mateus Badan de Pieri <mateuspieri@gmail.com>
Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
Co-authored-by: Peter Clement <PClmnt@users.noreply.github.com>
2023-04-24 09:31:48 +01:00
mike12345567
cf5316ec8d General fixes for open handles, attempting to find and close all issues in server which are stopping shutdown of Jest suite. 2023-03-27 19:38:49 +01:00
adrinr
84d450a931 Renames 2023-03-07 12:47:27 +01:00
adrinr
c1462a7c9c Clean code 2023-02-28 14:54:43 +01:00
adrinr
66217d6b08 Return redlock execution info 2023-02-28 12:52:43 +01:00
adrinr
24161d12bf Type locks 2023-02-28 12:47:28 +01:00
mike12345567
d6b0b7610a Fixing circular dependencies which were blocking tests. 2023-02-24 17:31:58 +00:00
Rory Powell
6a88cfc32d Update locks error logging (#9768)
* Fix intermittent backend-core migration test failure

* Update lock logging
2023-02-22 08:32:03 +00:00
Rory Powell
940de8b6a0 Run CI steps in parallel (#9760)
* Parallel CI

* Add build to integration test

* Add checkout to top of each run

* Revert branch update for ci job

* Experiment with --runInBand for CI

* Fix intermittent backend-core migration test failure

* Fix hanging worker redis connection

* Update naming from reset to newTenant
2023-02-21 17:13:24 +00:00
Rory Powell
eb5aa8786d Merge pull request #9670 from Budibase/budi-6559-enable-higher-concurrency-and-resiliency
Enable higher concurrency and resiliency in worker tests
2023-02-13 14:31:14 +00:00
Rory Powell
07e5598538 Enable use of redis container in worker tests 2023-02-13 12:09:16 +00:00
Rory Powell
ae9979929a Enable higher concurrency and resiliency in worker tests 2023-02-13 11:57:30 +00:00
Rory Powell
fba7ef26ad Replace ts-jest with swc/jest (#9289)
- Add swc dependencies
- Add transform change to jest.config.ts
- Replace `export =` with `export default` in src code to enable code coverage to work with swc transformer
- Restructure backend-core errors package to allow for exporting error classes with ESM syntax
- Update backend-core to no longer use `export =`, export individual packages instead of replacing with `export default` for backwards compatibility
- Update event publishers to use `export default` - this was required for the `jest.spyOn` usage inside backend-core common mocks
- Restructure some jest.mock usages where declaring the jest.fn as a variable outside of the package mock threw an error
2023-01-11 09:37:37 +00:00
Martin McKeaveney
7fb565e9f4 lock changes (#9047) 2022-12-15 12:45:53 +00:00
mike12345567
d3c4c99e16 Complete conversion of backend-core to Typescript. 2022-11-24 18:48:51 +00:00
Rory Powell
56456f3f6d Startup fixes 2022-10-25 13:37:26 +01:00
Rory Powell
281eb151e2 Merge pull request #8164 from Budibase/feature/quota-emails
Approaching and Exceeded Usage Limit Notifications
2022-10-12 12:31:17 +01:00
Martin McKeaveney
536b7238e9 redis fix for new instances, removing relationship options from google sheets integration 2022-10-10 11:00:45 +01:00
Rory Powell
84685d3340 Add locking framework 2022-10-10 08:21:17 +01:00
mike12345567
197d195cb4 Switching from scan for app locks to mget - which is a fast O(N) operation that only retrieves what we need. 2022-10-07 18:05:18 +01:00
mike12345567
53e9f79c65 Adding explanation of the difference between Database and SelectableDatabase in Redis. 2022-06-24 14:42:15 +01:00
mike12345567
b4bed6c0ce Updating redis to use typescript and adding the option of a writethrough cache which can be used, by passing a DB and a value to be written + a delay for writes. 2022-06-23 20:22:51 +01:00
Rory Powell
361ffd224f Merge branch 'release' into feature/event-backfill 2022-06-06 09:50:06 +01:00
Rory Powell
86d094dda4 Migration locks and add optional preventRetry option 2022-06-01 17:52:41 +01:00
Rory Powell
081db8423e Fix key eviction 2022-06-01 15:31:27 +01:00
Rory Powell
7ebfaf8a31 Failover protection with event cache 2022-05-31 21:04:41 +01:00
mike12345567
fa30457f2b Fixing issue with server not shutting down correctly when an error occurs, making sure that everything clears up gracefully. 2022-05-30 21:22:06 +01:00
mike12345567
76f19def26 Moving the withCache function to backend-core so that it can be shared, as well as adding a cache busting function which can be used when one of the checklist configs changes. 2022-05-23 15:03:52 +01:00
Martin McKeaveney
6d3aa6a806 withCache wrapper for working with redis 2022-05-23 00:09:03 +01:00
Rory Powell
715d42d3e6 Merge branch 'develop' into feature/licensing 2022-03-21 17:13:16 +00:00
mike12345567
d55a385072 Fixing an issue with detecting fully qualified Redis URL despite not in use. 2022-03-14 16:50:01 +00:00
mike12345567
30651e81d5 Making the URL optional for application creation/update in OpenAPI spec, removing unused parameter and fixing getRedisOptions function to work with fully qualified URLs. 2022-03-10 15:53:23 +00:00
Rory Powell
a81041bc40 Add shared licensing cache 2022-03-03 13:37:04 +00:00
mike12345567
12ae9993ca Changing directory. 2022-01-10 19:30:35 +00:00