Adria Navarro
1f4cdf348f
Encrypt files
2023-06-12 11:49:38 +01:00
Adria Navarro
978591e2ba
Enable encrypting
2023-06-09 16:26:48 +01:00
Adria Navarro
4acfc623b4
Use import for tar
2023-06-09 14:52:19 +01:00
mike12345567
9ee06e8e42
Updating fetch within main datasource UI to mimic that of the onboarding.
2023-06-07 18:34:32 +01:00
Peter Clement
f680fbd822
Merge remote-tracking branch 'origin/develop' into feature/sync-automations
2023-05-18 11:35:25 +01:00
Adria Navarro
a21039d5b4
Fix auth checks when specified auth
2023-05-16 11:43:13 +02:00
Peter Clement
d43fc819c7
refactor check for collect step into sdk
2023-05-16 09:29:40 +01:00
Adria Navarro
c3856a48aa
Fix types
2023-05-15 13:16:56 +02:00
mike12345567
5363693ac3
Removing automation logs from app exports as they are not useful - they take up space and the imported app won't have ran any automations yet. I have added it as a flag so that this does not affect backups (as you would want to maintain logs for backups).
2023-04-27 11:36:52 +01:00
mike12345567
060dc05afa
Removing export all apps functionality from cloud - this was a very niche feature and often confusing, now there is a consistent flow for starting out in self host.
2023-04-20 17:19:17 +01:00
mike12345567
b4e6cbce7a
Some final updates to get tests passing again, issue with publisher tenant ID being the 'unique' format which cannot actually be used.
2023-04-17 15:59:59 +01:00
Michael Drury
ef5bcc4b66
Main body of PR comments, switching event handling to be a publisher of async events, a generic event; then adding a new async document update processor, which can later be converted to be part of the event processing pipeline.
2023-04-15 00:37:22 +01:00
mike12345567
b2d7f24ff3
Adding builder test case.
2023-04-12 20:02:37 +01:00
mike12345567
23e99ca4b7
Adding group and user tests for user sync.
2023-04-12 19:59:05 +01:00
mike12345567
e623820478
Adding test cases.
2023-04-11 17:33:52 +01:00
Michael Drury
58d0a82108
First sync test, checking that a user is created correctly.
2023-04-05 22:35:01 +01:00
Michael Drury
70c9d06832
Adding group management to background user/group sync.
2023-04-05 20:51:47 +01:00
mike12345567
e699f4684a
Updating the global user sync to be more accurate and also remove old user metadata from apps that users don't have access to anymore.
2023-04-04 18:03:56 +01:00
Michael Drury
ff98ba5a0a
Adding a document update queue based on the events which can be used to track when certain documents have changed, using this for users and groups to detect when a re-sync is needed.
2023-04-04 00:25:15 +01:00
mike12345567
5b150e7748
Removing migration change and fixing #9738 - making sure that exports with excludeRows enabled don't include attachments.
2023-03-01 19:43:40 +00:00
mike12345567
eba18a4159
Fix for #9739 - there was an issue with the mango syntax, when working with multi attachment columns it was using an AND comparator instead of OR, it should be searching for rows that contain any attachment column, not all attachment columns.
2023-03-01 19:01:13 +00:00
mike12345567
0c80ea2083
Merge branch 'master' of github.com:Budibase/budibase into develop
2023-02-10 17:42:56 +00:00
mike12345567
4acfae6af6
Updating so that datasources always have correct types after environment variable enrichment.
2023-02-10 12:47:23 +00:00
mike12345567
0d21801ece
Fix based on testing.
2023-02-08 19:44:17 +00:00
mike12345567
68912f8030
Minor fix for the plugin datasource error that occurred when opening apps in an environment where the datasource no longer exists.
2023-02-08 19:17:15 +00:00
mike12345567
4d86df057b
Some fixes for #8770 - the CLI was very memory intensive when working with large databases, that contained many revisions. We don't need the revisions for app exports/backups, therefore I've updated our export systems to drop the revision history as it compacts the database significantly and speeds up exports/reduces memory usage.
2023-02-06 17:54:40 +00:00
mike12345567
c2eb8fb976
Update to cover the authentication for REST, replacing it with secret value and converting to password field, as well as minor update to env dropdown to only be a password field when not containing an environment variable - #9480 .
2023-01-30 18:50:35 +00:00
mike12345567
ec719b251c
Adding test case for secret-value replacement, to confirm that datasources API correctly replaces secret values with the actual secret.
2023-01-27 18:27:03 +00:00
mike12345567
fec16e23a0
Fixes for tests, allow any handlebars for datasources (dynamic vars).
2023-01-27 18:06:39 +00:00
mike12345567
3a40a64d15
Updating datasources to use new onlyFound option.
2023-01-27 15:21:04 +00:00
mike12345567
d667276fa1
Adding a onlyFound option to our handlebars system so that we can enrich only the parts we have and leave other components for further enrichment.
2023-01-27 15:15:07 +00:00
mike12345567
6349b08888
Don't check whether or not a datasource uses env vars when the function specifically includes the env vars, always fetch them so that they can be returned and put into context for queries to use.
2023-01-24 18:56:19 +00:00
Peter Clement
22e1c1cccf
ensure object is passed to processObjectSync
2023-01-24 15:44:34 +00:00
mike12345567
f4d389797c
Some minor updates after reviewing.
2023-01-19 19:10:41 +00:00
Mateus Badan de Pieri
eae7b529a7
adding validation for envvar bindings
...
adding validation for envvar bindings
2023-01-19 15:07:56 +00:00
mike12345567
b659060beb
Updating datasource API to remove secrets when returning datasources.
2023-01-18 19:16:19 +00:00
mike12345567
5e68a4d814
Minor updates to the context system and also adding env vars to automation context.
2023-01-16 16:47:59 +00:00
mike12345567
fc5f6ca530
Adding the ability to store environment variables to context, to make them more easily available/re-usable, as well as exposing them to queries.
2023-01-13 19:53:46 +00:00
mike12345567
87cf52192d
Small update to properly support environments with environment variables, making sure that datasources correctly select based on app ID.
2023-01-12 14:38:12 +00:00
mike12345567
03df57d077
Adding in basic implementation of variable usage, getting from pro and enriching through new datasource SDK.
2023-01-11 17:57:51 +00:00
mike12345567
f4f730aae2
Merge branch 'master' of github.com:Budibase/budibase into develop
2023-01-06 17:29:35 +00:00
mike12345567
76825c38b9
Adding test cases for the attachment re-write feature, mocking the direct couch find functionality.
2023-01-06 13:33:22 +00:00
mike12345567
cf90282303
Moving some stuff around to make more testable.
2023-01-06 12:47:12 +00:00
mike12345567
0c37c57a4b
Fix for #9249 - there was an issue with the updating of attachment URLs when importing an app/backup, this resolves it by utilising pagination in the CouchDB find API, making sure to iterate through all documents when there is a larger number.
2023-01-05 18:03:36 +00:00
Rory Powell
786039055e
File authentication using presigned URLs ( #8883 )
...
* file auth wip
* Private plugin bucket wip
* Add etag to logo request, fix circular dependency
* Resolve cyclic dependency
* Ensure browser doesn't cache custom component
* Global file fixes
* File fixes / remove permaUrl
* Add ctx local storage
* Enable presigned url to work from any host
* Test fix
* Self hosting w/ S3 + other fixes
* Fix for serving dev app in multi tenant
* Fix backup restore and import attachment url processing, never store attachment urls
* Test fixes
* Unit tests WIP
* Replace file path with base64 encoded private key
* Multi tenancy tests for files
* Environment files fixes
* Post-merge build and test fixes
* Add debug conditional to all jest logging and revert/fix prod minio proxy_pass
* Re-add default '/' minio route :(
* Rename iconFile -> iconFileName
* Add cloudfront vars to k8s templates
* Remove public bucket policy
* Remove duplicated test
2022-12-15 11:35:22 +00:00
mike12345567
a120ba0c8d
Fixing repl options.
2022-12-07 18:22:02 +00:00
mike12345567
be0b15428b
Linting.
2022-12-07 14:43:27 +00:00
mike12345567
617863ddd1
Fix for #7865 - if you deleted a cron automation immediately after creating/publishing it there was a scenario where prod automations would be out of sync with dev automations and it wouldn't really delete. To get around this, we do an automation sync back to dev DB - meaning that the cron ID is known and can be disabled.
2022-12-07 14:32:29 +00:00
mike12345567
dbde23a4de
Fixes based on test failures.
2022-12-06 12:47:48 +00:00
mike12345567
2ba9088faa
PR comments, moving more stuff into SDK.
2022-12-06 12:22:41 +00:00
mike12345567
53a6c0e74b
Fix for #8896 - the automation logs were already being sync'd to the dev database, but when unpublished they are in-accessible. Some minor updates to make sure that before unpublishing there is a sync, and then most of the changes are in pro.
2022-12-05 18:24:25 +00:00
mike12345567
8f94664ddd
Merge branch 'develop' of github.com:Budibase/budibase into refactor/backend-core-import
2022-11-22 12:03:31 +00:00
mike12345567
de82eca502
Some typescript conversions, moving a few imports around.
2022-11-21 18:33:34 +00:00
Dean
64e7239d42
Merge remote-tracking branch 'origin/develop' into feature/default-internal-datasource
2022-11-21 15:40:11 +00:00
mike12345567
cfbed0b236
Merge branch 'master' of github.com:Budibase/budibase into develop
2022-11-21 15:05:48 +00:00
Dean
02a902ebd3
Merge remote-tracking branch 'origin/develop' into feature/default-internal-datasource
2022-11-21 11:10:29 +00:00
mike12345567
58e59bfacc
Updating PouchLike to be an interface in types, with its impl stored in backend-core, now called Database.
2022-11-17 14:35:03 +00:00
Dean
ff7a3b8cef
Default Datasource and sample row data
2022-11-13 21:37:50 +00:00
Mel O'Hagan
6d1449431e
Null pointer fix
2022-11-10 15:22:05 +00:00
mike12345567
c744d23832
Major update - removing the use of context for PouchDB instances, swapping knowledge of PouchDB to the PouchLike structure that replaces it.
2022-11-09 16:53:42 +00:00
mike12345567
eab9806ee3
Fixing an issue with some browsers sending the gzip type as x-gzip rather than gzip, which caused the processing to fail.
2022-10-27 19:14:31 +01:00
mike12345567
a24694a4ea
Refactoring a lot of content around webhooks to Typescript, as well as fixing webhooks and automation app IDs on import of new app.
2022-10-25 18:19:18 +01:00
Rory Powell
9face50a07
Fix build failure around PouchDB type
2022-10-25 09:02:55 +01:00
Rory Powell
39c83d8fb1
Move backup processing into pro
2022-10-24 15:28:43 +01:00
mike12345567
fbed8923dc
Moving statistics under backups sdk subfolder.
2022-10-24 10:14:35 +01:00
mike12345567
e1dca87507
Fixing test case.
2022-10-21 18:52:39 +01:00
mike12345567
fe5d11232f
Update with PR comments.
2022-10-21 18:25:35 +01:00
mike12345567
854cb23947
Getting the import system to carry out a backup of the app before the restore.
2022-10-20 19:07:10 +01:00
mike12345567
f11c0bbfd7
Some type updates and processor handling for backup document being created before backup/restore occurs.
2022-10-19 19:19:36 +01:00
mike12345567
10c81c1adc
Fixing issue with using templates.
2022-10-19 16:57:14 +01:00
mike12345567
113a23ba3d
Adding statistics output for backups.
2022-10-19 16:48:32 +01:00
mike12345567
59bf052b70
Fixing issue with metadata name not being provided for uploaded backup on publish.
2022-10-19 13:52:56 +01:00
mike12345567
f5845a8a6e
Protection for real app IDs.
2022-10-19 11:47:07 +01:00
mike12345567
bdc4e29b2d
Finishing import processor - download backup, delete dev DB and then import over the top of this. Also includes a rollback feature if the backup fails to restore for whatever reason.
2022-10-18 19:43:19 +01:00
mike12345567
b52f413aa5
Removing attachment dir constant - no longer needed.
2022-10-18 17:04:18 +01:00
mike12345567
caa4954d8e
Updating export/import to include the budibase client and manifest.
2022-10-18 16:42:25 +01:00
mike12345567
f795cb0e33
Reformatting types to allow queue to be used for import and export.
2022-10-17 19:42:36 +01:00
mike12345567
1373630b33
Minor fixes after testing manual backup system.
2022-10-14 20:10:44 +01:00
mike12345567
0bd2a18e46
Main types and work for the CRUD operations of app backup backend in pro + the listeners to handle exporting apps from the server.
2022-10-14 19:24:03 +01:00
mike12345567
907b838db3
Fixing multi-app import from cloud.
2022-10-13 15:46:53 +01:00
mike12345567
d1c9a56e9a
Full import implementation - needs further testing, untars the file with all apps, then adds each of them individually.
2022-10-12 19:15:28 +01:00
mike12345567
f5dd87f8f9
Adding first pass of attachments updating.
2022-10-12 17:34:17 +01:00
mike12345567
8d7f40e443
Refactoring more to Typescript, adding the ability to use the _find API of CouchDB.
2022-10-12 17:02:23 +01:00
mike12345567
19133f08e6
Adding import functionality - still need to update the attachments URL.
2022-10-11 20:25:22 +01:00
mike12345567
f237befbce
Some fixes + cleanup of tmp directory.
2022-10-11 19:28:13 +01:00
mike12345567
7c71f76b70
Export to tarball through tmp.
2022-10-11 18:21:58 +01:00
mike12345567
1f36eec89a
Some updates towards supporting attachments in app exports.
2022-10-10 20:08:59 +01:00
mike12345567
ef52bde670
Building out initial SDK work - converting some existing exporting work to typescript.
2022-10-06 19:10:45 +01:00
