CodeMirror/budibase
1
0
Fork 0
mirror of synced 2024-07-07 15:25:52 +12:00
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into fix/pc-fixes

Browse source
This commit is contained in:
Peter Clement 2024-01-30 16:53:38 +00:00 committed by GitHub
parent 9c47671032 c01518e5d4
commit c915442a9e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
103 changed files with 2116 additions and 1561 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

49
README.md
View file

@ -11,7 +11,7 @@
The low code platform you'll enjoy using
</h3>
<p align="center">
Budibase is an open source low-code platform, and the easiest way to build internal apps that improve productivity.
Budibase is an open-source low-code platform that saves engineers 100s of hours building forms, portals, and approval apps, securely.
</p>
<h3 align="center">
@ -20,7 +20,7 @@
<br>
<p align="center">
<img alt="Budibase design ui" src="https://res.cloudinary.com/daog6scxm/image/upload/v1633524049/ui/design-ui-wide-mobile_gdaveq.jpg">
<img alt="Budibase design ui" src="https://res.cloudinary.com/daog6scxm/image/upload/v1680181644/ui/homepage-design-ui_sizp7b.png">
</p>
<p align="center">
@ -57,7 +57,7 @@
## ✨ Features
### Build and ship real software
Unlike other platforms, with Budibase you build and ship single page applications. Budibase applications have performance baked in and can be designed responsively, providing your users with a great experience.
Unlike other platforms, with Budibase you build and ship single page applications. Budibase applications have performance baked in and can be designed responsively, providing users with a great experience.
<br /><br />
### Open source and extensible
@ -65,40 +65,36 @@ Budibase is open-source - licensed as GPL v3. This should fill you with confiden
<br /><br />
### Load data or start from scratch
Budibase pulls in data from multiple sources, including MongoDB, CouchDB, PostgreSQL, MySQL, Airtable, S3, DynamoDB, or a REST API. And unlike other platforms, with Budibase you can start from scratch and create business apps with no datasources. [Request new datasources](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
Budibase pulls data from multiple sources, including MongoDB, CouchDB, PostgreSQL, MySQL, Airtable, S3, DynamoDB, or a REST API. And unlike other platforms, with Budibase you can start from scratch and create business apps with no data sources. [Request new datasources](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
<p align="center">
<img alt="Budibase data" src="https://res.cloudinary.com/daog6scxm/image/upload/v1636970242/Out%20of%20beta%20launch/data_n1tlhf.png">
<img alt="Budibase data" src="https://res.cloudinary.com/daog6scxm/image/upload/v1680281798/ui/data_klbuna.png">
</p>
<br /><br />
### Design and build apps with powerful pre-made components
Budibase comes out of the box with beautifully designed, powerful components which you can use like building blocks to build your UI. We also expose a lot of your favourite CSS styling options so you can go that extra creative mile. [Request new component](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
Budibase comes out of the box with beautifully designed, powerful components which you can use like building blocks to build your UI. We also expose many of your favourite CSS styling options so you can go that extra creative mile. [Request new component](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
<p align="center">
<img alt="Budibase design" src="https://res.cloudinary.com/daog6scxm/image/upload/v1636970243/Out%20of%20beta%20launch/design-like-a-pro_qhlfeu.gif">
<img alt="Budibase design" src="https://res.cloudinary.com/daog6scxm/image/upload/v1675437167/ui/form_2x_mbli8y.png">
</p>
<br /><br />
### Automate processes, integrate with other tools, and connect to webhooks
Save time by automating manual processes and workflows. From connecting to webhooks, to automating emails, simply tell Budibase what to do and let it work for you. You can easily [create new automations for Budibase here](https://github.com/Budibase/automations) or [Request new automation](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
<p align="center">
<img alt="Budibase automations" src="https://res.cloudinary.com/daog6scxm/image/upload/v1636970486/Out%20of%20beta%20launch/automation_riro7u.png">
</p>
### Automate processes, integrate with other tools and connect to webhooks
Save time by automating manual processes and workflows. From connecting to webhooks to automating emails, simply tell Budibase what to do and let it work for you. You can easily [create new automations for Budibase here](https://github.com/Budibase/automations) or [Request new automation](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
<br /><br />
### Integrate with your favorite tools
Budibase integrates with a number of popular tools allowing you to build apps that perfectly fit your stack.
<p align="center">
<img alt="Budibase integrations" src="https://res.cloudinary.com/daog6scxm/image/upload/v1636970242/Out%20of%20beta%20launch/integrations_kc7dqt.png">
<img alt="Budibase integrations" src="https://res.cloudinary.com/daog6scxm/image/upload/v1680195228/ui/automate_fg9z07.png">
</p>
<br /><br />
### Admin paradise
Budibase is made to scale. With Budibase, you can self-host on your own infrastructure and globally manage users, onboarding, SMTP, apps, groups, theming and more. You can also provide users/groups with an app portal and disseminate user-management to the group manager.
### Deploy with confidence and security
Budibase is made to scale. With Budibase, you can self-host on your own infrastructure and globally manage users, onboarding, SMTP, apps, groups, theming and more. You can also provide users/groups with an app portal and disseminate user management to the group manager.
- Checkout the promo video: https://youtu.be/xoljVpty_Kw
@ -119,17 +115,14 @@ As with anything that we build in Budibase, our new public API is simple to use,
#### Docs
You can learn more about the Budibase API at the following places:
- [General documentation](https://docs.budibase.com/docs/public-api) : Learn how to get your API key, how to use spec, and how to use with Postman
- [General documentation](https://docs.budibase.com/docs/public-api): Learn how to get your API key, how to use spec, and how to use Postman
- [Interactive API documentation](https://docs.budibase.com/reference/post_applications) : Learn how to interact with the API
#### Guides
- [Build an app with Budibase and Next.js](https://budibase.com/blog/building-a-crud-app-with-budibase-and-next.js/)
<br /><br />
## 🏁 Get started
Deploy Budibase self-hosted in your existing infrastructure, using Docker, Kubernetes, and Digital Ocean.
Or use Budibase Cloud if you don't need to self-host, and would like to get started quickly.
Deploy Budibase using Docker, Kubernetes, and Digital Ocean on your existing infrastructure. Or use Budibase Cloud if you don't need to self-host and would like to get started quickly.
### [Get started with self-hosting Budibase](https://docs.budibase.com/docs/hosting-methods)
@ -162,7 +155,7 @@ If you have a question or would like to talk with other Budibase users and join
## ❗ Code of conduct
Budibase is dedicated to providing a welcoming, diverse, and harrassment-free experience for everyone. We expect everyone in the Budibase community to abide by our [**Code of Conduct**](https://github.com/Budibase/budibase/blob/HEAD/docs/CODE_OF_CONDUCT.md). Please read it.
Budibase is dedicated to providing everyone a welcoming, diverse, and harassment-free experience. We expect everyone in the Budibase community to abide by our [**Code of Conduct**](https://github.com/Budibase/budibase/blob/HEAD/docs/CODE_OF_CONDUCT.md). Please read it.
<br />
@ -171,16 +164,16 @@ Budibase is dedicated to providing a welcoming, diverse, and harrassment-free ex
## 🙌 Contributing to Budibase
From opening a bug report to creating a pull request: every contribution is appreciated and welcomed. If you're planning to implement a new feature or change the API please create an issue first. This way we can ensure your work is not in vain.
Environment setup instructions are available for [Debian](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-DEBIAN.md) and [MacOSX](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-MACOSX.md)
From opening a bug report to creating a pull request: every contribution is appreciated and welcomed. If you're planning to implement a new feature or change the API, please create an issue first. This way, we can ensure your work is not in vain.
Environment setup instructions are available [here](https://github.com/Budibase/budibase/tree/HEAD/docs/CONTRIBUTING.md).
### Not Sure Where to Start?
A good place to start contributing, is the [First time issues project](https://github.com/Budibase/budibase/projects/22).
A good place to start contributing is the [First time issues project](https://github.com/Budibase/budibase/projects/22).
### How the repository is organized
Budibase is a monorepo managed by lerna. Lerna manages the building and publishing of the budibase packages. At a high level, here are the packages that make up Budibase.
- [packages/builder](https://github.com/Budibase/budibase/tree/HEAD/packages/builder) - contains code for the budibase builder client side svelte application.
- [packages/builder](https://github.com/Budibase/budibase/tree/HEAD/packages/builder) - contains code for the budibase builder client-side svelte application.
- [packages/client](https://github.com/Budibase/budibase/tree/HEAD/packages/client) - A module that runs in the browser responsible for reading JSON definition and creating living, breathing web apps from it.
@ -193,7 +186,7 @@ For more information, see [CONTRIBUTING.md](https://github.com/Budibase/budibase
## 📝 License
Budibase is open-source, licensed as [GPL v3](https://www.gnu.org/licenses/gpl-3.0.en.html). The client and component libraries are licensed as [MPL](https://directory.fsf.org/wiki/License:MPL-2.0) - so the apps that you build can be licensed however you like.
Budibase is open-source, licensed as [GPL v3](https://www.gnu.org/licenses/gpl-3.0.en.html). The client and component libraries are licensed as [MPL](https://directory.fsf.org/wiki/License:MPL-2.0) - so the apps you build can be licensed however you like.
<br /><br />

76
docs/DEV-SETUP-DEBIAN.md
View file

@ -1,76 +0,0 @@
## Dev Environment on Debian 11
### Install NVM & Node 14
NVM documentation: https://github.com/nvm-sh/nvm#installing-and-updating
Install NVM
```
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
```
Install Node 14
```
nvm install 14
```
### Install npm requirements
```
npm install -g yarn jest lerna
```
### Install Docker and Docker Compose
```
apt install docker.io
pip3 install docker-compose
```
### Clone the repo
```
git clone https://github.com/Budibase/budibase.git
```
### Check Versions
This setup process was tested on Debian 11 (bullseye) with version numbers show below. Your mileage may vary using anything else.
- Docker: 20.10.5
- Docker-Compose: 1.29.2
- Node: v14.20.1
- Yarn: 1.22.19
- Lerna: 5.1.4
### Build
```
cd budibase
yarn setup
```
The yarn setup command runs several build steps i.e.
```
node ./hosting/scripts/setup.js && yarn && yarn build && yarn dev
```
So this command will actually run the application in dev mode. It creates .env files under `./packages/server` and `./packages/worker` and runs docker containers for each service via docker-compose.
The dev version will be available on port 10000 i.e.
http://127.0.0.1:10000/builder/admin
### File descriptor issues with Vite and Chrome in Linux
If your dev environment stalls forever, with some network requests stuck in flight, it's likely that Chrome is trying to open more file descriptors than your system allows.
To fix this, apply the following tweaks.
Debian based distros:
Add `* - nofile 65536` to `/etc/security/limits.conf`.
Arch:
Add `DefaultLimitNOFILE=65536` to `/etc/systemd/system.conf`.

84
docs/DEV-SETUP-MACOSX.md
View file

@ -1,84 +0,0 @@
## Dev Environment on MAC OSX 12 (Monterey)
### Install Homebrew
Install instructions [here](https://brew.sh/)
| **NOTE**: If you are working on a M1 Apple Silicon which is running Z shell, you could need to add
`eval $(/opt/homebrew/bin/brew shellenv)` line to your `.zshrc`. This will make your zsh to find the apps you install
through brew.
### Install Node
Budibase requires a recent version of node 14:
```
brew install node npm
node -v
```
### Install npm requirements
```
npm install -g yarn jest lerna
```
### Install Docker and Docker Compose
```
brew install docker docker-compose
```
### Clone the repo
```
git clone https://github.com/Budibase/budibase.git
```
### Check Versions
This setup process was tested on Mac OSX 12 (Monterey) with version numbers shown below. Your mileage may vary using anything else.
- Docker: 20.10.14
- Docker-Compose: 2.6.0
- Node: 14.20.1
- Yarn: 1.22.19
- Lerna: 5.1.4
### Build
```
cd budibase
yarn setup
```
The yarn setup command runs several build steps i.e.
```
node ./hosting/scripts/setup.js && yarn && yarn build && yarn dev
```
So this command will actually run the application in dev mode. It creates .env files under `./packages/server` and `./packages/worker` and runs docker containers for each service via docker-compose.
The dev version will be available on port 10000 i.e.
http://127.0.0.1:10000/builder/admin
| **NOTE**: If you are working on a M1 Apple Silicon, you will need to uncomment `# platform: linux/amd64` line in
[hosting/docker-compose-dev.yaml](../hosting/docker-compose.dev.yaml)
### Troubleshootings
#### Yarn setup errors
If there are errors with the `yarn setup` command, you can try installing nvm and node 14. This is the same as the instructions for Debian 11.
#### Node 14.20.1 not supported for arm64
If you are working with M1 or M2 Mac and trying the Node installation via `nvm`, probably you will find the error `curl: (22) The requested URL returned error: 404`.
Version `v14.20.1` is not supported for arm64; in order to use it, you can switch the CPU architecture for this by the following command:
```shell
arch -x86_64 zsh #Run this before nvm install
```

92
docs/DEV-SETUP-WINDOWS.md
View file

@ -1,92 +0,0 @@
## Dev Environment on Windows 10/11 (WSL2)
### Install WSL with Ubuntu LTS
Enable WSL 2 on Windows 10/11 for docker support.
```
wsl --set-default-version 2
```
Install Ubuntu LTS.
```
wsl --install Ubuntu
```
Or follow the instruction here:
https://learn.microsoft.com/en-us/windows/wsl/install
### Install Docker in windows
Download the installer from docker and install it.
Check this url for more detailed instructions:
https://docs.docker.com/desktop/install/windows-install/
You should follow the next steps from within the Ubuntu terminal.
### Install NVM & Node 14
NVM documentation: https://github.com/nvm-sh/nvm#installing-and-updating
Install NVM
```
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
```
Install Node 14
```
nvm install 14
```
### Install npm requirements
```
npm install -g yarn jest lerna
```
### Clone the repo
```
git clone https://github.com/Budibase/budibase.git
```
### Check Versions
This setup process was tested on Windows 11 with version numbers show below. Your mileage may vary using anything else.
- Docker: 20.10.7
- Docker-Compose: 2.10.2
- Node: v14.20.1
- Yarn: 1.22.19
- Lerna: 5.5.4
### Build
```
cd budibase
yarn setup
```
The yarn setup command runs several build steps i.e.
```
node ./hosting/scripts/setup.js && yarn && yarn build && yarn dev
```
So this command will actually run the application in dev mode. It creates .env files under `./packages/server` and `./packages/worker` and runs docker containers for each service via docker-compose.
The dev version will be available on port 10000 i.e.
http://127.0.0.1:10000/builder/admin
### Working with the code
Here are the instructions to work on the application from within Visual Studio Code (in Windows) through the WSL. All the commands and files are within the Ubuntu system and it should run as if you were working on a Linux machine.
https://code.visualstudio.com/docs/remote/wsl
Note you will be able to run the application from within the WSL terminal and you will be able to access the application from the a browser in Windows.

3
i18n/README.es.md
View file

@ -207,8 +207,7 @@ Desde comunicar un bug a solventar un error en el codigo, toda contribucion es a
implementar una nueva funcionalidad o un realizar un cambio en la API, por favor crea un [nuevo mensaje aqui](https://github.com/Budibase/budibase/issues),
de esta manera nos encargaremos que tu trabajo no sea en vano.
Aqui tienes instrucciones de como configurar tu entorno Budibase para [Debian](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-DEBIAN.md)
y [MacOSX](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-MACOSX.md)
Aqui tienes instrucciones de como configurar tu entorno Budibase para [aquí](https://github.com/Budibase/budibase/tree/HEAD/docs/CONTRIBUTING.md).
### No estas seguro por donde empezar?
Un buen lugar para empezar a contribuir con nosotros es [aqui](https://github.com/Budibase/budibase/projects/22).

2
lerna.json
View file

@ -1,5 +1,5 @@
{
"version": "2.15.6",
"version": "2.16.0",
"npmClient": "yarn",
"packages": [
"packages/*",

2
package.json
View file

@ -46,7 +46,7 @@
"kill-accountportal": "kill-port 3001 4003",
"kill-all": "yarn run kill-builder && yarn run kill-server && yarn kill-accountportal",
"dev": "yarn run kill-all && lerna run --parallel prebuild && lerna run --stream dev --ignore=@budibase/account-portal-ui --ignore @budibase/account-portal-server",
"dev:noserver": "yarn run kill-builder && lerna run --stream dev:stack:up && lerna run --stream dev --ignore @budibase/backend-core --ignore @budibase/server --ignore @budibase/worker --ignore=@budibase/account-portal-ui --ignore @budibase/account-portal-server",
"dev:noserver": "yarn run kill-builder && lerna run --stream dev:stack:up --ignore @budibase/account-portal-server && lerna run --stream dev --ignore @budibase/backend-core --ignore @budibase/server --ignore @budibase/worker --ignore=@budibase/account-portal-ui --ignore @budibase/account-portal-server",
"dev:server": "yarn run kill-server && lerna run --stream dev --scope @budibase/worker --scope @budibase/server",
"dev:accountportal": "yarn kill-accountportal && lerna run dev --stream --scope @budibase/account-portal-ui --scope @budibase/account-portal-server",
"dev:all": "yarn run kill-all && lerna run --stream dev",

2
packages/account-portal

@ -1 +1 @@
Subproject commit 1b9fa56fd7b0991b4963de9f3e8b4711abdcae71
Subproject commit 485ec16a9eed48c548a5f1239772139f3319f028

9
packages/builder/src/builderStore/componentUtils.js
View file

@ -92,7 +92,14 @@ export const findAllMatchingComponents = (rootComponent, selector) => {
}
/**
* Finds the closes parent component which matches certain criteria
* Recurses through the component tree and finds all components.
*/
export const findAllComponents = rootComponent => {
return findAllMatchingComponents(rootComponent, () => true)
}
/**
* Finds the closest parent component which matches certain criteria
*/
export const findClosestMatchingComponent = (
rootComponent,

411
packages/builder/src/builderStore/dataBinding.js
View file

@ -1,6 +1,7 @@
import { cloneDeep } from "lodash/fp"
import { get } from "svelte/store"
import {
findAllComponents,
findAllMatchingComponents,
findComponent,
findComponentPath,
@ -102,6 +103,9 @@ export const getAuthBindings = () => {
return bindings
}
/**
* Gets all bindings for environment variables
*/
export const getEnvironmentBindings = () => {
let envVars = get(environment).variables
return envVars.map(variable => {
@ -130,26 +134,22 @@ export const toBindingsArray = (valueMap, prefix, category) => {
if (!binding) {
return acc
}
let config = {
type: "context",
runtimeBinding: binding,
readableBinding: `${prefix}.${binding}`,
icon: "Brackets",
}
if (category) {
config.category = category
}
acc.push(config)
return acc
}, [])
}
/**
* Utility - coverting a map of readable bindings to runtime
* Utility to covert a map of readable bindings to runtime
*/
export const readableToRuntimeMap = (bindings, ctx) => {
if (!bindings || !ctx) {
@ -162,7 +162,7 @@ export const readableToRuntimeMap = (bindings, ctx) => {
}
/**
* Utility - coverting a map of runtime bindings to readable
* Utility to covert a map of runtime bindings to readable bindings
*/
export const runtimeToReadableMap = (bindings, ctx) => {
if (!bindings || !ctx) {
@ -188,15 +188,23 @@ export const getComponentBindableProperties = (asset, componentId) => {
if (!def?.context) {
return []
}
const contexts = Array.isArray(def.context) ? def.context : [def.context]
// Get the bindings for the component
return getProviderContextBindings(asset, component)
const componentContext = {
component,
definition: def,
contexts,
}
return generateComponentContextBindings(asset, componentContext)
}
/**
* Gets all data provider components above a component.
* Gets all component contexts available to a certain component. This handles
* both global and local bindings, taking into account a component's position
* in the component tree.
*/
export const getContextProviderComponents = (
export const getComponentContexts = (
asset,
componentId,
type,
@ -205,30 +213,55 @@ export const getContextProviderComponents = (
if (!asset || !componentId) {
return []
}
let map = {}
// Get the component tree leading up to this component, ignoring the component
// itself
const path = findComponentPath(asset.props, componentId)
if (!options?.includeSelf) {
path.pop()
}
// Filter by only data provider components
return path.filter(component => {
// Processes all contexts exposed by a component
const processContexts = scope => component => {
const def = store.actions.components.getDefinition(component._component)
if (!def?.context) {
return false
return
}
// If no type specified, return anything that exposes context
if (!type) {
return true
if (!map[component._id]) {
map[component._id] = {
component,
definition: def,
contexts: [],
}
}
// Otherwise only match components with the specific context type
const contexts = Array.isArray(def.context) ? def.context : [def.context]
return contexts.find(context => context.type === type) != null
})
contexts.forEach(context => {
// Ensure type matches
if (type && context.type !== type) {
return
}
// Ensure scope matches
let contextScope = context.scope || "global"
if (contextScope !== scope) {
return
}
// Ensure the context is compatible with the component's current settings
if (!isContextCompatibleWithComponent(context, component)) {
return
}
map[component._id].contexts.push(context)
})
}
// Process all global contexts
const allComponents = findAllComponents(asset.props)
allComponents.forEach(processContexts("global"))
// Process all local contexts
const localComponents = findComponentPath(asset.props, componentId)
localComponents.forEach(processContexts("local"))
// Exclude self if required
if (!options?.includeSelf) {
delete map[componentId]
}
// Only return components which provide at least 1 matching context
return Object.values(map).filter(x => x.contexts.length > 0)
}
/**
@ -240,20 +273,19 @@ export const getActionProviders = (
actionType,
options = { includeSelf: false }
) => {
if (!asset || !componentId) {
if (!asset) {
return []
}
// Get the component tree leading up to this component, ignoring the component
// itself
const path = findComponentPath(asset.props, componentId)
if (!options?.includeSelf) {
path.pop()
}
// Get all components
const components = findAllComponents(asset.props)
// Find matching contexts and generate bindings
let providers = []
path.forEach(component => {
components.forEach(component => {
if (!options?.includeSelf && component._id === componentId) {
return
}
const def = store.actions.components.getDefinition(component._component)
const actions = (def?.actions || []).map(action => {
return typeof action === "string" ? { type: action } : action
@ -317,142 +349,132 @@ export const getDatasourceForProvider = (asset, component) => {
* Gets all bindable data properties from component data contexts.
*/
const getContextBindings = (asset, componentId) => {
// Extract any components which provide data contexts
const dataProviders = getContextProviderComponents(asset, componentId)
// Get all available contexts for this component
const componentContexts = getComponentContexts(asset, componentId)
// Generate bindings for all matching components
return getProviderContextBindings(asset, dataProviders)
// Generate bindings for each context
return componentContexts
.map(componentContext => {
return generateComponentContextBindings(asset, componentContext)
})
.flat()
}
/**
* Gets the context bindings exposed by a set of data provider components.
* Generates a set of bindings for a given component context
*/
const getProviderContextBindings = (asset, dataProviders) => {
if (!asset || !dataProviders) {
const generateComponentContextBindings = (asset, componentContext) => {
console.log("Hello ")
const { component, definition, contexts } = componentContext
if (!component || !definition || !contexts?.length) {
return []
}
// Ensure providers is an array
if (!Array.isArray(dataProviders)) {
dataProviders = [dataProviders]
}
// Create bindings for each data provider
let bindings = []
dataProviders.forEach(component => {
const def = store.actions.components.getDefinition(component._component)
const contexts = Array.isArray(def.context) ? def.context : [def.context]
contexts.forEach(context => {
if (!context?.type) {
return
}
// Create bindings for each context block provided by this data provider
contexts.forEach(context => {
if (!context?.type) {
let schema
let table
let readablePrefix
let runtimeSuffix = context.suffix
if (context.type === "form") {
// Forms do not need table schemas
// Their schemas are built from their component field names
schema = buildFormSchema(component, asset)
readablePrefix = "Fields"
} else if (context.type === "static") {
// Static contexts are fully defined by the components
schema = {}
const values = context.values || []
values.forEach(value => {
schema[value.key] = {
name: value.label,
type: value.type || "string",
}
})
} else if (context.type === "schema") {
// Schema contexts are generated dynamically depending on their data
const datasource = getDatasourceForProvider(asset, component)
if (!datasource) {
return
}
const info = getSchemaForDatasource(asset, datasource)
schema = info.schema
table = info.table
let schema
let table
let readablePrefix
let runtimeSuffix = context.suffix
if (context.type === "form") {
// Forms do not need table schemas
// Their schemas are built from their component field names
schema = buildFormSchema(component, asset)
readablePrefix = "Fields"
} else if (context.type === "static") {
// Static contexts are fully defined by the components
schema = {}
const values = context.values || []
values.forEach(value => {
schema[value.key] = {
name: value.label,
type: value.type || "string",
}
})
} else if (context.type === "schema") {
// Schema contexts are generated dynamically depending on their data
const datasource = getDatasourceForProvider(asset, component)
if (!datasource) {
return
}
const info = getSchemaForDatasource(asset, datasource)
schema = info.schema
table = info.table
// Determine what to prefix bindings with
if (datasource.type === "jsonarray") {
// For JSON arrays, use the array name as the readable prefix
const split = datasource.label.split(".")
readablePrefix = split[split.length - 1]
} else if (datasource.type === "viewV2") {
// For views, use the view name
const view = Object.values(table?.views || {}).find(
view => view.id === datasource.id
)
readablePrefix = view?.name
} else {
// Otherwise use the table name
readablePrefix = info.table?.name
}
}
if (!schema) {
return
}
const keys = Object.keys(schema).sort()
// Generate safe unique runtime prefix
let providerId = component._id
if (runtimeSuffix) {
providerId += `-${runtimeSuffix}`
}
if (!filterCategoryByContext(component, context)) {
return
}
const safeComponentId = makePropSafe(providerId)
// Create bindable properties for each schema field
keys.forEach(key => {
const fieldSchema = schema[key]
// Make safe runtime binding
const safeKey = key.split(".").map(makePropSafe).join(".")
const runtimeBinding = `${safeComponentId}.${safeKey}`
// Optionally use a prefix with readable bindings
let readableBinding = component._instanceName
if (readablePrefix) {
readableBinding += `.${readablePrefix}`
}
readableBinding += `.${fieldSchema.name || key}`
const bindingCategory = getComponentBindingCategory(
component,
context,
def
// Determine what to prefix bindings with
if (datasource.type === "jsonarray") {
// For JSON arrays, use the array name as the readable prefix
const split = datasource.label.split(".")
readablePrefix = split[split.length - 1]
} else if (datasource.type === "viewV2") {
// For views, use the view name
const view = Object.values(table?.views || {}).find(
view => view.id === datasource.id
)
readablePrefix = view?.name
} else {
// Otherwise use the table name
readablePrefix = info.table?.name
}
}
if (!schema) {
return
}
// Create the binding object
bindings.push({
type: "context",
runtimeBinding,
readableBinding,
// Field schema and provider are required to construct relationship
// datasource options, based on bindable properties
fieldSchema,
providerId,
// Table ID is used by JSON fields to know what table the field is in
tableId: table?._id,
component: component._component,
category: bindingCategory.category,
icon: bindingCategory.icon,
display: {
name: fieldSchema.name || key,
type: fieldSchema.type,
},
})
const keys = Object.keys(schema).sort()
// Generate safe unique runtime prefix
let providerId = component._id
if (runtimeSuffix) {
providerId += `-${runtimeSuffix}`
}
const safeComponentId = makePropSafe(providerId)
// Create bindable properties for each schema field
keys.forEach(key => {
const fieldSchema = schema[key]
// Make safe runtime binding
const safeKey = key.split(".").map(makePropSafe).join(".")
const runtimeBinding = `${safeComponentId}.${safeKey}`
// Optionally use a prefix with readable bindings
let readableBinding = component._instanceName
if (readablePrefix) {
readableBinding += `.${readablePrefix}`
}
readableBinding += `.${fieldSchema.name || key}`
// Determine which category this binding belongs in
const bindingCategory = getComponentBindingCategory(
component,
context,
definition
)
// Create the binding object
bindings.push({
type: "context",
runtimeBinding,
readableBinding: `${readableBinding}`,
// Field schema and provider are required to construct relationship
// datasource options, based on bindable properties
fieldSchema,
providerId,
// Table ID is used by JSON fields to know what table the field is in
tableId: table?._id,
component: component._component,
category: bindingCategory.category,
icon: bindingCategory.icon,
display: {
name: `${fieldSchema.name || key}`,
type: fieldSchema.type,
},
})
})
})
@ -460,25 +482,38 @@ const getProviderContextBindings = (asset, dataProviders) => {
return bindings
}
// Exclude a data context based on the component settings
const filterCategoryByContext = (component, context) => {
const { _component } = component
/**
* Checks if a certain data context is compatible with a certain instance of a
* configured component.
*/
const isContextCompatibleWithComponent = (context, component) => {
if (!component) {
return false
}
const { _component, actionType } = component
const { type } = context
// Certain types of form blocks only allow certain contexts
if (_component.endsWith("formblock")) {
if (
(component.actionType === "Create" && context.type === "schema") ||
(component.actionType === "View" && context.type === "form")
(actionType === "Create" && type === "schema") ||
(actionType === "View" && type === "form")
) {
return false
}
}
// Allow the context by default
return true
}
// Enrich binding category information for certain components
const getComponentBindingCategory = (component, context, def) => {
// Default category to component name
let icon = def.icon
let category = component._instanceName
// Form block edge case
if (component._component.endsWith("formblock")) {
if (context.type === "form") {
category = `${component._instanceName} - Fields`
@ -496,7 +531,7 @@ const getComponentBindingCategory = (component, context, def) => {
}
/**
* Gets all bindable properties from the logged in user.
* Gets all bindable properties from the logged-in user.
*/
export const getUserBindings = () => {
let bindings = []
@ -566,6 +601,7 @@ const getDeviceBindings = () => {
/**
* Gets all selected rows bindings for tables in the current asset.
* TODO: remove in future because we don't need a separate store for this
*/
const getSelectedRowsBindings = asset => {
let bindings = []
@ -608,6 +644,9 @@ const getSelectedRowsBindings = asset => {
return bindings
}
/**
* Generates a state binding for a certain key name
*/
export const makeStateBinding = key => {
return {
type: "context",
@ -662,6 +701,9 @@ const getUrlBindings = asset => {
return urlParamBindings.concat([queryParamsBinding])
}
/**
* Generates all bindings for role IDs
*/
const getRoleBindings = () => {
return (get(rolesStore) || []).map(role => {
return {
@ -1035,11 +1077,48 @@ export const getAllStateVariables = () => {
getAllAssets().forEach(asset => {
findAllMatchingComponents(asset.props, component => {
const settings = getComponentSettings(component._component)
settings
.filter(setting => setting.type === "event")
.forEach(setting => {
eventSettings.push(component[setting.key])
})
const parseEventSettings = (settings, comp) => {
settings
.filter(setting => setting.type === "event")
.forEach(setting => {
eventSettings.push(comp[setting.key])
})
}
const parseComponentSettings = (settings, component) => {
// Parse the nested button configurations
settings
.filter(setting => setting.type === "buttonConfiguration")
.forEach(setting => {
const buttonConfig = component[setting.key]
if (Array.isArray(buttonConfig)) {
buttonConfig.forEach(button => {
const nestedSettings = getComponentSettings(button._component)
parseEventSettings(nestedSettings, button)
})
}
})
parseEventSettings(settings, component)
}
// Parse the base component settings
parseComponentSettings(settings, component)
// Parse step configuration
const stepSetting = settings.find(
setting => setting.type === "stepConfiguration"
)
const steps = stepSetting ? component[stepSetting.key] : []
const stepDefinition = getComponentSettings(
"@budibase/standard-components/multistepformblockstep"
)
steps.forEach(step => {
parseComponentSettings(stepDefinition, step)
})
})
})

19
packages/builder/src/builderStore/store/frontend.js
View file

@ -707,10 +707,9 @@ export const getFrontendStore = () => {
else {
if (setting.type === "dataProvider") {
// Validate data provider exists, or else clear it
const treeId = parent?._id || component._id
const path = findComponentPath(screen?.props, treeId)
const providers = path.filter(component =>
component._component?.endsWith("/dataprovider")
const providers = findAllMatchingComponents(
screen?.props,
component => component._component?.endsWith("/dataprovider")
)
// Validate non-empty values
const valid = providers?.some(dp => value.includes?.(dp._id))
@ -732,6 +731,16 @@ export const getFrontendStore = () => {
return null
}
// Find all existing components of this type so that we can give this
// component a unique name
const screen = get(selectedScreen).props
const otherComponents = findAllMatchingComponents(
screen,
x => x._component === definition.component && x._id !== screen._id
)
let name = definition.friendlyName || definition.name
name = `${name} ${otherComponents.length + 1}`
// Generate basic component structure
let instance = {
_id: Helpers.uuid(),
@ -741,7 +750,7 @@ export const getFrontendStore = () => {
hover: {},
active: {},
},
_instanceName: `New ${definition.friendlyName || definition.name}`,
_instanceName: name,
...presetProps,
}

1
packages/builder/src/components/backend/Datasources/CreateEditRelationship.svelte
View file

@ -373,6 +373,7 @@
confirmText="Save"
onConfirm={saveRelationship}
disabled={!valid}
size="L"
>
<div class="headings">
<Detail>Tables</Detail>

10
packages/builder/src/components/common/RelationshipSelector.svelte
View file

@ -17,7 +17,7 @@
</script>
<div class="relationship-container">
<div class="relationship-part">
<div class="relationship-type">
<Select
disabled={linkEditDisabled}
bind:value={relationshipPart1}
@ -39,7 +39,7 @@
</div>
</div>
<div class="relationship-container">
<div class="relationship-part">
<div class="relationship-type">
<Select
disabled={linkEditDisabled}
bind:value={relationshipPart2}
@ -79,6 +79,10 @@
}
.relationship-part {
flex-basis: 60%;
flex-basis: 70%;
}
.relationship-type {
flex-basis: 30%;
}
</style>

58
packages/builder/src/components/design/settings/controls/ButtonActionEditor/actions/utils.js
View file

@ -1,5 +1,4 @@
import { getContextProviderComponents } from "builderStore/dataBinding"
import { store } from "builderStore"
import { getComponentContexts } from "builderStore/dataBinding"
import { capitalise } from "helpers"
// Generates bindings for all components that provider "datasource like"
@ -8,58 +7,49 @@ import { capitalise } from "helpers"
// Some examples are saving rows or duplicating rows.
export const getDatasourceLikeProviders = ({ asset, componentId, nested }) => {
// Get all form context providers
const formComponents = getContextProviderComponents(
const formComponentContexts = getComponentContexts(
asset,
componentId,
"form",
{ includeSelf: nested }
{
includeSelf: nested,
}
)
// Get all schema context providers
const schemaComponents = getContextProviderComponents(
const schemaComponentContexts = getComponentContexts(
asset,
componentId,
"schema",
{ includeSelf: nested }
{
includeSelf: nested,
}
)
// Generate contexts for all form providers
const formContexts = formComponents.map(component => ({
component,
context: extractComponentContext(component, "form"),
}))
// Generate contexts for all schema providers
const schemaContexts = schemaComponents.map(component => ({
component,
context: extractComponentContext(component, "schema"),
}))
// Check for duplicate contexts by the same component. In this case, attempt
// to label contexts with their suffixes
schemaContexts.forEach(schemaContext => {
schemaComponentContexts.forEach(schemaContext => {
// Check if we have a form context for this component
const id = schemaContext.component._id
const existing = formContexts.find(x => x.component._id === id)
const existing = formComponentContexts.find(x => x.component._id === id)
if (existing) {
if (existing.context.suffix) {
const suffix = capitalise(existing.context.suffix)
if (existing.contexts[0].suffix) {
const suffix = capitalise(existing.contexts[0].suffix)
existing.readableSuffix = ` - ${suffix}`
}
if (schemaContext.context.suffix) {
const suffix = capitalise(schemaContext.context.suffix)
if (schemaContext.contexts[0].suffix) {
const suffix = capitalise(schemaContext.contexts[0].suffix)
schemaContext.readableSuffix = ` - ${suffix}`
}
}
})
// Generate bindings for all contexts
const allContexts = formContexts.concat(schemaContexts)
return allContexts.map(({ component, context, readableSuffix }) => {
const allContexts = formComponentContexts.concat(schemaComponentContexts)
return allContexts.map(({ component, contexts, readableSuffix }) => {
let readableBinding = component._instanceName
let runtimeBinding = component._id
if (context.suffix) {
runtimeBinding += `-${context.suffix}`
if (contexts[0].suffix) {
runtimeBinding += `-${contexts[0].suffix}`
}
if (readableSuffix) {
readableBinding += readableSuffix
@ -70,13 +60,3 @@ export const getDatasourceLikeProviders = ({ asset, componentId, nested }) => {
}
})
}
// Gets a context definition of a certain type from a component definition
const extractComponentContext = (component, contextType) => {
const def = store.actions.components.getDefinition(component?._component)
if (!def) {
return null
}
const contexts = Array.isArray(def.context) ? def.context : [def.context]
return contexts.find(context => context?.type === contextType)
}

9
packages/builder/src/components/design/settings/controls/DataProviderSelect.svelte
View file

@ -1,15 +1,16 @@
<script>
import { Select } from "@budibase/bbui"
import { makePropSafe } from "@budibase/string-templates"
import { currentAsset, store } from "builderStore"
import { findComponentPath } from "builderStore/componentUtils"
import { currentAsset } from "builderStore"
import { findAllMatchingComponents } from "builderStore/componentUtils"
export let value
const getValue = component => `{{ literal ${makePropSafe(component._id)} }}`
$: path = findComponentPath($currentAsset?.props, $store.selectedComponentId)
$: providers = path.filter(c => c._component?.endsWith("/dataprovider"))
$: providers = findAllMatchingComponents($currentAsset?.props, c =>
c._component?.endsWith("/dataprovider")
)
</script>
<Select

15
packages/builder/src/components/design/settings/controls/DataSourceSelect/DataSourceSelect.svelte
View file

@ -1,6 +1,5 @@
<script>
import {
getContextProviderComponents,
readableToRuntimeBinding,
runtimeToReadableBinding,
} from "builderStore/dataBinding"
@ -30,6 +29,7 @@
import BindingBuilder from "components/integration/QueryBindingBuilder.svelte"
import IntegrationQueryEditor from "components/integration/index.svelte"
import { makePropSafe as safe } from "@budibase/string-templates"
import { findAllComponents } from "builderStore/componentUtils"
import ClientBindingPanel from "components/common/bindings/ClientBindingPanel.svelte"
import DataSourceCategory from "components/design/settings/controls/DataSourceSelect/DataSourceCategory.svelte"
import { API } from "api"
@ -75,12 +75,13 @@
...query,
type: "query",
}))
$: contextProviders = getContextProviderComponents(
$currentAsset,
$store.selectedComponentId
)
$: dataProviders = contextProviders
.filter(component => component._component?.endsWith("/dataprovider"))
$: dataProviders = findAllComponents($currentAsset.props)
.filter(component => {
return (
component._component?.endsWith("/dataprovider") &&
component._id !== $store.selectedComponentId
)
})
.map(provider => ({
label: provider._instanceName,
name: provider._instanceName,

4
packages/builder/src/pages/builder/app/[application]/_components/BuilderSidePanel.svelte
View file

@ -392,6 +392,10 @@
}
const openInviteFlow = () => {
// prevent email from getting overwritten if changes are made
if (!email) {
email = query
}
$licensing.userLimitReached
? userLimitReachedModal.show()
: (invitingFlow = true)

81
packages/client/manifest.json
View file

@ -573,7 +573,6 @@
"description": "A configurable data list that attaches to your backend tables.",
"icon": "JourneyData",
"illegalChildren": ["section"],
"requiredAncestors": ["dataprovider"],
"hasChildren": true,
"size": {
"width": 400,
@ -711,10 +710,12 @@
],
"context": [
{
"type": "schema"
"type": "schema",
"scope": "local"
},
{
"type": "static",
"scope": "local",
"values": [
{
"label": "Row index",
@ -1564,7 +1565,6 @@
"name": "Bar Chart",
"description": "Bar chart",
"icon": "GraphBarVertical",
"requiredAncestors": ["dataprovider"],
"size": {
"width": 600,
"height": 400
@ -1727,7 +1727,6 @@
"width": 600,
"height": 400
},
"requiredAncestors": ["dataprovider"],
"settings": [
{
"type": "text",
@ -1881,7 +1880,6 @@
"width": 600,
"height": 400
},
"requiredAncestors": ["dataprovider"],
"settings": [
{
"type": "text",
@ -2047,7 +2045,6 @@
"width": 600,
"height": 400
},
"requiredAncestors": ["dataprovider"],
"settings": [
{
"type": "text",
@ -2177,7 +2174,6 @@
"width": 600,
"height": 400
},
"requiredAncestors": ["dataprovider"],
"settings": [
{
"type": "text",
@ -2307,7 +2303,6 @@
"width": 600,
"height": 400
},
"requiredAncestors": ["dataprovider"],
"settings": [
{
"type": "text",
@ -4087,7 +4082,6 @@
"width": 400,
"height": 320
},
"requiredAncestors": ["dataprovider"],
"settings": [
{
"type": "dataProvider",
@ -4643,7 +4637,6 @@
"name": "Table",
"icon": "Table",
"illegalChildren": ["section"],
"requiredAncestors": ["dataprovider"],
"hasChildren": true,
"showEmptyState": false,
"size": {
@ -4734,7 +4727,6 @@
"name": "Date Range",
"icon": "Calendar",
"styles": ["size"],
"requiredAncestors": ["dataprovider"],
"hasChildren": false,
"size": {
"width": 200,
@ -4842,7 +4834,6 @@
"width": 100,
"height": 35
},
"requiredAncestors": ["dataprovider"],
"settings": [
{
"type": "dataProvider",
@ -5617,7 +5608,38 @@
}
]
}
]
],
"context": {
"type": "static",
"suffix": "provider",
"values": [
{
"label": "Rows",
"key": "rows",
"type": "array"
},
{
"label": "Extra Info",
"key": "info",
"type": "string"
},
{
"label": "Rows Length",
"key": "rowsLength",
"type": "number"
},
{
"label": "Schema",
"key": "schema",
"type": "object"
},
{
"label": "Page Number",
"key": "pageNumber",
"type": "number"
}
]
}
},
"cardsblock": {
"block": true,
@ -5796,7 +5818,8 @@
],
"context": {
"type": "schema",
"suffix": "repeater"
"suffix": "repeater",
"scope": "local"
}
},
"repeaterblock": {
@ -6020,7 +6043,8 @@
},
{
"type": "schema",
"suffix": "repeater"
"suffix": "repeater",
"scope": "local"
}
]
},
@ -6166,6 +6190,10 @@
"type": "form",
"suffix": "form"
},
{
"type": "schema",
"suffix": "repeater"
},
{
"type": "static",
"suffix": "form",
@ -6479,9 +6507,27 @@
],
"context": {
"type": "schema",
"suffix": "repeater"
"suffix": "repeater",
"scope": "local"
}
},
"grid": {
"name": "Grid",
"icon": "ViewGrid",
"hasChildren": true,
"settings": [
{
"type": "number",
"key": "cols",
"label": "Columns"
},
{
"type": "number",
"key": "rows",
"label": "Rows"
}
]
},
"gridblock": {
"name": "Grid Block",
"icon": "Table",
@ -6625,7 +6671,8 @@
}
],
"context": {
"type": "schema"
"type": "schema",
"scope": "local"
},
"actions": ["RefreshDatasource"]
},

140
packages/client/src/components/Component.svelte
View file

@ -9,7 +9,7 @@
</script>
<script>
import { getContext, setContext, onMount, onDestroy } from "svelte"
import { getContext, setContext, onMount } from "svelte"
import { writable, get } from "svelte/store"
import {
enrichProps,
@ -30,6 +30,15 @@
import ScreenPlaceholder from "components/app/ScreenPlaceholder.svelte"
import ComponentErrorState from "components/error-states/ComponentErrorState.svelte"
import { BudibasePrefix } from "../stores/components.js"
import {
decodeJSBinding,
findHBSBlocks,
isJSBinding,
} from "@budibase/string-templates"
import {
getActionContextKey,
getActionDependentContextKeys,
} from "../utils/buttonActions.js"
export let instance = {}
export let isLayout = false
@ -81,7 +90,6 @@
// Keep track of stringified representations of context and instance
// to avoid enriching bindings as much as possible
let lastContextKey
let lastInstanceKey
// Visibility flag used by conditional UI
@ -98,6 +106,13 @@
// We clear these whenever a new instance is received.
let ephemeralStyles
// Single string of all HBS blocks, used to check if we use a certain binding
// or not
let bindingString = ""
// List of context keys which we use inside bindings
let knownContextKeyMap = {}
// Set up initial state for each new component instance
$: initialise(instance)
@ -155,9 +170,6 @@
hasMissingRequiredSettings)
$: emptyState = empty && showEmptyState
// Enrich component settings
$: enrichComponentSettings($context, settingsDefinitionMap)
// Evaluate conditional UI settings and store any component setting changes
// which need to be made
$: evaluateConditions(conditions)
@ -206,6 +218,7 @@
errorState,
parent: id,
ancestors: [...($component?.ancestors ?? []), instance._component],
path: [...($component?.path ?? []), id],
})
const initialise = (instance, force = false) => {
@ -214,7 +227,8 @@
}
// Ensure we're processing a new instance
const instanceKey = Helpers.hashString(JSON.stringify(instance))
const stringifiedInstance = JSON.stringify(instance)
const instanceKey = Helpers.hashString(stringifiedInstance)
if (instanceKey === lastInstanceKey && !force) {
return
} else {
@ -274,13 +288,54 @@
return missing
})
// When considering bindings we can ignore children, so we remove that
// before storing the reference stringified version
const noChildren = JSON.stringify({ ...instance, _children: null })
const bindings = findHBSBlocks(noChildren).map(binding => {
let sanitizedBinding = binding.replace(/\\"/g, '"')
if (isJSBinding(sanitizedBinding)) {
return decodeJSBinding(sanitizedBinding)
} else {
return sanitizedBinding
}
})
// The known context key map is built up at runtime, as changes to keys are
// encountered. We manually seed this to the required action keys as these
// are not encountered at runtime and so need computed in advance.
knownContextKeyMap = generateActionKeyMap(instance, settingsDefinition)
bindingString = bindings.join(" ")
// Run any migrations
runMigrations(instance, settingsDefinition)
// Force an initial enrichment of the new settings
enrichComponentSettings(get(context), settingsDefinitionMap, {
force: true,
enrichComponentSettings(get(context), settingsDefinitionMap)
}
// Extracts a map of all context keys which are required by action settings
// to provide the functions to evaluate at runtime. This needs done manually
// as the action definitions themselves do not specify bindings for action
// keys, meaning we cannot do this while doing the other normal bindings.
const generateActionKeyMap = (instance, settingsDefinition) => {
let map = {}
settingsDefinition.forEach(setting => {
if (setting.type === "event") {
instance[setting.key]?.forEach(action => {
// We depend on the actual action key
const actionKey = getActionContextKey(action)
if (actionKey) {
map[actionKey] = true
}
// We also depend on any manually declared context keys
getActionDependentContextKeys(action)?.forEach(key => {
map[key] = true
})
})
}
})
return map
}
const runMigrations = (instance, settingsDefinition) => {
@ -381,17 +436,7 @@
}
// Enriches any string component props using handlebars
const enrichComponentSettings = (
context,
settingsDefinitionMap,
options = { force: false }
) => {
const contextChanged = context.key !== lastContextKey
if (!contextChanged && !options?.force) {
return
}
lastContextKey = context.key
const enrichComponentSettings = (context, settingsDefinitionMap) => {
// Record the timestamp so we can reference it after enrichment
latestUpdateTime = Date.now()
const enrichmentTime = latestUpdateTime
@ -506,31 +551,46 @@
})
}
const handleContextChange = key => {
// Check if we already know if this key is used
let used = knownContextKeyMap[key]
// If we don't know, check and cache
if (used == null) {
used = bindingString.indexOf(`[${key}]`) !== -1
knownContextKeyMap[key] = used
}
// Enrich settings if we use this key
if (used) {
enrichComponentSettings($context, settingsDefinitionMap)
}
}
// Register an unregister component instance
onMount(() => {
if (
$appStore.isDevApp &&
!componentStore.actions.isComponentRegistered(id)
) {
componentStore.actions.registerInstance(id, {
component: instance._component,
getSettings: () => cachedSettings,
getRawSettings: () => ({ ...staticSettings, ...dynamicSettings }),
getDataContext: () => get(context),
reload: () => initialise(instance, true),
setEphemeralStyles: styles => (ephemeralStyles = styles),
state: store,
})
if ($appStore.isDevApp) {
if (!componentStore.actions.isComponentRegistered(id)) {
componentStore.actions.registerInstance(id, {
component: instance._component,
getSettings: () => cachedSettings,
getRawSettings: () => ({ ...staticSettings, ...dynamicSettings }),
getDataContext: () => get(context),
reload: () => initialise(instance, true),
setEphemeralStyles: styles => (ephemeralStyles = styles),
state: store,
})
}
return () => {
if (componentStore.actions.isComponentRegistered(id)) {
componentStore.actions.unregisterInstance(id)
}
}
}
})
onDestroy(() => {
if (
$appStore.isDevApp &&
componentStore.actions.isComponentRegistered(id)
) {
componentStore.actions.unregisterInstance(id)
}
})
// Observe changes to context
onMount(() => context.actions.observeChanges(handleContextChange))
</script>
{#if constructor && initialSettings && (visible || inSelectedPath) && !builderHidden}

2
packages/client/src/components/app/DataProvider.svelte
View file

@ -71,7 +71,7 @@
datasource: dataSource || {},
schema,
rowsLength: $fetch.rows.length,
pageNumber: $fetch.pageNumber + 1,
// Undocumented properties. These aren't supposed to be used in builder
// bindings, but are used internally by other components
id: $component?.id,

21
packages/client/src/components/app/EmptyPlaceholder.svelte
View file

@ -1,6 +1,5 @@
<script>
import { getContext } from "svelte"
import { Icon } from "@budibase/bbui"
const component = getContext("component")
const { builderStore, componentStore } = getContext("sdk")
@ -10,15 +9,7 @@
{#if $builderStore.inBuilder}
<div class="component-placeholder">
<Icon name="Help" color="var(--spectrum-global-color-blue-600)" />
<span
class="spectrum-Link"
on:click={() => {
builderStore.actions.requestAddComponent()
}}
>
Add components inside your {definition?.name || $component.type}
</span>
{$component.name || definition?.name || "Component"}
</div>
{/if}
@ -32,14 +23,4 @@
font-size: var(--font-size-s);
gap: var(--spacing-s);
}
/* Common styles for all error states to use */
.component-placeholder :global(mark) {
background-color: var(--spectrum-global-color-gray-400);
padding: 0 4px;
border-radius: 2px;
}
.component-placeholder :global(.spectrum-Link) {
cursor: pointer;
}
</style>

106
packages/client/src/components/app/GridBlock.svelte
View file

@ -19,7 +19,36 @@
export let onRowClick = null
export let buttons = null
// parses columns to fix older formats
const context = getContext("context")
const component = getContext("component")
const {
styleable,
API,
builderStore,
notificationStore,
enrichButtonActions,
ActionTypes,
createContextStore,
Provider,
} = getContext("sdk")
let grid
$: columnWhitelist = parsedColumns
?.filter(col => col.active)
?.map(col => col.field)
$: schemaOverrides = getSchemaOverrides(parsedColumns)
$: enrichedButtons = enrichButtons(buttons)
$: parsedColumns = getParsedColumns(columns)
$: actions = [
{
type: ActionTypes.RefreshDatasource,
callback: () => grid?.getContext()?.rows.actions.refreshData(),
metadata: { dataSource: table },
},
]
// Parses columns to fix older formats
const getParsedColumns = columns => {
// If the first element has an active key all elements should be in the new format
if (columns?.length && columns[0]?.active !== undefined) {
@ -33,28 +62,6 @@
}))
}
$: parsedColumns = getParsedColumns(columns)
const context = getContext("context")
const component = getContext("component")
const {
styleable,
API,
builderStore,
notificationStore,
enrichButtonActions,
ActionTypes,
createContextStore,
} = getContext("sdk")
let grid
$: columnWhitelist = parsedColumns
?.filter(col => col.active)
?.map(col => col.field)
$: schemaOverrides = getSchemaOverrides(parsedColumns)
$: enrichedButtons = enrichButtons(buttons)
const getSchemaOverrides = columns => {
let overrides = {}
columns?.forEach(column => {
@ -78,11 +85,6 @@
const id = get(component).id
const gridContext = createContextStore(context)
gridContext.actions.provideData(id, row)
gridContext.actions.provideAction(
id,
ActionTypes.RefreshDatasource,
() => grid?.getContext()?.rows.actions.refreshData()
)
const fn = enrichButtonActions(settings.onClick, get(gridContext))
return await fn?.({ row })
},
@ -94,29 +96,31 @@
use:styleable={$component.styles}
class:in-builder={$builderStore.inBuilder}
>
<Grid
bind:this={grid}
datasource={table}
{API}
{stripeRows}
{initialFilter}
{initialSortColumn}
{initialSortOrder}
{fixedRowHeight}
{columnWhitelist}
{schemaOverrides}
canAddRows={allowAddRows}
canEditRows={allowEditRows}
canDeleteRows={allowDeleteRows}
canEditColumns={false}
canExpandRows={false}
canSaveSchema={false}
showControls={false}
notifySuccess={notificationStore.actions.success}
notifyError={notificationStore.actions.error}
buttons={enrichedButtons}
on:rowclick={e => onRowClick?.({ row: e.detail })}
/>
<Provider {actions}>
<Grid
bind:this={grid}
datasource={table}
{API}
{stripeRows}
{initialFilter}
{initialSortColumn}
{initialSortOrder}
{fixedRowHeight}
{columnWhitelist}
{schemaOverrides}
canAddRows={allowAddRows}
canEditRows={allowEditRows}
canDeleteRows={allowDeleteRows}
canEditColumns={false}
canExpandRows={false}
canSaveSchema={false}
showControls={false}
notifySuccess={notificationStore.actions.success}
notifyError={notificationStore.actions.error}
buttons={enrichedButtons}
on:rowclick={e => onRowClick?.({ row: e.detail })}
/>
</Provider>
</div>
<style>

4
packages/client/src/components/app/Repeater.svelte
View file

@ -2,6 +2,7 @@
import { getContext } from "svelte"
import Placeholder from "./Placeholder.svelte"
import Container from "./Container.svelte"
import { ContextScopes } from "constants"
export let dataProvider
export let noRowsMessage
@ -9,6 +10,7 @@
export let hAlign
export let vAlign
export let gap
export let scope = ContextScopes.Local
const { Provider } = getContext("sdk")
const component = getContext("component")
@ -22,7 +24,7 @@
<Placeholder />
{:else if rows.length > 0}
{#each rows as row, index}
<Provider data={{ ...row, index }}>
<Provider data={{ ...row, index }} {scope}>
<slot />
</Provider>
{/each}

9
packages/client/src/components/app/blocks/MultiStepFormblock.svelte
View file

@ -1,5 +1,6 @@
<script>
import BlockComponent from "components/BlockComponent.svelte"
import { Helpers } from "@budibase/bbui"
import { getContext, setContext } from "svelte"
import { builderStore } from "stores"
import { Utils } from "@budibase/frontend-core"
@ -41,7 +42,7 @@
let schema
$: fetchSchema(dataSource)
$: enrichedSteps = enrichSteps(steps, schema, $component.id)
$: enrichedSteps = enrichSteps(steps, schema, $component.id, $currentStep)
$: updateCurrentStep(enrichedSteps, $builderStore, $component)
const updateCurrentStep = (steps, builderStore, component) => {
@ -115,6 +116,7 @@
dataSource,
})
return {
_stepId: Helpers.uuid(),
fields: getDefaultFields(fields || [], schema),
title: title ?? defaultProps.title,
desc,
@ -142,7 +144,7 @@
},
}}
>
{#each enrichedSteps as step, stepIdx}
{#each enrichedSteps as step, stepIdx (step._stepId)}
<BlockComponent
type="formstep"
props={{ step: stepIdx + 1, _instanceName: `Step ${stepIdx + 1}` }}
@ -186,12 +188,13 @@
</BlockComponent>
</BlockComponent>
<BlockComponent type="text" props={{ text: step.desc }} order={1} />
<BlockComponent type="container" order={2}>
<div
class="form-block fields"
class:mobile={$context.device.mobile}
>
{#each step.fields as field, fieldIdx (`${field.field || field.name}_${stepIdx}_${fieldIdx}`)}
{#each step.fields as field, fieldIdx (`${field.field || field.name}_${fieldIdx}`)}
{#if getComponentForField(field)}
<BlockComponent
type={getComponentForField(field)}

1
packages/client/src/components/app/blocks/TableBlock.svelte
View file

@ -231,6 +231,7 @@
paginate,
limit: rowCount,
}}
context="provider"
order={1}
>
<BlockComponent

2
packages/client/src/components/app/blocks/form/FormBlockWrapper.svelte
View file

@ -10,6 +10,7 @@
export let noRowsMessage
const component = getContext("component")
const { ContextScopes } = getContext("sdk")
$: providerId = `${$component.id}-provider`
$: dataProvider = `{{ literal ${safe(providerId)} }}`
@ -55,6 +56,7 @@
noRowsMessage: noRowsMessage || "We couldn't find a row to display",
direction: "column",
hAlign: "center",
scope: ContextScopes.Global,
}}
>
<slot />

38
packages/client/src/components/app/forms/Form.svelte
View file

@ -21,6 +21,7 @@
export let editAutoColumns = false
const context = getContext("context")
const component = getContext("component")
const { API, fetchDatasourceSchema } = getContext("sdk")
const getInitialFormStep = () => {
@ -38,28 +39,47 @@
$: fetchSchema(dataSource)
$: schemaKey = generateSchemaKey(schema)
$: initialValues = getInitialValues(actionType, dataSource, $context)
$: initialValues = getInitialValues(
actionType,
dataSource,
$component.path,
$context
)
$: resetKey = Helpers.hashString(
schemaKey + JSON.stringify(initialValues) + disabled + readonly
)
// Returns the closes data context which isn't a built in context
const getInitialValues = (type, dataSource, context) => {
const getInitialValues = (type, dataSource, path, context) => {
// Only inherit values for update forms
if (type !== "Update") {
return {}
}
// Only inherit values for forms targeting internal tables
if (!dataSource?.tableId) {
const dsType = dataSource?.type
if (dsType !== "table" && dsType !== "viewV2") {
return {}
}
// Don't inherit values representing built in contexts
if (["user", "url"].includes(context.closestComponentId)) {
return {}
// Look up the component tree and find something that is provided by an
// ancestor that matches our datasource. This is for backwards compatibility
// as previously we could use the "closest" context.
for (let id of path.reverse().slice(1)) {
// Check for matching view datasource
if (
dataSource.type === "viewV2" &&
context[id]?._viewId === dataSource.id
) {
return context[id]
}
// Check for matching table datasource
if (
dataSource.type === "table" &&
context[id]?.tableId === dataSource.tableId
) {
return context[id]
}
}
// Always inherit the closest datasource
const closestContext = context[`${context.closestComponentId}`] || {}
return closestContext || {}
return {}
}
// Fetches the form schema from this form's dataSource

7
packages/client/src/components/app/forms/RelationshipField.svelte
View file

@ -108,16 +108,13 @@
}
}
$: forceFetchRows(filter, fieldApi)
$: forceFetchRows(filter)
$: debouncedFetchRows(searchTerm, primaryDisplay, defaultValue)
const forceFetchRows = async () => {
if (!fieldApi) {
return
}
// if the filter has changed, then we need to reset the options, clear the selection, and re-fetch
optionsObj = {}
fieldApi.setValue([])
fieldApi?.setValue([])
selectedValue = []
debouncedFetchRows(searchTerm, primaryDisplay, defaultValue)
}

19
packages/client/src/components/context/Provider.svelte
View file

@ -1,21 +1,24 @@
<script>
import { getContext, setContext, onDestroy } from "svelte"
import { dataSourceStore, createContextStore } from "stores"
import { ActionTypes } from "constants"
import { ActionTypes, ContextScopes } from "constants"
import { generate } from "shortid"
export let data
export let actions
export let key
export let scope = ContextScopes.Global
// Clone and create new data context for this component tree
const context = getContext("context")
let context = getContext("context")
const component = getContext("component")
const newContext = createContextStore(context)
setContext("context", newContext)
const providerKey = key || $component.id
// Create a new layer of context if we are only locally scoped
if (scope === ContextScopes.Local) {
context = createContextStore(context)
setContext("context", context)
}
// Generate a permanent unique ID for this component and use it to register
// any datasource actions
const instanceId = generate()
@ -30,7 +33,7 @@
const provideData = newData => {
const dataKey = JSON.stringify(newData)
if (dataKey !== lastDataKey) {
newContext.actions.provideData(providerKey, newData)
context.actions.provideData(providerKey, newData, scope)
lastDataKey = dataKey
}
}
@ -40,7 +43,7 @@
if (actionsKey !== lastActionsKey) {
lastActionsKey = actionsKey
newActions?.forEach(({ type, callback, metadata }) => {
newContext.actions.provideAction(providerKey, type, callback)
context.actions.provideAction(providerKey, type, callback, scope)
// Register any "refresh datasource" actions with a singleton store
// so we can easily refresh data at all levels for any datasource

5
packages/client/src/constants.js
View file

@ -12,5 +12,10 @@ export const ActionTypes = {
ScrollTo: "ScrollTo",
}
export const ContextScopes = {
Local: "local",
Global: "global",
}
export const DNDPlaceholderID = "dnd-placeholder"
export const ScreenslotType = "screenslot"

3
packages/client/src/sdk.js
View file

@ -23,7 +23,7 @@ import { getAction } from "utils/getAction"
import Provider from "components/context/Provider.svelte"
import Block from "components/Block.svelte"
import BlockComponent from "components/BlockComponent.svelte"
import { ActionTypes } from "./constants"
import { ActionTypes, ContextScopes } from "./constants"
import { fetchDatasourceSchema } from "./utils/schema.js"
import { getAPIKey } from "./utils/api.js"
import { enrichButtonActions } from "./utils/buttonActions.js"
@ -54,6 +54,7 @@ export default {
linkable,
getAction,
fetchDatasourceSchema,
ContextScopes,
getAPIKey,
enrichButtonActions,
processStringSync,

123
packages/client/src/stores/context.js
View file

@ -1,59 +1,98 @@
import { writable, derived } from "svelte/store"
import { Helpers } from "@budibase/bbui"
import { ContextScopes } from "constants"
export const createContextStore = oldContext => {
const newContext = writable({})
const contexts = oldContext ? [oldContext, newContext] : [newContext]
export const createContextStore = parentContext => {
const context = writable({})
let observers = []
// Derive the total context state at this point in the tree
const contexts = parentContext ? [parentContext, context] : [context]
const totalContext = derived(contexts, $contexts => {
// The key is the serialized representation of context
let key = ""
for (let i = 0; i < $contexts.length - 1; i++) {
key += $contexts[i].key
}
key = Helpers.hashString(
key + JSON.stringify($contexts[$contexts.length - 1])
)
// Reduce global state
const reducer = (total, context) => ({ ...total, ...context })
const context = $contexts.reduce(reducer, {})
return {
...context,
key,
}
return $contexts.reduce((total, context) => ({ ...total, ...context }), {})
})
// Adds a data context layer to the tree
const provideData = (providerId, data) => {
if (!providerId || data === undefined) {
return
}
newContext.update(state => {
state[providerId] = data
// Keep track of the closest component ID so we can later hydrate a "data" prop.
// This is only required for legacy bindings that used "data" rather than a
// component ID.
state.closestComponentId = providerId
return state
// Subscribe to updates in the parent context, so that we can proxy on any
// change messages to our own subscribers
if (parentContext) {
parentContext.actions.observeChanges(key => {
broadcastChange(key)
})
}
// Adds an action context layer to the tree
const provideAction = (providerId, actionType, callback) => {
// Provide some data in context
const provideData = (providerId, data, scope = ContextScopes.Global) => {
if (!providerId || data === undefined) {
return
}
// Proxy message up the chain if we have a parent and are providing global
// context
if (scope === ContextScopes.Global && parentContext) {
parentContext.actions.provideData(providerId, data, scope)
}
// Otherwise this is either the context root, or we're providing a local
// context override, so we need to update the local context instead
else {
context.update(state => {
state[providerId] = data
return state
})
broadcastChange(providerId)
}
}
// Provides some action in context
const provideAction = (
providerId,
actionType,
callback,
scope = ContextScopes.Global
) => {
if (!providerId || !actionType) {
return
}
newContext.update(state => {
state[`${providerId}_${actionType}`] = callback
return state
})
// Proxy message up the chain if we have a parent and are providing global
// context
if (scope === ContextScopes.Global && parentContext) {
parentContext.actions.provideAction(
providerId,
actionType,
callback,
scope
)
}
// Otherwise this is either the context root, or we're providing a local
// context override, so we need to update the local context instead
else {
const key = `${providerId}_${actionType}`
context.update(state => {
state[key] = callback
return state
})
broadcastChange(key)
}
}
const observeChanges = callback => {
observers.push(callback)
return () => {
observers = observers.filter(cb => cb !== callback)
}
}
const broadcastChange = key => {
observers.forEach(cb => cb(key))
}
return {
subscribe: totalContext.subscribe,
actions: { provideData, provideAction },
actions: {
provideData,
provideAction,
observeChanges,
},
}
}

147
packages/client/src/utils/buttonActions.js
View file

@ -17,6 +17,54 @@ import { ActionTypes } from "constants"
import { enrichDataBindings } from "./enrichDataBinding"
import { Helpers } from "@budibase/bbui"
// Default action handler, which extracts an action from context that was
// provided by another component and executes it with all action parameters
const contextActionHandler = async (action, context) => {
const key = getActionContextKey(action)
const fn = context[key]
if (fn) {
return await fn(action.parameters)
}
}
// Generates the context key, which is the key that this action depends on in
// context to provide the function it will run. This is broken out as a util
// because we reuse this inside the core Component.svelte file to determine
// what the required action context keys are for all action settings.
export const getActionContextKey = action => {
const type = action?.["##eventHandlerType"]
const key = (componentId, type) => `${componentId}_${type}`
switch (type) {
case "Scroll To Field":
return key(action.parameters.componentId, ActionTypes.ScrollTo)
case "Update Field Value":
return key(action.parameters.componentId, ActionTypes.UpdateFieldValue)
case "Validate Form":
return key(action.parameters.componentId, ActionTypes.ValidateForm)
case "Refresh Data Provider":
return key(action.parameters.componentId, ActionTypes.RefreshDatasource)
case "Clear Form":
return key(action.parameters.componentId, ActionTypes.ClearForm)
case "Change Form Step":
return key(action.parameters.componentId, ActionTypes.ChangeFormStep)
default:
return null
}
}
// If button actions depend on context, they must declare which keys they need
export const getActionDependentContextKeys = action => {
const type = action?.["##eventHandlerType"]
switch (type) {
case "Save Row":
case "Duplicate Row":
if (action.parameters?.providerId) {
return [action.parameters.providerId]
}
}
return []
}
const saveRowHandler = async (action, context) => {
const { fields, providerId, tableId, notificationOverride } =
action.parameters
@ -32,20 +80,21 @@ const saveRowHandler = async (action, context) => {
}
}
if (tableId) {
payload.tableId = tableId
if (tableId.startsWith("view")) {
payload._viewId = tableId
} else {
payload.tableId = tableId
}
}
try {
const row = await API.saveRow(payload)
if (!notificationOverride) {
notificationStore.actions.success("Row saved")
}
// Refresh related datasources
await dataSourceStore.actions.invalidateDataSource(tableId, {
invalidateRelationships: true,
})
return { row }
} catch (error) {
// Abort next actions
@ -64,7 +113,11 @@ const duplicateRowHandler = async (action, context) => {
}
}
if (tableId) {
payload.tableId = tableId
if (tableId.startsWith("view")) {
payload._viewId = tableId
} else {
payload.tableId = tableId
}
}
delete payload._id
delete payload._rev
@ -73,12 +126,10 @@ const duplicateRowHandler = async (action, context) => {
if (!notificationOverride) {
notificationStore.actions.success("Row saved")
}
// Refresh related datasources
await dataSourceStore.actions.invalidateDataSource(tableId, {
invalidateRelationships: true,
})
return { row }
} catch (error) {
// Abort next actions
@ -188,17 +239,6 @@ const navigationHandler = action => {
routeStore.actions.navigate(url, peek, externalNewTab)
}
const scrollHandler = async (action, context) => {
return await executeActionHandler(
context,
action.parameters.componentId,
ActionTypes.ScrollTo,
{
field: action.parameters.field,
}
)
}
const queryExecutionHandler = async action => {
const { datasourceId, queryId, queryParams, notificationOverride } =
action.parameters
@ -234,47 +274,6 @@ const queryExecutionHandler = async action => {
}
}
const executeActionHandler = async (
context,
componentId,
actionType,
params
) => {
const fn = context[`${componentId}_${actionType}`]
if (fn) {
return await fn(params)
}
}
const updateFieldValueHandler = async (action, context) => {
return await executeActionHandler(
context,
action.parameters.componentId,
ActionTypes.UpdateFieldValue,
{
type: action.parameters.type,
field: action.parameters.field,
value: action.parameters.value,
}
)
}
const validateFormHandler = async (action, context) => {
return await executeActionHandler(
context,
action.parameters.componentId,
ActionTypes.ValidateForm
)
}
const refreshDataProviderHandler = async (action, context) => {
return await executeActionHandler(
context,
action.parameters.componentId,
ActionTypes.RefreshDatasource
)
}
const logoutHandler = async action => {
await authStore.actions.logOut()
let redirectUrl = "/builder/auth/login"
@ -291,23 +290,6 @@ const logoutHandler = async action => {
}
}
const clearFormHandler = async (action, context) => {
return await executeActionHandler(
context,
action.parameters.componentId,
ActionTypes.ClearForm
)
}
const changeFormStepHandler = async (action, context) => {
return await executeActionHandler(
context,
action.parameters.componentId,
ActionTypes.ChangeFormStep,
action.parameters
)
}
const closeScreenModalHandler = action => {
let url
if (action?.parameters) {
@ -415,16 +397,10 @@ const handlerMap = {
["Duplicate Row"]: duplicateRowHandler,
["Delete Row"]: deleteRowHandler,
["Navigate To"]: navigationHandler,
["Scroll To Field"]: scrollHandler,
["Execute Query"]: queryExecutionHandler,
["Trigger Automation"]: triggerAutomationHandler,
["Validate Form"]: validateFormHandler,
["Update Field Value"]: updateFieldValueHandler,
["Refresh Data Provider"]: refreshDataProviderHandler,
["Log Out"]: logoutHandler,
["Clear Form"]: clearFormHandler,
["Close Screen Modal"]: closeScreenModalHandler,
["Change Form Step"]: changeFormStepHandler,
["Update State"]: updateStateHandler,
["Upload File to S3"]: s3UploadHandler,
["Export Data"]: exportDataHandler,
@ -459,7 +435,12 @@ export const enrichButtonActions = (actions, context) => {
return actions
}
const handlers = actions.map(def => handlerMap[def["##eventHandlerType"]])
// Get handlers for each action. If no bespoke handler is configured, fall
// back to simply executing this action from context.
const handlers = actions.map(def => {
return handlerMap[def["##eventHandlerType"]] || contextActionHandler
})
return async eventContext => {
// Button context is built up as actions are executed.
// Inherit any previous button context which may have come from actions

16
packages/client/src/utils/componentProps.js
View file

@ -23,16 +23,6 @@ export const propsAreSame = (a, b) => {
* Data bindings are enriched, and button actions are enriched.
*/
export const enrichProps = (props, context, settingsDefinitionMap) => {
// Create context of all bindings and data contexts
// Duplicate the closest context as "data" which the builder requires
const totalContext = {
...context,
// This is only required for legacy bindings that used "data" rather than a
// component ID.
data: context[context.closestComponentId],
}
// We want to exclude any button actions from enrichment at this stage.
// Extract top level button action settings.
let normalProps = { ...props }
@ -49,13 +39,13 @@ export const enrichProps = (props, context, settingsDefinitionMap) => {
let rawConditions = normalProps._conditions
// Enrich all props except button actions
let enrichedProps = enrichDataBindings(normalProps, totalContext)
let enrichedProps = enrichDataBindings(normalProps, context)
// Enrich button actions.
// Actions are enriched into a function at this stage, but actual data
// binding enrichment is done dynamically at runtime.
Object.keys(actionProps).forEach(prop => {
enrichedProps[prop] = enrichButtonActions(actionProps[prop], totalContext)
enrichedProps[prop] = enrichButtonActions(actionProps[prop], context)
})
// Conditions
@ -66,7 +56,7 @@ export const enrichProps = (props, context, settingsDefinitionMap) => {
// action
condition.settingValue = enrichButtonActions(
rawConditions[idx].settingValue,
totalContext
context
)
// Since we can't compare functions, we need to assume that conditions

10
packages/frontend-core/src/api/rows.js
View file

@ -19,11 +19,12 @@ export const buildRowEndpoints = API => ({
* @param suppressErrors whether or not to suppress error notifications
*/
saveRow: async (row, suppressErrors = false) => {
if (!row?.tableId) {
const resourceId = row?._viewId || row?.tableId
if (!resourceId) {
return
}
return await API.post({
url: `/api/${row._viewId || row.tableId}/rows`,
url: `/api/${resourceId}/rows`,
body: row,
suppressErrors,
})
@ -35,11 +36,12 @@ export const buildRowEndpoints = API => ({
* @param suppressErrors whether or not to suppress error notifications
*/
patchRow: async (row, suppressErrors = false) => {
if (!row?.tableId && !row?._viewId) {
const resourceId = row?._viewId || row?.tableId
if (!resourceId) {
return
}
return await API.patch({
url: `/api/${row._viewId || row.tableId}/rows`,
url: `/api/${resourceId}/rows`,
body: row,
suppressErrors,
})

4
packages/frontend-core/src/components/grid/cells/HeaderCell.svelte
View file

@ -13,7 +13,7 @@
import { getColumnIcon } from "../lib/utils"
import MigrationModal from "../controls/MigrationModal.svelte"
import { debounce } from "../../../utils/utils"
import { FieldType, FormulaTypes } from "@budibase/types"
import { FieldType, FormulaType } from "@budibase/types"
import { TableNames } from "../../../constants"
export let column
@ -96,7 +96,7 @@
const { type, formulaType } = col.schema
return (
searchableTypes.includes(type) ||
(type === FieldType.FORMULA && formulaType === FormulaTypes.STATIC)
(type === FieldType.FORMULA && formulaType === FormulaType.STATIC)
)
}

3
packages/server/package.json
View file

@ -119,8 +119,8 @@
"@types/google-spreadsheet": "3.1.5",
"@types/jest": "29.5.5",
"@types/koa": "2.13.4",
"@types/koa__router": "8.0.8",
"@types/koa-send": "^4.1.6",
"@types/koa__router": "8.0.8",
"@types/lodash": "4.14.200",
"@types/mssql": "9.1.4",
"@types/node-fetch": "2.6.4",
@ -142,6 +142,7 @@
"rimraf": "3.0.2",
"supertest": "6.3.3",
"swagger-jsdoc": "6.1.0",
"testcontainers": "10.6.0",
"timekeeper": "2.2.0",
"ts-node": "10.8.1",
"tsconfig-paths": "4.0.0",

2
packages/server/scripts/integrations/postgres/docker-compose.yml
View file

@ -2,7 +2,7 @@ version: "3.8"
services:
db:
container_name: postgres
image: postgres:15-bullseye
image: postgres:16.1-bullseye
restart: unless-stopped
environment:
POSTGRES_USER: root

8
packages/server/scripts/test.sh
View file

@ -5,10 +5,10 @@ if [[ -n $CI ]]
then
# Running in ci, where resources are limited
export NODE_OPTIONS="--max-old-space-size=4096"
echo "jest --coverage --maxWorkers=2 --forceExit --workerIdleMemoryLimit=2000MB --bail"
jest --coverage --maxWorkers=2 --forceExit --workerIdleMemoryLimit=2000MB --bail
echo "jest --coverage --maxWorkers=2 --forceExit --workerIdleMemoryLimit=2000MB --bail $@"
jest --coverage --maxWorkers=2 --forceExit --workerIdleMemoryLimit=2000MB --bail $@
else
# --maxWorkers performs better in development
echo "jest --coverage --maxWorkers=2 --forceExit"
jest --coverage --maxWorkers=2 --forceExit
echo "jest --coverage --maxWorkers=2 --forceExit $@"
jest --coverage --maxWorkers=2 --forceExit $@
fi

10
packages/server/specs/resources/table.ts
View file

@ -1,4 +1,4 @@
import { FieldTypes, RelationshipType, FormulaTypes } from "../../src/constants"
import { FieldType, FormulaType, RelationshipType } from "@budibase/types"
import { object } from "./utils"
import Resource from "./utils/Resource"
@ -27,7 +27,7 @@ const table = {
const baseColumnDef = {
type: {
type: "string",
enum: Object.values(FieldTypes),
enum: Object.values(FieldType),
description:
"Defines the type of the column, most explain themselves, a link column is a relationship.",
},
@ -81,7 +81,7 @@ const tableSchema = {
...baseColumnDef,
type: {
type: "string",
enum: [FieldTypes.LINK],
enum: [FieldType.LINK],
description: "A relationship column.",
},
fieldName: {
@ -128,7 +128,7 @@ const tableSchema = {
...baseColumnDef,
type: {
type: "string",
enum: [FieldTypes.FORMULA],
enum: [FieldType.FORMULA],
description: "A formula column.",
},
formula: {
@ -138,7 +138,7 @@ const tableSchema = {
},
formulaType: {
type: "string",
enum: Object.values(FormulaTypes),
enum: Object.values(FormulaType),
description:
"Defines whether this is a static or dynamic formula.",
},

2
packages/server/src/api/controllers/query/import/tests/index.spec.js
View file

@ -42,7 +42,7 @@ const datasets = {
}
describe("Rest Importer", () => {
const config = new TestConfig(false)
const config = new TestConfig()
beforeAll(async () => {
await config.init()

33
packages/server/src/api/controllers/row/ExternalRequest.ts
View file

@ -1,4 +1,5 @@
import {
AutoFieldSubType,
AutoReason,
Datasource,
FieldSchema,
@ -27,7 +28,6 @@ import {
isSQL,
} from "../../../integrations/utils"
import { getDatasourceAndQuery } from "../../../sdk/app/rows/utils"
import { AutoFieldSubTypes, FieldTypes } from "../../../constants"
import { processObjectSync } from "@budibase/string-templates"
import { cloneDeep } from "lodash/fp"
import { processDates, processFormulas } from "../../../utilities/rowProcessor"
@ -111,10 +111,10 @@ function buildFilters(
*/
function cleanupConfig(config: RunConfig, table: Table): RunConfig {
const primaryOptions = [
FieldTypes.STRING,
FieldTypes.LONGFORM,
FieldTypes.OPTIONS,
FieldTypes.NUMBER,
FieldType.STRING,
FieldType.LONGFORM,
FieldType.OPTIONS,
FieldType.NUMBER,
]
// filter out fields which cannot be keys
const fieldNames = Object.entries(table.schema)
@ -241,10 +241,7 @@ function basicProcessing({
function fixArrayTypes(row: Row, table: Table) {
for (let [fieldName, schema] of Object.entries(table.schema)) {
if (
schema.type === FieldTypes.ARRAY &&
typeof row[fieldName] === "string"
) {
if (schema.type === FieldType.ARRAY && typeof row[fieldName] === "string") {
try {
row[fieldName] = JSON.parse(row[fieldName])
} catch (err) {
@ -274,8 +271,8 @@ function isEditableColumn(column: FieldSchema) {
const isExternalAutoColumn =
column.autocolumn &&
column.autoReason !== AutoReason.FOREIGN_KEY &&
column.subtype !== AutoFieldSubTypes.AUTO_ID
const isFormula = column.type === FieldTypes.FORMULA
column.subtype !== AutoFieldSubType.AUTO_ID
const isFormula = column.type === FieldType.FORMULA
return !(isExternalAutoColumn || isFormula)
}
@ -322,11 +319,11 @@ export class ExternalRequest<T extends Operation> {
continue
}
// parse floats/numbers
if (field.type === FieldTypes.NUMBER && !isNaN(parseFloat(row[key]))) {
if (field.type === FieldType.NUMBER && !isNaN(parseFloat(row[key]))) {
newRow[key] = parseFloat(row[key])
}
// if its not a link then just copy it over
if (field.type !== FieldTypes.LINK) {
if (field.type !== FieldType.LINK) {
newRow[key] = row[key]
continue
}
@ -532,7 +529,7 @@ export class ExternalRequest<T extends Operation> {
buildRelationships(table: Table): RelationshipsJson[] {
const relationships = []
for (let [fieldName, field] of Object.entries(table.schema)) {
if (field.type !== FieldTypes.LINK) {
if (field.type !== FieldType.LINK) {
continue
}
const { tableName: linkTableName } = breakExternalTableId(field.tableId)
@ -586,7 +583,7 @@ export class ExternalRequest<T extends Operation> {
// we need this to work out if any relationships need removed
for (const field of Object.values(table.schema)) {
if (
field.type !== FieldTypes.LINK ||
field.type !== FieldType.LINK ||
!field.fieldName ||
isOneSide(field)
) {
@ -730,15 +727,15 @@ export class ExternalRequest<T extends Operation> {
return Object.entries(table.schema)
.filter(
column =>
column[1].type !== FieldTypes.LINK &&
column[1].type !== FieldTypes.FORMULA &&
column[1].type !== FieldType.LINK &&
column[1].type !== FieldType.FORMULA &&
!existing.find((field: string) => field === column[0])
)
.map(column => `${table.name}.${column[0]}`)
}
let fields = extractRealFields(table)
for (let field of Object.values(table.schema)) {
if (field.type !== FieldTypes.LINK || !includeRelations) {
if (field.type !== FieldType.LINK || !includeRelations) {
continue
}
const { tableName: linkTableName } = breakExternalTableId(field.tableId)

4
packages/server/src/api/controllers/row/external.ts
View file

@ -1,4 +1,3 @@
import { FieldTypes } from "../../../constants"
import {
breakExternalTableId,
breakRowIdField,
@ -9,6 +8,7 @@ import {
RunConfig,
} from "./ExternalRequest"
import {
FieldType,
Datasource,
IncludeRelationship,
Operation,
@ -154,7 +154,7 @@ export async function fetchEnrichedRow(ctx: UserCtx) {
// for a single row, there is probably a better way to do this with some smart multi-layer joins
for (let [fieldName, field] of Object.entries(table.schema)) {
if (
field.type !== FieldTypes.LINK ||
field.type !== FieldType.LINK ||
!row[fieldName] ||
row[fieldName].length === 0
) {

4
packages/server/src/api/controllers/row/internal.ts
View file

@ -6,12 +6,12 @@ import {
inputProcessing,
outputProcessing,
} from "../../../utilities/rowProcessor"
import { FieldTypes } from "../../../constants"
import * as utils from "./utils"
import { cloneDeep } from "lodash/fp"
import { context } from "@budibase/backend-core"
import { finaliseRow, updateRelatedFormula } from "./staticFormula"
import {
FieldType,
LinkDocumentValue,
PatchRowRequest,
PatchRowResponse,
@ -225,7 +225,7 @@ export async function fetchEnrichedRow(ctx: UserCtx) {
// insert the link rows in the correct place throughout the main row
for (let fieldName of Object.keys(table.schema)) {
let field = table.schema[fieldName]
if (field.type === FieldTypes.LINK) {
if (field.type === FieldType.LINK) {
// find the links that pertain to this field
const links = linkVals.filter(link => link.fieldName === fieldName)
// find the rows that the links state are linked to this field

16
packages/server/src/api/controllers/row/staticFormula.ts
View file

@ -4,9 +4,15 @@ import {
processAutoColumn,
processFormulas,
} from "../../../utilities/rowProcessor"
import { FieldTypes, FormulaTypes } from "../../../constants"
import { context, locks } from "@budibase/backend-core"
import { Table, Row, LockType, LockName } from "@budibase/types"
import {
Table,
Row,
LockType,
LockName,
FormulaType,
FieldType,
} from "@budibase/types"
import * as linkRows from "../../../db/linkedRows"
import sdk from "../../../sdk"
import isEqual from "lodash/isEqual"
@ -35,7 +41,7 @@ export async function updateRelatedFormula(
let relatedRows: Record<string, Row[]> = {}
for (let [key, field] of Object.entries(enrichedRow)) {
const columnDefinition = table.schema[key]
if (columnDefinition && columnDefinition.type === FieldTypes.LINK) {
if (columnDefinition && columnDefinition.type === FieldType.LINK) {
const relatedTableId = columnDefinition.tableId!
if (!relatedRows[relatedTableId]) {
relatedRows[relatedTableId] = []
@ -63,8 +69,8 @@ export async function updateRelatedFormula(
for (let column of Object.values(relatedTable!.schema)) {
// needs updated in related rows
if (
column.type === FieldTypes.FORMULA &&
column.formulaType === FormulaTypes.STATIC
column.type === FieldType.FORMULA &&
column.formulaType === FormulaType.STATIC
) {
// re-enrich rows for all the related, don't update the related formula for them
promises = promises.concat(

6
packages/server/src/api/controllers/table/bulkFormula.ts
View file

@ -1,4 +1,3 @@
import { FormulaTypes } from "../../../constants"
import { clearColumns } from "./utils"
import { doesContainStrings } from "@budibase/string-templates"
import { cloneDeep } from "lodash/fp"
@ -7,6 +6,7 @@ import uniq from "lodash/uniq"
import { updateAllFormulasInTable } from "../row/staticFormula"
import { context } from "@budibase/backend-core"
import {
FormulaType,
FieldSchema,
FieldType,
FormulaFieldMetadata,
@ -17,10 +17,10 @@ import { isRelationshipColumn } from "../../../db/utils"
function isStaticFormula(
column: FieldSchema
): column is FormulaFieldMetadata & { formulaType: FormulaTypes.STATIC } {
): column is FormulaFieldMetadata & { formulaType: FormulaType.STATIC } {
return (
column.type === FieldType.FORMULA &&
column.formulaType === FormulaTypes.STATIC
column.formulaType === FormulaType.STATIC
)
}

7
packages/server/src/api/controllers/table/tests/utils.spec.ts
View file

@ -1,5 +1,4 @@
import { FieldType } from "@budibase/types"
import { AutoFieldSubTypes } from "../../../../constants"
import { AutoFieldSubType, FieldType } from "@budibase/types"
import TestConfiguration from "../../../../tests/utilities/TestConfiguration"
import { importToRows } from "../utils"
@ -22,7 +21,7 @@ describe("utils", () => {
autoId: {
name: "autoId",
type: FieldType.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
subtype: AutoFieldSubType.AUTO_ID,
autocolumn: true,
constraints: {
type: FieldType.NUMBER,
@ -69,7 +68,7 @@ describe("utils", () => {
autoId: {
name: "autoId",
type: FieldType.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
subtype: AutoFieldSubType.AUTO_ID,
autocolumn: true,
constraints: {
type: FieldType.NUMBER,

13
packages/server/src/api/controllers/table/utils.ts
View file

@ -2,8 +2,6 @@ import { parse, isSchema, isRows } from "../../../utilities/schema"
import { getRowParams, generateRowID, InternalTables } from "../../../db/utils"
import isEqual from "lodash/isEqual"
import {
AutoFieldSubTypes,
FieldTypes,
GOOGLE_SHEETS_PRIMARY_KEY,
USERS_TABLE_SCHEMA,
SwitchableTypes,
@ -19,6 +17,7 @@ import { cloneDeep } from "lodash/fp"
import { quotas } from "@budibase/pro"
import { events, context } from "@budibase/backend-core"
import {
AutoFieldSubType,
ContextUser,
Datasource,
Row,
@ -106,7 +105,7 @@ export function makeSureTableUpToDate(table: Table, tableToSave: Table) {
for ([field, column] of Object.entries(table.schema)) {
if (
column.autocolumn &&
column.subtype === AutoFieldSubTypes.AUTO_ID &&
column.subtype === AutoFieldSubType.AUTO_ID &&
tableToSave.schema[field]
) {
const tableCol = tableToSave.schema[field] as NumberFieldMetadata
@ -144,8 +143,8 @@ export async function importToRows(
? row[fieldName]
: [row[fieldName]]
if (
(schema.type === FieldTypes.OPTIONS ||
schema.type === FieldTypes.ARRAY) &&
(schema.type === FieldType.OPTIONS ||
schema.type === FieldType.ARRAY) &&
row[fieldName]
) {
let merged = [...schema.constraints!.inclusion!, ...rowVal]
@ -403,7 +402,7 @@ export async function checkForViewUpdates(
)
const newViewTemplate = viewTemplate(
viewMetadata,
groupByField?.type === FieldTypes.ARRAY
groupByField?.type === FieldType.ARRAY
)
const viewName = view.name!
await saveView(null, viewName, newViewTemplate)
@ -434,7 +433,7 @@ export function generateJunctionTableName(
export function foreignKeyStructure(keyName: string, meta?: any) {
const structure: any = {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
constraints: {},
name: keyName,
}

4
packages/server/src/api/controllers/view/views.ts
View file

@ -6,8 +6,8 @@ import { fetchView } from "../row"
import { context, events } from "@budibase/backend-core"
import { DocumentType } from "../../../db/utils"
import sdk from "../../../sdk"
import { FieldTypes } from "../../../constants"
import {
FieldType,
Ctx,
Row,
Table,
@ -37,7 +37,7 @@ export async function save(ctx: Ctx) {
(field: any) => field.name == viewToSave.groupBy
)
const view = viewTemplate(viewToSave, groupByField?.type === FieldTypes.ARRAY)
const view = viewTemplate(viewToSave, groupByField?.type === FieldType.ARRAY)
const viewName = viewToSave.name
if (!viewName) {

6
packages/server/src/api/routes/public/tests/compare.spec.ts
View file

@ -12,7 +12,7 @@ let apiKey: string, table: Table, app: App, makeRequest: any
beforeAll(async () => {
app = await config.init()
table = await config.updateTable()
table = await config.upsertTable()
apiKey = await config.generateApiKey()
makeRequest = generateMakeRequest(apiKey)
})
@ -69,7 +69,7 @@ describe("check the applications endpoints", () => {
describe("check the tables endpoints", () => {
it("should allow retrieving tables through search", async () => {
await config.createApp("new app 1")
table = await config.updateTable()
table = await config.upsertTable()
const res = await makeRequest("post", "/tables/search")
expect(res).toSatisfyApiSpec()
})
@ -108,7 +108,7 @@ describe("check the tables endpoints", () => {
describe("check the rows endpoints", () => {
let row: Row
it("should allow retrieving rows through search", async () => {
table = await config.updateTable()
table = await config.upsertTable()
const res = await makeRequest("post", `/tables/${table._id}/rows/search`, {
query: {},
})

98
packages/server/src/api/routes/tests/query.seq.spec.js → packages/server/src/api/routes/tests/query.seq.spec.ts
View file

@ -1,5 +1,4 @@
const tk = require("timekeeper")
tk.freeze(Date.now())
import tk from "timekeeper"
// Mock out postgres for this
jest.mock("pg")
@ -17,16 +16,24 @@ jest.mock("@budibase/backend-core", () => {
},
}
})
const setup = require("./utilities")
const { checkBuilderEndpoint } = require("./utilities/TestFunctions")
const { checkCacheForDynamicVariable } = require("../../../threads/utils")
import * as setup from "./utilities"
import { checkBuilderEndpoint } from "./utilities/TestFunctions"
import { checkCacheForDynamicVariable } from "../../../threads/utils"
const { basicQuery, basicDatasource } = setup.structures
const { events, db: dbCore } = require("@budibase/backend-core")
import { events, db as dbCore } from "@budibase/backend-core"
import { Datasource, Query, SourceName } from "@budibase/types"
tk.freeze(Date.now())
const mockIsProdAppID = dbCore.isProdAppID as jest.MockedFunction<
typeof dbCore.isProdAppID
>
describe("/queries", () => {
let request = setup.getRequest()
let config = setup.getConfig()
let datasource, query
let datasource: Datasource & Required<Pick<Datasource, "_id">>, query: Query
afterAll(setup.afterAll)
@ -40,18 +47,7 @@ describe("/queries", () => {
await setupTest()
})
async function createInvalidIntegration() {
const datasource = await config.createDatasource({
datasource: {
...basicDatasource().datasource,
source: "INVALID_INTEGRATION",
},
})
const query = await config.createQuery()
return { datasource, query }
}
const createQuery = async query => {
const createQuery = async (query: Query) => {
return request
.post(`/api/queries`)
.send(query)
@ -67,7 +63,7 @@ describe("/queries", () => {
jest.clearAllMocks()
const res = await createQuery(query)
expect(res.res.statusMessage).toEqual(
expect((res as any).res.statusMessage).toEqual(
`Query ${query.name} saved successfully.`
)
expect(res.body).toEqual({
@ -92,7 +88,7 @@ describe("/queries", () => {
query._rev = res.body._rev
await createQuery(query)
expect(res.res.statusMessage).toEqual(
expect((res as any).res.statusMessage).toEqual(
`Query ${query.name} saved successfully.`
)
expect(res.body).toEqual({
@ -168,8 +164,8 @@ describe("/queries", () => {
it("should remove sensitive info for prod apps", async () => {
// Mock isProdAppID to pretend we are using a prod app
dbCore.isProdAppID.mockClear()
dbCore.isProdAppID.mockImplementation(() => true)
mockIsProdAppID.mockClear()
mockIsProdAppID.mockImplementation(() => true)
const query = await config.createQuery()
const res = await request
@ -184,7 +180,7 @@ describe("/queries", () => {
// Reset isProdAppID mock
expect(dbCore.isProdAppID).toHaveBeenCalledTimes(1)
dbCore.isProdAppID.mockImplementation(() => false)
mockIsProdAppID.mockImplementation(() => false)
})
})
@ -211,10 +207,11 @@ describe("/queries", () => {
})
it("should apply authorization to endpoint", async () => {
const query = await config.createQuery()
await checkBuilderEndpoint({
config,
method: "DELETE",
url: `/api/queries/${config._id}/${config._rev}`,
url: `/api/queries/${query._id}/${query._rev}`,
})
})
})
@ -272,20 +269,21 @@ describe("/queries", () => {
})
it("should fail with invalid integration type", async () => {
let error
try {
await createInvalidIntegration()
} catch (err) {
error = err
}
expect(error).toBeDefined()
expect(error.message).toBe("No datasource implementation found.")
const response = await config.api.datasource.create(
{
...basicDatasource().datasource,
source: "INVALID_INTEGRATION" as SourceName,
},
{ expectStatus: 500, rawResponse: true }
)
expect(response.body.message).toBe("No datasource implementation found.")
})
})
describe("variables", () => {
async function preview(datasource, fields) {
return config.previewQuery(request, config, datasource, fields)
async function preview(datasource: Datasource, fields: any) {
return config.previewQuery(request, config, datasource, fields, undefined)
}
it("should work with static variables", async () => {
@ -370,11 +368,19 @@ describe("/queries", () => {
})
describe("Current User Request Mapping", () => {
async function previewGet(datasource, fields, params) {
async function previewGet(
datasource: Datasource,
fields: any,
params: any
) {
return config.previewQuery(request, config, datasource, fields, params)
}
async function previewPost(datasource, fields, params) {
async function previewPost(
datasource: Datasource,
fields: any,
params: any
) {
return config.previewQuery(
request,
config,
@ -394,14 +400,18 @@ describe("/queries", () => {
emailHdr: "{{[user].[email]}}",
},
})
const res = await previewGet(datasource, {
path: "www.google.com",
queryString: "email={{[user].[email]}}",
headers: {
queryHdr: "{{[user].[firstName]}}",
secondHdr: "1234",
const res = await previewGet(
datasource,
{
path: "www.google.com",
queryString: "email={{[user].[email]}}",
headers: {
queryHdr: "{{[user].[firstName]}}",
secondHdr: "1234",
},
},
})
undefined
)
const parsedRequest = JSON.parse(res.body.extra.raw)
expect(parsedRequest.opts.headers).toEqual({

16
packages/server/src/api/routes/tests/row.spec.ts
View file

@ -6,11 +6,11 @@ import * as setup from "./utilities"
import { context, InternalTable, roles, tenancy } from "@budibase/backend-core"
import { quotas } from "@budibase/pro"
import {
AutoFieldSubTypes,
AutoFieldSubType,
FieldSchema,
FieldType,
FieldTypeSubtypes,
FormulaTypes,
FormulaType,
INTERNAL_TABLE_SOURCE_ID,
MonthlyQuotaName,
PermissionLevel,
@ -192,7 +192,7 @@ describe.each([
"Row ID": {
name: "Row ID",
type: FieldType.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
subtype: AutoFieldSubType.AUTO_ID,
icon: "ri-magic-line",
autocolumn: true,
constraints: {
@ -581,7 +581,7 @@ describe.each([
tableId: InternalTable.USER_METADATA,
}
let table = await config.api.table.create({
let table = await config.api.table.save({
name: "TestTable",
type: "table",
sourceType: TableSourceType.INTERNAL,
@ -1690,7 +1690,7 @@ describe.each([
tableConfig.sourceType = TableSourceType.EXTERNAL
}
}
const table = await config.api.table.create({
const table = await config.api.table.save({
...tableConfig,
schema: {
...tableConfig.schema,
@ -2032,7 +2032,7 @@ describe.each([
name: "formula",
type: FieldType.FORMULA,
formula: "{{ links.0.name }}",
formulaType: FormulaTypes.DYNAMIC,
formulaType: FormulaType.DYNAMIC,
},
},
}
@ -2086,7 +2086,7 @@ describe.each([
name: "formula",
type: FieldType.FORMULA,
formula: `{{ js "${js}"}}`,
formulaType: FormulaTypes.DYNAMIC,
formulaType: FormulaType.DYNAMIC,
},
},
})
@ -2129,7 +2129,7 @@ describe.each([
name: "formula",
type: FieldType.FORMULA,
formula: `{{ js "${js}"}}`,
formulaType: FormulaTypes.DYNAMIC,
formulaType: FormulaType.DYNAMIC,
},
},
})

14
packages/server/src/api/routes/tests/table.spec.ts
View file

@ -1,6 +1,6 @@
import { context, events } from "@budibase/backend-core"
import {
AutoFieldSubTypes,
AutoFieldSubType,
FieldSubtype,
FieldType,
INTERNAL_TABLE_SOURCE_ID,
@ -205,7 +205,7 @@ describe("/tables", () => {
autoId: {
name: "id",
type: FieldType.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
subtype: AutoFieldSubType.AUTO_ID,
autocolumn: true,
constraints: {
type: "number",
@ -438,7 +438,7 @@ describe("/tables", () => {
})
it("should successfully migrate a one-to-many user relationship to a user column", async () => {
const table = await config.api.table.create({
const table = await config.api.table.save({
name: "table",
type: "table",
sourceId: INTERNAL_TABLE_SOURCE_ID,
@ -496,7 +496,7 @@ describe("/tables", () => {
// We found a bug just after releasing this feature where if the row was created from the
// users table, not the table linking to it, the migration would succeed but lose the data.
// This happened because the order of the documents in the link was reversed.
const table = await config.api.table.create({
const table = await config.api.table.save({
name: "table",
type: "table",
sourceId: INTERNAL_TABLE_SOURCE_ID,
@ -554,7 +554,7 @@ describe("/tables", () => {
})
it("should successfully migrate a many-to-many user relationship to a users column", async () => {
const table = await config.api.table.create({
const table = await config.api.table.save({
name: "table",
type: "table",
sourceId: INTERNAL_TABLE_SOURCE_ID,
@ -611,7 +611,7 @@ describe("/tables", () => {
})
it("should successfully migrate a many-to-one user relationship to a users column", async () => {
const table = await config.api.table.create({
const table = await config.api.table.save({
name: "table",
type: "table",
sourceId: INTERNAL_TABLE_SOURCE_ID,
@ -670,7 +670,7 @@ describe("/tables", () => {
describe("unhappy paths", () => {
let table: Table
beforeAll(async () => {
table = await config.api.table.create({
table = await config.api.table.save({
name: "table",
type: "table",
sourceId: INTERNAL_TABLE_SOURCE_ID,

25
packages/server/src/automations/automationUtils.ts
View file

@ -5,7 +5,7 @@ import {
} from "@budibase/string-templates"
import sdk from "../sdk"
import { Row } from "@budibase/types"
import { LoopStep, LoopStepType, LoopInput } from "../definitions/automations"
import { LoopInput, LoopStep, LoopStepType } from "../definitions/automations"
/**
* When values are input to the system generally they will be of type string as this is required for template strings.
@ -128,23 +128,28 @@ export function substituteLoopStep(hbsString: string, substitute: string) {
}
export function stringSplit(value: string | string[]) {
if (value == null || Array.isArray(value)) {
return value || []
if (value == null) {
return []
}
if (value.split("\n").length > 1) {
value = value.split("\n")
} else {
value = value.split(",")
if (Array.isArray(value)) {
return value
}
return value
if (typeof value !== "string") {
throw new Error(`Unable to split value of type ${typeof value}: ${value}`)
}
const splitOnNewLine = value.split("\n")
if (splitOnNewLine.length > 1) {
return splitOnNewLine
}
return value.split(",")
}
export function typecastForLooping(loopStep: LoopStep, input: LoopInput) {
export function typecastForLooping(input: LoopInput) {
if (!input || !input.binding) {
return null
}
try {
switch (loopStep.inputs.option) {
switch (input.option) {
case LoopStepType.ARRAY:
if (typeof input.binding === "string") {
return JSON.parse(input.binding)

8
packages/server/src/automations/steps/queryRows.ts
View file

@ -1,9 +1,9 @@
import * as rowController from "../../api/controllers/row"
import * as tableController from "../../api/controllers/table"
import { FieldTypes } from "../../constants"
import { buildCtx } from "./utils"
import * as automationUtils from "../automationUtils"
import {
FieldType,
AutomationActionStepId,
AutomationCustomIOType,
AutomationFeature,
@ -115,7 +115,7 @@ function typeCoercion(filters: SearchFilters, table: Table) {
if (!column || typeof value !== "string") {
continue
}
if (column.type === FieldTypes.NUMBER) {
if (column.type === FieldType.NUMBER) {
if (key === "oneOf") {
searchParam[property] = value
.split(",")
@ -148,11 +148,11 @@ export async function run({ inputs, appId }: AutomationStepInput) {
}
}
const table = await getTable(appId, tableId)
let sortType = FieldTypes.STRING
let sortType = FieldType.STRING
if (table && table.schema && table.schema[sortColumn] && sortColumn) {
const fieldType = table.schema[sortColumn].type
sortType =
fieldType === FieldTypes.NUMBER ? FieldTypes.NUMBER : FieldTypes.STRING
fieldType === FieldType.NUMBER ? FieldType.NUMBER : FieldType.STRING
}
const ctx: any = buildCtx(appId, null, {
params: {

20
packages/server/src/automations/tests/loop.spec.ts
View file

@ -3,11 +3,13 @@ import * as triggers from "../triggers"
import { loopAutomation } from "../../tests/utilities/structures"
import { context } from "@budibase/backend-core"
import * as setup from "./utilities"
import { Row, Table } from "@budibase/types"
import { LoopInput, LoopStepType } from "../../definitions/automations"
describe("Attempt to run a basic loop automation", () => {
let config = setup.getConfig(),
table: any,
row: any
table: Table,
row: Row
beforeEach(async () => {
await automation.init()
@ -18,12 +20,12 @@ describe("Attempt to run a basic loop automation", () => {
afterAll(setup.afterAll)
async function runLoop(loopOpts?: any) {
async function runLoop(loopOpts?: LoopInput) {
const appId = config.getAppId()
return await context.doInAppContext(appId, async () => {
const params = { fields: { appId } }
return await triggers.externalTrigger(
loopAutomation(table._id, loopOpts),
loopAutomation(table._id!, loopOpts),
params,
{ getResponses: true }
)
@ -37,9 +39,17 @@ describe("Attempt to run a basic loop automation", () => {
it("test a loop with a string", async () => {
const resp = await runLoop({
type: "String",
option: LoopStepType.STRING,
binding: "a,b,c",
})
expect(resp.steps[2].outputs.iterations).toBe(3)
})
it("test a loop with a binding that returns an integer", async () => {
const resp = await runLoop({
option: LoopStepType.ARRAY,
binding: "{{ 1 }}",
})
expect(resp.steps[2].outputs.iterations).toBe(1)
})
})

4
packages/server/src/automations/tests/updateRow.spec.ts
View file

@ -67,7 +67,7 @@ describe("test the update row action", () => {
tableId: InternalTable.USER_METADATA,
}
let table = await config.api.table.create({
let table = await config.api.table.save({
name: uuid.v4(),
type: "table",
sourceType: TableSourceType.INTERNAL,
@ -120,7 +120,7 @@ describe("test the update row action", () => {
tableId: InternalTable.USER_METADATA,
}
let table = await config.api.table.create({
let table = await config.api.table.save({
name: uuid.v4(),
type: "table",
sourceType: TableSourceType.INTERNAL,

5
packages/server/src/automations/triggers.ts
View file

@ -9,7 +9,7 @@ import * as utils from "./utils"
import env from "../environment"
import { context, db as dbCore } from "@budibase/backend-core"
import { Automation, Row, AutomationData, AutomationJob } from "@budibase/types"
import { executeSynchronously } from "../threads/automation"
import { executeInThread } from "../threads/automation"
export const TRIGGER_DEFINITIONS = definitions
const JOB_OPTS = {
@ -117,8 +117,7 @@ export async function externalTrigger(
appId: context.getAppId(),
automation,
}
const job = { data } as AutomationJob
return executeSynchronously(job)
return executeInThread({ data } as AutomationJob)
} else {
return automationQueue.add(data, JOB_OPTS)
}

42
packages/server/src/automations/unitTests/automationUtils.spec.ts
View file

@ -1,10 +1,15 @@
const automationUtils = require("../automationUtils")
import { LoopStep, LoopStepType } from "../../definitions/automations"
import {
typecastForLooping,
cleanInputValues,
substituteLoopStep,
} from "../automationUtils"
describe("automationUtils", () => {
describe("substituteLoopStep", () => {
it("should allow multiple loop binding substitutes", () => {
expect(
automationUtils.substituteLoopStep(
substituteLoopStep(
`{{ loop.currentItem._id }} {{ loop.currentItem._id }} {{ loop.currentItem._id }}`,
"step.2"
)
@ -15,7 +20,7 @@ describe("automationUtils", () => {
it("should handle not subsituting outside of curly braces", () => {
expect(
automationUtils.substituteLoopStep(
substituteLoopStep(
`loop {{ loop.currentItem._id }}loop loop{{ loop.currentItem._id }}loop`,
"step.2"
)
@ -28,37 +33,20 @@ describe("automationUtils", () => {
describe("typeCastForLooping", () => {
it("should parse to correct type", () => {
expect(
automationUtils.typecastForLooping(
{ inputs: { option: "Array" } },
{ binding: [1, 2, 3] }
)
typecastForLooping({ option: LoopStepType.ARRAY, binding: [1, 2, 3] })
).toEqual([1, 2, 3])
expect(
automationUtils.typecastForLooping(
{ inputs: { option: "Array" } },
{ binding: "[1, 2, 3]" }
)
typecastForLooping({ option: LoopStepType.ARRAY, binding: "[1,2,3]" })
).toEqual([1, 2, 3])
expect(
automationUtils.typecastForLooping(
{ inputs: { option: "String" } },
{ binding: [1, 2, 3] }
)
typecastForLooping({ option: LoopStepType.STRING, binding: [1, 2, 3] })
).toEqual("1,2,3")
})
it("should handle null values", () => {
// expect it to handle where the binding is null
expect(
automationUtils.typecastForLooping(
{ inputs: { option: "Array" } },
{ binding: null }
)
).toEqual(null)
expect(typecastForLooping({ option: LoopStepType.ARRAY })).toEqual(null)
expect(() =>
automationUtils.typecastForLooping(
{ inputs: { option: "Array" } },
{ binding: "test" }
)
typecastForLooping({ option: LoopStepType.ARRAY, binding: "test" })
).toThrow()
})
})
@ -80,7 +68,7 @@ describe("automationUtils", () => {
},
}
expect(
automationUtils.cleanInputValues(
cleanInputValues(
{
row: {
relationship: `[{"_id": "ro_ta_users_us_3"}]`,
@ -113,7 +101,7 @@ describe("automationUtils", () => {
},
}
expect(
automationUtils.cleanInputValues(
cleanInputValues(
{
row: {
relationship: `ro_ta_users_us_3`,

41
packages/server/src/constants/index.ts
View file

@ -1,18 +1,11 @@
import { constants, objectStore, roles } from "@budibase/backend-core"
import {
FieldType as FieldTypes,
FieldType,
INTERNAL_TABLE_SOURCE_ID,
Table,
TableSourceType,
} from "@budibase/types"
export {
FieldType as FieldTypes,
RelationshipType,
AutoFieldSubTypes,
FormulaTypes,
} from "@budibase/types"
export enum FilterTypes {
STRING = "string",
FUZZY = "fuzzy",
@ -36,14 +29,14 @@ export const NoEmptyFilterStrings = [
]
export const CanSwitchTypes = [
[FieldTypes.JSON, FieldTypes.ARRAY],
[FieldType.JSON, FieldType.ARRAY],
[
FieldTypes.STRING,
FieldTypes.OPTIONS,
FieldTypes.LONGFORM,
FieldTypes.BARCODEQR,
FieldType.STRING,
FieldType.OPTIONS,
FieldType.LONGFORM,
FieldType.BARCODEQR,
],
[FieldTypes.BOOLEAN, FieldTypes.NUMBER],
[FieldType.BOOLEAN, FieldType.NUMBER],
]
export const SwitchableTypes = CanSwitchTypes.reduce((prev, current) =>
@ -86,9 +79,9 @@ export const USERS_TABLE_SCHEMA: Table = {
// TODO: ADMIN PANEL - when implemented this doesn't need to be carried out
schema: {
email: {
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
email: true,
length: {
maximum: "",
@ -99,34 +92,34 @@ export const USERS_TABLE_SCHEMA: Table = {
},
firstName: {
name: "firstName",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
presence: false,
},
},
lastName: {
name: "lastName",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
presence: false,
},
},
roleId: {
name: "roleId",
type: FieldTypes.OPTIONS,
type: FieldType.OPTIONS,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
presence: false,
inclusion: Object.values(roles.BUILTIN_ROLE_IDS),
},
},
status: {
name: "status",
type: FieldTypes.OPTIONS,
type: FieldType.OPTIONS,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
presence: false,
inclusion: Object.values(constants.UserStatus),
},

179
packages/server/src/db/defaultData/datasource_bb_default.ts
View file

@ -1,6 +1,4 @@
import {
AutoFieldSubTypes,
FieldTypes,
DEFAULT_BB_DATASOURCE_ID,
DEFAULT_INVENTORY_TABLE_ID,
DEFAULT_EMPLOYEE_TABLE_ID,
@ -16,6 +14,7 @@ import { jobsImport } from "./jobsImport"
import { expensesImport } from "./expensesImport"
import { db as dbCore } from "@budibase/backend-core"
import {
AutoFieldSubType,
FieldType,
RelationshipType,
Row,
@ -40,7 +39,7 @@ function syncLastIds(table: Table, rowCount: number) {
if (
entry.autocolumn &&
entry.type === FieldType.NUMBER &&
entry.subtype == AutoFieldSubTypes.AUTO_ID
entry.subtype == AutoFieldSubType.AUTO_ID
) {
entry.lastID = rowCount
}
@ -58,12 +57,12 @@ async function tableImport(table: Table, data: Row[]) {
const AUTO_COLUMNS: TableSchema = {
"Created At": {
name: "Created At",
type: FieldTypes.DATETIME,
subtype: AutoFieldSubTypes.CREATED_AT,
type: FieldType.DATETIME,
subtype: AutoFieldSubType.CREATED_AT,
icon: "ri-magic-line",
autocolumn: true,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -74,12 +73,12 @@ const AUTO_COLUMNS: TableSchema = {
},
"Updated At": {
name: "Updated At",
type: FieldTypes.DATETIME,
subtype: AutoFieldSubTypes.UPDATED_AT,
type: FieldType.DATETIME,
subtype: AutoFieldSubType.UPDATED_AT,
icon: "ri-magic-line",
autocolumn: true,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -101,12 +100,12 @@ export const DEFAULT_INVENTORY_TABLE_SCHEMA: Table = {
schema: {
"Item ID": {
name: "Item ID",
type: FieldTypes.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
type: FieldType.NUMBER,
subtype: AutoFieldSubType.AUTO_ID,
icon: "ri-magic-line",
autocolumn: true,
constraints: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
presence: false,
numericality: {
greaterThanOrEqualTo: "",
@ -115,9 +114,9 @@ export const DEFAULT_INVENTORY_TABLE_SCHEMA: Table = {
},
},
"Item Name": {
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {
maximum: null,
},
@ -128,9 +127,9 @@ export const DEFAULT_INVENTORY_TABLE_SCHEMA: Table = {
name: "Item Name",
},
"Item Tags": {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
constraints: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
presence: {
allowEmpty: false,
},
@ -140,9 +139,9 @@ export const DEFAULT_INVENTORY_TABLE_SCHEMA: Table = {
sortable: false,
},
Notes: {
type: FieldTypes.LONGFORM,
type: FieldType.LONGFORM,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
@ -150,9 +149,9 @@ export const DEFAULT_INVENTORY_TABLE_SCHEMA: Table = {
useRichText: null,
},
Status: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
constraints: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
presence: {
allowEmpty: false,
},
@ -162,18 +161,18 @@ export const DEFAULT_INVENTORY_TABLE_SCHEMA: Table = {
sortable: false,
},
SKU: {
type: FieldTypes.BARCODEQR,
type: FieldType.BARCODEQR,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
name: "SKU",
},
"Purchase Date": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -185,9 +184,9 @@ export const DEFAULT_INVENTORY_TABLE_SCHEMA: Table = {
ignoreTimezones: true,
},
"Purchase Price": {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
constraints: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
presence: false,
numericality: {
greaterThanOrEqualTo: null,
@ -211,75 +210,75 @@ export const DEFAULT_EMPLOYEE_TABLE_SCHEMA: Table = {
schema: {
"First Name": {
name: "First Name",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
},
"Last Name": {
name: "Last Name",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
},
Email: {
name: "Email",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
},
Address: {
name: "Address",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
},
City: {
name: "City",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
},
Postcode: {
name: "Postcode",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
},
Phone: {
name: "Phone",
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
},
"EMPLOYEE ID": {
name: "EMPLOYEE ID",
type: FieldTypes.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
type: FieldType.NUMBER,
subtype: AutoFieldSubType.AUTO_ID,
icon: "ri-magic-line",
autocolumn: true,
constraints: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
presence: false,
numericality: {
greaterThanOrEqualTo: "",
@ -288,9 +287,9 @@ export const DEFAULT_EMPLOYEE_TABLE_SCHEMA: Table = {
},
},
"Employee Level": {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
constraints: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
presence: false,
inclusion: ["Manager", "Junior", "Senior", "Apprentice", "Contractor"],
},
@ -298,18 +297,18 @@ export const DEFAULT_EMPLOYEE_TABLE_SCHEMA: Table = {
sortable: false,
},
"Badge Photo": {
type: FieldTypes.ATTACHMENT,
type: FieldType.ATTACHMENT,
constraints: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
presence: false,
},
name: "Badge Photo",
sortable: false,
},
Jobs: {
type: FieldTypes.LINK,
type: FieldType.LINK,
constraints: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
presence: false,
},
fieldName: "Assigned",
@ -318,9 +317,9 @@ export const DEFAULT_EMPLOYEE_TABLE_SCHEMA: Table = {
tableId: DEFAULT_JOBS_TABLE_ID,
},
"Start Date": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -332,9 +331,9 @@ export const DEFAULT_EMPLOYEE_TABLE_SCHEMA: Table = {
ignoreTimezones: true,
},
"End Date": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -359,12 +358,12 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
schema: {
"Job ID": {
name: "Job ID",
type: FieldTypes.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
type: FieldType.NUMBER,
subtype: AutoFieldSubType.AUTO_ID,
icon: "ri-magic-line",
autocolumn: true,
constraints: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
presence: false,
numericality: {
greaterThanOrEqualTo: "",
@ -373,9 +372,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
},
},
"Quote Date": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: {
allowEmpty: false,
@ -389,9 +388,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
ignoreTimezones: true,
},
"Quote Price": {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
constraints: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
presence: {
allowEmpty: false,
},
@ -403,9 +402,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
name: "Quote Price",
},
"Works Start": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -417,9 +416,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
ignoreTimezones: true,
},
Address: {
type: FieldTypes.LONGFORM,
type: FieldType.LONGFORM,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
@ -427,9 +426,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
useRichText: null,
},
"Customer Name": {
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {
maximum: null,
},
@ -438,9 +437,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
name: "Customer Name",
},
Notes: {
type: FieldTypes.LONGFORM,
type: FieldType.LONGFORM,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
@ -448,9 +447,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
useRichText: null,
},
"Customer Phone": {
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {
maximum: null,
},
@ -459,9 +458,9 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
name: "Customer Phone",
},
"Customer Email": {
type: FieldTypes.STRING,
type: FieldType.STRING,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {
maximum: null,
},
@ -471,14 +470,14 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
},
Assigned: {
name: "Assigned",
type: FieldTypes.LINK,
type: FieldType.LINK,
tableId: DEFAULT_EMPLOYEE_TABLE_ID,
fieldName: "Jobs",
relationshipType: RelationshipType.MANY_TO_MANY,
// sortable: true,
},
"Works End": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: "string",
length: {},
@ -492,7 +491,7 @@ export const DEFAULT_JOBS_TABLE_SCHEMA: Table = {
ignoreTimezones: true,
},
"Updated Price": {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
constraints: {
type: "number",
presence: false,
@ -518,12 +517,12 @@ export const DEFAULT_EXPENSES_TABLE_SCHEMA: Table = {
schema: {
"Expense ID": {
name: "Expense ID",
type: FieldTypes.NUMBER,
subtype: AutoFieldSubTypes.AUTO_ID,
type: FieldType.NUMBER,
subtype: AutoFieldSubType.AUTO_ID,
icon: "ri-magic-line",
autocolumn: true,
constraints: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
presence: false,
numericality: {
greaterThanOrEqualTo: "",
@ -532,9 +531,9 @@ export const DEFAULT_EXPENSES_TABLE_SCHEMA: Table = {
},
},
"Expense Tags": {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
constraints: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
presence: {
allowEmpty: false,
},
@ -554,9 +553,9 @@ export const DEFAULT_EXPENSES_TABLE_SCHEMA: Table = {
sortable: false,
},
Cost: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
constraints: {
type: FieldTypes.NUMBER,
type: FieldType.NUMBER,
presence: {
allowEmpty: false,
},
@ -568,9 +567,9 @@ export const DEFAULT_EXPENSES_TABLE_SCHEMA: Table = {
name: "Cost",
},
Notes: {
type: FieldTypes.LONGFORM,
type: FieldType.LONGFORM,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
},
@ -578,9 +577,9 @@ export const DEFAULT_EXPENSES_TABLE_SCHEMA: Table = {
useRichText: null,
},
"Payment Due": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -592,9 +591,9 @@ export const DEFAULT_EXPENSES_TABLE_SCHEMA: Table = {
ignoreTimezones: true,
},
"Date Paid": {
type: FieldTypes.DATETIME,
type: FieldType.DATETIME,
constraints: {
type: FieldTypes.STRING,
type: FieldType.STRING,
length: {},
presence: false,
datetime: {
@ -606,9 +605,9 @@ export const DEFAULT_EXPENSES_TABLE_SCHEMA: Table = {
ignoreTimezones: true,
},
Attachment: {
type: FieldTypes.ATTACHMENT,
type: FieldType.ATTACHMENT,
constraints: {
type: FieldTypes.ARRAY,
type: FieldType.ARRAY,
presence: false,
},
name: "Attachment",

18
packages/server/src/db/linkedRows/LinkController.ts
View file

@ -1,6 +1,5 @@
import { IncludeDocs, getLinkDocuments } from "./linkUtils"
import { InternalTables, getUserMetadataParams } from "../utils"
import { FieldTypes } from "../../constants"
import { context, logging } from "@budibase/backend-core"
import LinkDocument from "./LinkDocument"
import {
@ -62,7 +61,7 @@ class LinkController {
}
for (let fieldName of Object.keys(table.schema)) {
const { type } = table.schema[fieldName]
if (type === FieldTypes.LINK) {
if (type === FieldType.LINK) {
return true
}
}
@ -96,7 +95,7 @@ class LinkController {
validateTable(table: Table) {
const usedAlready = []
for (let schema of Object.values(table.schema)) {
if (schema.type !== FieldTypes.LINK) {
if (schema.type !== FieldType.LINK) {
continue
}
const unique = schema.tableId! + schema?.fieldName
@ -172,7 +171,7 @@ class LinkController {
// get the links this row wants to make
const rowField = row[fieldName]
const field = table.schema[fieldName]
if (field.type === FieldTypes.LINK && rowField != null) {
if (field.type === FieldType.LINK && rowField != null) {
// check which links actual pertain to the update in this row
const thisFieldLinkDocs = linkDocs.filter(
linkDoc =>
@ -353,7 +352,7 @@ class LinkController {
const schema = table.schema
for (let fieldName of Object.keys(schema)) {
const field = schema[fieldName]
if (field.type === FieldTypes.LINK && field.fieldName) {
if (field.type === FieldType.LINK && field.fieldName) {
// handle this in a separate try catch, want
// the put to bubble up as an error, if can't update
// table for some reason
@ -366,7 +365,7 @@ class LinkController {
}
const fields = this.handleRelationshipType(field, {
name: field.fieldName,
type: FieldTypes.LINK,
type: FieldType.LINK,
// these are the props of the table that initiated the link
tableId: table._id!,
fieldName: fieldName,
@ -413,10 +412,7 @@ class LinkController {
for (let fieldName of Object.keys(oldTable?.schema || {})) {
const field = oldTable?.schema[fieldName] as FieldSchema
// this field has been removed from the table schema
if (
field.type === FieldTypes.LINK &&
newTable.schema[fieldName] == null
) {
if (field.type === FieldType.LINK && newTable.schema[fieldName] == null) {
await this.removeFieldFromTable(fieldName)
}
}
@ -437,7 +433,7 @@ class LinkController {
for (let fieldName of Object.keys(schema)) {
const field = schema[fieldName]
try {
if (field.type === FieldTypes.LINK && field.fieldName) {
if (field.type === FieldType.LINK && field.fieldName) {
const linkedTable = await this._db.get<Table>(field.tableId)
delete linkedTable.schema[field.fieldName]
field.tableRev = (await this._db.put(linkedTable)).rev

5
packages/server/src/db/linkedRows/LinkDocument.ts
View file

@ -1,6 +1,5 @@
import { generateLinkID } from "../utils"
import { FieldTypes } from "../../constants"
import { LinkDocument } from "@budibase/types"
import { FieldType, LinkDocument } from "@budibase/types"
/**
* Creates a new link document structure which can be put to the database. It is important to
@ -43,7 +42,7 @@ class LinkDocumentImpl implements LinkDocument {
fieldName1,
fieldName2
)
this.type = FieldTypes.LINK
this.type = FieldType.LINK
this.doc1 = {
tableId: tableId1,
fieldName: fieldName1,

6
packages/server/src/db/linkedRows/linkUtils.ts
View file

@ -1,8 +1,8 @@
import { ViewName, getQueryIndex, isRelationshipColumn } from "../utils"
import { FieldTypes } from "../../constants"
import { createLinkView } from "../views/staticViews"
import { context, logging } from "@budibase/backend-core"
import {
FieldType,
DatabaseQueryOpts,
LinkDocument,
LinkDocumentValue,
@ -131,11 +131,11 @@ export async function getLinkedTable(id: string, tables: Table[]) {
export function getRelatedTableForField(table: Table, fieldName: string) {
// look to see if its on the table, straight in the schema
const field = table.schema[fieldName]
if (field?.type === FieldTypes.LINK) {
if (field?.type === FieldType.LINK) {
return field.tableId
}
for (let column of Object.values(table.schema)) {
if (column.type === FieldTypes.LINK && column.fieldName === fieldName) {
if (column.type === FieldType.LINK && column.fieldName === fieldName) {
return column.tableId
}
}

135
packages/server/src/db/tests/linkController.spec.js → packages/server/src/db/tests/linkController.spec.ts
View file

@ -1,17 +1,57 @@
const TestConfig = require("../../tests/utilities/TestConfiguration")
const {
basicRow,
import TestConfig from "../../tests/utilities/TestConfiguration"
import {
basicLinkedRow,
basicRow,
basicTable,
} = require("../../tests/utilities/structures")
const LinkController = require("../linkedRows/LinkController").default
const { context } = require("@budibase/backend-core")
const { RelationshipType } = require("../../constants")
const { cloneDeep } = require("lodash/fp")
} from "../../tests/utilities/structures"
import LinkController from "../linkedRows/LinkController"
import { context } from "@budibase/backend-core"
import {
FieldType,
ManyToManyRelationshipFieldMetadata,
ManyToOneRelationshipFieldMetadata,
OneToManyRelationshipFieldMetadata,
RelationshipFieldMetadata,
RelationshipType,
Row,
Table,
} from "@budibase/types"
import { cloneDeep } from "lodash"
const baseColumn = {
type: FieldType.LINK,
fieldName: "",
tableId: "",
name: "",
}
function mockManyToManyColumn(): ManyToManyRelationshipFieldMetadata {
return <ManyToManyRelationshipFieldMetadata>{
...baseColumn,
through: "",
throughFrom: "",
throughTo: "",
relationshipType: RelationshipType.MANY_TO_MANY,
}
}
function mockManyToOneColumn(): ManyToOneRelationshipFieldMetadata {
return <ManyToOneRelationshipFieldMetadata>{
...baseColumn,
relationshipType: RelationshipType.MANY_TO_ONE,
}
}
function mockOneToManyColumn(): OneToManyRelationshipFieldMetadata {
return <OneToManyRelationshipFieldMetadata>{
...baseColumn,
relationshipType: RelationshipType.ONE_TO_MANY,
}
}
describe("test the link controller", () => {
let config = new TestConfig()
let table1, table2, appId
let table1: Table, table2: Table, appId: string
beforeAll(async () => {
const app = await config.init()
@ -30,9 +70,18 @@ describe("test the link controller", () => {
afterAll(config.end)
async function createLinkController(table, row = null, oldTable = null) {
async function createLinkController(
table: Table,
row?: Row,
oldTable?: Table
) {
return context.doInAppContext(appId, () => {
const linkConfig = {
const linkConfig: {
tableId?: string
table: Table
row?: Row
oldTable?: Table
} = {
tableId: table._id,
table,
}
@ -47,11 +96,11 @@ describe("test the link controller", () => {
}
async function createLinkedRow(linkField = "link", t1 = table1, t2 = table2) {
const row = await config.createRow(basicRow(t2._id))
const row = await config.createRow(basicRow(t2._id!))
const { _id } = await config.createRow(
basicLinkedRow(t1._id, row._id, linkField)
basicLinkedRow(t1._id!, row._id!, linkField)
)
return config.getRow(t1._id, _id)
return config.getRow(t1._id!, _id!)
}
it("should be able to confirm if two table schemas are equal", async () => {
@ -71,6 +120,7 @@ describe("test the link controller", () => {
it("should be able to check the relationship types across two fields", async () => {
const controller = await createLinkController(table1)
// empty case
//@ts-ignore
let output = controller.handleRelationshipType({}, {})
expect(output.linkedField.relationshipType).toEqual(
RelationshipType.MANY_TO_MANY
@ -79,8 +129,8 @@ describe("test the link controller", () => {
RelationshipType.MANY_TO_MANY
)
output = controller.handleRelationshipType(
{ relationshipType: RelationshipType.MANY_TO_MANY },
{}
mockManyToManyColumn(),
{} as any
)
expect(output.linkedField.relationshipType).toEqual(
RelationshipType.MANY_TO_MANY
@ -88,20 +138,14 @@ describe("test the link controller", () => {
expect(output.linkerField.relationshipType).toEqual(
RelationshipType.MANY_TO_MANY
)
output = controller.handleRelationshipType(
{ relationshipType: RelationshipType.MANY_TO_ONE },
{}
)
output = controller.handleRelationshipType(mockManyToOneColumn(), {} as any)
expect(output.linkedField.relationshipType).toEqual(
RelationshipType.ONE_TO_MANY
)
expect(output.linkerField.relationshipType).toEqual(
RelationshipType.MANY_TO_ONE
)
output = controller.handleRelationshipType(
{ relationshipType: RelationshipType.ONE_TO_MANY },
{}
)
output = controller.handleRelationshipType(mockOneToManyColumn(), {} as any)
expect(output.linkedField.relationshipType).toEqual(
RelationshipType.MANY_TO_ONE
)
@ -115,16 +159,16 @@ describe("test the link controller", () => {
const controller = await createLinkController(table1, row)
await context.doInAppContext(appId, async () => {
// get initial count
const beforeLinks = await controller.getRowLinkDocs(row._id)
const beforeLinks = await controller.getRowLinkDocs(row._id!)
await controller.rowDeleted()
let afterLinks = await controller.getRowLinkDocs(row._id)
let afterLinks = await controller.getRowLinkDocs(row._id!)
expect(beforeLinks.length).toEqual(1)
expect(afterLinks.length).toEqual(0)
})
})
it("shouldn't throw an error when deleting a row with no links", async () => {
const row = await config.createRow(basicRow(table1._id))
const row = await config.createRow(basicRow(table1._id!))
const controller = await createLinkController(table1, row)
await context.doInAppContext(appId, async () => {
let error
@ -142,12 +186,13 @@ describe("test the link controller", () => {
const copyTable = {
...table1,
}
//@ts-ignore
copyTable.schema.otherTableLink = {
type: "link",
type: FieldType.LINK,
fieldName: "link",
tableId: table2._id,
tableId: table2._id!,
}
let error
let error: any
try {
controller.validateTable(copyTable)
} catch (err) {
@ -166,7 +211,7 @@ describe("test the link controller", () => {
const controller = await createLinkController(table1, row)
await context.doInAppContext(appId, async () => {
await controller.rowSaved()
let links = await controller.getRowLinkDocs(row._id)
let links = await controller.getRowLinkDocs(row._id!)
expect(links.length).toEqual(0)
})
})
@ -186,7 +231,7 @@ describe("test the link controller", () => {
it("should be able to remove a linked field from a table", async () => {
await createLinkedRow()
await createLinkedRow("link2")
const controller = await createLinkController(table1, null, table1)
const controller = await createLinkController(table1, undefined, table1)
await context.doInAppContext(appId, async () => {
let before = await controller.getTableLinkDocs()
await controller.removeFieldFromTable("link")
@ -199,7 +244,8 @@ describe("test the link controller", () => {
it("should throw an error when overwriting a link column", async () => {
const update = cloneDeep(table1)
update.schema.link.relationshipType = RelationshipType.MANY_TO_ONE
const linkSchema = update.schema.link as ManyToOneRelationshipFieldMetadata
linkSchema.relationshipType = RelationshipType.MANY_TO_ONE
let error
try {
const controller = await createLinkController(update)
@ -215,7 +261,7 @@ describe("test the link controller", () => {
await createLinkedRow()
const newTable = cloneDeep(table1)
delete newTable.schema.link
const controller = await createLinkController(newTable, null, table1)
const controller = await createLinkController(newTable, undefined, table1)
await context.doInAppContext(appId, async () => {
await controller.tableUpdated()
const links = await controller.getTableLinkDocs()
@ -235,7 +281,7 @@ describe("test the link controller", () => {
let error
try {
// create another row to initiate the error
await config.createRow(basicLinkedRow(row.tableId, row.link[0]))
await config.createRow(basicLinkedRow(row.tableId!, row.link[0]))
} catch (err) {
error = err
}
@ -245,7 +291,7 @@ describe("test the link controller", () => {
it("should not error if a link being created doesn't exist", async () => {
let error
try {
await config.createRow(basicLinkedRow(table1._id, "invalid"))
await config.createRow(basicLinkedRow(table1._id!, "invalid"))
} catch (err) {
error = err
}
@ -255,10 +301,11 @@ describe("test the link controller", () => {
it("make sure auto column goes onto other row too", async () => {
const table = await config.createTable()
const tableCfg = basicTable()
//@ts-ignore
tableCfg.schema.link = {
type: "link",
type: FieldType.LINK,
fieldName: "link",
tableId: table._id,
tableId: table._id!,
name: "link",
autocolumn: true,
}
@ -269,21 +316,23 @@ describe("test the link controller", () => {
it("should be able to link to self", async () => {
const table = await config.createTable()
//@ts-ignore
table.schema.link = {
type: "link",
type: FieldType.LINK,
fieldName: "link",
tableId: table._id,
tableId: table._id!,
name: "link",
autocolumn: true,
}
await config.updateTable(table)
await config.upsertTable(table)
})
it("should be able to remove a linked field from a table, even if the linked table does not exist", async () => {
await createLinkedRow()
await createLinkedRow("link2")
table1.schema["link"].tableId = "not_found"
const controller = await createLinkController(table1, null, table1)
const linkSchema = table1.schema["link"] as RelationshipFieldMetadata
linkSchema.tableId = "not_found"
const controller = await createLinkController(table1, undefined, table1)
await context.doInAppContext(appId, async () => {
let before = await controller.getTableLinkDocs()
await controller.removeFieldFromTable("link")

23
packages/server/src/db/tests/linkTests.spec.js → packages/server/src/db/tests/linkTests.spec.ts
View file

@ -1,14 +1,15 @@
const TestConfig = require("../../tests/utilities/TestConfiguration")
const { basicTable } = require("../../tests/utilities/structures")
const linkUtils = require("../linkedRows/linkUtils")
const { context } = require("@budibase/backend-core")
import TestConfig from "../../tests/utilities/TestConfiguration"
import { basicTable } from "../../tests/utilities/structures"
import * as linkUtils from "../linkedRows/linkUtils"
import { context } from "@budibase/backend-core"
import { FieldType, RelationshipType, Table } from "@budibase/types"
describe("test link functionality", () => {
const config = new TestConfig()
let appId
let appId: string
describe("getLinkedTable", () => {
let table
let table: Table
beforeAll(async () => {
const app = await config.init()
appId = app.appId
@ -17,15 +18,15 @@ describe("test link functionality", () => {
it("should be able to retrieve a linked table from a list", async () => {
await context.doInAppContext(appId, async () => {
const retrieved = await linkUtils.getLinkedTable(table._id, [table])
const retrieved = await linkUtils.getLinkedTable(table._id!, [table])
expect(retrieved._id).toBe(table._id)
})
})
it("should be able to retrieve a table from DB and update list", async () => {
const tables = []
const tables: Table[] = []
await context.doInAppContext(appId, async () => {
const retrieved = await linkUtils.getLinkedTable(table._id, tables)
const retrieved = await linkUtils.getLinkedTable(table._id!, tables)
expect(retrieved._id).toBe(table._id)
expect(tables[0]).toBeDefined()
})
@ -35,9 +36,11 @@ describe("test link functionality", () => {
describe("getRelatedTableForField", () => {
let link = basicTable()
link.schema.link = {
name: "link",
relationshipType: RelationshipType.ONE_TO_MANY,
fieldName: "otherLink",
tableId: "tableID",
type: "link",
type: FieldType.LINK,
}
it("should get the field from the table directly", () => {

4
packages/server/src/db/utils.ts
View file

@ -1,6 +1,7 @@
import newid from "./newid"
import { db as dbCore } from "@budibase/backend-core"
import {
FieldType,
DocumentType,
FieldSchema,
RelationshipFieldMetadata,
@ -8,7 +9,6 @@ import {
INTERNAL_TABLE_SOURCE_ID,
DatabaseQueryOpts,
} from "@budibase/types"
import { FieldTypes } from "../constants"
export { DocumentType, VirtualDocumentType } from "@budibase/types"
@ -315,5 +315,5 @@ export function extractViewInfoFromID(viewId: string) {
export function isRelationshipColumn(
column: FieldSchema
): column is RelationshipFieldMetadata {
return column.type === FieldTypes.LINK
return column.type === FieldType.LINK
}

10
packages/server/src/definitions/automations.ts
View file

@ -6,14 +6,14 @@ export enum LoopStepType {
}
export interface LoopStep extends AutomationStep {
inputs: {
option: LoopStepType
[key: string]: any
}
inputs: LoopInput
}
export interface LoopInput {
binding: string[] | string
option: LoopStepType
binding?: string[] | string | number[]
iterations?: string
failure?: any
}
export interface TriggerOutput {

2
packages/server/src/integrations/base/sqlTable.ts
View file

@ -1,5 +1,6 @@
import { Knex, knex } from "knex"
import {
RelationshipType,
FieldSubtype,
NumberFieldMetadata,
Operation,
@ -11,7 +12,6 @@ import {
import { breakExternalTableId } from "../utils"
import SchemaBuilder = Knex.SchemaBuilder
import CreateTableBuilder = Knex.CreateTableBuilder
import { RelationshipType } from "../../constants"
import { utils } from "@budibase/shared-core"
function isIgnoredType(type: FieldType) {

4
packages/server/src/integrations/oracle.ts
View file

@ -1,4 +1,5 @@
import {
FieldType,
DatasourceFieldType,
Integration,
Operation,
@ -21,7 +22,6 @@ import {
SqlClient,
} from "./utils"
import Sql from "./base/sql"
import { FieldTypes } from "../constants"
import {
BindParameters,
Connection,
@ -302,7 +302,7 @@ class OracleIntegration extends Sql implements DatasourcePlus {
})
if (this.isBooleanType(oracleColumn)) {
fieldSchema.type = FieldTypes.BOOLEAN
fieldSchema.type = FieldType.BOOLEAN
}
table.schema[columnName] = fieldSchema

14
packages/server/src/integrations/tests/utils/postgres.ts
View file

@ -1,27 +1,23 @@
import { Datasource, SourceName } from "@budibase/types"
import { GenericContainer, Wait, StartedTestContainer } from "testcontainers"
import env from "../../../environment"
let container: StartedTestContainer | undefined
const isMac = process.platform === "darwin"
export async function getDsConfig(): Promise<Datasource> {
try {
if (!container) {
// postgres 15-bullseye safer bet on Linux
const version = isMac ? undefined : "15-bullseye"
container = await new GenericContainer("postgres", version)
container = await new GenericContainer("postgres:16.1-bullseye")
.withExposedPorts(5432)
.withEnv("POSTGRES_PASSWORD", "password")
.withEnvironment({ POSTGRES_PASSWORD: "password" })
.withWaitStrategy(
Wait.forLogMessage(
"PostgreSQL init process complete; ready for start up."
"database system is ready to accept connections",
2
)
)
.start()
}
const host = container.getContainerIpAddress()
const host = container.getHost()
const port = container.getMappedPort(5432)
return {

2
packages/server/src/migrations/functions/usageQuotas/tests/syncRows.spec.ts
View file

@ -5,7 +5,7 @@ import { QuotaUsageType, StaticQuotaName } from "@budibase/types"
import { db as dbCore, context } from "@budibase/backend-core"
describe("syncRows", () => {
let config = new TestConfig(false)
const config = new TestConfig()
beforeEach(async () => {
await config.init()

5
packages/server/src/sdk/app/rows/attachments.ts
View file

@ -1,7 +1,6 @@
import { CouchFindOptions, Table, Row } from "@budibase/types"
import { FieldType, CouchFindOptions, Table, Row } from "@budibase/types"
import { db as dbCore } from "@budibase/backend-core"
import { DocumentType, SEPARATOR } from "../../../db/utils"
import { FieldTypes } from "../../../constants"
// default limit - seems to work well for performance
export const FIND_LIMIT = 25
@ -31,7 +30,7 @@ export async function getRowsWithAttachments(appId: string, table: Table) {
const db = dbCore.getDB(appId)
const attachmentCols: string[] = []
for (let [key, column] of Object.entries(table.schema)) {
if (column.type === FieldTypes.ATTACHMENT) {
if (column.type === FieldType.ATTACHMENT) {
attachmentCols.push(key)
}
}

13
packages/server/src/sdk/app/rows/search/tests/external.spec.ts
View file

@ -16,7 +16,6 @@ import {
expectAnyExternalColsAttributes,
generator,
} from "@budibase/backend-core/tests"
import datasource from "../../../../../api/routes/datasource"
jest.unmock("mysql2/promise")
@ -30,13 +29,15 @@ describe.skip("external", () => {
beforeAll(async () => {
const container = await new GenericContainer("mysql")
.withExposedPorts(3306)
.withEnv("MYSQL_ROOT_PASSWORD", "admin")
.withEnv("MYSQL_DATABASE", "db")
.withEnv("MYSQL_USER", "user")
.withEnv("MYSQL_PASSWORD", "password")
.withEnvironment({
MYSQL_ROOT_PASSWORD: "admin",
MYSQL_DATABASE: "db",
MYSQL_USER: "user",
MYSQL_PASSWORD: "password",
})
.start()
const host = container.getContainerIpAddress()
const host = container.getHost()
const port = container.getMappedPort(3306)
await config.init()

14
packages/server/src/sdk/app/rows/tests/internal.spec.ts
View file

@ -7,11 +7,11 @@ import {
TableSourceType,
FieldType,
Table,
AutoFieldSubTypes,
AutoFieldSubType,
AutoColumnFieldMetadata,
} from "@budibase/types"
import TestConfiguration from "../../../../tests/utilities/TestConfiguration"
import { cache } from "@budibase/backend-core"
tk.freeze(Date.now())
@ -117,7 +117,7 @@ describe("sdk >> rows >> internal", () => {
id: {
name: "id",
type: FieldType.AUTO,
subtype: AutoFieldSubTypes.AUTO_ID,
subtype: AutoFieldSubType.AUTO_ID,
autocolumn: true,
lastID: 0,
},
@ -181,7 +181,7 @@ describe("sdk >> rows >> internal", () => {
id: {
name: "id",
type: FieldType.AUTO,
subtype: AutoFieldSubTypes.AUTO_ID,
subtype: AutoFieldSubType.AUTO_ID,
autocolumn: true,
lastID: 0,
},
@ -213,8 +213,10 @@ describe("sdk >> rows >> internal", () => {
)
const persistedTable = await config.getTable(table._id)
expect((table as any).schema.id.lastID).toBe(0)
expect(persistedTable.schema.id.lastID).toBe(20)
expect((table.schema.id as AutoColumnFieldMetadata).lastID).toBe(0)
expect((persistedTable.schema.id as AutoColumnFieldMetadata).lastID).toBe(
20
)
})
})
})

15
packages/server/src/sdk/app/rows/utils.ts
View file

@ -1,7 +1,6 @@
import cloneDeep from "lodash/cloneDeep"
import validateJs from "validate.js"
import { Row, Table, TableSchema } from "@budibase/types"
import { FieldTypes } from "../../../constants"
import { FieldType, Row, Table, TableSchema } from "@budibase/types"
import { makeExternalQuery } from "../../../integrations/base/query"
import { Format } from "../../../api/controllers/view/exporters"
import sdk from "../.."
@ -22,7 +21,7 @@ export function cleanExportRows(
let cleanRows = [...rows]
const relationships = Object.entries(schema)
.filter((entry: any[]) => entry[1].type === FieldTypes.LINK)
.filter((entry: any[]) => entry[1].type === FieldType.LINK)
.map(entry => entry[0])
relationships.forEach(column => {
@ -88,17 +87,17 @@ export async function validate({
continue
}
// formulas shouldn't validated, data will be deleted anyway
if (type === FieldTypes.FORMULA || column.autocolumn) {
if (type === FieldType.FORMULA || column.autocolumn) {
continue
}
// special case for options, need to always allow unselected (empty)
if (type === FieldTypes.OPTIONS && constraints?.inclusion) {
if (type === FieldType.OPTIONS && constraints?.inclusion) {
constraints.inclusion.push(null as any, "")
}
let res
// Validate.js doesn't seem to handle array
if (type === FieldTypes.ARRAY && row[fieldName]) {
if (type === FieldType.ARRAY && row[fieldName]) {
if (row[fieldName].length) {
if (!Array.isArray(row[fieldName])) {
row[fieldName] = row[fieldName].split(",")
@ -116,13 +115,13 @@ export async function validate({
errors[fieldName] = [`${fieldName} is required`]
}
} else if (
(type === FieldTypes.ATTACHMENT || type === FieldTypes.JSON) &&
(type === FieldType.ATTACHMENT || type === FieldType.JSON) &&
typeof row[fieldName] === "string"
) {
// this should only happen if there is an error
try {
const json = JSON.parse(row[fieldName])
if (type === FieldTypes.ATTACHMENT) {
if (type === FieldType.ATTACHMENT) {
if (Array.isArray(json)) {
row[fieldName] = json
} else {

4
packages/server/src/sdk/app/tables/external/index.ts vendored
View file

@ -1,4 +1,5 @@
import {
FieldType,
Operation,
RelationshipType,
RenameColumn,
@ -14,7 +15,6 @@ import {
setStaticSchemas,
} from "../../../../api/controllers/table/utils"
import { cloneDeep } from "lodash/fp"
import { FieldTypes } from "../../../../constants"
import { makeTableRequest } from "../../../../api/controllers/table/ExternalRequest"
import {
isRelationshipSetup,
@ -78,7 +78,7 @@ export async function save(
// check if relations need setup
for (let schema of Object.values(tableToSave.schema)) {
if (schema.type !== FieldTypes.LINK || isRelationshipSetup(schema)) {
if (schema.type !== FieldType.LINK || isRelationshipSetup(schema)) {
continue
}
const schemaTableId = schema.tableId

3
packages/server/src/sdk/app/tables/external/utils.ts vendored
View file

@ -9,7 +9,6 @@ import {
Table,
TableSourceType,
} from "@budibase/types"
import { FieldTypes } from "../../../../constants"
import {
foreignKeyStructure,
generateForeignKey,
@ -27,7 +26,7 @@ export function cleanupRelationships(
// clean up relationships in couch table schemas
for (let [key, schema] of Object.entries(tableToIterate.schema)) {
if (
schema.type === FieldTypes.LINK &&
schema.type === FieldType.LINK &&
(!oldTable || table.schema[key] == null)
) {
const schemaTableId = schema.tableId

4
packages/server/src/sdk/app/tables/internal/index.ts
View file

@ -1,4 +1,5 @@
import {
FieldType,
RenameColumn,
Table,
ViewStatisticsSchema,
@ -10,7 +11,6 @@ import {
hasTypeChanged,
TableSaveFunctions,
} from "../../../../api/controllers/table/utils"
import { FieldTypes } from "../../../../constants"
import { EventType, updateLinks } from "../../../../db/linkedRows"
import { cloneDeep } from "lodash/fp"
import isEqual from "lodash/isEqual"
@ -63,7 +63,7 @@ export async function save(
}
// rename row fields when table column is renamed
if (renaming && table.schema[renaming.updated].type === FieldTypes.LINK) {
if (renaming && table.schema[renaming.updated].type === FieldType.LINK) {
throw new Error("Cannot rename a linked column.")
}

2
packages/server/src/sdk/tests/tables.spec.ts
View file

@ -9,7 +9,7 @@ describe("tables", () => {
beforeAll(async () => {
await config.init()
table = await config.api.table.create(basicTable())
table = await config.api.table.save(basicTable())
})
describe("getTables", () => {

100
packages/server/src/tests/utilities/TestConfiguration.ts
View file

@ -27,7 +27,18 @@ import {
sessions,
tenancy,
} from "@budibase/backend-core"
import * as controllers from "./controllers"
import {
app as appController,
deploy as deployController,
role as roleController,
automation as automationController,
webhook as webhookController,
query as queryController,
screen as screenController,
layout as layoutController,
view as viewController,
} from "./controllers"
import { cleanup } from "../../utilities/fileSystem"
import newid from "../../db/newid"
import { generateUserMetadataID } from "../../db/utils"
@ -44,13 +55,14 @@ import {
RelationshipFieldMetadata,
RelationshipType,
Row,
SearchFilters,
SearchParams,
SourceName,
Table,
TableSourceType,
User,
UserRoles,
View,
WithRequired,
} from "@budibase/types"
import API from "./api"
@ -543,11 +555,7 @@ class TestConfiguration {
// clear any old app
this.appId = null
this.app = await context.doInTenant(this.tenantId!, async () => {
const app = await this._req(
{ name: appName },
null,
controllers.app.create
)
const app = await this._req({ name: appName }, null, appController.create)
this.appId = app.appId!
return app
})
@ -563,7 +571,7 @@ class TestConfiguration {
}
async publish() {
await this._req(null, null, controllers.deploy.publishApp)
await this._req(null, null, deployController.publishApp)
// @ts-ignore
const prodAppId = this.getAppId().replace("_dev", "")
this.prodAppId = prodAppId
@ -578,7 +586,7 @@ class TestConfiguration {
const response = await this._req(
null,
{ appId: this.appId },
controllers.app.unpublish
appController.unpublish
)
this.prodAppId = null
this.prodApp = null
@ -587,14 +595,16 @@ class TestConfiguration {
// TABLE
async updateTable(
async upsertTable(
config?: TableToBuild,
{ skipReassigning } = { skipReassigning: false }
): Promise<Table> {
config = config || basicTable()
config.sourceType = config.sourceType || TableSourceType.INTERNAL
config.sourceId = config.sourceId || INTERNAL_TABLE_SOURCE_ID
const response = await this._req(config, null, controllers.table.save)
const response = await this.api.table.save({
...config,
sourceType: config.sourceType || TableSourceType.INTERNAL,
sourceId: config.sourceId || INTERNAL_TABLE_SOURCE_ID,
})
if (!skipReassigning) {
this.table = response
}
@ -612,7 +622,7 @@ class TestConfiguration {
if (!config.sourceId) {
config.sourceId = INTERNAL_TABLE_SOURCE_ID
}
return this.updateTable(config, options)
return this.upsertTable(config, options)
}
async createExternalTable(
@ -627,12 +637,12 @@ class TestConfiguration {
config.sourceId = this.datasource._id
config.sourceType = TableSourceType.EXTERNAL
}
return this.updateTable(config, options)
return this.upsertTable(config, options)
}
async getTable(tableId?: string) {
tableId = tableId || this.table!._id!
return this._req(null, { tableId }, controllers.table.find)
return this.api.table.get(tableId)
}
async createLinkedTable(
@ -680,37 +690,35 @@ class TestConfiguration {
if (!this.table) {
throw "Test requires table to be configured."
}
const tableId = (config && config.tableId) || this.table._id
const tableId = (config && config.tableId) || this.table._id!
config = config || basicRow(tableId!)
return this._req(config, { tableId }, controllers.row.save)
return this.api.row.save(tableId, config)
}
async getRow(tableId: string, rowId: string): Promise<Row> {
return this._req(null, { tableId, rowId }, controllers.row.find)
const res = await this.api.row.get(tableId, rowId)
return res.body
}
async getRows(tableId: string) {
if (!tableId && this.table) {
tableId = this.table._id!
}
return this._req(null, { tableId }, controllers.row.fetch)
return this.api.row.fetch(tableId)
}
async searchRows(tableId: string, searchParams: SearchFilters = {}) {
async searchRows(tableId: string, searchParams?: SearchParams) {
if (!tableId && this.table) {
tableId = this.table._id!
}
const body = {
query: searchParams,
}
return this._req(body, { tableId }, controllers.row.search)
return this.api.row.search(tableId, searchParams)
}
// ROLE
async createRole(config?: any) {
config = config || basicRole()
return this._req(config, null, controllers.role.save)
return this._req(config, null, roleController.save)
}
// VIEW
@ -723,7 +731,7 @@ class TestConfiguration {
tableId: this.table!._id,
name: generator.guid(),
}
return this._req(view, null, controllers.view.v1.save)
return this._req(view, null, viewController.v1.save)
}
async createView(
@ -753,13 +761,13 @@ class TestConfiguration {
delete config._rev
}
this.automation = (
await this._req(config, null, controllers.automation.create)
await this._req(config, null, automationController.create)
).automation
return this.automation
}
async getAllAutomations() {
return this._req(null, null, controllers.automation.fetch)
return this._req(null, null, automationController.fetch)
}
async deleteAutomation(automation?: any) {
@ -770,7 +778,7 @@ class TestConfiguration {
return this._req(
null,
{ id: automation._id, rev: automation._rev },
controllers.automation.destroy
automationController.destroy
)
}
@ -779,28 +787,27 @@ class TestConfiguration {
throw "Must create an automation before creating webhook."
}
config = config || basicWebhook(this.automation._id)
return (await this._req(config, null, controllers.webhook.save)).webhook
return (await this._req(config, null, webhookController.save)).webhook
}
// DATASOURCE
async createDatasource(config?: {
datasource: Datasource
}): Promise<Datasource> {
}): Promise<WithRequired<Datasource, "_id">> {
config = config || basicDatasource()
const response = await this._req(config, null, controllers.datasource.save)
this.datasource = response.datasource
return this.datasource!
const response = await this.api.datasource.create(config.datasource)
this.datasource = response
return { ...this.datasource, _id: this.datasource!._id! }
}
async updateDatasource(datasource: Datasource): Promise<Datasource> {
const response = await this._req(
datasource,
{ datasourceId: datasource._id },
controllers.datasource.update
)
this.datasource = response.datasource
return this.datasource!
async updateDatasource(
datasource: Datasource
): Promise<WithRequired<Datasource, "_id">> {
const response = await this.api.datasource.update(datasource)
this.datasource = response
return { ...this.datasource, _id: this.datasource!._id! }
}
async restDatasource(cfg?: any) {
@ -815,6 +822,7 @@ class TestConfiguration {
async dynamicVariableDatasource() {
let datasource = await this.restDatasource()
const basedOnQuery = await this.createQuery({
...basicQuery(datasource._id!),
fields: {
@ -886,21 +894,21 @@ class TestConfiguration {
throw "No datasource created for query."
}
config = config || basicQuery(this.datasource!._id!)
return this._req(config, null, controllers.query.save)
return this._req(config, null, queryController.save)
}
// SCREEN
async createScreen(config?: any) {
config = config || basicScreen()
return this._req(config, null, controllers.screen.save)
return this._req(config, null, screenController.save)
}
// LAYOUT
async createLayout(config?: any) {
config = config || basicLayout()
return await this._req(config, null, controllers.layout.save)
return await this._req(config, null, layoutController.save)
}
}

18
packages/server/src/tests/utilities/api/datasource.ts
View file

@ -2,20 +2,23 @@ import {
CreateDatasourceRequest,
Datasource,
VerifyDatasourceRequest,
VerifyDatasourceResponse,
} from "@budibase/types"
import TestConfiguration from "../TestConfiguration"
import { TestAPI } from "./base"
import supertest from "supertest"
export class DatasourceAPI extends TestAPI {
constructor(config: TestConfiguration) {
super(config)
}
create = async (
create = async <B extends boolean = false>(
config: Datasource,
{ expectStatus } = { expectStatus: 200 }
): Promise<Datasource> => {
{
expectStatus,
rawResponse,
}: { expectStatus?: number; rawResponse?: B } = {}
): Promise<B extends false ? Datasource : supertest.Response> => {
const body: CreateDatasourceRequest = {
datasource: config,
tablesFilter: [],
@ -25,8 +28,11 @@ export class DatasourceAPI extends TestAPI {
.send(body)
.set(this.config.defaultHeaders())
.expect("Content-Type", /json/)
.expect(expectStatus)
return result.body.datasource as Datasource
.expect(expectStatus || 200)
if (rawResponse) {
return result as any
}
return result.body.datasource
}
update = async (

3
packages/server/src/tests/utilities/api/row.ts
View file

@ -7,6 +7,7 @@ import {
BulkImportRequest,
BulkImportResponse,
SearchRowResponse,
SearchParams,
} from "@budibase/types"
import TestConfiguration from "../TestConfiguration"
import { TestAPI } from "./base"
@ -154,10 +155,12 @@ export class RowAPI extends TestAPI {
search = async (
sourceId: string,
params?: SearchParams,
{ expectStatus } = { expectStatus: 200 }
): Promise<SearchRowResponse> => {
const request = this.request
.post(`/api/${sourceId}/search`)
.send(params)
.set(this.config.defaultHeaders())
.expect(expectStatus)

2
packages/server/src/tests/utilities/api/table.ts
View file

@ -13,7 +13,7 @@ export class TableAPI extends TestAPI {
super(config)
}
create = async (
save = async (
data: SaveTableRequest,
{ expectStatus } = { expectStatus: 200 }
): Promise<SaveTableResponse> => {

14
packages/server/src/tests/utilities/structures.ts
View file

@ -21,8 +21,9 @@ import {
Table,
INTERNAL_TABLE_SOURCE_ID,
TableSourceType,
AutomationIOType,
Query,
} from "@budibase/types"
import { LoopInput, LoopStepType } from "../../definitions/automations"
const { BUILTIN_ROLE_IDS } = roles
@ -204,10 +205,13 @@ export function serverLogAutomation(appId?: string): Automation {
}
}
export function loopAutomation(tableId: string, loopOpts?: any): Automation {
export function loopAutomation(
tableId: string,
loopOpts?: LoopInput
): Automation {
if (!loopOpts) {
loopOpts = {
option: "Array",
option: LoopStepType.ARRAY,
binding: "{{ steps.1.rows }}",
}
}
@ -360,7 +364,7 @@ export function basicDatasource(): { datasource: Datasource } {
}
}
export function basicQuery(datasourceId: string) {
export function basicQuery(datasourceId: string): Query {
return {
datasourceId: datasourceId,
name: "New Query",
@ -368,6 +372,8 @@ export function basicQuery(datasourceId: string) {
fields: {},
schema: {},
queryVerb: "read",
transformer: null,
readable: true,
}
}

29
packages/server/src/threads/automation.ts
View file

@ -43,22 +43,19 @@ const CRON_STEP_ID = triggerDefs.CRON.stepId
const STOPPED_STATUS = { success: true, status: AutomationStatus.STOPPED }
function getLoopIterations(loopStep: LoopStep) {
let binding = loopStep.inputs.binding
const binding = loopStep.inputs.binding
if (!binding) {
return 0
}
const isString = typeof binding === "string"
try {
if (isString) {
binding = JSON.parse(binding)
const json = typeof binding === "string" ? JSON.parse(binding) : binding
if (Array.isArray(json)) {
return json.length
}
} catch (err) {
// ignore error - wasn't able to parse
}
if (Array.isArray(binding)) {
return binding.length
}
if (isString) {
if (typeof binding === "string") {
return automationUtils.stringSplit(binding).length
}
return 0
@ -256,7 +253,7 @@ class Orchestrator {
this._context.env = await sdkUtils.getEnvironmentVariables()
let automation = this._automation
let stopped = false
let loopStep: AutomationStep | undefined = undefined
let loopStep: LoopStep | undefined = undefined
let stepCount = 0
let loopStepNumber: any = undefined
@ -311,7 +308,7 @@ class Orchestrator {
stepCount++
if (step.stepId === LOOP_STEP_ID) {
loopStep = step
loopStep = step as LoopStep
loopStepNumber = stepCount
continue
}
@ -331,7 +328,6 @@ class Orchestrator {
}
try {
loopStep.inputs.binding = automationUtils.typecastForLooping(
loopStep as LoopStep,
loopStep.inputs as LoopInput
)
} catch (err) {
@ -348,7 +344,7 @@ class Orchestrator {
loopStep = undefined
break
}
let item = []
let item: any[] = []
if (
typeof loopStep.inputs.binding === "string" &&
loopStep.inputs.option === "String"
@ -399,7 +395,8 @@ class Orchestrator {
if (
index === env.AUTOMATION_MAX_ITERATIONS ||
index === parseInt(loopStep.inputs.iterations)
(loopStep.inputs.iterations &&
index === parseInt(loopStep.inputs.iterations))
) {
this.updateContextAndOutput(
loopStepNumber,
@ -615,7 +612,7 @@ export function execute(job: Job<AutomationData>, callback: WorkerCallback) {
})
}
export function executeSynchronously(job: Job) {
export async function executeInThread(job: Job<AutomationData>) {
const appId = job.data.event.appId
if (!appId) {
throw new Error("Unable to execute, event doesn't contain app ID.")
@ -627,10 +624,10 @@ export function executeSynchronously(job: Job) {
}, job.data.event.timeout || 12000)
})
return context.doInAppContext(appId, async () => {
return await context.doInAppContext(appId, async () => {
const envVars = await sdkUtils.getEnvironmentVariables()
// put into automation thread for whole context
return context.doInEnvironmentContext(envVars, async () => {
return await context.doInEnvironmentContext(envVars, async () => {
const automationOrchestrator = new Orchestrator(job)
return await Promise.race([
automationOrchestrator.execute(),

16
packages/server/src/utilities/rowProcessor/attachments.ts
View file

@ -1,6 +1,12 @@
import { FieldTypes, ObjectStoreBuckets } from "../../constants"
import { ObjectStoreBuckets } from "../../constants"
import { context, db as dbCore, objectStore } from "@budibase/backend-core"
import { RenameColumn, Row, RowAttachment, Table } from "@budibase/types"
import {
FieldType,
RenameColumn,
Row,
RowAttachment,
Table,
} from "@budibase/types"
export class AttachmentCleanup {
static async coreCleanup(fileListFn: () => string[]): Promise<void> {
@ -28,7 +34,7 @@ export class AttachmentCleanup {
let files: string[] = []
const tableSchema = opts.oldTable?.schema || table.schema
for (let [key, schema] of Object.entries(tableSchema)) {
if (schema.type !== FieldTypes.ATTACHMENT) {
if (schema.type !== FieldType.ATTACHMENT) {
continue
}
const columnRemoved = opts.oldTable && !table.schema[key]
@ -62,7 +68,7 @@ export class AttachmentCleanup {
return AttachmentCleanup.coreCleanup(() => {
let files: string[] = []
for (let [key, schema] of Object.entries(table.schema)) {
if (schema.type !== FieldTypes.ATTACHMENT) {
if (schema.type !== FieldType.ATTACHMENT) {
continue
}
rows.forEach(row => {
@ -79,7 +85,7 @@ export class AttachmentCleanup {
return AttachmentCleanup.coreCleanup(() => {
let files: string[] = []
for (let [key, schema] of Object.entries(table.schema)) {
if (schema.type !== FieldTypes.ATTACHMENT) {
if (schema.type !== FieldType.ATTACHMENT) {
continue
}
const oldKeys =

30
packages/server/src/utilities/rowProcessor/index.ts
View file

@ -1,10 +1,16 @@
import * as linkRows from "../../db/linkedRows"
import { FieldTypes, AutoFieldSubTypes } from "../../constants"
import { processFormulas, fixAutoColumnSubType } from "./utils"
import { objectStore, utils } from "@budibase/backend-core"
import { InternalTables } from "../../db/utils"
import { TYPE_TRANSFORM_MAP } from "./map"
import { FieldSubtype, Row, RowAttachment, Table } from "@budibase/types"
import {
FieldType,
AutoFieldSubType,
FieldSubtype,
Row,
RowAttachment,
Table,
} from "@budibase/types"
import { cloneDeep } from "lodash/fp"
import {
processInputBBReferences,
@ -54,25 +60,25 @@ export function processAutoColumn(
schema = fixAutoColumnSubType(schema)
}
switch (schema.subtype) {
case AutoFieldSubTypes.CREATED_BY:
case AutoFieldSubType.CREATED_BY:
if (creating && shouldUpdateUserFields && userId) {
row[key] = [userId]
}
break
case AutoFieldSubTypes.CREATED_AT:
case AutoFieldSubType.CREATED_AT:
if (creating) {
row[key] = now
}
break
case AutoFieldSubTypes.UPDATED_BY:
case AutoFieldSubType.UPDATED_BY:
if (shouldUpdateUserFields && userId) {
row[key] = [userId]
}
break
case AutoFieldSubTypes.UPDATED_AT:
case AutoFieldSubType.UPDATED_AT:
row[key] = now
break
case AutoFieldSubTypes.AUTO_ID:
case AutoFieldSubType.AUTO_ID:
if (creating) {
schema.lastID = !schema.lastID ? BASE_AUTO_ID : schema.lastID + 1
row[key] = schema.lastID
@ -134,7 +140,7 @@ export async function inputProcessing(
continue
}
// remove any formula values, they are to be generated
if (field.type === FieldTypes.FORMULA) {
if (field.type === FieldType.FORMULA) {
delete clonedRow[key]
}
// otherwise coerce what is there to correct types
@ -143,7 +149,7 @@ export async function inputProcessing(
}
// remove any attachment urls, they are generated on read
if (field.type === FieldTypes.ATTACHMENT) {
if (field.type === FieldType.ATTACHMENT) {
const attachments = clonedRow[key]
if (attachments?.length) {
attachments.forEach((attachment: RowAttachment) => {
@ -152,7 +158,7 @@ export async function inputProcessing(
}
}
if (field.type === FieldTypes.BB_REFERENCE && value) {
if (field.type === FieldType.BB_REFERENCE && value) {
clonedRow[key] = await processInputBBReferences(
value,
field.subtype as FieldSubtype
@ -214,7 +220,7 @@ export async function outputProcessing<T extends Row[] | Row>(
// process complex types: attachements, bb references...
for (let [property, column] of Object.entries(table.schema)) {
if (column.type === FieldTypes.ATTACHMENT) {
if (column.type === FieldType.ATTACHMENT) {
for (let row of enriched) {
if (row[property] == null || !Array.isArray(row[property])) {
continue
@ -227,7 +233,7 @@ export async function outputProcessing<T extends Row[] | Row>(
}
} else if (
!opts.skipBBReferences &&
column.type == FieldTypes.BB_REFERENCE
column.type == FieldType.BB_REFERENCE
) {
for (let row of enriched) {
row[property] = await processOutputBBReferences(

67
packages/server/src/utilities/rowProcessor/map.ts
View file

@ -1,7 +1,6 @@
// @ts-nocheck
import { FieldTypes } from "../../constants"
import { FieldType } from "@budibase/types"
const parseArrayString = value => {
const parseArrayString = (value: any) => {
if (typeof value === "string") {
if (value === "") {
return []
@ -21,11 +20,13 @@ const parseArrayString = value => {
* A map of how we convert various properties in rows to each other based on the row type.
*/
export const TYPE_TRANSFORM_MAP: any = {
[FieldTypes.LINK]: {
[FieldType.LINK]: {
"": [],
//@ts-ignore
[null]: [],
//@ts-ignore
[undefined]: undefined,
parse: link => {
parse: (link: any) => {
if (Array.isArray(link) && typeof link[0] === "object") {
return link.map(el => (el && el._id ? el._id : el))
}
@ -35,75 +36,97 @@ export const TYPE_TRANSFORM_MAP: any = {
return link
},
},
[FieldTypes.OPTIONS]: {
[FieldType.OPTIONS]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
},
[FieldTypes.ARRAY]: {
[FieldType.ARRAY]: {
//@ts-ignore
[null]: [],
//@ts-ignore
[undefined]: undefined,
parse: parseArrayString,
},
[FieldTypes.STRING]: {
[FieldType.STRING]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
},
[FieldTypes.BARCODEQR]: {
[FieldType.BARCODEQR]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
},
[FieldTypes.FORMULA]: {
[FieldType.FORMULA]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
},
[FieldTypes.LONGFORM]: {
[FieldType.LONGFORM]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
},
[FieldTypes.NUMBER]: {
[FieldType.NUMBER]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
parse: n => parseFloat(n),
parse: (n: any) => parseFloat(n),
},
[FieldTypes.BIGINT]: {
[FieldType.BIGINT]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
},
[FieldTypes.DATETIME]: {
[FieldType.DATETIME]: {
"": null,
[undefined]: undefined,
//@ts-ignore
[null]: null,
parse: date => {
//@ts-ignore
[undefined]: undefined,
parse: (date: any) => {
if (date instanceof Date) {
return date.toISOString()
}
return date
},
},
[FieldTypes.ATTACHMENT]: {
[FieldType.ATTACHMENT]: {
//@ts-ignore
[null]: [],
//@ts-ignore
[undefined]: undefined,
parse: parseArrayString,
},
[FieldTypes.BOOLEAN]: {
[FieldType.BOOLEAN]: {
"": null,
//@ts-ignore
[null]: null,
//@ts-ignore
[undefined]: undefined,
true: true,
false: false,
},
[FieldTypes.AUTO]: {
[FieldType.AUTO]: {
parse: () => undefined,
},
[FieldTypes.JSON]: {
parse: input => {
[FieldType.JSON]: {
parse: (input: any) => {
try {
if (input === "") {
return undefined

21
packages/server/src/utilities/rowProcessor/tests/utils.spec.ts
View file

@ -1,6 +1,11 @@
import { fixAutoColumnSubType } from "../utils"
import { AutoFieldDefaultNames, AutoFieldSubTypes } from "../../../constants"
import { FieldSchema, FieldType, RelationshipType } from "@budibase/types"
import { AutoFieldDefaultNames } from "../../../constants"
import {
AutoFieldSubType,
FieldSchema,
FieldType,
RelationshipType,
} from "@budibase/types"
describe("rowProcessor utility", () => {
describe("fixAutoColumnSubType", () => {
@ -20,37 +25,37 @@ describe("rowProcessor utility", () => {
it("updates the schema with the correct subtype", async () => {
schema.name = AutoFieldDefaultNames.CREATED_BY
expect(fixAutoColumnSubType(schema).subtype).toEqual(
AutoFieldSubTypes.CREATED_BY
AutoFieldSubType.CREATED_BY
)
schema.subtype = undefined
schema.name = AutoFieldDefaultNames.UPDATED_BY
expect(fixAutoColumnSubType(schema).subtype).toEqual(
AutoFieldSubTypes.UPDATED_BY
AutoFieldSubType.UPDATED_BY
)
schema.subtype = undefined
schema.name = AutoFieldDefaultNames.CREATED_AT
expect(fixAutoColumnSubType(schema).subtype).toEqual(
AutoFieldSubTypes.CREATED_AT
AutoFieldSubType.CREATED_AT
)
schema.subtype = undefined
schema.name = AutoFieldDefaultNames.UPDATED_AT
expect(fixAutoColumnSubType(schema).subtype).toEqual(
AutoFieldSubTypes.UPDATED_AT
AutoFieldSubType.UPDATED_AT
)
schema.subtype = undefined
schema.name = AutoFieldDefaultNames.AUTO_ID
expect(fixAutoColumnSubType(schema).subtype).toEqual(
AutoFieldSubTypes.AUTO_ID
AutoFieldSubType.AUTO_ID
)
schema.subtype = undefined
})
it("returns the column if subtype exists", async () => {
schema.subtype = AutoFieldSubTypes.CREATED_BY
schema.subtype = AutoFieldSubType.CREATED_BY
schema.name = AutoFieldDefaultNames.CREATED_AT
expect(fixAutoColumnSubType(schema)).toEqual(schema)
})

26
packages/server/src/utilities/rowProcessor/utils.ts
View file

@ -1,15 +1,13 @@
import {
AutoFieldDefaultNames,
AutoFieldSubTypes,
FieldTypes,
FormulaTypes,
} from "../../constants"
import { AutoFieldDefaultNames } from "../../constants"
import { processStringSync } from "@budibase/string-templates"
import {
AutoColumnFieldMetadata,
FieldSchema,
Row,
Table,
FormulaType,
AutoFieldSubType,
FieldType,
} from "@budibase/types"
import tracer from "dd-trace"
@ -30,15 +28,15 @@ export function fixAutoColumnSubType(
}
// the columns which get auto generated
if (column.name.endsWith(AutoFieldDefaultNames.CREATED_BY)) {
column.subtype = AutoFieldSubTypes.CREATED_BY
column.subtype = AutoFieldSubType.CREATED_BY
} else if (column.name.endsWith(AutoFieldDefaultNames.UPDATED_BY)) {
column.subtype = AutoFieldSubTypes.UPDATED_BY
column.subtype = AutoFieldSubType.UPDATED_BY
} else if (column.name.endsWith(AutoFieldDefaultNames.CREATED_AT)) {
column.subtype = AutoFieldSubTypes.CREATED_AT
column.subtype = AutoFieldSubType.CREATED_AT
} else if (column.name.endsWith(AutoFieldDefaultNames.UPDATED_AT)) {
column.subtype = AutoFieldSubTypes.UPDATED_AT
column.subtype = AutoFieldSubType.UPDATED_AT
} else if (column.name.endsWith(AutoFieldDefaultNames.AUTO_ID)) {
column.subtype = AutoFieldSubTypes.AUTO_ID
column.subtype = AutoFieldSubType.AUTO_ID
}
return column
}
@ -57,11 +55,11 @@ export function processFormulas<T extends Row | Row[]>(
const rows = Array.isArray(inputRows) ? inputRows : [inputRows]
if (rows) {
for (let [column, schema] of Object.entries(table.schema)) {
if (schema.type !== FieldTypes.FORMULA) {
if (schema.type !== FieldType.FORMULA) {
continue
}
const isStatic = schema.formulaType === FormulaTypes.STATIC
const isStatic = schema.formulaType === FormulaType.STATIC
if (
schema.formula == null ||
@ -100,7 +98,7 @@ export function processDates<T extends Row | Row[]>(
let rows = Array.isArray(inputRows) ? inputRows : [inputRows]
let datesWithTZ: string[] = []
for (let [column, schema] of Object.entries(table.schema)) {
if (schema.type !== FieldTypes.DATETIME) {
if (schema.type !== FieldType.DATETIME) {
continue
}
if (!schema.timeOnly && !schema.ignoreTimezones) {

26
packages/server/src/utilities/schema.ts
View file

@ -1,12 +1,11 @@
import { FieldSubtype } from "@budibase/types"
import { FieldTypes } from "../constants"
import { FieldType, FieldSubtype } from "@budibase/types"
import { ValidColumnNameRegex, utils } from "@budibase/shared-core"
import { db } from "@budibase/backend-core"
import { parseCsvExport } from "../api/controllers/view/exporters"
interface SchemaColumn {
readonly name: string
readonly type: FieldTypes
readonly type: FieldType
readonly subtype: FieldSubtype
readonly autocolumn?: boolean
readonly constraints?: {
@ -36,13 +35,13 @@ interface ValidationResults {
}
const PARSERS: any = {
[FieldTypes.NUMBER]: (attribute?: string) => {
[FieldType.NUMBER]: (attribute?: string) => {
if (!attribute) {
return attribute
}
return Number(attribute)
},
[FieldTypes.DATETIME]: (attribute?: string) => {
[FieldType.DATETIME]: (attribute?: string) => {
if (!attribute) {
return attribute
}
@ -60,7 +59,7 @@ export function isSchema(schema: any): schema is Schema {
column !== null &&
typeof column === "object" &&
typeof column.type === "string" &&
Object.values(FieldTypes).includes(column.type as FieldTypes)
Object.values(FieldType).includes(column.type as FieldType)
)
})
)
@ -110,20 +109,17 @@ export function validate(rows: Rows, schema: Schema): ValidationResults {
isAutoColumn
) {
return
} else if (
columnType === FieldTypes.NUMBER &&
isNaN(Number(columnData))
) {
} else if (columnType === FieldType.NUMBER && isNaN(Number(columnData))) {
// If provided must be a valid number
results.schemaValidation[columnName] = false
} else if (
// If provided must be a valid date
columnType === FieldTypes.DATETIME &&
columnType === FieldType.DATETIME &&
isNaN(new Date(columnData).getTime())
) {
results.schemaValidation[columnName] = false
} else if (
columnType === FieldTypes.BB_REFERENCE &&
columnType === FieldType.BB_REFERENCE &&
!isValidBBReference(columnData, columnSubtype)
) {
results.schemaValidation[columnName] = false
@ -155,15 +151,15 @@ export function parse(rows: Rows, schema: Schema): Rows {
const columnType = schema[columnName].type
const columnSubtype = schema[columnName].subtype
if (columnType === FieldTypes.NUMBER) {
if (columnType === FieldType.NUMBER) {
// If provided must be a valid number
parsedRow[columnName] = columnData ? Number(columnData) : columnData
} else if (columnType === FieldTypes.DATETIME) {
} else if (columnType === FieldType.DATETIME) {
// If provided must be a valid date
parsedRow[columnName] = columnData
? new Date(columnData).toISOString()
: columnData
} else if (columnType === FieldTypes.BB_REFERENCE) {
} else if (columnType === FieldType.BB_REFERENCE) {
const parsedValues =
!!columnData && parseCsvExport<{ _id: string }[]>(columnData)
if (!parsedValues) {

430
packages/string-templates/manifest.json
View file

File diff suppressed because it is too large Load diff

2
packages/string-templates/package.json
View file

@ -25,7 +25,7 @@
"manifest": "node ./scripts/gen-collection-info.js"
},
"dependencies": {
"@budibase/handlebars-helpers": "^0.13.0",
"@budibase/handlebars-helpers": "^0.13.1",
"dayjs": "^1.10.8",
"handlebars": "^4.7.6",
"lodash.clonedeep": "^4.5.0",

7
packages/string-templates/scripts/gen-collection-info.js
View file

@ -10,8 +10,8 @@ const marked = require("marked")
* https://github.com/budibase/handlebars-helpers
*/
const { join } = require("path")
const path = require("path")
const DIRECTORY = join(__dirname, "..", "..", "..")
const COLLECTIONS = [
"math",
"array",
@ -115,6 +115,8 @@ function getCommentInfo(file, func) {
docs.example = docs.example.replace("product", "multiply")
}
docs.description = blocks[0].trim()
docs.acceptsBlock = docs.tags.some(el => el.title === "block")
docs.acceptsInline = docs.tags.some(el => el.title === "inline")
return docs
}
@ -127,7 +129,7 @@ function run() {
const foundNames = []
for (let collection of COLLECTIONS) {
const collectionFile = fs.readFileSync(
`${DIRECTORY}/node_modules/${HELPER_LIBRARY}/lib/${collection}.js`,
`${path.dirname(require.resolve(HELPER_LIBRARY))}/lib/${collection}.js`,
"utf8"
)
const collectionInfo = {}
@ -159,6 +161,7 @@ function run() {
numArgs: args.length,
example: jsDocInfo.example || undefined,
description: jsDocInfo.description,
requiresBlock: jsDocInfo.acceptsBlock && !jsDocInfo.acceptsInline,
})
}
outputJSON[collection] = collectionInfo

2
packages/string-templates/src/helpers/date.js
View file

@ -115,7 +115,7 @@ module.exports.duration = (str, pattern, format) => {
setLocale(config.str, config.pattern)
const duration = dayjs.duration(config.str, config.pattern)
if (!isOptions(format)) {
if (format && !isOptions(format)) {
return duration.format(format)
} else {
return duration.humanize()

9
packages/string-templates/src/helpers/list.js
View file

@ -3,6 +3,8 @@ const helperList = require("@budibase/handlebars-helpers")
let helpers = undefined
const helpersToRemove = ["sortBy"]
module.exports.getHelperList = () => {
if (helpers) {
return helpers
@ -15,12 +17,17 @@ module.exports.getHelperList = () => {
}
for (let collection of constructed) {
for (let [key, func] of Object.entries(collection)) {
helpers[key] = func
// Handlebars injects the hbs options to the helpers by default. We are adding an empty {} as a last parameter to simulate it
helpers[key] = (...props) => func(...props, {})
}
}
for (let key of Object.keys(externalHandlebars.addedHelpers)) {
helpers[key] = externalHandlebars.addedHelpers[key]
}
for (const toRemove of helpersToRemove) {
delete helpers[toRemove]
}
Object.freeze(helpers)
return helpers
}

97
packages/string-templates/test/manifest.spec.js
View file

@ -16,21 +16,55 @@ jest.mock("@budibase/handlebars-helpers/lib/uuid", () => {
})
const fs = require("fs")
const { processString } = require("../src/index.cjs")
const {
processString,
convertToJS,
processStringSync,
encodeJSBinding,
} = require("../src/index.cjs")
const tk = require("timekeeper")
const { getHelperList } = require("../src/helpers")
tk.freeze("2021-01-21T12:00:00")
const processJS = (js, context) => {
return processStringSync(encodeJSBinding(js), context)
}
const manifest = JSON.parse(
fs.readFileSync(require.resolve("../manifest.json"), "utf8")
)
const collections = Object.keys(manifest)
const examples = collections.reduce((acc, collection) => {
const functions = Object.keys(manifest[collection]).filter(
fnc => manifest[collection][fnc].example
)
if (functions.length) {
const functions = Object.entries(manifest[collection])
.filter(([_, details]) => details.example)
.map(([name, details]) => {
const example = details.example
let [hbs, js] = example.split("->").map(x => x.trim())
if (!js) {
// The function has no return value
return
}
// Trim 's
js = js.replace(/^\'|\'$/g, "")
if ((parsedExpected = tryParseJson(js))) {
if (Array.isArray(parsedExpected)) {
if (typeof parsedExpected[0] === "object") {
js = JSON.stringify(parsedExpected)
} else {
js = parsedExpected.join(",")
}
}
}
const requiresHbsBody = details.requiresBlock
return [name, { hbs, js, requiresHbsBody }]
})
.filter(x => !!x)
if (Object.keys(functions).length) {
acc[collection] = functions
}
return acc
@ -55,11 +89,7 @@ function tryParseJson(str) {
describe("manifest", () => {
describe("examples are valid", () => {
describe.each(Object.keys(examples))("%s", collection => {
it.each(examples[collection])("%s", async func => {
const example = manifest[collection][func].example
let [hbs, js] = example.split("->").map(x => x.trim())
it.each(examples[collection])("%s", async (_, { hbs, js }) => {
const context = {
double: i => i * 2,
isString: x => typeof x === "string",
@ -71,23 +101,40 @@ describe("manifest", () => {
context[`array${i}`] = JSON.parse(arrayString.replace(/\'/g, '"'))
})
if (js === undefined) {
// The function has no return value
return
let result = await processString(hbs, context)
result = result.replace(/&nbsp;/g, " ")
expect(result).toEqual(js)
})
})
})
describe("can be parsed and run as js", () => {
const jsHelpers = getHelperList()
const jsExamples = Object.keys(examples).reduce((acc, v) => {
acc[v] = examples[v].filter(([key]) => jsHelpers[key])
return acc
}, {})
describe.each(Object.keys(jsExamples))("%s", collection => {
it.each(
jsExamples[collection].filter(
([_, { requiresHbsBody }]) => !requiresHbsBody
)
)("%s", async (_, { hbs, js }) => {
const context = {
double: i => i * 2,
isString: x => typeof x === "string",
}
let result = await processString(hbs, context)
// Trim 's
js = js.replace(/^\'|\'$/g, "")
if ((parsedExpected = tryParseJson(js))) {
if (Array.isArray(parsedExpected)) {
if (typeof parsedExpected[0] === "object") {
js = JSON.stringify(parsedExpected)
} else {
js = parsedExpected.join(",")
}
}
}
const arrays = hbs.match(/\[[^/\]]+\]/)
arrays?.forEach((arrayString, i) => {
hbs = hbs.replace(new RegExp(escapeRegExp(arrayString)), `array${i}`)
context[`array${i}`] = JSON.parse(arrayString.replace(/\'/g, '"'))
})
let convertedJs = convertToJS(hbs)
let result = processJS(convertedJs, context)
result = result.replace(/&nbsp;/g, " ")
expect(result).toEqual(js)
})

4
packages/types/src/documents/app/table/constants.ts
View file

@ -8,7 +8,7 @@ export enum AutoReason {
FOREIGN_KEY = "foreign_key",
}
export enum AutoFieldSubTypes {
export enum AutoFieldSubType {
CREATED_BY = "createdBy",
CREATED_AT = "createdAt",
UPDATED_BY = "updatedBy",
@ -16,7 +16,7 @@ export enum AutoFieldSubTypes {
AUTO_ID = "autoID",
}
export enum FormulaTypes {
export enum FormulaType {
STATIC = "static",
DYNAMIC = "dynamic",
}

14
packages/types/src/documents/app/table/schema.ts
View file

@ -2,9 +2,9 @@
// column size, position and whether it can be viewed
import { FieldSubtype, FieldType } from "../row"
import {
AutoFieldSubTypes,
AutoFieldSubType,
AutoReason,
FormulaTypes,
FormulaType,
RelationshipType,
} from "./constants"
@ -22,7 +22,7 @@ interface BaseRelationshipFieldMetadata
fieldName: string
tableId: string
tableRev?: string
subtype?: AutoFieldSubTypes.CREATED_BY | AutoFieldSubTypes.UPDATED_BY
subtype?: AutoFieldSubType.CREATED_BY | AutoFieldSubType.UPDATED_BY
}
// External tables use junction tables, internal tables don't require them
@ -62,7 +62,7 @@ export interface AutoColumnFieldMetadata
extends Omit<BaseFieldSchema, "subtype"> {
type: FieldType.AUTO
autocolumn: true
subtype?: AutoFieldSubTypes
subtype?: AutoFieldSubType
lastID?: number
// if the column was turned to an auto-column for SQL, explains why (primary, foreign etc)
autoReason?: AutoReason
@ -70,7 +70,7 @@ export interface AutoColumnFieldMetadata
export interface NumberFieldMetadata extends Omit<BaseFieldSchema, "subtype"> {
type: FieldType.NUMBER
subtype?: AutoFieldSubTypes.AUTO_ID
subtype?: AutoFieldSubType.AUTO_ID
lastID?: number
autoReason?: AutoReason.FOREIGN_KEY
// used specifically when Budibase generates external tables, this denotes if a number field
@ -85,7 +85,7 @@ export interface DateFieldMetadata extends Omit<BaseFieldSchema, "subtype"> {
type: FieldType.DATETIME
ignoreTimezones?: boolean
timeOnly?: boolean
subtype?: AutoFieldSubTypes.CREATED_AT | AutoFieldSubTypes.UPDATED_AT
subtype?: AutoFieldSubType.CREATED_AT | AutoFieldSubType.UPDATED_AT
}
export interface LongFormFieldMetadata extends BaseFieldSchema {
@ -96,7 +96,7 @@ export interface LongFormFieldMetadata extends BaseFieldSchema {
export interface FormulaFieldMetadata extends BaseFieldSchema {
type: FieldType.FORMULA
formula: string
formulaType?: FormulaTypes
formulaType?: FormulaType
}
export interface BBReferenceFieldMetadata

Some files were not shown because too many files have changed in this diff Show more