CodeMirror/budibase
1
0
Fork 0
mirror of synced 2024-06-26 18:10:51 +12:00
Code Issues Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/develop' into feature/current-user-rest-bindings

Browse source
This commit is contained in:
Dean 2022-06-29 18:50:14 +01:00
parent 523bb63455 fccf14eb95
commit 4b296f9bce
134 changed files with 4724 additions and 1957 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

76
.github/CODE_OF_CONDUCT.md vendored
View file

@ -1,76 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at community@budibase.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

1
.github/CODE_OF_CONDUCT.md vendored Symbolic link
View file

@ -0,0 +1 @@
../docs/CODE_OF_CONDUCT.md

208
.github/CONTRIBUTING.md vendored
View file

@ -1,208 +0,0 @@
# Contributing
From opening a bug report to creating a pull request: every contribution is appreciated and welcome. If you're planning to implement a new feature or change the api please create an issue first. This way we can ensure that your precious work is not in vain.
## Not Sure Where to Start?
Budibase is a low-code web application builder that creates svelte based web applications.
Budibase is a monorepo managed by [lerna](https://github.com/lerna/lerna). Lerna manages the building and publishing of the budibase packages. At a high level, here are the packages that make up budibase.
- **packages/builder** - contains code for the budibase builder client side svelte application.
- **packages/client** - A module that runs in the browser responsible for reading JSON definition and creating living, breathing web apps from it.
- **packages/server** - The budibase server. This [Koa](https://koajs.com/) app is responsible for serving the JS for the builder and budibase apps, as well as providing the API for interaction with the database and file system.
- **packages/worker** - This [Koa](https://koajs.com/) app is responsible for providing global apis for managing your budibase installation. Authentication, Users, Email, Org and Auth configs are all provided by the worker.
## Contributor License Agreement (CLA)
In order to accept your pull request, we need you to submit a CLA. You only need to do this once. If you are submitting a pull request for the first time, just submit a Pull Request and our CLA Bot will give you instructions on how to sign the CLA before merging your Pull Request.
All contributors must sign an [Individual Contributor License Agreement](https://github.com/budibase/budibase/blob/next/.github/cla/individual-cla.md).
If contributing on behalf of your company, your company must sign a [Corporate Contributor License Agreement](https://github.com/budibase/budibase/blob/next/.github/cla/corporate-cla.md). If so, please contact us via community@budibase.com.
## Glossary of Terms
To understand the budibase API, it can be helpful to understand the top level entities that make up Budibase.
### Client
A client represents a single budibase customer. Each budibase client will have 1 or more budibase servers. Every client is assigned a unique ID.
### App
A client can have one or more budibase applications. Budibase applications would be things like "Developer Inventory Management" or "Goat Herder CRM". Think of a budibase application as a tree.
### Database
An App can have one or more databases. Keeping with our [dendrology](https://en.wikipedia.org/wiki/Dendrology) analogy - think of an database as a branch on the tree. Databases are used to keep data separate for different instances of your app. For example, if you had a CRM app, you may create a database for your US office, and a database for your Australian office. Databases allow us to support [multitenancy](https://www.gartner.com/en/information-technology/glossary/multitenancy) in budibase applications.
### Table
Tables in budibase are almost akin to tables in relational databases. A table may be a "Car" or an "Employee". They are the main building blocks for the creation and management of backend data in budibase.
### View
A View is an advanced feature in budibase that allows you to write a custom query using [MapReduce](https://pouchdb.com/guides/queries.html) queries. Views enable powerful query functionality and calculations, allowing you to do more with your data.
### Page
A page in budibase is actually a single, self contained svelte web app. There are only 2 pages in budibase. The **login** page and the **main** page.
### Screen
A screen is a component within a single page. Generally, screens represent client side routes, and can be switched without refreshing the page.
### Component
A component is the basic frontend building block of a budibase app.
### Component Library
Component libraries are collections of components as well as the definition of their props contained in a file called `components.json`.
## Contributing to Budibase
* Please maintain the existing code style.
* Please try to keep your commits small and focused.
* Please write tests.
* If the project diverges from your branch, please rebase instead of merging. This makes the commit graph easier to read.
* Once your work is completed, please raise a PR against the `develop` branch with some information about what has changed and why.
### Getting Started For Contributors
#### 1. Prerequisites
NodeJS Version `14.x.x`
*yarn -* `npm install -g yarn`
*jest* - `npm install -g jest`
#### 2. Clone this repository
`git clone https://github.com/Budibase/budibase.git`
then `cd ` into your local copy.
#### 3. Install and Build
| **NOTE**: On Windows, all yarn commands must be executed on a bash shell (e.g. git bash)
To develop the Budibase platform you'll need [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) installed.
##### Quick method
`yarn setup` will check that all necessary components are installed and setup the repo for usage.
##### Manual method
The following commands can be executed to manually get Budibase up and running (assuming Docker/Docker Compose has been installed).
`yarn` to install project dependencies
`yarn bootstrap` will install all budibase modules and symlink them together using lerna.
`yarn build` will build all budibase packages.
#### 4. Running
To run the budibase server and builder in dev mode (i.e. with live reloading):
1. Open a new console
2. `yarn dev` (from root)
3. Access the builder on http://localhost:10000/builder
This will enable watch mode for both the builder app, server, client library and any component libraries.
#### 5. Debugging using VS Code
To debug the budibase server and worker a VS Code launch configuration has been provided.
Visit the debug window and select `Budibase Server` or `Budibase Worker` to debug the respective component.
Alternatively to start both components simultaneously select `Start Budibase`.
In addition to the above, the remaining budibase components may be ran in dev mode using: `yarn dev:noserver`.
#### 6. Cleanup
If you wish to delete all the apps created in development and reset the environment then run the following:
1. `yarn nuke:docker` will wipe all the Budibase services
2. `yarn dev` will restart all the services
### Backend
For the backend we run [Redis](https://redis.io/), [CouchDB](https://couchdb.apache.org/), [MinIO](https://min.io/) and [NGINX](https://www.nginx.com/) in Docker compose. This means that to develop Budibase you will need Docker and Docker compose installed. The backend services are then ran separately as Node services with nodemon so that they can be debugged outside of Docker.
### Data Storage
When you are running locally, budibase stores data on disk using docker volumes. The volumes and the types of data associated with each are:
- `redis_data`
- Sessions, email tokens
- `couchdb3_data`
- Global and app databases
- `minio_data`
- App manifest, budibase client, static assets
### Devlopment Modes
A combination of environment variables controls the mode that budibase runs in.
Yarn commands can be used to mimic the different modes that budibase can be ran in
#### Self Hosted
The default mode. A single tenant installation with no usage restrictions.
To enable this mode, use:
```
yarn mode:self
```
#### Cloud
The cloud mode, with account portal turned off.
To enable this mode, use:
```
yarn mode:cloud
```
#### Cloud & Account
The cloud mode, with account portal turned on. This is a replica of the mode that runs at https://budibase.app
To enable this mode, use:
```
yarn mode:account
```
### CI
An overview of the CI pipelines can be found [here](./workflows/README.md)
### Troubleshooting
Sometimes, things go wrong. This can be due to incompatible updates on the budibase platform. To clear down your development environment and start again follow **Step 6. Cleanup**, then proceed from **Step 3. Install and Build** in the setup guide above. You should have a fresh Budibase installation.
### Running tests
#### End-to-end Tests
Budibase uses Cypress to run a number of E2E tests. To run the tests execute the following command in the root folder:
```
yarn test:e2e
```
Or if you are in the builder you can run `yarn cy:test`.
### Other Useful Information
* The contributors are listed in [AUTHORS.md](https://github.com/Budibase/budibase/blob/master/.github/AUTHORS.md) (add yourself).
* This project uses a modified version of the MPLv2 license, see [LICENSE](https://github.com/budibase/server/blob/master/LICENSE).
* We use the [C4 (Collective Code Construction Contract)](https://rfc.zeromq.org/spec:42/C4/) process for contributions.
Please read this if you are unfamiliar with it.

1
.github/CONTRIBUTING.md vendored Symbolic link
View file

@ -0,0 +1 @@
../docs/CONTRIBUTING.md

106
.github/workflows/README.md vendored
View file

@ -6,7 +6,7 @@ Welcome to the budibase CI pipelines directory. This document details what each
## All CI Pipelines
### Note
- When running workflow dispatch jobs, ensure you always run them off the `master` branch. It defaults to `develop`, so double check before running any jobs.
- When running workflow dispatch jobs, ensure you always run them off the `master` branch. It defaults to `develop`, so double check before running any jobs. The exception to this case is the `deploy-release` job which requires the develop branch.
### Standard CI Build Job (budibase_ci.yml)
Triggers:
@ -24,14 +24,14 @@ The standard CI Build job is what runs when you raise a PR to develop or master.
Triggers:
- Push to develop
The job responsible for building, tagging and pushing docker images out to the test and staging environments.
The job responsible for building, tagging and pushing docker images out to the test and release environments.
- Installs all dependencies
- builds the project
- run the unit tests
- publish the budibase JS packages under a prerelease tag to NPM
- build, tag and push docker images under the `develop` tag to docker hub
These images will then be pulled by the test and staging environments, updating the latest automatically. Discord notifications are sent to the #infra channel when this occurs.
These images will then be pulled by the test and release environments, updating the latest automatically. Discord notifications are sent to the #infra channel when this occurs.
### Release Job (release.yml)
Triggers:
@ -57,8 +57,33 @@ This job relies on the release job to have run first, so the latest image is pus
- Build and release the budibase helm chart for kubernetes users
- Perform a github release with the latest version. You can see previous releases here (https://github.com/Budibase/budibase/releases)
### Deploy Release (deploy-release.yml)
Triggers:
- Manual Workflow Dispatch Trigger
### Cloud Deploy (deploy-cloud.yml)
This job is responsible for deploying to our release, cloud kubernetes environment. You must run the release job first, to ensure that the latest images have been built and pushed to docker hub. After kicking off this job, the following will occur:
- Checks out the release branch
- Pulls the latest `values.yaml` from budibase infra, a private repo containing budibases infrastructure configuration
- Gets the latest budibase version from `lerna.json`, if it hasn't been specified in the workflow when you kicked it off
- Configures AWS Credentials
- Deploys the helm chart in the budibase repo to our preproduction EKS cluster, injecting the `values.yaml` we pulled from budibase-infra
- Fires off a discord webhook in the #infra channel to show that the deployment completely successfully.
### Deploy Preprod (deploy-preprod.yml)
Triggers:
- Manual Workflow Dispatch Trigger
This job is responsible for deploying to our preprod, cloud kubernetes environment. You must run the release job first, to ensure that the latest images have been built and pushed to docker hub. After kicking off this job, the following will occur:
- Checks out the master branch
- Pulls the latest `values.yaml` from budibase infra, a private repo containing budibases infrastructure configuration
- Gets the latest budibase version from `lerna.json`, if it hasn't been specified in the workflow when you kicked it off
- Configures AWS Credentials
- Deploys the helm chart in the budibase repo to our preprod EKS cluster, injecting the `values.yaml` we pulled from budibase-infra
- Fires off a discord webhook in the #infra channel to show that the deployment completely successfully.
### Deploy Production (deploy-cloud.yml)
Triggers:
- Manual Workflow Dispatch Trigger
@ -90,4 +115,75 @@ This job is responsible for deploying to our production, cloud kubernetes enviro
### Rollback A Bad Cloud Deployment
- Kick off cloud deploy job
- Ensure you are running off master
- Enter the version number of the last known good version of budibase. For example `1.0.0`
- Enter the version number of the last known good version of budibase. For example `1.0.0`
## Pro
### Installing Pro
The pro package is always installed from source in our CI jobs.
This is done to prevent pro needing to be published prior to CI runs in budiabse. This is required for two reasons:
- To reduce developer need to manually bump versions, i.e:
- release pro, bump pro dep in budibase, now ci can run successfully
- The cyclic dependency on backend-core, i.e:
- pro depends on backend-core
- server depends on pro
- backend-core lives in the monorepo, so it can't be released independently to be used in pro
- therefore the only option is to pull pro from source and release it as a part of the monorepo release, as if it were a mono package
The install is performed using the same steps as local development, via the `yarn bootstrap` command, see the [Contributing Guide#Pro](../CONTRIBUTING.md#pro)
The branch to install pro from can vary depending on ref of the commit that triggered the budibase CI job. This is done to enable branches which have changes in both the monorepo and the pro repo to have their CI pass successfully.
This is done using the [pro/install.sh](../../scripts/pro/install.sh) script. The script will:
- Clone pro to it's default branch (`develop`)
- Check if the clone worked, on forked versions of budibase this will fail due to no access
- This is fine as the `yarn` command will install the version from NPM
- Community PRs should never touch pro so this will always work
- Checkout the `BRANCH` argument, if this fails fallback to `BASE_BRANCH`
- This enables the more complex case of a feature branch being merged to another feature branch, e.g.
- I am working on a branch `epic/stonks` which exists on budibase and pro.
- I want to merge a change to this branch in budibase from `feature/stonks-ui`, which only exists in budibase
- The base branch ensures that `epic/stonks` in pro will still be checked out for the CI run, rather than falling back to `develop`
- Run `yarn setup` to build and install dependencies
- `yarn`
- `yarn bootstrap`
- `yarn build`
- The will build .ts files, and also update the `main` and `types` of `package.json` to point to `dist` rather than src
- The build command will only ever work in CI, it is prevented in local dev
#### `BRANCH` and `BASE_BRANCH` arguments
These arguments are supplied by the various budibase build and release pipelines
- `budibase_ci`
- `BRANCH: ${{ github.event.pull_request.head.ref }}` -> The branch being merged
- `BASE_BRANCH: ${{ github.event.pull_request.base.ref}}` -> The base branch
- `release-develop`
- `BRANCH: develop` -> always use the `develop` branch in pro
- `release`
- `BRANCH: master` -> always use the `master` branch in pro
### Releasing Pro
After budibase dependencies have been released we will release the new version of pro to match the release version of budibase dependencies. This is to ensure that we are always keeping the version of `backend-core` in sync in the pro package and in budibase packages. Without this we could run into scenarios where different versions are being used when installed via `yarn` inside the docker images, creating very difficult to debug cases.
Pro is released using the [pro/release.sh](../../scripts/pro/release.sh) script. The script will:
- Inspect the `VERSION` from the `lerna.json` file in budibase
- Determine whether to use the `latest` or `develop` tag based on the command argument
- Go to pro directory
- install npm creds
- update the version of `backend-core` to be `VERSION`, the version just released by lerna
- publish to npm. Uses a `lerna publish` command, pro itself is a mono repo.
- force the version to be the same as `VERSION` to keep pro and budibase in sync
- reverts the changes to `main` and `types` in `package.json` that were made by the build step, to point back to source
- commit & push: `Prep next development iteration`
- Go to budibase
- Update to the new version of pro in `server` and `worker` so the latest pro version is used in the docker builds
- commit & push: `Update pro version to $VERSION`
#### `COMMAND` argument
This argument is supplied by the existing `release` and `release:develop` budibase commands, which invoke the pro release
- `release` will supply no command and default to use `latest`
- `release:develop` will supply `develop`

17
.github/workflows/budibase_ci.yml vendored
View file

@ -7,6 +7,7 @@ on:
branches:
- master
- develop
- new-design-ui
pull_request:
branches:
- master
@ -59,3 +60,19 @@ jobs:
with:
install: false
command: yarn test:e2e:ci
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-1
- name: Upload to S3
if: github.ref == 'refs/heads/new-design-ui'
run: |
tar -czvf new_ui.tar.gz packages/server/assets packages/server/index.html
aws s3 cp new_ui.tar.gz s3://prod-budi-app-assets/beta:design_ui/
aws s3 cp packages/client/dist/budibase-client.js s3://prod-budi-app-assets/beta:design_ui/budibase-client.js
aws cloudfront create-invalidation --distribution-id E3ELKP4RCEHVLW --paths "/beta:design_ui/*"

2
.github/workflows/deploy-cloud.yaml vendored
View file

@ -1,4 +1,4 @@
name: Budibase Cloud Deploy
name: Budibase Deploy Production
on:
workflow_dispatch:

2
.github/workflows/deploy-preprod.yml vendored
View file

@ -1,4 +1,4 @@
name: Budibase Release Preprod
name: Budibase Deploy Preprod
on:
workflow_dispatch:

77
.github/workflows/deploy-release.yml vendored Normal file
View file

@ -0,0 +1,77 @@
name: Budibase Deploy Release
on:
workflow_dispatch:
jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-1
- name: Fail if branch is not develop
if: github.ref != 'refs/heads/develop'
run: |
echo "Ref is not develop, you must run this job from develop."
exit 1
- name: Get the latest budibase release version
id: version
run: |
release_version=$(cat lerna.json | jq -r '.version')
echo "RELEASE_VERSION=$release_version" >> $GITHUB_ENV
- name: Tag and release Proxy service docker image
run: |
docker login -u $DOCKER_USER -p $DOCKER_PASSWORD
yarn build:docker:proxy:release
docker tag proxy-service budibase/proxy:$RELEASE_TAG
docker push budibase/proxy:$RELEASE_TAG
env:
DOCKER_USER: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_API_KEY }}
RELEASE_TAG: k8s-release
- name: Pull values.yaml from budibase-infra
run: |
curl -H "Authorization: token ${{ secrets.GH_PERSONAL_TOKEN }}" \
-H 'Accept: application/vnd.github.v3.raw' \
-o values.release.yaml \
-L https://api.github.com/repos/budibase/budibase-infra/contents/kubernetes/budibase-release/values.yaml
wc -l values.release.yaml
- name: Deploy to Release Environment
uses: glopezep/helm@v1.7.1
with:
release: budibase-release
namespace: budibase
chart: charts/budibase
token: ${{ github.token }}
helm: helm3
values: |
globals:
appVersion: develop
ingress:
enabled: true
nginx: true
value-files: >-
[
"values.release.yaml"
]
env:
KUBECONFIG_FILE: '${{ secrets.RELEASE_KUBECONFIG }}'
- name: Discord Webhook Action
uses: tsickert/discord-webhook@v4.0.0
with:
webhook-url: ${{ secrets.PROD_DEPLOY_WEBHOOK_URL }}
content: "Release Env Deployment Complete: ${{ env.RELEASE_VERSION }} deployed to Budibase Release Env."
embed-title: ${{ env.RELEASE_VERSION }}

21
.github/workflows/release-develop.yml vendored
View file

@ -1,10 +1,10 @@
name: Budibase Release Staging
concurrency: release-develop
name: Budibase Prerelease
concurrency: release-prerelease
on:
push:
branches:
- release
- develop
paths:
- '.aws/**'
- '.github/**'
@ -21,18 +21,19 @@ env:
# Posthog token used by ui at build time
POSTHOG_TOKEN: phc_uDYOfnFt6wAbBAXkC6STjcrTpAFiWIhqgFcsC1UVO5F
INTERCOM_TOKEN: ${{ secrets.INTERCOM_TOKEN }}
PERSONAL_ACCESS_TOKEN : ${{ secrets.PERSONAL_ACCESS_TOKEN }}
PERSONAL_ACCESS_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
FEATURE_PREVIEW_URL: https://budirelease.live
jobs:
release:
runs-on: ubuntu-latest
steps:
# - name: Fail if branch is not develop
# if: github.ref != 'refs/heads/develop'
# run: |
# echo "Ref is not develop, you must run this job from develop."
# exit 1
- name: Fail if branch is not develop
if: github.ref != 'refs/heads/develop'
run: |
echo "Ref is not develop, you must run this job from develop."
exit 1
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
@ -124,4 +125,4 @@ jobs:
with:
webhook-url: ${{ secrets.PROD_DEPLOY_WEBHOOK_URL }}
content: "Release Env Deployment Complete: ${{ env.RELEASE_VERSION }} deployed to Budibase Release Env."
embed-title: ${{ env.RELEASE_VERSION }}
embed-title: ${{ env.RELEASE_VERSION }}

5
README.md
View file

@ -174,6 +174,7 @@ Budibase is dedicated to providing a welcoming, diverse, and harrassment-free ex
## 🙌 Contributing to Budibase
From opening a bug report to creating a pull request: every contribution is appreciated and welcomed. If you're planning to implement a new feature or change the API please create an issue first. This way we can ensure your work is not in vain.
Environment setup instructions are available for [Debian](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-DEBIAN.md) and [MacOSX](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-MACOSX.md)
### Not Sure Where to Start?
A good place to start contributing, is the [First time issues project](https://github.com/Budibase/budibase/projects/22).
@ -187,7 +188,7 @@ Budibase is a monorepo managed by lerna. Lerna manages the building and publishi
- [packages/server](https://github.com/Budibase/budibase/tree/HEAD/packages/server) - The budibase server. This Koa app is responsible for serving the JS for the builder and budibase apps, as well as providing the API for interaction with the database and file system.
For more information, see [CONTRIBUTING.md](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md)
For more information, see [CONTRIBUTING.md](https://github.com/Budibase/budibase/blob/HEAD/docs/CONTRIBUTING.md)
<br /><br />
@ -202,7 +203,7 @@ Budibase is open-source, licensed as [GPL v3](https://www.gnu.org/licenses/gpl-3
[![Stargazers over time](https://starchart.cc/Budibase/budibase.svg)](https://starchart.cc/Budibase/budibase)
If you are having issues between updates of the builder, please use the guide [here](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md#troubleshooting) to clear down your environment.
If you are having issues between updates of the builder, please use the guide [here](https://github.com/Budibase/budibase/blob/HEAD/docs/CONTRIBUTING.md#troubleshooting) to clear down your environment.
<br /><br />

2
charts/budibase/templates/worker-service-deployment.yaml
View file

@ -123,6 +123,8 @@ spec:
value: {{ .Values.globals.google.clientId | quote }}
- name: GOOGLE_CLIENT_SECRET
value: {{ .Values.globals.google.secret | quote }}
- name: TENANT_FEATURE_FLAGS
value: {{ .Values.globals.tenantFeatureFlags | quote }}
image: budibase/worker:{{ .Values.globals.appVersion }}
imagePullPolicy: Always
livenessProbe:

2
charts/budibase/values.yaml
View file

@ -103,7 +103,7 @@ globals:
google:
clientId: ""
secret: ""
automationMaxIterations: "500"
automationMaxIterations: "200"
createSecrets: true # creates an internal API key, JWT secrets and redis password for you

76
docs/CODE_OF_CONDUCT.md Normal file
View file

@ -0,0 +1,76 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at community@budibase.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

231
docs/CONTRIBUTING.md Normal file
View file

@ -0,0 +1,231 @@
# Contributing
From opening a bug report to creating a pull request: every contribution is appreciated and welcome. If you're planning to implement a new feature or change the api please [create an issue](https://github.com/Budibase/budibase/issues/new/choose) first. This way we can ensure that your precious work is not in vain.
## Table of contents
- [Quick start](#quick-start)
- [Status](#status)
- [What's included](#whats-included)
- [Bugs and feature requests](#bugs-and-feature-requests)
## Not Sure Where to Start?
Budibase is a low-code web application builder that creates svelte-based web applications.
Budibase is a monorepo managed by [lerna](https://github.com/lerna/lerna). Lerna manages the building and publishing of the budibase packages. At a high level, here are the packages that make up budibase.
- **packages/builder** - contains code for the budibase builder client side svelte application.
- **packages/client** - A module that runs in the browser responsible for reading JSON definition and creating living, breathing web apps from it.
- **packages/server** - The budibase server. This [Koa](https://koajs.com/) app is responsible for serving the JS for the builder and budibase apps, as well as providing the API for interaction with the database and file system.
- **packages/worker** - This [Koa](https://koajs.com/) app is responsible for providing global apis for managing your budibase installation. Authentication, Users, Email, Org and Auth configs are all provided by the worker.
## Contributor License Agreement (CLA)
In order to accept your pull request, we need you to submit a CLA. You only need to do this once. If you are submitting a pull request for the first time, just submit a Pull Request and our CLA Bot will give you instructions on how to sign the CLA before merging your Pull Request.
All contributors must sign an [Individual Contributor License Agreement](https://github.com/budibase/budibase/blob/next/.github/cla/individual-cla.md).
If contributing on behalf of your company, your company must sign a [Corporate Contributor License Agreement](https://github.com/budibase/budibase/blob/next/.github/cla/corporate-cla.md). If so, please contact us via community@budibase.com.
## Glossary of Terms
To understand the budibase API, it can be helpful to understand the top level entities that make up Budibase.
### Client
A client represents a single budibase customer. Each budibase client will have 1 or more budibase servers. Every client is assigned a unique ID.
### App
A client can have one or more budibase applications. Budibase applications would be things like "Developer Inventory Management" or "Goat Herder CRM". Think of a budibase application as a tree.
### Database
An App can have one or more databases. Keeping with our [dendrology](https://en.wikipedia.org/wiki/Dendrology) analogy - think of an database as a branch on the tree. Databases are used to keep data separate for different instances of your app. For example, if you had a CRM app, you may create a database for your US office, and a database for your Australian office. Databases allow us to support [multitenancy](https://www.gartner.com/en/information-technology/glossary/multitenancy) in budibase applications.
### Table
Tables in budibase are almost akin to tables in relational databases. A table may be a "Car" or an "Employee". They are the main building blocks for the creation and management of backend data in budibase.
### View
A View is an advanced feature in budibase that allows you to write a custom query using [MapReduce](https://pouchdb.com/guides/queries.html) queries. Views enable powerful query functionality and calculations, allowing you to do more with your data.
### Page
A page in budibase is actually a single, self contained svelte web app. There are only 2 pages in budibase. The **login** page and the **main** page.
### Screen
A screen is a component within a single page. Generally, screens represent client side routes, and can be switched without refreshing the page.
### Component
A component is the basic frontend building block of a budibase app.
### Component Library
Component libraries are collections of components as well as the definition of their props contained in a file called `components.json`.
## Contributing to Budibase
* Please maintain the existing code style.
* Please try to keep your commits small and focused.
* Please write tests.
* If the project diverges from your branch, please rebase instead of merging. This makes the commit graph easier to read.
* Once your work is completed, please raise a PR against the `develop` branch with some information about what has changed and why.
### Getting Started For Contributors
#### 1. Prerequisites
NodeJS Version `14.x.x`
*yarn -* `npm install -g yarn`
*jest* - `npm install -g jest`
#### 2. Clone this repository
`git clone https://github.com/Budibase/budibase.git`
then `cd ` into your local copy.
#### 3. Install and Build
| **NOTE**: On Windows, all yarn commands must be executed on a bash shell (e.g. git bash)
To develop the Budibase platform you'll need [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) installed.
##### Quick method
`yarn setup` will check that all necessary components are installed and setup the repo for usage.
##### Manual method
The following commands can be executed to manually get Budibase up and running (assuming Docker/Docker Compose has been installed).
`yarn` to install project dependencies
`yarn bootstrap` will install all budibase modules and symlink them together using lerna.
`yarn build` will build all budibase packages.
#### 4. Running
To run the budibase server and builder in dev mode (i.e. with live reloading):
1. Open a new console
2. `yarn dev` (from root)
3. Access the builder on http://localhost:10000/builder
This will enable watch mode for both the builder app, server, client library and any component libraries.
#### 5. Debugging using VS Code
To debug the budibase server and worker a VS Code launch configuration has been provided.
Visit the debug window and select `Budibase Server` or `Budibase Worker` to debug the respective component.
Alternatively to start both components simultaneously select `Start Budibase`.
In addition to the above, the remaining budibase components may be run in dev mode using: `yarn dev:noserver`.
#### 6. Cleanup
If you wish to delete all the apps created in development and reset the environment then run the following:
1. `yarn nuke:docker` will wipe all the Budibase services
2. `yarn dev` will restart all the services
### Backend
For the backend we run [Redis](https://redis.io/), [CouchDB](https://couchdb.apache.org/), [MinIO](https://min.io/) and [NGINX](https://www.nginx.com/) in Docker compose. This means that to develop Budibase you will need Docker and Docker compose installed. The backend services are then run separately as Node services with nodemon so that they can be debugged outside of Docker.
### Data Storage
When you are running locally, budibase stores data on disk using docker volumes. The volumes and the types of data associated with each are:
- `redis_data`
- Sessions, email tokens
- `couchdb3_data`
- Global and app databases
- `minio_data`
- App manifest, budibase client, static assets
### Development Modes
A combination of environment variables controls the mode budibase runs in.
Yarn commands can be used to mimic the different modes as described in the sections below:
#### Self Hosted
The default mode. A single tenant installation with no usage restrictions.
To enable this mode, use:
```
yarn mode:self
```
#### Cloud
The cloud mode, with account portal turned off.
To enable this mode, use:
```
yarn mode:cloud
```
#### Cloud & Account
The cloud mode, with account portal turned on. This is a replica of the mode that runs at https://budibase.app
To enable this mode, use:
```
yarn mode:account
```
### CI
An overview of the CI pipelines can be found [here](./workflows/README.md)
### Pro
@budibase/pro is the closed source package that supports licensed features in budibase. By default the package will be pulled from NPM and will not normally need to be touched in local development. If you require to update code inside the pro package it can be cloned to the same root level as budibase, e.g.
```
.
|_ budibase
|_ budibase-pro
```
Note that only budibase maintainers will be able to access the pro repo.
The `yarn bootstrap` command can be used to replace the NPM supplied dependency with the local source aware version. This is achieved using the `yarn link` command. To see specifically how dependencies are linked see [scripts/link-dependencies.sh](../scripts/link-dependencies.sh). The same link script is used to link dependencies to account-portal in local dev.
### Troubleshooting
Sometimes, things go wrong. This can be due to incompatible updates on the budibase platform. To clear down your development environment and start again follow **Step 6. Cleanup**, then proceed from **Step 3. Install and Build** in the setup guide above to create a fresh Budibase installation.
### Running tests
#### End-to-end Tests
Budibase uses Cypress to run a number of E2E tests. To run the tests execute the following command in the root folder:
```
yarn test:e2e
```
Or if you are in the builder you can run `yarn cy:test`.
### Other Useful Information
* The contributors are listed in [AUTHORS.md](https://github.com/Budibase/budibase/blob/master/.github/AUTHORS.md) (add yourself).
* This project uses a modified version of the MPLv2 license, see [LICENSE](https://github.com/budibase/server/blob/master/LICENSE).
* We use the [C4 (Collective Code Construction Contract)](https://rfc.zeromq.org/spec:42/C4/) process for contributions.
Please read this if you are unfamiliar with it.

52
docs/DEV-SETUP-DEBIAN.md Normal file
View file

@ -0,0 +1,52 @@
## Dev Environment on Debian 11
### Install Node
Budibase requires a recent version of node (14+):
```
curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
apt -y install nodejs
node -v
```
### Install npm requirements
```
npm install -g yarn jest lerna
```
### Install Docker and Docker Compose
```
apt install docker.io
pip3 install docker-compose
```
### Clone the repo
```
git clone https://github.com/Budibase/budibase.git
```
### Check Versions
This setup process was tested on Debian 11 (bullseye) with version numbers show below. Your mileage may vary using anything else.
- Docker: 20.10.5
- Docker-Compose: 1.29.2
- Node: v16.15.1
- Yarn: 1.22.19
- Lerna: 5.1.4
### Build
```
cd budibase
yarn setup
```
The yarn setup command runs several build steps i.e.
```
node ./hosting/scripts/setup.js && yarn && yarn bootstrap && yarn build && yarn dev
```
So this command will actually run the application in dev mode. It creates .env files under `./packages/server` and `./packages/worker` and runs docker containers for each service via docker-compose.
The dev version will be available on port 10000 i.e.
http://127.0.0.1:10000/builder/admin

54
docs/DEV-SETUP-MACOSX.md Normal file
View file

@ -0,0 +1,54 @@
## Dev Environment on MAC OSX 12 (Monterey)
### Install Homebrew
Install instructions [here](https://brew.sh/)
### Install Node
Budibase requires a recent version of node (14+):
```
brew install node npm
node -v
```
### Install npm requirements
```
npm install -g yarn jest lerna
```
### Install Docker and Docker Compose
```
brew install docker docker-compose
```
### Clone the repo
```
git clone https://github.com/Budibase/budibase.git
```
### Check Versions
This setup process was tested on Mac OSX 12 (Monterey) with version numbers shown below. Your mileage may vary using anything else.
- Docker: 20.10.14
- Docker-Compose: 2.6.0
- Node: 18.3.0
- Yarn: 1.22.19
- Lerna: 5.1.4
### Build
```
cd budibase
yarn setup
```
The yarn setup command runs several build steps i.e.
```
node ./hosting/scripts/setup.js && yarn && yarn bootstrap && yarn build && yarn dev
```
So this command will actually run the application in dev mode. It creates .env files under `./packages/server` and `./packages/worker` and runs docker containers for each service via docker-compose.
The dev version will be available on port 10000 i.e.
http://127.0.0.1:10000/builder/admin

13
hosting/letsencrypt/certificate-renew.sh Normal file
View file

@ -0,0 +1,13 @@
#!/bin/bash
CUSTOM_DOMAIN="$1"
if [[ ! -z "${CUSTOM_DOMAIN}" ]]; then
certbot certonly --webroot --webroot-path="/var/www/html" \
--register-unsafely-without-email \
--domains $CUSTOM_DOMAIN \
--rsa-key-size 4096 \
--agree-tos \
--force-renewal
nginx -s reload
fi

23
hosting/letsencrypt/certificate-request.sh Normal file
View file

@ -0,0 +1,23 @@
#!/bin/bash
CUSTOM_DOMAIN="$1"
# Request from Lets Encrypt
certbot certonly --webroot --webroot-path="/var/www/html" \
--register-unsafely-without-email \
--domains $CUSTOM_DOMAIN \
--rsa-key-size 4096 \
--agree-tos \
--force-renewal
if (($? != 0)); then
echo "ERROR: certbot request failed for $CUSTOM_DOMAIN use http on port 80 - exiting"
exit 1
else
cp /app/letsencrypt/options-ssl-nginx.conf /etc/letsencrypt/options-ssl-nginx.conf
cp /app/letsencrypt/ssl-dhparams.pem /etc/letsencrypt/ssl-dhparams.pem
cp /app/letsencrypt/nginx-ssl.conf /etc/nginx/sites-available/nginx-ssl.conf
sed -i "s/CUSTOM_DOMAIN/$CUSTOM_DOMAIN/g" /etc/nginx/sites-available/nginx-ssl.conf
ln -s /etc/nginx/sites-available/nginx-ssl.conf /etc/nginx/sites-enabled/nginx-ssl.conf
echo "INFO: restart nginx after certbot request"
/etc/init.d/nginx restart
fi

96
hosting/letsencrypt/nginx-ssl.conf Normal file
View file

@ -0,0 +1,96 @@
server {
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
server_name _;
ssl_certificate /etc/letsencrypt/live/CUSTOM_DOMAIN/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/CUSTOM_DOMAIN/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
client_max_body_size 1000m;
ignore_invalid_headers off;
proxy_buffering off;
# port_in_redirect off;
location ^~ /.well-known/acme-challenge/ {
default_type "text/plain";
root /var/www/html;
break;
}
location = /.well-known/acme-challenge/ {
return 404;
}
location /app {
proxy_pass http://127.0.0.1:4001;
}
location = / {
proxy_pass http://127.0.0.1:4001;
}
location ~ ^/(builder|app_) {
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:4001;
}
location ~ ^/api/(system|admin|global)/ {
proxy_pass http://127.0.0.1:4002;
}
location /worker/ {
proxy_pass http://127.0.0.1:4002;
rewrite ^/worker/(.*)$ /$1 break;
}
location /api/ {
# calls to the API are rate limited with bursting
limit_req zone=ratelimit burst=20 nodelay;
# 120s timeout on API requests
proxy_read_timeout 120s;
proxy_connect_timeout 120s;
proxy_send_timeout 120s;
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:4001;
}
location /db/ {
proxy_pass http://127.0.0.1:5984;
rewrite ^/db/(.*)$ /$1 break;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://127.0.0.1:9000;
}
client_header_timeout 60;
client_body_timeout 60;
keepalive_timeout 60;
# gzip
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css text/xml application/json application/javascript application/rss+xml application/atom+xml image/svg+xml;
}

13
hosting/letsencrypt/options-ssl-nginx.conf Normal file
View file

@ -0,0 +1,13 @@
# This file contains important security parameters. If you modify this file
# manually, Certbot will be unable to automatically provide future security
# updates. Instead, Certbot will print and log an error message with a path to
# the up-to-date file that you will need to refer to when manually updating
# this file.
ssl_session_cache shared:le_nginx_SSL:10m;
ssl_session_timeout 1440m;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;
ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";

8
hosting/letsencrypt/ssl-dhparams.pem Normal file
View file

@ -0,0 +1,8 @@
-----BEGIN DH PARAMETERS-----
MIIBCAKCAQEA//////////+t+FRYortKmq/cViAnPTzx2LnFg84tNpWp4TZBFGQz
+8yTnc4kmz75fS/jY2MMddj2gbICrsRhetPfHtXV/WVhJDP1H18GbtCFY2VVPe0a
87VXE15/V8k1mE8McODmi3fipona8+/och3xWKE2rec1MKzKT0g6eXq8CrGCsyT7
YdEIqUuyyOP7uWrat2DX9GgdT0Kj3jlN9K5W7edjcrsZCwenyO4KbXCeAvzhzffi
7MA0BM0oNC9hkXL+nOmFg/+OTxIy7vKBg8P+OxtMb61zO7X8vC7CIAXFjvGDfRaD
ssbzSibBsu/6iGtCOGEoXJf//////////wIBAg==
-----END DH PARAMETERS-----

17
hosting/scripts/build-target-paths.sh Normal file
View file

@ -0,0 +1,17 @@
#!/bin/bash
echo ${TARGETBUILD} > /buildtarget.txt
if [[ "${TARGETBUILD}" = "aas" ]]; then
# Azure AppService uses /home for persisent data & SSH on port 2222
mkdir -p /home/budibase/{minio,couchdb}
mkdir -p /home/budibase/couchdb/data
chown -R couchdb:couchdb /home/budibase/couchdb/
apt update
apt-get install -y openssh-server
sed -i 's#dir=/opt/couchdb/data/search#dir=/home/budibase/couchdb/data/search#' /opt/clouseau/clouseau.ini
sed -i 's#/minio/minio server /minio &#/minio/minio server /home/budibase/minio &#' /runner.sh
sed -i 's#database_dir = ./data#database_dir = /home/budibase/couchdb/data#' /opt/couchdb/etc/default.ini
sed -i 's#view_index_dir = ./data#view_index_dir = /home/budibase/couchdb/data#' /opt/couchdb/etc/default.ini
sed -i "s/#Port 22/Port 2222/" /etc/ssh/sshd_config
/etc/init.d/ssh restart
fi

40
hosting/scripts/healthcheck.sh Normal file
View file

@ -0,0 +1,40 @@
#!/usr/bin/env bash
healthy=true
if [[ $(curl -Lfk -s -w "%{http_code}\n" http://localhost/ -o /dev/null) -ne 200 ]]; then
echo 'ERROR: Budibase is not running';
healthy=false
fi
if [[ $(curl -s -w "%{http_code}\n" http://localhost:4001/health -o /dev/null) -ne 200 ]]; then
echo 'ERROR: Budibase backend is not running';
healthy=false
fi
if [[ $(curl -s -w "%{http_code}\n" http://localhost:4002/health -o /dev/null) -ne 200 ]]; then
echo 'ERROR: Budibase worker is not running';
healthy=false
fi
if [[ $(curl -s -w "%{http_code}\n" http://localhost:5984/ -o /dev/null) -ne 200 ]]; then
echo 'ERROR: CouchDB is not running';
healthy=false
fi
if [[ $(redis-cli -a $REDIS_PASSWORD --no-auth-warning ping) != 'PONG' ]]; then
echo 'ERROR: Redis is down';
healthy=false
fi
# mino, clouseau,
nginx -t -q
NGINX_STATUS=$?
if [[ $NGINX_STATUS -gt 0 ]]; then
echo 'ERROR: Nginx config problem';
healthy=false
fi
if [ $healthy == true ]; then
exit 0
else
exit 1
fi

177
hosting/single/Dockerfile
View file

@ -1,79 +1,93 @@
FROM couchdb
FROM node:14-slim as build
ENV DEPLOYMENT_ENVIRONMENT=docker
ENV POSTHOG_TOKEN=phc_fg5I3nDOf6oJVMHSaycEhpPdlgS8rzXG2r6F2IpxCHS
ENV COUCHDB_PASSWORD=budibase
ENV COUCHDB_USER=budibase
ENV COUCH_DB_URL=http://budibase:budibase@localhost:5984
ENV BUDIBASE_ENVIRONMENT=PRODUCTION
ENV MINIO_URL=http://localhost:9000
ENV REDIS_URL=localhost:6379
ENV WORKER_URL=http://localhost:4002
ENV INTERNAL_API_KEY=budibase
ENV JWT_SECRET=testsecret
ENV MINIO_ACCESS_KEY=budibase
ENV MINIO_SECRET_KEY=budibase
ENV SELF_HOSTED=1
ENV CLUSTER_PORT=10000
ENV REDIS_PASSWORD=budibase
ENV ARCHITECTURE=amd
ENV APP_PORT=4001
ENV WORKER_PORT=4002
# install node-gyp dependencies
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends apt-utils cron g++ make python
RUN apt-get update
RUN apt-get install software-properties-common wget nginx -y
RUN apt-add-repository 'deb http://security.debian.org/debian-security stretch/updates main'
RUN apt-get update
# add pin script
WORKDIR /
ADD scripts/pinVersions.js scripts/cleanup.sh ./
RUN chmod +x /cleanup.sh
# build server
WORKDIR /app
ADD packages/server .
RUN node /pinVersions.js && yarn && yarn build && /cleanup.sh
# build worker
WORKDIR /worker
ADD packages/worker .
RUN node /pinVersions.js && yarn && yarn build && /cleanup.sh
FROM couchdb:3.2.1
# TARGETARCH can be amd64 or arm e.g. docker build --build-arg TARGETARCH=amd64
ARG TARGETARCH amd64
#TARGETBUILD can be set to single (for single docker image) or aas (for azure app service)
# e.g. docker build --build-arg TARGETBUILD=aas ....
ARG TARGETBUILD single
ENV TARGETBUILD $TARGETBUILD
COPY --from=build /app /app
COPY --from=build /worker /worker
ENV \
APP_PORT=4001 \
ARCHITECTURE=amd \
BUDIBASE_ENVIRONMENT=PRODUCTION \
CLUSTER_PORT=80 \
COUCHDB_PASSWORD=budibase \
COUCHDB_USER=budibase \
COUCH_DB_URL=http://budibase:budibase@localhost:5984 \
# CUSTOM_DOMAIN=budi001.custom.com \
DEPLOYMENT_ENVIRONMENT=docker \
INTERNAL_API_KEY=budibase \
JWT_SECRET=testsecret \
MINIO_ACCESS_KEY=budibase \
MINIO_SECRET_KEY=budibase \
MINIO_URL=http://localhost:9000 \
POSTHOG_TOKEN=phc_fg5I3nDOf6oJVMHSaycEhpPdlgS8rzXG2r6F2IpxCHS \
REDIS_PASSWORD=budibase \
REDIS_URL=localhost:6379 \
SELF_HOSTED=1 \
TARGETBUILD=$TARGETBUILD \
WORKER_PORT=4002 \
WORKER_URL=http://localhost:4002
# install base dependencies
RUN apt-get update && \
apt-get install -y software-properties-common wget nginx && \
apt-add-repository 'deb http://security.debian.org/debian-security stretch/updates main' && \
apt-get update
# install other dependencies, nodejs, oracle requirements, jdk8, redis, nginx
WORKDIR /nodejs
RUN curl -sL https://deb.nodesource.com/setup_16.x -o /tmp/nodesource_setup.sh && \
bash /tmp/nodesource_setup.sh && \
apt-get install -y libaio1 nodejs nginx openjdk-8-jdk redis-server unzip && \
npm install --global yarn pm2
# setup nginx
ADD hosting/single/nginx.conf /etc/nginx
RUN mkdir /etc/nginx/logs
RUN useradd www
RUN touch /etc/nginx/logs/error.log
RUN touch /etc/nginx/logs/nginx.pid
ADD hosting/single/nginx-default-site.conf /etc/nginx/sites-enabled/default
RUN mkdir -p /var/log/nginx && \
touch /var/log/nginx/error.log && \
touch /var/run/nginx.pid
# install java
RUN apt-get install openjdk-8-jdk -y
# setup nodejs
WORKDIR /nodejs
RUN curl -sL https://deb.nodesource.com/setup_16.x -o /tmp/nodesource_setup.sh
RUN bash /tmp/nodesource_setup.sh
RUN apt-get install nodejs
RUN npm install --global yarn
RUN npm install --global pm2
# setup redis
RUN apt install redis-server -y
# setup server
WORKDIR /app
ADD packages/server .
RUN ls -al
RUN yarn
RUN yarn build
# Install client for oracle datasource
RUN apt-get install unzip libaio1
RUN /bin/bash -e scripts/integrations/oracle/instantclient/linux/x86-64/install.sh
# setup worker
WORKDIR /worker
ADD packages/worker .
RUN yarn
RUN yarn build
WORKDIR /
RUN mkdir -p scripts/integrations/oracle
ADD packages/server/scripts/integrations/oracle scripts/integrations/oracle
RUN /bin/bash -e ./scripts/integrations/oracle/instantclient/linux/install.sh
# setup clouseau
WORKDIR /
RUN wget https://github.com/cloudant-labs/clouseau/releases/download/2.21.0/clouseau-2.21.0-dist.zip
RUN unzip clouseau-2.21.0-dist.zip
RUN mv clouseau-2.21.0 /opt/clouseau
RUN rm clouseau-2.21.0-dist.zip
RUN wget https://github.com/cloudant-labs/clouseau/releases/download/2.21.0/clouseau-2.21.0-dist.zip && \
unzip clouseau-2.21.0-dist.zip && \
mv clouseau-2.21.0 /opt/clouseau && \
rm clouseau-2.21.0-dist.zip
WORKDIR /opt/clouseau
RUN mkdir ./bin
ADD hosting/single/clouseau ./bin/
ADD hosting/single/log4j.properties .
ADD hosting/single/clouseau.ini .
ADD hosting/single/log4j.properties hosting/single/clouseau.ini ./
RUN chmod +x ./bin/clouseau
# setup CouchDB
@ -82,18 +96,49 @@ ADD hosting/single/vm.args ./etc/
# setup minio
WORKDIR /minio
RUN wget https://dl.min.io/server/minio/release/linux-${ARCHITECTURE}64/minio
RUN chmod +x minio
ADD scripts/install-minio.sh ./install.sh
RUN chmod +x install.sh && ./install.sh
# setup runner file
WORKDIR /
ADD hosting/single/runner.sh .
RUN chmod +x ./runner.sh
ADD hosting/scripts/healthcheck.sh .
RUN chmod +x ./healthcheck.sh
EXPOSE 10000
ADD hosting/scripts/build-target-paths.sh .
RUN chmod +x ./build-target-paths.sh
# For Azure App Service install SSH & point data locations to /home
RUN /build-target-paths.sh
# cleanup cache
RUN yarn cache clean -f
EXPOSE 80
EXPOSE 443
VOLUME /opt/couchdb/data
VOLUME /minio
# setup letsencrypt certificate
RUN apt-get install -y certbot python3-certbot-nginx
ADD hosting/letsencrypt /app/letsencrypt
RUN chmod +x /app/letsencrypt/certificate-request.sh /app/letsencrypt/certificate-renew.sh
# Remove cached files
RUN rm -rf \
/root/.cache \
/root/.npm \
/root/.pip \
/usr/local/share/doc \
/usr/share/doc \
/usr/share/man \
/var/lib/apt/lists/* \
/tmp/*
HEALTHCHECK --interval=15s --timeout=15s --start-period=45s CMD "/healthcheck.sh"
# must set this just before running
ENV NODE_ENV=production
WORKDIR /
CMD ["./runner.sh"]

112
hosting/single/README.md Normal file
View file

@ -0,0 +1,112 @@
# Docker Single Image for Budibase
## Overview
As an alternative to running several docker containers via docker-compose, the files under ./hosting/single can be used to build a docker image containing all of the Budibase components (minio, couch, clouseau etc).
We call this the 'single image' container as the Dockerfile adds all the components to a single docker image.
## Usage
- Amend Environment Variables
- Build Requirements
- Build the Image
- Run the Container
### Amend Environment Variables
Edit the Dockerfile in this directory amending the environment variables to suit your usage. Pay particular attention to changing passwords.
The CUSTOM_DOMAIN variable will be used to request a certificate from LetsEncrypt and if successful you can point traffic to port 443. If you choose to use the CUSTOM_DOMAIN variable ensure that the DNS for your custom domain points to the public IP address where you are running Budibase - otherwise the certificate issuance will fail.
If you have other arrangements for a proxy in front of the single image container you can omit the CUSTOM_DOMAIN environment variable and the request to LetsEncrypt will be skipped. You can then point traffic to port 80.
### Build Requirements
We would suggest building the image with 6GB of RAM and 20GB of free disk space for build artifacts. The resulting image size will use approx 2GB of disk space.
### Build the Image
The guidance below is based on building the Budibase single image on Debian 11 and AlmaLinux 8. If you use another distro or OS you will need to amend the commands to suit.
#### Install Node
Budibase requires a more recent version of node (14+) than is available in the base Debian repos so:
```
curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
apt install -y nodejs
node -v
```
Install yarn and lerna:
```
npm install -g yarn jest lerna
```
#### Install Docker
```
apt install -y docker.io
```
Check the versions of each installed version. This process was tested with the version numbers below so YMMV using anything else:
- Docker: 20.10.5
- node: 16.15.1
- yarn: 1.22.19
- lerna: 5.1.4
#### Get the Code
Clone the Budibase repo
```
git clone https://github.com/Budibase/budibase.git
cd budibase
```
#### Setup Node
Node setup:
```
node ./hosting/scripts/setup.js
yarn
yarn bootstrap
yarn build
```
#### Build Image
The following yarn command does some prep and then runs the docker build command:
```
yarn build:docker:single
```
If the docker build step fails try running that step again manually with:
```
docker build --build-arg TARGETARCH=amd --no-cache -t budibase:latest -f ./hosting/single/Dockerfile .
```
#### Azure App Services
Azure have some specific requirements for running a container in their App Service. Specifically, installation of SSH to port 2222 and data storage under /home. If you would like to build a budibase container for Azure App Service add the build argument shown below setting it to 'aas'. You can remove the CUSTOM_DOMAIN env variable from the Dockerfile too as Azure terminate SSL before requests reach the container.
```
docker build --build-arg TARGETARCH=amd --build-arg TARGETBUILD=aas -t budibase:latest -f ./hosting/single/Dockerfile .
```
### Run the Container
```
docker run -d -p 80:80 -p 443:443 --name budibase budibase:latest
```
Where:
- -d runs the container in detached mode
- -p forwards ports from your host to the ports inside the container. If you are already using port 80 on your host for something else you can try running with an alternative port e.g. `-p 8080:80`
- --name is the name for the container as shown in `docker ps` and can be used with other docker commands e.g. `docker restart budibase`
When the container runs you should be able to access the container over http at your host address e.g. http://1.2.3.4/ or using your custom domain e.g. https://my.custom.domain/
When the Budibase UI appears you will be prompted to create an account to get started.
### Podman
The single image container builds fine when using podman in place of docker. You may be prompted for the registry to use for the CouchDB image and the HEALTHCHECK parameter is not OCI compliant so is ignored.
### Check
There are many things that could go wrong so if your container is not building or running as expected please check the following before opening a support issue.
Verify the healthcheck status of the container:
```
docker ps
```
Check the container logs:
```
docker logs budibase
```
### Support
This single image build is still a work-in-progress so if you open an issue please provide the following information:
- The OS and OS version you are building on
- The versions you are using of docker, docker-compose, yarn, node, lerna
- For build errors please provide zipped output
- For container errors please provide zipped container logs

94
hosting/single/nginx-default-site.conf Normal file
View file

@ -0,0 +1,94 @@
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
client_max_body_size 1000m;
ignore_invalid_headers off;
proxy_buffering off;
# port_in_redirect off;
location ^~ /.well-known/acme-challenge/ {
default_type "text/plain";
root /var/www/html;
break;
}
location = /.well-known/acme-challenge/ {
return 404;
}
location /app {
proxy_pass http://127.0.0.1:4001;
}
location = / {
proxy_pass http://127.0.0.1:4001;
}
location ~ ^/(builder|app_) {
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:4001;
}
location ~ ^/api/(system|admin|global)/ {
proxy_pass http://127.0.0.1:4002;
}
location /worker/ {
proxy_pass http://127.0.0.1:4002;
rewrite ^/worker/(.*)$ /$1 break;
}
location /api/ {
# calls to the API are rate limited with bursting
limit_req zone=ratelimit burst=20 nodelay;
# 120s timeout on API requests
proxy_read_timeout 120s;
proxy_connect_timeout 120s;
proxy_send_timeout 120s;
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:4001;
}
location /db/ {
proxy_pass http://127.0.0.1:5984;
rewrite ^/db/(.*)$ /$1 break;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://127.0.0.1:9000;
}
client_header_timeout 60;
client_body_timeout 60;
keepalive_timeout 60;
# gzip
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css text/xml application/json application/javascript application/rss+xml application/atom+xml image/svg+xml;
}

87
hosting/single/nginx.conf
View file

@ -1,6 +1,6 @@
user www www;
error_log /etc/nginx/logs/error.log;
pid /etc/nginx/logs/nginx.pid;
user www-data www-data;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 8192;
@ -32,85 +32,6 @@ http {
default "upgrade";
}
server {
listen 10000 default_server;
listen [::]:10000 default_server;
server_name _;
client_max_body_size 1000m;
ignore_invalid_headers off;
proxy_buffering off;
# port_in_redirect off;
include /etc/nginx/sites-enabled/*;
location /app {
proxy_pass http://127.0.0.1:4001;
}
location = / {
proxy_pass http://127.0.0.1:4001;
}
location ~ ^/(builder|app_) {
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:4001;
}
location ~ ^/api/(system|admin|global)/ {
proxy_pass http://127.0.0.1:4002;
}
location /worker/ {
proxy_pass http://127.0.0.1:4002;
rewrite ^/worker/(.*)$ /$1 break;
}
location /api/ {
# calls to the API are rate limited with bursting
limit_req zone=ratelimit burst=20 nodelay;
# 120s timeout on API requests
proxy_read_timeout 120s;
proxy_connect_timeout 120s;
proxy_send_timeout 120s;
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:4001;
}
location /db/ {
proxy_pass http://127.0.0.1:5984;
rewrite ^/db/(.*)$ /$1 break;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://127.0.0.1:9000;
}
client_header_timeout 60;
client_body_timeout 60;
keepalive_timeout 60;
# gzip
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css text/xml application/json application/javascript application/rss+xml application/atom+xml image/svg+xml;
}
}

16
hosting/single/runner.sh
View file

@ -2,6 +2,15 @@ redis-server --requirepass $REDIS_PASSWORD &
/opt/clouseau/bin/clouseau &
/minio/minio server /minio &
/docker-entrypoint.sh /opt/couchdb/bin/couchdb &
/etc/init.d/nginx restart
if [[ ! -z "${CUSTOM_DOMAIN}" ]]; then
# Add monthly cron job to renew certbot certificate
echo -n "* * 2 * * root exec /app/letsencrypt/certificate-renew.sh ${CUSTOM_DOMAIN}" >> /etc/cron.d/certificate-renew
chmod +x /etc/cron.d/certificate-renew
# Request the certbot certificate
/app/letsencrypt/certificate-request.sh ${CUSTOM_DOMAIN}
fi
/etc/init.d/nginx restart
pushd app
pm2 start --name app "yarn run:docker"
@ -10,7 +19,6 @@ pushd worker
pm2 start --name worker "yarn run:docker"
popd
sleep 10
URL=http://${COUCHDB_USER}:${COUCHDB_PASSWORD}@localhost:5984
curl -X PUT ${URL}/_users
curl -X PUT ${URL}/_replicator
sleep infinity
curl -X PUT ${COUCH_DB_URL}/_users
curl -X PUT ${COUCH_DB_URL}/_replicator
sleep infinity

4
hosting/single/test.sh Executable file
View file

@ -0,0 +1,4 @@
#!/bin/bash
id=$(docker run -t -d -p 80:80 budibase:latest)
docker exec -it $id bash
docker kill $id

2
lerna.json
View file

@ -1,5 +1,5 @@
{
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"npmClient": "yarn",
"packages": [
"packages/*"

1
package.json
View file

@ -62,6 +62,7 @@
"build:docker:develop": "node scripts/pinVersions && lerna run build:docker && npm run build:docker:proxy:compose && cd hosting/scripts/linux/ && ./release-to-docker-hub.sh develop && cd -",
"build:docker:airgap": "node hosting/scripts/airgapped/airgappedDockerBuild",
"build:digitalocean": "cd hosting/digitalocean && ./build.sh && cd -",
"build:docker:single:multiarch": "docker buildx build --platform linux/arm64,linux/amd64 -f hosting/single/Dockerfile -t budibase:latest .",
"build:docker:single:image": "docker build -f hosting/single/Dockerfile -t budibase:latest .",
"build:docker:single": "lerna run build && lerna run predocker && npm run build:docker:single:image",
"build:docs": "lerna run build:docs",

1
packages/backend-core/cache.js
View file

@ -3,5 +3,6 @@ const generic = require("./src/cache/generic")
module.exports = {
user: require("./src/cache/user"),
app: require("./src/cache/appMetadata"),
writethrough: require("./src/cache/writethrough"),
...generic,
}

5
packages/backend-core/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/backend-core",
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"description": "Budibase backend core libraries used in server and worker",
"main": "dist/src/index.js",
"types": "dist/src/index.d.ts",
@ -20,6 +20,7 @@
"test:watch": "jest --watchAll"
},
"dependencies": {
"@budibase/types": "^1.0.212-alpha.6",
"@techpass/passport-openidconnect": "0.3.2",
"aws-sdk": "2.1030.0",
"bcrypt": "5.0.1",
@ -57,12 +58,12 @@
]
},
"devDependencies": {
"@budibase/types": "^1.0.206",
"@shopify/jest-koa-mocks": "3.1.5",
"@types/jest": "27.5.1",
"@types/koa": "2.0.52",
"@types/node": "14.18.20",
"@types/node-fetch": "2.6.1",
"@types/pouchdb": "6.4.0",
"@types/redlock": "4.0.3",
"@types/semver": "7.3.7",
"@types/tar-fs": "2.0.1",

2
packages/backend-core/redis.js
View file

@ -1,5 +1,5 @@
module.exports = {
Client: require("./src/redis"),
utils: require("./src/redis/utils"),
clients: require("./src/redis/authRedis"),
clients: require("./src/redis/init"),
}

2
packages/backend-core/src/cache/appMetadata.js vendored
View file

@ -1,4 +1,4 @@
const redis = require("../redis/authRedis")
const redis = require("../redis/init")
const { doWithDB } = require("../db")
const { DocumentTypes } = require("../db/constants")

92
packages/backend-core/src/cache/base/index.ts vendored Normal file
View file

@ -0,0 +1,92 @@
import { getTenantId } from "../../context"
import redis from "../../redis/init"
import RedisWrapper from "../../redis"
function generateTenantKey(key: string) {
const tenantId = getTenantId()
return `${key}:${tenantId}`
}
export = class BaseCache {
client: RedisWrapper | undefined
constructor(client: RedisWrapper | undefined = undefined) {
this.client = client
}
async getClient() {
return !this.client ? await redis.getCacheClient() : this.client
}
async keys(pattern: string) {
const client = await this.getClient()
return client.keys(pattern)
}
/**
* Read only from the cache.
*/
async get(key: string, opts = { useTenancy: true }) {
key = opts.useTenancy ? generateTenantKey(key) : key
const client = await this.getClient()
return client.get(key)
}
/**
* Write to the cache.
*/
async store(
key: string,
value: any,
ttl: number | null = null,
opts = { useTenancy: true }
) {
key = opts.useTenancy ? generateTenantKey(key) : key
const client = await this.getClient()
await client.store(key, value, ttl)
}
/**
* Remove from cache.
*/
async delete(key: string, opts = { useTenancy: true }) {
key = opts.useTenancy ? generateTenantKey(key) : key
const client = await this.getClient()
return client.delete(key)
}
/**
* Read from the cache. Write to the cache if not exists.
*/
async withCache(
key: string,
ttl: number,
fetchFn: any,
opts = { useTenancy: true }
) {
const cachedValue = await this.get(key, opts)
if (cachedValue) {
return cachedValue
}
try {
const fetchedValue = await fetchFn()
await this.store(key, fetchedValue, ttl, opts)
return fetchedValue
} catch (err) {
console.error("Error fetching before cache - ", err)
throw err
}
}
async bustCache(key: string, opts = { client: null }) {
const client = await this.getClient()
try {
await client.delete(generateTenantKey(key))
} catch (err) {
console.error("Error busting cache - ", err)
throw err
}
}
}

75
packages/backend-core/src/cache/generic.js vendored
View file

@ -1,5 +1,6 @@
const redis = require("../redis/authRedis")
const { getTenantId } = require("../context")
const BaseCache = require("./base")
const GENERIC = new BaseCache()
exports.CacheKeys = {
CHECKLIST: "checklist",
@ -16,67 +17,13 @@ exports.TTL = {
ONE_DAY: 86400,
}
function generateTenantKey(key) {
const tenantId = getTenantId()
return `${key}:${tenantId}`
function performExport(funcName) {
return (...args) => GENERIC[funcName](...args)
}
exports.keys = async pattern => {
const client = await redis.getCacheClient()
return client.keys(pattern)
}
/**
* Read only from the cache.
*/
exports.get = async (key, opts = { useTenancy: true }) => {
key = opts.useTenancy ? generateTenantKey(key) : key
const client = await redis.getCacheClient()
const value = await client.get(key)
return value
}
/**
* Write to the cache.
*/
exports.store = async (key, value, ttl, opts = { useTenancy: true }) => {
key = opts.useTenancy ? generateTenantKey(key) : key
const client = await redis.getCacheClient()
await client.store(key, value, ttl)
}
exports.delete = async (key, opts = { useTenancy: true }) => {
key = opts.useTenancy ? generateTenantKey(key) : key
const client = await redis.getCacheClient()
return client.delete(key)
}
/**
* Read from the cache. Write to the cache if not exists.
*/
exports.withCache = async (key, ttl, fetchFn, opts = { useTenancy: true }) => {
const cachedValue = await exports.get(key, opts)
if (cachedValue) {
return cachedValue
}
try {
const fetchedValue = await fetchFn()
await exports.store(key, fetchedValue, ttl, opts)
return fetchedValue
} catch (err) {
console.error("Error fetching before cache - ", err)
throw err
}
}
exports.bustCache = async key => {
const client = await redis.getCacheClient()
try {
await client.delete(generateTenantKey(key))
} catch (err) {
console.error("Error busting cache - ", err)
throw err
}
}
exports.keys = performExport("keys")
exports.get = performExport("get")
exports.store = performExport("store")
exports.delete = performExport("delete")
exports.withCache = performExport("withCache")
exports.bustCache = performExport("bustCache")

59
packages/backend-core/src/cache/tests/writethrough.spec.js vendored Normal file
View file

@ -0,0 +1,59 @@
require("../../../tests/utilities/TestConfiguration")
const { Writethrough } = require("../writethrough")
const { dangerousGetDB } = require("../../db")
const tk = require("timekeeper")
const START_DATE = Date.now()
tk.freeze(START_DATE)
const DELAY = 5000
const db = dangerousGetDB("test")
const db2 = dangerousGetDB("test2")
const writethrough = new Writethrough(db, DELAY), writethrough2 = new Writethrough(db2, DELAY)
describe("writethrough", () => {
describe("put", () => {
let first
it("should be able to store, will go to DB", async () => {
const response = await writethrough.put({ _id: "test", value: 1 })
const output = await db.get(response.id)
first = output
expect(output.value).toBe(1)
})
it("second put shouldn't update DB", async () => {
const response = await writethrough.put({ ...first, value: 2 })
const output = await db.get(response.id)
expect(first._rev).toBe(output._rev)
expect(output.value).toBe(1)
})
it("should put it again after delay period", async () => {
tk.freeze(START_DATE + DELAY + 1)
const response = await writethrough.put({ ...first, value: 3 })
const output = await db.get(response.id)
expect(response.rev).not.toBe(first._rev)
expect(output.value).toBe(3)
})
})
describe("get", () => {
it("should be able to retrieve", async () => {
const response = await writethrough.get("test")
expect(response.value).toBe(3)
})
})
describe("same doc, different databases (tenancy)", () => {
it("should be able to two different databases", async () => {
const resp1 = await writethrough.put({ _id: "db1", value: "first" })
const resp2 = await writethrough2.put({ _id: "db1", value: "second" })
expect(resp1.rev).toBeDefined()
expect(resp2.rev).toBeDefined()
expect((await db.get("db1")).value).toBe("first")
expect((await db2.get("db1")).value).toBe("second")
})
})
})

2
packages/backend-core/src/cache/user.js vendored
View file

@ -1,4 +1,4 @@
const redis = require("../redis/authRedis")
const redis = require("../redis/init")
const { getTenantId, lookupTenantId, doWithGlobalDB } = require("../tenancy")
const env = require("../environment")
const accounts = require("../cloud/accounts")

120
packages/backend-core/src/cache/writethrough.ts vendored Normal file
View file

@ -0,0 +1,120 @@
import BaseCache from "./base"
import { getWritethroughClient } from "../redis/init"
const DEFAULT_WRITE_RATE_MS = 10000
let CACHE: BaseCache | null = null
interface CacheItem {
doc: any
lastWrite: number
}
async function getCache() {
if (!CACHE) {
const client = await getWritethroughClient()
CACHE = new BaseCache(client)
}
return CACHE
}
function makeCacheKey(db: PouchDB.Database, key: string) {
return db.name + key
}
function makeCacheItem(doc: any, lastWrite: number | null = null): CacheItem {
return { doc, lastWrite: lastWrite || Date.now() }
}
export async function put(
db: PouchDB.Database,
doc: any,
writeRateMs: number = DEFAULT_WRITE_RATE_MS
) {
const cache = await getCache()
const key = doc._id
let cacheItem: CacheItem | undefined = await cache.get(makeCacheKey(db, key))
const updateDb = !cacheItem || cacheItem.lastWrite < Date.now() - writeRateMs
let output = doc
if (updateDb) {
const writeDb = async (toWrite: any) => {
// doc should contain the _id and _rev
const response = await db.put(toWrite)
output = {
...doc,
_id: response.id,
_rev: response.rev,
}
}
try {
await writeDb(doc)
} catch (err: any) {
if (err.status !== 409) {
throw err
} else {
// get the rev, update over it - this is risky, may change in future
const readDoc = await db.get(doc._id)
doc._rev = readDoc._rev
await writeDb(doc)
}
}
}
// if we are updating the DB then need to set the lastWrite to now
cacheItem = makeCacheItem(output, updateDb ? null : cacheItem?.lastWrite)
await cache.store(makeCacheKey(db, key), cacheItem)
return { ok: true, id: output._id, rev: output._rev }
}
export async function get(db: PouchDB.Database, id: string): Promise<any> {
const cache = await getCache()
const cacheKey = makeCacheKey(db, id)
let cacheItem: CacheItem = await cache.get(cacheKey)
if (!cacheItem) {
const doc = await db.get(id)
cacheItem = makeCacheItem(doc)
await cache.store(cacheKey, cacheItem)
}
return cacheItem.doc
}
export async function remove(
db: PouchDB.Database,
docOrId: any,
rev?: any
): Promise<void> {
const cache = await getCache()
if (!docOrId) {
throw new Error("No ID/Rev provided.")
}
const id = typeof docOrId === "string" ? docOrId : docOrId._id
rev = typeof docOrId === "string" ? rev : docOrId._rev
try {
await cache.delete(makeCacheKey(db, id))
} finally {
await db.remove(id, rev)
}
}
export class Writethrough {
db: PouchDB.Database
writeRateMs: number
constructor(
db: PouchDB.Database,
writeRateMs: number = DEFAULT_WRITE_RATE_MS
) {
this.db = db
this.writeRateMs = writeRateMs
}
async put(doc: any) {
return put(this.db, doc, this.writeRateMs)
}
async get(id: string) {
return get(this.db, id)
}
async remove(docOrId: any, rev?: any) {
return remove(this.db, docOrId, rev)
}
}

49
packages/backend-core/src/db/pouch.js
View file

@ -1,21 +1,42 @@
const PouchDB = require("pouchdb")
const env = require("../environment")
function getUrlInfo() {
let url = env.COUCH_DB_URL
let username, password, host
const [protocol, rest] = url.split("://")
if (url.includes("@")) {
const hostParts = rest.split("@")
host = hostParts[1]
const authParts = hostParts[0].split(":")
username = authParts[0]
password = authParts[1]
} else {
host = rest
exports.getUrlInfo = (url = env.COUCH_DB_URL) => {
let cleanUrl, username, password, host
if (url) {
// Ensure the URL starts with a protocol
const protoRegex = /^https?:\/\//i
if (!protoRegex.test(url)) {
url = `http://${url}`
}
// Split into protocol and remainder
const split = url.split("://")
const protocol = split[0]
const rest = split.slice(1).join("://")
// Extract auth if specified
if (url.includes("@")) {
// Split into host and remainder
let parts = rest.split("@")
host = parts[parts.length - 1]
let auth = parts.slice(0, -1).join("@")
// Split auth into username and password
if (auth.includes(":")) {
const authParts = auth.split(":")
username = authParts[0]
password = authParts.slice(1).join(":")
} else {
username = auth
}
} else {
host = rest
}
cleanUrl = `${protocol}://${host}`
}
return {
url: `${protocol}://${host}`,
url: cleanUrl,
auth: {
username,
password,
@ -24,7 +45,7 @@ function getUrlInfo() {
}
exports.getCouchInfo = () => {
const urlInfo = getUrlInfo()
const urlInfo = exports.getUrlInfo()
let username
let password
if (env.COUCH_DB_USERNAME) {

62
packages/backend-core/src/db/tests/pouch.spec.js Normal file
View file

@ -0,0 +1,62 @@
require("../../../tests/utilities/TestConfiguration")
const getUrlInfo = require("../pouch").getUrlInfo
describe("pouch", () => {
describe("Couch DB URL parsing", () => {
it("should handle a null Couch DB URL", () => {
const info = getUrlInfo(null)
expect(info.url).toBeUndefined()
expect(info.auth.username).toBeUndefined()
})
it("should be able to parse a basic Couch DB URL", () => {
const info = getUrlInfo("http://host.com")
expect(info.url).toBe("http://host.com")
expect(info.auth.username).toBeUndefined()
})
it("should be able to parse a Couch DB basic URL with HTTPS", () => {
const info = getUrlInfo("https://host.com")
expect(info.url).toBe("https://host.com")
expect(info.auth.username).toBeUndefined()
})
it("should be able to parse a basic Couch DB URL with a custom port", () => {
const info = getUrlInfo("https://host.com:1234")
expect(info.url).toBe("https://host.com:1234")
expect(info.auth.username).toBeUndefined()
})
it("should be able to parse a Couch DB URL with auth", () => {
const info = getUrlInfo("https://user:pass@host.com:1234")
expect(info.url).toBe("https://host.com:1234")
expect(info.auth.username).toBe("user")
expect(info.auth.password).toBe("pass")
})
it("should be able to parse a Couch DB URL with auth and special chars", () => {
const info = getUrlInfo("https://user:s:p@s://@://:d@;][~s@host.com:1234")
expect(info.url).toBe("https://host.com:1234")
expect(info.auth.username).toBe("user")
expect(info.auth.password).toBe("s:p@s://@://:d@;][~s")
})
it("should be able to parse a Couch DB URL without a protocol", () => {
const info = getUrlInfo("host.com:1234")
expect(info.url).toBe("http://host.com:1234")
expect(info.auth.username).toBeUndefined()
})
it("should be able to parse a Couch DB URL with auth and without a protocol", () => {
const info = getUrlInfo("user:s:p@s://@://:d@;][~s@host.com:1234")
expect(info.url).toBe("http://host.com:1234")
expect(info.auth.username).toBe("user")
expect(info.auth.password).toBe("s:p@s://@://:d@;][~s")
})
it("should be able to parse a Couch DB URL with only username auth", () => {
const info = getUrlInfo("https://user@host.com:1234")
expect(info.url).toBe("https://host.com:1234")
expect(info.auth.username).toBe("user")
expect(info.auth.password).toBeUndefined()
})
it("should be able to parse a Couch DB URL with only username auth and without a protocol", () => {
const info = getUrlInfo("user@host.com:1234")
expect(info.url).toBe("http://host.com:1234")
expect(info.auth.username).toBe("user")
expect(info.auth.password).toBeUndefined()
})
})
})

3
packages/backend-core/src/environment.ts
View file

@ -16,7 +16,7 @@ if (!LOADED && isDev() && !isTest()) {
LOADED = true
}
const env: any = {
const env = {
isTest,
isDev,
JWT_SECRET: process.env.JWT_SECRET,
@ -66,6 +66,7 @@ const env: any = {
for (let [key, value] of Object.entries(env)) {
// handle the edge case of "0" to disable an environment variable
if (value === "0") {
// @ts-ignore
env[key] = 0
}
}

10
packages/backend-core/src/objectStore/index.ts
View file

@ -294,6 +294,16 @@ export const uploadDirectory = async (
await Promise.all(uploads)
}
exports.downloadTarballDirect = async (url: string, path: string) => {
path = sanitizeKey(path)
const response = await fetch(url)
if (!response.ok) {
throw new Error(`unexpected response ${response.statusText}`)
}
await streamPipeline(response.body, zlib.Unzip(), tar.extract(path))
}
export const downloadTarball = async (url: any, bucketName: any, path: any) => {
bucketName = sanitizeBucket(bucketName)
path = sanitizeKey(path)

2
packages/backend-core/src/pkg/redis.ts
View file

@ -2,7 +2,7 @@
// The outer exports can't be used as they now reference dist directly
import Client from "../redis"
import utils from "../redis/utils"
import clients from "../redis/authRedis"
import clients from "../redis/init"
export = {
Client,

132
packages/backend-core/src/redis/index.js → packages/backend-core/src/redis/index.ts
View file

@ -1,3 +1,4 @@
import RedisWrapper from "../redis"
const env = require("../environment")
// ioredis mock is all in memory
const Redis = env.isTest() ? require("ioredis-mock") : require("ioredis")
@ -6,24 +7,34 @@ const {
removeDbPrefix,
getRedisOptions,
SEPARATOR,
SelectableDatabases,
} = require("./utils")
const RETRY_PERIOD_MS = 2000
const STARTUP_TIMEOUT_MS = 5000
const CLUSTERED = false
const DEFAULT_SELECT_DB = SelectableDatabases.DEFAULT
// for testing just generate the client once
let CLOSED = false
let CLIENT = env.isTest() ? new Redis(getRedisOptions()) : null
let CLIENTS: { [key: number]: any } = {}
// if in test always connected
let CONNECTED = !!env.isTest()
let CONNECTED = env.isTest()
function connectionError(timeout, err) {
function pickClient(selectDb: number): any {
return CLIENTS[selectDb]
}
function connectionError(
selectDb: number,
timeout: NodeJS.Timeout,
err: Error | string
) {
// manually shut down, ignore errors
if (CLOSED) {
return
}
CLIENT.disconnect()
pickClient(selectDb).disconnect()
CLOSED = true
// always clear this on error
clearTimeout(timeout)
@ -38,59 +49,69 @@ function connectionError(timeout, err) {
* Inits the system, will error if unable to connect to redis cluster (may take up to 10 seconds) otherwise
* will return the ioredis client which will be ready to use.
*/
function init() {
let timeout
function init(selectDb = DEFAULT_SELECT_DB) {
let timeout: NodeJS.Timeout
CLOSED = false
// testing uses a single in memory client
if (env.isTest() || (CLIENT && CONNECTED)) {
let client = pickClient(selectDb)
// already connected, ignore
if (client && CONNECTED) {
return
}
// testing uses a single in memory client
if (env.isTest()) {
CLIENTS[selectDb] = new Redis(getRedisOptions())
}
// start the timer - only allowed 5 seconds to connect
timeout = setTimeout(() => {
if (!CONNECTED) {
connectionError(timeout, "Did not successfully connect in timeout")
connectionError(
selectDb,
timeout,
"Did not successfully connect in timeout"
)
}
}, STARTUP_TIMEOUT_MS)
// disconnect any lingering client
if (CLIENT) {
CLIENT.disconnect()
if (client) {
client.disconnect()
}
const { redisProtocolUrl, opts, host, port } = getRedisOptions(CLUSTERED)
if (CLUSTERED) {
CLIENT = new Redis.Cluster([{ host, port }], opts)
client = new Redis.Cluster([{ host, port }], opts)
} else if (redisProtocolUrl) {
CLIENT = new Redis(redisProtocolUrl)
client = new Redis(redisProtocolUrl)
} else {
CLIENT = new Redis(opts)
client = new Redis(opts)
}
// attach handlers
CLIENT.on("end", err => {
connectionError(timeout, err)
client.on("end", (err: Error) => {
connectionError(selectDb, timeout, err)
})
CLIENT.on("error", err => {
connectionError(timeout, err)
client.on("error", (err: Error) => {
connectionError(selectDb, timeout, err)
})
CLIENT.on("connect", () => {
client.on("connect", () => {
clearTimeout(timeout)
CONNECTED = true
})
CLIENTS[selectDb] = client
}
function waitForConnection() {
function waitForConnection(selectDb: number = DEFAULT_SELECT_DB) {
return new Promise(resolve => {
if (CLIENT == null) {
if (pickClient(selectDb) == null) {
init()
} else if (CONNECTED) {
resolve()
resolve("")
return
}
// check if the connection is ready
const interval = setInterval(() => {
if (CONNECTED) {
clearInterval(interval)
resolve()
resolve("")
}
}, 500)
})
@ -100,25 +121,26 @@ function waitForConnection() {
* Utility function, takes a redis stream and converts it to a promisified response -
* this can only be done with redis streams because they will have an end.
* @param stream A redis stream, specifically as this type of stream will have an end.
* @param client The client to use for further lookups.
* @return {Promise<object>} The final output of the stream
*/
function promisifyStream(stream) {
function promisifyStream(stream: any, client: RedisWrapper) {
return new Promise((resolve, reject) => {
const outputKeys = new Set()
stream.on("data", keys => {
stream.on("data", (keys: string[]) => {
keys.forEach(key => {
outputKeys.add(key)
})
})
stream.on("error", err => {
stream.on("error", (err: Error) => {
reject(err)
})
stream.on("end", async () => {
const keysArray = Array.from(outputKeys)
const keysArray: string[] = Array.from(outputKeys) as string[]
try {
let getPromises = []
for (let key of keysArray) {
getPromises.push(CLIENT.get(key))
getPromises.push(client.get(key))
}
const jsonArray = await Promise.all(getPromises)
resolve(
@ -134,48 +156,52 @@ function promisifyStream(stream) {
})
}
class RedisWrapper {
constructor(db) {
export = class RedisWrapper {
_db: string
_select: number
constructor(db: string, selectDb: number | null = null) {
this._db = db
this._select = selectDb || DEFAULT_SELECT_DB
}
getClient() {
return CLIENT
return pickClient(this._select)
}
async init() {
CLOSED = false
init()
await waitForConnection()
init(this._select)
await waitForConnection(this._select)
return this
}
async finish() {
CLOSED = true
CLIENT.disconnect()
this.getClient().disconnect()
}
async scan(key = "") {
async scan(key = ""): Promise<any> {
const db = this._db
key = `${db}${SEPARATOR}${key}`
let stream
if (CLUSTERED) {
let node = CLIENT.nodes("master")
let node = this.getClient().nodes("master")
stream = node[0].scanStream({ match: key + "*", count: 100 })
} else {
stream = CLIENT.scanStream({ match: key + "*", count: 100 })
stream = this.getClient().scanStream({ match: key + "*", count: 100 })
}
return promisifyStream(stream)
return promisifyStream(stream, this.getClient())
}
async keys(pattern) {
async keys(pattern: string) {
const db = this._db
return CLIENT.keys(addDbPrefix(db, pattern))
return this.getClient().keys(addDbPrefix(db, pattern))
}
async get(key) {
async get(key: string) {
const db = this._db
let response = await CLIENT.get(addDbPrefix(db, key))
let response = await this.getClient().get(addDbPrefix(db, key))
// overwrite the prefixed key
if (response != null && response.key) {
response.key = key
@ -188,39 +214,37 @@ class RedisWrapper {
}
}
async store(key, value, expirySeconds = null) {
async store(key: string, value: any, expirySeconds: number | null = null) {
const db = this._db
if (typeof value === "object") {
value = JSON.stringify(value)
}
const prefixedKey = addDbPrefix(db, key)
await CLIENT.set(prefixedKey, value)
await this.getClient().set(prefixedKey, value)
if (expirySeconds) {
await CLIENT.expire(prefixedKey, expirySeconds)
await this.getClient().expire(prefixedKey, expirySeconds)
}
}
async getTTL(key) {
async getTTL(key: string) {
const db = this._db
const prefixedKey = addDbPrefix(db, key)
return CLIENT.ttl(prefixedKey)
return this.getClient().ttl(prefixedKey)
}
async setExpiry(key, expirySeconds) {
async setExpiry(key: string, expirySeconds: number | null) {
const db = this._db
const prefixedKey = addDbPrefix(db, key)
await CLIENT.expire(prefixedKey, expirySeconds)
await this.getClient().expire(prefixedKey, expirySeconds)
}
async delete(key) {
async delete(key: string) {
const db = this._db
await CLIENT.del(addDbPrefix(db, key))
await this.getClient().del(addDbPrefix(db, key))
}
async clear() {
let items = await this.scan()
await Promise.all(items.map(obj => this.delete(obj.key)))
await Promise.all(items.map((obj: any) => this.delete(obj.key)))
}
}
module.exports = RedisWrapper

13
packages/backend-core/src/redis/authRedis.js → packages/backend-core/src/redis/init.js
View file

@ -2,7 +2,7 @@ const Client = require("./index")
const utils = require("./utils")
const { getRedlock } = require("./redlock")
let userClient, sessionClient, appClient, cacheClient
let userClient, sessionClient, appClient, cacheClient, writethroughClient
let migrationsRedlock
// turn retry off so that only one instance can ever hold the lock
@ -13,6 +13,10 @@ async function init() {
sessionClient = await new Client(utils.Databases.SESSIONS).init()
appClient = await new Client(utils.Databases.APP_METADATA).init()
cacheClient = await new Client(utils.Databases.GENERIC_CACHE).init()
writethroughClient = await new Client(
utils.Databases.WRITE_THROUGH,
utils.SelectableDatabases.WRITE_THROUGH
).init()
// pass the underlying ioredis client to redlock
migrationsRedlock = getRedlock(
cacheClient.getClient(),
@ -25,6 +29,7 @@ process.on("exit", async () => {
if (sessionClient) await sessionClient.finish()
if (appClient) await appClient.finish()
if (cacheClient) await cacheClient.finish()
if (writethroughClient) await writethroughClient.finish()
})
module.exports = {
@ -52,6 +57,12 @@ module.exports = {
}
return cacheClient
},
getWritethroughClient: async () => {
if (!writethroughClient) {
await init()
}
return writethroughClient
},
getMigrationsRedlock: async () => {
if (!migrationsRedlock) {
await init()

37
packages/backend-core/src/redis/utils.js
View file

@ -6,6 +6,14 @@ const SEPARATOR = "-"
const REDIS_URL = !env.REDIS_URL ? "localhost:6379" : env.REDIS_URL
const REDIS_PASSWORD = !env.REDIS_PASSWORD ? "budibase" : env.REDIS_PASSWORD
/**
* These Redis databases help us to segment up a Redis keyspace by prepending the
* specified database name onto the cache key. This means that a single real Redis database
* can be split up a bit; allowing us to use scans on small databases to find some particular
* keys within.
* If writing a very large volume of keys is expected (say 10K+) then it is better to keep these out
* of the default keyspace and use a separate one - the SelectableDatabases can be used for this.
*/
exports.Databases = {
PW_RESETS: "pwReset",
VERIFICATIONS: "verification",
@ -19,6 +27,35 @@ exports.Databases = {
QUERY_VARS: "queryVars",
LICENSES: "license",
GENERIC_CACHE: "data_cache",
WRITE_THROUGH: "writeThrough",
}
/**
* These define the numeric Redis databases that can be access with the SELECT command -
* (https://redis.io/commands/select/). By default a Redis server/cluster will have 16 selectable
* databases, increasing this count increases the amount of CPU/memory required to run the server.
* Ideally new Redis keyspaces should be used sparingly, only when absolutely necessary for performance
* to be maintained. Generally a keyspace can grow to be very large is scans are not needed or desired,
* but if you need to walk through all values in a database periodically then a separate selectable
* keyspace should be used.
*/
exports.SelectableDatabases = {
DEFAULT: 0,
WRITE_THROUGH: 1,
UNUSED_1: 2,
UNUSED_2: 3,
UNUSED_3: 4,
UNUSED_4: 5,
UNUSED_5: 6,
UNUSED_6: 7,
UNUSED_7: 8,
UNUSED_8: 9,
UNUSED_9: 10,
UNUSED_10: 11,
UNUSED_11: 12,
UNUSED_12: 13,
UNUSED_13: 14,
UNUSED_14: 15,
}
exports.SEPARATOR = SEPARATOR

2
packages/backend-core/src/security/sessions.js
View file

@ -1,4 +1,4 @@
const redis = require("../redis/authRedis")
const redis = require("../redis/init")
const { v4: uuidv4 } = require("uuid")
// a week in seconds

4
packages/backend-core/src/utils.js
View file

@ -197,3 +197,7 @@ exports.platformLogout = async ({ ctx, userId, keepActiveSession }) => {
await events.auth.logout()
await userCache.invalidateUser(userId)
}
exports.timeout = timeMs => {
return new Promise(resolve => setTimeout(resolve, timeMs))
}

158
packages/backend-core/yarn.lock
View file

@ -656,6 +656,13 @@
"@types/keygrip" "*"
"@types/node" "*"
"@types/debug@*":
version "4.1.7"
resolved "https://registry.yarnpkg.com/@types/debug/-/debug-4.1.7.tgz#7cc0ea761509124709b8b2d1090d8f6c17aadb82"
integrity sha512-9AonUzyTjXXhEOa0DnqpzZi6VHlqKMswga9EXjpXnnqxwLtdvPPtlO8evrI5D9S6asFRCQ6v+wpiUKbw+vKqyg==
dependencies:
"@types/ms" "*"
"@types/express-serve-static-core@^4.17.18":
version "4.17.28"
resolved "https://registry.yarnpkg.com/@types/express-serve-static-core/-/express-serve-static-core-4.17.28.tgz#c47def9f34ec81dc6328d0b1b5303d1ec98d86b8"
@ -762,6 +769,11 @@
resolved "https://registry.yarnpkg.com/@types/mime/-/mime-1.3.2.tgz#93e25bf9ee75fe0fd80b594bc4feb0e862111b5a"
integrity sha512-YATxVxgRqNH6nHEIsvg6k2Boc1JHI9ZbH5iWFFv/MTkchz3b1ieGDa5T0a9RznNdI0KhVbdbWSN+KWWrQZRxTw==
"@types/ms@*":
version "0.7.31"
resolved "https://registry.yarnpkg.com/@types/ms/-/ms-0.7.31.tgz#31b7ca6407128a3d2bbc27fe2d21b345397f6197"
integrity sha512-iiUgKzV9AuaEkZqkOLDIvlQiL6ltuZd9tGcW3gwpnX8JbuiuhFlEGmmFXEXkN50Cvq7Os88IY2v0dkDqXYWVgA==
"@types/node-fetch@2.6.1":
version "2.6.1"
resolved "https://registry.yarnpkg.com/@types/node-fetch/-/node-fetch-2.6.1.tgz#8f127c50481db65886800ef496f20bbf15518975"
@ -780,6 +792,152 @@
resolved "https://registry.yarnpkg.com/@types/node/-/node-14.18.20.tgz#268f028b36eaf51181c3300252f605488c4f0650"
integrity sha512-Q8KKwm9YqEmUBRsqJ2GWJDtXltBDxTdC4m5vTdXBolu2PeQh8LX+f6BTwU+OuXPu37fLxoN6gidqBmnky36FXA==
"@types/pouchdb-adapter-cordova-sqlite@*":
version "1.0.1"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-cordova-sqlite/-/pouchdb-adapter-cordova-sqlite-1.0.1.tgz#49e5ee6df7cc0c23196fcb340f43a560e74eb1d6"
integrity sha512-nqlXpW1ho3KBg1mUQvZgH2755y3z/rw4UA7ZJCPMRTHofxGMY8izRVw5rHBL4/7P615or0J2udpRYxgkT3D02g==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-fruitdown@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-fruitdown/-/pouchdb-adapter-fruitdown-6.1.3.tgz#9b140ad9645cc56068728acf08ec19ac0046658e"
integrity sha512-Wz1Z1JLOW1hgmFQjqnSkmyyfH7by/iWb4abKn684WMvQfmxx6BxKJpJ4+eulkVPQzzgMMSgU1MpnQOm9FgRkbw==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-http@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-http/-/pouchdb-adapter-http-6.1.3.tgz#6e592d5f48deb6274a21ddac1498dd308096bcf3"
integrity sha512-9Z4TLbF/KJWy/D2sWRPBA+RNU0odQimfdvlDX+EY7rGcd3aVoH8qjD/X0Xcd/0dfBH5pKrNIMFFQgW/TylRCmA==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-idb@*":
version "6.1.4"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-idb/-/pouchdb-adapter-idb-6.1.4.tgz#cb9a18864585d600820cd325f007614c5c3989cd"
integrity sha512-KIAXbkF4uYUz0ZwfNEFLtEkK44mEWopAsD76UhucH92XnJloBysav+TjI4FFfYQyTjoW3S1s6V+Z14CUJZ0F6w==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-leveldb@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-leveldb/-/pouchdb-adapter-leveldb-6.1.3.tgz#17c7e75d75b992050bca15991e97fba575c61bb3"
integrity sha512-ex8NFqQGFwEpFi7AaZ5YofmuemfZNsL3nTFZBUCAKYMBkazQij1pe2ILLStSvJr0XS0qxgXjCEW19T5Wqiiskg==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-localstorage@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-localstorage/-/pouchdb-adapter-localstorage-6.1.3.tgz#0dde02ba6b9d6073a295a20196563942ba9a54bd"
integrity sha512-oor040tye1KKiGLWYtIy7rRT7C2yoyX3Tf6elEJRpjOA7Ja/H8lKc4LaSh9ATbptIcES6MRqZDxtp7ly9hsW3Q==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-memory@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-memory/-/pouchdb-adapter-memory-6.1.3.tgz#9eabdbc890fcf58960ee8b68b8685f837e75c844"
integrity sha512-gVbsIMzDzgZYThFVT4eVNsmuZwVm/4jDxP1sjlgc3qtDIxbtBhGgyNfcskwwz9Zu5Lv1avkDsIWvcxQhnvRlHg==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-node-websql@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-node-websql/-/pouchdb-adapter-node-websql-6.1.3.tgz#aa18bc68af8cf509acd12c400010dcd5fab2243d"
integrity sha512-F/P+os6Jsa7CgHtH64+Z0HfwIcj0hIRB5z8gNhF7L7dxPWoAfkopK5H2gydrP3sQrlGyN4WInF+UJW/Zu1+FKg==
dependencies:
"@types/pouchdb-adapter-websql" "*"
"@types/pouchdb-core" "*"
"@types/pouchdb-adapter-websql@*":
version "6.1.4"
resolved "https://registry.yarnpkg.com/@types/pouchdb-adapter-websql/-/pouchdb-adapter-websql-6.1.4.tgz#359fbe42ccac0ac90b492ddb8c32fafd0aa96d79"
integrity sha512-zMJQCtXC40hBsIDRn0GhmpeGMK0f9l/OGWfLguvczROzxxcOD7REI+e6SEmX7gJKw5JuMvlfuHzkQwjmvSJbtg==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-browser@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-browser/-/pouchdb-browser-6.1.3.tgz#8f33d6ef58d6817d1f6d36979148a1c7f63244d8"
integrity sha512-EdYowrWxW9SWBMX/rux2eq7dbHi5Zeyzz+FF/IAsgQKnUxgeCO5VO2j4zTzos0SDyJvAQU+EYRc11r7xGn5tvA==
dependencies:
"@types/pouchdb-adapter-http" "*"
"@types/pouchdb-adapter-idb" "*"
"@types/pouchdb-adapter-websql" "*"
"@types/pouchdb-core" "*"
"@types/pouchdb-mapreduce" "*"
"@types/pouchdb-replication" "*"
"@types/pouchdb-core@*":
version "7.0.10"
resolved "https://registry.yarnpkg.com/@types/pouchdb-core/-/pouchdb-core-7.0.10.tgz#d1ea1549e7fad6cb579f71459b1bc27252e06a5a"
integrity sha512-mKhjLlWWXyV3PTTjDhzDV1kc2dolO7VYFa75IoKM/hr8Er9eo8RIbS7mJLfC8r/C3p6ihZu9yZs1PWC1LQ0SOA==
dependencies:
"@types/debug" "*"
"@types/pouchdb-find" "*"
"@types/pouchdb-find@*":
version "6.3.7"
resolved "https://registry.yarnpkg.com/@types/pouchdb-find/-/pouchdb-find-6.3.7.tgz#f713534a53c1a7f3fd8fbbfb74131a1b04711ddc"
integrity sha512-b2dr9xoZRK5Mwl8UiRA9l5j9mmCxNfqXuu63H1KZHwJLILjoIIz7BntCvM0hnlnl7Q8P8wORq0IskuaMq5Nnnw==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-http@*":
version "6.1.3"
resolved "https://registry.yarnpkg.com/@types/pouchdb-http/-/pouchdb-http-6.1.3.tgz#09576c0d409da1f8dee34ec5b768415e2472ea52"
integrity sha512-0e9E5SqNOyPl/3FnEIbENssB4FlJsNYuOy131nxrZk36S+y1R/6qO7ZVRypWpGTqBWSuVd7gCsq2UDwO/285+w==
dependencies:
"@types/pouchdb-adapter-http" "*"
"@types/pouchdb-core" "*"
"@types/pouchdb-mapreduce@*":
version "6.1.7"
resolved "https://registry.yarnpkg.com/@types/pouchdb-mapreduce/-/pouchdb-mapreduce-6.1.7.tgz#9ab32d1e0f234f1bf6d1e4c5d7e216e9e23ac0a3"
integrity sha512-WzBwm7tmO9QhfRzVaWT4v6JQSS/fG2OoUDrWrhX87rPe2Pn6laPvdK5li6myNRxCoI/l5e8Jd+oYBAFnaiFucA==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-node@*":
version "6.1.4"
resolved "https://registry.yarnpkg.com/@types/pouchdb-node/-/pouchdb-node-6.1.4.tgz#5214c0169fcfd2237d373380bbd65a934feb5dfb"
integrity sha512-wnTCH8X1JOPpNOfVhz8HW0AvmdHh6pt40MuRj0jQnK7QEHsHS79WujsKTKSOF8QXtPwpvCNSsI7ut7H7tfxxJQ==
dependencies:
"@types/pouchdb-adapter-http" "*"
"@types/pouchdb-adapter-leveldb" "*"
"@types/pouchdb-core" "*"
"@types/pouchdb-mapreduce" "*"
"@types/pouchdb-replication" "*"
"@types/pouchdb-replication@*":
version "6.4.4"
resolved "https://registry.yarnpkg.com/@types/pouchdb-replication/-/pouchdb-replication-6.4.4.tgz#743406c90f13a988fa3e346ea74ce40acd170d00"
integrity sha512-BsE5LKpjJK4iAf6Fx5kyrMw+33V+Ip7uWldUnU2BYrrvtR+MLD22dcImm7DZN1st2wPPb91i0XEnQzvP0w1C/Q==
dependencies:
"@types/pouchdb-core" "*"
"@types/pouchdb-find" "*"
"@types/pouchdb@6.4.0":
version "6.4.0"
resolved "https://registry.yarnpkg.com/@types/pouchdb/-/pouchdb-6.4.0.tgz#f9c41ca64b23029f9bf2eb4bf6956e6431cb79f8"
integrity sha512-eGCpX+NXhd5VLJuJMzwe3L79fa9+IDTrAG3CPaf4s/31PD56hOrhDJTSmRELSXuiqXr6+OHzzP0PldSaWsFt7w==
dependencies:
"@types/pouchdb-adapter-cordova-sqlite" "*"
"@types/pouchdb-adapter-fruitdown" "*"
"@types/pouchdb-adapter-http" "*"
"@types/pouchdb-adapter-idb" "*"
"@types/pouchdb-adapter-leveldb" "*"
"@types/pouchdb-adapter-localstorage" "*"
"@types/pouchdb-adapter-memory" "*"
"@types/pouchdb-adapter-node-websql" "*"
"@types/pouchdb-adapter-websql" "*"
"@types/pouchdb-browser" "*"
"@types/pouchdb-core" "*"
"@types/pouchdb-http" "*"
"@types/pouchdb-mapreduce" "*"
"@types/pouchdb-node" "*"
"@types/pouchdb-replication" "*"
"@types/prettier@^2.1.5":
version "2.6.3"
resolved "https://registry.yarnpkg.com/@types/prettier/-/prettier-2.6.3.tgz#68ada76827b0010d0db071f739314fa429943d0a"

4
packages/bbui/package.json
View file

@ -1,7 +1,7 @@
{
"name": "@budibase/bbui",
"description": "A UI solution used in the different Budibase projects.",
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"license": "MPL-2.0",
"svelte": "src/index.js",
"module": "dist/bbui.es.js",
@ -38,7 +38,7 @@
],
"dependencies": {
"@adobe/spectrum-css-workflow-icons": "^1.2.1",
"@budibase/string-templates": "^1.0.206",
"@budibase/string-templates": "^1.0.212-alpha.6",
"@spectrum-css/actionbutton": "^1.0.1",
"@spectrum-css/actiongroup": "^1.0.1",
"@spectrum-css/avatar": "^3.0.2",

18
packages/bbui/src/Button/Button.svelte
View file

@ -14,6 +14,7 @@
export let active = false
export let tooltip = undefined
export let dataCy
export let newStyles = false
let showTooltip = false
</script>
@ -25,6 +26,7 @@
class:spectrum-Button--warning={warning}
class:spectrum-Button--overBackground={overBackground}
class:spectrum-Button--quiet={quiet}
class:new-styles={newStyles}
class:active
class="spectrum-Button spectrum-Button--size{size.toUpperCase()}"
{disabled}
@ -93,4 +95,20 @@
padding-left: var(--spacing-m);
line-height: 0;
}
.spectrum-Button--primary.new-styles {
background: var(--spectrum-global-color-gray-800);
border-color: transparent;
color: var(--spectrum-global-color-gray-50);
}
.spectrum-Button--primary.new-styles:hover {
background: var(--spectrum-global-color-gray-900);
}
.spectrum-Button--secondary.new-styles {
background: var(--spectrum-global-color-gray-200);
border-color: transparent;
color: var(--spectrum-global-color-gray-900);
}
.spectrum-Button--secondary.new-styles:hover {
background: var(--spectrum-global-color-gray-300);
}
</style>

4
packages/bbui/src/Form/Core/TextField.svelte
View file

@ -112,4 +112,8 @@
.spectrum-Textfield {
width: 100%;
}
input:disabled {
color: var(--spectrum-global-color-gray-600) !important;
-webkit-text-fill-color: var(--spectrum-global-color-gray-600) !important;
}
</style>

2
packages/bbui/src/Typography/Detail.svelte
View file

@ -1,9 +1,7 @@
<script>
import "@spectrum-css/typography/dist/index-vars.css"
// Sizes
export let size = "M"
export let serif = false
</script>

408
packages/builder/cypress/integration/appOverview.spec.js
View file

@ -1,7 +1,7 @@
import filterTests from "../support/filterTests"
import clientPackage from "@budibase/client/package.json"
filterTests(['all'], () => {
filterTests(["all"], () => {
context("Application Overview screen", () => {
before(() => {
cy.login()
@ -10,31 +10,19 @@ filterTests(['all'], () => {
it("Should be accessible from the applications list", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .title").eq(0)
.invoke('attr', 'data-cy')
.then(($dataCy) => {
const dataCy = $dataCy;
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .title")
.eq(0)
.invoke("attr", "data-cy")
.then($dataCy => {
const dataCy = $dataCy
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.click({ force: true })
cy.location().should((loc) => {
expect(loc.pathname).to.eq('/builder/portal/overview/' + dataCy)
cy.location().should(loc => {
expect(loc.pathname).to.eq("/builder/portal/overview/" + dataCy)
})
})
})
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .title").eq(0)
.invoke('attr', 'data-cy')
.then(($dataCy) => {
const dataCy = $dataCy;
cy.get(".appTable .app-row-actions button").contains("View").click({force: true})
cy.location().should((loc) => {
expect(loc.pathname).to.eq('/builder/portal/overview/' + dataCy)
})
})
})
// Find a more suitable place for this.
@ -43,24 +31,28 @@ filterTests(['all'], () => {
cy.get(".appTable .lock-status").eq(0).contains("Locked by you").click()
cy.unlockApp({ owned : true })
cy.unlockApp({ owned: true })
cy.get(".appTable").should("exist")
cy.get(".lock-status").should('not.be.visible')
cy.get(".lock-status").should("not.be.visible")
})
it("Should allow unlocking in the app overview screen", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .app-row-actions button").contains("Edit").eq(0).click({force: true})
cy.get(".appTable .app-row-actions button")
.contains("Edit")
.eq(0)
.click({ force: true })
cy.wait(1000)
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".lock-status").eq(0).contains("Locked by you").click()
cy.unlockApp({ owned : true })
cy.unlockApp({ owned: true })
cy.get(".lock-status").should("not.be.visible")
})
@ -68,107 +60,149 @@ filterTests(['all'], () => {
it("Should reflect the deploy state of an app that hasn't been published.", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".header-right button.spectrum-Button[data-cy='view-app']").should("be.disabled")
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".header-right button.spectrum-Button[data-cy='view-app']").should(
"be.disabled"
)
cy.get(".spectrum-Tabs-item.is-selected").contains("Overview")
cy.get(".overview-tab").should("be.visible")
cy.get(".overview-tab [data-cy='app-status']").within(() => {
cy.get(".status-display").contains("Unpublished")
cy.get(".status-display .icon svg[aria-label='GlobeStrike']").should("exist")
cy.get(".status-display .icon svg[aria-label='GlobeStrike']").should(
"exist"
)
cy.get(".status-text").contains("-")
})
})
it("Should reflect the app deployment state", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .app-row-actions button").contains("Edit").eq(0).click({force: true})
cy.get(".toprightnav button.spectrum-Button").contains("Publish").click({ force : true })
cy.get(".spectrum-Modal [data-cy='deploy-app-modal']").should("be.visible")
.within(() => {
cy.get(".spectrum-Button").contains("Publish").click({ force : true })
cy.wait(1000)
});
cy.get(".appTable .app-row-actions button")
.contains("Edit")
.eq(0)
.click({ force: true })
cy.get(".toprightnav button.spectrum-Button")
.contains("Publish")
.click({ force: true })
cy.get(".spectrum-Modal [data-cy='deploy-app-modal']")
.should("be.visible")
.within(() => {
cy.get(".spectrum-Button").contains("Publish").click({ force: true })
cy.wait(1000)
})
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".header-right button.spectrum-Button[data-cy='view-app']").should("not.be.disabled")
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".header-right button.spectrum-Button[data-cy='view-app']").should(
"not.be.disabled"
)
cy.get(".overview-tab [data-cy='app-status']").within(() => {
cy.get(".status-display").contains("Published")
cy.get(".status-display .icon svg[aria-label='GlobeCheck']").should("exist")
cy.get(".status-display .icon svg[aria-label='GlobeCheck']").should(
"exist"
)
cy.get(".status-text").contains("Last published a few seconds ago")
})
})
it("Should reflect an application that has been unpublished", () => {
it("Should reflect an application that has been unpublished", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .app-row-actions button").contains("Edit").eq(0).click({force: true})
cy.get(".appTable .app-row-actions button")
.contains("Edit")
.eq(0)
.click({ force: true })
cy.get(".deployment-top-nav svg[aria-label='Globe']")
.click({ force: true })
cy.get(".deployment-top-nav svg[aria-label='Globe']").click({
force: true,
})
cy.get("[data-cy='publish-popover-menu']").should("be.visible")
cy.get("[data-cy='publish-popover-menu'] [data-cy='publish-popover-action']")
.click({ force : true })
cy.get("[data-cy='unpublish-modal']").should("be.visible")
.within(() => {
cy.get(".confirm-wrap button").click({ force: true }
)})
cy.get(
"[data-cy='publish-popover-menu'] [data-cy='publish-popover-action']"
).click({ force: true })
cy.get("[data-cy='unpublish-modal']")
.should("be.visible")
.within(() => {
cy.get(".confirm-wrap button").click({ force: true })
})
cy.wait(1000)
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".overview-tab [data-cy='app-status']").within(() => {
cy.get(".status-display").contains("Unpublished")
cy.get(".status-display .icon svg[aria-label='GlobeStrike']").should("exist")
cy.get(".status-display .icon svg[aria-label='GlobeStrike']").should(
"exist"
)
cy.get(".status-text").contains("Last published a few seconds ago")
})
})
it("Should allow the editing of the application icon and colour", () => {
it("Should allow the editing of the application icon and colour", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable", { timeout: 2000})
.within(() => {
cy.get(".app-row-actions-icon").eq(0).click()
})
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.wait(1000)
cy.get(".app-overview-actions-icon").within(() => {
cy.get(".spectrum-Icon").click({ force: true })
})
cy.get(".spectrum-Menu").contains("Edit icon").click()
// Select random icon
cy.get(".grid").within(() => {
cy.get(".icon-item").eq(Math.floor(Math.random() * 23) + 1).click()
cy.get(".icon-item")
.eq(Math.floor(Math.random() * 23) + 1)
.click()
})
// Select random colour
cy.get(".fill").click()
cy.get(".colors").within(() => {
cy.get(".color").eq(Math.floor(Math.random() * 33) + 1).click()
cy.get(".color")
.eq(Math.floor(Math.random() * 33) + 1)
.click()
})
cy.intercept('**/applications/**').as('iconChange')
cy.intercept("**/applications/**").as("iconChange")
cy.get(".spectrum-Button").contains("Save").click({ force: true })
cy.wait("@iconChange")
cy.get("@iconChange").its('response.statusCode')
.should('eq', 200)
cy.get("@iconChange").its("response.statusCode").should("eq", 200)
// Confirm icon has changed from default
// Confirm colour has been applied
cy.get(".appTable", { timeout: 2000})
.within(() => {
cy.get('[aria-label]').eq(0).children()
.should('have.attr', 'xlink:href').and('not.contain', '#spectrum-icon-18-Apps')
cy.get(".title").children().children()
.should('have.attr', 'style').and('contains', 'color')
})
cy.get(".appTable", { timeout: 2000 }).within(() => {
cy.get("[aria-label]")
.eq(0)
.children()
.should("have.attr", "xlink:href")
.and("not.contain", "#spectrum-icon-18-Apps")
cy.get(".title")
.children()
.children()
.should("have.attr", "style")
.and("contains", "color")
})
})
it("Should reflect the last time the application was edited", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".header-right button").contains("Edit").click({ force: true });
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".header-right button").contains("Edit").click({ force: true })
cy.navigateToFrontend()
@ -177,41 +211,51 @@ filterTests(['all'], () => {
})
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".overview-tab [data-cy='edited-by']").within(() => {
cy.get(".editor-name").contains("You")
cy.get(".last-edit-text").contains("Last edited a few seconds ago")
})
});
})
it("Should reflect application version is up-to-date", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".overview-tab [data-cy='app-version']").within(() => {
cy.get(".version-status").contains("You're running the latest!")
})
});
})
it("Should navigate to the settings tab when clicking the App Version card header", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".spectrum-Tabs-item.is-selected").contains("Overview")
cy.get(".overview-tab").should("be.visible")
cy.get(".overview-tab [data-cy='app-version'] .dash-card-header").click({ force : true })
cy.get(".overview-tab [data-cy='app-version'] .dash-card-header").click({
force: true,
})
cy.get(".spectrum-Tabs-item.is-selected").contains("Settings")
cy.get(".settings-tab").should("be.visible")
cy.get(".overview-tab").should("not.exist")
});
})
it("Should allow the upgrading of an application, if available.", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.wait(500)
cy.location().then(loc => {
@ -219,8 +263,7 @@ filterTests(['all'], () => {
const appId = params[params.length - 1]
cy.log(appId)
//Downgrade the app for the test
cy.alterAppVersion(appId, "0.0.1-alpha.0")
.then(()=>{
cy.alterAppVersion(appId, "0.0.1-alpha.0").then(() => {
cy.reload()
cy.wait(1000)
cy.log("Current deployment version: " + clientPackage.version)
@ -228,115 +271,163 @@ filterTests(['all'], () => {
cy.get(".version-status a").contains("Update").click()
cy.get(".spectrum-Tabs-item.is-selected").contains("Settings")
cy.get(".version-section .page-action button").contains("Update").click({ force: true })
cy.intercept('POST', '**/applications/**/client/update').as('updateVersion')
cy.get(".spectrum-Modal.is-open button").contains("Update").click({ force: true })
cy.get(".version-section .page-action button")
.contains("Update")
.click({ force: true })
cy.intercept("POST", "**/applications/**/client/update").as(
"updateVersion"
)
cy.get(".spectrum-Modal.is-open button")
.contains("Update")
.click({ force: true })
cy.wait("@updateVersion")
.its('response.statusCode').should('eq', 200)
.then(() => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".spectrum-Tabs-item").contains("Overview").click({ force: true })
cy.get(".overview-tab [data-cy='app-version']").within(() => {
cy.get(".spectrum-Heading").contains(clientPackage.version)
cy.get(".version-status").contains("You're running the latest!")
.its("response.statusCode")
.should("eq", 200)
.then(() => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".spectrum-Tabs-item")
.contains("Overview")
.click({ force: true })
cy.get(".overview-tab [data-cy='app-version']").within(() => {
cy.get(".spectrum-Heading").contains(clientPackage.version)
cy.get(".version-status").contains("You're running the latest!")
})
})
})
})
});
})
})
it("Should allow editing of the app details.", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".spectrum-Tabs-item").contains("Settings").click()
cy.get(".spectrum-Tabs-item.is-selected").contains("Settings")
cy.get(".settings-tab").should("be.visible")
cy.get(".details-section .page-action button").contains("Edit").click({ force: true })
cy.get(".details-section .page-action button")
.contains("Edit")
.click({ force: true })
cy.updateAppName("sample name")
//publish and check its disabled
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .app-row-actions button").contains("Edit").eq(0).click({force: true})
cy.get(".appTable .app-row-actions button")
.contains("Edit")
.eq(0)
.click({ force: true })
cy.get(".toprightnav button.spectrum-Button").contains("Publish").click({ force : true })
cy.get(".spectrum-Modal [data-cy='deploy-app-modal']").should("be.visible")
.within(() => {
cy.get(".spectrum-Button").contains("Publish").click({ force : true })
cy.wait(1000)
});
cy.get(".toprightnav button.spectrum-Button")
.contains("Publish")
.click({ force: true })
cy.get(".spectrum-Modal [data-cy='deploy-app-modal']")
.should("be.visible")
.within(() => {
cy.get(".spectrum-Button").contains("Publish").click({ force: true })
cy.wait(1000)
})
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.wait(1000)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".spectrum-Tabs-item").contains("Settings").click()
cy.get(".spectrum-Tabs-item.is-selected").contains("Settings")
cy.get(".details-section .page-action .spectrum-Button").scrollIntoView()
cy.wait(1000)
cy.get(".details-section .page-action .spectrum-Button").should("be.disabled")
cy.get(".details-section .page-action .spectrum-Button").should(
"be.disabled"
)
})
xit("Should allow copying of the published application Id", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .app-row-actions").eq(0)
.within(() => {
cy.get(".spectrum-Button").contains("Edit").click({ force: true })
})
cy.get(".appTable .app-row-actions")
.eq(0)
.within(() => {
cy.get(".spectrum-Button").contains("Edit").click({ force: true })
})
cy.publishApp("sample-name")
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".app-overview-actions-icon > .icon").click({ force: true })
cy.get(".app-overview-actions-icon > .icon").click({ force : true })
cy.get("[data-cy='app-overview-menu-popover']")
.eq(0)
.within(() => {
cy.get(".spectrum-Menu-item")
.contains("Copy App ID")
.click({ force: true })
})
cy.get("[data-cy='app-overview-menu-popover']").eq(0).within(() => {
cy.get(".spectrum-Menu-item").contains("Copy App ID").click({ force: true })
})
cy.get(".spectrum-Toast-content").contains("App ID copied to clipboard.").should("be.visible")
cy.get(".spectrum-Toast-content")
.contains("App ID copied to clipboard.")
.should("be.visible")
})
it("Should allow unpublishing of the application", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".app-overview-actions-icon > .icon").click({ force: true })
cy.get(".app-overview-actions-icon > .icon").click({ force : true })
cy.get("[data-cy='app-overview-menu-popover']")
.eq(0)
.within(() => {
cy.get(".spectrum-Menu-item")
.contains("Unpublish")
.click({ force: true })
cy.wait(500)
})
cy.get("[data-cy='app-overview-menu-popover']").eq(0).within(() => {
cy.get(".spectrum-Menu-item").contains("Unpublish").click({ force: true })
cy.wait(500)
})
cy.get("[data-cy='unpublish-modal']").should("be.visible")
.within(() => {
cy.get(".confirm-wrap button").click({ force: true }
)})
cy.get("[data-cy='unpublish-modal']")
.should("be.visible")
.within(() => {
cy.get(".confirm-wrap button").click({ force: true })
})
cy.get(".overview-tab [data-cy='app-status']").within(() => {
cy.get(".status-display").contains("Unpublished")
cy.get(".status-display .icon svg[aria-label='GlobeStrike']").should("exist")
cy.get(".status-display .icon svg[aria-label='GlobeStrike']").should(
"exist"
)
})
})
it("Should allow deleting of the application", () => {
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.get(".appTable .name").eq(0).click()
cy.get(".appTable .app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
cy.get(".app-overview-actions-icon > .icon").click({ force: true })
cy.get(".app-overview-actions-icon > .icon").click({ force : true })
cy.get("[data-cy='app-overview-menu-popover']").eq(0).within(() => {
cy.get(".spectrum-Menu-item").contains("Delete").click({ force: true })
cy.wait(500)
})
cy.get("[data-cy='app-overview-menu-popover']")
.eq(0)
.within(() => {
cy.get(".spectrum-Menu-item")
.contains("Delete")
.click({ force: true })
cy.wait(500)
})
//The test application was renamed earlier in the spec
cy.get(".spectrum-Dialog-grid").within(() => {
@ -344,18 +435,17 @@ filterTests(['all'], () => {
cy.get(".spectrum-Button--warning").click()
})
cy.location().should((loc) => {
expect(loc.pathname).to.eq('/builder/portal/apps')
cy.location().should(loc => {
expect(loc.pathname).to.eq("/builder/portal/apps")
})
cy.get(".appTable").should("not.exist")
cy.get(".welcome .container h1").contains("Let's create your first app!")
})
after(() => {
cy.deleteAllApps()
})
})
})

321
packages/builder/cypress/integration/createComponents.spec.js
View file

@ -1,7 +1,7 @@
import filterTests from "../support/filterTests"
const interact = require('../support/interact')
const interact = require("../support/interact")
filterTests(['all'], () => {
filterTests(["all"], () => {
context("Create Components", () => {
let headlineId
@ -19,17 +19,15 @@ filterTests(['all'], () => {
//Use the tree to delete a selected component
const deleteSelectedComponent = () => {
cy.get(".nav-items-container .nav-item.selected .actions > div > .icon").click({
cy.get(
".nav-items-container .nav-item.selected .actions > div > .icon"
).click({
force: true,
})
cy.get(".spectrum-Popover.is-open li").contains("Delete").click()
cy.get(".spectrum-Modal button").contains("Delete Component").click({
force: true,
})
cy.get(".spectrum-Popover.is-open li")
.contains("Delete")
.click()
cy.get(".spectrum-Modal button")
.contains("Delete Component")
.click({
force: true,
})
}
it("should add a container", () => {
@ -47,18 +45,18 @@ filterTests(['all'], () => {
it("should change the text of the headline", () => {
const text = "Lorem ipsum dolor sit amet."
cy.get("[data-cy=setting-text] input")
.type(text)
.blur()
cy.get("[data-cy=setting-text] input").type(text).blur()
cy.getComponent(headlineId).should("have.text", text)
})
it("should change the size of the headline", () => {
cy.get("[data-cy=setting-size]").scrollIntoView().click()
cy.get("[data-cy=setting-size]").within(() => {
cy.get(".spectrum-Form-item li.spectrum-Menu-item").contains("3XL").click()
cy.get(".spectrum-Form-item li.spectrum-Menu-item")
.contains("3XL")
.click()
})
cy.getComponent(headlineId).within(() => {
cy.get(".spectrum-Heading").should("have.css", "font-size", "60px")
})
@ -66,13 +64,9 @@ filterTests(['all'], () => {
it("should create a form and reset to match schema", () => {
cy.addComponent("Form", "Form").then(() => {
cy.get("[data-cy=setting-dataSource]")
.contains("Custom")
.click()
cy.get(interact.DROPDOWN)
.contains("dog")
.click()
cy.wait(500)
cy.get("[data-cy=setting-dataSource]").contains("Custom").click()
cy.get(interact.DROPDOWN).contains("dog").click()
cy.wait(500)
cy.addComponent("Form", "Field Group").then(fieldGroupId => {
cy.contains("Update form fields").click()
cy.get(".spectrum-Modal")
@ -83,11 +77,9 @@ filterTests(['all'], () => {
cy.contains("name").should("exist")
cy.contains("age").should("exist")
cy.contains("breed").should("exist")
// cy.contains("image").should("exist")
// cy.contains("image").should("exist")
})
cy.getComponent(fieldGroupId)
.find("input")
.should("have.length", 2)
cy.getComponent(fieldGroupId).find("input").should("have.length", 2)
cy.getComponent(fieldGroupId)
.find(interact.SPECTRUM_PICKER)
.should("have.length", 1)
@ -97,191 +89,102 @@ filterTests(['all'], () => {
it("deletes a component", () => {
cy.addComponent("Elements", "Paragraph").then(componentId => {
cy.get("[data-cy=setting-_instanceName] input")
.type(componentId)
.blur()
cy.get(".nav-items-container .nav-item.selected .actions > div > .icon").click({
cy.get("[data-cy=setting-_instanceName] input").type(componentId).blur()
cy.get(
".nav-items-container .nav-item.selected .actions > div > .icon"
).click({
force: true,
})
cy.get(".spectrum-Popover.is-open li").contains("Delete").click()
cy.get(".spectrum-Modal button").contains("Delete Component").click({
force: true,
})
cy.get(".spectrum-Popover.is-open li")
.contains("Delete")
.click()
cy.get(".spectrum-Modal button")
.contains("Delete Component")
.click({
force: true,
})
cy.getComponent(componentId).should("not.exist")
})
})
it("should set focus to the field setting when fields are added to a form", () => {
cy.addComponent("Form", "Form").then((formId) => {
//For deletion
cy.get("[data-cy=setting-_instanceName] input")
.clear()
.type(formId)
.blur()
const componentTypeLabels = ["Text Field", "Number Field", "Password Field",
"Options Picker", "Checkbox", "Long Form Field", "Date Picker", "Attachment",
"JSON Field", "Multi-select Picker", "Relationship Picker"]
const refocusTest = (componentId) => {
cy.getComponent(componentId)
.find(".showMe").should("exist").click({ force: true })
cy.get("[data-cy=setting-field] .spectrum-InputGroup")
.should("have.class", "is-focused")
}
const testFieldFocusOnCreate = (componentLabel) => {
cy.log("Adding: " + componentLabel)
return cy.addComponent("Form", componentLabel).then((componentId) => {
refocusTest(componentId)
cy.get("[data-cy=setting-field] .spectrum-InputGroup")
.should("have.class", "is-focused")
})
}
cy.wait(1000)
cy.wrap(componentTypeLabels).each((label) => {
return testFieldFocusOnCreate(label)
}).then(()=>{
cy.get(".nav-items-container .nav-item").contains(formId).click({ force: true })
deleteSelectedComponent()
})
})
})
it("should clear the iframe place holder when a form field has been set", () => {
cy.addComponent("Form", "Form").then((formId) => {
cy.addComponent("Form", "Form").then(formId => {
//For deletion
cy.get("[data-cy=setting-_instanceName] input")
.clear()
.type(formId)
.blur()
cy.get("[data-cy=setting-dataSource]")
.contains("Custom")
.click()
cy.get(".dropdown")
.contains("dog")
.click()
cy.get("[data-cy=setting-dataSource]").contains("Custom").click()
cy.get(".dropdown").contains("dog").click()
const fieldTypeToColumnName = {
"Text Field" : "name",
"Number Field": "age",
"Options Picker": "breed"
"Text Field": "name",
"Number Field": "age",
"Options Picker": "breed",
}
const componentTypeLabels = Object.keys(fieldTypeToColumnName)
const testFieldFocusOnCreate = (componentLabel) => {
const testFieldFocusOnCreate = componentLabel => {
cy.log("Adding: " + componentLabel)
return cy.addComponent("Form", componentLabel).then((componentId) => {
return cy.addComponent("Form", componentLabel).then(componentId => {
cy.getComponent(componentId)
.find(".placeholder_wrap").should("exist")
cy.get("[data-cy=setting-field] .spectrum-InputGroup")
.should("have.class", "is-focused")
.find(".component-placeholder")
.should("exist")
cy.get("[data-cy=setting-field] button.spectrum-Picker").click()
//Click the first appropriate field. They are filtered by type
cy.get("[data-cy=setting-field] .spectrum-Popover.is-open li.spectrum-Menu-item")
.contains(fieldTypeToColumnName[componentLabel]).click()
cy.get(
"[data-cy=setting-field] .spectrum-Popover.is-open li.spectrum-Menu-item"
)
.contains(fieldTypeToColumnName[componentLabel])
.click()
cy.wait(500)
cy.getComponent(componentId)
.find(".placeholder_wrap").should("not.exist")
.find(".component-placeholder")
.should("not.exist")
})
}
cy.wait(500)
cy.wrap(componentTypeLabels).each((label) => {
return testFieldFocusOnCreate(label)
}).then(()=>{
cy.get(".nav-items-container .nav-item").contains(formId).click({ force: true })
deleteSelectedComponent()
})
})
})
it("should focus a charts settings on data provider if not nested in provider ", () => {
cy.addComponent("Layout", "Container").then((containerId) => {
//For deletion
cy.get("[data-cy=setting-_instanceName] input")
.clear()
.type(containerId)
.blur()
const chartTypeLabels = ["Bar Chart", "Line Chart", "Area Chart", "Pie Chart",
"Donut Chart", "Candlestick Chart"]
const refocusTest = (componentId) => {
cy.getComponent(componentId)
.find(".showMe").should("exist").click({ force: true })
cy.get("[data-cy=dataProvider-prop-control] .spectrum-Picker")
.should("have.class", "is-focused")
}
const testFocusOnCreate = (chartLabel) => {
cy.log("Adding: " + chartLabel)
cy.addComponent("Chart", chartLabel).then((componentId) => {
refocusTest(componentId)
cy.get("[data-cy=dataProvider-prop-control] .spectrum-Picker")
.should("have.class", "is-focused")
cy.wrap(componentTypeLabels)
.each(label => {
return testFieldFocusOnCreate(label)
})
.then(() => {
cy.get(".nav-items-container .nav-item")
.contains(formId)
.click({ force: true })
deleteSelectedComponent()
})
}
cy.wait(1000)
cy.wrap(chartTypeLabels).each((label) => {
return testFocusOnCreate(label)
})
.then(()=>{
cy.get(".nav-items-container .nav-item").contains(containerId).click({ force: true })
deleteSelectedComponent()
})
})
})
it("should populate the provider for charts with a data provider in its path", () => {
cy.addComponent("Data", "Data Provider").then((providerId) => {
cy.addComponent("Data", "Data Provider").then(providerId => {
//For deletion
cy.get("[data-cy=setting-_instanceName] input")
.clear()
.type(providerId)
.blur()
cy.get("[data-cy=setting-dataSource]")
.contains("Choose an option")
.click()
cy.get(`[data-cy=dataSource-popover-${providerId}] ul li`)
.contains("dog")
.click()
const chartTypeLabels = ["Bar Chart", "Line Chart", "Area Chart", "Pie Chart",
"Donut Chart", "Candlestick Chart"]
const chartTypeLabels = [
"Bar Chart",
"Line Chart",
"Area Chart",
"Pie Chart",
"Donut Chart",
"Candlestick Chart",
]
const testFocusOnCreate = (chartLabel) => {
const testFocusOnCreate = chartLabel => {
cy.log("Adding: " + chartLabel)
cy.addComponent("Chart", chartLabel).then((componentId) => {
cy.get("[data-cy=dataProvider-prop-control] .spectrum-Picker")
.should("not.have.class", "is-focused")
cy.addComponent("Chart", chartLabel).then(componentId => {
cy.get(
"[data-cy=dataProvider-prop-control] .spectrum-Picker"
).should("not.have.class", "is-focused")
// Pre populated.
cy.get("[data-cy=dataProvider-prop-control] .spectrum-Picker-label")
@ -289,114 +192,86 @@ filterTests(['all'], () => {
.should("exist")
})
}
cy.wait(1000)
cy.wrap(chartTypeLabels).each((label) => {
return testFocusOnCreate(label)
})
.then(()=>{
cy.get(".nav-items-container .nav-item").contains(providerId).click({ force: true })
deleteSelectedComponent()
})
cy.wrap(chartTypeLabels)
.each(label => {
return testFocusOnCreate(label)
})
.then(() => {
cy.get(".nav-items-container .nav-item")
.contains(providerId)
.click({ force: true })
deleteSelectedComponent()
})
})
})
it("should replace the placeholder when a url is set on an image", () => {
cy.addComponent("Elements", "Image").then((imageId) => {
cy.get("[data-cy=url-prop-control] .spectrum-InputGroup")
.should("have.class", "is-focused")
cy.addComponent("Elements", "Image").then(imageId => {
cy.get("[data-cy=setting-_instanceName] input")
.clear()
.type(imageId)
.blur()
//return $("New Data Provider.Rows")[0]["Attachment"][0]["url"]
//No minio, so just enter something local that will not reslove
cy.get("[data-cy=url-prop-control] input[type=text]")
.type("cypress/fixtures/ghost.png")
.blur()
cy.getComponent(imageId)
.find(".placeholder_wrap").should("not.exist")
cy.getComponent(imageId)
.find(`img[alt=${imageId}]`).should("exist")
.find(".component-placeholder")
.should("not.exist")
cy.getComponent(imageId).find(`img[alt=${imageId}]`).should("exist")
cy.get(".nav-items-container .nav-item")
.contains(imageId)
.click({ force: true })
deleteSelectedComponent()
})
})
it("should add a markdown component.", () => {
cy.addComponent("Elements", "Markdown Viewer").then((markdownId) => {
cy.get("[data-cy=value-prop-control] .spectrum-InputGroup")
.should("have.class", "is-focused")
cy.addComponent("Elements", "Markdown Viewer").then(markdownId => {
cy.get("[data-cy=setting-_instanceName] input")
.clear()
.type(markdownId)
.blur()
cy.get("[data-cy=value-prop-control] input[type=text].spectrum-Textfield-input")
.type("# Hi").blur()
cy.get(
"[data-cy=value-prop-control] input[type=text].spectrum-Textfield-input"
)
.type("# Hi")
.blur()
cy.getComponent(markdownId)
.find(".placeholder_wrap").should("not.exist")
.find(".component-placeholder")
.should("not.exist")
cy.getComponent(markdownId)
.find(".editor-preview-full h1").contains("Hi")
.find(".editor-preview-full h1")
.contains("Hi")
cy.get(".nav-items-container .nav-item")
.contains(markdownId)
.click({ force: true })
deleteSelectedComponent()
})
})
it("should direct the user when adding an Icon component.", () => {
cy.addComponent("Elements", "Icon").then((iconId) => {
cy.get("[data-cy=icon-prop-control] .spectrum-ActionButton")
.should("have.class", "is-focused")
cy.getComponent(iconId)
.find(".placeholder_wrap").should("exist")
cy.addComponent("Elements", "Icon").then(iconId => {
cy.getComponent(iconId).find(".component-placeholder").should("exist")
cy.get("[data-cy=setting-_instanceName] input")
.clear()
.type(iconId)
.blur()
cy.get("[data-cy=icon-prop-control] .spectrum-ActionButton").click()
cy.get("[data-cy=icon-popover].spectrum-Popover.is-open").within(() => {
cy.get(".search-input input")
.type("save")
.blur()
cy.get(".search-input input").type("save").blur()
cy.get(".search-input button").click({ force: true })
cy.get(".icon-area .icon-container").eq(0).click({ force: true })
})
cy.getComponent(iconId)
.find(".placeholder_wrap").should("not.exist")
cy.getComponent(iconId)
.find("i.ri-save-fill").should("exist")
.find(".component-placeholder")
.should("not.exist")
cy.getComponent(iconId).find("i.ri-save-fill").should("exist")
cy.get(".nav-items-container .nav-item")
.contains(iconId)
.click({ force: true })
deleteSelectedComponent()
})
})

71
packages/builder/cypress/integration/renameAnApplication.spec.js
View file

@ -1,7 +1,7 @@
import filterTests from "../support/filterTests"
const interact = require('../support/interact')
const interact = require("../support/interact")
filterTests(['all'], () => {
filterTests(["all"], () => {
context("Rename an App", () => {
beforeEach(() => {
cy.login()
@ -32,12 +32,15 @@ filterTests(['all'], () => {
const appRename = "Cypress Renamed"
// Publish the app
cy.get(interact.TOP_RIGHT_NAV)
cy.get(interact.SPECTRUM_BUTTON).contains("Publish").click({ force: true })
cy.get(interact.SPECTRUM_DIALOG_GRID)
.within(() => {
// Click publish again within the modal
cy.get(interact.SPECTRUM_BUTTON).contains("Publish").click({ force: true })
})
cy.get(interact.SPECTRUM_BUTTON)
.contains("Publish")
.click({ force: true })
cy.get(interact.SPECTRUM_DIALOG_GRID).within(() => {
// Click publish again within the modal
cy.get(interact.SPECTRUM_BUTTON)
.contains("Publish")
.click({ force: true })
})
// Rename app, Search for app, Confirm name was changed
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.wait(500)
@ -64,14 +67,20 @@ filterTests(['all'], () => {
const appName = "Cypress Tests"
cy.visit(`${Cypress.config().baseUrl}/builder`)
cy.wait(500)
cy.get(interact.SPECTRUM_BUTTON).contains("Create app").click({ force: true })
cy.get(interact.SPECTRUM_BUTTON)
.contains("Create app")
.click({ force: true })
cy.contains(/Start from scratch/).click()
cy.get(interact.SPECTRUM_MODAL)
.within(() => {
cy.get("input").eq(0).type(appName)
cy.get(interact.SPECTRUM_BUTTON_GROUP).contains("Create app").click({ force: true })
cy.get(interact.ERROR).should("have.text", "Another app with the same name already exists")
})
cy.get(interact.SPECTRUM_MODAL).within(() => {
cy.get("input").eq(0).type(appName)
cy.get(interact.SPECTRUM_BUTTON_GROUP)
.contains("Create app")
.click({ force: true })
cy.get(interact.ERROR).should(
"have.text",
"Another app with the same name already exists"
)
})
})
it("should validate application names", () => {
@ -89,7 +98,10 @@ filterTests(['all'], () => {
cy.reload()
cy.wait(1000)
renameApp(numberName, specialCharName)
cy.get(interact.ERROR).should("have.text", "App name must be letters, numbers and spaces only")
cy.get(interact.ERROR).should(
"have.text",
"App name must be letters, numbers and spaces only"
)
// Set app name back to Cypress Tests
cy.reload()
cy.wait(1000)
@ -98,24 +110,19 @@ filterTests(['all'], () => {
const renameApp = (originalName, changedName, published, noName) => {
cy.searchForApplication(originalName)
cy.get(interact.APP_TABLE)
.within(() => {
cy.get(interact.AREA_LABEL_MORE).eq(0).click()
})
// Check for when an app is published
if (published == true) {
// Should not have Edit as option, will unpublish app
cy.should("not.have.value", "Edit")
cy.get(interact.SPECTRUM_MENU).contains("Unpublish").click()
cy.get(interact.SPECTRUM_DIALOG_GRID).contains("Unpublish app").click()
cy.get(".appTable > :nth-child(5) > :nth-child(2) > .spectrum-Icon").click()
}
cy.get(interact.APP_ROW_ACTION_MENU_POPOVER).eq(0).within(() => {
cy.get(interact.SPECTRUM_MENU_ITEMM).contains("Edit").click({ force: true })
cy.get(interact.APP_TABLE).within(() => {
cy.get(".app-row-actions button")
.contains("Manage")
.eq(0)
.click({ force: true })
})
cy.get(".spectrum-Tabs-item").contains("Settings").click()
cy.get(".spectrum-Tabs-item.is-selected").contains("Settings")
cy.get(".settings-tab").should("be.visible")
cy.get(".details-section .page-action button")
.contains("Edit")
.click({ force: true })
cy.updateAppName(changedName, noName)
}
})
})

35
packages/builder/cypress/support/commands.js
View file

@ -144,20 +144,27 @@ Cypress.Commands.add("deleteApp", name => {
return
}
// Go to app overview
const appIdParsed = appId.split("_").pop()
const actionEleId = `[data-cy=row_actions_${appIdParsed}]`
cy.get(actionEleId).within(() => {
cy.get(".spectrum-Icon").eq(0).click({ force: true })
cy.contains("Manage").click({ force: true })
})
cy.get(".spectrum-Menu").then($menu => {
if ($menu.text().includes("Unpublish")) {
cy.get(".spectrum-Menu").contains("Unpublish").click()
cy.get(".spectrum-Dialog-grid").contains("Unpublish app").click()
cy.wait(1000)
// Unpublish first if needed
cy.get(`[data-cy="app-status"]`).then($status => {
if ($status.text().includes("Last published")) {
cy.contains("Unpublish").click()
cy.get(".spectrum-Modal").within(() => {
cy.contains("Unpublish app").click()
})
}
})
cy.get(actionEleId).within(() => {
cy.get(".spectrum-Icon").eq(0).click({ force: true })
// Delete app
cy.get(".app-overview-actions-icon").within(() => {
cy.get(".spectrum-Icon").click({ force: true })
})
cy.get(".spectrum-Menu").contains("Delete").click()
cy.get(".spectrum-Dialog-grid").within(() => {
@ -180,17 +187,7 @@ Cypress.Commands.add("deleteAllApps", () => {
.its("body")
.then(val => {
for (let i = 0; i < val.length; i++) {
const appIdParsed = val[i].appId.split("_").pop()
const actionEleId = `[data-cy=row_actions_${appIdParsed}]`
cy.get(actionEleId).within(() => {
cy.get(".spectrum-Icon").eq(0).click({ force: true })
})
cy.get(".spectrum-Menu").contains("Delete").click()
cy.get(".spectrum-Dialog-grid").within(() => {
cy.get("input").type(val[i].name)
cy.get(".spectrum-Button--warning").click()
})
cy.deleteApp(val[i].name)
cy.reload()
}
})
@ -364,7 +361,7 @@ Cypress.Commands.add("createTable", (tableName, initialTable) => {
cy.get(`[data-cy="new-table"]`).click()
}
cy.wait(5000)
cy.get(".spectrum-Dialog-grid")
cy.get(".item")
.contains("Budibase DB")
.click({ force: true })
.then(() => {

10
packages/builder/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/builder",
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"license": "GPL-3.0",
"private": true,
"scripts": {
@ -69,10 +69,10 @@
}
},
"dependencies": {
"@budibase/bbui": "^1.0.206",
"@budibase/client": "^1.0.206",
"@budibase/frontend-core": "^1.0.206",
"@budibase/string-templates": "^1.0.206",
"@budibase/bbui": "^1.0.212-alpha.6",
"@budibase/client": "^1.0.212-alpha.6",
"@budibase/frontend-core": "^1.0.212-alpha.6",
"@budibase/string-templates": "^1.0.212-alpha.6",
"@sentry/browser": "5.19.1",
"@spectrum-css/page": "^3.0.1",
"@spectrum-css/vars": "^3.0.1",

116
packages/builder/src/builderStore/store/frontend.js
View file

@ -190,6 +190,7 @@ export const getFrontendStore = () => {
// Build array of promises to speed up bulk deletions
const promises = []
let deleteUrls = []
screensToDelete.forEach(screen => {
// Delete the screen
promises.push(
@ -199,14 +200,10 @@ export const getFrontendStore = () => {
})
)
// Remove links to this screen
promises.push(
store.actions.components.links.delete(
screen.routing.route,
screen.props._instanceName
)
)
deleteUrls.push(screen.routing.route)
})
promises.push(store.actions.links.delete(deleteUrls))
await Promise.all(promises)
const deletedIds = screensToDelete.map(screen => screen._id)
store.update(state => {
@ -578,89 +575,38 @@ export const getFrontendStore = () => {
})
await store.actions.preview.saveSelected()
},
links: {
save: async (url, title) => {
const layout = get(mainLayout)
if (!layout) {
return
}
},
links: {
save: async (url, title) => {
const layout = get(mainLayout)
if (!layout) {
return
}
// Add link setting to main layout
if (layout.props._component.endsWith("layout")) {
// If using a new SDK, add to the layout component settings
if (!layout.props.links) {
layout.props.links = []
}
layout.props.links.push({
text: title,
url,
})
} else {
// If using an old SDK, add to the navigation component
// TODO: remove this when we can assume everyone has updated
const nav = findComponentType(
layout.props,
"@budibase/standard-components/navigation"
)
if (!nav) {
return
}
// Add link setting to main layout
if (!layout.props.links) {
layout.props.links = []
}
layout.props.links.push({
text: title,
url,
})
let newLink
if (nav._children && nav._children.length) {
// Clone an existing link if one exists
newLink = cloneDeep(nav._children[0])
await store.actions.layouts.save(layout)
},
delete: async urls => {
const layout = get(mainLayout)
if (!layout?.props.links?.length) {
return
}
// Set our new props
newLink._id = Helpers.uuid()
newLink._instanceName = `${title} Link`
newLink.url = url
newLink.text = title
} else {
// Otherwise create vanilla new link
newLink = {
...store.actions.components.createInstance("link"),
url,
text: title,
_instanceName: `${title} Link`,
}
nav._children = [...nav._children, newLink]
}
}
// Filter out the URLs to delete
urls = Array.isArray(urls) ? urls : [urls]
layout.props.links = layout.props.links.filter(
link => !urls.includes(link.url)
)
// Save layout
await store.actions.layouts.save(layout)
},
delete: async (url, title) => {
const layout = get(mainLayout)
if (!layout) {
return
}
// Add link setting to main layout
if (layout.props._component.endsWith("layout")) {
// If using a new SDK, add to the layout component settings
layout.props.links = layout.props.links.filter(
link => !(link.text === title && link.url === url)
)
} else {
// If using an old SDK, add to the navigation component
// TODO: remove this when we can assume everyone has updated
const nav = findComponentType(
layout.props,
"@budibase/standard-components/navigation"
)
if (!nav) {
return
}
nav._children = nav._children.filter(
child => !(child.url === url && child.text === title)
)
}
// Save layout
await store.actions.layouts.save(layout)
},
await store.actions.layouts.save(layout)
},
},
settings: {

2
packages/builder/src/builderStore/store/screenTemplates/newRowScreen.js
View file

@ -15,7 +15,7 @@ export default function (tables) {
name: `${table.name} - New`,
create: () => createScreen(table),
id: NEW_ROW_TEMPLATE,
table: table.name,
table: table._id,
}
})
}

2
packages/builder/src/builderStore/store/screenTemplates/rowDetailScreen.js
View file

@ -17,7 +17,7 @@ export default function (tables) {
name: `${table.name} - Detail`,
create: () => createScreen(table),
id: ROW_DETAIL_TEMPLATE,
table: table.name,
table: table._id,
}
})
}

2
packages/builder/src/builderStore/store/screenTemplates/rowListScreen.js
View file

@ -10,7 +10,7 @@ export default function (tables) {
name: `${table.name} - List`,
create: () => createScreen(table),
id: ROW_LIST_TEMPLATE,
table: table.name,
table: table._id,
}
})
}

80
packages/builder/src/components/backend/DatasourceNavigator/modals/CreateDatasourceModal.svelte
View file

@ -5,12 +5,13 @@
Body,
Layout,
Detail,
Heading,
notifications,
} from "@budibase/bbui"
import { onMount } from "svelte"
import ICONS from "../icons"
import { API } from "api"
import { IntegrationNames, IntegrationTypes } from "constants/backend"
import { IntegrationTypes } from "constants/backend"
import CreateTableModal from "components/backend/TableNavigator/modals/CreateTableModal.svelte"
import DatasourceConfigModal from "components/backend/DatasourceNavigator/modals/DatasourceConfigModal.svelte"
import GoogleDatasourceConfigModal from "components/backend/DatasourceNavigator/modals/GoogleDatasourceConfigModal.svelte"
@ -118,7 +119,7 @@
<Modal bind:this={modal}>
<ModalContent
disabled={!Object.keys(integration).length}
title="Data"
title="Add data source"
confirmText="Continue"
showSecondaryButton={showImportButton}
secondaryButtonText="Import"
@ -129,27 +130,25 @@
chooseNextModal()
}}
>
<Layout noPadding>
<Body size="S"
>All apps need data. You can connect to a data source below, or add data
to your app using Budibase's built-in database.
</Body>
<Layout noPadding gap="XS">
<Body size="S">Get started with Budibase DB</Body>
<div
class:selected={integration.type === IntegrationTypes.INTERNAL}
on:click={() => selectIntegration(IntegrationTypes.INTERNAL)}
class="item hoverable"
>
<div class="item-body">
<svelte:component this={ICONS.BUDIBASE} height="18" width="18" />
<span class="icon-spacing"> <Body size="S">Budibase DB</Body></span>
<div class="item-body with-type">
<svelte:component this={ICONS.BUDIBASE} height="20" width="20" />
<div class="text">
<Heading size="XXS">Budibase DB</Heading>
<Detail size="S" class="type">Non-relational</Detail>
</div>
</div>
</div>
</Layout>
<Layout gap="XS" noPadding>
<div class="title-spacing">
<Detail size="S">Connect to data source</Detail>
</div>
<Layout noPadding gap="XS">
<Body size="S">Connect to an external data source</Body>
<div class="item-list">
{#each Object.entries(integrations).filter(([key]) => key !== IntegrationTypes.INTERNAL) as [integrationType, schema]}
<div
@ -157,18 +156,18 @@
on:click={() => selectIntegration(integrationType)}
class="item hoverable"
>
<div class="item-body">
<div class="item-body" class:with-type={!!schema.type}>
<svelte:component
this={ICONS[integrationType]}
height="18"
width="18"
height="20"
width="20"
/>
<span class="icon-spacing">
<Body size="S"
>{schema.name || IntegrationNames[integrationType]}</Body
></span
>
<div class="text">
<Heading size="XXS">{schema.friendlyName}</Heading>
{#if schema.type}
<Detail size="S">{schema.type || ""}</Detail>
{/if}
</div>
</div>
</div>
{/each}
@ -178,13 +177,6 @@
</Modal>
<style>
.icon-spacing {
margin-left: var(--spacing-m);
}
.item-body {
display: flex;
margin-left: var(--spacing-m);
}
.item-list {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(150px, 1fr));
@ -195,21 +187,35 @@
cursor: pointer;
display: grid;
grid-gap: var(--spectrum-alias-grid-margin-xsmall);
padding: var(--spectrum-alias-item-padding-s);
padding: var(--spectrum-alias-item-padding-s)
var(--spectrum-alias-item-padding-m);
background: var(--spectrum-alias-background-color-secondary);
transition: 0.3s all;
transition: background 0.13s ease-out;
border: solid var(--spectrum-alias-border-color);
border-radius: 5px;
box-sizing: border-box;
border-width: 2px;
}
.selected {
.item:hover,
.item.selected {
background: var(--spectrum-alias-background-color-tertiary);
}
.item:hover,
.selected {
background: var(--spectrum-alias-background-color-tertiary);
.item-body {
display: flex;
flex-direction: row;
justify-content: flex-start;
align-items: center;
gap: var(--spacing-m);
}
.item-body.with-type {
align-items: flex-start;
}
.item-body.with-type :global(svg) {
margin-top: 4px;
}
.text :global(.spectrum-Detail) {
color: var(--spectrum-global-color-gray-700);
}
</style>

6
packages/builder/src/components/common/AppLockModal.svelte
View file

@ -87,7 +87,7 @@
{#if lockedByYou && getExpiryDuration(app) > 0}
<span class="lock-expiry-body">
{processStringSync(
"This lock will expire in {{ duration time 'millisecond' }} from now",
"This lock will expire in {{ duration time 'millisecond' }} from now.",
{
time: getExpiryDuration(app),
}
@ -141,4 +141,8 @@
gap: var(--spacing-s);
max-width: 175px;
}
.lock-status-text {
font-weight: 400;
color: var(--spectrum-global-color-gray-800);
}
</style>

16
packages/builder/src/components/design/NavigationPanel/DatasourceModal.svelte
View file

@ -14,14 +14,14 @@
let selectedScreens = [...initalScreens]
const toggleScreenSelection = (table, datasource) => {
if (selectedScreens.find(s => s.table === table.name)) {
if (selectedScreens.find(s => s.table === table._id)) {
selectedScreens = selectedScreens.filter(
screen => screen.table !== table.name
screen => screen.table !== table._id
)
} else {
let partialTemplates = getTemplates($store, $tables.list).reduce(
(acc, template) => {
if (template.table === table.name) {
if (template.table === table._id) {
template.datasource = datasource.name
acc.push(template)
}
@ -88,7 +88,7 @@
<div
class="data-source-entry"
class:selected={selectedScreens.find(
x => x.table === table.name
x => x.table === table._id
)}
on:click={() => toggleScreenSelection(table, datasource)}
>
@ -102,8 +102,7 @@
<use xlink:href="#spectrum-icon-18-Table" />
</svg>
{table.name}
{#if selectedScreens.find(x => x.table === table.name)}
{#if selectedScreens.find(x => x.table === table._id)}
<span class="data-source-check">
<Icon size="S" name="CheckmarkCircle" />
</span>
@ -116,7 +115,7 @@
<div
class="data-source-entry"
class:selected={selectedScreens.find(
x => x.table === datasource.entities[table_key].name
x => x.table === datasource.entities[table_key]._id
)}
on:click={() =>
toggleScreenSelection(
@ -134,8 +133,7 @@
<use xlink:href="#spectrum-icon-18-Table" />
</svg>
{datasource.entities[table_key].name}
{#if selectedScreens.find(x => x.table === datasource.entities[table_key].name)}
{#if selectedScreens.find(x => x.table === datasource.entities[table_key]._id)}
<span class="data-source-check">
<Icon size="S" name="CheckmarkCircle" />
</span>

3
packages/builder/src/components/design/NavigationPanel/ScreenWizard.svelte
View file

@ -66,7 +66,7 @@
// Add link in layout for list screens
if (screen.props._instanceName.endsWith("List")) {
await store.actions.components.links.save(
await store.actions.links.save(
screen.routing.route,
screen.routing.route.split("/")[1]
)
@ -131,6 +131,7 @@
const screens = selectedTemplates.map(template => {
let screenTemplate = template.create()
screenTemplate.datasource = template.datasource
screenTemplate.autoTableId = template.table
return screenTemplate
})
await createScreens({ screens, screenAccessRole })

63
packages/builder/src/components/design/PropertiesPanel/PropertyControls/ButtonActionEditor/actions/ExportData.svelte
View file

@ -1,27 +1,18 @@
<script>
import { Label, Select, Body } from "@budibase/bbui"
import { findAllMatchingComponents } from "builderStore/componentUtils"
import { Label, Select, Body, Multiselect } from "@budibase/bbui"
import {
findAllMatchingComponents,
findComponent,
} from "builderStore/componentUtils"
import { currentAsset } from "builderStore"
import { onMount } from "svelte"
import {
getDatasourceForProvider,
getSchemaForDatasource,
} from "builderStore/dataBinding"
export let parameters
$: tables = findAllMatchingComponents($currentAsset?.props, component =>
component._component.endsWith("table")
).map(table => ({
label: table._instanceName,
value: table._id,
}))
$: tableBlocks = findAllMatchingComponents($currentAsset?.props, component =>
component._component.endsWith("tableblock")
).map(block => ({
label: block._instanceName,
value: `${block._id}-table`,
}))
$: componentOptions = tables.concat(tableBlocks)
const FORMATS = [
{
label: "CSV",
@ -33,6 +24,32 @@
},
]
$: tables = findAllMatchingComponents($currentAsset?.props, component =>
component._component.endsWith("table")
).map(table => ({
label: table._instanceName,
value: table._id,
}))
$: tableBlocks = findAllMatchingComponents($currentAsset?.props, component =>
component._component.endsWith("tableblock")
).map(block => ({
label: block._instanceName,
value: `${block._id}-table`,
}))
$: componentOptions = tables.concat(tableBlocks)
$: columnOptions = getColumnOptions(parameters.tableComponentId)
const getColumnOptions = tableId => {
// Strip block suffix if block component
if (tableId?.includes("-")) {
tableId = tableId.split("-")[0]
}
const selectedTable = findComponent($currentAsset?.props, tableId)
const datasource = getDatasourceForProvider($currentAsset, selectedTable)
const { schema } = getSchemaForDatasource($currentAsset, datasource)
return Object.keys(schema || {})
}
onMount(() => {
if (!parameters.type) {
parameters.type = "csv"
@ -53,10 +70,16 @@
<Select
bind:value={parameters.tableComponentId}
options={componentOptions}
on:change={() => (parameters.columns = [])}
/>
<Label small>Export as</Label>
<Select bind:value={parameters.type} options={FORMATS} />
<Label small>Export columns</Label>
<Multiselect
placeholder="All columns"
bind:value={parameters.columns}
options={columnOptions}
/>
</div>
</div>
@ -80,7 +103,7 @@
display: grid;
column-gap: var(--spacing-xs);
row-gap: var(--spacing-s);
grid-template-columns: 70px 1fr;
grid-template-columns: 90px 1fr;
align-items: center;
}
</style>

3
packages/builder/src/components/design/PropertiesPanel/PropertyControls/FilterEditor/FilterDrawer.svelte
View file

@ -54,8 +54,9 @@
}
const onFieldChange = (expression, field) => {
// Update the field type
// Update the field types
expression.type = enrichedSchemaFields.find(x => x.name === field)?.type
expression.externalType = getSchema(expression)?.externalType
// Ensure a valid operator is set
const validOperators = LuceneUtils.getValidOperatorsForType(

53
packages/builder/src/components/start/AppRow.svelte
View file

@ -1,18 +1,11 @@
<script>
import { Heading, Button, Icon, ActionMenu, MenuItem } from "@budibase/bbui"
import { Heading, Button, Icon } from "@budibase/bbui"
import AppLockModal from "../common/AppLockModal.svelte"
import { processStringSync } from "@budibase/string-templates"
export let app
export let exportApp
export let editApp
export let updateApp
export let deleteApp
export let unpublishApp
export let appOverview
export let releaseLock
export let editIcon
export let copyAppId
</script>
<div class="title" data-cy={`${app.devId}`}>
@ -20,7 +13,7 @@
<div class="app-icon" style="color: {app.icon?.color || ''}">
<Icon size="XL" name={app.icon?.name || "Apps"} />
</div>
<div class="name" data-cy="app-name-link" on:click={() => appOverview(app)}>
<div class="name" data-cy="app-name-link" on:click={() => editApp(app)}>
<Heading size="XS">
{app.name}
</Heading>
@ -37,7 +30,7 @@
{/if}
</div>
<div class="desktop">
<AppLockModal {app} buttonSize="S" />
<AppLockModal {app} buttonSize="M" />
</div>
<div class="desktop">
<div class="app-status">
@ -52,47 +45,27 @@
</div>
<div data-cy={`row_actions_${app.appId}`}>
<div class="app-row-actions">
<Button size="S" secondary newStyles on:click={() => appOverview(app)}>
Manage
</Button>
<Button
size="S"
secondary
quiet
primary
newStyles
disabled={app.lockedOther}
on:click={() => editApp(app)}
>Edit
>
Edit
</Button>
<Button size="S" cta on:click={() => appOverview(app)}>View</Button>
</div>
<ActionMenu align="right" dataCy="app-row-actions-menu-popover">
<span slot="control" class="app-row-actions-icon">
<Icon hoverable name="More" />
</span>
{#if app.lockedYou}
<MenuItem on:click={() => releaseLock(app)} icon="LockOpen">
Release lock
</MenuItem>
{/if}
<MenuItem on:click={() => exportApp(app)} icon="Download">Export</MenuItem>
{#if app.deployed}
<MenuItem on:click={() => unpublishApp(app)} icon="GlobeRemove">
Unpublish
</MenuItem>
<MenuItem on:click={() => copyAppId(app)} icon="Copy">
Copy App ID
</MenuItem>
{/if}
{#if !app.deployed}
<MenuItem on:click={() => updateApp(app)} icon="Edit">Edit</MenuItem>
<MenuItem on:click={() => deleteApp(app)} icon="Delete">Delete</MenuItem>
{/if}
<MenuItem on:click={() => editIcon(app)} icon="Brush">Edit icon</MenuItem>
</ActionMenu>
</div>
<style>
.app-row-actions {
grid-gap: var(--spacing-s);
display: grid;
grid-template-columns: 75px 75px;
display: flex;
flex-direction: row;
justify-content: flex-end;
}
.app-status {
display: grid;

27
packages/builder/src/pages/builder/app/[application]/_layout.svelte
View file

@ -1,7 +1,14 @@
<script>
import { store, automationStore } from "builderStore"
import { roles, flags } from "stores/backend"
import { Icon, ActionGroup, Tabs, Tab, notifications } from "@budibase/bbui"
import {
Icon,
ActionGroup,
Tabs,
Tab,
notifications,
Banner,
} from "@budibase/bbui"
import RevertModal from "components/deploy/RevertModal.svelte"
import VersionModal from "components/deploy/VersionModal.svelte"
import DeployNavigation from "components/deploy/DeployNavigation.svelte"
@ -17,6 +24,7 @@
// Get Package and set store
let promise = getPackage()
let betaAccess = false
// Sync once when you load the app
let hasSynced = false
@ -58,10 +66,18 @@
})
}
async function newDesignUi() {
await flags.toggleUiFeature("design_ui")
window.location.reload()
}
onMount(async () => {
if (!hasSynced && application) {
try {
await API.syncApp(application)
// check if user has beta access
const betaResponse = await API.checkBetaAccess($auth?.user?.email)
betaAccess = betaResponse.access
} catch (error) {
notifications.error("Failed to sync with production database")
}
@ -79,6 +95,15 @@
<div class="loading" />
{:then _}
<div class="root">
{#if betaAccess}
<Banner
extraButtonText="Try New UI (Beta)"
extraButtonAction={newDesignUi}
>
Try the <b>all new</b> budibase design interface. (Not recommended for existing
budibase apps)
</Banner>
{/if}
<div class="top-nav">
<div class="topleftnav">
<button class="home-logo">

137
packages/builder/src/pages/builder/portal/apps/index.svelte
View file

@ -3,20 +3,17 @@
Heading,
Layout,
Button,
Input,
Select,
Modal,
Page,
notifications,
Body,
Search,
Helpers,
} from "@budibase/bbui"
import TemplateDisplay from "components/common/TemplateDisplay.svelte"
import Spinner from "components/common/Spinner.svelte"
import CreateAppModal from "components/start/CreateAppModal.svelte"
import UpdateAppModal from "components/start/UpdateAppModal.svelte"
import ChooseIconModal from "components/start/ChooseIconModal.svelte"
import ExportAppModal from "components/start/ExportAppModal.svelte"
import { store, automationStore } from "builderStore"
@ -25,10 +22,8 @@
import { apps, auth, admin, templates } from "stores/portal"
import download from "downloadjs"
import { goto } from "@roxi/routify"
import ConfirmDialog from "components/common/ConfirmDialog.svelte"
import AppRow from "components/start/AppRow.svelte"
import { AppStatus } from "constants"
import analytics, { Events, EventSource } from "analytics"
import Logo from "assets/bb-space-man.svg"
let sortBy = "name"
@ -36,15 +31,11 @@
let selectedApp
let creationModal
let updatingModal
let deletionModal
let unpublishModal
let exportModal
let iconModal
let creatingApp = false
let loaded = $apps?.length || $templates?.length
let searchTerm = ""
let cloud = $admin.cloud
let appName = ""
let creatingFromTemplate = false
const resolveWelcomeMessage = (auth, apps) => {
@ -164,18 +155,6 @@
creatingApp = false
}
const viewApp = app => {
analytics.captureEvent(Events.APP_VIEW_PUBLISHED, {
appId: app.appId,
eventSource: EventSource.PORTAL,
})
if (app.url) {
window.open(`/app${app.url}`)
} else {
window.open(`/${app.prodId}`)
}
}
const appOverview = app => {
$goto(`../overview/${app.devId}`)
}
@ -190,76 +169,6 @@
$goto(`../../app/${app.devId}`)
}
const editIcon = app => {
selectedApp = app
iconModal.show()
}
const exportApp = app => {
exportModal.show()
selectedApp = app
}
const unpublishApp = app => {
selectedApp = app
unpublishModal.show()
}
const confirmUnpublishApp = async () => {
if (!selectedApp) {
return
}
try {
await API.unpublishApp(selectedApp.prodId)
await apps.load()
notifications.success("App unpublished successfully")
} catch (err) {
notifications.error("Error unpublishing app")
}
}
const deleteApp = app => {
selectedApp = app
deletionModal.show()
}
const confirmDeleteApp = async () => {
if (!selectedApp) {
return
}
try {
await API.deleteApp(selectedApp?.devId)
await apps.load()
// Get checklist, just in case that was the last app
await admin.init()
notifications.success("App deleted successfully")
} catch (err) {
notifications.error("Error deleting app")
}
selectedApp = null
appName = null
}
const updateApp = async app => {
selectedApp = app
updatingModal.show()
}
const releaseLock = async app => {
try {
await API.releaseAppLock(app.devId)
await apps.load()
notifications.success("Lock released successfully")
} catch (err) {
notifications.error("Error releasing lock")
}
}
const copyAppId = async app => {
await Helpers.copyToClipboard(app.prodId)
notifications.success("App ID copied to clipboard.")
}
function createAppFromTemplateUrl(templateKey) {
// validate the template key just to make sure
const templateParts = templateKey.split("/")
@ -407,19 +316,7 @@
<div class="appTable" class:unlocked>
{#each filteredApps as app (app.appId)}
<AppRow
{copyAppId}
{releaseLock}
{editIcon}
{app}
{unpublishApp}
{viewApp}
{editApp}
{exportApp}
{deleteApp}
{updateApp}
{appOverview}
/>
<AppRow {app} {editApp} {appOverview} />
{/each}
</div>
</Layout>
@ -453,35 +350,6 @@
<ExportAppModal app={selectedApp} />
</Modal>
<ConfirmDialog
bind:this={deletionModal}
title="Confirm deletion"
okText="Delete app"
onOk={confirmDeleteApp}
onCancel={() => (appName = null)}
disabled={appName !== selectedApp?.name}
>
Are you sure you want to delete the app <b>{selectedApp?.name}</b>?
<p>Please enter the app name below to confirm.</p>
<Input
bind:value={appName}
data-cy="delete-app-confirmation"
placeholder={selectedApp?.name}
/>
</ConfirmDialog>
<ConfirmDialog
bind:this={unpublishModal}
title="Confirm unpublish"
okText="Unpublish app"
onOk={confirmUnpublishApp}
dataCy={"unpublish-modal"}
>
Are you sure you want to unpublish the app <b>{selectedApp?.name}</b>?
</ConfirmDialog>
<ChooseIconModal app={selectedApp} bind:this={iconModal} />
<style>
.appTable {
border-top: var(--border-light);
@ -538,12 +406,9 @@
height: 70px;
display: grid;
align-items: center;
grid-gap: var(--spacing-xl);
grid-template-columns: auto 1fr;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
padding: 0 var(--spacing-s);
}
.appTable :global(> div) {
border-bottom: var(--border-light);

50
packages/builder/src/pages/builder/portal/manage/users/_components/AddUserModal.svelte
View file

@ -2,22 +2,47 @@
import {
Body,
Input,
Select,
Label,
ModalContent,
notifications,
Select,
Toggle,
Label,
} from "@budibase/bbui"
import { createValidationStore, emailValidator } from "helpers/validation"
import { users } from "stores/portal"
export let disabled
const password = Math.random().toString(36).substring(2, 22)
const options = ["Email onboarding", "Basic onboarding"]
let selected = options[0]
let builder, admin
const [email, error, touched] = createValidationStore("", emailValidator)
let disabled
let builder
let admin
let selected = "Email onboarding"
$: basic = selected === "Basic onboarding"
function addUser() {
if (basic) {
createUser()
} else {
createUserFlow()
}
}
async function createUser() {
try {
await users.create({
email: $email,
password,
builder,
admin,
forceResetPassword: true,
})
notifications.success("Successfully created user")
} catch (error) {
notifications.error("Error creating user")
}
}
async function createUserFlow() {
try {
@ -30,7 +55,7 @@
</script>
<ModalContent
onConfirm={createUserFlow}
onConfirm={addUser}
size="M"
title="Add new user"
confirmText="Add user"
@ -47,17 +72,22 @@
<Select
placeholder={null}
bind:value={selected}
on:change
{options}
label="Add new user via:"
/>
<Input
type="email"
label="Email"
bind:value={$email}
error={$touched && $error}
placeholder="john@doe.com"
label="Email"
/>
{#if basic}
<Input disabled label="Password" value={password} />
{/if}
<div>
<div class="toggle">
<Label size="L">Development access</Label>

74
packages/builder/src/pages/builder/portal/manage/users/_components/BasicOnboardingModal.svelte
View file

@ -1,74 +0,0 @@
<script>
import {
ModalContent,
Body,
Input,
notifications,
Toggle,
Label,
} from "@budibase/bbui"
import { createValidationStore, emailValidator } from "helpers/validation"
import { users } from "stores/portal"
const [email, error, touched] = createValidationStore("", emailValidator)
const password = Math.random().toString(36).slice(2, 20)
let builder = false,
admin = false
async function createUser() {
try {
await users.create({
email: $email,
password,
builder,
admin,
forceResetPassword: true,
})
notifications.success("Successfully created user")
} catch (error) {
notifications.error(error.message)
}
}
</script>
<ModalContent
onConfirm={createUser}
size="M"
title="Basic user onboarding"
confirmText="Continue"
cancelText="Cancel"
disabled={$error}
error={$touched && $error}
showCloseIcon={false}
>
<Body size="S">
Below you will find the user’s username and password. The password will not
be accessible from this point. Please save the credentials.
</Body>
<Input
type="email"
label="Email"
bind:value={$email}
error={$touched && $error}
/>
<Input disabled label="Password" value={password} />
<div>
<div class="toggle">
<Label size="L">Development access</Label>
<Toggle text="" bind:value={builder} />
</div>
<div class="toggle">
<Label size="L">Administration access</Label>
<Toggle text="" bind:value={admin} />
</div>
</div>
</ModalContent>
<style>
.toggle {
display: grid;
grid-template-columns: 78% 1fr;
align-items: center;
width: 50%;
}
</style>

11
packages/builder/src/pages/builder/portal/manage/users/index.svelte
View file

@ -15,7 +15,6 @@
} from "@budibase/bbui"
import TagsRenderer from "./_components/TagsTableRenderer.svelte"
import AddUserModal from "./_components/AddUserModal.svelte"
import BasicOnboardingModal from "./_components/BasicOnboardingModal.svelte"
import { users } from "stores/portal"
import { onMount } from "svelte"
@ -30,7 +29,6 @@
}
let search
let email
$: filteredUsers = $users
.filter(user => user.email.includes(search || ""))
.map(user => ({
@ -41,12 +39,6 @@
}))
let createUserModal
let basicOnboardingModal
function openBasicOnboardingModal() {
createUserModal.hide()
basicOnboardingModal.show()
}
onMount(async () => {
try {
@ -93,9 +85,8 @@
</Layout>
<Modal bind:this={createUserModal}>
<AddUserModal on:change={openBasicOnboardingModal} />
<AddUserModal />
</Modal>
<Modal bind:this={basicOnboardingModal}><BasicOnboardingModal {email} /></Modal>
<style>
.field {

3
packages/builder/src/stores/backend/flags.js
View file

@ -16,6 +16,9 @@ export function createFlagsStore() {
})
await actions.fetch()
},
toggleUiFeature: async feature => {
await API.toggleUiFeature({ value: feature })
},
}
return {

2
packages/cli/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/cli",
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"description": "Budibase CLI, for developers, self hosting and migrations.",
"main": "src/index.js",
"bin": {

8
packages/client/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/client",
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"license": "MPL-2.0",
"module": "dist/budibase-client.js",
"main": "dist/budibase-client.js",
@ -19,9 +19,9 @@
"dev:builder": "rollup -cw"
},
"dependencies": {
"@budibase/bbui": "^1.0.206",
"@budibase/frontend-core": "^1.0.206",
"@budibase/string-templates": "^1.0.206",
"@budibase/bbui": "^1.0.212-alpha.6",
"@budibase/frontend-core": "^1.0.212-alpha.6",
"@budibase/string-templates": "^1.0.212-alpha.6",
"@spectrum-css/button": "^3.0.3",
"@spectrum-css/card": "^3.0.3",
"@spectrum-css/divider": "^1.0.3",

1
packages/client/src/utils/buttonActions.js
View file

@ -270,6 +270,7 @@ const exportDataHandler = async action => {
tableId: selection.tableId,
rows: selection.selectedRows,
format: action.parameters.type,
columns: action.parameters.columns,
})
download(data, `${selection.tableId}.${action.parameters.type}`)
} catch (error) {

1293
packages/client/yarn.lock
View file

File diff suppressed because it is too large Load diff

4
packages/frontend-core/package.json
View file

@ -1,12 +1,12 @@
{
"name": "@budibase/frontend-core",
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"description": "Budibase frontend core libraries used in builder and client",
"author": "Budibase",
"license": "MPL-2.0",
"svelte": "src/index.js",
"dependencies": {
"@budibase/bbui": "^1.0.206",
"@budibase/bbui": "^1.0.212-alpha.6",
"lodash": "^4.17.21",
"svelte": "^3.46.2"
}

9
packages/frontend-core/src/api/flags.js
View file

@ -22,4 +22,13 @@ export const buildFlagEndpoints = API => ({
},
})
},
/**
* Allows us to experimentally toggle a beta UI feature through a cookie.
* @param value the feature to toggle
*/
toggleUiFeature: async ({ value }) => {
return await API.post({
url: `/api/beta/${value}`,
})
},
})

10
packages/frontend-core/src/api/other.js
View file

@ -54,4 +54,14 @@ export const buildOtherEndpoints = API => ({
url: "/api/permission/builtin",
})
},
/**
* Check if they are part of the budibase beta program.
*/
checkBetaAccess: async email => {
return await API.get({
url: `/api/beta/access?email=${email}`,
external: true,
})
},
})

5
packages/frontend-core/src/api/rows.js
View file

@ -65,12 +65,15 @@ export const buildRowEndpoints = API => ({
* Exports rows.
* @param tableId the table ID to export the rows from
* @param rows the array of rows to export
* @param format the format to export (csv or json)
* @param columns which columns to export (all if undefined)
*/
exportRows: async ({ tableId, rows, format }) => {
exportRows: async ({ tableId, rows, format, columns }) => {
return await API.post({
url: `/api/${tableId}/rows/exportRows?format=${format}`,
body: {
rows,
columns,
},
parseResponse: async response => {
return await response.text()

26
packages/frontend-core/src/constants.js
View file

@ -42,6 +42,10 @@ export const OperatorOptions = {
value: "notEqual",
label: "Does Not Contain",
},
In: {
value: "oneOf",
label: "Is in",
},
}
// Cookie names
@ -63,3 +67,25 @@ export const TableNames = {
* - Coerce types for search endpoint
*/
export const ApiVersion = "1"
/**
* Maximum minimum range for SQL number values
*/
export const SqlNumberTypeRangeMap = {
integer: {
max: 2147483647,
min: -2147483648,
},
int: {
max: 2147483647,
min: -2147483648,
},
smallint: {
max: 32767,
min: -32768,
},
mediumint: {
max: 8388607,
min: -8388608,
},
}

44
packages/frontend-core/src/utils/lucene.js
View file

@ -1,5 +1,5 @@
import { Helpers } from "@budibase/bbui"
import { OperatorOptions } from "../constants"
import { OperatorOptions, SqlNumberTypeRangeMap } from "../constants"
/**
* Returns the valid operator options for a certain data type
@ -14,6 +14,7 @@ export const getValidOperatorsForType = type => {
Op.Like,
Op.Empty,
Op.NotEmpty,
Op.In,
]
const numOps = [
Op.Equals,
@ -22,6 +23,7 @@ export const getValidOperatorsForType = type => {
Op.LessThan,
Op.Empty,
Op.NotEmpty,
Op.In,
]
if (type === "string") {
return stringOps
@ -91,31 +93,34 @@ export const buildLuceneQuery = filter => {
notEmpty: {},
contains: {},
notContains: {},
oneOf: {},
}
if (Array.isArray(filter)) {
filter.forEach(expression => {
let { operator, field, type, value } = expression
let { operator, field, type, value, externalType } = expression
// Parse all values into correct types
if (type === "datetime" && value) {
value = new Date(value).toISOString()
}
if (type === "number") {
value = parseFloat(value)
if (type === "number" && !Array.isArray(value)) {
if (operator === "oneOf") {
value = value.split(",").map(item => parseFloat(item))
} else {
value = parseFloat(value)
}
}
if (type === "boolean") {
value = `${value}`?.toLowerCase() === "true"
}
if (operator.startsWith("range")) {
const minint =
SqlNumberTypeRangeMap[externalType]?.min || Number.MIN_SAFE_INTEGER
const maxint =
SqlNumberTypeRangeMap[externalType]?.max || Number.MAX_SAFE_INTEGER
if (!query.range[field]) {
query.range[field] = {
low:
type === "number"
? Number.MIN_SAFE_INTEGER
: "0000-00-00T00:00:00.000Z",
high:
type === "number"
? Number.MAX_SAFE_INTEGER
: "9999-00-00T00:00:00.000Z",
low: type === "number" ? minint : "0000-00-00T00:00:00.000Z",
high: type === "number" ? maxint : "9999-00-00T00:00:00.000Z",
}
}
if (operator === "rangeLow" && value != null && value !== "") {
@ -141,7 +146,6 @@ export const buildLuceneQuery = filter => {
}
})
}
return query
}
@ -213,6 +217,17 @@ export const runLuceneQuery = (docs, query) => {
return docValue == null || docValue === ""
})
// Process an includes match (fails if the value is not included)
const oneOf = match("oneOf", (docValue, testValue) => {
if (typeof testValue === "string") {
testValue = testValue.split(",")
if (typeof docValue === "number") {
testValue = testValue.map(item => parseFloat(item))
}
}
return !testValue?.includes(docValue)
})
// Match a document against all criteria
const docMatch = doc => {
return (
@ -222,7 +237,8 @@ export const runLuceneQuery = (docs, query) => {
equalMatch(doc) &&
notEqualMatch(doc) &&
emptyMatch(doc) &&
notEmptyMatch(doc)
notEmptyMatch(doc) &&
oneOf(doc)
)
}

1
packages/server/.gitignore vendored
View file

@ -2,6 +2,7 @@ node_modules/
myapps/
.env
builder/*
new_design_ui/*
client/*
db/dev.db/
dist

12
packages/server/package.json
View file

@ -1,7 +1,7 @@
{
"name": "@budibase/server",
"email": "hi@budibase.com",
"version": "1.0.206",
"version": "1.0.212-alpha.6",
"description": "Budibase Web Server",
"main": "src/index.ts",
"repository": {
@ -77,10 +77,11 @@
"license": "GPL-3.0",
"dependencies": {
"@apidevtools/swagger-parser": "10.0.3",
"@budibase/backend-core": "^1.0.206",
"@budibase/client": "^1.0.206",
"@budibase/pro": "1.0.206",
"@budibase/string-templates": "^1.0.206",
"@budibase/backend-core": "^1.0.212-alpha.6",
"@budibase/client": "^1.0.212-alpha.6",
"@budibase/pro": "1.0.212-alpha.6",
"@budibase/string-templates": "^1.0.212-alpha.6",
"@budibase/types": "^1.0.212-alpha.6",
"@bull-board/api": "3.7.0",
"@bull-board/koa": "3.9.4",
"@elastic/elasticsearch": "7.10.0",
@ -151,7 +152,6 @@
"@babel/core": "7.17.4",
"@babel/preset-env": "7.16.11",
"@budibase/standard-components": "^0.9.139",
"@budibase/types": "^1.0.206",
"@jest/test-sequencer": "24.9.0",
"@types/apidoc": "0.50.0",
"@types/bson": "4.2.0",

BIN
packages/server/scripts/integrations/oracle/instantclient/linux/arm64/basiclite-19.10.zip Normal file
View file

Binary file not shown.

23
packages/server/scripts/integrations/oracle/instantclient/linux/arm64/install.sh Executable file
View file

@ -0,0 +1,23 @@
#!/bin/bash
# Must be root to continue
if [[ $(id -u) -ne 0 ]] ; then echo "Please run as root" ; exit 1 ; fi
# Allow for re-runs
rm -rf /opt/oracle
echo "Installing oracle instant client"
# copy and unzip package
mkdir -p /opt/oracle
cp scripts/integrations/oracle/instantclient/linux/arm64/basiclite-19.10.zip /opt/oracle
cd /opt/oracle
unzip -qq basiclite-19.10.zip -d .
rm *.zip
mv instantclient* instantclient
# update runtime link path
sh -c "echo /opt/oracle/instantclient > /etc/ld.so.conf.d/oracle-instantclient.conf"
ldconfig /etc/ld.so.conf.d
echo "Installation complete"

10
packages/server/scripts/integrations/oracle/instantclient/linux/install.sh Executable file
View file

@ -0,0 +1,10 @@
#!/bin/bash
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]:-$0}"; )" &> /dev/null && pwd 2> /dev/null; )"
if [[ $TARGETARCH == arm* ]] ;
then
echo "Installing ARM Oracle instant client..."
$SCRIPT_DIR/arm64/install.sh
else
echo "Installing x86-64 Oracle instant client..."
$SCRIPT_DIR/x86-64/install.sh
fi

6
packages/server/src/api/controllers/application.ts
View file

@ -227,7 +227,11 @@ export const fetchAppPackage = async (ctx: any) => {
application,
screens,
layouts,
clientLibPath: clientLibraryPath(ctx.params.appId, application.version),
clientLibPath: clientLibraryPath(
ctx.params.appId,
application.version,
ctx
),
}
}

55
packages/server/src/api/controllers/dev.js
View file

@ -3,8 +3,12 @@ const env = require("../../environment")
const { checkSlashesInUrl } = require("../../utilities")
const { request } = require("../../utilities/workerRequests")
const { clearLock } = require("../../utilities/redis")
const { Replication, getProdAppID } = require("@budibase/backend-core/db")
const { DocumentTypes } = require("../../db/utils")
const {
Replication,
getProdAppID,
dangerousGetDB,
} = require("@budibase/backend-core/db")
const { DocumentTypes, getRowParams } = require("../../db/utils")
const { app: appCache } = require("@budibase/backend-core/cache")
const { getProdAppDB, getAppDB } = require("@budibase/backend-core/context")
const { events } = require("@budibase/backend-core")
@ -133,3 +137,50 @@ exports.getBudibaseVersion = async ctx => {
}
await events.installation.versionChecked(version)
}
// TODO: remove as part of beta program
exports.checkBetaAccess = async ctx => {
// go to the cloud platform if running self hosted
if (env.SELF_HOSTED || !env.MULTI_TENANCY) {
let baseUrl = ""
if (env.ACCOUNT_PORTAL_URL) {
baseUrl = env.ACCOUNT_PORTAL_URL.replace("account.", "")
} else {
baseUrl = "https://budibase.app"
}
const response = await fetch(
`${baseUrl}/api/beta/access?email=${ctx.query.email}`
)
const json = await response.json()
ctx.body = json
return
}
const userToCheck = ctx.query.email
const BETA_USERS_DB = "app_bb_f9b77d06b9db4e3ca185476ab87a2364"
const BETA_USERS_TABLE = "ta_8c2c6df1c03f49cfb6340e85e066dd15"
try {
const db = dangerousGetDB(BETA_USERS_DB)
const betaUsers = (
await db.allDocs(
getRowParams(BETA_USERS_TABLE, null, {
include_docs: true,
})
)
).rows.map(row => row.doc)
let access = false
for (let betaUser of betaUsers) {
if (betaUser["Email address"].trim() === userToCheck) {
access = true
break
}
}
ctx.body = { access }
} catch (err) {
console.error(err)
ctx.body = { access: false }
}
}

21
packages/server/src/api/controllers/row/external.js
View file

@ -157,7 +157,8 @@ exports.validate = async () => {
exports.exportRows = async ctx => {
const { datasourceId } = breakExternalTableId(ctx.params.tableId)
const db = getAppDB()
let format = ctx.query.format
const format = ctx.query.format
const { columns } = ctx.request.body
const datasource = await db.get(datasourceId)
if (!datasource || !datasource.entities) {
ctx.throw(400, "Datasource has not been configured for plus API.")
@ -171,13 +172,27 @@ exports.exportRows = async ctx => {
}
let result = await exports.search(ctx)
let headers = Object.keys(result.rows[0])
let rows = []
// Filter data to only specified columns if required
if (columns && columns.length) {
for (let i = 0; i < result.rows.length; i++) {
rows[i] = {}
for (let column of columns) {
rows[i][column] = result.rows[i][column]
}
}
} else {
rows = result.rows
}
let headers = Object.keys(rows[0])
const exporter = exporters[format]
const filename = `export.${format}`
// send down the file
ctx.attachment(filename)
return apiFileReturn(exporter(headers, result.rows))
return apiFileReturn(exporter(headers, rows))
}
exports.fetchEnrichedRow = async ctx => {

30
packages/server/src/api/controllers/row/internalSearch.js
View file

@ -17,6 +17,7 @@ class QueryBuilder {
notEqual: {},
empty: {},
notEmpty: {},
oneOf: {},
...base,
}
this.limit = 50
@ -112,6 +113,11 @@ class QueryBuilder {
return this
}
addOneOf(key, value) {
this.query.oneOf[key] = value
return this
}
/**
* Preprocesses a value before going into a lucene search.
* Transforms strings to lowercase and wraps strings and bools in quotes.
@ -145,7 +151,7 @@ class QueryBuilder {
function build(structure, queryFn) {
for (let [key, value] of Object.entries(structure)) {
key = builder.preprocess(key.replace(/ /, "_"), {
key = builder.preprocess(key.replace(/ /g, "_"), {
escape: true,
})
const expression = queryFn(key, value)
@ -220,6 +226,28 @@ class QueryBuilder {
if (this.query.notEmpty) {
build(this.query.notEmpty, key => `${key}:["" TO *]`)
}
if (this.query.oneOf) {
build(this.query.oneOf, (key, value) => {
if (!Array.isArray(value)) {
if (typeof value === "string") {
value = value.split(",")
} else {
return ""
}
}
let orStatement = `${builder.preprocess(
value[0],
allPreProcessingOpts
)}`
for (let i = 1; i < value.length; i++) {
orStatement += ` OR ${builder.preprocess(
value[i],
allPreProcessingOpts
)}`
}
return `${key}:(${orStatement})`
})
}
return query
}

43
packages/server/src/api/controllers/static/index.js
View file

@ -16,9 +16,15 @@ const { upload } = require("../../../utilities/fileSystem")
const { attachmentsRelativeURL } = require("../../../utilities")
const { DocumentTypes, isDevAppID } = require("../../../db/utils")
const { getAppDB, getAppId } = require("@budibase/backend-core/context")
const { setCookie, clearCookie } = require("@budibase/backend-core/utils")
const AWS = require("aws-sdk")
const { events } = require("@budibase/backend-core")
const fs = require("fs")
const {
downloadTarballDirect,
} = require("../../../utilities/fileSystem/utilities")
async function prepareUpload({ s3Key, bucket, metadata, file }) {
const response = await upload({
bucket,
@ -38,8 +44,41 @@ async function prepareUpload({ s3Key, bucket, metadata, file }) {
}
}
exports.toggleBetaUiFeature = async function (ctx) {
const cookieName = `beta:${ctx.params.feature}`
if (ctx.cookies.get(cookieName)) {
clearCookie(ctx, cookieName)
ctx.body = {
message: `${ctx.params.feature} disabled`,
}
return
}
let builderPath = resolve(TOP_LEVEL_PATH, "new_design_ui")
// // download it from S3
if (!fs.existsSync(builderPath)) {
fs.mkdirSync(builderPath)
}
await downloadTarballDirect(
"https://cdn.budi.live/beta:design_ui/new_ui.tar.gz",
builderPath
)
setCookie(ctx, {}, cookieName)
ctx.body = {
message: `${ctx.params.feature} enabled`,
}
}
exports.serveBuilder = async function (ctx) {
let builderPath = resolve(TOP_LEVEL_PATH, "builder")
// Temporary: New Design UI
const designUiCookie = ctx.cookies.get("beta:design_ui")
// TODO: get this from the tmp Dir that we downloaded from MinIO
const uiPath = designUiCookie ? "new_design_ui" : "builder"
let builderPath = resolve(TOP_LEVEL_PATH, uiPath)
await send(ctx, ctx.file, { root: builderPath })
if (!ctx.file.includes("assets/")) {
await events.serve.servedBuilder()
@ -78,7 +117,7 @@ exports.serveApp = async function (ctx) {
title: appInfo.name,
production: env.isProd(),
appId,
clientLibPath: clientLibraryPath(appId, appInfo.version),
clientLibPath: clientLibraryPath(appId, appInfo.version, ctx),
})
const appHbs = loadHandlebarsFile(`${__dirname}/templates/app.hbs`)

1
packages/server/src/api/routes/dev.js
View file

@ -22,5 +22,6 @@ router
.get("/api/dev/version", authorized(BUILDER), controller.getBudibaseVersion)
.delete("/api/dev/:appId/lock", authorized(BUILDER), controller.clearLock)
.post("/api/dev/:appId/revert", authorized(BUILDER), controller.revert)
.get("/api/beta/access", controller.checkBetaAccess)
module.exports = router

1
packages/server/src/api/routes/static.ts
View file

@ -38,6 +38,7 @@ router
// TODO: for now this builder endpoint is not authorized/secured, will need to be
.get("/builder/:file*", controller.serveBuilder)
.post("/api/attachments/process", authorized(BUILDER), controller.uploadFile)
.post("/api/beta/:feature", controller.toggleBetaUiFeature)
.post(
"/api/attachments/:tableId/upload",
paramResource("tableId"),

Some files were not shown because too many files have changed in this diff Show more