CodeMirror/budibase
1
0
Fork 0
mirror of synced 2024-09-08 13:41:09 +12:00
Code Issues Projects Releases Wiki Activity

Merge branch 'develop' into api-create-app

Browse source
This commit is contained in:
Pedro Silva 2022-10-07 11:06:31 +01:00
commit 955897a313
38 changed files with 383 additions and 194 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
docs/DEV-SETUP-DEBIAN.md
View file

@ -1,12 +1,15 @@
## Dev Environment on Debian 11
### Install Node
### Install NVM & Node 14
NVM documentation: https://github.com/nvm-sh/nvm#installing-and-updating
Budibase requires a recent version of node (14+):
Install NVM
```
curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
apt -y install nodejs
node -v
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
```
Install Node 14
```
nvm install 14
```
### Install npm requirements
@ -31,7 +34,7 @@ This setup process was tested on Debian 11 (bullseye) with version numbers show
- Docker: 20.10.5
- Docker-Compose: 1.29.2
- Node: v16.15.1
- Node: v14.20.1
- Yarn: 1.22.19
- Lerna: 5.1.4

9
docs/DEV-SETUP-MACOSX.md
View file

@ -11,7 +11,7 @@ through brew.
### Install Node
Budibase requires a recent version of node (14+):
Budibase requires a recent version of node 14:
```
brew install node npm
node -v
@ -38,7 +38,7 @@ This setup process was tested on Mac OSX 12 (Monterey) with version numbers show
- Docker: 20.10.14
- Docker-Compose: 2.6.0
- Node: 18.3.0
- Node: 14.20.1
- Yarn: 1.22.19
- Lerna: 5.1.4
@ -59,4 +59,7 @@ The dev version will be available on port 10000 i.e.
http://127.0.0.1:10000/builder/admin
| **NOTE**: If you are working on a M1 Apple Silicon, you will need to uncomment `# platform: linux/amd64` line in
[hosting/docker-compose-dev.yaml](../hosting/docker-compose.dev.yaml)
[hosting/docker-compose-dev.yaml](../hosting/docker-compose.dev.yaml)
### Troubleshooting
If there are errors with the `yarn setup` command, you can try installing nvm and node 14. This is the same as the instructions for Debian 11.

81
docs/DEV-SETUP-WINDOWS.md Normal file
View file

@ -0,0 +1,81 @@
## Dev Environment on Windows 10/11 (WSL2)
### Install WSL with Ubuntu LTS
Enable WSL 2 on Windows 10/11 for docker support.
```
wsl --set-default-version 2
```
Install Ubuntu LTS.
```
wsl --install Ubuntu
```
Or follow the instruction here:
https://learn.microsoft.com/en-us/windows/wsl/install
### Install Docker in windows
Download the installer from docker and install it.
Check this url for more detailed instructions:
https://docs.docker.com/desktop/install/windows-install/
You should follow the next steps from within the Ubuntu terminal.
### Install NVM & Node 14
NVM documentation: https://github.com/nvm-sh/nvm#installing-and-updating
Install NVM
```
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
```
Install Node 14
```
nvm install 14
```
### Install npm requirements
```
npm install -g yarn jest lerna
```
### Clone the repo
```
git clone https://github.com/Budibase/budibase.git
```
### Check Versions
This setup process was tested on Windows 11 with version numbers show below. Your mileage may vary using anything else.
- Docker: 20.10.7
- Docker-Compose: 2.10.2
- Node: v14.20.1
- Yarn: 1.22.19
- Lerna: 5.5.4
### Build
```
cd budibase
yarn setup
```
The yarn setup command runs several build steps i.e.
```
node ./hosting/scripts/setup.js && yarn && yarn bootstrap && yarn build && yarn dev
```
So this command will actually run the application in dev mode. It creates .env files under `./packages/server` and `./packages/worker` and runs docker containers for each service via docker-compose.
The dev version will be available on port 10000 i.e.
http://127.0.0.1:10000/builder/admin
### Working with the code
Here are the instructions to work on the application from within Visual Studio Code (in Windows) through the WSL. All the commands and files are within the Ubuntu system and it should run as if you were working on a Linux machine.
https://code.visualstudio.com/docs/remote/wsl
Note you will be able to run the application from within the WSL terminal and you will be able to access the application from the a browser in Windows.

4
hosting/single/Dockerfile
View file

@ -19,8 +19,8 @@ ADD packages/worker .
RUN node /pinVersions.js && yarn && yarn build && /cleanup.sh
FROM couchdb:3.2.1
# TARGETARCH can be amd64 or arm e.g. docker build --build-arg TARGETARCH=amd64
ARG TARGETARCH=amd64
ARG TARGETARCH
ENV TARGETARCH $TARGETARCH
#TARGETBUILD can be set to single (for single docker image) or aas (for azure app service)
# e.g. docker build --build-arg TARGETBUILD=aas ....
ARG TARGETBUILD=single

2
lerna.json
View file

@ -1,5 +1,5 @@
{
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"npmClient": "yarn",
"packages": [
"packages/*"

4
packages/backend-core/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/backend-core",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Budibase backend core libraries used in server and worker",
"main": "dist/src/index.js",
"types": "dist/src/index.d.ts",
@ -20,7 +20,7 @@
"test:watch": "jest --watchAll"
},
"dependencies": {
"@budibase/types": "2.0.14-alpha.4",
"@budibase/types": "2.0.24-alpha.0",
"@shopify/jest-koa-mocks": "5.0.1",
"@techpass/passport-openidconnect": "0.3.2",
"aws-sdk": "2.1030.0",

4
packages/bbui/package.json
View file

@ -1,7 +1,7 @@
{
"name": "@budibase/bbui",
"description": "A UI solution used in the different Budibase projects.",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"license": "MPL-2.0",
"svelte": "src/index.js",
"module": "dist/bbui.es.js",
@ -38,7 +38,7 @@
],
"dependencies": {
"@adobe/spectrum-css-workflow-icons": "^1.2.1",
"@budibase/string-templates": "2.0.14-alpha.4",
"@budibase/string-templates": "2.0.24-alpha.0",
"@spectrum-css/actionbutton": "^1.0.1",
"@spectrum-css/actiongroup": "^1.0.1",
"@spectrum-css/avatar": "^3.0.2",

10
packages/builder/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/builder",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"license": "GPL-3.0",
"private": true,
"scripts": {
@ -71,10 +71,10 @@
}
},
"dependencies": {
"@budibase/bbui": "2.0.14-alpha.4",
"@budibase/client": "2.0.14-alpha.4",
"@budibase/frontend-core": "2.0.14-alpha.4",
"@budibase/string-templates": "2.0.14-alpha.4",
"@budibase/bbui": "2.0.24-alpha.0",
"@budibase/client": "2.0.24-alpha.0",
"@budibase/frontend-core": "2.0.24-alpha.0",
"@budibase/string-templates": "2.0.24-alpha.0",
"@sentry/browser": "5.19.1",
"@spectrum-css/page": "^3.0.1",
"@spectrum-css/vars": "^3.0.1",

24
packages/builder/src/builderStore/dataBinding.js
View file

@ -396,19 +396,17 @@ export const getUserBindings = () => {
bindings = keys.reduce((acc, key) => {
const fieldSchema = schema[key]
if (fieldSchema.type !== "link") {
acc.push({
type: "context",
runtimeBinding: `${safeUser}.${makePropSafe(key)}`,
readableBinding: `Current User.${key}`,
// Field schema and provider are required to construct relationship
// datasource options, based on bindable properties
fieldSchema,
providerId: "user",
category: "Current User",
icon: "User",
})
}
acc.push({
type: "context",
runtimeBinding: `${safeUser}.${makePropSafe(key)}`,
readableBinding: `Current User.${key}`,
// Field schema and provider are required to construct relationship
// datasource options, based on bindable properties
fieldSchema,
providerId: "user",
category: "Current User",
icon: "User",
})
return acc
}, [])

8
packages/cli/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/cli",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Budibase CLI, for developers, self hosting and migrations.",
"main": "src/index.js",
"bin": {
@ -26,9 +26,9 @@
"outputPath": "build"
},
"dependencies": {
"@budibase/backend-core": "2.0.14-alpha.4",
"@budibase/string-templates": "2.0.14-alpha.4",
"@budibase/types": "2.0.14-alpha.4",
"@budibase/backend-core": "2.0.24-alpha.0",
"@budibase/string-templates": "2.0.24-alpha.0",
"@budibase/types": "2.0.24-alpha.0",
"axios": "0.21.2",
"chalk": "4.1.0",
"cli-progress": "3.11.2",

8
packages/client/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/client",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"license": "MPL-2.0",
"module": "dist/budibase-client.js",
"main": "dist/budibase-client.js",
@ -19,9 +19,9 @@
"dev:builder": "rollup -cw"
},
"dependencies": {
"@budibase/bbui": "2.0.14-alpha.4",
"@budibase/frontend-core": "2.0.14-alpha.4",
"@budibase/string-templates": "2.0.14-alpha.4",
"@budibase/bbui": "2.0.24-alpha.0",
"@budibase/frontend-core": "2.0.24-alpha.0",
"@budibase/string-templates": "2.0.24-alpha.0",
"@spectrum-css/button": "^3.0.3",
"@spectrum-css/card": "^3.0.3",
"@spectrum-css/divider": "^1.0.3",

6
packages/client/src/components/ClientApp.svelte
View file

@ -16,7 +16,6 @@
themeStore,
appStore,
devToolsStore,
environmentStore,
} from "stores"
import NotificationDisplay from "components/overlay/NotificationDisplay.svelte"
import ConfirmationDisplay from "components/overlay/ConfirmationDisplay.svelte"
@ -48,8 +47,6 @@
!$builderStore.inBuilder &&
$devToolsStore.enabled &&
!$routeStore.queryParams?.peek
$: objectStoreUrl = $environmentStore.cloud ? "https://cdn.budi.live" : ""
$: pluginsUrl = `${objectStoreUrl}/plugins`
// Handle no matching route
$: {
@ -95,8 +92,7 @@
<svelte:head>
{#if $builderStore.usedPlugins?.length}
{#each $builderStore.usedPlugins as plugin (plugin.hash)}
<script
src={`${pluginsUrl}/${plugin.jsUrl}?r=${plugin.hash || ""}`}></script>
<script src={`${plugin.jsUrl}?r=${plugin.hash || ""}`}></script>
{/each}
{/if}
</svelte:head>

58
packages/client/src/components/app/charts/ApexOptionsBuilder.js
View file

@ -1,37 +1,39 @@
export class ApexOptionsBuilder {
formatters = {
["Default"]: val => (isNaN(val) ? val : Math.round(val * 100) / 100),
["Thousands"]: val => `${Math.round(val / 1000)}K`,
["Millions"]: val => `${Math.round(val / 1000000)}M`,
}
options = {
series: [],
legend: {
show: false,
position: "top",
horizontalAlign: "right",
showForSingleSeries: true,
showForNullSeries: true,
showForZeroSeries: true,
},
chart: {
toolbar: {
constructor() {
this.formatters = {
["Default"]: val => (isNaN(val) ? val : Math.round(val * 100) / 100),
["Thousands"]: val => `${Math.round(val / 1000)}K`,
["Millions"]: val => `${Math.round(val / 1000000)}M`,
}
this.options = {
series: [],
legend: {
show: false,
position: "top",
horizontalAlign: "right",
showForSingleSeries: true,
showForNullSeries: true,
showForZeroSeries: true,
},
zoom: {
enabled: false,
chart: {
toolbar: {
show: false,
},
zoom: {
enabled: false,
},
},
},
xaxis: {
labels: {
formatter: this.formatters.Default,
xaxis: {
labels: {
formatter: this.formatters.Default,
},
},
},
yaxis: {
labels: {
formatter: this.formatters.Default,
yaxis: {
labels: {
formatter: this.formatters.Default,
},
},
},
}
}
setOption(path, value) {

4
packages/frontend-core/package.json
View file

@ -1,12 +1,12 @@
{
"name": "@budibase/frontend-core",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Budibase frontend core libraries used in builder and client",
"author": "Budibase",
"license": "MPL-2.0",
"svelte": "src/index.js",
"dependencies": {
"@budibase/bbui": "2.0.14-alpha.4",
"@budibase/bbui": "2.0.24-alpha.0",
"lodash": "^4.17.21",
"svelte": "^3.46.2"
}

82
packages/frontend-core/src/fetch/DataFetch.js
View file

@ -14,52 +14,52 @@ import { convertJSONSchemaToTableSchema } from "../utils/json"
* For other types of datasource, this class is overridden and extended.
*/
export default class DataFetch {
// API client
API = null
// Feature flags
featureStore = writable({
supportsSearch: false,
supportsSort: false,
supportsPagination: false,
})
// Config
options = {
datasource: null,
limit: 10,
// Search config
filter: null,
query: null,
// Sorting config
sortColumn: null,
sortOrder: "ascending",
sortType: null,
// Pagination config
paginate: true,
}
// State of the fetch
store = writable({
rows: [],
info: null,
schema: null,
loading: false,
loaded: false,
query: null,
pageNumber: 0,
cursor: null,
cursors: [],
})
/**
* Constructs a new DataFetch instance.
* @param opts the fetch options
*/
constructor(opts) {
// API client
this.API = null
// Feature flags
this.featureStore = writable({
supportsSearch: false,
supportsSort: false,
supportsPagination: false,
})
// Config
this.options = {
datasource: null,
limit: 10,
// Search config
filter: null,
query: null,
// Sorting config
sortColumn: null,
sortOrder: "ascending",
sortType: null,
// Pagination config
paginate: true,
}
// State of the fetch
this.store = writable({
rows: [],
info: null,
schema: null,
loading: false,
loaded: false,
query: null,
pageNumber: 0,
cursor: null,
cursors: [],
})
// Merge options with their default values
this.API = opts?.API
this.options = {

7
packages/frontend-core/src/utils/lucene.js
View file

@ -121,7 +121,12 @@ export const buildLuceneQuery = filter => {
query.allOr = true
return
}
if (type === "datetime" && !isHbs) {
if (
type === "datetime" &&
!isHbs &&
operator !== "empty" &&
operator !== "notEmpty"
) {
// Ensure date value is a valid date and parse into correct format
if (!value) {
return

2
packages/sdk/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/sdk",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Budibase Public API SDK",
"author": "Budibase",
"license": "MPL-2.0",

12
packages/server/package.json
View file

@ -1,7 +1,7 @@
{
"name": "@budibase/server",
"email": "hi@budibase.com",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Budibase Web Server",
"main": "src/index.ts",
"repository": {
@ -77,11 +77,11 @@
"license": "GPL-3.0",
"dependencies": {
"@apidevtools/swagger-parser": "10.0.3",
"@budibase/backend-core": "2.0.14-alpha.4",
"@budibase/client": "2.0.14-alpha.4",
"@budibase/pro": "2.0.14-alpha.4",
"@budibase/string-templates": "2.0.14-alpha.4",
"@budibase/types": "2.0.14-alpha.4",
"@budibase/backend-core": "2.0.24-alpha.0",
"@budibase/client": "2.0.24-alpha.0",
"@budibase/pro": "2.0.24-alpha.0",
"@budibase/string-templates": "2.0.24-alpha.0",
"@budibase/types": "2.0.24-alpha.0",
"@bull-board/api": "3.7.0",
"@bull-board/koa": "3.9.4",
"@elastic/elasticsearch": "7.10.0",

6
packages/server/src/api/controllers/application.ts
View file

@ -50,6 +50,7 @@ import { errors, events, migrations } from "@budibase/backend-core"
import { App, Layout, Screen, MigrationType } from "@budibase/types"
import { BASE_LAYOUT_PROP_IDS } from "../../constants/layouts"
import { groups } from "@budibase/pro"
import { enrichPluginURLs } from "../../utilities/plugins"
const URL_REGEX_SLASH = /\/|\\/g
@ -208,10 +209,13 @@ export const fetchAppDefinition = async (ctx: any) => {
export const fetchAppPackage = async (ctx: any) => {
const db = context.getAppDB()
const application = await db.get(DocumentType.APP_METADATA)
let application = await db.get(DocumentType.APP_METADATA)
const layouts = await getLayouts()
let screens = await getScreens()
// Enrich plugin URLs
application.usedPlugins = enrichPluginURLs(application.usedPlugins)
// Only filter screens if the user is not a builder
if (!(ctx.user.builder && ctx.user.builder.global)) {
const userRoleId = getUserRoleId(ctx)

33
packages/server/src/api/controllers/datasource.js
View file

@ -68,6 +68,7 @@ exports.buildSchemaFromDb = async function (ctx) {
datasource.entities = tables
}
setDefaultDisplayColumns(datasource)
const dbResp = await db.put(datasource)
datasource._rev = dbResp.rev
@ -78,6 +79,24 @@ exports.buildSchemaFromDb = async function (ctx) {
ctx.body = response
}
/**
* Make sure all datasource entities have a display name selected
*/
const setDefaultDisplayColumns = datasource => {
//
for (let entity of Object.values(datasource.entities)) {
if (entity.primaryDisplay) {
continue
}
const notAutoColumn = Object.values(entity.schema).find(
schema => !schema.autocolumn
)
if (notAutoColumn) {
entity.primaryDisplay = notAutoColumn.name
}
}
}
/**
* Check for variables that have been updated or removed and invalidate them.
*/
@ -155,6 +174,7 @@ exports.save = async function (ctx) {
const { tables, error } = await buildSchemaHelper(datasource)
schemaError = error
datasource.entities = tables
setDefaultDisplayColumns(datasource)
}
const dbResp = await db.put(datasource)
@ -238,19 +258,6 @@ const buildSchemaHelper = async datasource => {
const connector = new Connector(datasource.config)
await connector.buildSchema(datasource._id, datasource.entities)
// make sure they all have a display name selected
for (let entity of Object.values(datasource.entities ?? {})) {
if (entity.primaryDisplay) {
continue
}
const notAutoColumn = Object.values(entity.schema).find(
schema => !schema.autocolumn
)
if (notAutoColumn) {
entity.primaryDisplay = notAutoColumn.name
}
}
const errors = connector.schemaErrors
let error = null
if (errors && Object.keys(errors).length > 0) {

9
packages/server/src/api/controllers/public/rows.ts
View file

@ -52,14 +52,19 @@ export async function read(ctx: any, next: any) {
}
export async function update(ctx: any, next: any) {
ctx.request.body = await addRev(fixRow(ctx.request.body, ctx.params))
const { tableId } = ctx.params
ctx.request.body = await addRev(fixRow(ctx.request.body, ctx.params), tableId)
await rowController.save(ctx)
await next()
}
export async function destroy(ctx: any, next: any) {
const { tableId } = ctx.params
// set the body as expected, with the _id and _rev fields
ctx.request.body = await addRev(fixRow({ _id: ctx.params.rowId }, ctx.params))
ctx.request.body = await addRev(
fixRow({ _id: ctx.params.rowId }, ctx.params),
tableId
)
await rowController.destroy(ctx)
// destroy controller doesn't currently return the row as the body, need to adjust this
// in the public API to be correct

2
packages/server/src/api/controllers/public/utils.ts
View file

@ -22,7 +22,7 @@ export async function addRev(
}
/**
* Performs a case insensitive search on the provided documents, using the
* Performs a case in-sensitive search on the provided documents, using the
* provided key and value. This will be a string based search, using the
* startsWith function.
*/

4
packages/server/src/api/controllers/query/index.ts
View file

@ -240,6 +240,10 @@ async function execute(
const { rows, pagination, extra } = await quotas.addQuery(runFn, {
datasourceId: datasource._id,
})
// remove the raw from execution incase transformer being used to hide data
if (extra?.raw) {
delete extra.raw
}
if (opts && opts.rowsOnly) {
ctx.body = rows
} else {

9
packages/server/src/api/controllers/row/internalSearch.js
View file

@ -145,7 +145,7 @@ class QueryBuilder {
* @param options The preprocess options
* @returns {string|*}
*/
preprocess(value, { escape, lowercase, wrap } = {}) {
preprocess(value, { escape, lowercase, wrap, type } = {}) {
const hasVersion = !!this.version
// Determine if type needs wrapped
const originalType = typeof value
@ -157,8 +157,11 @@ class QueryBuilder {
if (escape && originalType === "string") {
value = `${value}`.replace(/[ #+\-&|!(){}\]^"~*?:\\]/g, "\\$&")
}
// Wrap in quotes
if (hasVersion && wrap) {
if (originalType === "string" && !isNaN(value) && !type) {
value = `"${value}"`
} else if (hasVersion && wrap) {
value = originalType === "number" ? value : `"${value}"`
}
return value
@ -253,6 +256,7 @@ class QueryBuilder {
value = builder.preprocess(value, {
escape: true,
lowercase: true,
type: "string",
})
return `${key}:${value}*`
})
@ -281,6 +285,7 @@ class QueryBuilder {
value = builder.preprocess(value, {
escape: true,
lowercase: true,
type: "fuzzy",
})
return `${key}:${value}~`
})

5
packages/server/src/api/controllers/static/index.ts
View file

@ -1,3 +1,5 @@
import { enrichPluginURLs } from "../../../utilities/plugins"
require("svelte/register")
const send = require("koa-send")
@ -107,12 +109,13 @@ export const serveApp = async function (ctx: any) {
if (!env.isJest()) {
const App = require("./templates/BudibaseApp.svelte").default
const plugins = enrichPluginURLs(appInfo.usedPlugins)
const { head, html, css } = App.render({
title: appInfo.name,
production: env.isProd(),
appId,
clientLibPath: clientLibraryPath(appId, appInfo.version, ctx),
usedPlugins: appInfo.usedPlugins,
usedPlugins: plugins,
})
const appHbs = loadHandlebarsFile(`${__dirname}/templates/app.hbs`)

4
packages/server/src/api/controllers/static/templates/BudibaseApp.svelte
View file

@ -88,9 +88,7 @@
<!-- But before loadBudibase is called -->
{#if usedPlugins?.length}
{#each usedPlugins as plugin}
<script
type="application/javascript"
src={`/plugins/${plugin.jsUrl}`}></script>
<script type="application/javascript" src={plugin.jsUrl}></script>
{/each}
{/if}
<script type="application/javascript">

12
packages/server/src/api/controllers/table/internal.ts
View file

@ -14,8 +14,10 @@ import {
fixAutoColumnSubType,
} from "../../../utilities/rowProcessor"
import { runStaticFormulaChecks } from "./bulkFormula"
import { Table } from "../../../definitions/common"
import { Table } from "@budibase/types"
import { quotas } from "@budibase/pro"
import { isEqual } from "lodash"
import { cloneDeep } from "lodash/fp"
function checkAutoColumns(table: Table, oldTable: Table) {
if (!table.schema) {
@ -123,10 +125,16 @@ export async function save(ctx: any) {
if (updatedRows && updatedRows.length !== 0) {
await db.bulkDocs(updatedRows)
}
const result = await db.put(tableToSave)
let result = await db.put(tableToSave)
tableToSave._rev = result.rev
const savedTable = cloneDeep(tableToSave)
tableToSave = await tableSaveFunctions.after(tableToSave)
// the table may be updated as part of the table save after functionality - need to write it
if (!isEqual(savedTable, tableToSave)) {
result = await db.put(tableToSave)
tableToSave._rev = result.rev
}
// has to run after, make sure it has _id
await runStaticFormulaChecks(tableToSave, { oldTable, deletion: null })
return tableToSave

2
packages/server/src/api/controllers/table/utils.ts
View file

@ -247,7 +247,7 @@ class TableSaveFunctions {
// after saving
async after(table: any) {
table = await handleSearchIndexes(table)
await handleDataImport(this.user, table, this.dataImport)
table = await handleDataImport(this.user, table, this.dataImport)
return table
}

20
packages/server/src/api/routes/tests/internalSearch.spec.js
View file

@ -173,4 +173,24 @@ describe("internal search", () => {
}, PARAMS)
checkLucene(response, `*:* AND NOT column:(a AND b AND c)`, PARAMS)
})
it("test equal without version query", async () => {
PARAMS.version = null
const response = await search.paginatedSearch({
equal: {
"column": "1",
}
}, PARAMS)
const query = response.rows[0].query
const json = JSON.parse(query)
if (PARAMS.sort) {
expect(json.sort).toBe(`${PARAMS.sort}<${PARAMS.sortType}>`)
}
if (PARAMS.bookmark) {
expect(json.bookmark).toBe(PARAMS.bookmark)
}
expect(json.include_docs).toBe(true)
expect(json.q).toBe(`(*:* AND column:"1") AND tableId:${PARAMS.tableId}`)
})
})

29
packages/server/src/threads/automation.ts
View file

@ -252,7 +252,7 @@ class Orchestrator {
let loopStepNumber: any = undefined
let loopSteps: LoopStep[] | undefined = []
let metadata
let wasLoopStep = false
// check if this is a recurring automation,
if (isProdAppID(this._appId) && isRecurring(automation)) {
metadata = await this.getMetadata()
@ -267,6 +267,7 @@ class Orchestrator {
let input,
iterations = 1,
iterationCount = 0
if (step.stepId === LOOP_STEP_ID) {
loopStep = step
loopStepNumber = stepCount
@ -277,10 +278,8 @@ class Orchestrator {
input = await processObject(loopStep.inputs, this._context)
iterations = getLoopIterations(loopStep as LoopStep, input)
}
for (let index = 0; index < iterations; index++) {
let originalStepInput = cloneDeep(step.inputs)
// Handle if the user has set a max iteration count or if it reaches the max limit set by us
if (loopStep && input.binding) {
let newInput = await processObject(
@ -313,7 +312,6 @@ class Orchestrator {
} else {
item = loopStep.inputs.binding
}
this._context.steps[loopStepNumber] = {
currentItem: item[index],
}
@ -331,6 +329,16 @@ class Orchestrator {
innerValue,
`steps.${loopStepNumber}`
)
} else if (typeof value === "object") {
for (let [innerObject, innerValue] of Object.entries(
originalStepInput[key][innerKey]
)) {
originalStepInput[key][innerKey][innerObject] =
automationUtils.substituteLoopStep(
innerValue,
`steps.${loopStepNumber}`
)
}
}
}
} else {
@ -386,6 +394,7 @@ class Orchestrator {
let stepFn = await this.getStepFunctionality(step.stepId)
let inputs = await processObject(originalStepInput, this._context)
inputs = automationUtils.cleanInputValues(inputs, step.schema.inputs)
try {
// appId is always passed
const outputs = await stepFn({
@ -394,6 +403,7 @@ class Orchestrator {
emitter: this._emitter,
context: this._context,
})
this._context.steps[stepCount] = outputs
// if filter causes us to stop execution don't break the loop, set a var
// so that we can finish iterating through the steps and record that it stopped
@ -419,6 +429,7 @@ class Orchestrator {
console.error(`Automation error - ${step.stepId} - ${err}`)
return err
}
if (loopStep) {
iterationCount++
if (index === iterations - 1) {
@ -429,6 +440,13 @@ class Orchestrator {
}
}
// Delete the step after the loop step as it's irrelevant, since information is included
// in the loop step
if (wasLoopStep) {
this._context.steps.splice(loopStepNumber + 1, 1)
wasLoopStep = false
}
if (loopSteps && loopSteps.length) {
let tempOutput = {
success: true,
@ -441,9 +459,10 @@ class Orchestrator {
outputs: tempOutput,
inputs: step.inputs,
})
this._context.steps[loopStepNumber] = tempOutput
this._context.steps.splice(loopStepNumber, 0, tempOutput)
loopSteps = undefined
wasLoopStep = true
}
}

21
packages/server/src/utilities/plugins.js Normal file
View file

@ -0,0 +1,21 @@
const env = require("../environment")
const { plugins: ProPlugins } = require("@budibase/pro")
const { objectStore } = require("@budibase/backend-core")
exports.enrichPluginURLs = plugins => {
if (!plugins || !plugins.length) {
return []
}
return plugins.map(plugin => {
const cloud = !env.SELF_HOSTED
const bucket = objectStore.ObjectStoreBuckets.PLUGINS
const jsFileName = "plugin.min.js"
// In self host we need to prefix the path, as the bucket name is not part
// of the bucket path. In cloud, it's already part of the bucket path.
let jsUrl = cloud ? "https://cdn.budi.live/" : `/${bucket}/`
jsUrl += ProPlugins.getBucketPath(plugin.name)
jsUrl += jsFileName
return { ...plugin, jsUrl }
})
}

30
packages/server/yarn.lock
View file

@ -1094,12 +1094,12 @@
resolved "https://registry.yarnpkg.com/@bcoe/v8-coverage/-/v8-coverage-0.2.3.tgz#75a2e8b51cb758a7553d6804a5932d7aace75c39"
integrity sha512-0hYQ8SB4Db5zvZB4axdMHGwEaQjkZzFjQiN9LVYvIFB2nSUHW9tYpxWriPrWDASIxiaXax83REcLxuSdnGPZtw==
"@budibase/backend-core@2.0.14-alpha.4":
version "2.0.14-alpha.4"
resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-2.0.14-alpha.4.tgz#213446a9e04385fb38ab8785742cdb8b4eaf2cac"
integrity sha512-ma1Ipst4AQVi4sx+ULs3bX9xT4eHqavCvM/BXtZzV23SOTdjJTi9wX2KWbIvAUT5xo2NgN0uRf5zf7B6CBRuXw==
"@budibase/backend-core@2.0.24-alpha.0":
version "2.0.24-alpha.0"
resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-2.0.24-alpha.0.tgz#db53e562cd40aa21783f6f4d6eaf880ec8225a8f"
integrity sha512-o8Ooa5Ga3hcbovoO+Fjyp++TcJmt8v83Y7GfurLznjAc5lakdEe8RWKR9JV7xqenXpZ49ZwWYTSOLwxFBiJuvw==
dependencies:
"@budibase/types" "2.0.14-alpha.4"
"@budibase/types" "2.0.24-alpha.0"
"@shopify/jest-koa-mocks" "5.0.1"
"@techpass/passport-openidconnect" "0.3.2"
aws-sdk "2.1030.0"
@ -1180,13 +1180,13 @@
svelte-flatpickr "^3.2.3"
svelte-portal "^1.0.0"
"@budibase/pro@2.0.14-alpha.4":
version "2.0.14-alpha.4"
resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-2.0.14-alpha.4.tgz#9796271365f4ffb875655f86df84a2b4111b793a"
integrity sha512-I5QIv04hfoUPEYHPkCD5L/5WU7cJurmfhjy62ZMTNbJzFzhIGDzQlOjM0MnQO5kJvGv1NWl2PQvSty1rYzXGsw==
"@budibase/pro@2.0.24-alpha.0":
version "2.0.24-alpha.0"
resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-2.0.24-alpha.0.tgz#c7c70f431d9d17603d965b353dbb7c4956b6f7a0"
integrity sha512-bp+m9mXEonvxiLo4s9Gt4vhHuVrderFB4vfCISMRcIpZTo/CPVkMTOEM7C4liZyXOiLB9hh4Y+vQY1T6ilIr3Q==
dependencies:
"@budibase/backend-core" "2.0.14-alpha.4"
"@budibase/types" "2.0.14-alpha.4"
"@budibase/backend-core" "2.0.24-alpha.0"
"@budibase/types" "2.0.24-alpha.0"
"@koa/router" "8.0.8"
joi "17.6.0"
node-fetch "^2.6.1"
@ -1209,10 +1209,10 @@
svelte-apexcharts "^1.0.2"
svelte-flatpickr "^3.1.0"
"@budibase/types@2.0.14-alpha.4":
version "2.0.14-alpha.4"
resolved "https://registry.yarnpkg.com/@budibase/types/-/types-2.0.14-alpha.4.tgz#37b1ff362d2551fd978e8c8153a4cec357eda0b0"
integrity sha512-l4dfMh5it1N56nc+jCtdXklrZA1cMt6WSVBoKCXBedZGpguJr18Wo/cfn1YWryuUMuSlpFVDTrlQQXCZ+ku92g==
"@budibase/types@2.0.24-alpha.0":
version "2.0.24-alpha.0"
resolved "https://registry.yarnpkg.com/@budibase/types/-/types-2.0.24-alpha.0.tgz#d5ff01dc4a6f1d584077b1808c0a6c8cffbd1eea"
integrity sha512-eiTumTQp6d9+bizhImqK+OJ4g4b9IuUTCOdNp7K2rwu0v94xxLuQEtJUA5MKiqClx7Uu27IlybeQmP+oM99gTg==
"@bull-board/api@3.7.0":
version "3.7.0"

2
packages/string-templates/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/string-templates",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Handlebars wrapper for Budibase templating.",
"main": "src/index.cjs",
"module": "dist/bundle.mjs",

2
packages/types/package.json
View file

@ -1,6 +1,6 @@
{
"name": "@budibase/types",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Budibase types",
"main": "dist/index.js",
"types": "dist/index.d.ts",

10
packages/worker/package.json
View file

@ -1,7 +1,7 @@
{
"name": "@budibase/worker",
"email": "hi@budibase.com",
"version": "2.0.14-alpha.4",
"version": "2.0.24-alpha.0",
"description": "Budibase background service",
"main": "src/index.ts",
"repository": {
@ -36,10 +36,10 @@
"author": "Budibase",
"license": "GPL-3.0",
"dependencies": {
"@budibase/backend-core": "2.0.14-alpha.4",
"@budibase/pro": "2.0.14-alpha.4",
"@budibase/string-templates": "2.0.14-alpha.4",
"@budibase/types": "2.0.14-alpha.4",
"@budibase/backend-core": "2.0.24-alpha.0",
"@budibase/pro": "2.0.24-alpha.0",
"@budibase/string-templates": "2.0.24-alpha.0",
"@budibase/types": "2.0.24-alpha.0",
"@koa/router": "8.0.8",
"@sentry/node": "6.17.7",
"@techpass/passport-openidconnect": "0.3.2",

30
packages/worker/yarn.lock
View file

@ -291,12 +291,12 @@
resolved "https://registry.yarnpkg.com/@bcoe/v8-coverage/-/v8-coverage-0.2.3.tgz#75a2e8b51cb758a7553d6804a5932d7aace75c39"
integrity sha512-0hYQ8SB4Db5zvZB4axdMHGwEaQjkZzFjQiN9LVYvIFB2nSUHW9tYpxWriPrWDASIxiaXax83REcLxuSdnGPZtw==
"@budibase/backend-core@2.0.14-alpha.4":
version "2.0.14-alpha.4"
resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-2.0.14-alpha.4.tgz#213446a9e04385fb38ab8785742cdb8b4eaf2cac"
integrity sha512-ma1Ipst4AQVi4sx+ULs3bX9xT4eHqavCvM/BXtZzV23SOTdjJTi9wX2KWbIvAUT5xo2NgN0uRf5zf7B6CBRuXw==
"@budibase/backend-core@2.0.24-alpha.0":
version "2.0.24-alpha.0"
resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-2.0.24-alpha.0.tgz#db53e562cd40aa21783f6f4d6eaf880ec8225a8f"
integrity sha512-o8Ooa5Ga3hcbovoO+Fjyp++TcJmt8v83Y7GfurLznjAc5lakdEe8RWKR9JV7xqenXpZ49ZwWYTSOLwxFBiJuvw==
dependencies:
"@budibase/types" "2.0.14-alpha.4"
"@budibase/types" "2.0.24-alpha.0"
"@shopify/jest-koa-mocks" "5.0.1"
"@techpass/passport-openidconnect" "0.3.2"
aws-sdk "2.1030.0"
@ -327,21 +327,21 @@
uuid "8.3.2"
zlib "1.0.5"
"@budibase/pro@2.0.14-alpha.4":
version "2.0.14-alpha.4"
resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-2.0.14-alpha.4.tgz#9796271365f4ffb875655f86df84a2b4111b793a"
integrity sha512-I5QIv04hfoUPEYHPkCD5L/5WU7cJurmfhjy62ZMTNbJzFzhIGDzQlOjM0MnQO5kJvGv1NWl2PQvSty1rYzXGsw==
"@budibase/pro@2.0.24-alpha.0":
version "2.0.24-alpha.0"
resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-2.0.24-alpha.0.tgz#c7c70f431d9d17603d965b353dbb7c4956b6f7a0"
integrity sha512-bp+m9mXEonvxiLo4s9Gt4vhHuVrderFB4vfCISMRcIpZTo/CPVkMTOEM7C4liZyXOiLB9hh4Y+vQY1T6ilIr3Q==
dependencies:
"@budibase/backend-core" "2.0.14-alpha.4"
"@budibase/types" "2.0.14-alpha.4"
"@budibase/backend-core" "2.0.24-alpha.0"
"@budibase/types" "2.0.24-alpha.0"
"@koa/router" "8.0.8"
joi "17.6.0"
node-fetch "^2.6.1"
"@budibase/types@2.0.14-alpha.4":
version "2.0.14-alpha.4"
resolved "https://registry.yarnpkg.com/@budibase/types/-/types-2.0.14-alpha.4.tgz#37b1ff362d2551fd978e8c8153a4cec357eda0b0"
integrity sha512-l4dfMh5it1N56nc+jCtdXklrZA1cMt6WSVBoKCXBedZGpguJr18Wo/cfn1YWryuUMuSlpFVDTrlQQXCZ+ku92g==
"@budibase/types@2.0.24-alpha.0":
version "2.0.24-alpha.0"
resolved "https://registry.yarnpkg.com/@budibase/types/-/types-2.0.24-alpha.0.tgz#d5ff01dc4a6f1d584077b1808c0a6c8cffbd1eea"
integrity sha512-eiTumTQp6d9+bizhImqK+OJ4g4b9IuUTCOdNp7K2rwu0v94xxLuQEtJUA5MKiqClx7Uu27IlybeQmP+oM99gTg==
"@cspotcode/source-map-consumer@0.8.0":
version "0.8.0"

2
scripts/install-minio.sh
View file

@ -1,8 +1,10 @@
#!/bin/bash
if [[ $TARGETARCH == arm* ]] ;
then
echo "INSTALLING ARM64 MINIO"
wget https://dl.min.io/server/minio/release/linux-arm64/minio
else
echo "INSTALLING AMD64 MINIO"
wget https://dl.min.io/server/minio/release/linux-amd64/minio
fi
chmod +x minio

5
scripts/pro/install.sh
View file

@ -18,6 +18,11 @@ git clone https://$PERSONAL_ACCESS_TOKEN@github.com/Budibase/budibase-pro.git
if [[ -d "budibase-pro" ]]; then
cd budibase-pro
if [[ -z "${BRANCH}" ]]; then
echo Using GITHUB_REF_NAME: $GITHUB_REF_NAME
export BRANCH=$GITHUB_REF_NAME
fi
# Try to checkout the matching pro branch
git checkout $BRANCH