alnoda-workspaces/workspaces/sqlite-workspace/README.md

SQLite Workspace

Collection of tools to develop SQLite databases. Useful if you need to create an embedded database for your application, and fill it with data.

Helps to get started fast locally, because SQLite and toolset are already installed and configured. It is only needed to start a container.

Also can be self-hosted and used for collaboration when developing embedded database.

Try it out:

docker run --name space-1 -d -p 8020-8035:8020-8035 alnoda/sqlite-workspace

Open localhost:8020

Features

SQLite Features:

1. sqlean. The ultimate set of compiled SQLite extensions.
2. nocodb. Great UI to fill SQLite with data. NocoDB is an open source Airtable alternative.
3. sqlite-web. Web-based SQLite database browser.
4. litecli. CLI for SQLite.
5. sqlite-viewer. View SQLite file online.
6. Web-GUI-for-SQLite. Alternative tool to view SQLite file online.
7. DBdesigner. Draw ERD diagrams and use it to generate DDL code.
8. tbls. Generate documentation (essentially data catalog) from the database

Workspace Features:

1. Browser-based VS-code with extensions:
   • PostgreSQL Client for vscode
   • SQLTools PostgreSQL
2. Browser-based terminal. Finally no need to ssh into a running docker container.
3. Cronicle. Browser-based jobs scheduler. Schedule backups, data extracts, maintenance scripts.
4. Ubuntu with many apps installed:
   • text editors
   • Zsh
   • cron
   • mc
   • and many more
5. Rclone. Save SQLite databases to S3.
6. Programming languages. Develop/execute directly inside the workspace using browser-based IDE.
   • python
   • nodejs
7. Filebrowser. Browse and download files (finally no need to mount folders for that).
8. Static file server. View generated HTML reports.
9. Ungit. The easiest way to use git.
10. Browser-based process and resource utilization monitor
11. Mkdocs. Create documentation with Markdown.
12. Workspace own UI, from which you can open all applications and documentation.

Get started

NocoDB

NocoDB. NocoDB is an open source Airtable alternative. Use it to create SQLite tables, develop databases without SQL.

Launch the workspace, navigate to the Workspace UI and open NocoDB. Or open localhost:8030 directly in browser. Create new user (provide any email,pass). Create tables, enter data, import data files, create forms and much more.

query NocoDB data

NocoDB is backed by SQLite3, database file /home/nocodb/noco.db .

You can query NocoDB databse directly with sqlite-web. Open it through the Workspace UI, or navigate to localhost:8031 directly.

At any moment you can copy SQLite database from NocoDB with all the data, edit, modify and configure for the needs of your application.

• Copy of NocoDB database and continue developing with Sqlite-web

mv -f /home/nocodb/noco.db /home/project/sqlite-db/db-main.sqlite 

Open Sqlite-web on localhost:8032

NocoDB with another SQLite database

If yopu create NocoDB project with external database, tables created by NocoDB will not have prefixes.

For example, create project with the example Chinook SQLite database file /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite, and query data also with sqlite-web on localhost:8033.

Or create NocoDB project with external SQLite database file /home/project/sqlite-db/db-main.sqlite and query with sqlite-web on localhost:8032.

Sqlite

SQLite3 is installed. Open terminal http://localhost:8026/ and execute

sqlite3

Note: tcheck out a better CLI litecli

Sqlite extensions

The following compiled extensions are present in the folder /home/sqlite-extensions

• crypto: secure hashes
• fileio: read and write files
• fuzzy: fuzzy string matching and phonetics
• ipaddr: IP address manipulation
• re: regular expressions
• stats: math statistics
• text: string functions
• unicode: Unicode support
• uuid: Universally Unique IDentifiers

Example start SQLite, and load extension:

sqlite3
.load /home/sqlite-extensions/stats

sqlite-web

Serve any SQLite database. There are 2 SQlite database served by default:

• NocoDB database. Sqlite-web is served on localhost:8031
• "Main" database. Empty SQlite3 database. Sqlite-web is served on localhost:8032
• "Example/test" database. Filled with test "Chinook" database. Sqlite-web is served on localhost:8033

Copy SQlite databases

SQlite is just a file, jou can copy it and replace other databases. For example, if you want to copy NocoDB database into the "Main" database, which is served by Sqlite-web on localhost:8032 simply execute

mv -f /home/nocodb/noco.db /home/project/sqlite-db/db-main.sqlite 

Serve on-demand SQlite database

One port 8034 is reserved for serving on-demand database with Sqlite-web. Serve it with the following command

sqlite_web <path-to-your-sqlite-database-file> --host 0.0.0.0 --port 8034

Example, start sqlite-web on port 8034 for SQLite database file /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite

sqlite_web /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite --host 0.0.0.0 --port 8034 

and with extensions loaded:

sqlite_web /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite --host 0.0.0.0 --port 8034  -e /home/sqlite-extensions/text.so -e /home/sqlite-extensions/unicode.so -e /home/sqlite-extensions/uuid.so

sqlite-viewer. View SQLite file online.

Upload SQLite database file (from your local machine) and explore.

Web-GUI-for-SQLite

Upload SQLite database file (from your local machine) and explore.

litecli

A command-line client for SQLite databases that has auto-completion and syntax highlighting. Useful if you prefer CLI.

Open Terminal and try with example database:

litecli /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite

Load extensions

.load /home/sqlite-extensions/stats

Advanced

SQLite Json

Examples

CREATE TABLE jsontest   (
  id INTEGER NOT NULL PRIMARY KEY,
  jss JSON
);

INSERT INTO jsontest  VALUES 
(1, json(' { "key1" : "my key 1", "key2": [ "test" ] } ')),
(2, json(' { "key1" : "my new key", "key2": [ "test", "test" ] } ')),
(3, json(' { "key1" : "my yet another key", "key2": [ "test", "test", "test2" ] } '));

SELECT * FROM jsontest;

Links: - Complete SQlite JSON docs

rclone

rclone is a command line program to manage files on cloud storage. It is a feature rich alternative to cloud vendors' web storage interfaces.
Copy dump to S3:
Create file ~/.config/rclone/rclone.conf with the following content

[remote]
type = s3
provider = AWS
access_key_id = XXXXXXXXXXXXXXXXXXXXXX
secret_access_key = XXxxXXxXXXxxxXXxXXXxxXXXxxxXXXxXXXX
region = xx-xxxx-x

Use Rclone to copy to S3 and delete from local

rclone move /home/project/sqlite-db remote:my-s3-bucket/sqlite/

Restore from S3 to local: When there is a need to get database from S3 to local folder.

# copy 
rclone copy remote:my-s3-bucket/sqlite/ /home/sqlite-db/
# mount S3 without copying
rclone sync remote:my-s3-bucket/sqlite/ /home/sqlite-db/

tbls

Generate docs for the example database

tbls doc sqlite:////home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite /home/static-server/sql-tbls

and view SVG files with Static File Server