alnoda-workspaces/experimental/sqlite-workspace/README.md

SQLite Workspace

Collection of tools to work with SQLite.

Helps to get started fast locally, because SQLite and toolset are already installed and configured. It is only needed to start a container.

Also can be self-hosted and used for collaboration when developing embedded database.

Try it out:

docker run --name space-1 -d -p 8020-8035:8020-8035 alnoda/sqlite-workspace

Open localhost:8020

Features

SQLite Features:

1. sqlean. The ultimate set of compiled SQLite extensions.
2. sqlite-web. Web-based SQLite database browser.
3. litecli. CLI for SQLite.
4. sqlite-viewer. View SQLite file online.
5. Web-GUI-for-SQLite. Alternative tool to view SQLite file online.
6. DBdesigner. Draw ERD diagrams and use it to generate DDL code.
7. tbls. Generate documentation (essentially data catalog) from the database

Workspace Features:

1. Browser-based VS-code with extensions:
   • PostgreSQL Client for vscode
   • SQLTools PostgreSQL
2. Browser-based terminal. Finally no need to ssh into a running docker container.
3. Cronicle. Browser-based jobs scheduler. Schedule backups, data extracts, maintenance scripts.
4. Ubuntu with many apps installed:
   • text editors
   • Zsh
   • cron
   • mc
   • and many more
5. Rclone. Save SQLite databases to S3.
6. Programming languages. Develop/execute directly inside the workspace using browser-based IDE.
   • python
   • nodejs
7. Filebrowser. Browse and download files (finally no need to mount folders for that).
8. Static file server. View generated HTML reports.
9. Ungit. The easiest way to use git.
10. Browser-based process and resource utilization monitor
11. Mkdocs. Create documentation with Markdown.
12. Workspace own UI, from which you can open all applications and documentation.

Get started

Sqlite

SQLite3 is installed. Open terminal http://localhost:8026/ and execute

sqlite3

Note: tcheck out a better CLI litecli

Sqlite extensions

The following compiled extensions are present in the folder /home/sqlite-extensions

• crypto: secure hashes
• fileio: read and write files
• fuzzy: fuzzy string matching and phonetics
• ipaddr: IP address manipulation
• re: regular expressions
• stats: math statistics
• text: string functions
• unicode: Unicode support
• uuid: Universally Unique IDentifiers

Example start SQLite, and load extension:

sqlite3
.load /home/sqlite-extensions/stats

sqlite-web

Serve any SQLite database. There is a port assigned - 8033. Use this port to start sqlite-web for your database. Workspace UI has a shorcut to quickly open sqlite-web running on port 8033.

Example, start sqlite-web on port 8033 for SQLite database file /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite

sqlite_web /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite --host 0.0.0.0 --port 8033 

and with extensions loaded:

sqlite_web /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite --host 0.0.0.0 --port 8033  -e /home/sqlite-extensions/text.so -e /home/sqlite-extensions/unicode.so -e /home/sqlite-extensions/uuid.so

sqlite-viewer. View SQLite file online.

Upload SQLite database file (from your local machine) annd explore.

Web-GUI-for-SQLite

Upload SQLite database file (from your local machine) annd explore.

litecli

A command-line client for SQLite databases that has auto-completion and syntax highlighting. Useful if you prefer CLI.

Open Terminal and try with example database:

litecli /home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite

Load extensions

.load /home/sqlite-extensions/stats

Advanced

SQLite Json

Examples

CREATE TABLE jsontest   (
  id INTEGER NOT NULL PRIMARY KEY,
  jss JSON
);

INSERT INTO jsontest  VALUES 
(1, json(' { "key1" : "my key 1", "key2": [ "test" ] } ')),
(2, json(' { "key1" : "my new key", "key2": [ "test", "test" ] } ')),
(3, json(' { "key1" : "my yet another key", "key2": [ "test", "test", "test2" ] } '));

SELECT * FROM jsontest;

Links: - Complete SQlite JSON docs

rclone

rclone is a command line program to manage files on cloud storage. It is a feature rich alternative to cloud vendors' web storage interfaces.
Copy dump to S3:
Create file ~/.config/rclone/rclone.conf with the following content

[remote]
type = s3
provider = AWS
access_key_id = XXXXXXXXXXXXXXXXXXXXXX
secret_access_key = XXxxXXxXXXxxxXXxXXXxxXXXxxxXXXxXXXX
region = xx-xxxx-x

Use Rclone to copy to S3 and delete from local

rclone move /home/project/sqlite-db remote:my-s3-bucket/sqlite/

Restore from S3 to local: When there is a need to get database from S3 to local folder.

# copy 
rclone copy remote:my-s3-bucket/sqlite/ /home/sqlite-db/
# mount S3 without copying
rclone sync remote:my-s3-bucket/sqlite/ /home/sqlite-db/

tbls

Generate docs for the example database

tbls doc sqlite:////home/static-server/sqlite-viewer/examples/Chinook_Sqlite.sqlite /home/static-server/sql-tbls

and view SVG files with Static File Server