jdcaballerov
14ef5c1536
Add sonic dependency for archivebox service
2020-12-07 15:39:18 -05:00
jdcaballerov
da5a8775cb
Add sonic client dependency
2020-12-07 15:36:56 -05:00
Nick Sweeting
e634b70904
Merge pull request #574 from jdcaballerov/patch-3
...
Set info as default logging level for sonic
2020-12-07 15:03:58 -05:00
Nick Sweeting
b5862afcf1
Merge pull request #572 from jdcaballerov/patch-1
...
Add ripgrep
2020-12-07 15:03:36 -05:00
jdcaballerov
030183cf44
Update docker-compose.yml
2020-12-07 14:39:49 -05:00
jdcaballerov
4f6be55753
Set info as default logging level for sonic
2020-12-07 13:23:37 -05:00
jdcaballerov
2082b98fd7
Use sonic as default search backend
2020-12-07 13:22:09 -05:00
jdcaballerov
d8c6100b13
Add ripgrep
2020-12-07 12:46:14 -05:00
Nick Sweeting
60f5b94d74
Merge pull request #571 from cdvv7788/remove-walrus
2020-12-06 15:17:53 -05:00
Cristian
db73d92f83
docs: Update shell message to import models
2020-12-06 12:26:22 -05:00
Cristian
8d22ebf988
feat: Remove walrus operator (we still need to support python3.7)
2020-12-06 12:23:02 -05:00
Nick Sweeting
8aee5c408a
improve workflow names for clarity in PR status
2020-12-06 16:26:44 +02:00
Nick Sweeting
6ac48d7c35
tweak warning msg
2020-12-06 02:11:36 +02:00
Nick Sweeting
a0a79cead8
move utils and vendored libs into subfolders
2020-12-06 02:01:18 +02:00
Nick Sweeting
8440858751
bump pip_dist build
2020-12-06 01:38:51 +02:00
Nick Sweeting
8d103687d0
Merge pull request #570 from ArchiveBox/sonic-search
2020-12-05 18:22:17 -05:00
jdcaballerov
172197ae01
refactor: Remove if LENGTH and use text chunker for every input
2020-12-06 01:14:39 +02:00
jdcaballerov
5a6b814c79
Add exception handling for indexable content reader
2020-12-06 01:14:38 +02:00
JDC
b1d70185ed
Increase word_objects for Sonic default config
2020-12-06 01:14:38 +02:00
JDC
15fbd81480
Change MAX_SONIC_TEXT_LENGTH
2020-12-06 01:14:38 +02:00
JDC
db9c2edccc
Add log print for url indexing
2020-12-06 01:14:38 +02:00
JDC
0acf479b70
Partition long strings in chunks for sonic
2020-12-06 01:14:38 +02:00
JDC
caf4660ac8
Add indexing to update command and utilities
2020-12-06 01:14:37 +02:00
JDC
273c9d91c6
Add tag filter to update command
2020-12-06 01:13:39 +02:00
JDC
7903db6dfb
Add ArchiveResult Manager and sorted indexable filter
2020-12-06 01:13:39 +02:00
JDC
23a9beb4e0
Add ignored extensions in ripgrep search
2020-12-06 01:13:39 +02:00
JDC
95382b3812
Add ripgrep rg search backend and set as default
2020-12-06 01:13:39 +02:00
JDC
8484bdb973
Fix add search filter to update
2020-12-06 01:13:39 +02:00
JDC
c5b1b91708
fix: flush_search_index must be called before removing snapshots
2020-12-06 01:13:39 +02:00
JDC
70cc0c1950
Add search filter-type
2020-12-06 01:13:39 +02:00
JDC
4eeedae815
Exception handling for indexing and searching
2020-12-06 01:13:39 +02:00
JDC
0ed53cc117
Add search filter type for update
2020-12-06 01:13:39 +02:00
JDC
9bd40ed7f6
Max out number of queries
2020-12-06 01:13:39 +02:00
JDC
a38e3e0c90
Get searc backend password from env var SEARCH_BACKEND_PASSWORD
2020-12-06 01:13:39 +02:00
JDC
0773f12034
Add sonic to docker-compose
2020-12-06 01:13:38 +02:00
JDC
0f7dba07df
feat: add search filter-type to list command
2020-12-06 01:13:37 +02:00
JDC
fb67d6684c
fix: Return empty QuerySet instead of list
2020-12-06 01:12:47 +02:00
JDC
823df34080
Use QuerySets for search backend API instead of pks
2020-12-06 01:12:47 +02:00
JDC
f383648ffc
Use a generator for snapshot flush from index
2020-12-06 01:12:47 +02:00
JDC
47daa038eb
Implement flush for search backend after remove command
2020-12-06 01:12:47 +02:00
JDC
c2c01af3ad
Add config for search backend
2020-12-06 01:12:47 +02:00
JDC
5f6673c72c
Implement backend architecture for search engines
2020-12-06 01:12:46 +02:00
JDC
b1f70b2197
Initial implementation
2020-12-06 01:12:45 +02:00
Nick Sweeting
7bc13204e6
Merge branch 'master' into v0.5.0
2020-12-05 17:45:16 -05:00
Nick Sweeting
3b280e6b02
Merge pull request #569 from cdvv7788/extract-command-update
...
feat: Add --extract flag to update command
2020-12-05 17:43:28 -05:00
Nick Sweeting
a05671f2be
Merge pull request #568 from cdvv7788/#544-update-index-only
...
#544 update: --index-only flag
2020-12-05 17:40:22 -05:00
Cristian
35389608d1
feat: Add --extract flag to update command
2020-12-05 12:20:47 -05:00
Cristian
34cad4fe8d
fix: Update function with --index-only flag was not behaving as expected
2020-12-05 12:10:17 -05:00
Nick Sweeting
f467435797
Merge pull request #528 from mAAdhaTTah/pull-from-pocket-api
...
Add parser for Pocket API
2020-12-04 22:58:41 -05:00
mAAdhaTTah
ac7ad9e942
Add parser for Pocket API
...
Pass a url like `pocket://Username` to import that username's archived Pocket
library. Tokens need to be stored in ArchveBox.conf with the following keys:
```
POCKET_CONSUMER_KEY = key-from-custom-pocket-app
POCKET_ACCESS_TOKENS = {"YourUsername": "pocket-token-for-app"}
```
`POCKET_ACCESS_TOKENS` MUST be on a single line, or the JSON will be
misinterpreted by the parser as a new key/value pair.
2020-12-04 22:54:39 -05:00
