luulam/AFFiNE
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

feat(server): improve indexer perf (#15512)

Browse Source 
#### PR Dependency Tree


* **PR #15512** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Search now supports generation-based indexing with embedded and remote
providers.
* Added automatic search reconciliation and improved handling of
document, workspace, and permission changes.
* Added clearer search status errors for unavailable, syncing, unready,
or failed indexes.
* Added Manticore Search end-to-end support and provider-specific search
behavior.

* **Improvements**
* Search and aggregate pagination now report returned results and
continuation status more accurately.
* Improved permission filtering to prevent inaccessible documents from
appearing in results.
  * Admin provider selection now consistently enables indexing.

* **Documentation**
* Clarified search pagination, aggregation counts, provider
configuration, and end-to-end setup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
DarkSky
2026-08-23 17:54:49 +08:00
committed by GitHub
parent a8eef53966
commit b530198a3b
107 changed files with 8612 additions and 5306 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
packages/frontend/admin/src/config.json
View File

@@ -35,10 +35,6 @@
"type": "Object",
"desc": "The config for doc job queue"
},
"queues.indexer": {
"type": "Object",
"desc": "The config for indexer job queue"
},
"queues.notification": {
"type": "Object",
"desc": "The config for notification job queue"
@@ -398,39 +394,29 @@
"indexer": {
"enabled": {
"type": "Boolean",
"desc": "Enable indexer plugin",
"env": "AFFINE_INDEXER_ENABLED"
"desc": "Enable indexer plugin"
},
"provider.type": {
"type": "String",
"desc": "Indexer search provider. Self-hosted uses the embedded provider by default; remote providers require an endpoint.",
"env": "AFFINE_INDEXER_SEARCH_PROVIDER"
"desc": "Indexer search provider. Self-hosted uses the embedded provider by default; remote providers require an endpoint."
},
"provider.endpoint": {
"type": "String",
"desc": "Remote indexer endpoint. Not used by the embedded provider.",
"env": "AFFINE_INDEXER_SEARCH_ENDPOINT"
"desc": "Remote indexer endpoint. Not used by the embedded provider."
},
"provider.apiKey": {
"type": "String",
"desc": "Indexer search service api key. Optional for elasticsearch",
"link": "https://www.elastic.co/guide/server/current/api-key.html",
"env": "AFFINE_INDEXER_SEARCH_API_KEY"
"desc": "Indexer search service api key. Optional for remote providers",
"link": "https://www.elastic.co/guide/server/current/api-key.html"
},
"provider.username": {
"type": "String",
"desc": "Indexer search service auth username, if not set, basic auth will be disabled. Optional for elasticsearch",
"link": "https://www.elastic.co/guide/en/elasticsearch/reference/current/http-clients.html",
"env": "AFFINE_INDEXER_SEARCH_USERNAME"
"desc": "Indexer search service auth username, if not set, basic auth will be disabled. Optional for remote providers",
"link": "https://www.elastic.co/guide/en/elasticsearch/reference/current/http-clients.html"
},
"provider.password": {
"type": "String",
"desc": "Indexer search service auth password, if not set, basic auth will be disabled. Optional for elasticsearch",
"env": "AFFINE_INDEXER_SEARCH_PASSWORD"
},
"autoIndex.batchSize": {
"type": "Number",
"desc": "Number of workspaces automatically indexed per batch"
"desc": "Indexer search service auth password, if not set, basic auth will be disabled. Optional for remote providers"
}
},
"oauth": {