feat(server): improve indexer perf (#15512)

#### PR Dependency Tree


* **PR #15512** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Search now supports generation-based indexing with embedded and remote
providers.
* Added automatic search reconciliation and improved handling of
document, workspace, and permission changes.
* Added clearer search status errors for unavailable, syncing, unready,
or failed indexes.
* Added Manticore Search end-to-end support and provider-specific search
behavior.

* **Improvements**
* Search and aggregate pagination now report returned results and
continuation status more accurately.
* Improved permission filtering to prevent inaccessible documents from
appearing in results.
  * Admin provider selection now consistently enables indexing.

* **Documentation**
* Clarified search pagination, aggregation counts, provider
configuration, and end-to-end setup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
DarkSky
2026-08-23 17:54:49 +08:00
committed by GitHub
parent a8eef53966
commit b530198a3b
107 changed files with 8612 additions and 5306 deletions

View File

@@ -0,0 +1,45 @@
# Manticore Search provider E2E
This fixture validates the shared immutable projection contract against a real
Manticore Search RT table. Aggregate queries remain unsupported and are validated
through the typed fallback boundary instead.
Start the external dependency:
```bash
docker compose -f packages/backend/server/e2e/manticore-provider/compose.yml up -d --wait
```
Use a disposable PostgreSQL database and run the provider-gated E2E:
```bash
DATABASE_URL=postgresql://ds:ds@localhost:55433/affine_rfc6_manticore_e2e \
yarn workspace @affine/server prisma migrate deploy
```
Configure `packages/backend/server/config.json` with:
```json
{
"indexer": {
"enabled": true,
"provider": {
"type": "manticoresearch",
"endpoint": "http://127.0.0.1:9308"
}
}
}
```
Then run:
```bash
DATABASE_URL=postgresql://ds:ds@localhost:55433/affine_rfc6_manticore_e2e \
yarn af server e2e src/__tests__/e2e/indexer/manticore-provider.spec.ts
```
Stop only this disposable dependency when finished:
```bash
docker compose -f packages/backend/server/e2e/manticore-provider/compose.yml down -v
```