luulam/AFFiNE
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

feat(core): add collection rules module (#11683)

Browse Source 
whats changed:

### orm

add a new `select$` method, can subscribe on only one field to improve batch subscribe performance

### yjs-observable

add a new `yjsObservePath` method, which can subscribe to changes from specific path in yjs. Improves batch subscribe performance

```ts
yjsGetPath(
      this.workspaceService.workspace.rootYDoc.getMap('meta'),
      'pages'
    ).pipe(
      switchMap(pages => yjsObservePath(pages, '*.tags')),
      map(pages => {
          // only when tags changed
      })
)
```

### standard property naming

All `DocProperty` components renamed to `WorkspaceProperty` which is consistent with the product definition.

### `WorkspacePropertyService`

Split the workspace property management logic from the `doc` module and create a new `WorkspacePropertyService`. The new service manages the creation and modification of properties, and the `docService` is only responsible for storing the property value data.

### new `<Filters />` component

in `core/component/filter`

### new `<ExplorerDisplayMenuButton />` component

in `core/component/explorer/display-menu`

![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/g3jz87HxbjOJpXV3FPT7/c47ab43c-ac53-4ab6-922e-03127d07bef3.png)

### new `/workspace/xxx/all-new` route

New route for test components and functions

### new collection role service

Implemented some filter group order rules

see `collection-rules/index.ts`

### standard property type definition

define type in `modules\workspace-property\types.ts`

![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/g3jz87HxbjOJpXV3FPT7/4324453a-4fab-4d1e-83bb-53693e68e87a.png)

define components (name,icon,....) in `components\workspace-property-types\index.ts`

![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/g3jz87HxbjOJpXV3FPT7/93a23947-aaff-480d-a158-dd4075baae17.png)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Introduced comprehensive filtering, grouping, and ordering capabilities for workspace documents with reactive updates.
  - Added a new "All Pages" workspace view supporting dynamic filters and display preferences.
  - Developed UI components for filter creation, condition editing, and display menu controls.
  - Launched enhanced tag management with inline editors, selection, creation, and deletion workflows.
  - Added workspace property types with dedicated filter UIs including checkbox, date, tags, and text.
  - Introduced workspace property management replacing document property handling.
  - Added modular providers for filters, group-by, and order-by operations supporting various property types and system attributes.

- **Improvements**
  - Standardized tag and property naming conventions across the application (using `name` instead of `value` or `title`).
  - Migrated document property handling to workspace property-centric logic.
  - Enhanced internationalization with additional filter and display menu labels.
  - Improved styling for filter conditions, display menus, and workspace pages.
  - Optimized reactive data subscriptions and state management for performance.
  - Refined schema typings and type safety for workspace properties.
  - Updated imports and component references to workspace property equivalents throughout frontend.

- **Bug Fixes**
  - Resolved tag property inconsistencies affecting display and filtering.
  - Fixed filter and tag selection behaviors for accurate and reliable UI interactions.

- **Chores**
  - Added and refined test cases for ORM, observables, and filtering logic.
  - Cleaned up legacy document property code and improved type safety.
  - Modularized and restructured components for better maintainability.
  - Introduced new CSS styles for workspace pages and display menus.
  - Added framework module configurations for collection rules and workspace property features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
EYHN
2025-05-08 08:38:56 +00:00
parent f9e003d220
commit 8399d99e79
174 changed files with 5644 additions and 980 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
packages/frontend/core/src/modules/doc/constants.ts
View File

@@ -1,60 +0,0 @@
import type { DocCustomPropertyInfo } from '../db';
/**
* default built-in custom property, user can update and delete them
*
* 'id' and 'type' is request, 'index' is a manually maintained incremental key.
*/
export const BUILT_IN_CUSTOM_PROPERTY_TYPE: DocCustomPropertyInfo[] = [
{
id: 'tags',
type: 'tags',
index: 'a0000001',
},
{
id: 'docPrimaryMode',
type: 'docPrimaryMode',
show: 'always-hide',
index: 'a0000002',
},
{
id: 'journal',
type: 'journal',
show: 'always-hide',
index: 'a0000003',
},
{
id: 'template',
type: 'template',
index: 'a00000031',
show: 'always-hide',
},
{
id: 'createdAt',
type: 'createdAt',
index: 'a0000004',
},
{
id: 'updatedAt',
type: 'updatedAt',
index: 'a0000005',
},
{
id: 'createdBy',
type: 'createdBy',
show: 'always-hide',
index: 'a0000006',
},
{
id: 'edgelessTheme',
type: 'edgelessTheme',
show: 'always-hide',
index: 'a0000007',
},
{
id: 'pageWidth',
type: 'pageWidth',
show: 'always-hide',
index: 'a0000008',
},
];

77
packages/frontend/core/src/modules/doc/entities/property-list.ts
View File

@@ -1,77 +0,0 @@
import {
Entity,
generateFractionalIndexingKeyBetween,
LiveData,
} from '@toeverything/infra';
import type { DocCustomPropertyInfo } from '../../db/schema/schema';
import type { DocPropertiesStore } from '../stores/doc-properties';
export class DocPropertyList extends Entity {
constructor(private readonly docPropertiesStore: DocPropertiesStore) {
super();
}
properties$ = LiveData.from(
this.docPropertiesStore.watchDocPropertyInfoList(),
[]
);
sortedProperties$ = this.properties$.map(list =>
// default index key is '', so always before any others
list.toSorted((a, b) => ((a.index ?? '') > (b.index ?? '') ? 1 : -1))
);
propertyInfo$(id: string) {
return this.properties$.map(list => list.find(info => info.id === id));
}
updatePropertyInfo(id: string, properties: Partial<DocCustomPropertyInfo>) {
this.docPropertiesStore.updateDocPropertyInfo(id, properties);
}
createProperty(
properties: Omit<DocCustomPropertyInfo, 'id'> & { id?: string }
) {
return this.docPropertiesStore.createDocPropertyInfo(properties);
}
removeProperty(id: string) {
this.docPropertiesStore.removeDocPropertyInfo(id);
}
indexAt(at: 'before' | 'after', targetId?: string) {
const sortedChildren = this.sortedProperties$.value.filter(
node => node.index
) as (DocCustomPropertyInfo & { index: string })[];
const targetIndex = targetId
? sortedChildren.findIndex(node => node.id === targetId)
: -1;
if (targetIndex === -1) {
if (at === 'before') {
const first = sortedChildren.at(0);
return generateFractionalIndexingKeyBetween(null, first?.index ?? null);
} else {
const last = sortedChildren.at(-1);
return generateFractionalIndexingKeyBetween(last?.index ?? null, null);
}
} else {
const target = sortedChildren[targetIndex];
const before: DocCustomPropertyInfo | null =
sortedChildren[targetIndex - 1] || null;
const after: DocCustomPropertyInfo | null =
sortedChildren[targetIndex + 1] || null;
if (at === 'before') {
return generateFractionalIndexingKeyBetween(
before?.index ?? null,
target.index
);
} else {
return generateFractionalIndexingKeyBetween(
target.index,
after?.index ?? null
);
}
}
}
}

2
packages/frontend/core/src/modules/doc/index.ts
View File

@@ -11,7 +11,6 @@ import type { Framework } from '@toeverything/infra';
import { WorkspaceDBService } from '../db/services/db';
import { WorkspaceScope, WorkspaceService } from '../workspace';
import { Doc } from './entities/doc';
import { DocPropertyList } from './entities/property-list';
import { DocRecord } from './entities/record';
import { DocRecordList } from './entities/record-list';
import { DocCreateMiddleware } from './providers/doc-create-middleware';
@@ -35,7 +34,6 @@ export function configureDocModule(framework: Framework) {
.store(DocsStore, [WorkspaceService, DocPropertiesStore])
.entity(DocRecord, [DocsStore, DocPropertiesStore])
.entity(DocRecordList, [DocsStore])
.entity(DocPropertyList, [DocPropertiesStore])
.scope(DocScope)
.entity(Doc, [DocScope, DocsStore, WorkspaceService])
.service(DocService);

57
packages/frontend/core/src/modules/doc/services/docs.ts
View File

@@ -4,15 +4,12 @@ import { replaceIdMiddleware } from '@blocksuite/affine/shared/adapters';
import type { AffineTextAttributes } from '@blocksuite/affine/shared/types';
import type { DeltaInsert } from '@blocksuite/affine/store';
import { Slice, Text, Transformer } from '@blocksuite/affine/store';
import { LiveData, ObjectPool, Service } from '@toeverything/infra';
import { omitBy } from 'lodash-es';
import { ObjectPool, Service } from '@toeverything/infra';
import { combineLatest, map } from 'rxjs';
import { initDocFromProps } from '../../../blocksuite/initialization';
import type { DocProperties } from '../../db';
import { getAFFiNEWorkspaceSchema } from '../../workspace';
import type { Doc } from '../entities/doc';
import { DocPropertyList } from '../entities/property-list';
import { DocRecordList } from '../entities/record-list';
import { DocCreated, DocInitialized } from '../events';
import type { DocCreateMiddleware } from '../providers/doc-create-middleware';
@@ -33,26 +30,44 @@ export class DocsService extends Service {
},
});
propertyList = this.framework.createEntity(DocPropertyList);
/**
* Get all property values of a property, used for search
*
* Results may include docs in trash or deleted docs
* Legacy property data such as old `journal` will not be included in the values
*/
propertyValues$(propertyKey: string) {
return combineLatest([
this.store.watchDocIds(),
this.docPropertiesStore.watchPropertyAllValues(propertyKey),
]).pipe(
map(([docIds, propertyValues]) => {
const result = new Map<string, string | undefined>();
for (const docId of docIds) {
result.set(docId, propertyValues.get(docId));
}
return result;
})
);
}
/**
* used for search doc by properties, for convenience of search, all non-exist doc or trash doc have been filtered
* used for search
*/
allDocProperties$: LiveData<Record<string, DocProperties>> = LiveData.from(
combineLatest([
this.docPropertiesStore.watchAllDocProperties(),
this.store.watchNonTrashDocIds(),
]).pipe(
map(([properties, docIds]) => {
const allIds = new Set(docIds);
return omitBy(
properties as Record<string, DocProperties>,
(_, id) => !allIds.has(id)
);
})
),
{}
);
allDocsCreatedDate$() {
return this.store.watchAllDocCreateDate();
}
/**
* used for search
*/
allDocsUpdatedDate$() {
return this.store.watchAllDocUpdatedDate();
}
allDocsTagIds$() {
return this.store.watchAllDocTagIds();
}
constructor(
private readonly store: DocsStore,

227
packages/frontend/core/src/modules/doc/stores/doc-properties.ts
View File

@@ -1,33 +1,22 @@
import { Store, yjsObserveByPath, yjsObserveDeep } from '@toeverything/infra';
import { differenceBy, isNil, omitBy } from 'lodash-es';
import {
LiveData,
Store,
yjsGetPath,
yjsObserveDeep,
} from '@toeverything/infra';
import { isNil, omitBy } from 'lodash-es';
import { combineLatest, map, switchMap } from 'rxjs';
import { AbstractType as YAbstractType } from 'yjs';
import type { WorkspaceDBService } from '../../db';
import type {
DocCustomPropertyInfo,
DocProperties,
} from '../../db/schema/schema';
import type { DocProperties } from '../../db/schema/schema';
import type { WorkspaceService } from '../../workspace';
import { BUILT_IN_CUSTOM_PROPERTY_TYPE } from '../constants';
interface LegacyDocProperties {
custom?: Record<string, { value: unknown } | undefined>;
system?: Record<string, { value: unknown } | undefined>;
}
type LegacyDocPropertyInfo = {
id?: string;
name?: string;
type?: string;
icon?: string;
};
type LegacyDocPropertyInfoList = Record<
string,
LegacyDocPropertyInfo | undefined
>;
export class DocPropertiesStore extends Store {
constructor(
private readonly workspaceService: WorkspaceService,
@@ -43,92 +32,6 @@ export class DocPropertiesStore extends Store {
});
}
getDocPropertyInfoList() {
const db = this.dbService.db.docCustomPropertyInfo.find();
const legacy = this.upgradeLegacyDocPropertyInfoList(
this.getLegacyDocPropertyInfoList()
);
const builtIn = BUILT_IN_CUSTOM_PROPERTY_TYPE;
const withLegacy = [...db, ...differenceBy(legacy, db, i => i.id)];
const all = [
...withLegacy,
...differenceBy(builtIn, withLegacy, i => i.id),
];
return all.filter(i => !i.isDeleted);
}
createDocPropertyInfo(
config: Omit<DocCustomPropertyInfo, 'id'> & { id?: string }
) {
return this.dbService.db.docCustomPropertyInfo.create(config);
}
removeDocPropertyInfo(id: string) {
this.updateDocPropertyInfo(id, {
additionalData: {}, // also remove additional data to reduce size
isDeleted: true,
});
}
updateDocPropertyInfo(id: string, config: Partial<DocCustomPropertyInfo>) {
const needMigration = !this.dbService.db.docCustomPropertyInfo.get(id);
const isBuiltIn =
needMigration && BUILT_IN_CUSTOM_PROPERTY_TYPE.some(i => i.id === id);
if (isBuiltIn) {
this.createPropertyFromBuiltIn(id, config);
} else if (needMigration) {
// if this property is not in db, we need to migration it from legacy to db, only type and name is needed
this.migrateLegacyDocPropertyInfo(id, config);
} else {
this.dbService.db.docCustomPropertyInfo.update(id, config);
}
}
migrateLegacyDocPropertyInfo(
id: string,
override: Partial<DocCustomPropertyInfo>
) {
const legacy = this.getLegacyDocPropertyInfo(id);
this.dbService.db.docCustomPropertyInfo.create({
id,
type:
legacy?.type ??
'unknown' /* should never reach here, just for safety, we need handle unknown property type */,
name: legacy?.name,
...override,
});
}
createPropertyFromBuiltIn(
id: string,
override: Partial<DocCustomPropertyInfo>
) {
const builtIn = BUILT_IN_CUSTOM_PROPERTY_TYPE.find(i => i.id === id);
if (!builtIn) {
return;
}
this.createDocPropertyInfo({ ...builtIn, ...override });
}
watchDocPropertyInfoList() {
return combineLatest([
this.watchLegacyDocPropertyInfoList().pipe(
map(this.upgradeLegacyDocPropertyInfoList)
),
this.dbService.db.docCustomPropertyInfo.find$(),
]).pipe(
map(([legacy, db]) => {
const builtIn = BUILT_IN_CUSTOM_PROPERTY_TYPE;
const withLegacy = [...db, ...differenceBy(legacy, db, i => i.id)];
const all = [
...withLegacy,
...differenceBy(builtIn, withLegacy, i => i.id),
];
return all.filter(i => !i.isDeleted);
})
);
}
getDocProperties(id: string) {
return {
...this.upgradeLegacyDocProperties(this.getLegacyDocProperties(id)),
@@ -137,28 +40,6 @@ export class DocPropertiesStore extends Store {
};
}
watchAllDocProperties() {
const allDocProperties$ = this.dbService.db.docProperties.find$();
const allLegacyDocProperties$ = this.watchAllLegacyDocProperties();
return combineLatest([allDocProperties$, allLegacyDocProperties$]).pipe(
map(([db, legacy]) => {
const map = new Map(db.map(i => [i.id, i]));
const allIds = new Set([...map.keys(), ...Object.keys(legacy ?? {})]);
const result = {} as Record<string, Record<string, any>>;
for (const id of allIds) {
result[id] = {
...this.upgradeLegacyDocProperties(legacy?.[id]),
...omitBy(map.get(id), isNil),
};
}
return result;
})
);
}
watchDocProperties(id: string) {
return combineLatest([
this.watchLegacyDocProperties(id).pipe(
@@ -176,6 +57,20 @@ export class DocPropertiesStore extends Store {
);
}
/**
* find doc ids by property key and value
*
* this apis will not include legacy properties
*/
watchPropertyAllValues(propertyKey: string) {
return LiveData.from<Map<string, string | undefined>>(
this.dbService.db.docProperties
.select$(propertyKey)
.pipe(map(o => new Map(o.map(i => [i.id, i[propertyKey]])))),
new Map()
);
}
private upgradeLegacyDocProperties(properties?: LegacyDocProperties) {
if (!properties) {
return {};
@@ -194,29 +89,6 @@ export class DocPropertiesStore extends Store {
return newProperties;
}
private upgradeLegacyDocPropertyInfoList(
infoList?: LegacyDocPropertyInfoList
) {
if (!infoList) {
return [];
}
const newInfoList: DocCustomPropertyInfo[] = [];
for (const [id, info] of Object.entries(infoList ?? {})) {
if (info?.type) {
newInfoList.push({
id,
name: info.name,
type: info.type,
icon: info.icon,
});
}
}
return newInfoList;
}
private getLegacyDocProperties(id: string) {
return this.workspaceService.workspace.rootYDoc
.getMap<any>('affine:workspace-properties')
@@ -225,25 +97,8 @@ export class DocPropertiesStore extends Store {
?.toJSON() as LegacyDocProperties | undefined;
}
private watchAllLegacyDocProperties() {
return yjsObserveByPath(
this.workspaceService.workspace.rootYDoc.getMap<any>(
'affine:workspace-properties'
),
`pageProperties`
).pipe(
switchMap(yjsObserveDeep),
map(
p =>
(p instanceof YAbstractType ? p.toJSON() : p) as
| { [docId: string]: LegacyDocProperties }
| undefined
)
);
}
private watchLegacyDocProperties(id: string) {
return yjsObserveByPath(
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap<any>(
'affine:workspace-properties'
),
@@ -258,40 +113,4 @@ export class DocPropertiesStore extends Store {
)
);
}
private getLegacyDocPropertyInfoList() {
return this.workspaceService.workspace.rootYDoc
.getMap<any>('affine:workspace-properties')
.get('schema')
?.get('pageProperties')
?.get('custom')
?.toJSON() as LegacyDocPropertyInfoList | undefined;
}
private watchLegacyDocPropertyInfoList() {
return yjsObserveByPath(
this.workspaceService.workspace.rootYDoc.getMap<any>(
'affine:workspace-properties'
),
'schema.pageProperties.custom'
).pipe(
switchMap(yjsObserveDeep),
map(
p =>
(p instanceof YAbstractType ? p.toJSON() : p) as
| LegacyDocPropertyInfoList
| undefined
)
);
}
private getLegacyDocPropertyInfo(id: string) {
return this.workspaceService.workspace.rootYDoc
.getMap<any>('affine:workspace-properties')
.get('schema')
?.get('pageProperties')
?.get('custom')
?.get(id)
?.toJSON() as LegacyDocPropertyInfo | undefined;
}
}

68
packages/frontend/core/src/modules/doc/stores/docs.ts
View File

@@ -2,9 +2,10 @@ import type { DocMode } from '@blocksuite/affine/model';
import type { DocMeta } from '@blocksuite/affine/store';
import {
Store,
yjsGetPath,
yjsObserve,
yjsObserveByPath,
yjsObserveDeep,
yjsObservePath,
} from '@toeverything/infra';
import { nanoid } from 'nanoid';
import { distinctUntilChanged, map, switchMap } from 'rxjs';
@@ -63,7 +64,7 @@ export class DocsStore extends Store {
}
watchDocIds() {
return yjsObserveByPath(
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'pages'
).pipe(
@@ -78,8 +79,65 @@ export class DocsStore extends Store {
);
}
watchAllDocUpdatedDate() {
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'pages'
).pipe(
switchMap(pages => yjsObservePath(pages, '*.updatedDate')),
map(pages => {
if (pages instanceof YArray) {
return pages.map(v => ({
id: v.get('id') as string,
updatedDate: v.get('updatedDate') as number | undefined,
}));
} else {
return [];
}
})
);
}
watchAllDocTagIds() {
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'pages'
).pipe(
switchMap(pages => yjsObservePath(pages, '*.tags')),
map(pages => {
if (pages instanceof YArray) {
return pages.map(v => ({
id: v.get('id') as string,
tags: (v.get('tags')?.toJSON() ?? []) as string[],
}));
} else {
return [];
}
})
);
}
watchAllDocCreateDate() {
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'pages'
).pipe(
switchMap(pages => yjsObservePath(pages, '*.createDate')),
map(pages => {
if (pages instanceof YArray) {
return pages.map(v => ({
id: v.get('id') as string,
createDate: (v.get('createDate') ?? 0) as number,
}));
} else {
return [];
}
})
);
}
watchNonTrashDocIds() {
return yjsObserveByPath(
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'pages'
).pipe(
@@ -97,7 +155,7 @@ export class DocsStore extends Store {
}
watchTrashDocIds() {
return yjsObserveByPath(
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'pages'
).pipe(
@@ -116,7 +174,7 @@ export class DocsStore extends Store {
watchDocMeta(id: string) {
let docMetaIndexCache = -1;
return yjsObserveByPath(
return yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'pages'
).pipe(