Brooooooklyn
c7ddd679fd
feat(server): use native tokenizer impl (#6960)
### Benchmark
`yarn workspace @affine/server-native bench`
```
┌─────────┬────────────┬─────────┬────────────────────┬──────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ns) │ Margin │ Samples │
├─────────┼────────────┼─────────┼────────────────────┼──────────┼─────────┤
│ 0 │ 'tiktoken' │ '5' │ 176932518.76000002 │ '±4.71%' │ 100 │
│ 1 │ 'native' │ '16' │ 61041597.51000003 │ '±0.60%' │ 100 │
└─────────┴────────────┴─────────┴────────────────────┴──────────┴─────────┘
```
2024-05-16 07:55:10 +00:00
..
2024-04-30 15:19:30 +00:00
2024-05-14 09:32:29 +00:00
2024-05-15 14:03:09 +00:00
2024-05-16 07:55:10 +00:00
2024-01-31 08:43:03 +00:00
2024-05-16 07:55:10 +00:00
2024-05-06 06:59:50 +00:00
2024-04-17 18:29:04 +08:00
2024-04-17 16:32:26 +08:00
2024-04-17 16:32:26 +08:00
2024-02-02 08:32:07 +00:00
2024-05-16 07:55:10 +00:00
2024-03-12 06:41:42 +00:00
2024-05-06 12:42:12 +00:00