Commit graph

399 commits

Author SHA1 Message Date
boojack
0038295bbc feat(config): provision settings from secret files
- Load IdPs and supported instance-setting groups as runtime overlays from /etc/secrets.
- Reject API mutations of deployment-managed resources and serialize authentication safety checks across database drivers.
- Preserve upgrade compatibility, demo SSO policy, stable IdP ordering, and driver-specific transaction retries.
2026-07-13 22:34:24 +08:00
boojack
4bc3928029 fix(user): implement ListUsers pagination
Rework ListUsers to match the ListMemos pagination contract: opaque
PageToken, normalizePageSize, DB-level limit+1 look-ahead, and a
next_page_token. Adds Offset to store.FindUser with an OFFSET clause in
all three dialects, and an `id DESC` ORDER BY tiebreaker so offset pages
stay stable when created_ts ties.

Also align pagination across list endpoints:
- Bump DefaultPageSize 10 -> 50 to match the documented default; use
  normalizePageSize in ListAttachments.
- Remove the never-implemented total_size field from all six list
  responses (ListUsers, ListAttachments, ListMemoComments,
  ListMemoReactions, ListUserSettings, ListPersonalAccessTokens) and
  regenerate.
- useListUsers now pages through next_page_token so the admin members
  view still loads every user past the default page size.
2026-07-12 20:51:04 +08:00
grandpig
76aee4e177
refactor: use the built-in max/min to simplify the code (#6060)
Signed-off-by: grandpig <grandpig@outlook.com>
2026-07-02 08:35:47 +08:00
boojack
f0e4a5624f feat(filter): expand CEL filter surface with startsWith/endsWith, matches(), and all()
Let users write three more CEL constructs in the filter field, each compiled to
SQL across SQLite/MySQL/Postgres:

- Scalar startsWith()/endsWith() on content/filename/mime_type (case-insensitive)
- matches() regex: PG ~, MySQL/SQLite REGEXP (Go-backed SQLite fn), validated at
  compile time via cel.ValidateRegexLiterals()
- all() comprehension over tags via per-element subqueries, non-empty required

Also: contains() now escapes LIKE metacharacters (%, _, \); cross-dialect render
tests plus behavioral tests; cel-go bumped to v0.28.1; new operators surfaced in
the frontend shortcut guide.
2026-06-15 23:22:28 +08:00
boojack
e53b7d96e7
fix: delete user cleanup (#5981) 2026-05-25 22:10:29 +08:00
Steven
ea0625da45 feat(stats): admin instance resource statistics 2026-05-01 23:15:56 +08:00
boojack
ee1799851e
feat: redesign account and SSO management (#5886) 2026-04-24 09:08:58 +08:00
boojack
d688914b28
feat(auth): add SSO user identity linkage (#5883) 2026-04-23 08:51:45 +08:00
boojack
50638040f6
fix: reduce list memo query overhead (#5880) 2026-04-22 09:31:48 +08:00
boojack
c45663761d fix(api): reduce memory pressure in backend paths 2026-04-16 23:08:48 +08:00
memoclaw
24fc8ab8ca
feat(mentions): add memo mention parsing, notifications, and rendering (#5811)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-04-06 22:16:53 +08:00
boojack
10a955fd62 refactor: move plugin packages under internal 2026-04-06 11:10:32 +08:00
boojack
4b4e719470
feat(attachments): add Live Photo and Motion Photo support (#5810) 2026-04-06 10:47:01 +08:00
memoclaw
acddef1f3d
fix(api): switch user resource names to usernames (#5779)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-25 09:11:17 +08:00
google-labs-jules[bot]
e0cc247823
chore: optimize multi-user RSS feed generation by fixing N+1 query (#5749)
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
2026-03-20 18:09:24 +08:00
memoclaw
3f3133d6e2
feat(memo): add share links for private memos (#5742)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-19 23:47:22 +08:00
memoclaw
f759b416af
refactor(inbox): store memo comment payloads without activity records (#5741)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-19 19:33:25 +08:00
memoclaw
8f43e8075b
fix: correct typos in comments, error messages, and identifiers (#5704)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 18:40:17 +08:00
memoclaw
1e82714a52
perf: batch load memo relations when listing memos (#5692)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 11:19:19 +08:00
memoclaw
92d937b1aa
feat: replace auto-increment ID with UID for identity provider resource names (#5687)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 21:01:22 +08:00
Biplav Barua
def123232a
feat: implement ListActivities pagination (#5468) 2026-03-04 19:08:47 +08:00
Mudkip
09d73e8b6e
fix(mysql): handle CreateMemo custom timestamps with FROM_UNIXTIME (#5673) 2026-03-01 19:01:54 +08:00
Steven
2b19d8a969 fix(tests): update PAT tests to accept both custom and SQL errors
The PostgreSQL implementation returns 'PAT not found' when no rows match,
while SQLite/MySQL return 'sql: no rows in result set' from QueryRowContext.

Both behaviors are correct - the key fix is that PostgreSQL no longer
throws JSONB errors on missing/malformed data.

Changes:
- Update test assertions to accept either error type
- Fix comment punctuation for godot linter
- Maintain backward compatibility across all database drivers
2026-02-09 21:09:19 +08:00
Steven
d9e8387d63 fix(postgres): handle missing PAT data gracefully and add comprehensive tests
Fixes #5612, #5611

Root cause: PostgreSQL's jsonb_array_elements() throws errors when the
'tokens' key is missing or malformed, while SQLite/MySQL return NULL
gracefully. This caused:
- 502 errors when creating admin after v0.25.3 → v0.26.0 upgrade
- Settings not persisting and users unable to stay logged in

Changes to store/db/postgres/user_setting.go:
- Remove strict JSONB operations from GetUserByPATHash query
- Fetch all PERSONAL_ACCESS_TOKENS rows and filter in Go
- Skip malformed/invalid JSON rows with continue (error recovery)
- Match SQLite/MySQL's forgiving error handling

New integration tests (store/test/user_setting_test.go):
- TestUserSettingGetUserByPATHashNoTokensKey
- TestUserSettingGetUserByPATHashEmptyTokensArray
- TestUserSettingGetUserByPATHashWithOtherUsers

New PostgreSQL-specific tests (store/db/postgres/user_setting_test.go):
- TestGetUserByPATHashWithMissingData (comprehensive edge cases)
- TestGetUserByPATHashPerformance (100+ users)
- TestUpsertUserSetting (basic upsert)

Test coverage:
✅ Missing PERSONAL_ACCESS_TOKENS key
✅ Empty/malformed JSON data
✅ Multiple users with mixed valid/invalid data
✅ Performance with 100+ users
✅ Error recovery without crashes

Benefits:
- No database migration required (TEXT column works fine)
- Backward compatible with v0.25.3 upgrades
- Handles missing/corrupt data gracefully
- Consistent behavior across all database drivers
2026-02-09 21:02:09 +08:00
Steven
8770b186e4 fix: add Unicode case-insensitive search for SQLite (#5559)
Add custom memos_unicode_lower() SQLite function to enable proper
case-insensitive text search for non-English languages (Cyrillic,
Greek, CJK, etc.).

Previously, SQLite's LOWER() only worked for ASCII characters due to
modernc.org/sqlite lacking ICU extension. This caused searches for
non-English text to be case-sensitive (e.g., searching 'блины' wouldn't
find 'Блины').

Changes:
- Add store/db/sqlite/functions.go with Unicode case folding function
- Register custom function using golang.org/x/text/cases.Fold()
- Update filter renderer to use custom function for SQLite dialect
- Add test for Unicode case-insensitive search
- Make golang.org/x/text a direct dependency

Fixes #5559
2026-02-02 21:10:07 +08:00
Johnny
05f31e457e fix: add mmap size setting to database connection to prevent OOM errors 2026-01-20 21:53:31 +08:00
Steven
00f21b86e2 chore: remove redundant tests 2026-01-20 09:12:36 +08:00
Johnny
7089db06c2 test: enhance memo filter tests with COALESCE for JSON extraction and add migration data persistence tests 2026-01-19 23:09:17 +08:00
Johnny
dc7ec8a8ad feat: allow setting custom timestamps when creating memos and comments
Allow API users to set custom create_time, update_time, and display_time
when creating memos and comments. This enables importing historical data
with accurate timestamps.

Changes:
- Update proto definitions: change create_time and update_time from
  OUTPUT_ONLY to OPTIONAL to allow setting on creation
- Modify CreateMemo service to handle custom timestamps from request
- Update database drivers (SQLite, MySQL, PostgreSQL) to support
  inserting custom timestamps when provided
- Add comprehensive test coverage for custom timestamp functionality
- Maintain backward compatibility: auto-generated timestamps still
  work when custom values are not provided
- Fix golangci-lint issues in plugin/filter (godot and revive)

Fixes #5483
2026-01-17 12:56:03 +08:00
Johnny
69b62cccdb test: optimize store tests performance by reusing docker image and reducing build context 2026-01-12 23:30:56 +08:00
Johnny
31f634b71a chore: add more tests 2026-01-12 22:28:42 +08:00
Johnny
cc9a214be8 revert: revert system_setting to instance_setting rename changes
Reverts only the system_setting → instance_setting rename related changes from commit d326c71.
Keeps the resource → attachment rename changes intact.

- Reverts table name back to system_setting in all database drivers (MySQL, PostgreSQL, SQLite)
- Removes migration files for the system_setting rename
- Reverts LATEST.sql files to use system_setting table
2026-01-07 20:38:02 +08:00
Johnny
d326c71078 refactor(db): rename tables for clarity - resource→attachment, system_setting→instance_setting 2026-01-06 23:36:42 +08:00
Johnny
bd02de9895
chore: add store tests (#5397) 2025-12-31 21:26:35 +08:00
Johnny
78aa41336a feat: implement attachment filtering functionality 2025-12-28 18:47:59 +08:00
Johnny
7932f6d0d0
refactor: user auth improvements (#5360) 2025-12-18 18:15:51 +08:00
Johnny
310590b278 chore: fix golang checks 2025-12-16 22:53:36 +08:00
Johnny
40e129b8af refactor(auth): streamline session authentication and cookie handling 2025-12-16 22:23:59 +08:00
Steven
0610257562 refactor(store): remove deprecated migration_history table and backward compatibility code
Complete removal of migration_history system in favor of instance_setting based schema versioning.

Changes:
- Remove migration_history table creation from all LATEST.sql files
- Delete all migration_history model and implementation files (~300 lines)
- Remove FindMigrationHistoryList and UpsertMigrationHistory from Driver interface
- Replace complex backward compatibility functions with simple version check
- Update health check to use instance_setting instead of migration_history
- Simplify checkMinimumUpgradeVersion to detect pre-v0.22 installations

Breaking change:
Users on versions < v0.22.0 (May 2024) must upgrade to v0.25.x first before upgrading to this version.
Clear error message with upgrade instructions will be shown for old installations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 22:54:30 +08:00
Steven
d1492007ab fix(store): filter inbox notifications by message type at database level
Add MessageType filter to FindInbox to exclude legacy VERSION_UPDATE
notifications from inbox queries. This resolves the issue where users
saw notification counts but no items displayed, as VERSION_UPDATE
entries cannot be rendered in the new UserNotification API.

Changes:
- Add MessageType field to FindInbox struct for database-level filtering
- Implement JSON extraction filters in SQLite, MySQL, and PostgreSQL drivers
- Update ListUserNotifications to filter MEMO_COMMENT type at store level

This approach improves performance by filtering at the database rather
than in application code, reducing unnecessary data transfer for users
with many legacy inbox entries.

Fixes #5278

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 08:43:05 +08:00
Steven
07a030ddfd fix(postgres): update tag filtering SQL to ensure proper type casting for LIKE comparisons 2025-11-26 23:04:07 +08:00
Steven
424f11f227 fix(store): fix PostgreSQL tag filtering type inference error
Resolves issue where tag filtering in PostgreSQL databases failed with "operator does not exist: jsonb ~~ unknown" error. The hierarchical tag filtering feature introduced in commit 5e47f25b generated SQL with implicit type placeholders that PostgreSQL couldn't infer.

The fix explicitly casts the LIKE comparison placeholder to text (::text) in the PostgreSQL dialect, ensuring proper type resolution for the query parameter.

Fixes #5275

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 07:40:39 +08:00
boojack
21d31e3609
fix(security): implement security review recommendations (#5228)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-06 23:32:27 +08:00
Steven
4c1d1c70d1 refactor: rename workspace to instance throughout codebase
Remove work-related terminology by renaming "workspace" to "instance"
across the entire application. This change better reflects that Memos
is a self-hosted tool suitable for personal and non-work use cases.

Breaking Changes:
- API endpoints: /api/v1/workspace/* → /api/v1/instance/*
- gRPC service: WorkspaceService → InstanceService
- Proto types: WorkspaceSetting → InstanceSetting
- Frontend translation keys: workspace-section → instance-section

Backend Changes:
- Renamed proto definitions and regenerated code
- Updated all store layer methods and database drivers
- Renamed service implementations and API handlers
- Updated cache from workspaceSettingCache to instanceSettingCache

Frontend Changes:
- Renamed service client: workspaceServiceClient → instanceServiceClient
- Updated all React components and state management
- Refactored stores: workspace.ts → instance.ts
- Updated all 32 locale translation files

All tests pass and both backend and frontend build successfully.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 23:35:35 +08:00
Steven
5e47f25bf5 feat(store): add hierarchical tag filtering support
Tag filters now support hierarchical matching where searching for a tag (e.g., "book") will match both the exact tag and any tags with that prefix (e.g., "book/fiction", "book/non-fiction"). This applies across all database backends (SQLite, MySQL, PostgreSQL) with corresponding test updates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 00:21:53 +08:00
Steven
e0b1153269 fix(web): resolve MobX observable reactivity issue in filter computation
Fixes filtering functionality that was broken due to improper use of
useMemo with MobX observables. The issue occurred because useMemo's
dependency array uses reference equality, but MobX observable arrays
are mutated in place (reference doesn't change when items are added/removed).

Changes:
- Remove useMemo from filter computation in Home, UserProfile, and Archived pages
- Calculate filters directly in render since components are already MobX observers
- Fix typo: memoFitler -> memoFilter in Archived.tsx

This ensures filters are recalculated whenever memoFilterStore.filters changes,
making tag clicks and other filter interactions work correctly.

Fixes #5189

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 06:59:13 +08:00
Steven
46ce0bc62e fix(store): correct PostgreSQL placeholder generation in IN clauses
Fixes a regression introduced in v0.25.2 where PostgreSQL IN clause
placeholders were not properly incremented, causing all parameters to
use the same placeholder index (e.g., $1, $1, $1 instead of $1, $2, $3).

This bug affected:
- ListReactions (ContentIDList) - caused "failed to list reactions" errors
- ListAttachments (MemoIDList)
- ListMemos (IDList and UIDList)

The fix combines placeholder generation and argument appending into a
single loop to ensure proper incrementing.

Fixes #5188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 06:47:06 +08:00
Claude
e35f16306e feat: add id as final tie-breaker in ListMemos 2025-10-23 19:44:41 +08:00
Steven
b4ea7d843f feat: enhance memo sorting functionality to support multiple fields 2025-10-20 23:41:58 +08:00
Steven
95de5cc700 refactor: update migration history methods 2025-10-20 23:05:50 +08:00