Commit graph

76 commits

Author SHA1 Message Date
amblued
fe9a9bdf64
refactor(reaction): link reactions to memo IDs (#6221) 2026-08-22 00:12:35 +08:00
amblued
38412eb75a
fix(storage): proxy S3 attachments through authenticated routes (#6210) 2026-08-18 23:38:35 +08:00
Wally
f954434920
fix(store): begin sqlite memo mutations with BEGIN IMMEDIATE (#6192) 2026-08-16 22:18:08 +08:00
Johnny
7d971b1c61
feat(storage): add named attachment storage drivers (#6184) 2026-08-13 22:37:11 +08:00
Johnny
da81b6b48f
feat: persist and display client media metadata (#6180) 2026-08-11 23:47:21 +08:00
Johnny
bd636a4365
feat: support memo-scoped Markdown attachment images (#6169) 2026-08-09 23:27:18 +08:00
Johnny
4e8b262d6d
refactor(memo-views): replace shortcuts with saved views (#6167) 2026-08-09 20:27:27 +08:00
Johnny
8648b97b9e
feat(user): define username format and mention syntax (#6134) 2026-08-02 19:57:16 +08:00
Johnny
c68e3d5bf9
feat(markdown): unify tag syntax and recognition (#6132) 2026-08-02 14:46:04 +08:00
johnnyjoygh
1236d9ba94 test(ci): add upgrade and startup smoke coverage
Exercise real server startup on backend test runs and gate releases on fresh-install, previous-stable upgrade, and entrypoint smoke checks across supported databases.
2026-07-26 21:21:15 +08:00
Johnny
019f4f9adc
fix(auth): provision SSO users atomically (#6114) 2026-07-25 09:47:16 +08:00
johnnyjoygh
715306ea66 chore: enrich access tokens setting page
Settings drops the all-in-one bordered card for a de-carded layout in
the property-rail design language: a sticky table-of-contents rail at
md+ (Settings wordmark, uppercase group labels, quiet anchor rows) and
a horizontally swipeable chip strip below md, replacing the mobile
section dropdown. Nav items are real anchors with aria-current, and
switching sections scrolls back to the top.

Access Tokens becomes a first-class section with an explainer panel:
what a PAT is and a copyable curl example (real instance origin,
memos_pat_ prefix) beside token-safety guidelines in a two-column band,
with a Learn more docs link and the tokens table beneath. Successful
PAT authentication now records the token's lastUsedAt asynchronously
inside resolveBearer, with a clone-before-mutate cache guard and
monotonic writes in the store, surfaced in a Last used column.

Also localizes the create dialog's 90 Days label, lets the My Account
row wrap instead of clipping on narrow screens, and drops the dead
select-section key from all locales.
2026-07-18 10:41:53 +08:00
boojack
0038295bbc feat(config): provision settings from secret files
- Load IdPs and supported instance-setting groups as runtime overlays from /etc/secrets.
- Reject API mutations of deployment-managed resources and serialize authentication safety checks across database drivers.
- Preserve upgrade compatibility, demo SSO policy, stable IdP ordering, and driver-specific transaction retries.
2026-07-13 22:34:24 +08:00
boojack
26f4b73cb9 feat(filter): standard CEL now variable, time accessors, set ops
Replace the custom now() function with an idiomatic `now` timestamp variable (host-injected, frozen once per compile) and retype created_ts/updated_ts/create_time to CEL timestamp. Filters now use standard timestamp/duration arithmetic, e.g. `created_ts >= now - duration("24h")` and `timestamp("2025-01-01T00:00:00Z")`.

Add standard CEL surface that compiles to SQL across SQLite/MySQL/Postgres: timestamp accessors (getFullYear/getMonth/getDate/getDayOfWeek/..., with 0-based month and weekday normalized), ext.Sets() (sets.contains/intersects/equivalent over tags), tags.exists_one(), size() on string fields, and division/modulo folding. A frozen clock is injectable for deterministic tests.

BREAKING CHANGE: now() is removed (use the `now` variable) and time fields are timestamps, so bare-epoch comparisons need timestamp(<epoch>). Existing saved shortcuts using the old syntax must be updated.
2026-06-22 22:42:44 +08:00
boojack
f0e4a5624f feat(filter): expand CEL filter surface with startsWith/endsWith, matches(), and all()
Let users write three more CEL constructs in the filter field, each compiled to
SQL across SQLite/MySQL/Postgres:

- Scalar startsWith()/endsWith() on content/filename/mime_type (case-insensitive)
- matches() regex: PG ~, MySQL/SQLite REGEXP (Go-backed SQLite fn), validated at
  compile time via cel.ValidateRegexLiterals()
- all() comprehension over tags via per-element subqueries, non-empty required

Also: contains() now escapes LIKE metacharacters (%, _, \); cross-dialect render
tests plus behavioral tests; cel-go bumped to v0.28.1; new operators surfaced in
the frontend shortcut guide.
2026-06-15 23:22:28 +08:00
boojack
9eabb554d5
feat(settings): move tag metadata to user settings (#6017) 2026-06-07 23:58:00 +08:00
boojack
e53b7d96e7
fix: delete user cleanup (#5981) 2026-05-25 22:10:29 +08:00
boojack
02096836c3 test: stabilize backend tests in CI
- Avoid requiring built frontend assets in cache header tests.
- Skip Testcontainers-backed store tests when Docker is unavailable.
2026-05-08 22:24:54 +08:00
Steven
ea0625da45 feat(stats): admin instance resource statistics 2026-05-01 23:15:56 +08:00
boojack
616487fa37
chore: update backend and frontend dependencies (#5900)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-04-28 08:25:26 +08:00
boojack
d688914b28
feat(auth): add SSO user identity linkage (#5883) 2026-04-23 08:51:45 +08:00
boojack
101704c8ea
feat(ai): add BYOK audio transcription (#5832) 2026-04-13 22:09:24 +08:00
memoclaw
83ed32f119
feat(ai): add instance AI providers and transcription (#5829)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-04-12 19:23:34 +08:00
memoclaw
1921b57662
fix(tags): allow blur-only tag metadata (#5800)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-31 21:38:55 +08:00
boojack
0e89407ee9 fix(filter): enforce CEL syntax semantics
Reject non-standard truthy numeric expressions in filters and document the parser as a supported subset of standard CEL syntax.

- remove legacy filter rewrites
- support standard equality in tag exists predicates
- add regression coverage for accepted and rejected expressions
2026-03-31 08:10:49 +08:00
boojack
7c708ee27e
chore: add migration upgrade coverage (#5796) 2026-03-30 23:51:57 +08:00
memoclaw
4add9b04ad fix: prevent local attachment uploads from overwriting files 2026-03-26 21:46:51 +08:00
memoclaw
acddef1f3d
fix(api): switch user resource names to usernames (#5779)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-25 09:11:17 +08:00
memoclaw
4818bf3559
refactor(store): remove synthetic system bot user lookup (#5778)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-24 22:16:30 +08:00
google-labs-jules[bot]
e0cc247823
chore: optimize multi-user RSS feed generation by fixing N+1 query (#5749)
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
2026-03-20 18:09:24 +08:00
memoclaw
e164517773 test: remove raw SQL migration backfill test setup 2026-03-19 19:54:39 +08:00
memoclaw
f759b416af
refactor(inbox): store memo comment payloads without activity records (#5741)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-19 19:33:25 +08:00
memoclaw
a249d06e2e
feat(instance): add notification transport setting (#5737)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-03-19 09:18:28 +08:00
boojack
65d14fbb63
feat(instance): add canonical tag metadata setting (#5736) 2026-03-19 08:54:43 +08:00
boojack
330291d4d9
feat(user): add per-user tag metadata settings (#5735) 2026-03-18 23:15:14 +08:00
memoclaw
1e82714a52
perf: batch load memo relations when listing memos (#5692)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 11:19:19 +08:00
memoclaw
92d937b1aa
feat: replace auto-increment ID with UID for identity provider resource names (#5687)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 21:01:22 +08:00
Johnny
78efa6802e feat(store): change default storage type to local filesystem
Add migration scripts for existing instances without a storage setting
to explicitly preserve DATABASE as their storage type, ensuring backward
compatibility. Change the default for new installations to LOCAL to
improve out-of-the-box performance.
2026-03-01 19:42:01 +08:00
Johnny
252b3eff90 test(store): add tests for CreateMemo with custom timestamps 2026-03-01 19:08:27 +08:00
Steven
2b19d8a969 fix(tests): update PAT tests to accept both custom and SQL errors
The PostgreSQL implementation returns 'PAT not found' when no rows match,
while SQLite/MySQL return 'sql: no rows in result set' from QueryRowContext.

Both behaviors are correct - the key fix is that PostgreSQL no longer
throws JSONB errors on missing/malformed data.

Changes:
- Update test assertions to accept either error type
- Fix comment punctuation for godot linter
- Maintain backward compatibility across all database drivers
2026-02-09 21:09:19 +08:00
Steven
d9e8387d63 fix(postgres): handle missing PAT data gracefully and add comprehensive tests
Fixes #5612, #5611

Root cause: PostgreSQL's jsonb_array_elements() throws errors when the
'tokens' key is missing or malformed, while SQLite/MySQL return NULL
gracefully. This caused:
- 502 errors when creating admin after v0.25.3 → v0.26.0 upgrade
- Settings not persisting and users unable to stay logged in

Changes to store/db/postgres/user_setting.go:
- Remove strict JSONB operations from GetUserByPATHash query
- Fetch all PERSONAL_ACCESS_TOKENS rows and filter in Go
- Skip malformed/invalid JSON rows with continue (error recovery)
- Match SQLite/MySQL's forgiving error handling

New integration tests (store/test/user_setting_test.go):
- TestUserSettingGetUserByPATHashNoTokensKey
- TestUserSettingGetUserByPATHashEmptyTokensArray
- TestUserSettingGetUserByPATHashWithOtherUsers

New PostgreSQL-specific tests (store/db/postgres/user_setting_test.go):
- TestGetUserByPATHashWithMissingData (comprehensive edge cases)
- TestGetUserByPATHashPerformance (100+ users)
- TestUpsertUserSetting (basic upsert)

Test coverage:
✅ Missing PERSONAL_ACCESS_TOKENS key
✅ Empty/malformed JSON data
✅ Multiple users with mixed valid/invalid data
✅ Performance with 100+ users
✅ Error recovery without crashes

Benefits:
- No database migration required (TEXT column works fine)
- Backward compatible with v0.25.3 upgrades
- Handles missing/corrupt data gracefully
- Consistent behavior across all database drivers
2026-02-09 21:02:09 +08:00
Steven
8770b186e4 fix: add Unicode case-insensitive search for SQLite (#5559)
Add custom memos_unicode_lower() SQLite function to enable proper
case-insensitive text search for non-English languages (Cyrillic,
Greek, CJK, etc.).

Previously, SQLite's LOWER() only worked for ASCII characters due to
modernc.org/sqlite lacking ICU extension. This caused searches for
non-English text to be case-sensitive (e.g., searching 'блины' wouldn't
find 'Блины').

Changes:
- Add store/db/sqlite/functions.go with Unicode case folding function
- Register custom function using golang.org/x/text/cases.Fold()
- Update filter renderer to use custom function for SQLite dialect
- Add test for Unicode case-insensitive search
- Make golang.org/x/text a direct dependency

Fixes #5559
2026-02-02 21:10:07 +08:00
Johnny
0f3c9a467d refactor: migrate HOST roles to ADMIN
- Updated the isSuperUser function to only check for ADMIN role.
- Added SQL migration scripts for MySQL, PostgreSQL, and SQLite to change user roles from HOST to ADMIN.
- Created a new SQLite migration to alter the user table structure and ensure data integrity during the migration process.
2026-01-20 23:38:30 +08:00
Johnny
552318209b
fix: resolve flaky migration tests and add stable upgrade test (#5514) 2026-01-20 19:25:00 +08:00
Johnny
7089db06c2 test: enhance memo filter tests with COALESCE for JSON extraction and add migration data persistence tests 2026-01-19 23:09:17 +08:00
Johnny
af2a2588bf chore(test): add edge case tests for user settings shortcuts and JSON fields 2026-01-19 22:50:14 +08:00
Johnny
cbf46a2988 feat(filter): add CEL list comprehension support for tag filtering
Add support for CEL exists() comprehension with startsWith, endsWith, and
contains predicates to enable powerful tag filtering patterns.

Features:
- tags.exists(t, t.startsWith("prefix")) - Match tags by prefix
- tags.exists(t, t.endsWith("suffix")) - Match tags by suffix
- tags.exists(t, t.contains("substring")) - Match tags by substring
- Negation: !tags.exists(...) to exclude matching tags
- Works with all operators (AND, OR, NOT) and other filters

Implementation:
- Added ListComprehensionCondition IR type for comprehension expressions
- Parser detects exists() macro and extracts predicates
- Renderer generates optimized SQL for SQLite, MySQL, PostgreSQL
- Proper NULL/empty array handling across all database dialects
- Helper functions reduce code duplication

Design decisions:
- Only exists() supported (all() rejected at parse time with clear error)
- Only simple predicates (matches() excluded to avoid regex complexity)
- Fail-fast validation with helpful error messages

Tests:
- Comprehensive test suite covering all predicates and edge cases
- Tests for NULL/empty arrays, combined filters, negation
- Real-world use case test for Issue #5480 (archive workflow)
- All tests pass on SQLite, MySQL, PostgreSQL

Closes #5480
2026-01-17 12:36:39 +08:00
Johnny
4bd0f29ee5 fix: resolve linter warning and cache race condition
- Suppress revive redundant-test-main-exit warning with //nolint
- Fix data race in cache.Clear() by using Delete() instead of map replacement
- Both changes maintain correct behavior while fixing CI failures
2026-01-14 23:50:37 +08:00
Johnny
db57f4456a fix: resolve data races in store tests with atomic operations
Fix all data races detected by -race flag in parallel store tests.

Problems fixed:
1. TestMain not propagating exit code
   - Was: m.Run(); return
   - Now: os.Exit(m.Run())

2. Data races on global DSN variables
   - mysqlBaseDSN and postgresBaseDSN written in sync.Once
   - Read outside Once without synchronization
   - Race detector: write/read without happens-before relationship

3. Data races on container pointers
   - mysqlContainer, postgresContainer, testDockerNetwork
   - Same pattern: write in Once, read in cleanup

Solution: Use atomic operations
- atomic.Value for DSN strings (Store/Load)
- atomic.Pointer for container pointers (Store/Load)
- Provides proper memory synchronization
- Race-free reads and writes

Why sync.Once alone wasn't enough:
- sync.Once guarantees function runs once
- Does NOT provide memory barrier for variables written inside
- Reads outside Once have no synchronization with writes
- Race detector correctly flags this as violation

Technical details:
- atomic.Value.Store() provides release semantics
- atomic.Value.Load() provides acquire semantics
- Guarantees happens-before relationship per Go memory model
- All 159 parallel tests can safely access globals

Impact:
- Tests now pass with -race flag
- No performance degradation (atomic ops are fast)
- Maintains parallel execution benefits (8-10x speedup)
- Proper Go memory model compliance

Related: Issues #2, #3 from race analysis
2026-01-14 23:37:12 +08:00
Johnny
411e8fc5b0 perf: enable parallel execution for all store tests
Add t.Parallel() to all 159 test functions in store/test/

Benefits:
- Tests run in parallel (8-16 concurrent on typical CI)
- Each test already has isolated database (safe to parallelize)
- No shared state between tests
- Go test runner handles synchronization

Expected performance:
- SQLite tests: 4-6min → 30-45sec (87% faster)
- MySQL tests: 6-8min → 45-60sec (88% faster)
- Better CPU utilization (10-15% → 80-95%)

Why it's safe:
- NewTestingStore() creates isolated DB per test
- No global state mutations
- Each test uses t.TempDir() for file isolation
- Container-based tests use unique database names

Impact on CI workflow:
- Backend tests workflow: 4-6min → 1-2min total
- Store test group: 3-4min → 20-30sec
- 8-10x speedup from parallelization alone
2026-01-14 22:44:19 +08:00