feat(transcription): explicit STT settings with provider, model, prompt (#5926)

This commit is contained in:
boojack 2026-05-02 19:35:18 +08:00 committed by GitHub
parent ef55013418
commit 238f27dea1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 2962 additions and 389 deletions

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,155 @@
# Transcription (STT) settings — design
**Date:** 2026-05-02
**Scope:** Backend + frontend. Schema-additive (no migration required).
## Problem
Memos has one AI feature today: audio transcription (speech-to-text). The current design has three concrete problems:
1. **Model is hard-coded per provider type.** `internal/ai/models.go` pins OpenAI to `gpt-4o-transcribe` and Gemini to `gemini-2.5-flash`. Users who want `whisper-1` (cheaper, often more accurate for non-English) or third-party Whisper-compatible endpoints (Groq's `whisper-large-v3-turbo`, self-hosted whisper.cpp / Speaches via OpenAI-compatible URL) cannot configure them at all.
2. **No explicit transcription configuration.** `InstanceAISetting.providers` is a generic credentials list. The frontend (`MemoEditor/index.tsx:65`) implicitly picks "the first provider with an API key whose type is in TRANSCRIPTION_PROVIDER_TYPES." Users cannot:
- Choose which provider runs transcription when they have multiple.
- Set a default language (Whisper API supports it but it is never sent).
- Set a `prompt` hint to bias spelling of proper nouns / jargon (a documented Whisper feature, surfaced by every other STT product).
3. **Gemini fails for browser-recorded audio.** `internal/ai/gemini.go:23` does not list `audio/webm` in `geminiSupportedContentTypes`, but `MediaRecorder` in browsers defaults to `audio/webm`. So selecting a Gemini provider for in-editor recording produces a content-type error every time.
## Goal
Let the operator configure transcription explicitly: which provider, which model, default language, and a spelling-hint prompt. Make the OpenAI provider work as a universal "OpenAI-compatible" engine so Groq / self-hosted Whisper / Speaches are reachable through endpoint override.
## Non-goals
- Adding STT engines beyond OpenAI and Gemini (Azure, Deepgram, AWS Transcribe — out of scope; the schema admits them later via `AIProviderType` enum).
- Other AI features (summarization, embeddings, tag suggestion). The schema is shaped so they fit later, but none are designed here.
- Per-call provider override at recording time. Research across all surveyed products (OpenWebUI, LibreChat, Whisper Memos, Superwhisper, etc.) confirms STT engine is a global preference, not an action-time choice. We follow the same pattern.
- Server-side audio transcoding (e.g., webm → wav for Gemini). See "Gemini webm" below for the chosen mitigation.
- Multi-user or per-user override of admin defaults. Memos' STT setting is instance-scoped, like every other instance setting.
## Naming
Field and message names follow cross-platform STT conventions, not Memos-internal shorthand:
| Concept | Chosen name | Rationale |
|---|---|---|
| Config message | `TranscriptionConfig` | AssemblyAI uses this exact identifier; matches OpenAI's `CreateTranscription*` verb family and Memos' existing `Transcribe` RPC. The `STT` acronym is not used as a type name in any major STT API. |
| Provider reference | `provider_id` (string) | Plain protobuf convention for a string-ID reference (`field_id`, `user_id` style). `engine` was rejected as an OpenWebUI-only term; typed message refs are not needed since providers are addressed by string ID. |
| Model | `model` | Unanimous across OpenAI, Google v2, Deepgram, OpenWebUI, LibreChat. Not `model_id`. |
| Default language | `language` | Bare `language` is the modern convention (OpenAI, Whisper family, Deepgram, Wyoming). `language_code` is the older Google/AWS form; we accept ISO 639-1 short codes the same way OpenAI does. |
| Spelling hint | `prompt` | OpenAI's public API field name and AssemblyAI's. Whisper's internal name is `initial_prompt`, but `prompt` is what users of `audio.transcriptions.create` recognize. |
A note on the message name collision: `proto/api/v1/ai_service.proto` already declares a `TranscriptionConfig` for **per-call** prompt/language overrides. The new store-level `TranscriptionConfig` lives in package `memos.store`, so the two compile cleanly. Memos already uses parallel `api.v1.X` / `store.X` message pairs (e.g. `User`, `Memo`); this matches that pattern.
## Architecture
### Schema (additive)
`proto/store/instance_setting.proto`:
```proto
message InstanceAISetting {
repeated AIProviderConfig providers = 1; // unchanged — credential pool
TranscriptionConfig transcription = 2; // NEW — feature config
}
message TranscriptionConfig {
// References an entry in providers[].id. Empty string = transcription disabled.
string provider_id = 1;
// Free text. Empty string = engine default (whisper-1 for OPENAI, gemini-2.5-flash for GEMINI).
string model = 2;
// ISO 639-1 short code. Empty string = auto-detect.
string language = 3;
// Up to ~200 tokens. Used as the OpenAI Whisper `prompt` parameter and as
// a "Context and spelling hints:" block in the Gemini prompt.
string prompt = 4;
}
```
`proto/api/v1/ai_service.proto`:
- `TranscribeRequest.provider_id` becomes optional. When omitted, the server resolves the provider from `InstanceAISetting.transcription.provider_id`.
- `TranscribeRequest.config` (per-call `TranscriptionConfig` with `prompt` / `language`) is kept for advanced overrides but its fields, when empty, fall back to the persisted defaults from `InstanceAISetting.transcription`.
### Backend changes
1. **`internal/ai/models.go`** — `DefaultTranscriptionModel` already exists; reuse it as the fallback when `TranscriptionConfig.model` is empty. No new code, just used from a new call site.
2. **`server/router/api/v1/ai_service.go`**:
- Read `InstanceAISetting.transcription` at the start of `Transcribe`.
- Resolve `provider_id` from request → fall back to `transcription.provider_id`. If both empty, return `FailedPrecondition` with a clear "transcription not configured" message.
- Resolve `model` similarly: request override → `transcription.model` → engine default via `DefaultTranscriptionModel`.
- Merge `language` and `prompt`: per-call overrides win; otherwise fall through to persisted defaults.
3. **`internal/ai/gemini.go`** — out of scope to fix the webm content-type list here. See mitigation below.
### Frontend changes
`web/src/components/Settings/AISection.tsx` is restructured into two settings groups inside the existing `SettingSection`:
1. **AI Integrations** (renamed from "Providers" — current behavior): list of credential entries (id, title, type, endpoint, api key). No functional changes; the rename communicates that this section is just credentials.
2. **Transcription** (new): three-segment form
- **Provider** — Select dropdown listing entries from group 1 by `title`. First option is "None — transcription disabled". Disabled with a hint "Add an AI integration first ↑" when group 1 is empty.
- **Model** — text input. Placeholder updates dynamically based on the selected provider's type (`whisper-1` for OPENAI, `gemini-2.5-flash` for GEMINI). Help text below: "Free text. Use the provider's model identifier — e.g., whisper-1, gpt-4o-transcribe, whisper-large-v3-turbo."
- **Default language** — text input, ISO 639-1 placeholder, empty = auto.
- **Prompt hints** — textarea, ~200 token soft limit, help text "Improves spelling of proper nouns and jargon. Whisper limit is ~224 tokens."
`web/src/components/MemoEditor/index.tsx:65` changes:
- Replace the "first provider with apiKey in TRANSCRIPTION_PROVIDER_TYPES" lookup with this enable rule: transcribe button shows iff `aiSetting.transcription.providerId` is non-empty AND the referenced provider exists in `aiSetting.providers` AND that provider has `apiKeySet === true`.
- The editor no longer needs to know the provider object itself for the call — see service change below.
`web/src/components/MemoEditor/services/transcriptionService.ts` is simplified: it stops accepting a `provider` argument and simply omits `provider_id` from the request. The server resolves the provider, model, language, and prompt from `InstanceAISetting.transcription`. (No override path is exposed at the editor layer; advanced callers can still pass `provider_id` directly via the proto if needed in the future.)
### How "OpenAI-compatible" backends work
To use Groq, Speaches, or self-hosted whisper.cpp:
1. In **AI Integrations**, add a provider with type `OPENAI`, set `endpoint` to e.g. `https://api.groq.com/openai/v1` or `http://speaches:8000/v1`, set the API key, give it a recognizable title ("Groq", "Self-hosted Whisper").
2. In **Transcription**, select that provider and set `model` to the backend's model identifier (`whisper-large-v3-turbo`, `Systran/faster-distil-whisper-large-v3`, etc.).
This is the universal escape hatch confirmed across OpenWebUI, LibreChat, and Whisper Obsidian plugin: don't enumerate every backend — let the OpenAI engine be a transport, not a brand.
## Gemini webm mitigation
The Gemini `audio/webm` failure is a real user-blocking bug but separate from the settings redesign. Three options were considered:
- **(a) Server-side transcode** with ffmpeg. Adds a heavy runtime dep; rejected as YAGNI.
- **(b) Switch MediaRecorder format** when STT engine is Gemini. Browser support for `audio/mp4` and `audio/wav` in `MediaRecorder` is patchy across Firefox / Safari / Chrome; rejected as fragile.
- **(c) Inline hint + accept the limitation.** Selected. The Transcription section shows a small warning under the model field when the chosen provider type is `GEMINI`: "Gemini does not accept browser-recorded `audio/webm`. For in-editor recording, use an OpenAI-compatible provider."
Server-side transcoding can be revisited later as a self-contained change if Gemini demand grows.
## Validation
Server validation (`server/router/api/v1/ai_service.go`):
- `transcription.provider_id`, when set, must reference an existing entry in `providers[]`. On `UpdateInstanceSetting` for the AI key, reject with `InvalidArgument` if it doesn't.
- `transcription.model` length cap: 256 chars (covers `Systran/faster-distil-whisper-large-v3`-style names with margin).
- `transcription.language` length cap: 32 chars (existing constant `maxTranscriptionLanguageLength`).
- `transcription.prompt` length cap: 4096 chars (existing constant `maxTranscriptionPromptLength`).
Frontend validation in `AISection.tsx`:
- "Save" disabled if `transcription.providerId` is set but the referenced provider was just deleted from the integrations list (in the same unsaved edit).
- Inline warning shown (but Save still allowed) if the referenced provider exists but has `apiKeySet === false` — surfacing the broken state so the operator can fix it without blocking unrelated edits to other settings.
## Backwards compatibility
The schema change is purely additive. Existing instances with `providers` configured but no `transcription` field default to `provider_id = ""`, which means transcription is disabled until the operator visits the new Transcription section and selects a provider.
This is a small UX regression for instances that were relying on the implicit "first provider wins" behavior — they now must make a one-click selection. Acceptable trade-off because:
- It makes the choice explicit (the implicit pick was the source of confusion when users had multiple providers).
- A one-time migration that auto-fills `transcription.provider_id` with the first STT-capable provider is feasible but adds complexity for a one-line user action. Skip the migration; document the change in the release notes.
## Testing
- `internal/ai/transcription_test.go` (existing) covers the transcribe RPC. Add cases for: empty `provider_id` falls back to setting; empty `model` falls back to `DefaultTranscriptionModel`; per-call overrides win over settings.
- `server/router/api/v1/test/ai_service_test.go` (existing) covers the API service. Add cases for the validation rules above (unknown provider_id, oversized model/language/prompt).
- Frontend: manual verification via the dev server (`pnpm dev` in `web/`) — load Settings, add a provider, configure transcription, verify the home editor's record button enables/disables based on `provider_id`. No new component tests required (existing AISection has none).
## Out of scope, explicitly
- Multiple transcription configurations / per-tag or per-user routing.
- Per-call provider override exposed in the editor UI.
- Test-transcription button in settings (worth doing later; deferred to keep this scope tight).
- Glossary / vocabulary list as a separate field — folded into `prompt` for now (Joplin/Superwhisper split this; we can add later if users ask).
- TTS settings. Memos has none today and none planned.

View file

@ -4,7 +4,7 @@ import "github.com/pkg/errors"
const ( const (
// DefaultOpenAITranscriptionModel is the built-in OpenAI transcription model. // DefaultOpenAITranscriptionModel is the built-in OpenAI transcription model.
DefaultOpenAITranscriptionModel = "gpt-4o-transcribe" DefaultOpenAITranscriptionModel = "whisper-1"
// DefaultGeminiTranscriptionModel is the built-in Gemini transcription model. // DefaultGeminiTranscriptionModel is the built-in Gemini transcription model.
DefaultGeminiTranscriptionModel = "gemini-2.5-flash" DefaultGeminiTranscriptionModel = "gemini-2.5-flash"
) )

View file

@ -15,27 +15,13 @@ service AIService {
post: "/api/v1/ai:transcribe" post: "/api/v1/ai:transcribe"
body: "*" body: "*"
}; };
option (google.api.method_signature) = "provider_id,config,audio"; option (google.api.method_signature) = "audio";
} }
} }
message TranscribeRequest { message TranscribeRequest {
// Required. The instance AI provider ID to use.
string provider_id = 1 [(google.api.field_behavior) = REQUIRED];
// Required. Transcription options.
TranscriptionConfig config = 2 [(google.api.field_behavior) = REQUIRED];
// Required. Audio input. // Required. Audio input.
TranscriptionAudio audio = 3 [(google.api.field_behavior) = REQUIRED]; TranscriptionAudio audio = 1 [(google.api.field_behavior) = REQUIRED];
}
message TranscriptionConfig {
// Optional. A prompt to improve transcription quality.
string prompt = 1 [(google.api.field_behavior) = OPTIONAL];
// Optional. The language of the input audio.
string language = 2 [(google.api.field_behavior) = OPTIONAL];
} }
message TranscriptionAudio { message TranscriptionAudio {

View file

@ -227,6 +227,10 @@ message InstanceSetting {
message AISetting { message AISetting {
// providers is the list of AI provider configurations available instance-wide. // providers is the list of AI provider configurations available instance-wide.
repeated AIProviderConfig providers = 1; repeated AIProviderConfig providers = 1;
// transcription is the speech-to-text feature configuration.
// When unset or transcription.provider_id is empty, transcription is disabled.
TranscriptionConfig transcription = 2;
} }
// AIProviderConfig represents one callable AI provider connection. // AIProviderConfig represents one callable AI provider connection.
@ -249,6 +253,25 @@ message InstanceSetting {
OPENAI = 1; OPENAI = 1;
GEMINI = 2; GEMINI = 2;
} }
// TranscriptionConfig configures the speech-to-text feature.
message TranscriptionConfig {
// provider_id references an entry in AISetting.providers[].id.
// Empty string means transcription is disabled.
string provider_id = 1;
// model is the provider-specific model identifier.
// Empty string falls back to the engine default
// (whisper-1 for OPENAI providers, gemini-2.5-flash for GEMINI providers).
string model = 2;
// language is the default ISO 639-1 language hint sent to the provider.
// Empty string lets the provider auto-detect.
string language = 3;
// prompt is a default spelling/vocabulary hint passed to the provider.
string prompt = 4;
}
} }
// Request message for GetInstanceSetting method. // Request message for GetInstanceSetting method.

View file

@ -24,12 +24,8 @@ const (
type TranscribeRequest struct { type TranscribeRequest struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
// Required. The instance AI provider ID to use.
ProviderId string `protobuf:"bytes,1,opt,name=provider_id,json=providerId,proto3" json:"provider_id,omitempty"`
// Required. Transcription options.
Config *TranscriptionConfig `protobuf:"bytes,2,opt,name=config,proto3" json:"config,omitempty"`
// Required. Audio input. // Required. Audio input.
Audio *TranscriptionAudio `protobuf:"bytes,3,opt,name=audio,proto3" json:"audio,omitempty"` Audio *TranscriptionAudio `protobuf:"bytes,1,opt,name=audio,proto3" json:"audio,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
} }
@ -64,20 +60,6 @@ func (*TranscribeRequest) Descriptor() ([]byte, []int) {
return file_api_v1_ai_service_proto_rawDescGZIP(), []int{0} return file_api_v1_ai_service_proto_rawDescGZIP(), []int{0}
} }
func (x *TranscribeRequest) GetProviderId() string {
if x != nil {
return x.ProviderId
}
return ""
}
func (x *TranscribeRequest) GetConfig() *TranscriptionConfig {
if x != nil {
return x.Config
}
return nil
}
func (x *TranscribeRequest) GetAudio() *TranscriptionAudio { func (x *TranscribeRequest) GetAudio() *TranscriptionAudio {
if x != nil { if x != nil {
return x.Audio return x.Audio
@ -85,60 +67,6 @@ func (x *TranscribeRequest) GetAudio() *TranscriptionAudio {
return nil return nil
} }
type TranscriptionConfig struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Optional. A prompt to improve transcription quality.
Prompt string `protobuf:"bytes,1,opt,name=prompt,proto3" json:"prompt,omitempty"`
// Optional. The language of the input audio.
Language string `protobuf:"bytes,2,opt,name=language,proto3" json:"language,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *TranscriptionConfig) Reset() {
*x = TranscriptionConfig{}
mi := &file_api_v1_ai_service_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *TranscriptionConfig) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*TranscriptionConfig) ProtoMessage() {}
func (x *TranscriptionConfig) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_ai_service_proto_msgTypes[1]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use TranscriptionConfig.ProtoReflect.Descriptor instead.
func (*TranscriptionConfig) Descriptor() ([]byte, []int) {
return file_api_v1_ai_service_proto_rawDescGZIP(), []int{1}
}
func (x *TranscriptionConfig) GetPrompt() string {
if x != nil {
return x.Prompt
}
return ""
}
func (x *TranscriptionConfig) GetLanguage() string {
if x != nil {
return x.Language
}
return ""
}
type TranscriptionAudio struct { type TranscriptionAudio struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
// Types that are valid to be assigned to Source: // Types that are valid to be assigned to Source:
@ -156,7 +84,7 @@ type TranscriptionAudio struct {
func (x *TranscriptionAudio) Reset() { func (x *TranscriptionAudio) Reset() {
*x = TranscriptionAudio{} *x = TranscriptionAudio{}
mi := &file_api_v1_ai_service_proto_msgTypes[2] mi := &file_api_v1_ai_service_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@ -168,7 +96,7 @@ func (x *TranscriptionAudio) String() string {
func (*TranscriptionAudio) ProtoMessage() {} func (*TranscriptionAudio) ProtoMessage() {}
func (x *TranscriptionAudio) ProtoReflect() protoreflect.Message { func (x *TranscriptionAudio) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_ai_service_proto_msgTypes[2] mi := &file_api_v1_ai_service_proto_msgTypes[1]
if x != nil { if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@ -181,7 +109,7 @@ func (x *TranscriptionAudio) ProtoReflect() protoreflect.Message {
// Deprecated: Use TranscriptionAudio.ProtoReflect.Descriptor instead. // Deprecated: Use TranscriptionAudio.ProtoReflect.Descriptor instead.
func (*TranscriptionAudio) Descriptor() ([]byte, []int) { func (*TranscriptionAudio) Descriptor() ([]byte, []int) {
return file_api_v1_ai_service_proto_rawDescGZIP(), []int{2} return file_api_v1_ai_service_proto_rawDescGZIP(), []int{1}
} }
func (x *TranscriptionAudio) GetSource() isTranscriptionAudio_Source { func (x *TranscriptionAudio) GetSource() isTranscriptionAudio_Source {
@ -251,7 +179,7 @@ type TranscribeResponse struct {
func (x *TranscribeResponse) Reset() { func (x *TranscribeResponse) Reset() {
*x = TranscribeResponse{} *x = TranscribeResponse{}
mi := &file_api_v1_ai_service_proto_msgTypes[3] mi := &file_api_v1_ai_service_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@ -263,7 +191,7 @@ func (x *TranscribeResponse) String() string {
func (*TranscribeResponse) ProtoMessage() {} func (*TranscribeResponse) ProtoMessage() {}
func (x *TranscribeResponse) ProtoReflect() protoreflect.Message { func (x *TranscribeResponse) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_ai_service_proto_msgTypes[3] mi := &file_api_v1_ai_service_proto_msgTypes[2]
if x != nil { if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@ -276,7 +204,7 @@ func (x *TranscribeResponse) ProtoReflect() protoreflect.Message {
// Deprecated: Use TranscribeResponse.ProtoReflect.Descriptor instead. // Deprecated: Use TranscribeResponse.ProtoReflect.Descriptor instead.
func (*TranscribeResponse) Descriptor() ([]byte, []int) { func (*TranscribeResponse) Descriptor() ([]byte, []int) {
return file_api_v1_ai_service_proto_rawDescGZIP(), []int{3} return file_api_v1_ai_service_proto_rawDescGZIP(), []int{2}
} }
func (x *TranscribeResponse) GetText() string { func (x *TranscribeResponse) GetText() string {
@ -290,15 +218,9 @@ var File_api_v1_ai_service_proto protoreflect.FileDescriptor
const file_api_v1_ai_service_proto_rawDesc = "" + const file_api_v1_ai_service_proto_rawDesc = "" +
"\n" + "\n" +
"\x17api/v1/ai_service.proto\x12\fmemos.api.v1\x1a\x1cgoogle/api/annotations.proto\x1a\x17google/api/client.proto\x1a\x1fgoogle/api/field_behavior.proto\"\xb6\x01\n" + "\x17api/v1/ai_service.proto\x12\fmemos.api.v1\x1a\x1cgoogle/api/annotations.proto\x1a\x17google/api/client.proto\x1a\x1fgoogle/api/field_behavior.proto\"P\n" +
"\x11TranscribeRequest\x12$\n" + "\x11TranscribeRequest\x12;\n" +
"\vprovider_id\x18\x01 \x01(\tB\x03\xe0A\x02R\n" + "\x05audio\x18\x01 \x01(\v2 .memos.api.v1.TranscriptionAudioB\x03\xe0A\x02R\x05audio\"\x9c\x01\n" +
"providerId\x12>\n" +
"\x06config\x18\x02 \x01(\v2!.memos.api.v1.TranscriptionConfigB\x03\xe0A\x02R\x06config\x12;\n" +
"\x05audio\x18\x03 \x01(\v2 .memos.api.v1.TranscriptionAudioB\x03\xe0A\x02R\x05audio\"S\n" +
"\x13TranscriptionConfig\x12\x1b\n" +
"\x06prompt\x18\x01 \x01(\tB\x03\xe0A\x01R\x06prompt\x12\x1f\n" +
"\blanguage\x18\x02 \x01(\tB\x03\xe0A\x01R\blanguage\"\x9c\x01\n" +
"\x12TranscriptionAudio\x12\x1f\n" + "\x12TranscriptionAudio\x12\x1f\n" +
"\acontent\x18\x01 \x01(\fB\x03\xe0A\x04H\x00R\acontent\x12\x12\n" + "\acontent\x18\x01 \x01(\fB\x03\xe0A\x04H\x00R\acontent\x12\x12\n" +
"\x03uri\x18\x02 \x01(\tH\x00R\x03uri\x12\x1f\n" + "\x03uri\x18\x02 \x01(\tH\x00R\x03uri\x12\x1f\n" +
@ -306,10 +228,10 @@ const file_api_v1_ai_service_proto_rawDesc = "" +
"\fcontent_type\x18\x04 \x01(\tB\x03\xe0A\x01R\vcontentTypeB\b\n" + "\fcontent_type\x18\x04 \x01(\tB\x03\xe0A\x01R\vcontentTypeB\b\n" +
"\x06source\"(\n" + "\x06source\"(\n" +
"\x12TranscribeResponse\x12\x12\n" + "\x12TranscribeResponse\x12\x12\n" +
"\x04text\x18\x01 \x01(\tR\x04text2\x9a\x01\n" + "\x04text\x18\x01 \x01(\tR\x04text2\x86\x01\n" +
"\tAIService\x12\x8c\x01\n" + "\tAIService\x12y\n" +
"\n" + "\n" +
"Transcribe\x12\x1f.memos.api.v1.TranscribeRequest\x1a .memos.api.v1.TranscribeResponse\";\xdaA\x18provider_id,config,audio\x82\xd3\xe4\x93\x02\x1a:\x01*\"\x15/api/v1/ai:transcribeB\xa6\x01\n" + "Transcribe\x12\x1f.memos.api.v1.TranscribeRequest\x1a .memos.api.v1.TranscribeResponse\"(\xdaA\x05audio\x82\xd3\xe4\x93\x02\x1a:\x01*\"\x15/api/v1/ai:transcribeB\xa6\x01\n" +
"\x10com.memos.api.v1B\x0eAiServiceProtoP\x01Z0github.com/usememos/memos/proto/gen/api/v1;apiv1\xa2\x02\x03MAX\xaa\x02\fMemos.Api.V1\xca\x02\fMemos\\Api\\V1\xe2\x02\x18Memos\\Api\\V1\\GPBMetadata\xea\x02\x0eMemos::Api::V1b\x06proto3" "\x10com.memos.api.v1B\x0eAiServiceProtoP\x01Z0github.com/usememos/memos/proto/gen/api/v1;apiv1\xa2\x02\x03MAX\xaa\x02\fMemos.Api.V1\xca\x02\fMemos\\Api\\V1\xe2\x02\x18Memos\\Api\\V1\\GPBMetadata\xea\x02\x0eMemos::Api::V1b\x06proto3"
var ( var (
@ -324,23 +246,21 @@ func file_api_v1_ai_service_proto_rawDescGZIP() []byte {
return file_api_v1_ai_service_proto_rawDescData return file_api_v1_ai_service_proto_rawDescData
} }
var file_api_v1_ai_service_proto_msgTypes = make([]protoimpl.MessageInfo, 4) var file_api_v1_ai_service_proto_msgTypes = make([]protoimpl.MessageInfo, 3)
var file_api_v1_ai_service_proto_goTypes = []any{ var file_api_v1_ai_service_proto_goTypes = []any{
(*TranscribeRequest)(nil), // 0: memos.api.v1.TranscribeRequest (*TranscribeRequest)(nil), // 0: memos.api.v1.TranscribeRequest
(*TranscriptionConfig)(nil), // 1: memos.api.v1.TranscriptionConfig (*TranscriptionAudio)(nil), // 1: memos.api.v1.TranscriptionAudio
(*TranscriptionAudio)(nil), // 2: memos.api.v1.TranscriptionAudio (*TranscribeResponse)(nil), // 2: memos.api.v1.TranscribeResponse
(*TranscribeResponse)(nil), // 3: memos.api.v1.TranscribeResponse
} }
var file_api_v1_ai_service_proto_depIdxs = []int32{ var file_api_v1_ai_service_proto_depIdxs = []int32{
1, // 0: memos.api.v1.TranscribeRequest.config:type_name -> memos.api.v1.TranscriptionConfig 1, // 0: memos.api.v1.TranscribeRequest.audio:type_name -> memos.api.v1.TranscriptionAudio
2, // 1: memos.api.v1.TranscribeRequest.audio:type_name -> memos.api.v1.TranscriptionAudio 0, // 1: memos.api.v1.AIService.Transcribe:input_type -> memos.api.v1.TranscribeRequest
0, // 2: memos.api.v1.AIService.Transcribe:input_type -> memos.api.v1.TranscribeRequest 2, // 2: memos.api.v1.AIService.Transcribe:output_type -> memos.api.v1.TranscribeResponse
3, // 3: memos.api.v1.AIService.Transcribe:output_type -> memos.api.v1.TranscribeResponse 2, // [2:3] is the sub-list for method output_type
3, // [3:4] is the sub-list for method output_type 1, // [1:2] is the sub-list for method input_type
2, // [2:3] is the sub-list for method input_type 1, // [1:1] is the sub-list for extension type_name
2, // [2:2] is the sub-list for extension type_name 1, // [1:1] is the sub-list for extension extendee
2, // [2:2] is the sub-list for extension extendee 0, // [0:1] is the sub-list for field type_name
0, // [0:2] is the sub-list for field type_name
} }
func init() { file_api_v1_ai_service_proto_init() } func init() { file_api_v1_ai_service_proto_init() }
@ -348,7 +268,7 @@ func file_api_v1_ai_service_proto_init() {
if File_api_v1_ai_service_proto != nil { if File_api_v1_ai_service_proto != nil {
return return
} }
file_api_v1_ai_service_proto_msgTypes[2].OneofWrappers = []any{ file_api_v1_ai_service_proto_msgTypes[1].OneofWrappers = []any{
(*TranscriptionAudio_Content)(nil), (*TranscriptionAudio_Content)(nil),
(*TranscriptionAudio_Uri)(nil), (*TranscriptionAudio_Uri)(nil),
} }
@ -358,7 +278,7 @@ func file_api_v1_ai_service_proto_init() {
GoPackagePath: reflect.TypeOf(x{}).PkgPath(), GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_api_v1_ai_service_proto_rawDesc), len(file_api_v1_ai_service_proto_rawDesc)), RawDescriptor: unsafe.Slice(unsafe.StringData(file_api_v1_ai_service_proto_rawDesc), len(file_api_v1_ai_service_proto_rawDesc)),
NumEnums: 0, NumEnums: 0,
NumMessages: 4, NumMessages: 3,
NumExtensions: 0, NumExtensions: 0,
NumServices: 1, NumServices: 1,
}, },

View file

@ -1138,6 +1138,9 @@ type InstanceSetting_AISetting struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
// providers is the list of AI provider configurations available instance-wide. // providers is the list of AI provider configurations available instance-wide.
Providers []*InstanceSetting_AIProviderConfig `protobuf:"bytes,1,rep,name=providers,proto3" json:"providers,omitempty"` Providers []*InstanceSetting_AIProviderConfig `protobuf:"bytes,1,rep,name=providers,proto3" json:"providers,omitempty"`
// transcription is the speech-to-text feature configuration.
// When unset or transcription.provider_id is empty, transcription is disabled.
Transcription *InstanceSetting_TranscriptionConfig `protobuf:"bytes,2,opt,name=transcription,proto3" json:"transcription,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
} }
@ -1179,6 +1182,13 @@ func (x *InstanceSetting_AISetting) GetProviders() []*InstanceSetting_AIProvider
return nil return nil
} }
func (x *InstanceSetting_AISetting) GetTranscription() *InstanceSetting_TranscriptionConfig {
if x != nil {
return x.Transcription
}
return nil
}
// AIProviderConfig represents one callable AI provider connection. // AIProviderConfig represents one callable AI provider connection.
type InstanceSetting_AIProviderConfig struct { type InstanceSetting_AIProviderConfig struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
@ -1275,6 +1285,83 @@ func (x *InstanceSetting_AIProviderConfig) GetApiKeyHint() string {
return "" return ""
} }
// TranscriptionConfig configures the speech-to-text feature.
type InstanceSetting_TranscriptionConfig struct {
state protoimpl.MessageState `protogen:"open.v1"`
// provider_id references an entry in AISetting.providers[].id.
// Empty string means transcription is disabled.
ProviderId string `protobuf:"bytes,1,opt,name=provider_id,json=providerId,proto3" json:"provider_id,omitempty"`
// model is the provider-specific model identifier.
// Empty string falls back to the engine default
// (whisper-1 for OPENAI providers, gemini-2.5-flash for GEMINI providers).
Model string `protobuf:"bytes,2,opt,name=model,proto3" json:"model,omitempty"`
// language is the default ISO 639-1 language hint sent to the provider.
// Empty string lets the provider auto-detect.
Language string `protobuf:"bytes,3,opt,name=language,proto3" json:"language,omitempty"`
// prompt is a default spelling/vocabulary hint passed to the provider.
Prompt string `protobuf:"bytes,4,opt,name=prompt,proto3" json:"prompt,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *InstanceSetting_TranscriptionConfig) Reset() {
*x = InstanceSetting_TranscriptionConfig{}
mi := &file_api_v1_instance_service_proto_msgTypes[16]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *InstanceSetting_TranscriptionConfig) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*InstanceSetting_TranscriptionConfig) ProtoMessage() {}
func (x *InstanceSetting_TranscriptionConfig) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_instance_service_proto_msgTypes[16]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use InstanceSetting_TranscriptionConfig.ProtoReflect.Descriptor instead.
func (*InstanceSetting_TranscriptionConfig) Descriptor() ([]byte, []int) {
return file_api_v1_instance_service_proto_rawDescGZIP(), []int{2, 8}
}
func (x *InstanceSetting_TranscriptionConfig) GetProviderId() string {
if x != nil {
return x.ProviderId
}
return ""
}
func (x *InstanceSetting_TranscriptionConfig) GetModel() string {
if x != nil {
return x.Model
}
return ""
}
func (x *InstanceSetting_TranscriptionConfig) GetLanguage() string {
if x != nil {
return x.Language
}
return ""
}
func (x *InstanceSetting_TranscriptionConfig) GetPrompt() string {
if x != nil {
return x.Prompt
}
return ""
}
// Custom profile configuration for instance branding. // Custom profile configuration for instance branding.
type InstanceSetting_GeneralSetting_CustomProfile struct { type InstanceSetting_GeneralSetting_CustomProfile struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
@ -1287,7 +1374,7 @@ type InstanceSetting_GeneralSetting_CustomProfile struct {
func (x *InstanceSetting_GeneralSetting_CustomProfile) Reset() { func (x *InstanceSetting_GeneralSetting_CustomProfile) Reset() {
*x = InstanceSetting_GeneralSetting_CustomProfile{} *x = InstanceSetting_GeneralSetting_CustomProfile{}
mi := &file_api_v1_instance_service_proto_msgTypes[16] mi := &file_api_v1_instance_service_proto_msgTypes[17]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@ -1299,7 +1386,7 @@ func (x *InstanceSetting_GeneralSetting_CustomProfile) String() string {
func (*InstanceSetting_GeneralSetting_CustomProfile) ProtoMessage() {} func (*InstanceSetting_GeneralSetting_CustomProfile) ProtoMessage() {}
func (x *InstanceSetting_GeneralSetting_CustomProfile) ProtoReflect() protoreflect.Message { func (x *InstanceSetting_GeneralSetting_CustomProfile) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_instance_service_proto_msgTypes[16] mi := &file_api_v1_instance_service_proto_msgTypes[17]
if x != nil { if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@ -1352,7 +1439,7 @@ type InstanceSetting_StorageSetting_S3Config struct {
func (x *InstanceSetting_StorageSetting_S3Config) Reset() { func (x *InstanceSetting_StorageSetting_S3Config) Reset() {
*x = InstanceSetting_StorageSetting_S3Config{} *x = InstanceSetting_StorageSetting_S3Config{}
mi := &file_api_v1_instance_service_proto_msgTypes[17] mi := &file_api_v1_instance_service_proto_msgTypes[18]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@ -1364,7 +1451,7 @@ func (x *InstanceSetting_StorageSetting_S3Config) String() string {
func (*InstanceSetting_StorageSetting_S3Config) ProtoMessage() {} func (*InstanceSetting_StorageSetting_S3Config) ProtoMessage() {}
func (x *InstanceSetting_StorageSetting_S3Config) ProtoReflect() protoreflect.Message { func (x *InstanceSetting_StorageSetting_S3Config) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_instance_service_proto_msgTypes[17] mi := &file_api_v1_instance_service_proto_msgTypes[18]
if x != nil { if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@ -1441,7 +1528,7 @@ type InstanceSetting_NotificationSetting_EmailSetting struct {
func (x *InstanceSetting_NotificationSetting_EmailSetting) Reset() { func (x *InstanceSetting_NotificationSetting_EmailSetting) Reset() {
*x = InstanceSetting_NotificationSetting_EmailSetting{} *x = InstanceSetting_NotificationSetting_EmailSetting{}
mi := &file_api_v1_instance_service_proto_msgTypes[19] mi := &file_api_v1_instance_service_proto_msgTypes[20]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@ -1453,7 +1540,7 @@ func (x *InstanceSetting_NotificationSetting_EmailSetting) String() string {
func (*InstanceSetting_NotificationSetting_EmailSetting) ProtoMessage() {} func (*InstanceSetting_NotificationSetting_EmailSetting) ProtoMessage() {}
func (x *InstanceSetting_NotificationSetting_EmailSetting) ProtoReflect() protoreflect.Message { func (x *InstanceSetting_NotificationSetting_EmailSetting) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_instance_service_proto_msgTypes[19] mi := &file_api_v1_instance_service_proto_msgTypes[20]
if x != nil { if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@ -1552,7 +1639,7 @@ type InstanceStats_DatabaseStats struct {
func (x *InstanceStats_DatabaseStats) Reset() { func (x *InstanceStats_DatabaseStats) Reset() {
*x = InstanceStats_DatabaseStats{} *x = InstanceStats_DatabaseStats{}
mi := &file_api_v1_instance_service_proto_msgTypes[20] mi := &file_api_v1_instance_service_proto_msgTypes[21]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@ -1564,7 +1651,7 @@ func (x *InstanceStats_DatabaseStats) String() string {
func (*InstanceStats_DatabaseStats) ProtoMessage() {} func (*InstanceStats_DatabaseStats) ProtoMessage() {}
func (x *InstanceStats_DatabaseStats) ProtoReflect() protoreflect.Message { func (x *InstanceStats_DatabaseStats) ProtoReflect() protoreflect.Message {
mi := &file_api_v1_instance_service_proto_msgTypes[20] mi := &file_api_v1_instance_service_proto_msgTypes[21]
if x != nil { if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@ -1605,7 +1692,7 @@ const file_api_v1_instance_service_proto_rawDesc = "" +
"\finstance_url\x18\x06 \x01(\tR\vinstanceUrl\x12(\n" + "\finstance_url\x18\x06 \x01(\tR\vinstanceUrl\x12(\n" +
"\x05admin\x18\a \x01(\v2\x12.memos.api.v1.UserR\x05admin\x12\x16\n" + "\x05admin\x18\a \x01(\v2\x12.memos.api.v1.UserR\x05admin\x12\x16\n" +
"\x06commit\x18\b \x01(\tR\x06commit\"\x1b\n" + "\x06commit\x18\b \x01(\tR\x06commit\"\x1b\n" +
"\x19GetInstanceProfileRequest\"\xf0\x19\n" + "\x19GetInstanceProfileRequest\"\xcd\x1b\n" +
"\x0fInstanceSetting\x12\x17\n" + "\x0fInstanceSetting\x12\x17\n" +
"\x04name\x18\x01 \x01(\tB\x03\xe0A\bR\x04name\x12W\n" + "\x04name\x18\x01 \x01(\tB\x03\xe0A\bR\x04name\x12W\n" +
"\x0fgeneral_setting\x18\x02 \x01(\v2,.memos.api.v1.InstanceSetting.GeneralSettingH\x00R\x0egeneralSetting\x12W\n" + "\x0fgeneral_setting\x18\x02 \x01(\v2,.memos.api.v1.InstanceSetting.GeneralSettingH\x00R\x0egeneralSetting\x12W\n" +
@ -1671,9 +1758,10 @@ const file_api_v1_instance_service_proto_rawDesc = "" +
"\breply_to\x18\b \x01(\tR\areplyTo\x12\x17\n" + "\breply_to\x18\b \x01(\tR\areplyTo\x12\x17\n" +
"\ause_tls\x18\t \x01(\bR\x06useTls\x12\x17\n" + "\ause_tls\x18\t \x01(\bR\x06useTls\x12\x17\n" +
"\ause_ssl\x18\n" + "\ause_ssl\x18\n" +
" \x01(\bR\x06useSsl\x1aY\n" + " \x01(\bR\x06useSsl\x1a\xb2\x01\n" +
"\tAISetting\x12L\n" + "\tAISetting\x12L\n" +
"\tproviders\x18\x01 \x03(\v2..memos.api.v1.InstanceSetting.AIProviderConfigR\tproviders\x1a\x80\x02\n" + "\tproviders\x18\x01 \x03(\v2..memos.api.v1.InstanceSetting.AIProviderConfigR\tproviders\x12W\n" +
"\rtranscription\x18\x02 \x01(\v21.memos.api.v1.InstanceSetting.TranscriptionConfigR\rtranscription\x1a\x80\x02\n" +
"\x10AIProviderConfig\x12\x0e\n" + "\x10AIProviderConfig\x12\x0e\n" +
"\x02id\x18\x01 \x01(\tR\x02id\x12\x14\n" + "\x02id\x18\x01 \x01(\tR\x02id\x12\x14\n" +
"\x05title\x18\x02 \x01(\tR\x05title\x12@\n" + "\x05title\x18\x02 \x01(\tR\x05title\x12@\n" +
@ -1682,7 +1770,13 @@ const file_api_v1_instance_service_proto_rawDesc = "" +
"\aapi_key\x18\x05 \x01(\tB\x03\xe0A\x04R\x06apiKey\x12#\n" + "\aapi_key\x18\x05 \x01(\tB\x03\xe0A\x04R\x06apiKey\x12#\n" +
"\vapi_key_set\x18\b \x01(\bB\x03\xe0A\x03R\tapiKeySet\x12%\n" + "\vapi_key_set\x18\b \x01(\bB\x03\xe0A\x03R\tapiKeySet\x12%\n" +
"\fapi_key_hint\x18\t \x01(\tB\x03\xe0A\x03R\n" + "\fapi_key_hint\x18\t \x01(\tB\x03\xe0A\x03R\n" +
"apiKeyHint\"j\n" + "apiKeyHint\x1a\x80\x01\n" +
"\x13TranscriptionConfig\x12\x1f\n" +
"\vprovider_id\x18\x01 \x01(\tR\n" +
"providerId\x12\x14\n" +
"\x05model\x18\x02 \x01(\tR\x05model\x12\x1a\n" +
"\blanguage\x18\x03 \x01(\tR\blanguage\x12\x16\n" +
"\x06prompt\x18\x04 \x01(\tR\x06prompt\"j\n" +
"\x03Key\x12\x13\n" + "\x03Key\x12\x13\n" +
"\x0fKEY_UNSPECIFIED\x10\x00\x12\v\n" + "\x0fKEY_UNSPECIFIED\x10\x00\x12\v\n" +
"\aGENERAL\x10\x01\x12\v\n" + "\aGENERAL\x10\x01\x12\v\n" +
@ -1739,7 +1833,7 @@ func file_api_v1_instance_service_proto_rawDescGZIP() []byte {
} }
var file_api_v1_instance_service_proto_enumTypes = make([]protoimpl.EnumInfo, 3) var file_api_v1_instance_service_proto_enumTypes = make([]protoimpl.EnumInfo, 3)
var file_api_v1_instance_service_proto_msgTypes = make([]protoimpl.MessageInfo, 21) var file_api_v1_instance_service_proto_msgTypes = make([]protoimpl.MessageInfo, 22)
var file_api_v1_instance_service_proto_goTypes = []any{ var file_api_v1_instance_service_proto_goTypes = []any{
(InstanceSetting_Key)(0), // 0: memos.api.v1.InstanceSetting.Key (InstanceSetting_Key)(0), // 0: memos.api.v1.InstanceSetting.Key
(InstanceSetting_AIProviderType)(0), // 1: memos.api.v1.InstanceSetting.AIProviderType (InstanceSetting_AIProviderType)(0), // 1: memos.api.v1.InstanceSetting.AIProviderType
@ -1760,19 +1854,20 @@ var file_api_v1_instance_service_proto_goTypes = []any{
(*InstanceSetting_NotificationSetting)(nil), // 16: memos.api.v1.InstanceSetting.NotificationSetting (*InstanceSetting_NotificationSetting)(nil), // 16: memos.api.v1.InstanceSetting.NotificationSetting
(*InstanceSetting_AISetting)(nil), // 17: memos.api.v1.InstanceSetting.AISetting (*InstanceSetting_AISetting)(nil), // 17: memos.api.v1.InstanceSetting.AISetting
(*InstanceSetting_AIProviderConfig)(nil), // 18: memos.api.v1.InstanceSetting.AIProviderConfig (*InstanceSetting_AIProviderConfig)(nil), // 18: memos.api.v1.InstanceSetting.AIProviderConfig
(*InstanceSetting_GeneralSetting_CustomProfile)(nil), // 19: memos.api.v1.InstanceSetting.GeneralSetting.CustomProfile (*InstanceSetting_TranscriptionConfig)(nil), // 19: memos.api.v1.InstanceSetting.TranscriptionConfig
(*InstanceSetting_StorageSetting_S3Config)(nil), // 20: memos.api.v1.InstanceSetting.StorageSetting.S3Config (*InstanceSetting_GeneralSetting_CustomProfile)(nil), // 20: memos.api.v1.InstanceSetting.GeneralSetting.CustomProfile
nil, // 21: memos.api.v1.InstanceSetting.TagsSetting.TagsEntry (*InstanceSetting_StorageSetting_S3Config)(nil), // 21: memos.api.v1.InstanceSetting.StorageSetting.S3Config
(*InstanceSetting_NotificationSetting_EmailSetting)(nil), // 22: memos.api.v1.InstanceSetting.NotificationSetting.EmailSetting nil, // 22: memos.api.v1.InstanceSetting.TagsSetting.TagsEntry
(*InstanceStats_DatabaseStats)(nil), // 23: memos.api.v1.InstanceStats.DatabaseStats (*InstanceSetting_NotificationSetting_EmailSetting)(nil), // 23: memos.api.v1.InstanceSetting.NotificationSetting.EmailSetting
(*User)(nil), // 24: memos.api.v1.User (*InstanceStats_DatabaseStats)(nil), // 24: memos.api.v1.InstanceStats.DatabaseStats
(*fieldmaskpb.FieldMask)(nil), // 25: google.protobuf.FieldMask (*User)(nil), // 25: memos.api.v1.User
(*timestamppb.Timestamp)(nil), // 26: google.protobuf.Timestamp (*fieldmaskpb.FieldMask)(nil), // 26: google.protobuf.FieldMask
(*color.Color)(nil), // 27: google.type.Color (*timestamppb.Timestamp)(nil), // 27: google.protobuf.Timestamp
(*emptypb.Empty)(nil), // 28: google.protobuf.Empty (*color.Color)(nil), // 28: google.type.Color
(*emptypb.Empty)(nil), // 29: google.protobuf.Empty
} }
var file_api_v1_instance_service_proto_depIdxs = []int32{ var file_api_v1_instance_service_proto_depIdxs = []int32{
24, // 0: memos.api.v1.InstanceProfile.admin:type_name -> memos.api.v1.User 25, // 0: memos.api.v1.InstanceProfile.admin:type_name -> memos.api.v1.User
11, // 1: memos.api.v1.InstanceSetting.general_setting:type_name -> memos.api.v1.InstanceSetting.GeneralSetting 11, // 1: memos.api.v1.InstanceSetting.general_setting:type_name -> memos.api.v1.InstanceSetting.GeneralSetting
12, // 2: memos.api.v1.InstanceSetting.storage_setting:type_name -> memos.api.v1.InstanceSetting.StorageSetting 12, // 2: memos.api.v1.InstanceSetting.storage_setting:type_name -> memos.api.v1.InstanceSetting.StorageSetting
13, // 3: memos.api.v1.InstanceSetting.memo_related_setting:type_name -> memos.api.v1.InstanceSetting.MemoRelatedSetting 13, // 3: memos.api.v1.InstanceSetting.memo_related_setting:type_name -> memos.api.v1.InstanceSetting.MemoRelatedSetting
@ -1780,34 +1875,35 @@ var file_api_v1_instance_service_proto_depIdxs = []int32{
16, // 5: memos.api.v1.InstanceSetting.notification_setting:type_name -> memos.api.v1.InstanceSetting.NotificationSetting 16, // 5: memos.api.v1.InstanceSetting.notification_setting:type_name -> memos.api.v1.InstanceSetting.NotificationSetting
17, // 6: memos.api.v1.InstanceSetting.ai_setting:type_name -> memos.api.v1.InstanceSetting.AISetting 17, // 6: memos.api.v1.InstanceSetting.ai_setting:type_name -> memos.api.v1.InstanceSetting.AISetting
5, // 7: memos.api.v1.UpdateInstanceSettingRequest.setting:type_name -> memos.api.v1.InstanceSetting 5, // 7: memos.api.v1.UpdateInstanceSettingRequest.setting:type_name -> memos.api.v1.InstanceSetting
25, // 8: memos.api.v1.UpdateInstanceSettingRequest.update_mask:type_name -> google.protobuf.FieldMask 26, // 8: memos.api.v1.UpdateInstanceSettingRequest.update_mask:type_name -> google.protobuf.FieldMask
22, // 9: memos.api.v1.TestInstanceEmailSettingRequest.email:type_name -> memos.api.v1.InstanceSetting.NotificationSetting.EmailSetting 23, // 9: memos.api.v1.TestInstanceEmailSettingRequest.email:type_name -> memos.api.v1.InstanceSetting.NotificationSetting.EmailSetting
23, // 10: memos.api.v1.InstanceStats.database:type_name -> memos.api.v1.InstanceStats.DatabaseStats 24, // 10: memos.api.v1.InstanceStats.database:type_name -> memos.api.v1.InstanceStats.DatabaseStats
26, // 11: memos.api.v1.InstanceStats.generated_time:type_name -> google.protobuf.Timestamp 27, // 11: memos.api.v1.InstanceStats.generated_time:type_name -> google.protobuf.Timestamp
19, // 12: memos.api.v1.InstanceSetting.GeneralSetting.custom_profile:type_name -> memos.api.v1.InstanceSetting.GeneralSetting.CustomProfile 20, // 12: memos.api.v1.InstanceSetting.GeneralSetting.custom_profile:type_name -> memos.api.v1.InstanceSetting.GeneralSetting.CustomProfile
2, // 13: memos.api.v1.InstanceSetting.StorageSetting.storage_type:type_name -> memos.api.v1.InstanceSetting.StorageSetting.StorageType 2, // 13: memos.api.v1.InstanceSetting.StorageSetting.storage_type:type_name -> memos.api.v1.InstanceSetting.StorageSetting.StorageType
20, // 14: memos.api.v1.InstanceSetting.StorageSetting.s3_config:type_name -> memos.api.v1.InstanceSetting.StorageSetting.S3Config 21, // 14: memos.api.v1.InstanceSetting.StorageSetting.s3_config:type_name -> memos.api.v1.InstanceSetting.StorageSetting.S3Config
27, // 15: memos.api.v1.InstanceSetting.TagMetadata.background_color:type_name -> google.type.Color 28, // 15: memos.api.v1.InstanceSetting.TagMetadata.background_color:type_name -> google.type.Color
21, // 16: memos.api.v1.InstanceSetting.TagsSetting.tags:type_name -> memos.api.v1.InstanceSetting.TagsSetting.TagsEntry 22, // 16: memos.api.v1.InstanceSetting.TagsSetting.tags:type_name -> memos.api.v1.InstanceSetting.TagsSetting.TagsEntry
22, // 17: memos.api.v1.InstanceSetting.NotificationSetting.email:type_name -> memos.api.v1.InstanceSetting.NotificationSetting.EmailSetting 23, // 17: memos.api.v1.InstanceSetting.NotificationSetting.email:type_name -> memos.api.v1.InstanceSetting.NotificationSetting.EmailSetting
18, // 18: memos.api.v1.InstanceSetting.AISetting.providers:type_name -> memos.api.v1.InstanceSetting.AIProviderConfig 18, // 18: memos.api.v1.InstanceSetting.AISetting.providers:type_name -> memos.api.v1.InstanceSetting.AIProviderConfig
1, // 19: memos.api.v1.InstanceSetting.AIProviderConfig.type:type_name -> memos.api.v1.InstanceSetting.AIProviderType 19, // 19: memos.api.v1.InstanceSetting.AISetting.transcription:type_name -> memos.api.v1.InstanceSetting.TranscriptionConfig
14, // 20: memos.api.v1.InstanceSetting.TagsSetting.TagsEntry.value:type_name -> memos.api.v1.InstanceSetting.TagMetadata 1, // 20: memos.api.v1.InstanceSetting.AIProviderConfig.type:type_name -> memos.api.v1.InstanceSetting.AIProviderType
4, // 21: memos.api.v1.InstanceService.GetInstanceProfile:input_type -> memos.api.v1.GetInstanceProfileRequest 14, // 21: memos.api.v1.InstanceSetting.TagsSetting.TagsEntry.value:type_name -> memos.api.v1.InstanceSetting.TagMetadata
6, // 22: memos.api.v1.InstanceService.GetInstanceSetting:input_type -> memos.api.v1.GetInstanceSettingRequest 4, // 22: memos.api.v1.InstanceService.GetInstanceProfile:input_type -> memos.api.v1.GetInstanceProfileRequest
7, // 23: memos.api.v1.InstanceService.UpdateInstanceSetting:input_type -> memos.api.v1.UpdateInstanceSettingRequest 6, // 23: memos.api.v1.InstanceService.GetInstanceSetting:input_type -> memos.api.v1.GetInstanceSettingRequest
8, // 24: memos.api.v1.InstanceService.TestInstanceEmailSetting:input_type -> memos.api.v1.TestInstanceEmailSettingRequest 7, // 24: memos.api.v1.InstanceService.UpdateInstanceSetting:input_type -> memos.api.v1.UpdateInstanceSettingRequest
9, // 25: memos.api.v1.InstanceService.GetInstanceStats:input_type -> memos.api.v1.GetInstanceStatsRequest 8, // 25: memos.api.v1.InstanceService.TestInstanceEmailSetting:input_type -> memos.api.v1.TestInstanceEmailSettingRequest
3, // 26: memos.api.v1.InstanceService.GetInstanceProfile:output_type -> memos.api.v1.InstanceProfile 9, // 26: memos.api.v1.InstanceService.GetInstanceStats:input_type -> memos.api.v1.GetInstanceStatsRequest
5, // 27: memos.api.v1.InstanceService.GetInstanceSetting:output_type -> memos.api.v1.InstanceSetting 3, // 27: memos.api.v1.InstanceService.GetInstanceProfile:output_type -> memos.api.v1.InstanceProfile
5, // 28: memos.api.v1.InstanceService.UpdateInstanceSetting:output_type -> memos.api.v1.InstanceSetting 5, // 28: memos.api.v1.InstanceService.GetInstanceSetting:output_type -> memos.api.v1.InstanceSetting
28, // 29: memos.api.v1.InstanceService.TestInstanceEmailSetting:output_type -> google.protobuf.Empty 5, // 29: memos.api.v1.InstanceService.UpdateInstanceSetting:output_type -> memos.api.v1.InstanceSetting
10, // 30: memos.api.v1.InstanceService.GetInstanceStats:output_type -> memos.api.v1.InstanceStats 29, // 30: memos.api.v1.InstanceService.TestInstanceEmailSetting:output_type -> google.protobuf.Empty
26, // [26:31] is the sub-list for method output_type 10, // 31: memos.api.v1.InstanceService.GetInstanceStats:output_type -> memos.api.v1.InstanceStats
21, // [21:26] is the sub-list for method input_type 27, // [27:32] is the sub-list for method output_type
21, // [21:21] is the sub-list for extension type_name 22, // [22:27] is the sub-list for method input_type
21, // [21:21] is the sub-list for extension extendee 22, // [22:22] is the sub-list for extension type_name
0, // [0:21] is the sub-list for field type_name 22, // [22:22] is the sub-list for extension extendee
0, // [0:22] is the sub-list for field type_name
} }
func init() { file_api_v1_instance_service_proto_init() } func init() { file_api_v1_instance_service_proto_init() }
@ -1830,7 +1926,7 @@ func file_api_v1_instance_service_proto_init() {
GoPackagePath: reflect.TypeOf(x{}).PkgPath(), GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_api_v1_instance_service_proto_rawDesc), len(file_api_v1_instance_service_proto_rawDesc)), RawDescriptor: unsafe.Slice(unsafe.StringData(file_api_v1_instance_service_proto_rawDesc), len(file_api_v1_instance_service_proto_rawDesc)),
NumEnums: 3, NumEnums: 3,
NumMessages: 21, NumMessages: 22,
NumExtensions: 0, NumExtensions: 0,
NumServices: 1, NumServices: 1,
}, },

View file

@ -2701,6 +2701,12 @@ components:
items: items:
$ref: '#/components/schemas/InstanceSetting_AIProviderConfig' $ref: '#/components/schemas/InstanceSetting_AIProviderConfig'
description: providers is the list of AI provider configurations available instance-wide. description: providers is the list of AI provider configurations available instance-wide.
transcription:
allOf:
- $ref: '#/components/schemas/InstanceSetting_TranscriptionConfig'
description: |-
transcription is the speech-to-text feature configuration.
When unset or transcription.provider_id is empty, transcription is disabled.
description: AI provider configuration settings. description: AI provider configuration settings.
InstanceSetting_GeneralSetting: InstanceSetting_GeneralSetting:
type: object type: object
@ -2808,6 +2814,29 @@ components:
so a single entry like "project/.*" matches all tags under that prefix. so a single entry like "project/.*" matches all tags under that prefix.
Exact tag names are also valid (they are trivially valid regex patterns). Exact tag names are also valid (they are trivially valid regex patterns).
description: Tag metadata configuration. description: Tag metadata configuration.
InstanceSetting_TranscriptionConfig:
type: object
properties:
providerId:
type: string
description: |-
provider_id references an entry in AISetting.providers[].id.
Empty string means transcription is disabled.
model:
type: string
description: |-
model is the provider-specific model identifier.
Empty string falls back to the engine default
(whisper-1 for OPENAI providers, gemini-2.5-flash for GEMINI providers).
language:
type: string
description: |-
language is the default ISO 639-1 language hint sent to the provider.
Empty string lets the provider auto-detect.
prompt:
type: string
description: prompt is a default spelling/vocabulary hint passed to the provider.
description: TranscriptionConfig configures the speech-to-text feature.
InstanceStats: InstanceStats:
type: object type: object
properties: properties:
@ -3538,18 +3567,9 @@ components:
description: Request message for TestInstanceEmailSetting method. description: Request message for TestInstanceEmailSetting method.
TranscribeRequest: TranscribeRequest:
required: required:
- providerId
- config
- audio - audio
type: object type: object
properties: properties:
providerId:
type: string
description: Required. The instance AI provider ID to use.
config:
allOf:
- $ref: '#/components/schemas/TranscriptionConfig'
description: Required. Transcription options.
audio: audio:
allOf: allOf:
- $ref: '#/components/schemas/TranscriptionAudio' - $ref: '#/components/schemas/TranscriptionAudio'
@ -3577,15 +3597,6 @@ components:
contentType: contentType:
type: string type: string
description: Optional. The MIME type of the input audio. description: Optional. The MIME type of the input audio.
TranscriptionConfig:
type: object
properties:
prompt:
type: string
description: Optional. A prompt to improve transcription quality.
language:
type: string
description: Optional. The language of the input audio.
UpsertMemoReactionRequest: UpsertMemoReactionRequest:
required: required:
- name - name

View file

@ -963,6 +963,9 @@ type InstanceAISetting struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
// providers is the list of AI provider configurations available instance-wide. // providers is the list of AI provider configurations available instance-wide.
Providers []*AIProviderConfig `protobuf:"bytes,1,rep,name=providers,proto3" json:"providers,omitempty"` Providers []*AIProviderConfig `protobuf:"bytes,1,rep,name=providers,proto3" json:"providers,omitempty"`
// transcription is the speech-to-text feature configuration.
// When unset or transcription.provider_id is empty, transcription is disabled.
Transcription *TranscriptionConfig `protobuf:"bytes,2,opt,name=transcription,proto3" json:"transcription,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
} }
@ -1004,6 +1007,13 @@ func (x *InstanceAISetting) GetProviders() []*AIProviderConfig {
return nil return nil
} }
func (x *InstanceAISetting) GetTranscription() *TranscriptionConfig {
if x != nil {
return x.Transcription
}
return nil
}
type AIProviderConfig struct { type AIProviderConfig struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"` Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
@ -1081,6 +1091,85 @@ func (x *AIProviderConfig) GetApiKey() string {
return "" return ""
} }
// TranscriptionConfig configures the speech-to-text feature.
type TranscriptionConfig struct {
state protoimpl.MessageState `protogen:"open.v1"`
// provider_id references an entry in InstanceAISetting.providers[].id.
// Empty string means transcription is disabled.
ProviderId string `protobuf:"bytes,1,opt,name=provider_id,json=providerId,proto3" json:"provider_id,omitempty"`
// model is the provider-specific model identifier.
// Empty string falls back to the engine default
// (whisper-1 for OPENAI providers, gemini-2.5-flash for GEMINI providers).
Model string `protobuf:"bytes,2,opt,name=model,proto3" json:"model,omitempty"`
// language is the default ISO 639-1 language hint sent to the provider.
// Empty string lets the provider auto-detect.
Language string `protobuf:"bytes,3,opt,name=language,proto3" json:"language,omitempty"`
// prompt is a default spelling/vocabulary hint passed to the provider.
// Used as the OpenAI Whisper "prompt" parameter and folded into the Gemini
// generation prompt as a "Context and spelling hints" block.
Prompt string `protobuf:"bytes,4,opt,name=prompt,proto3" json:"prompt,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *TranscriptionConfig) Reset() {
*x = TranscriptionConfig{}
mi := &file_store_instance_setting_proto_msgTypes[12]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *TranscriptionConfig) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*TranscriptionConfig) ProtoMessage() {}
func (x *TranscriptionConfig) ProtoReflect() protoreflect.Message {
mi := &file_store_instance_setting_proto_msgTypes[12]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use TranscriptionConfig.ProtoReflect.Descriptor instead.
func (*TranscriptionConfig) Descriptor() ([]byte, []int) {
return file_store_instance_setting_proto_rawDescGZIP(), []int{12}
}
func (x *TranscriptionConfig) GetProviderId() string {
if x != nil {
return x.ProviderId
}
return ""
}
func (x *TranscriptionConfig) GetModel() string {
if x != nil {
return x.Model
}
return ""
}
func (x *TranscriptionConfig) GetLanguage() string {
if x != nil {
return x.Language
}
return ""
}
func (x *TranscriptionConfig) GetPrompt() string {
if x != nil {
return x.Prompt
}
return ""
}
type InstanceNotificationSetting_EmailSetting struct { type InstanceNotificationSetting_EmailSetting struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
Enabled bool `protobuf:"varint,1,opt,name=enabled,proto3" json:"enabled,omitempty"` Enabled bool `protobuf:"varint,1,opt,name=enabled,proto3" json:"enabled,omitempty"`
@ -1099,7 +1188,7 @@ type InstanceNotificationSetting_EmailSetting struct {
func (x *InstanceNotificationSetting_EmailSetting) Reset() { func (x *InstanceNotificationSetting_EmailSetting) Reset() {
*x = InstanceNotificationSetting_EmailSetting{} *x = InstanceNotificationSetting_EmailSetting{}
mi := &file_store_instance_setting_proto_msgTypes[13] mi := &file_store_instance_setting_proto_msgTypes[14]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@ -1111,7 +1200,7 @@ func (x *InstanceNotificationSetting_EmailSetting) String() string {
func (*InstanceNotificationSetting_EmailSetting) ProtoMessage() {} func (*InstanceNotificationSetting_EmailSetting) ProtoMessage() {}
func (x *InstanceNotificationSetting_EmailSetting) ProtoReflect() protoreflect.Message { func (x *InstanceNotificationSetting_EmailSetting) ProtoReflect() protoreflect.Message {
mi := &file_store_instance_setting_proto_msgTypes[13] mi := &file_store_instance_setting_proto_msgTypes[14]
if x != nil { if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@ -1273,15 +1362,22 @@ const file_store_instance_setting_proto_rawDesc = "" +
"\breply_to\x18\b \x01(\tR\areplyTo\x12\x17\n" + "\breply_to\x18\b \x01(\tR\areplyTo\x12\x17\n" +
"\ause_tls\x18\t \x01(\bR\x06useTls\x12\x17\n" + "\ause_tls\x18\t \x01(\bR\x06useTls\x12\x17\n" +
"\ause_ssl\x18\n" + "\ause_ssl\x18\n" +
" \x01(\bR\x06useSsl\"P\n" + " \x01(\bR\x06useSsl\"\x98\x01\n" +
"\x11InstanceAISetting\x12;\n" + "\x11InstanceAISetting\x12;\n" +
"\tproviders\x18\x01 \x03(\v2\x1d.memos.store.AIProviderConfigR\tproviders\"\x9e\x01\n" + "\tproviders\x18\x01 \x03(\v2\x1d.memos.store.AIProviderConfigR\tproviders\x12F\n" +
"\rtranscription\x18\x02 \x01(\v2 .memos.store.TranscriptionConfigR\rtranscription\"\x9e\x01\n" +
"\x10AIProviderConfig\x12\x0e\n" + "\x10AIProviderConfig\x12\x0e\n" +
"\x02id\x18\x01 \x01(\tR\x02id\x12\x14\n" + "\x02id\x18\x01 \x01(\tR\x02id\x12\x14\n" +
"\x05title\x18\x02 \x01(\tR\x05title\x12/\n" + "\x05title\x18\x02 \x01(\tR\x05title\x12/\n" +
"\x04type\x18\x03 \x01(\x0e2\x1b.memos.store.AIProviderTypeR\x04type\x12\x1a\n" + "\x04type\x18\x03 \x01(\x0e2\x1b.memos.store.AIProviderTypeR\x04type\x12\x1a\n" +
"\bendpoint\x18\x04 \x01(\tR\bendpoint\x12\x17\n" + "\bendpoint\x18\x04 \x01(\tR\bendpoint\x12\x17\n" +
"\aapi_key\x18\x05 \x01(\tR\x06apiKey*\x95\x01\n" + "\aapi_key\x18\x05 \x01(\tR\x06apiKey\"\x80\x01\n" +
"\x13TranscriptionConfig\x12\x1f\n" +
"\vprovider_id\x18\x01 \x01(\tR\n" +
"providerId\x12\x14\n" +
"\x05model\x18\x02 \x01(\tR\x05model\x12\x1a\n" +
"\blanguage\x18\x03 \x01(\tR\blanguage\x12\x16\n" +
"\x06prompt\x18\x04 \x01(\tR\x06prompt*\x95\x01\n" +
"\x12InstanceSettingKey\x12$\n" + "\x12InstanceSettingKey\x12$\n" +
" INSTANCE_SETTING_KEY_UNSPECIFIED\x10\x00\x12\t\n" + " INSTANCE_SETTING_KEY_UNSPECIFIED\x10\x00\x12\t\n" +
"\x05BASIC\x10\x01\x12\v\n" + "\x05BASIC\x10\x01\x12\v\n" +
@ -1312,7 +1408,7 @@ func file_store_instance_setting_proto_rawDescGZIP() []byte {
} }
var file_store_instance_setting_proto_enumTypes = make([]protoimpl.EnumInfo, 3) var file_store_instance_setting_proto_enumTypes = make([]protoimpl.EnumInfo, 3)
var file_store_instance_setting_proto_msgTypes = make([]protoimpl.MessageInfo, 14) var file_store_instance_setting_proto_msgTypes = make([]protoimpl.MessageInfo, 15)
var file_store_instance_setting_proto_goTypes = []any{ var file_store_instance_setting_proto_goTypes = []any{
(InstanceSettingKey)(0), // 0: memos.store.InstanceSettingKey (InstanceSettingKey)(0), // 0: memos.store.InstanceSettingKey
(AIProviderType)(0), // 1: memos.store.AIProviderType (AIProviderType)(0), // 1: memos.store.AIProviderType
@ -1329,9 +1425,10 @@ var file_store_instance_setting_proto_goTypes = []any{
(*InstanceNotificationSetting)(nil), // 12: memos.store.InstanceNotificationSetting (*InstanceNotificationSetting)(nil), // 12: memos.store.InstanceNotificationSetting
(*InstanceAISetting)(nil), // 13: memos.store.InstanceAISetting (*InstanceAISetting)(nil), // 13: memos.store.InstanceAISetting
(*AIProviderConfig)(nil), // 14: memos.store.AIProviderConfig (*AIProviderConfig)(nil), // 14: memos.store.AIProviderConfig
nil, // 15: memos.store.InstanceTagsSetting.TagsEntry (*TranscriptionConfig)(nil), // 15: memos.store.TranscriptionConfig
(*InstanceNotificationSetting_EmailSetting)(nil), // 16: memos.store.InstanceNotificationSetting.EmailSetting nil, // 16: memos.store.InstanceTagsSetting.TagsEntry
(*color.Color)(nil), // 17: google.type.Color (*InstanceNotificationSetting_EmailSetting)(nil), // 17: memos.store.InstanceNotificationSetting.EmailSetting
(*color.Color)(nil), // 18: google.type.Color
} }
var file_store_instance_setting_proto_depIdxs = []int32{ var file_store_instance_setting_proto_depIdxs = []int32{
0, // 0: memos.store.InstanceSetting.key:type_name -> memos.store.InstanceSettingKey 0, // 0: memos.store.InstanceSetting.key:type_name -> memos.store.InstanceSettingKey
@ -1345,17 +1442,18 @@ var file_store_instance_setting_proto_depIdxs = []int32{
6, // 8: memos.store.InstanceGeneralSetting.custom_profile:type_name -> memos.store.InstanceCustomProfile 6, // 8: memos.store.InstanceGeneralSetting.custom_profile:type_name -> memos.store.InstanceCustomProfile
2, // 9: memos.store.InstanceStorageSetting.storage_type:type_name -> memos.store.InstanceStorageSetting.StorageType 2, // 9: memos.store.InstanceStorageSetting.storage_type:type_name -> memos.store.InstanceStorageSetting.StorageType
8, // 10: memos.store.InstanceStorageSetting.s3_config:type_name -> memos.store.StorageS3Config 8, // 10: memos.store.InstanceStorageSetting.s3_config:type_name -> memos.store.StorageS3Config
17, // 11: memos.store.InstanceTagMetadata.background_color:type_name -> google.type.Color 18, // 11: memos.store.InstanceTagMetadata.background_color:type_name -> google.type.Color
15, // 12: memos.store.InstanceTagsSetting.tags:type_name -> memos.store.InstanceTagsSetting.TagsEntry 16, // 12: memos.store.InstanceTagsSetting.tags:type_name -> memos.store.InstanceTagsSetting.TagsEntry
16, // 13: memos.store.InstanceNotificationSetting.email:type_name -> memos.store.InstanceNotificationSetting.EmailSetting 17, // 13: memos.store.InstanceNotificationSetting.email:type_name -> memos.store.InstanceNotificationSetting.EmailSetting
14, // 14: memos.store.InstanceAISetting.providers:type_name -> memos.store.AIProviderConfig 14, // 14: memos.store.InstanceAISetting.providers:type_name -> memos.store.AIProviderConfig
1, // 15: memos.store.AIProviderConfig.type:type_name -> memos.store.AIProviderType 15, // 15: memos.store.InstanceAISetting.transcription:type_name -> memos.store.TranscriptionConfig
10, // 16: memos.store.InstanceTagsSetting.TagsEntry.value:type_name -> memos.store.InstanceTagMetadata 1, // 16: memos.store.AIProviderConfig.type:type_name -> memos.store.AIProviderType
17, // [17:17] is the sub-list for method output_type 10, // 17: memos.store.InstanceTagsSetting.TagsEntry.value:type_name -> memos.store.InstanceTagMetadata
17, // [17:17] is the sub-list for method input_type 18, // [18:18] is the sub-list for method output_type
17, // [17:17] is the sub-list for extension type_name 18, // [18:18] is the sub-list for method input_type
17, // [17:17] is the sub-list for extension extendee 18, // [18:18] is the sub-list for extension type_name
0, // [0:17] is the sub-list for field type_name 18, // [18:18] is the sub-list for extension extendee
0, // [0:18] is the sub-list for field type_name
} }
func init() { file_store_instance_setting_proto_init() } func init() { file_store_instance_setting_proto_init() }
@ -1378,7 +1476,7 @@ func file_store_instance_setting_proto_init() {
GoPackagePath: reflect.TypeOf(x{}).PkgPath(), GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_store_instance_setting_proto_rawDesc), len(file_store_instance_setting_proto_rawDesc)), RawDescriptor: unsafe.Slice(unsafe.StringData(file_store_instance_setting_proto_rawDesc), len(file_store_instance_setting_proto_rawDesc)),
NumEnums: 3, NumEnums: 3,
NumMessages: 14, NumMessages: 15,
NumExtensions: 0, NumExtensions: 0,
NumServices: 0, NumServices: 0,
}, },

View file

@ -149,6 +149,10 @@ message InstanceNotificationSetting {
message InstanceAISetting { message InstanceAISetting {
// providers is the list of AI provider configurations available instance-wide. // providers is the list of AI provider configurations available instance-wide.
repeated AIProviderConfig providers = 1; repeated AIProviderConfig providers = 1;
// transcription is the speech-to-text feature configuration.
// When unset or transcription.provider_id is empty, transcription is disabled.
TranscriptionConfig transcription = 2;
} }
message AIProviderConfig { message AIProviderConfig {
@ -165,3 +169,24 @@ enum AIProviderType {
OPENAI = 1; OPENAI = 1;
GEMINI = 2; GEMINI = 2;
} }
// TranscriptionConfig configures the speech-to-text feature.
message TranscriptionConfig {
// provider_id references an entry in InstanceAISetting.providers[].id.
// Empty string means transcription is disabled.
string provider_id = 1;
// model is the provider-specific model identifier.
// Empty string falls back to the engine default
// (whisper-1 for OPENAI providers, gemini-2.5-flash for GEMINI providers).
string model = 2;
// language is the default ISO 639-1 language hint sent to the provider.
// Empty string lets the provider auto-detect.
string language = 3;
// prompt is a default spelling/vocabulary hint passed to the provider.
// Used as the OpenAI Whisper "prompt" parameter and folded into the Gemini
// generation prompt as a "Context and spelling hints" block.
string prompt = 4;
}

View file

@ -17,8 +17,6 @@ import (
const ( const (
maxTranscriptionAudioSizeBytes = 25 * MebiByte maxTranscriptionAudioSizeBytes = 25 * MebiByte
maxTranscriptionPromptLength = 4096
maxTranscriptionLanguageLength = 32
maxTranscriptionFilenameLength = 255 maxTranscriptionFilenameLength = 255
) )
@ -51,20 +49,6 @@ func (s *APIV1Service) Transcribe(ctx context.Context, request *v1pb.TranscribeR
return nil, status.Errorf(codes.Unauthenticated, "user not authenticated") return nil, status.Errorf(codes.Unauthenticated, "user not authenticated")
} }
if strings.TrimSpace(request.ProviderId) == "" {
return nil, status.Errorf(codes.InvalidArgument, "provider_id is required")
}
if request.Config == nil {
return nil, status.Errorf(codes.InvalidArgument, "config is required")
}
prompt := strings.TrimSpace(request.Config.GetPrompt())
if len(prompt) > maxTranscriptionPromptLength {
return nil, status.Errorf(codes.InvalidArgument, "prompt is too long; maximum length is %d characters", maxTranscriptionPromptLength)
}
language := strings.TrimSpace(request.Config.GetLanguage())
if len(language) > maxTranscriptionLanguageLength {
return nil, status.Errorf(codes.InvalidArgument, "language is too long; maximum length is %d characters", maxTranscriptionLanguageLength)
}
if request.Audio == nil { if request.Audio == nil {
return nil, status.Errorf(codes.InvalidArgument, "audio is required") return nil, status.Errorf(codes.InvalidArgument, "audio is required")
} }
@ -90,10 +74,31 @@ func (s *APIV1Service) Transcribe(ctx context.Context, request *v1pb.TranscribeR
return nil, status.Errorf(codes.InvalidArgument, "audio content type %q is not supported", contentType) return nil, status.Errorf(codes.InvalidArgument, "audio content type %q is not supported", contentType)
} }
provider, model, err := s.resolveAIProviderForTranscription(ctx, request.ProviderId) aiSetting, err := s.Store.GetInstanceAISetting(ctx)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to get AI setting: %v", err)
}
persisted := aiSetting.GetTranscription()
providerID := persisted.GetProviderId()
if providerID == "" {
return nil, status.Errorf(codes.FailedPrecondition, "transcription is not configured")
}
provider, err := s.resolveAIProvider(aiSetting, providerID)
if err != nil { if err != nil {
return nil, err return nil, err
} }
model := persisted.GetModel()
if model == "" {
defaultModel, err := ai.DefaultTranscriptionModel(provider.Type)
if err != nil {
return nil, status.Errorf(codes.InvalidArgument, "%v", err)
}
model = defaultModel
}
transcriber, err := ai.NewTranscriber(provider) transcriber, err := ai.NewTranscriber(provider)
if err != nil { if err != nil {
return nil, status.Errorf(codes.InvalidArgument, "failed to create AI transcriber: %v", err) return nil, status.Errorf(codes.InvalidArgument, "failed to create AI transcriber: %v", err)
@ -105,8 +110,8 @@ func (s *APIV1Service) Transcribe(ctx context.Context, request *v1pb.TranscribeR
ContentType: contentType, ContentType: contentType,
Audio: bytes.NewReader(content), Audio: bytes.NewReader(content),
Size: int64(len(content)), Size: int64(len(content)),
Prompt: prompt, Prompt: persisted.GetPrompt(),
Language: language, Language: persisted.GetLanguage(),
}) })
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "failed to transcribe audio: %v", err) return nil, status.Errorf(codes.Internal, "failed to transcribe audio: %v", err)
@ -116,12 +121,7 @@ func (s *APIV1Service) Transcribe(ctx context.Context, request *v1pb.TranscribeR
}, nil }, nil
} }
func (s *APIV1Service) resolveAIProviderForTranscription(ctx context.Context, providerID string) (ai.ProviderConfig, string, error) { func (*APIV1Service) resolveAIProvider(setting *storepb.InstanceAISetting, providerID string) (ai.ProviderConfig, error) {
setting, err := s.Store.GetInstanceAISetting(ctx)
if err != nil {
return ai.ProviderConfig{}, "", status.Errorf(codes.Internal, "failed to get AI setting: %v", err)
}
providers := make([]ai.ProviderConfig, 0, len(setting.GetProviders())) providers := make([]ai.ProviderConfig, 0, len(setting.GetProviders()))
for _, provider := range setting.GetProviders() { for _, provider := range setting.GetProviders() {
if provider == nil { if provider == nil {
@ -132,13 +132,9 @@ func (s *APIV1Service) resolveAIProviderForTranscription(ctx context.Context, pr
provider, err := ai.FindProvider(providers, providerID) provider, err := ai.FindProvider(providers, providerID)
if err != nil { if err != nil {
return ai.ProviderConfig{}, "", status.Errorf(codes.NotFound, "AI provider not found") return ai.ProviderConfig{}, status.Errorf(codes.FailedPrecondition, "transcription provider is not configured")
} }
selectedModel, err := ai.DefaultTranscriptionModel(provider.Type) return *provider, nil
if err != nil {
return ai.ProviderConfig{}, "", status.Errorf(codes.InvalidArgument, "%v", err)
}
return *provider, selectedModel, nil
} }
func convertAIProviderConfigFromStore(provider *storepb.AIProviderConfig) ai.ProviderConfig { func convertAIProviderConfigFromStore(provider *storepb.AIProviderConfig) ai.ProviderConfig {

View file

@ -20,6 +20,12 @@ import (
"github.com/usememos/memos/store" "github.com/usememos/memos/store"
) )
const (
maxTranscriptionConfigModelLength = 256
maxTranscriptionConfigLanguageLength = 32
maxTranscriptionConfigPromptLength = 4096
)
// GetInstanceProfile returns the instance profile. // GetInstanceProfile returns the instance profile.
func (s *APIV1Service) GetInstanceProfile(ctx context.Context, _ *v1pb.GetInstanceProfileRequest) (*v1pb.InstanceProfile, error) { func (s *APIV1Service) GetInstanceProfile(ctx context.Context, _ *v1pb.GetInstanceProfileRequest) (*v1pb.InstanceProfile, error) {
admin, err := s.GetInstanceAdmin(ctx) admin, err := s.GetInstanceAdmin(ctx)
@ -91,6 +97,7 @@ func (s *APIV1Service) GetInstanceSetting(ctx context.Context, request *v1pb.Get
return nil, status.Errorf(codes.PermissionDenied, "permission denied") return nil, status.Errorf(codes.PermissionDenied, "permission denied")
} }
} }
isAdminCaller := false
if instanceSetting.Key == storepb.InstanceSettingKey_AI { if instanceSetting.Key == storepb.InstanceSettingKey_AI {
user, err := s.fetchCurrentUser(ctx) user, err := s.fetchCurrentUser(ctx)
if err != nil { if err != nil {
@ -99,9 +106,22 @@ func (s *APIV1Service) GetInstanceSetting(ctx context.Context, request *v1pb.Get
if user == nil { if user == nil {
return nil, status.Errorf(codes.Unauthenticated, "user not authenticated") return nil, status.Errorf(codes.Unauthenticated, "user not authenticated")
} }
isAdminCaller = user.Role == store.RoleAdmin
} }
return convertInstanceSettingFromStore(instanceSetting), nil result := convertInstanceSettingFromStore(instanceSetting)
if instanceSetting.Key == storepb.InstanceSettingKey_AI && !isAdminCaller {
// Non-admin callers only need transcription.provider_id to gate the
// editor's Transcribe button. Model / language / prompt are
// admin-entered defaults that may contain proprietary glossary terms,
// so they are redacted from non-admin responses.
if ai := result.GetAiSetting(); ai != nil && ai.Transcription != nil {
ai.Transcription.Model = ""
ai.Transcription.Language = ""
ai.Transcription.Prompt = ""
}
}
return result, nil
} }
func (s *APIV1Service) UpdateInstanceSetting(ctx context.Context, request *v1pb.UpdateInstanceSettingRequest) (*v1pb.InstanceSetting, error) { func (s *APIV1Service) UpdateInstanceSetting(ctx context.Context, request *v1pb.UpdateInstanceSettingRequest) (*v1pb.InstanceSetting, error) {
@ -509,6 +529,7 @@ func convertInstanceAISettingFromStore(setting *storepb.InstanceAISetting) *v1pb
aiSetting := &v1pb.InstanceSetting_AISetting{ aiSetting := &v1pb.InstanceSetting_AISetting{
Providers: make([]*v1pb.InstanceSetting_AIProviderConfig, 0, len(setting.Providers)), Providers: make([]*v1pb.InstanceSetting_AIProviderConfig, 0, len(setting.Providers)),
Transcription: convertTranscriptionConfigFromStore(setting.GetTranscription()),
} }
for _, provider := range setting.Providers { for _, provider := range setting.Providers {
if provider == nil { if provider == nil {
@ -534,6 +555,7 @@ func convertInstanceAISettingToStore(setting *v1pb.InstanceSetting_AISetting) *s
aiSetting := &storepb.InstanceAISetting{ aiSetting := &storepb.InstanceAISetting{
Providers: make([]*storepb.AIProviderConfig, 0, len(setting.Providers)), Providers: make([]*storepb.AIProviderConfig, 0, len(setting.Providers)),
Transcription: convertTranscriptionConfigToStore(setting.GetTranscription()),
} }
for _, provider := range setting.Providers { for _, provider := range setting.Providers {
if provider == nil { if provider == nil {
@ -550,6 +572,30 @@ func convertInstanceAISettingToStore(setting *v1pb.InstanceSetting_AISetting) *s
return aiSetting return aiSetting
} }
func convertTranscriptionConfigFromStore(setting *storepb.TranscriptionConfig) *v1pb.InstanceSetting_TranscriptionConfig {
if setting == nil {
return nil
}
return &v1pb.InstanceSetting_TranscriptionConfig{
ProviderId: setting.GetProviderId(),
Model: setting.GetModel(),
Language: setting.GetLanguage(),
Prompt: setting.GetPrompt(),
}
}
func convertTranscriptionConfigToStore(setting *v1pb.InstanceSetting_TranscriptionConfig) *storepb.TranscriptionConfig {
if setting == nil {
return nil
}
return &storepb.TranscriptionConfig{
ProviderId: setting.GetProviderId(),
Model: setting.GetModel(),
Language: setting.GetLanguage(),
Prompt: setting.GetPrompt(),
}
}
func validateInstanceSetting(setting *v1pb.InstanceSetting) error { func validateInstanceSetting(setting *v1pb.InstanceSetting) error {
key, err := ExtractInstanceSettingKeyFromName(setting.Name) key, err := ExtractInstanceSettingKeyFromName(setting.Name)
if err != nil { if err != nil {
@ -619,6 +665,53 @@ func (s *APIV1Service) prepareInstanceAISettingForUpdate(ctx context.Context, se
return errors.Errorf("provider %q API key is required", provider.Id) return errors.Errorf("provider %q API key is required", provider.Id)
} }
} }
if err := preparePersistedTranscriptionConfig(setting, existing); err != nil {
return err
}
return nil
}
func preparePersistedTranscriptionConfig(setting *storepb.InstanceAISetting, existing *storepb.InstanceAISetting) error {
// Preserve the previously stored transcription config when the request omits it,
// matching the same "absence == keep" semantics used for API keys. The preserved
// config still falls through to validation below, so a stale provider_id is
// rejected if the same update removed or renamed its referenced provider.
if setting.Transcription == nil && existing != nil {
setting.Transcription = existing.GetTranscription()
}
if setting.Transcription == nil {
return nil
}
cfg := setting.Transcription
cfg.ProviderId = strings.TrimSpace(cfg.ProviderId)
cfg.Model = strings.TrimSpace(cfg.Model)
cfg.Language = strings.TrimSpace(cfg.Language)
cfg.Prompt = strings.TrimSpace(cfg.Prompt)
if cfg.ProviderId != "" {
referenced := false
for _, provider := range setting.Providers {
if provider != nil && provider.Id == cfg.ProviderId {
referenced = true
break
}
}
if !referenced {
return errors.Errorf("transcription provider_id %q does not reference any configured provider", cfg.ProviderId)
}
}
if len(cfg.Model) > maxTranscriptionConfigModelLength {
return errors.Errorf("transcription model is too long; maximum length is %d characters", maxTranscriptionConfigModelLength)
}
if len(cfg.Language) > maxTranscriptionConfigLanguageLength {
return errors.Errorf("transcription language is too long; maximum length is %d characters", maxTranscriptionConfigLanguageLength)
}
if len(cfg.Prompt) > maxTranscriptionConfigPromptLength {
return errors.Errorf("transcription prompt is too long; maximum length is %d characters", maxTranscriptionConfigPromptLength)
}
return nil return nil
} }

View file

@ -21,8 +21,6 @@ func TestTranscribe(t *testing.T) {
defer ts.Cleanup() defer ts.Cleanup()
_, err := ts.Service.Transcribe(ctx, &v1pb.TranscribeRequest{ _, err := ts.Service.Transcribe(ctx, &v1pb.TranscribeRequest{
ProviderId: "openai-main",
Config: &v1pb.TranscriptionConfig{},
Audio: &v1pb.TranscriptionAudio{ Audio: &v1pb.TranscriptionAudio{
Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")}, Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")},
Filename: "voice.wav", Filename: "voice.wav",
@ -33,7 +31,7 @@ func TestTranscribe(t *testing.T) {
require.Contains(t, err.Error(), "user not authenticated") require.Contains(t, err.Error(), "user not authenticated")
}) })
t.Run("transcribes audio file with configured provider", func(t *testing.T) { t.Run("transcribes audio file using persisted transcription setting", func(t *testing.T) {
ts := NewTestService(t) ts := NewTestService(t)
defer ts.Cleanup() defer ts.Cleanup()
@ -45,7 +43,8 @@ func TestTranscribe(t *testing.T) {
require.Equal(t, "/audio/transcriptions", r.URL.Path) require.Equal(t, "/audio/transcriptions", r.URL.Path)
require.Equal(t, "Bearer sk-test", r.Header.Get("Authorization")) require.Equal(t, "Bearer sk-test", r.Header.Get("Authorization"))
require.NoError(t, r.ParseMultipartForm(10<<20)) require.NoError(t, r.ParseMultipartForm(10<<20))
require.Equal(t, "gpt-4o-transcribe", r.FormValue("model")) require.Equal(t, "whisper-1", r.FormValue("model"))
require.Equal(t, "fr", r.FormValue("language"))
require.Equal(t, "names: Alice", r.FormValue("prompt")) require.Equal(t, "names: Alice", r.FormValue("prompt"))
file, header, err := r.FormFile("file") file, header, err := r.FormFile("file")
@ -73,16 +72,18 @@ func TestTranscribe(t *testing.T) {
ApiKey: "sk-test", ApiKey: "sk-test",
}, },
}, },
Transcription: &storepb.TranscriptionConfig{
ProviderId: "openai-main",
Model: "whisper-1",
Language: "fr",
Prompt: "names: Alice",
},
}, },
}, },
}) })
require.NoError(t, err) require.NoError(t, err)
resp, err := ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{ resp, err := ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{
ProviderId: "openai-main",
Config: &v1pb.TranscriptionConfig{
Prompt: "names: Alice",
},
Audio: &v1pb.TranscriptionAudio{ Audio: &v1pb.TranscriptionAudio{
Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")}, Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")},
Filename: "voice.wav", Filename: "voice.wav",
@ -117,14 +118,15 @@ func TestTranscribe(t *testing.T) {
ApiKey: "sk-test", ApiKey: "sk-test",
}, },
}, },
Transcription: &storepb.TranscriptionConfig{
ProviderId: "openai-main",
},
}, },
}, },
}) })
require.NoError(t, err) require.NoError(t, err)
_, err = ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{ _, err = ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{
ProviderId: "openai-main",
Config: &v1pb.TranscriptionConfig{},
Audio: &v1pb.TranscriptionAudio{ Audio: &v1pb.TranscriptionAudio{
Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")}, Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")},
Filename: "voice.wav", Filename: "voice.wav",
@ -172,14 +174,15 @@ func TestTranscribe(t *testing.T) {
ApiKey: "gemini-key", ApiKey: "gemini-key",
}, },
}, },
Transcription: &storepb.TranscriptionConfig{
ProviderId: "gemini-main",
},
}, },
}, },
}) })
require.NoError(t, err) require.NoError(t, err)
resp, err := ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{ resp, err := ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{
ProviderId: "gemini-main",
Config: &v1pb.TranscriptionConfig{},
Audio: &v1pb.TranscriptionAudio{ Audio: &v1pb.TranscriptionAudio{
Source: &v1pb.TranscriptionAudio_Content{Content: []byte("mp3 bytes")}, Source: &v1pb.TranscriptionAudio_Content{Content: []byte("mp3 bytes")},
Filename: "voice.mp3", Filename: "voice.mp3",
@ -190,7 +193,7 @@ func TestTranscribe(t *testing.T) {
require.Equal(t, "gemini transcript", resp.Text) require.Equal(t, "gemini transcript", resp.Text)
}) })
t.Run("uses built-in transcription model", func(t *testing.T) { t.Run("falls back to engine default model when transcription model is empty", func(t *testing.T) {
ts := NewTestService(t) ts := NewTestService(t)
defer ts.Cleanup() defer ts.Cleanup()
@ -200,7 +203,7 @@ func TestTranscribe(t *testing.T) {
openAIServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { openAIServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
require.NoError(t, r.ParseMultipartForm(10<<20)) require.NoError(t, r.ParseMultipartForm(10<<20))
require.Equal(t, "gpt-4o-transcribe", r.FormValue("model")) require.Equal(t, "whisper-1", r.FormValue("model"))
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
require.NoError(t, json.NewEncoder(w).Encode(map[string]string{ require.NoError(t, json.NewEncoder(w).Encode(map[string]string{
"text": "built-in model", "text": "built-in model",
@ -221,14 +224,15 @@ func TestTranscribe(t *testing.T) {
ApiKey: "sk-test", ApiKey: "sk-test",
}, },
}, },
Transcription: &storepb.TranscriptionConfig{
ProviderId: "openai-main",
},
}, },
}, },
}) })
require.NoError(t, err) require.NoError(t, err)
resp, err := ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{ resp, err := ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{
ProviderId: "openai-main",
Config: &v1pb.TranscriptionConfig{},
Audio: &v1pb.TranscriptionAudio{ Audio: &v1pb.TranscriptionAudio{
Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")}, Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")},
Filename: "voice.wav", Filename: "voice.wav",
@ -247,27 +251,7 @@ func TestTranscribe(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
userCtx := ts.CreateUserContext(ctx, user.ID) userCtx := ts.CreateUserContext(ctx, user.ID)
_, err = ts.Store.UpsertInstanceSetting(ctx, &storepb.InstanceSetting{
Key: storepb.InstanceSettingKey_AI,
Value: &storepb.InstanceSetting_AiSetting{
AiSetting: &storepb.InstanceAISetting{
Providers: []*storepb.AIProviderConfig{
{
Id: "openai-main",
Title: "OpenAI",
Type: storepb.AIProviderType_OPENAI,
Endpoint: "https://example.com/v1",
ApiKey: "sk-test",
},
},
},
},
})
require.NoError(t, err)
_, err = ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{ _, err = ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{
ProviderId: "openai-main",
Config: &v1pb.TranscriptionConfig{},
Audio: &v1pb.TranscriptionAudio{ Audio: &v1pb.TranscriptionAudio{
Source: &v1pb.TranscriptionAudio_Content{Content: []byte("not audio")}, Source: &v1pb.TranscriptionAudio_Content{Content: []byte("not audio")},
Filename: "notes.txt", Filename: "notes.txt",
@ -277,4 +261,23 @@ func TestTranscribe(t *testing.T) {
require.Error(t, err) require.Error(t, err)
require.Contains(t, err.Error(), "not supported") require.Contains(t, err.Error(), "not supported")
}) })
t.Run("returns FailedPrecondition when transcription is not configured", func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()
user, err := ts.CreateRegularUser(ctx, "alice-empty")
require.NoError(t, err)
userCtx := ts.CreateUserContext(ctx, user.ID)
_, err = ts.Service.Transcribe(userCtx, &v1pb.TranscribeRequest{
Audio: &v1pb.TranscriptionAudio{
Source: &v1pb.TranscriptionAudio_Content{Content: []byte("RIFF")},
Filename: "voice.wav",
ContentType: "audio/wav",
},
})
require.Error(t, err)
require.Contains(t, err.Error(), "transcription is not configured")
})
} }

View file

@ -2,6 +2,7 @@ package test
import ( import (
"context" "context"
"strings"
"testing" "testing"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@ -731,4 +732,149 @@ func TestUpdateInstanceSetting(t *testing.T) {
"existing AI provider API key must be preserved when an empty value is sent") "existing AI provider API key must be preserved when an empty value is sent")
require.Equal(t, "OpenAI primary", stored.GetProviders()[0].GetTitle()) require.Equal(t, "OpenAI primary", stored.GetProviders()[0].GetTitle())
}) })
t.Run("UpdateInstanceSetting - transcription provider_id must reference an existing provider", func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()
hostUser, err := ts.CreateHostUser(ctx, "admin")
require.NoError(t, err)
adminCtx := ts.CreateUserContext(ctx, hostUser.ID)
_, err = ts.Service.UpdateInstanceSetting(adminCtx, &v1pb.UpdateInstanceSettingRequest{
Setting: &v1pb.InstanceSetting{
Name: "instance/settings/AI",
Value: &v1pb.InstanceSetting_AiSetting{
AiSetting: &v1pb.InstanceSetting_AISetting{
Providers: []*v1pb.InstanceSetting_AIProviderConfig{
{
Id: "openai-main",
Title: "OpenAI",
Type: v1pb.InstanceSetting_OPENAI,
ApiKey: "sk-test",
},
},
Transcription: &v1pb.InstanceSetting_TranscriptionConfig{
ProviderId: "does-not-exist",
},
},
},
},
})
require.Error(t, err)
require.Contains(t, err.Error(), "transcription provider_id")
})
t.Run("UpdateInstanceSetting - transcription strings are length-capped", func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()
hostUser, err := ts.CreateHostUser(ctx, "admin")
require.NoError(t, err)
adminCtx := ts.CreateUserContext(ctx, hostUser.ID)
base := &v1pb.InstanceSetting{
Name: "instance/settings/AI",
Value: &v1pb.InstanceSetting_AiSetting{
AiSetting: &v1pb.InstanceSetting_AISetting{
Providers: []*v1pb.InstanceSetting_AIProviderConfig{
{
Id: "openai-main",
Title: "OpenAI",
Type: v1pb.InstanceSetting_OPENAI,
ApiKey: "sk-test",
},
},
},
},
}
oversizedModel := strings.Repeat("a", 257)
base.GetAiSetting().Transcription = &v1pb.InstanceSetting_TranscriptionConfig{
ProviderId: "openai-main",
Model: oversizedModel,
}
_, err = ts.Service.UpdateInstanceSetting(adminCtx, &v1pb.UpdateInstanceSettingRequest{Setting: base})
require.Error(t, err)
require.Contains(t, err.Error(), "transcription model")
oversizedLanguage := strings.Repeat("a", 33)
base.GetAiSetting().Transcription = &v1pb.InstanceSetting_TranscriptionConfig{
ProviderId: "openai-main",
Language: oversizedLanguage,
}
_, err = ts.Service.UpdateInstanceSetting(adminCtx, &v1pb.UpdateInstanceSettingRequest{Setting: base})
require.Error(t, err)
require.Contains(t, err.Error(), "transcription language")
oversizedPrompt := strings.Repeat("a", 4097)
base.GetAiSetting().Transcription = &v1pb.InstanceSetting_TranscriptionConfig{
ProviderId: "openai-main",
Prompt: oversizedPrompt,
}
_, err = ts.Service.UpdateInstanceSetting(adminCtx, &v1pb.UpdateInstanceSettingRequest{Setting: base})
require.Error(t, err)
require.Contains(t, err.Error(), "transcription prompt")
})
t.Run("UpdateInstanceSetting - transcription is preserved when omitted on update", func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()
hostUser, err := ts.CreateHostUser(ctx, "admin")
require.NoError(t, err)
adminCtx := ts.CreateUserContext(ctx, hostUser.ID)
_, err = ts.Service.UpdateInstanceSetting(adminCtx, &v1pb.UpdateInstanceSettingRequest{
Setting: &v1pb.InstanceSetting{
Name: "instance/settings/AI",
Value: &v1pb.InstanceSetting_AiSetting{
AiSetting: &v1pb.InstanceSetting_AISetting{
Providers: []*v1pb.InstanceSetting_AIProviderConfig{
{
Id: "openai-main",
Title: "OpenAI",
Type: v1pb.InstanceSetting_OPENAI,
ApiKey: "sk-test",
},
},
Transcription: &v1pb.InstanceSetting_TranscriptionConfig{
ProviderId: "openai-main",
Model: "whisper-1",
Language: "en",
Prompt: "names: Alice",
},
},
},
},
})
require.NoError(t, err)
_, err = ts.Service.UpdateInstanceSetting(adminCtx, &v1pb.UpdateInstanceSettingRequest{
Setting: &v1pb.InstanceSetting{
Name: "instance/settings/AI",
Value: &v1pb.InstanceSetting_AiSetting{
AiSetting: &v1pb.InstanceSetting_AISetting{
Providers: []*v1pb.InstanceSetting_AIProviderConfig{
{
Id: "openai-main",
Title: "OpenAI",
Type: v1pb.InstanceSetting_OPENAI,
ApiKey: "",
},
},
},
},
},
})
require.NoError(t, err)
stored, err := ts.Store.GetInstanceAISetting(ctx)
require.NoError(t, err)
require.NotNil(t, stored.GetTranscription())
require.Equal(t, "openai-main", stored.GetTranscription().GetProviderId())
require.Equal(t, "whisper-1", stored.GetTranscription().GetModel())
require.Equal(t, "en", stored.GetTranscription().GetLanguage())
require.Equal(t, "names: Alice", stored.GetTranscription().GetPrompt())
})
} }

View file

@ -8,7 +8,7 @@ import { memoKeys } from "@/hooks/useMemoQueries";
import { userKeys } from "@/hooks/useUserQueries"; import { userKeys } from "@/hooks/useUserQueries";
import { handleError } from "@/lib/error"; import { handleError } from "@/lib/error";
import { cn } from "@/lib/utils"; import { cn } from "@/lib/utils";
import { InstanceSetting_AIProviderType, InstanceSetting_Key } from "@/types/proto/api/v1/instance_service_pb"; import { InstanceSetting_Key } from "@/types/proto/api/v1/instance_service_pb";
import { useTranslate } from "@/utils/i18n"; import { useTranslate } from "@/utils/i18n";
import { convertVisibilityFromString } from "@/utils/memo"; import { convertVisibilityFromString } from "@/utils/memo";
import { import {
@ -28,11 +28,6 @@ import { EditorProvider, useEditorContext } from "./state";
import type { MemoEditorProps } from "./types"; import type { MemoEditorProps } from "./types";
import type { LocalFile } from "./types/attachment"; import type { LocalFile } from "./types/attachment";
const TRANSCRIPTION_PROVIDER_TYPES: InstanceSetting_AIProviderType[] = [
InstanceSetting_AIProviderType.OPENAI,
InstanceSetting_AIProviderType.GEMINI,
];
const MemoEditor = (props: MemoEditorProps) => ( const MemoEditor = (props: MemoEditorProps) => (
<EditorProvider> <EditorProvider>
<MemoEditorImpl {...props} /> <MemoEditorImpl {...props} />
@ -61,10 +56,12 @@ const MemoEditorImpl: React.FC<MemoEditorProps> = ({
const [isTranscribingAudio, setIsTranscribingAudio] = useState(false); const [isTranscribingAudio, setIsTranscribingAudio] = useState(false);
const memoName = memo?.name; const memoName = memo?.name;
const transcriptionProvider = useMemo( const canTranscribe = useMemo(() => {
() => aiSetting.providers.find((provider) => provider.apiKeySet && TRANSCRIPTION_PROVIDER_TYPES.includes(provider.type)), const providerId = aiSetting.transcription?.providerId ?? "";
[aiSetting.providers], if (!providerId) return false;
); const provider = aiSetting.providers.find((p) => p.id === providerId);
return Boolean(provider?.apiKeySet);
}, [aiSetting.providers, aiSetting.transcription?.providerId]);
// Get default visibility from user settings // Get default visibility from user settings
const defaultVisibility = userGeneralSetting?.memoVisibility ? convertVisibilityFromString(userGeneralSetting.memoVisibility) : undefined; const defaultVisibility = userGeneralSetting?.memoVisibility ? convertVisibilityFromString(userGeneralSetting.memoVisibility) : undefined;
@ -129,7 +126,7 @@ const MemoEditorImpl: React.FC<MemoEditorProps> = ({
const handleTranscribeRecordedAudio = useCallback( const handleTranscribeRecordedAudio = useCallback(
async (localFile: LocalFile) => { async (localFile: LocalFile) => {
if (!transcriptionProvider) { if (!canTranscribe) {
dispatch(actions.addLocalFile(localFile)); dispatch(actions.addLocalFile(localFile));
setIsTranscribingAudio(false); setIsTranscribingAudio(false);
setIsAudioRecorderOpen(false); setIsAudioRecorderOpen(false);
@ -137,7 +134,7 @@ const MemoEditorImpl: React.FC<MemoEditorProps> = ({
} }
try { try {
const text = (await transcriptionService.transcribeFile(localFile.file, transcriptionProvider)).trim(); const text = (await transcriptionService.transcribeFile(localFile.file)).trim();
if (!text) { if (!text) {
dispatch(actions.addLocalFile(localFile)); dispatch(actions.addLocalFile(localFile));
toast.error(t("editor.audio-recorder.transcribe-empty")); toast.error(t("editor.audio-recorder.transcribe-empty"));
@ -155,7 +152,7 @@ const MemoEditorImpl: React.FC<MemoEditorProps> = ({
setIsAudioRecorderOpen(false); setIsAudioRecorderOpen(false);
} }
}, },
[actions, dispatch, insertTranscribedText, t, transcriptionProvider], [actions, canTranscribe, dispatch, insertTranscribedText, t],
); );
const audioRecorderActions = useMemo( const audioRecorderActions = useMemo(
@ -223,7 +220,7 @@ const MemoEditorImpl: React.FC<MemoEditorProps> = ({
}; };
const handleTranscribeAudioRecording = () => { const handleTranscribeAudioRecording = () => {
if (!transcriptionProvider || isTranscribingAudio) { if (!canTranscribe || isTranscribingAudio) {
return; return;
} }
@ -340,7 +337,7 @@ const MemoEditorImpl: React.FC<MemoEditorProps> = ({
onStop={audioRecorder.stopRecording} onStop={audioRecorder.stopRecording}
onCancel={handleCancelAudioRecording} onCancel={handleCancelAudioRecording}
onTranscribe={handleTranscribeAudioRecording} onTranscribe={handleTranscribeAudioRecording}
canTranscribe={!!transcriptionProvider} canTranscribe={canTranscribe}
isTranscribing={isTranscribingAudio} isTranscribing={isTranscribingAudio}
/> />
)} )}

View file

@ -1,15 +1,12 @@
import { create } from "@bufbuild/protobuf"; import { create } from "@bufbuild/protobuf";
import { aiServiceClient } from "@/connect"; import { aiServiceClient } from "@/connect";
import { TranscribeRequestSchema, TranscriptionAudioSchema, TranscriptionConfigSchema } from "@/types/proto/api/v1/ai_service_pb"; import { TranscribeRequestSchema, TranscriptionAudioSchema } from "@/types/proto/api/v1/ai_service_pb";
import type { InstanceSetting_AIProviderConfig } from "@/types/proto/api/v1/instance_service_pb";
export const transcriptionService = { export const transcriptionService = {
async transcribeFile(file: File, provider: InstanceSetting_AIProviderConfig): Promise<string> { async transcribeFile(file: File): Promise<string> {
const content = new Uint8Array(await file.arrayBuffer()); const content = new Uint8Array(await file.arrayBuffer());
const response = await aiServiceClient.transcribe( const response = await aiServiceClient.transcribe(
create(TranscribeRequestSchema, { create(TranscribeRequestSchema, {
providerId: provider.id,
config: create(TranscriptionConfigSchema, {}),
audio: create(TranscriptionAudioSchema, { audio: create(TranscriptionAudioSchema, {
source: { source: {
case: "content", case: "content",

View file

@ -1,7 +1,7 @@
import { create } from "@bufbuild/protobuf"; import { create } from "@bufbuild/protobuf";
import { isEqual } from "lodash-es"; import { isEqual } from "lodash-es";
import { MoreVerticalIcon, PlusIcon } from "lucide-react"; import { MoreVerticalIcon, PlusIcon } from "lucide-react";
import { useEffect, useMemo, useState } from "react"; import { useEffect, useMemo, useRef, useState } from "react";
import { toast } from "react-hot-toast"; import { toast } from "react-hot-toast";
import ConfirmDialog from "@/components/ConfirmDialog"; import ConfirmDialog from "@/components/ConfirmDialog";
import { Button } from "@/components/ui/button"; import { Button } from "@/components/ui/button";
@ -10,6 +10,7 @@ import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, DropdownMenuTrigge
import { Input } from "@/components/ui/input"; import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label"; import { Label } from "@/components/ui/label";
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select"; import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
import { Textarea } from "@/components/ui/textarea";
import { useInstance } from "@/contexts/InstanceContext"; import { useInstance } from "@/contexts/InstanceContext";
import { import {
InstanceSetting_AIProviderConfig, InstanceSetting_AIProviderConfig,
@ -17,6 +18,8 @@ import {
InstanceSetting_AIProviderType, InstanceSetting_AIProviderType,
InstanceSetting_AISettingSchema, InstanceSetting_AISettingSchema,
InstanceSetting_Key, InstanceSetting_Key,
InstanceSetting_TranscriptionConfig,
InstanceSetting_TranscriptionConfigSchema,
InstanceSettingSchema, InstanceSettingSchema,
} from "@/types/proto/api/v1/instance_service_pb"; } from "@/types/proto/api/v1/instance_service_pb";
import { useTranslate } from "@/utils/i18n"; import { useTranslate } from "@/utils/i18n";
@ -36,6 +39,13 @@ type LocalAIProvider = {
apiKeyHint: string; apiKeyHint: string;
}; };
type LocalTranscription = {
providerId: string;
model: string;
language: string;
prompt: string;
};
const providerTypeOptions = [InstanceSetting_AIProviderType.OPENAI, InstanceSetting_AIProviderType.GEMINI]; const providerTypeOptions = [InstanceSetting_AIProviderType.OPENAI, InstanceSetting_AIProviderType.GEMINI];
const byokNotes = ["setting.ai.byok-key-note", "setting.ai.byok-storage-note", "setting.ai.byok-model-note"] as const; const byokNotes = ["setting.ai.byok-key-note", "setting.ai.byok-storage-note", "setting.ai.byok-model-note"] as const;
@ -61,6 +71,13 @@ const toLocalProvider = (provider: InstanceSetting_AIProviderConfig): LocalAIPro
apiKeyHint: provider.apiKeyHint, apiKeyHint: provider.apiKeyHint,
}); });
const toLocalTranscription = (config: InstanceSetting_TranscriptionConfig | undefined): LocalTranscription => ({
providerId: config?.providerId ?? "",
model: config?.model ?? "",
language: config?.language ?? "",
prompt: config?.prompt ?? "",
});
const newProvider = (): LocalAIProvider => ({ const newProvider = (): LocalAIProvider => ({
id: createProviderID(), id: createProviderID(),
title: "", title: "",
@ -80,11 +97,20 @@ const toProviderConfig = (provider: LocalAIProvider) =>
apiKey: provider.apiKey, apiKey: provider.apiKey,
}); });
const toTranscriptionConfig = (transcription: LocalTranscription) =>
create(InstanceSetting_TranscriptionConfigSchema, {
providerId: transcription.providerId,
model: transcription.model.trim(),
language: transcription.language.trim(),
prompt: transcription.prompt,
});
const AISection = () => { const AISection = () => {
const t = useTranslate(); const t = useTranslate();
const saveInstanceSetting = useInstanceSettingUpdater(); const saveInstanceSetting = useInstanceSettingUpdater();
const { aiSetting: originalSetting } = useInstance(); const { aiSetting: originalSetting } = useInstance();
const [providers, setProviders] = useState<LocalAIProvider[]>(() => originalSetting.providers.map(toLocalProvider)); const [providers, setProviders] = useState<LocalAIProvider[]>(() => originalSetting.providers.map(toLocalProvider));
const [transcription, setTranscription] = useState<LocalTranscription>(() => toLocalTranscription(originalSetting.transcription));
const [editingProvider, setEditingProvider] = useState<LocalAIProvider | undefined>(); const [editingProvider, setEditingProvider] = useState<LocalAIProvider | undefined>();
const [deleteTarget, setDeleteTarget] = useState<LocalAIProvider | undefined>(); const [deleteTarget, setDeleteTarget] = useState<LocalAIProvider | undefined>();
@ -92,8 +118,50 @@ const AISection = () => {
setProviders(originalSetting.providers.map(toLocalProvider)); setProviders(originalSetting.providers.map(toLocalProvider));
}, [originalSetting.providers]); }, [originalSetting.providers]);
const originalProviders = useMemo(() => originalSetting.providers.map(toLocalProvider), [originalSetting.providers]); // Only re-sync the transcription draft when the server-side content actually
const hasChanges = !isEqual(providers, originalProviders); // changes — not on every originalSetting identity change. This prevents
// provider-side saves (which keep transcription unchanged on the server) from
// wiping an in-progress transcription draft.
const lastSyncedTranscription = useRef<LocalTranscription>(toLocalTranscription(originalSetting.transcription));
useEffect(() => {
const next = toLocalTranscription(originalSetting.transcription);
if (!isEqual(lastSyncedTranscription.current, next)) {
setTranscription(next);
lastSyncedTranscription.current = next;
}
}, [originalSetting.transcription]);
const originalTranscription = useMemo(() => toLocalTranscription(originalSetting.transcription), [originalSetting.transcription]);
const transcriptionHasChanges = !isEqual(transcription, originalTranscription);
const transcriptionProviderRef = useMemo(
() => providers.find((provider) => provider.id === transcription.providerId),
[providers, transcription.providerId],
);
// Persists the AI setting using a specific providers list and transcription
// value. Provider operations pass originalSetting.transcription so an
// in-progress transcription draft is never accidentally committed.
const persistAISetting = async (
nextProviders: LocalAIProvider[],
nextTranscription: InstanceSetting_TranscriptionConfig | undefined,
errorContext: string,
) => {
return saveInstanceSetting({
key: InstanceSetting_Key.AI,
setting: create(InstanceSettingSchema, {
name: buildInstanceSettingName(InstanceSetting_Key.AI),
value: {
case: "aiSetting",
value: create(InstanceSetting_AISettingSchema, {
providers: nextProviders.map(toProviderConfig),
transcription: nextTranscription,
}),
},
}),
errorContext,
});
};
const handleCreateProvider = () => { const handleCreateProvider = () => {
setEditingProvider(newProvider()); setEditingProvider(newProvider());
@ -103,7 +171,7 @@ const AISection = () => {
setEditingProvider({ ...provider, apiKey: "" }); setEditingProvider({ ...provider, apiKey: "" });
}; };
const handleSaveProvider = (provider: LocalAIProvider) => { const handleSaveProvider = async (provider: LocalAIProvider) => {
const title = provider.title.trim(); const title = provider.title.trim();
const endpoint = provider.endpoint.trim(); const endpoint = provider.endpoint.trim();
@ -116,41 +184,47 @@ const AISection = () => {
return; return;
} }
const normalizedProvider = { const normalizedProvider = { ...provider, title, endpoint };
...provider, const exists = providers.some((item) => item.id === normalizedProvider.id);
title, const nextProviders = exists
endpoint, ? providers.map((item) => (item.id === normalizedProvider.id ? normalizedProvider : item))
}; : [...providers, normalizedProvider];
setProviders((prev) => {
const exists = prev.some((item) => item.id === normalizedProvider.id); const ok = await persistAISetting(nextProviders, originalSetting.transcription, "Update AI provider");
if (!exists) { if (!ok) return;
return [...prev, normalizedProvider]; setProviders(nextProviders);
}
return prev.map((item) => (item.id === normalizedProvider.id ? normalizedProvider : item));
});
setEditingProvider(undefined); setEditingProvider(undefined);
}; };
const handleDeleteProvider = () => { const handleDeleteProvider = async () => {
if (!deleteTarget) return; if (!deleteTarget) return;
setProviders((prev) => prev.filter((provider) => provider.id !== deleteTarget.id)); const target = deleteTarget;
const nextProviders = providers.filter((provider) => provider.id !== target.id);
// If the persisted transcription references the deleted provider, the
// server would reject the save (provider_id must reference an existing
// provider). Send a cleared transcription in that case.
const persistedTranscription = originalSetting.transcription;
const nextTranscription =
persistedTranscription && persistedTranscription.providerId === target.id
? create(InstanceSetting_TranscriptionConfigSchema, {})
: persistedTranscription;
const ok = await persistAISetting(nextProviders, nextTranscription, "Delete AI provider");
if (!ok) return;
setProviders(nextProviders);
if (transcription.providerId === target.id) {
setTranscription((prev) => ({ ...prev, providerId: "" }));
}
setDeleteTarget(undefined); setDeleteTarget(undefined);
}; };
const handleSaveSetting = async () => { const handleSaveTranscription = async () => {
await saveInstanceSetting({ if (transcription.providerId && !transcriptionProviderRef) {
key: InstanceSetting_Key.AI, toast.error(t("setting.ai.transcription-empty-providers"));
setting: create(InstanceSettingSchema, { return;
name: buildInstanceSettingName(InstanceSetting_Key.AI), }
value: { await persistAISetting(providers, toTranscriptionConfig(transcription), "Update transcription");
case: "aiSetting",
value: create(InstanceSetting_AISettingSchema, {
providers: providers.map(toProviderConfig),
}),
},
}),
errorContext: "Update AI providers",
});
}; };
return ( return (
@ -183,7 +257,7 @@ const AISection = () => {
</div> </div>
</SettingPanel> </SettingPanel>
<SettingGroup title={t("setting.ai.providers")} description={t("setting.ai.description")}> <SettingGroup title={t("setting.ai.integrations-title")} description={t("setting.ai.integrations-description")}>
<SettingTable <SettingTable
columns={[ columns={[
{ {
@ -242,11 +316,23 @@ const AISection = () => {
/> />
</SettingGroup> </SettingGroup>
<div className="w-full flex justify-end"> <SettingGroup
<Button disabled={!hasChanges} onClick={handleSaveSetting}> title={t("setting.ai.transcription-title")}
description={t("setting.ai.transcription-description")}
showSeparator
actions={
<Button disabled={!transcriptionHasChanges} onClick={handleSaveTranscription}>
{t("common.save")} {t("common.save")}
</Button> </Button>
</div> }
>
<TranscriptionForm
providers={providers}
transcription={transcription}
onChange={setTranscription}
referencedProvider={transcriptionProviderRef}
/>
</SettingGroup>
<AIProviderDialog <AIProviderDialog
provider={editingProvider} provider={editingProvider}
@ -267,6 +353,98 @@ const AISection = () => {
); );
}; };
interface TranscriptionFormProps {
providers: LocalAIProvider[];
transcription: LocalTranscription;
referencedProvider: LocalAIProvider | undefined;
onChange: (next: LocalTranscription) => void;
}
const TranscriptionForm = ({ providers, transcription, referencedProvider, onChange }: TranscriptionFormProps) => {
const t = useTranslate();
const noProviders = providers.length === 0;
const update = (partial: Partial<LocalTranscription>) => {
onChange({ ...transcription, ...partial });
};
const placeholderForProvider = (provider: LocalAIProvider | undefined) => {
if (!provider) return "";
return provider.type === InstanceSetting_AIProviderType.GEMINI
? t("setting.ai.transcription-model-placeholder-gemini")
: t("setting.ai.transcription-model-placeholder-openai");
};
return (
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 max-w-3xl">
<div className="flex flex-col gap-1.5 sm:col-span-2">
<Label>{t("setting.ai.transcription-provider")}</Label>
<Select
value={transcription.providerId || "__none__"}
onValueChange={(value) => update({ providerId: value === "__none__" ? "" : value })}
disabled={noProviders}
>
<SelectTrigger className="w-full">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="__none__">{t("setting.ai.transcription-no-provider")}</SelectItem>
{providers.map((provider) => (
<SelectItem key={provider.id} value={provider.id}>
{provider.title || provider.id}
</SelectItem>
))}
</SelectContent>
</Select>
{noProviders && <p className="text-xs text-muted-foreground">{t("setting.ai.transcription-empty-providers")}</p>}
{referencedProvider && !referencedProvider.apiKeySet && (
<p className="text-xs text-destructive">{t("setting.ai.transcription-warning-no-key")}</p>
)}
{referencedProvider?.type === InstanceSetting_AIProviderType.GEMINI && (
<p className="text-xs text-muted-foreground">{t("setting.ai.transcription-warning-gemini-webm")}</p>
)}
</div>
<div className="flex flex-col gap-1.5 sm:col-span-2">
<Label>{t("setting.ai.transcription-model")}</Label>
<Input
value={transcription.model}
onChange={(e) => update({ model: e.target.value })}
placeholder={placeholderForProvider(referencedProvider)}
disabled={!transcription.providerId}
maxLength={256}
/>
<p className="text-xs text-muted-foreground">{t("setting.ai.transcription-model-help")}</p>
</div>
<div className="flex flex-col gap-1.5">
<Label>{t("setting.ai.transcription-language")}</Label>
<Input
value={transcription.language}
onChange={(e) => update({ language: e.target.value })}
placeholder={t("setting.ai.transcription-language-placeholder")}
disabled={!transcription.providerId}
maxLength={32}
/>
<p className="text-xs text-muted-foreground">{t("setting.ai.transcription-language-help")}</p>
</div>
<div className="flex flex-col gap-1.5 sm:col-span-2">
<Label>{t("setting.ai.transcription-prompt")}</Label>
<Textarea
value={transcription.prompt}
onChange={(e) => update({ prompt: e.target.value })}
placeholder={t("setting.ai.transcription-prompt-placeholder")}
rows={3}
disabled={!transcription.providerId}
maxLength={4096}
/>
<p className="text-xs text-muted-foreground">{t("setting.ai.transcription-prompt-help")}</p>
</div>
</div>
);
};
interface AIProviderDialogProps { interface AIProviderDialogProps {
provider?: LocalAIProvider; provider?: LocalAIProvider;
onOpenChange: (open: boolean) => void; onOpenChange: (open: boolean) => void;

View file

@ -427,13 +427,32 @@
"edit-provider": "Edit provider", "edit-provider": "Edit provider",
"endpoint": "Endpoint", "endpoint": "Endpoint",
"endpoint-hint": "Leave empty to use the official provider endpoint.", "endpoint-hint": "Leave empty to use the official provider endpoint.",
"integrations-description": "Provider keys are supplied by the instance owner and used by server-side AI features.",
"integrations-title": "AI integrations",
"keep-api-key": "Leave blank to keep the existing key", "keep-api-key": "Leave blank to keep the existing key",
"label": "AI", "label": "AI",
"no-providers": "No AI providers configured.", "no-providers": "No AI providers configured.",
"provider-title": "Provider name", "provider-title": "Provider name",
"provider-title-required": "Provider name is required.", "provider-title-required": "Provider name is required.",
"provider-type": "Provider type", "provider-type": "Provider type",
"providers": "Providers" "providers": "Providers",
"transcription-description": "Speech-to-text settings used when recording audio in the memo composer.",
"transcription-empty-providers": "Add an AI integration first to enable transcription.",
"transcription-language-help": "ISO 639-1 short code (e.g. en, de, zh). Leave empty to auto-detect.",
"transcription-language-placeholder": "auto-detect",
"transcription-language": "Default language",
"transcription-model-help": "Free text. Use the provider's model identifier — e.g. whisper-1, gpt-4o-transcribe, whisper-large-v3-turbo.",
"transcription-model-placeholder-gemini": "gemini-2.5-flash",
"transcription-model-placeholder-openai": "whisper-1",
"transcription-model": "Model",
"transcription-no-provider": "None — transcription disabled",
"transcription-prompt-help": "Improves spelling of proper nouns and jargon. Whisper limit is roughly 224 tokens.",
"transcription-prompt-placeholder": "Names: Alice, Bob. Glossary: kubernetes, OAuth.",
"transcription-prompt": "Prompt hints",
"transcription-provider": "Provider",
"transcription-title": "Transcription",
"transcription-warning-gemini-webm": "Gemini does not accept browser-recorded audio/webm. For in-editor recording, use an OpenAI-compatible provider.",
"transcription-warning-no-key": "The selected provider has no API key set. Edit the integration above to add one."
}, },
"instance": { "instance": {
"access-description": "Control sign-up, authentication, profile editing, and calendar defaults for this instance.", "access-description": "Control sign-up, authentication, profile editing, and calendar defaults for this instance.",

View file

@ -13,30 +13,16 @@ import type { Message } from "@bufbuild/protobuf";
* Describes the file api/v1/ai_service.proto. * Describes the file api/v1/ai_service.proto.
*/ */
export const file_api_v1_ai_service: GenFile = /*@__PURE__*/ export const file_api_v1_ai_service: GenFile = /*@__PURE__*/
fileDesc("ChdhcGkvdjEvYWlfc2VydmljZS5wcm90bxIMbWVtb3MuYXBpLnYxIpsBChFUcmFuc2NyaWJlUmVxdWVzdBIYCgtwcm92aWRlcl9pZBgBIAEoCUID4EECEjYKBmNvbmZpZxgCIAEoCzIhLm1lbW9zLmFwaS52MS5UcmFuc2NyaXB0aW9uQ29uZmlnQgPgQQISNAoFYXVkaW8YAyABKAsyIC5tZW1vcy5hcGkudjEuVHJhbnNjcmlwdGlvbkF1ZGlvQgPgQQIiQQoTVHJhbnNjcmlwdGlvbkNvbmZpZxITCgZwcm9tcHQYASABKAlCA+BBARIVCghsYW5ndWFnZRgCIAEoCUID4EEBIncKElRyYW5zY3JpcHRpb25BdWRpbxIWCgdjb250ZW50GAEgASgMQgPgQQRIABINCgN1cmkYAiABKAlIABIVCghmaWxlbmFtZRgDIAEoCUID4EEBEhkKDGNvbnRlbnRfdHlwZRgEIAEoCUID4EEBQggKBnNvdXJjZSIiChJUcmFuc2NyaWJlUmVzcG9uc2USDAoEdGV4dBgBIAEoCTKaAQoJQUlTZXJ2aWNlEowBCgpUcmFuc2NyaWJlEh8ubWVtb3MuYXBpLnYxLlRyYW5zY3JpYmVSZXF1ZXN0GiAubWVtb3MuYXBpLnYxLlRyYW5zY3JpYmVSZXNwb25zZSI72kEYcHJvdmlkZXJfaWQsY29uZmlnLGF1ZGlvgtPkkwIaOgEqIhUvYXBpL3YxL2FpOnRyYW5zY3JpYmVCpgEKEGNvbS5tZW1vcy5hcGkudjFCDkFpU2VydmljZVByb3RvUAFaMGdpdGh1Yi5jb20vdXNlbWVtb3MvbWVtb3MvcHJvdG8vZ2VuL2FwaS92MTthcGl2MaICA01BWKoCDE1lbW9zLkFwaS5WMcoCDE1lbW9zXEFwaVxWMeICGE1lbW9zXEFwaVxWMVxHUEJNZXRhZGF0YeoCDk1lbW9zOjpBcGk6OlYxYgZwcm90bzM", [file_google_api_annotations, file_google_api_client, file_google_api_field_behavior]); fileDesc("ChdhcGkvdjEvYWlfc2VydmljZS5wcm90bxIMbWVtb3MuYXBpLnYxIkkKEVRyYW5zY3JpYmVSZXF1ZXN0EjQKBWF1ZGlvGAEgASgLMiAubWVtb3MuYXBpLnYxLlRyYW5zY3JpcHRpb25BdWRpb0ID4EECIncKElRyYW5zY3JpcHRpb25BdWRpbxIWCgdjb250ZW50GAEgASgMQgPgQQRIABINCgN1cmkYAiABKAlIABIVCghmaWxlbmFtZRgDIAEoCUID4EEBEhkKDGNvbnRlbnRfdHlwZRgEIAEoCUID4EEBQggKBnNvdXJjZSIiChJUcmFuc2NyaWJlUmVzcG9uc2USDAoEdGV4dBgBIAEoCTKGAQoJQUlTZXJ2aWNlEnkKClRyYW5zY3JpYmUSHy5tZW1vcy5hcGkudjEuVHJhbnNjcmliZVJlcXVlc3QaIC5tZW1vcy5hcGkudjEuVHJhbnNjcmliZVJlc3BvbnNlIijaQQVhdWRpb4LT5JMCGjoBKiIVL2FwaS92MS9haTp0cmFuc2NyaWJlQqYBChBjb20ubWVtb3MuYXBpLnYxQg5BaVNlcnZpY2VQcm90b1ABWjBnaXRodWIuY29tL3VzZW1lbW9zL21lbW9zL3Byb3RvL2dlbi9hcGkvdjE7YXBpdjGiAgNNQViqAgxNZW1vcy5BcGkuVjHKAgxNZW1vc1xBcGlcVjHiAhhNZW1vc1xBcGlcVjFcR1BCTWV0YWRhdGHqAg5NZW1vczo6QXBpOjpWMWIGcHJvdG8z", [file_google_api_annotations, file_google_api_client, file_google_api_field_behavior]);
/** /**
* @generated from message memos.api.v1.TranscribeRequest * @generated from message memos.api.v1.TranscribeRequest
*/ */
export type TranscribeRequest = Message<"memos.api.v1.TranscribeRequest"> & { export type TranscribeRequest = Message<"memos.api.v1.TranscribeRequest"> & {
/**
* Required. The instance AI provider ID to use.
*
* @generated from field: string provider_id = 1;
*/
providerId: string;
/**
* Required. Transcription options.
*
* @generated from field: memos.api.v1.TranscriptionConfig config = 2;
*/
config?: TranscriptionConfig | undefined;
/** /**
* Required. Audio input. * Required. Audio input.
* *
* @generated from field: memos.api.v1.TranscriptionAudio audio = 3; * @generated from field: memos.api.v1.TranscriptionAudio audio = 1;
*/ */
audio?: TranscriptionAudio | undefined; audio?: TranscriptionAudio | undefined;
}; };
@ -48,32 +34,6 @@ export type TranscribeRequest = Message<"memos.api.v1.TranscribeRequest"> & {
export const TranscribeRequestSchema: GenMessage<TranscribeRequest> = /*@__PURE__*/ export const TranscribeRequestSchema: GenMessage<TranscribeRequest> = /*@__PURE__*/
messageDesc(file_api_v1_ai_service, 0); messageDesc(file_api_v1_ai_service, 0);
/**
* @generated from message memos.api.v1.TranscriptionConfig
*/
export type TranscriptionConfig = Message<"memos.api.v1.TranscriptionConfig"> & {
/**
* Optional. A prompt to improve transcription quality.
*
* @generated from field: string prompt = 1;
*/
prompt: string;
/**
* Optional. The language of the input audio.
*
* @generated from field: string language = 2;
*/
language: string;
};
/**
* Describes the message memos.api.v1.TranscriptionConfig.
* Use `create(TranscriptionConfigSchema)` to create a new message.
*/
export const TranscriptionConfigSchema: GenMessage<TranscriptionConfig> = /*@__PURE__*/
messageDesc(file_api_v1_ai_service, 1);
/** /**
* @generated from message memos.api.v1.TranscriptionAudio * @generated from message memos.api.v1.TranscriptionAudio
*/ */
@ -119,7 +79,7 @@ export type TranscriptionAudio = Message<"memos.api.v1.TranscriptionAudio"> & {
* Use `create(TranscriptionAudioSchema)` to create a new message. * Use `create(TranscriptionAudioSchema)` to create a new message.
*/ */
export const TranscriptionAudioSchema: GenMessage<TranscriptionAudio> = /*@__PURE__*/ export const TranscriptionAudioSchema: GenMessage<TranscriptionAudio> = /*@__PURE__*/
messageDesc(file_api_v1_ai_service, 2); messageDesc(file_api_v1_ai_service, 1);
/** /**
* @generated from message memos.api.v1.TranscribeResponse * @generated from message memos.api.v1.TranscribeResponse
@ -138,7 +98,7 @@ export type TranscribeResponse = Message<"memos.api.v1.TranscribeResponse"> & {
* Use `create(TranscribeResponseSchema)` to create a new message. * Use `create(TranscribeResponseSchema)` to create a new message.
*/ */
export const TranscribeResponseSchema: GenMessage<TranscribeResponse> = /*@__PURE__*/ export const TranscribeResponseSchema: GenMessage<TranscribeResponse> = /*@__PURE__*/
messageDesc(file_api_v1_ai_service, 3); messageDesc(file_api_v1_ai_service, 2);
/** /**
* @generated from service memos.api.v1.AIService * @generated from service memos.api.v1.AIService

File diff suppressed because one or more lines are too long