Skip to main content
Prerequisite: You should have followed the Development and Local Supabase Development setup guides.

Overview

Tuturuuu leverages the Vercel AI SDK to generate structured data from large language models (LLMs). This approach enables type-safe AI responses, improved reliability, and consistent data structures for features like flashcards, quizzes, and learning plans. This guide covers how to use AI structured data generation in the Tuturuuu development workflow.

Key Concepts

What is Structured Data Generation?

While text generation can be useful, many applications require generating structured data. For example, you might want to:
  • Extract specific information from text
  • Generate quizzes or flashcards from learning material
  • Create complex objects like learning plans or task lists
  • Ensure AI responses follow a consistent format
The AI SDK standardizes structured object generation across model providers with the generateObject and streamObject functions. You can use Zod schemas to specify the shape of the data that you want, and the AI model will generate data that conforms to that structure.

Architecture in Tuturuuu

Tuturuuu’s AI features follow this high-level architecture:
  1. Frontend UI - React components that display and interact with AI-generated content
  2. API Routes - Next.js routes that handle AI requests and responses
  3. AI SDK - Vercel AI SDK that manages model providers and generates structured data
  4. Supabase - Backend database for authentication, authorization, and storing AI-generated content

Chat request trust boundary

The shared Google chat handler accepts only user and assistant message roles from request JSON, after trimming and case normalization. Other roles, including system and tool, return 400 before authentication or model work. This applies to web Mira, web AI Chat, Rewise, Learn, mobile legacy chat and the subscription route that reuses the handler. Client histories remain accepted for compatibility, but assistant parts are text only. Client dynamic-tool and tool-* call/result parts are dropped from both roles rather than rejecting a normal send that echoes rendered tool history. Other non-text assistant parts, including sources and reasoning, are also dropped. User text and existing user source URL mapping remain supported; chat attachments continue through the server storage path below. Tool results in the current generation come from server tool execution; echoed results are never reconstructed as model context. This does not authenticate client-supplied assistant text as a persisted server response. Native conversations load persisted history on the server. Their adapter passes server-created split instructions and saved conversation prompts through the in-process createPOST({ serverSystemInstructions }) option, outside JSON. An identically named request-body field grants no instruction authority. Regression coverage: packages/ai/src/chat/google/chat-request-schema.test.ts, subscription-route.test.ts, and the native conversation adapter’s ai-message-shared.test.ts.

Mira Chat Attachments

Dashboard chat attachments are uploaded to Supabase Storage before the user message is sent. New chats may first place files under {wsId}/chats/ai/resources/temp/{userId} and then move them into {wsId}/chats/ai/resources/{chatId} once the chat exists. Tools that read those attachments, such as convert_file_to_markdown, should resolve bare filenames and stale same-workspace attachment paths against the current chat folder. They must still reject full paths from another workspace. Workspace membership alone never grants access to a chat’s resource folder. /api/ai/chat/upload-url, file-urls, signed-read-url and delete-file also require that an existing ai_chats row belongs to the actor (checked with the admin client, because creator-only RLS would hide another member’s chat as missing) and that temp/{userId} paths name the actor. Another member’s chat answers 404 and no URL is issued. Ids without an ai_chats row stay usable for Mind threads and pre-creation uploads. Uploads accept images (including HEIC/HEIF), audio (m4a, mp3, wav, aac, ogg, opus, flac), video and documents up to 32 MiB, the per-request model attachment budget; a larger declared size answers 413. The mobile Assistant uploads into the selected chat’s own store with no cross-store fallback: native conversations (bare UUIDs) use /api/v1/workspaces/:wsId/chat/conversations/:id/attachments/upload-url, and legacy chats (ai-chat-/legacy-ai- ids) use /api/ai/chat/upload-url with the stored chat id. A native send refuses any attachment stored outside chats/<conversationId>/ instead of silently dropping it. Before upload the app re-encodes photos on device to JPEG (quality 82, longest edge 2048 px, EXIF orientation applied, EXIF/GPS removed). It keeps a PNG with transparency, or one that JPEG would enlarge, minus its metadata chunks. Voice notes record AAC-LC .m4a, mono, 16 kHz at 32 kbps; neither platform encodes MP3 natively. Originals stay on the device. Each failed attachment shows its own reason (too large, unsupported, not allowed in this chat, connection, server) and offers Retry when retrying can help. Regression coverage: apps/web/src/app/api/ai/chat/**/*.test.ts and apps/mobile/test/features/assistant/{data,cubit}/assistant_*attachment*_test.dart plus assistant_media_optimizer_test.dart.

Attachment conversion URL policy

Both MarkItDown /markitdown endpoints in apps/discord accept signed Storage URLs only at the configured Supabase origin. Host comparison is case-insensitive and IDNA-normalized; userinfo, other hosts, unconfigured ports, and redirects are rejected before content is downloaded. HTTPS is the default. Local development may use HTTP only when the selected Supabase URL explicitly configures HTTP on localhost, 127.0.0.1, ::1, or host.docker.internal, with its matching port. The existing Supabase environment-variable precedence remains unchanged. Signed paths must retain the raw Storage signing prefix and contain no encoded slash/backslash or literal/encoded dot segments, including before URL normalization. This policy trusts the operator-configured hostname and its DNS; it does not resolve and filter private addresses. Keep that configuration and DNS under operator control. Signed downloads retain the 60-second timeout, Content-Length check, 50 MiB streaming cap, and temporary-file cleanup, including cancellation. Temporary-file suffixes come from supported extension constants; unknown or unsafe suffixes are omitted. The retained direct YouTube conversion path accepts video links on the existing YouTube host allowlist, extracts an 11-character video ID, and fetches a canonical HTTPS watch page on www.youtube.com without redirects. The response URL returned to callers is canonical. Transcript retrieval remains owned by MarkItDown’s YouTube integration. This does not change Mira’s Gemini-native YouTube routing. Regression coverage: apps/discord/tests/test_markitdown_service.py and apps/discord/tests/test_markitdown_security.py.

Mira Dashboard Chat Agent Loop

The dashboard Mira chat keeps the main assistant in fast mode by default. New sessions should ignore stale stored thinking preferences unless the user manually opts into the toolbar’s deep-check mode for that active session. Assistant text should stream as soon as it is useful, then tool calls may run inline, followed by more assistant text in the same response. Keep every assistant text surface on the shared Streamdown wrapper, including text rendered inside compact tool UIs, so code blocks, tables, Mermaid, math, and CJK spacing stay consistent. Mira enables Streamdown math with singleDollarTextMath because model output commonly uses $...$ inline LaTeX. Keep @streamdown/math in the Tailwind source scan alongside the base Streamdown dist files, and keep katex/dist/katex.min.css imported from the chat renderer path. While the request is submitted but no assistant text has arrived yet, the chat should render a lightweight assistant activity bubble with rotating status copy. Hide that placeholder as soon as real assistant text streams, and keep it separate from markdown/tool rendering so status cycling does not re-render heavy message content. Keep the first optional-tool model step lean. Unless a workflow must force a specific tool before answering, expose only select_tools and no_action_needed on the first step so the model can stream direct answers without carrying every tool schema. Stream smoothing should not add artificial per-chunk delay on the dashboard chat path; perceived smoothness belongs in the client activity state, not in delayed server chunks. Do not force select_tools as the universal first step. Force tool selection only when the workflow cannot answer safely before a tool runs, such as current web lookups, workspace context switching, workspace member lookups, file conversion, or writes. Complex verification, risk review, planning checks, or conflicting evidence should use run_parallel_checks, which delegates to bounded ToolLoopAgent subagents in parallel and returns a compact summary to the main assistant. Any Mira tool that spawns additional model calls must use MiraToolContext.creditWsId ?? wsId, resolve the plan model for that billed workspace, preflight AI credits, apply the workspace output-token cap, and deduct the subcall usage before exposing the generated result.

Schema Definitions

Schemas define the structure of the data that will be generated by the AI models. In Tuturuuu, these are defined in packages/ai/src/object/types.ts using Zod. Here are some examples of schemas used in Tuturuuu:

Flashcard Schema

Quiz Schema

Year Plan Schema

Creating an API Endpoint

To create an API endpoint that generates structured data, follow these steps:

1. Create a new route file

Create a new route file in the appropriate Next.js app, for example:

2. Implement authentication and validation

Use Supabase to authenticate the user and validate their permissions:

Fast session auth for Mira and assistant routes

Session-authenticated AI routes may accept the x-tuturuuu-ai-temp-auth header as an optimization before falling back to Supabase getUser(). The browser mints this token through POST /api/ai/temp-auth/token after the normal session, workspace normalization, membership, and selected billing workspace checks succeed. Tokens live only in memory on the client, expire after 60 seconds, and are stored in Redis only as SHA-256 digests. Revocation is version-based: ai:temp-auth:user-version:{userId} is bumped before logout or account removal, so any token minted under the old version is rejected. Redis is not authoritative for security; when Redis is unavailable or a token is missing/invalid, routes fall back to the existing Supabase session path. A revoked token returns 401 and does not fall back. Credit availability snapshots are also Redis-backed under ai:credits:snapshot:{billingWsId}:{userId}. They are UI/status hints only and must not authorize model execution. AI preflight must call the authoritative Postgres allowance RPC so daily credit limits, daily request limits, and feature-specific request limits are enforced before every model run. Actual reservations, deductions, and ledger writes remain in Postgres, and successful commits invalidate the snapshot.

3. Generate structured data

Use the AI SDK to generate structured data based on the schema:

Supported Models

Tuturuuu supports multiple AI models through the Vercel AI SDK. There is no static model array anymore — the catalog is sourced dynamically from the AI gateway and stored in the ai_gateway_models table (synced by packages/ai/src/credits/sync-gateway-models.ts). The browsable catalog is served from apps/infrastructure:
  • API: GET /api/v1/infrastructure/ai/models (apps/infrastructure/src/app/api/v1/infrastructure/ai/models/route.ts)
  • Docs: see Model Catalog for how metadata is published, browsed, and synced.
The set of provider integrations the SDK can wire up lives in packages/ai/src/supported-providers.ts:
For per-workspace plan resolution (which model a billed workspace is allowed to run), use the helpers under packages/ai/src/credits/ such as resolve-plan-model.ts and cap-output-tokens.ts, rather than hard-coding a model ID. To use a different model in your endpoint, change the model reference. Active callers default to gemini-3.1-flash-lite:

Calling from the Frontend

To call your AI endpoint from the frontend, you can use the appropriate hooks or fetch API:

Integration with Supabase

Tuturuuu’s AI features are tightly integrated with Supabase for several purposes:

TypeScript Types

Supabase-generated TypeScript types are available at packages/types/src/supabase.ts. These types are automatically generated when you run bun sb:typegen or bun sb:reset and are accessible to all apps that have the @tuturuuu/types package installed. You can use these types to ensure type safety when working with Supabase data in your AI features:

Short-hand Type Access

For more convenient access to common table types in your AI features, Tuturuuu also provides short-hand type definitions in packages/types/src/db.ts. These are easier to use and remember than the full database type paths:
The short-hand types can also include extended client-side properties that aren’t in the database schema, making them perfect for your AI feature implementations. This ensures that your AI features correctly interact with the database schema, reducing runtime errors and improving development experience.

Authentication and Authorization

Before making AI requests, ensure the user is authenticated and authorized to use the feature:

Feature Flags

Use the workspace_secrets table to enable or disable AI features for specific workspaces:

Storing Results

You can store AI-generated content in Supabase for future use:

Best Practices

Schema Design

When designing schemas for AI-generated content:
  1. Be specific - Use the .describe() method to provide clear instructions to the AI model
  2. Keep it simple - Break complex schemas into smaller, nested objects
  3. Add validations - Use Zod’s validation methods (.min(), .max(), .regex(), etc.)
  4. Use enums - For fields with a fixed set of values, use .enum()
Example of a well-designed schema:

Error Handling

Implement robust error handling for AI-generated content:

Response Processing

For complex AI-generated content, you may need to post-process the response:

Local Development and Testing

Setting Up API Keys

To test AI features locally, you need to set up the appropriate API keys in your environment:
  1. Create a .env.local file in the root of your Next.js app
  2. Add the necessary API keys:
  1. Restart your development server

Testing AI Endpoints

You can test your AI endpoints using tools like Postman or simple cURL commands:

Troubleshooting

Common Issues

  1. API Key Issues: Ensure your API keys are correctly set in your environment
  2. Model Unavailability: Some models may be unavailable in certain regions
  3. Token Limits: Large prompts may exceed token limits
  4. Schema Validation Errors: The AI might generate content that doesn’t match your schema

Debugging Tips

  1. Log the prompt: Print the full prompt being sent to the AI model
  2. Start with simple schemas: Begin with simple schemas and gradually increase complexity
  3. Check response format: Verify the raw response from the AI model before schema validation

Further Resources

Dashboard assistant settings receipts

Dashboard Soul metadata belongs to the verified account, independently of the selected workspace. Reads and edits use the shared internal API adapter, an account-specific query key, and a committed session lease. Account departure cancels requests and removes that account’s cached Soul; a later same-account session must receive fresh server admission. An abandoned React render does not revoke the currently committed lease. A failed optimistic edit may restore its previous value only while its intent, query instance and publication revision are still current. It must preserve a newer confirmed tool or settings publication. Malformed or foreign-owner Soul receipts are errors, rather than successful default values. Displayed names use the trimmed custom name, with Mira as the fallback for an empty name; this does not change the server’s existing name contract. Only definitive successful update_my_settings outputs in the current submitted conversation invalidate the current account’s Soul metadata. Failed, preliminary, duplicate, restored-history and departed account/workspace/chat outputs do not trigger a settings refresh. This read-side admission leaves real transport and persistence errors intact. It adds no server compare-and-swap or offline reconciliation guarantee; mobile pending-name policy is separate. Regression coverage lives in the dashboard use-mira-soul and use-mira-soul-scope hook tests, mira-settings-receipts tests, and the shared mira-soul adapter tests. These source regressions do not establish production or authenticated browser delivery.