feat: whyrating - initial project from turbostarter boilerplate
This commit is contained in:
Alejandro Gutiérrez
@@ -0,0 +1,122 @@
|
||||
---
|
||||
title: Overview
|
||||
description: Learn about background tasks & cron jobs and how they can power your application.
|
||||
url: /docs/web/background-tasks/overview
|
||||
---
|
||||
|
||||
# Overview
|
||||
|
||||
Background tasks and cron jobs are long-running processes that execute outside of your main application flow, allowing you to handle time-intensive operations and scheduled workflows without blocking user interactions or hitting serverless function timeouts.
|
||||
|
||||
<Callout title="Perfect for time-intensive & scheduled operations">
|
||||
Background tasks are ideal for operations that take longer than typical serverless function timeouts (10-60 seconds), such as processing large files, sending batch emails, or making multiple API calls.
|
||||
|
||||
Cron jobs are perfect for recurring operations like daily reports, cleanup tasks, or periodic data synchronization.
|
||||
</Callout>
|
||||
|
||||
## What are background tasks?
|
||||
|
||||
**Background tasks** are asynchronous processes that run separately from your main application thread. Instead of forcing users to wait for lengthy operations to complete, you can offload these tasks to run in the background while your application remains responsive.
|
||||
|
||||
**Cron jobs** are scheduled background tasks that run automatically at specific times or intervals. They're perfect for maintenance operations, reports, and recurring workflows that need to happen without user intervention.
|
||||
|
||||
Think of background tasks as your application's *worker threads* - they handle the heavy lifting while your main application stays fast and responsive for users.
|
||||
|
||||
<ThemedImage alt="Background tasks architecture diagram" light="/images/docs/web/background-tasks/light.png" dark="/images/docs/web/background-tasks/dark.png" width={1335} height={285} zoomable />
|
||||
|
||||
## Why use background tasks?
|
||||
|
||||
<Cards className="grid-cols-1">
|
||||
<Card title="Avoid timeouts">
|
||||
Most serverless platforms have strict execution limits:
|
||||
|
||||
* **[Vercel (Hobby)](https://vercel.com/docs/functions/serverless-functions/runtimes#max-duration)**: 300 seconds
|
||||
* **[Vercel (Pro)](https://vercel.com/docs/functions/serverless-functions/runtimes#max-duration)**: 800 seconds
|
||||
* **[Vercel (Enterprise)](https://vercel.com/docs/functions/serverless-functions/runtimes#max-duration)**: 800 seconds
|
||||
* **[AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/configuration-timeout.html)**: 900 seconds
|
||||
* **[Netlify Functions](https://docs.netlify.com/functions/overview/#default-deployment-options)**: 30 seconds
|
||||
|
||||
Background tasks let you bypass these limitations entirely.
|
||||
</Card>
|
||||
|
||||
<Card title="Better user experience">
|
||||
Users don't have to wait for long-running processes. They can continue using
|
||||
your application while tasks complete in the background.
|
||||
</Card>
|
||||
|
||||
<Card title="Automated workflows">
|
||||
Cron jobs enable hands-off automation of recurring tasks like daily backups,
|
||||
weekly reports, or monthly user engagement analysis - all running reliably
|
||||
without manual intervention.
|
||||
</Card>
|
||||
|
||||
<Card title="Improved reliability">
|
||||
Background tasks can be automatically retried if they fail, ensuring your
|
||||
critical processes eventually complete successfully.
|
||||
</Card>
|
||||
|
||||
<Card title="Resource optimization">
|
||||
Your main application servers stay available to handle user requests instead of being tied up with heavy processing tasks.
|
||||
</Card>
|
||||
</Cards>
|
||||
|
||||
## Common use cases
|
||||
|
||||
Here are some typical scenarios where background tasks shine:
|
||||
|
||||
<Accordions>
|
||||
<Accordion title="File processing">
|
||||
* **Video transcoding**: Converting uploaded videos to different formats or resolutions
|
||||
* **Image optimization**: Batch processing user-uploaded images
|
||||
* **Document parsing**: Extracting text from PDFs or generating thumbnails
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Data operations">
|
||||
* **Database migrations**: Moving or transforming large datasets
|
||||
* **Report generation**: Creating complex analytics reports
|
||||
* **Data synchronization**: Syncing data between different systems
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Communication">
|
||||
* **Email campaigns**: Sending personalized emails to large user lists
|
||||
* **Notification processing**: Delivering push notifications across multiple platforms
|
||||
* **SMS campaigns**: Bulk SMS sending with rate limiting
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="AI and ML tasks">
|
||||
* **Content generation**: Using AI models to generate text, images, or videos
|
||||
* **Data analysis**: Running machine learning models on large datasets
|
||||
* **Natural language processing**: Analyzing text content for insights
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Third-party integrations">
|
||||
* **API synchronization**: Syncing data with external services
|
||||
* **Webhook processing**: Handling incoming webhooks that trigger complex workflows
|
||||
* **Social media automation**: Posting content across multiple platforms
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Scheduled operations (Cron jobs)">
|
||||
* **Daily reports**: Generating and emailing daily analytics or performance reports
|
||||
* **Database maintenance**: Cleaning up old records, optimizing indexes, or running backups
|
||||
* **User engagement**: Sending weekly newsletters or monthly account summaries
|
||||
* **System monitoring**: Health checks, performance monitoring, and alert notifications
|
||||
* **Content management**: Auto-publishing scheduled content or archiving old posts
|
||||
</Accordion>
|
||||
</Accordions>
|
||||
|
||||
## When not to use background tasks?
|
||||
|
||||
Background tasks and cron jobs aren't always the right solution. Consider alternatives for:
|
||||
|
||||
* **Real-time operations**: Tasks that users need immediate results from
|
||||
* **Simple, fast operations**: Tasks that complete in under 5-10 seconds
|
||||
* **Database queries**: Standard CRUD operations that should remain synchronous
|
||||
* **User authentication**: Login/logout processes should be immediate
|
||||
|
||||
<Callout type="warn" title="Keep it simple">
|
||||
Start with synchronous processing for simple tasks and manual processes for infrequent operations. Only move to background tasks when you hit timeout limitations or user experience issues, and only use cron jobs when you need reliable automation.
|
||||
</Callout>
|
||||
|
||||
## Getting started
|
||||
|
||||
Ready to add background tasks to your TurboStarter application? Check out our [Trigger.dev integration guide](/docs/web/background-tasks/trigger) or [Upstash QStash integration guide](/docs/web/background-tasks/qstash) to learn how to implement background tasks using one of the most developer-friendly background job frameworks available.
|
||||
@@ -0,0 +1,648 @@
|
||||
---
|
||||
title: Upstash QStash
|
||||
description: Integrate Upstash QStash with your TurboStarter application for serverless-first background task processing.
|
||||
url: /docs/web/background-tasks/qstash
|
||||
---
|
||||
|
||||
# Upstash QStash
|
||||
|
||||
[Upstash QStash](https://upstash.com/docs/qstash) is a serverless message queue and task scheduler designed specifically for serverless and edge environments. It uses HTTP endpoints instead of persistent connections, making it perfect for modern web applications.
|
||||
|
||||
<Callout title="Why QStash?">
|
||||
QStash is built for the serverless world - no infrastructure to manage, automatic scaling, and pay-per-use pricing. It delivers messages to your HTTP endpoints with built-in retries, delays, and scheduling capabilities.
|
||||
</Callout>
|
||||
|
||||
<Steps>
|
||||
<Step>
|
||||
## Setup
|
||||
|
||||
Visit [Upstash Console](https://console.upstash.com) and create a free account. Create a new QStash project and note down your credentials.
|
||||
|
||||
Add your QStash credentials to your root environment variables:
|
||||
|
||||
```dotenv title=".env.local"
|
||||
QSTASH_URL=https://qstash.upstash.io
|
||||
QSTASH_TOKEN=your_qstash_token_here
|
||||
QSTASH_CURRENT_SIGNING_KEY=your_current_signing_key_here
|
||||
QSTASH_NEXT_SIGNING_KEY=your_next_signing_key_here
|
||||
```
|
||||
|
||||
You can find these values in your Upstash Console under the QStash project settings.
|
||||
|
||||
For production, make sure to add these environment variables to your deployment platform.
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Install dependencies
|
||||
|
||||
Add the QStash SDK to your API package:
|
||||
|
||||
```bash
|
||||
pnpm add --filter api @upstash/qstash
|
||||
```
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Create the QStash client
|
||||
|
||||
Create a utility file to initialize the QStash client in your API package:
|
||||
|
||||
```ts title="packages/api/src/lib/qstash.ts"
|
||||
import { Client } from "@upstash/qstash";
|
||||
|
||||
import { env } from "~/env";
|
||||
|
||||
export const qstashClient = new Client({
|
||||
baseUrl: env.QSTASH_URL,
|
||||
token: env.QSTASH_TOKEN,
|
||||
});
|
||||
```
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Create task handlers
|
||||
|
||||
QStash delivers messages to HTTP endpoints, so you'll create API routes to handle your background tasks.
|
||||
|
||||
Let's create task handlers for common operations:
|
||||
|
||||
<Tabs items={["Task router", "Process user data", "Daily cleanup", "Verification middleware"]}>
|
||||
<Tab value="Task router">
|
||||
```ts title="packages/api/src/modules/tasks/router.ts"
|
||||
import { Hono } from "hono";
|
||||
import * as z from "zod";
|
||||
|
||||
import { qstashVerifyMiddleware } from "../../middleware/qstash-verify";
|
||||
import { dailyCleanupHandler } from "./handlers/daily-cleanup";
|
||||
import { processUserDataHandler } from "./handlers/process-user-data";
|
||||
|
||||
const processUserDataSchema = z.object({
|
||||
userId: z.string(),
|
||||
operation: z.enum(["export", "analyze", "cleanup"]),
|
||||
});
|
||||
|
||||
export const tasksRouter = new Hono()
|
||||
.basePath("/tasks")
|
||||
// Apply QStash signature verification to all task routes
|
||||
.use(qstashVerifyMiddleware)
|
||||
.post("/process-user-data", processUserDataHandler)
|
||||
.post("/daily-cleanup", dailyCleanupHandler);
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab value="Process user data">
|
||||
```ts title="packages/api/src/modules/tasks/handlers/process-user-data.ts"
|
||||
import type { Context } from "hono";
|
||||
import * as z from "zod";
|
||||
|
||||
const ProcessUserDataSchema = z.object({
|
||||
userId: z.string(),
|
||||
operation: z.enum(["export", "analyze", "cleanup"]),
|
||||
});
|
||||
|
||||
export async function processUserDataHandler(c: Context) {
|
||||
try {
|
||||
const payload = ProcessUserDataSchema.parse(await c.req.json());
|
||||
const { userId, operation } = payload;
|
||||
|
||||
console.log("Starting user data processing", { userId, operation });
|
||||
|
||||
switch (operation) {
|
||||
case "export":
|
||||
// Simulate data export
|
||||
await new Promise((resolve) => setTimeout(resolve, 2000));
|
||||
console.log("User data exported successfully");
|
||||
return c.json({
|
||||
success: true,
|
||||
result: "Data exported to CSV",
|
||||
});
|
||||
|
||||
case "analyze":
|
||||
// Simulate data analysis
|
||||
await new Promise((resolve) => setTimeout(resolve, 5000));
|
||||
console.log("User data analysis completed");
|
||||
return c.json({
|
||||
success: true,
|
||||
result: { totalActions: 156, avgSessionTime: "4m 32s" },
|
||||
});
|
||||
|
||||
case "cleanup":
|
||||
// Simulate data cleanup
|
||||
await new Promise((resolve) => setTimeout(resolve, 3000));
|
||||
console.log("User data cleanup completed");
|
||||
return c.json({
|
||||
success: true,
|
||||
result: "Removed 23 obsolete records",
|
||||
});
|
||||
|
||||
default:
|
||||
throw new Error(`Unknown operation: ${operation}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Task failed:", error);
|
||||
return c.json({ error: "Task failed" }, 500);
|
||||
}
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab value="Daily cleanup">
|
||||
```ts title="packages/api/src/modules/tasks/handlers/daily-cleanup.ts"
|
||||
import type { Context } from "hono";
|
||||
|
||||
export async function dailyCleanupHandler(c: Context) {
|
||||
try {
|
||||
console.log("Starting daily cleanup");
|
||||
|
||||
// Cleanup old logs
|
||||
await new Promise((resolve) => setTimeout(resolve, 5000));
|
||||
console.log("Logs cleaned up");
|
||||
|
||||
// Cleanup temporary files
|
||||
await new Promise((resolve) => setTimeout(resolve, 3000));
|
||||
console.log("Temp files cleaned up");
|
||||
|
||||
// Generate daily reports
|
||||
await new Promise((resolve) => setTimeout(resolve, 8000));
|
||||
console.log("Reports generated");
|
||||
|
||||
return c.json({
|
||||
success: true,
|
||||
cleanupTime: new Date().toISOString(),
|
||||
itemsProcessed: 1247,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("Daily cleanup failed:", error);
|
||||
return c.json({ error: "Daily cleanup failed" }, 500);
|
||||
}
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab value="Verification middleware">
|
||||
```ts title="packages/api/src/middleware/qstash-verify.ts"
|
||||
import { Receiver } from "@upstash/qstash";
|
||||
import { createMiddleware } from "hono/factory";
|
||||
|
||||
export const qstashVerifyMiddleware = createMiddleware(async (c, next) => {
|
||||
const currentSigningKey = process.env.QSTASH_CURRENT_SIGNING_KEY;
|
||||
const nextSigningKey = process.env.QSTASH_NEXT_SIGNING_KEY;
|
||||
|
||||
if (!currentSigningKey || !nextSigningKey) {
|
||||
return c.json({ error: "QStash signing keys not configured" }, 500);
|
||||
}
|
||||
|
||||
const signature = c.req.header("upstash-signature");
|
||||
|
||||
if (!signature) {
|
||||
return c.json({ error: "Missing QStash signature" }, 401);
|
||||
}
|
||||
|
||||
try {
|
||||
const body = await c.req.text();
|
||||
|
||||
const receiver = new Receiver({
|
||||
currentSigningKey,
|
||||
nextSigningKey,
|
||||
});
|
||||
|
||||
const isValid = receiver.verify({
|
||||
body,
|
||||
signature,
|
||||
});
|
||||
|
||||
if (!isValid) {
|
||||
return c.json({ error: "Invalid QStash signature" }, 401);
|
||||
}
|
||||
|
||||
// Re-create the request with the body for the next handler
|
||||
const newRequest = new Request(c.req.url, {
|
||||
method: c.req.method,
|
||||
headers: c.req.headers,
|
||||
body,
|
||||
});
|
||||
|
||||
c.req = newRequest;
|
||||
await next();
|
||||
} catch (error) {
|
||||
console.error("QStash signature verification failed:", error);
|
||||
return c.json({ error: "Invalid signature" }, 401);
|
||||
}
|
||||
});
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Register task routes
|
||||
|
||||
Add the tasks router to your main API:
|
||||
|
||||
```ts title="packages/api/src/index.ts"
|
||||
import { tasksRouter } from "./modules/tasks/router";
|
||||
|
||||
const appRouter = new Hono()
|
||||
.basePath("/api")
|
||||
.route("/tasks", tasksRouter)
|
||||
// ... other existing routers
|
||||
.onError(onError);
|
||||
|
||||
export { appRouter };
|
||||
```
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Triggering tasks
|
||||
|
||||
You can trigger tasks from your TurboStarter application by publishing messages to QStash, which will then deliver them to your task endpoints.
|
||||
|
||||
Create a service to handle task triggering:
|
||||
|
||||
```ts title="packages/api/src/modules/tasks/service.ts"
|
||||
import { qstashClient } from "../../lib/qstash";
|
||||
|
||||
function getTaskUrl(taskName: string): string {
|
||||
const baseUrl = process.env.NEXT_PUBLIC_APP_URL || "http://localhost:3000";
|
||||
return `${baseUrl}/api/tasks/${taskName}`;
|
||||
}
|
||||
|
||||
export class TaskService {
|
||||
static async processUserData(
|
||||
userId: string,
|
||||
operation: "export" | "analyze" | "cleanup",
|
||||
) {
|
||||
return await qstashClient.publishJSON({
|
||||
url: getTaskUrl("process-user-data"),
|
||||
body: { userId, operation },
|
||||
});
|
||||
}
|
||||
|
||||
static async scheduleUserDataProcessing(
|
||||
userId: string,
|
||||
operation: "export" | "analyze" | "cleanup",
|
||||
delaySeconds: number,
|
||||
) {
|
||||
return await qstashClient.publishJSON({
|
||||
url: getTaskUrl("process-user-data"),
|
||||
body: { userId, operation },
|
||||
delay: `${delaySeconds}s`,
|
||||
});
|
||||
}
|
||||
|
||||
static async scheduleDailyCleanup() {
|
||||
return await qstashClient.schedules.create({
|
||||
destination: getTaskUrl("daily-cleanup"),
|
||||
cron: "0 2 * * *", // Daily at 2 AM
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Create API endpoints for triggering
|
||||
|
||||
Create endpoints to trigger tasks from your application:
|
||||
|
||||
```ts title="packages/api/src/modules/tasks/trigger/router.ts"
|
||||
import { Hono } from "hono";
|
||||
import * as z from "zod";
|
||||
|
||||
import { enforceAuth, validate } from "../../middleware";
|
||||
import { TaskService } from "./service";
|
||||
|
||||
const triggerUserDataSchema = z.object({
|
||||
userId: z.string(),
|
||||
operation: z.enum(["export", "analyze", "cleanup"]),
|
||||
delaySeconds: z.number().optional(),
|
||||
});
|
||||
|
||||
export const taskTriggerRouter = new Hono()
|
||||
.post(
|
||||
"/trigger/process-user-data",
|
||||
enforceAuth,
|
||||
validate("json", triggerUserDataSchema),
|
||||
async (c) => {
|
||||
const { userId, operation, delaySeconds } = c.req.valid("json");
|
||||
|
||||
const result = delaySeconds
|
||||
? await TaskService.scheduleUserDataProcessing(
|
||||
userId,
|
||||
operation,
|
||||
delaySeconds,
|
||||
)
|
||||
: await TaskService.processUserData(userId, operation);
|
||||
|
||||
return c.json({
|
||||
success: true,
|
||||
messageId: result.messageId,
|
||||
message: delaySeconds
|
||||
? `Task scheduled to run in ${delaySeconds} seconds`
|
||||
: "Task queued for immediate processing",
|
||||
});
|
||||
},
|
||||
)
|
||||
.post("/trigger/daily-cleanup", enforceAuth, async (c) => {
|
||||
const result = await TaskService.scheduleDailyCleanup();
|
||||
|
||||
return c.json({
|
||||
success: true,
|
||||
scheduleId: result.scheduleId,
|
||||
message: "Daily cleanup scheduled",
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Add it to your main router:
|
||||
|
||||
```ts title="packages/api/src/index.ts"
|
||||
import { taskTriggerRouter } from "./modules/tasks/trigger/router";
|
||||
|
||||
const appRouter = new Hono()
|
||||
.basePath("/api")
|
||||
.route("/tasks", tasksRouter)
|
||||
.route("/", taskTriggerRouter) // Trigger routes at root level
|
||||
// ... other existing routers
|
||||
.onError(onError);
|
||||
|
||||
export { appRouter };
|
||||
```
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Using tasks in your application
|
||||
|
||||
### From the client
|
||||
|
||||
```tsx title="apps/web/src/modules/tasks/process-data-button.tsx"
|
||||
"use client";
|
||||
|
||||
import { handle } from "@turbostarter/api/utils";
|
||||
import { useMutation } from "@tanstack/react-query";
|
||||
|
||||
import { api } from "~/lib/api/client";
|
||||
|
||||
export function ProcessDataButton({ userId }: { userId: string }) {
|
||||
const { mutate: processData, isPending } = useMutation({
|
||||
mutationFn: handle(api.trigger["process-user-data"].$post),
|
||||
onSuccess: (data) => {
|
||||
console.log("Task queued:", data.messageId);
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<button
|
||||
onClick={() =>
|
||||
processData({
|
||||
json: {
|
||||
userId,
|
||||
operation: "analyze",
|
||||
delaySeconds: 30, // Optional delay
|
||||
},
|
||||
})
|
||||
}
|
||||
disabled={isPending}
|
||||
>
|
||||
{isPending ? "Queueing..." : "Analyze User Data"}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### From a server action
|
||||
|
||||
```ts title="apps/web/src/app/actions/user-actions.ts"
|
||||
"use server";
|
||||
|
||||
import { handle } from "@turbostarter/api/utils";
|
||||
|
||||
import { api } from "~/lib/api/server";
|
||||
|
||||
export async function processUserData(
|
||||
userId: string,
|
||||
operation: "export" | "analyze" | "cleanup",
|
||||
) {
|
||||
try {
|
||||
const result = await handle(api.trigger["process-user-data"].$post)({
|
||||
json: { userId, operation },
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
messageId: result.messageId,
|
||||
};
|
||||
} catch (error) {
|
||||
console.error("Failed to queue background task:", error);
|
||||
throw new Error("Failed to queue background task");
|
||||
}
|
||||
}
|
||||
```
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
## Advanced features
|
||||
|
||||
### Cron jobs & scheduling
|
||||
|
||||
QStash makes it easy to schedule recurring tasks:
|
||||
|
||||
```ts
|
||||
// Schedule a task to run every day at 2 AM
|
||||
await qstashClient.schedules.create({
|
||||
destination: `${baseUrl}/api/tasks/daily-cleanup`,
|
||||
cron: "0 2 * * *",
|
||||
});
|
||||
|
||||
// Schedule a task to run every Monday at 9 AM
|
||||
await qstashClient.schedules.create({
|
||||
destination: `${baseUrl}/api/tasks/weekly-report`,
|
||||
cron: "0 9 * * 1",
|
||||
});
|
||||
|
||||
// One-time delayed task
|
||||
await qstashClient.publishJSON({
|
||||
url: `${baseUrl}/api/tasks/reminder`,
|
||||
body: { userId: "123", type: "follow-up" },
|
||||
delay: "3d", // 3 days from now
|
||||
});
|
||||
```
|
||||
|
||||
### Topics (Fanout pattern)
|
||||
|
||||
Create topics to send messages to multiple endpoints:
|
||||
|
||||
```ts
|
||||
// Create a topic
|
||||
await qstashClient.topics.upsert({
|
||||
name: "user-events",
|
||||
endpoints: [
|
||||
{ url: `${baseUrl}/api/tasks/update-analytics` },
|
||||
{ url: `${baseUrl}/api/tasks/send-notification` },
|
||||
{ url: `${baseUrl}/api/tasks/update-crm` },
|
||||
],
|
||||
});
|
||||
|
||||
// Publish to topic - all endpoints will receive the message
|
||||
await qstashClient.publishJSON({
|
||||
topic: "user-events",
|
||||
body: {
|
||||
userId: "123",
|
||||
event: "user-registered",
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Queues (Sequential processing)
|
||||
|
||||
Create queues for ordered task processing:
|
||||
|
||||
```ts
|
||||
// Create a queue
|
||||
const queue = qstashClient.queue({ queueName: "user-onboarding" });
|
||||
|
||||
// Add tasks to queue (they'll run in order)
|
||||
await queue.enqueueJSON({
|
||||
url: `${baseUrl}/api/tasks/send-welcome-email`,
|
||||
body: { userId: "123" },
|
||||
});
|
||||
|
||||
await queue.enqueueJSON({
|
||||
url: `${baseUrl}/api/tasks/setup-user-profile`,
|
||||
body: { userId: "123" },
|
||||
});
|
||||
|
||||
await queue.enqueueJSON({
|
||||
url: `${baseUrl}/api/tasks/trigger-onboarding-sequence`,
|
||||
body: { userId: "123" },
|
||||
});
|
||||
```
|
||||
|
||||
## Monitoring and debugging
|
||||
|
||||
### QStash Dashboard
|
||||
|
||||
Visit the [Upstash Console](https://console.upstash.com) to monitor your tasks:
|
||||
|
||||
* **Message tracking**: See all messages, their status, and delivery attempts
|
||||
* **Logs**: View detailed logs for each message delivery
|
||||
* **Analytics**: Monitor throughput, success rates, and error patterns
|
||||
* **Schedules**: Manage and monitor your cron jobs
|
||||
* **Dead letter queue**: Handle messages that failed after all retries
|
||||
|
||||
### Local development
|
||||
|
||||
During development, you can:
|
||||
|
||||
1. **Use ngrok** for local testing:
|
||||
|
||||
```bash
|
||||
# Install ngrok
|
||||
npm install -g ngrok
|
||||
|
||||
# Expose your local server
|
||||
ngrok http 3000
|
||||
|
||||
# Use the ngrok URL in your QStash configuration
|
||||
```
|
||||
|
||||
2. **Check message delivery** in the Upstash Console
|
||||
|
||||
3. **Use console.log** in your task handlers for debugging
|
||||
|
||||
## Best practices
|
||||
|
||||
<Accordions>
|
||||
<Accordion title="Always verify signatures">
|
||||
Use the QStash signature verification middleware to ensure messages are authentic:
|
||||
|
||||
```ts
|
||||
// ✅ Good - Always verify QStash signatures
|
||||
.use(qstashVerifyMiddleware)
|
||||
|
||||
// ❌ Not secure - Accepting unverified requests
|
||||
.post("/tasks/sensitive-operation", handler)
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Handle errors gracefully">
|
||||
Return appropriate HTTP status codes so QStash knows whether to retry:
|
||||
|
||||
```ts
|
||||
// ✅ Good - Clear error handling
|
||||
try {
|
||||
await processTask(payload);
|
||||
return c.json({ success: true });
|
||||
} catch (error) {
|
||||
console.error("Task failed:", error);
|
||||
// 5xx = QStash will retry, 4xx = won't retry
|
||||
return c.json({ error: "Task failed" }, 500);
|
||||
}
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Use idempotent operations">
|
||||
Make your tasks safe to run multiple times in case of retries:
|
||||
|
||||
```ts
|
||||
// ✅ Good - Check if work already done
|
||||
const existingResult = await db.findProcessedResult(payload.id);
|
||||
if (existingResult) {
|
||||
return c.json({ success: true, result: existingResult });
|
||||
}
|
||||
|
||||
// Proceed with processing...
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Set appropriate timeouts">
|
||||
Configure timeouts based on your expected processing time:
|
||||
|
||||
```ts
|
||||
// For quick tasks
|
||||
await qstashClient.publishJSON({
|
||||
url: taskUrl,
|
||||
body: payload,
|
||||
timeout: "30s",
|
||||
});
|
||||
|
||||
// For longer tasks
|
||||
await qstashClient.publishJSON({
|
||||
url: taskUrl,
|
||||
body: payload,
|
||||
timeout: "300s", // 5 minutes
|
||||
});
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Use structured logging">
|
||||
Include relevant context in your logs:
|
||||
|
||||
```ts
|
||||
console.log("Task started", {
|
||||
taskType: "process-user-data",
|
||||
userId: payload.userId,
|
||||
operation: payload.operation,
|
||||
timestamp: new Date().toISOString(),
|
||||
});
|
||||
```
|
||||
</Accordion>
|
||||
</Accordions>
|
||||
|
||||
## Next steps
|
||||
|
||||
With QStash integrated into your TurboStarter application, you can now:
|
||||
|
||||
* **Process background tasks** without worrying about serverless timeouts
|
||||
* **Schedule recurring operations** with reliable cron job functionality
|
||||
* **Handle high-volume messaging** with automatic retries and scaling
|
||||
* **Build complex workflows** using topics, queues, and delays
|
||||
|
||||
Ready to explore more advanced features? Check out the official documentation for webhooks, batch operations, and advanced routing patterns.
|
||||
|
||||
<Cards>
|
||||
<Card title="Documentation" description="upstash.com" href="https://upstash.com/docs/qstash" />
|
||||
|
||||
<Card title="Dashboard" description="console.upstash.com" href="https://console.upstash.com" />
|
||||
</Cards>
|
||||
@@ -0,0 +1,505 @@
|
||||
---
|
||||
title: trigger.dev
|
||||
description: Integrate trigger.dev with your TurboStarter application for reliable background task processing.
|
||||
url: /docs/web/background-tasks/trigger
|
||||
---
|
||||
|
||||
# trigger.dev
|
||||
|
||||
[trigger.dev](https://trigger.dev) is an open-source background jobs framework that lets you write reliable workflows in plain async code.
|
||||
|
||||
<Callout title="Why trigger.dev?">
|
||||
trigger.dev provides automatic retries, real-time monitoring, and seamless scaling - all while letting you write background tasks in familiar JavaScript/TypeScript code directly in your TurboStarter project.
|
||||
</Callout>
|
||||
|
||||
<Steps>
|
||||
<Step>
|
||||
## Setup
|
||||
|
||||
Visit [trigger.dev](https://trigger.dev) and create a free account. Create a new project and note down your API key.
|
||||
|
||||
Add your trigger.dev API key to your root environment variables:
|
||||
|
||||
```dotenv title=".env.local"
|
||||
TRIGGER_SECRET_KEY=your_secret_key_here
|
||||
```
|
||||
|
||||
For production, make sure to add the production API key to your deployment environment.
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Create a new package in your repository
|
||||
|
||||
You can use the [Turbo generator](/docs/web/customization/add-package) to quickly scaffold the package structure:
|
||||
|
||||
```bash
|
||||
turbo gen package
|
||||
```
|
||||
|
||||
When prompted, name your package `tasks`. This will create the basic structure for you.
|
||||
|
||||
Alternatively, create a new folder `tasks` in the `/packages` directory and add the following files:
|
||||
|
||||
<Tabs items={["package.json", "tsconfig.json", "trigger.config.ts"]}>
|
||||
<Tab value="package.json">
|
||||
```json
|
||||
{
|
||||
"name": "@turbostarter/tasks",
|
||||
"private": true,
|
||||
"version": "0.1.0",
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": "./src/index.ts"
|
||||
},
|
||||
"scripts": {
|
||||
"clean": "git clean -xdf .cache .turbo dist node_modules",
|
||||
"dev": "pnpm dlx trigger.dev@latest dev",
|
||||
"deploy": "pnpm dlx trigger.dev@latest deploy",
|
||||
"format": "prettier --check . --ignore-path ../../.gitignore",
|
||||
"lint": "eslint",
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@trigger.dev/sdk": "3.3.17"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@trigger.dev/build": "3.3.17",
|
||||
"@turbostarter/eslint-config": "workspace:*",
|
||||
"@turbostarter/prettier-config": "workspace:*",
|
||||
"@turbostarter/tsconfig": "workspace:*",
|
||||
"eslint": "catalog:",
|
||||
"prettier": "catalog:",
|
||||
"typescript": "catalog:"
|
||||
},
|
||||
"prettier": "@turbostarter/prettier-config"
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab value="tsconfig.json">
|
||||
```json
|
||||
{
|
||||
"extends": "@turbostarter/tsconfig/base.json",
|
||||
"include": ["**/*.ts"],
|
||||
"exclude": ["dist", "build", "node_modules"]
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab value="trigger.config.ts">
|
||||
```ts
|
||||
import { defineConfig } from "@trigger.dev/sdk";
|
||||
|
||||
export default defineConfig({
|
||||
project: "your_project_id", // Replace with your actual project ID
|
||||
runtime: "node",
|
||||
logLevel: "log",
|
||||
maxDuration: 300,
|
||||
dirs: ["./src/trigger"],
|
||||
});
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Create your first task
|
||||
|
||||
Now create your first task in the `packages/tasks/src/trigger` directory:
|
||||
|
||||
<Tabs items={["process-user-data.ts", "daily-cleanup.ts", "src/index.ts"]}>
|
||||
<Tab value="process-user-data.ts">
|
||||
```ts title="packages/tasks/src/trigger/process-user-data.ts"
|
||||
import { task, logger, wait } from "@trigger.dev/sdk";
|
||||
import * as z from "zod";
|
||||
|
||||
const ProcessUserDataSchema = z.object({
|
||||
userId: z.string(),
|
||||
operation: z.enum(["export", "analyze", "cleanup"]),
|
||||
});
|
||||
|
||||
export const processUserDataTask = task({
|
||||
id: "process-user-data",
|
||||
run: async (payload: z.infer<typeof ProcessUserDataSchema>) => {
|
||||
const { userId, operation } = payload;
|
||||
|
||||
logger.info("Starting user data processing", { userId, operation });
|
||||
|
||||
switch (operation) {
|
||||
case "export":
|
||||
await wait.for({ seconds: 2 });
|
||||
logger.info("User data exported successfully");
|
||||
return { success: true, result: "Data exported to CSV" };
|
||||
|
||||
case "analyze":
|
||||
await wait.for({ seconds: 5 });
|
||||
logger.info("User data analysis completed");
|
||||
return {
|
||||
success: true,
|
||||
result: { totalActions: 156, avgSessionTime: "4m 32s" },
|
||||
};
|
||||
|
||||
case "cleanup":
|
||||
await wait.for({ seconds: 3 });
|
||||
logger.info("User data cleanup completed");
|
||||
return { success: true, result: "Removed 23 obsolete records" };
|
||||
|
||||
default:
|
||||
throw new Error(`Unknown operation: ${operation}`);
|
||||
}
|
||||
},
|
||||
});
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab value="daily-cleanup.ts">
|
||||
```ts title="packages/tasks/src/trigger/daily-cleanup.ts"
|
||||
import { schedules, task, logger, wait } from "@trigger.dev/sdk";
|
||||
|
||||
export const dailyCleanupTask = task({
|
||||
id: "daily-cleanup",
|
||||
run: async () => {
|
||||
logger.info("Starting daily cleanup");
|
||||
|
||||
// Cleanup old logs
|
||||
await wait.for({ seconds: 5 });
|
||||
logger.info("Logs cleaned up");
|
||||
|
||||
// Cleanup temporary files
|
||||
await wait.for({ seconds: 3 });
|
||||
logger.info("Temp files cleaned up");
|
||||
|
||||
// Generate daily reports
|
||||
await wait.for({ seconds: 8 });
|
||||
logger.info("Reports generated");
|
||||
|
||||
return {
|
||||
success: true,
|
||||
cleanupTime: new Date().toISOString(),
|
||||
itemsProcessed: 1247,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
// Schedule the task to run daily at 2 AM
|
||||
schedules.create({
|
||||
task: "daily-cleanup",
|
||||
cron: "0 2 * * *",
|
||||
});
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab value="src/index.ts">
|
||||
```ts title="packages/tasks/src/index.ts"
|
||||
export * from "./trigger/process-user-data";
|
||||
export * from "./trigger/daily-cleanup";
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Test your task
|
||||
|
||||
You can test your tasks locally by running:
|
||||
|
||||
```bash
|
||||
# Start the development server
|
||||
pnpm --filter @turbostarter/tasks dev
|
||||
```
|
||||
|
||||
This will deploy your tasks to trigger.dev in the development environment, allowing you to trigger them from the dashboard or programmatically.
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Deploy your tasks
|
||||
|
||||
To deploy your tasks to production on trigger.dev, run:
|
||||
|
||||
```bash
|
||||
pnpm --filter @turbostarter/tasks deploy
|
||||
```
|
||||
|
||||
You can also add this command as an automated deployment step in your CI/CD pipeline by creating a new GitHub action.
|
||||
|
||||
Add the `TRIGGER_ACCESS_TOKEN` secret to your repository secrets, which you can create in the trigger.dev dashboard.
|
||||
|
||||
```yml title=".github/workflows/deploy-tasks.yml"
|
||||
name: Deploy to trigger.dev (prod)
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: lts/*
|
||||
- uses: pnpm/action-setup@v4
|
||||
- name: Install dependencies
|
||||
run: pnpm install
|
||||
- name: Deploy trigger tasks
|
||||
env:
|
||||
TRIGGER_ACCESS_TOKEN: ${{ secrets.TRIGGER_ACCESS_TOKEN }}
|
||||
run: |
|
||||
pnpm --filter @turbostarter/tasks deploy
|
||||
```
|
||||
</Step>
|
||||
|
||||
<Step>
|
||||
## Triggering tasks
|
||||
|
||||
You can trigger tasks from your TurboStarter application using the API layer.
|
||||
|
||||
<Callout type="warning" title="Direct task triggering not recommended">
|
||||
While you can trigger tasks directly from your frontend or server components using the trigger.dev SDK, it's recommended to use the API layer approach shown below.
|
||||
|
||||
This provides better security, validation, and separation of concerns.
|
||||
</Callout>
|
||||
|
||||
First, add the `@turbostarter/tasks` package as a dependency to your API package:
|
||||
|
||||
```json title="packages/api/package.json"
|
||||
{
|
||||
"dependencies": {
|
||||
"@turbostarter/tasks": "workspace:*"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### From an API endpoint
|
||||
|
||||
Create a new API module to handle task triggering:
|
||||
|
||||
```ts title="packages/api/src/modules/tasks/tasks.router.ts"
|
||||
import { tasks } from "@trigger.dev/sdk";
|
||||
import { Hono } from "hono";
|
||||
import * as z from "zod";
|
||||
import type { processUserDataTask } from "@turbostarter/tasks";
|
||||
|
||||
import { enforceAuth, validate } from "../../middleware";
|
||||
|
||||
const processUserDataSchema = z.object({
|
||||
userId: z.string(),
|
||||
operation: z.enum(["export", "analyze", "cleanup"]),
|
||||
});
|
||||
|
||||
export const tasksRouter = new Hono().post(
|
||||
"/process-user-data",
|
||||
enforceAuth,
|
||||
validate("json", processUserDataSchema),
|
||||
async (c) => {
|
||||
const { userId, operation } = c.req.valid("json");
|
||||
|
||||
const handle = await tasks.trigger<typeof processUserDataTask>(
|
||||
"process-user-data",
|
||||
{ userId, operation },
|
||||
);
|
||||
|
||||
return c.json({
|
||||
success: true,
|
||||
taskId: handle.id,
|
||||
message: "Background task started successfully",
|
||||
});
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
Then register it in your main API router:
|
||||
|
||||
```ts title="packages/api/src/index.ts"
|
||||
import { tasksRouter } from "./modules/tasks/tasks.router";
|
||||
|
||||
const appRouter = new Hono()
|
||||
.basePath("/api")
|
||||
.route("/tasks", tasksRouter)
|
||||
// ... other existing routers
|
||||
.onError(onError);
|
||||
|
||||
export { appRouter };
|
||||
```
|
||||
|
||||
### From the client
|
||||
|
||||
You can call the task endpoint from your web app using TurboStarter's API client:
|
||||
|
||||
```tsx title="apps/web/src/modules/tasks/process-data-button.tsx"
|
||||
"use client";
|
||||
|
||||
import { handle } from "@turbostarter/api/utils";
|
||||
import { useMutation } from "@tanstack/react-query";
|
||||
|
||||
import { api } from "~/lib/api/client";
|
||||
|
||||
export function ProcessDataButton({ userId }: { userId: string }) {
|
||||
const { mutate: processData, isPending } = useMutation({
|
||||
mutationFn: handle(api.tasks["process-user-data"].$post),
|
||||
onSuccess: (data) => {
|
||||
console.log("Task started:", data.taskId);
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<button
|
||||
onClick={() =>
|
||||
processData({
|
||||
json: { userId, operation: "analyze" },
|
||||
})
|
||||
}
|
||||
disabled={isPending}
|
||||
>
|
||||
{isPending ? "Processing..." : "Analyze User Data"}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### From a server action
|
||||
|
||||
```ts title="apps/web/src/app/actions/user-actions.ts"
|
||||
"use server";
|
||||
|
||||
import { handle } from "@turbostarter/api/utils";
|
||||
|
||||
import { api } from "~/lib/api/server";
|
||||
|
||||
export async function processUserData(userId: string, operation: string) {
|
||||
try {
|
||||
const result = await handle(api.tasks["process-user-data"].$post)({
|
||||
json: { userId, operation },
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
taskId: result.taskId,
|
||||
};
|
||||
} catch (error) {
|
||||
console.error("Failed to trigger background task:", error);
|
||||
throw new Error("Failed to start background task");
|
||||
}
|
||||
}
|
||||
```
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
## Monitoring and debugging
|
||||
|
||||
### Dashboard access
|
||||
|
||||
Visit the [trigger.dev dashboard](https://trigger.dev) to monitor your tasks:
|
||||
|
||||
* View task execution logs and performance metrics
|
||||
* Track success and failure rates across all your tasks
|
||||
* Monitor task duration and resource usage
|
||||
* Replay failed tasks with a single click
|
||||
* Set up alerts for task failures or performance issues
|
||||
|
||||
### Local development
|
||||
|
||||
During development, run your tasks locally while connected to trigger.dev:
|
||||
|
||||
```bash
|
||||
# Start everything in the workspace
|
||||
pnpm dev
|
||||
|
||||
# or start the tasks package only
|
||||
pnpm --filter @turbostarter/tasks dev
|
||||
```
|
||||
|
||||
This allows you to:
|
||||
|
||||
* Test tasks locally with real data
|
||||
* Debug with breakpoints and console logs
|
||||
* See immediate feedback as you develop
|
||||
|
||||
## Best practices
|
||||
|
||||
<Accordions>
|
||||
<Accordion title="Use descriptive task IDs">
|
||||
```ts
|
||||
// ✅ Good - Clear and descriptive
|
||||
id: "user-data-export-csv";
|
||||
id: "weekly-newsletter-campaign";
|
||||
id: "cleanup-temp-files";
|
||||
|
||||
// ❌ Not so good - Generic and unclear
|
||||
id: "task1";
|
||||
id: "job";
|
||||
id: "process";
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Include proper error handling">
|
||||
```ts
|
||||
run: async (payload) => {
|
||||
try {
|
||||
const result = await processData(payload);
|
||||
logger.info("Task completed successfully", { result });
|
||||
return result;
|
||||
} catch (error) {
|
||||
logger.error("Task failed:", error.message);
|
||||
throw error; // Re-throw to trigger retry logic
|
||||
}
|
||||
},
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Use structured logging">
|
||||
```ts
|
||||
logger.info("Processing started", {
|
||||
userId: payload.userId,
|
||||
operation: payload.operation,
|
||||
timestamp: new Date().toISOString(),
|
||||
});
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Keep tasks focused">
|
||||
Instead of one massive task, create focused, single-purpose tasks that can be composed together for complex workflows.
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Configure appropriate retries">
|
||||
Set retry policies based on your task's requirements:
|
||||
|
||||
```ts
|
||||
// For critical operations
|
||||
retry: {
|
||||
maxAttempts: 5,
|
||||
minTimeoutInMs: 2000,
|
||||
maxTimeoutInMs: 30000,
|
||||
factor: 2,
|
||||
}
|
||||
|
||||
// For less critical operations
|
||||
retry: {
|
||||
maxAttempts: 2,
|
||||
minTimeoutInMs: 1000,
|
||||
maxTimeoutInMs: 5000,
|
||||
factor: 1.5,
|
||||
}
|
||||
```
|
||||
</Accordion>
|
||||
</Accordions>
|
||||
|
||||
## Next steps
|
||||
|
||||
With trigger.dev integrated into your TurboStarter application, you can now:
|
||||
|
||||
* **Handle long-running operations** that would timeout in serverless functions
|
||||
* **Schedule recurring tasks** like reports, cleanups, and maintenance
|
||||
* **Process background jobs** reliably with automatic retries
|
||||
* **Scale your application** without worrying about task execution infrastructure
|
||||
|
||||
Ready to explore more advanced features? Check out the official documentation for additional capabilities like webhooks, batching, and custom integrations.
|
||||
|
||||
<Cards>
|
||||
<Card title="Documentation" description="trigger.dev" href="https://trigger.dev/docs" />
|
||||
|
||||
<Card title="Examples" description="trigger.dev" href="https://trigger.dev/docs/guides/introduction" />
|
||||
</Cards>
|
||||
Reference in New Issue
Block a user