I've heard about Telegram and its bots for a while but never understood exactly why it'd be useful. Yet another chat? I finally looked into and discovered how it's useful to me, someone who hacks on side projects for fun and maybe profit. 🤞

Push Notifications

It's usefulness really comes down to me being able to easily send myself push notifications and a simple interface to view them.

I'm hacking on a side project and I wanted to be notified if/when users do certain actions:

• sign up
• perform an onboarding action
• reach an api limit
• etc

This gives me an opportunity to each out to the user if I feel like it may be appropriate to do so. Yes, this won't scale but this is for my hobby projects. I'm not worried about scale.

I could use email but then I need to figure out email credentials to be able to actually send them and have to use an email client to read the messages which are only going to be a sentence long. I find a dedicated chat interface to be much nicer for this use case. I could use Slack but creating my own slack workspace for myself seems like overkill. Not to mention the Slack api takes some time to familiarize yourself with.

Telegram Speed Run

There's an extensive tutorial from Telegram that you can follow but it's pretty long and uses java. So here's a very condensed version that uses node.

First, install the app on your phone and computer.

Bots

There are two "bots" we're going to use to help set things up. A bot is just a chat application within Telegram.

Go here to start using the "BotFather" bot. This bot helps you create bots. Enter /newbot in a chat with the BotFather and follow the prompts to create new bot.

We'll be using the bot's username and the access token shortly.

Next, we need to get the id of our own account so we can send messages from the bot account to ourselves. We'll use another bot for this. Type userinfobot in the Contacts search bar, select the bot and click "Start". It will immediately respond with a message containing your Id.

Sending a message

You can use a super simple http request with your bot's access token and your chat id.

POST https://api.telegram.org/bot<telegram_bot_token>/sendMessage
Content-Type: application/json

{
    "chat_id": <chat_id>,
    "text": "Is this thing on??"
}

Note that the token is prefixed with bot and there's no space between it and the actual token.

If you've installed the mobile client you should see a push notification.

Sending messages in Node

In my project sending these notifications isn't critical to the business logic so I like to fire and forget the sending of these messages.

import { env } from "~/env.server";

export async function sendTelegram(message: string) {
  const url = `https://api.telegram.org/bot${env.TELEGRAM_BOT_TOKEN}/sendMessage`;
  return fetch(url, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      chat_id: env.TELEGRAM_MY_ID,
      text: message,
    }),
  });
}

And then somewhere else in the application:

const user = await createUser(data);
await sendConfirmationEmail(user)

sendTelegram(`New user signup: ${data.email}`).catch(console.error)

return redirect('/confirm-user')

Notice the sendTelegram function is not being awaited and there's a catch to ensure any error doesn't bubble up and crash the application. This means the user isn't waiting on the process of interacting with the telegram api to finish. I want the app to be as snappy as possible with the least amount of effort.

I just recently wrote a guide on Remix file uploads and asked the Remix folks for some feedback. Turns out Michael, from Remix, has been cooking up some new apis for this very thing that he encouraged me to use.

Here we go again 😅.

<Form />

To use the Remix <Form /> component with files you must use the encType prop and update the method to "post" or "put":

<Form method='post' encType='multipart/form-data'>
    <input type='file' name='my-file' />
    <button>Submit</button>
</Form>

Parse the form

Next, we need to receive the form data in a route action. We could just use a native function:

export async function action ({request}: ActionFunctionArgs) {
  const formData = await request.formData()
  const file = formData.get('my-file')  
  // process the file
}

However, this loads the entire uploaded file contents into server memory. A much more efficient way to handle files would be to stream the uploads to their destination. This way our server never holds the entire contents of large files in memory at one time.

This is exactly what the @mjackson/form-data-parser package is for. It creates intermediary FileUpload instances that extend the File class that can be streamed to their destination. Clever stuff.

Here's what it looks like.

import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'

export async function action ({request}: ActionFunctionArgs) {

    const uploadHandler = async (fileUpload: FileUpload) => {
        if (fileUpload.fieldName === 'my-file') {
            // process the upload and return a File
        }

        // ignore any unrecognized files
    }

    const formData = await parseFormData(request, uploadHandler)

    // 'my-file' has already been processed at this point 
    const file = formData.get('my-file')  
}

Adapters

The @mjackson/form-data-parser package is designed in a way that allows for implementing various storage destinations. It uses another package that defines a FileStorage interface. Your storage destination could be anything, you just need to create a class that implements this interface and you're good to go. However, you'll be dealing with streams which can be tricky. I found this helpful to understand the basics.

Luckily, this package comes with two basic adapters out of the box: MemoryFileStorage and LocalFileStorage.

Local file storage

Want to store uploads directly on your server? That's what LocalFileStorage is for.

import { LocalFileStorage } from '@mjackson/file-storage/local';
import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'

const fileStorage = new LocalFileStorage('/uploads/user-avatars');

export async function action ({request}: ActionFunctionArgs) {
    // however you authenticate users
    const user = await getUser(request)

    const uploadHandler = async (fileUpload: FileUpload) => {
        if (fileUpload.fieldName === 'my-file') {
            let storageKey = `user-${user.id}-avatar`;

            await fileStorage.set(storageKey, fileUpload);

            return fileStorage.get(storageKey);
        }

        // Ignore any files we don't recognize the name of...
    }
    const formData = await parseFormData(request, uploadHandler)
    const file = formData.get('my-file')  
    // file has been processed
}

Memory File Storage

This stores the uploads in an in-memory Map but it doesn't work the way I expect it to. I'm going to hold off on my description until I get some clarification.

S3

I have opened a pull request to merge in an S3 adapter. I don't know if/when that will be merged but you can grab the implementation here.

const fileStorage = new S3FileStorage(s3Client, 'bucket-name')

R2

Sergio Xalambrí just published a package that implements the FileStorage interface for R2. Check it out here. Be aware that it does not take advantage of streaming files as far as I can tell. It will load them into memory and then push them to R2.

SFTP

After lots of trial and error I was also able to come up with an SFTP implementation if you want to upload and read files to and from a remote server. You can check it out here.

const fileStorage = new SftpFileStorage(
    {
        host: "1.23.456.789",
        username: "some-user",
        privateKey: fs.readFileSync("/home/some-user/.ssh/id_rsa"),
    },
    "/somewhere/on/your/server",
);

Gotchas

There are a couple of gotchas you need to be aware of.

Limiting file size

You definitely don't want to give an individual the power to take up all of your disk space. You can limit file size with the 3rd argument of parseFormData. The _unstable_parseMultipartFormData function defaults to have a max file size of 3MB. I don't believe parseFormData has a default max which means it will allow everything.

Here's how you would do it with parseFormData.

const formData = await parseFormData(request, uploadHandler, {
    maxFileSize: 3000000, // 3MB
});

Accessing other form data

Due to the nature of processing the uploads this way, you don't have access to the other form values because the FormData instance hasn't been created yet. For example, say you have another input in the same form for the user to select a category for the file that you need to do some validation on.

const formData = await parseFormData(request, uploadHandler)
const category = formData.get('category')

// Now I just need to valida....oh wait, my file has already been uploaded...

To work around this, you can use the useSubmit hook and append values as search params to be read in the action.

const submit = useSubmit()

const handleSubmit = (e: React.FormEvent<HTMLFormElement>) => {
    const formData = new FormData(e.currentTarget)
    const params = new URLSearchParams({category: formData.get('category')})
    const options = {
        action: `/app/upload?${params}`,
        method: "post",
        encType: "multipart/form-data"
    }
    submit(formData, options)
}

return (
    <Form
        method="post"
        action="/app/upload"
        encType="multipart/form-data"
        onSubmit={handleSubmit}
    >
        ...
    </Form>
)

And then in your action:

export async function action({request}: ActionFuntionArgs) {
    const url = new URL(request.url)
    const cateogry = url.searchParams.get('category')

    // continue on to use parseFormData()
}

Streaming files to the user

To complete the circle, here's how you can stream files to your users from a loader.

import invariant from "tiny-invariant";
import { fileStorage } from "~/your-file-storage.server";

export async function loader() {
  const storageKey = "some-file.png";
  const file = await fileStorage.get(storageKey);
  invariant(file, "invalid file");

  return new Response(file, {
    headers: {
      "Content-Type": file.type,
      "Content-Disposition": `attachment; filename=${storageKey}`,
    },
  });
}

As of writing this there is an official guide on file uploads in Remix but it's basically a placeholder until they iron out some APIs and have time to revisit the guide.

This is my unofficial guide.

<Form />

To use the Remix <Form /> component with files you must use the encType prop and update the method to "post" or "put":

<Form method='post' encType='multipart/form-data'>
    <input type='file' name='my-file' />
    <button>Submit</button>
</Form>

Now, in your route's action you can pull that File out of the form data.

export async function action({request}: ActionFuntionArgs) {
    const formData = await request.formData()
    const file = formData.get('my-file')
    if (!(file instanceof File)) throw new Error('Invalid file')

    // process file

    return {
        message: "File received"
    }
}

The uploaded file(s) will be stored in memory and you'll need to decide what to do with them.

Base64

You could convert the upload into a base64 value:

const formData = await request.formData()
const file = formData.get('my-file')
if (!(file instanceof File)) throw new Error('Invalid file')
const arrayBuffer = await file.arrayBuffer()
const buffer = Buffer.from(arrayBuffer)
const base64 = buffer.toString('base64')

But most likely you'll want to store them somewhere...

Local file system

Luckily, Remix has a handy utility for storing uploads locally. Here's what that looks like:

import {
    unstable_createFileUploadHandler,
    unstable_parseMultipartFormData
} from '@remix-run/node'

export async function action({request}: ActionFuntionArgs) {
    const uploadHandler = unstable_createFileUploadHandler({
        // where to save the file
        directory: '/app/storage',
        // use the incoming filename instead of creating a new name
        file: (args) => args.filename
    })
    const formData = await unstable_parseMultipartFormData(request, uploadHandler)

    return {
        message: "File received"
    }
}

unstable_parseMultipartFormData

You might have noticed that request.formData() went away and was replaced with this unstable_parseMultipartFormData function. This is a Remix utility for iteratively deciding where uploads go.

• request.formData() - read everything into memory and then decide what to do with file data
• unstable_parseMultipartFormData - decide what to do with file data as it's being read

This allows users to upload big files without the application storing those entirely in memory all at once. Instead, you can stream those files into their destinations.

The unstable_parseMultipartFormData function will iterate over the form data keys and if the key contains a file, it will hand that off to be processed by the upload handler.

Gotcha

One "gotcha" with parsing the form data this way is that you don't have access to the other form values because the FormData instance hasn't been created yet. For example, say you had another input in the same form for the user to select a category for the file that determines where that file lives on disk of if you need to do some validation on the provided category.

const formData = await unstable_parseMultipartFormData(request, uploadHandler)
const category = formData.get('category')

// Now I just need to valida....oh wait, my file has already been uploaded...

To get around this, you can use the useSubmit hook and append values as search params to be read in the action.

const submit = useSubmit()

return (
    <Form method="post" onSubmit={(e) => {
        const formData = new FormData(e.currentTarget)
        const params = new URLSearchParams({category: formData.get('category')})
        const options = {
            action: `/app/upload?${params}`,
            method: "post",
            encType: "multipart/form-data"
        }
        submit(formData, options)
    }}>
)

And then in your action:

export async function action({request}: ActionFuntionArgs) {
    const url = new URL(request.url)
    const cateogry = url.searchParams.get('category')

    // continue on to use unstable_parseMultipartFormData()
}

Custom upload handlers

Remix allows you to create your very own upload handlers.

const uploadHandler: UploadHandler = async ({ data, filename }) => {

    // handle the data and filename however you need to

    // return File | string | null | undefined
    return "https://example.com/assets/newly-uploaded-file.png";
}
const formData = await unstable_parseMultipartFormData(request, uploadHandler);

// Remix swaps in whatever you return from your upload handler
const uploadUrl = formData.get('my-file')

Composing upload handlers

Remix also comes with a utility for using multiple upload handlers to handle certain uploads differently. The following is lifted directly from the Remix docs.

  const uploadHandler = unstable_composeUploadHandlers(
    // custom upload handler
    async ({ name, contentType, data, filename }) => {
        if (name !== "img") {
            return undefined;
        }
        const uploadedImage = await uploadImageToCloudinary(data);
        return uploadedImage.secure_url;
    },
    // fallback to memory for everything else
    unstable_createMemoryUploadHandler()
);

const formData = await unstable_parseMultipartFormData(
    request,
    uploadHandler
);

In this example, the first handler only applies to the upload coming from the input with the name of "img":

<input type='file' name='img' />

By returning undefined, Remix will try the next upload handler. In this case, it's the unstable_createMemoryUploadHandler.

unstable_createMemoryUploadHandler

Wait, why is there a unstable_createMemoryUploadHandler? Wouldn't that be the same as request.formData()? Yep, almost. This utility does come with the ability to limit the upload size. It defaults to 3MB.

const uploadHandler = unstable_createMemoryUploadHandler({maxPartSize: 5000000}) // 5MB
const formData = await unstable_parseMultipartFormData(
    request,
    uploadHandler
);
const file = formData.get('my-file')

// up to you to handle now

Upload to S3

Here's an example that illustrates efficiently streaming the upload to S3.

import {
  type ActionFunctionArgs,
  json,
  unstable_parseMultipartFormData,
  type UploadHandler,
} from "@remix-run/node";
import { S3Client } from "@aws-sdk/client-s3";
import { env } from "~/env.server";
import { Upload } from "@aws-sdk/lib-storage";
import { Readable } from "stream";

// you probably wouldn't actually create the s3 client here...
const s3 = new S3Client({
  region: env.AWS_REGION,
  credentials: {
    accessKeyId: env.AWS_ACCESS_KEY_ID,
    secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
  },
});

export async function action({ request }: ActionFunctionArgs) {
  const uploadHandler: UploadHandler = async ({
    data,
    filename,
  }) => {
    const readableStream = Readable.from(data);
    const upload = new Upload({
      client: s3,
      params: {
        Bucket: env.AWS_S3_BUCKET,
        Key: filename,
        Body: readableStream,
      },
    });
    await upload.done();
    return filename;
  };

  await unstable_parseMultipartFormData(request, uploadHandler);

  return json({
    success: true,
  });
}

Hope this helps!