File uploads with Remix (Updated)

I just recently wrote a guide on Remix file uploads and asked the Remix folks for some feedback. Turns out Michael, from Remix, has been cooking up some new apis for this very thing that he encouraged me to use.

Here we go again 😅.

<Form />

To use the Remix <Form /> component with files you must use the encType prop and update the method to "post" or "put":

<Form method='post' encType='multipart/form-data'>
    <input type='file' name='my-file' />
    <button>Submit</button>
</Form>

Parse the form

Next, we need to receive the form data in a route action. We could just use a native function:

export async function action ({request}: ActionFunctionArgs) {
  const formData = await request.formData()
  const file = formData.get('my-file')  
  // process the file
}

However, this loads the entire uploaded file contents into server memory. A much more efficient way to handle files would be to stream the uploads to their destination. This way our server never holds the entire contents of large files in memory at one time.

This is exactly what the @mjackson/form-data-parser package is for. It creates intermediary FileUpload instances that extend the File class that can be streamed to their destination. Clever stuff.

Here's what it looks like.

import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'

export async function action ({request}: ActionFunctionArgs) {

    const uploadHandler = async (fileUpload: FileUpload) => {
        if (fileUpload.fieldName === 'my-file') {
            // process the upload and return a File
        }

        // ignore any unrecognized files
    }

    const formData = await parseFormData(request, uploadHandler)

    // 'my-file' has already been processed at this point 
    const file = formData.get('my-file')  
}

Adapters

The @mjackson/form-data-parser package is designed in a way that allows for implementing various storage destinations. It uses another package that defines a FileStorage interface. Your storage destination could be anything, you just need to create a class that implements this interface and you're good to go. However, you'll be dealing with streams which can be tricky. I found this helpful to understand the basics.

Luckily, this package comes with two basic adapters out of the box: MemoryFileStorage and LocalFileStorage.

Local file storage

Want to store uploads directly on your server? That's what LocalFileStorage is for.

import { LocalFileStorage } from '@mjackson/file-storage/local';
import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'

const fileStorage = new LocalFileStorage('/uploads/user-avatars');

export async function action ({request}: ActionFunctionArgs) {
    // however you authenticate users
    const user = await getUser(request)

    const uploadHandler = async (fileUpload: FileUpload) => {
        if (fileUpload.fieldName === 'my-file') {
            let storageKey = `user-${user.id}-avatar`;

            await fileStorage.set(storageKey, fileUpload);

            return fileStorage.get(storageKey);
        }

        // Ignore any files we don't recognize the name of...
    }
    const formData = await parseFormData(request, uploadHandler)
    const file = formData.get('my-file')  
    // file has been processed
}

Memory File Storage

This stores the uploads in an in-memory Map but it doesn't work the way I expect it to. I'm going to hold off on my description until I get some clarification.

S3

I have opened a pull request to merge in an S3 adapter. I don't know if/when that will be merged but you can grab the implementation here.

const fileStorage = new S3FileStorage(s3Client, 'bucket-name')

R2

Sergio Xalambrí just published a package that implements the FileStorage interface for R2. Check it out here. Be aware that it does not take advantage of streaming files as far as I can tell. It will load them into memory and then push them to R2.

SFTP

After lots of trial and error I was also able to come up with an SFTP implementation if you want to upload and read files to and from a remote server. You can check it out here.

const fileStorage = new SftpFileStorage(
    {
        host: "1.23.456.789",
        username: "some-user",
        privateKey: fs.readFileSync("/home/some-user/.ssh/id_rsa"),
    },
    "/somewhere/on/your/server",
);

Gotchas

There are a couple of gotchas you need to be aware of.

Limiting file size

You definitely don't want to give an individual the power to take up all of your disk space. You can limit file size with the 3rd argument of parseFormData. The _unstable_parseMultipartFormData function defaults to have a max file size of 3MB. I don't believe parseFormData has a default max which means it will allow everything.

Here's how you would do it with parseFormData.

const formData = await parseFormData(request, uploadHandler, {
    maxFileSize: 3000000, // 3MB
});

Accessing other form data

Due to the nature of processing the uploads this way, you don't have access to the other form values because the FormData instance hasn't been created yet. For example, say you have another input in the same form for the user to select a category for the file that you need to do some validation on.

const formData = await parseFormData(request, uploadHandler)
const category = formData.get('category')

// Now I just need to valida....oh wait, my file has already been uploaded...

To work around this, you can use the useSubmit hook and append values as search params to be read in the action.

const submit = useSubmit()

const handleSubmit = (e: React.FormEvent<HTMLFormElement>) => {
    const formData = new FormData(e.currentTarget)
    const params = new URLSearchParams({category: formData.get('category')})
    const options = {
        action: `/app/upload?${params}`,
        method: "post",
        encType: "multipart/form-data"
    }
    submit(formData, options)
}

return (
    <Form
        method="post"
        action="/app/upload"
        encType="multipart/form-data"
        onSubmit={handleSubmit}
    >
        ...
    </Form>
)

And then in your action:

export async function action({request}: ActionFuntionArgs) {
    const url = new URL(request.url)
    const cateogry = url.searchParams.get('category')

    // continue on to use parseFormData()
}

Streaming files to the user

To complete the circle, here's how you can stream files to your users from a loader.

import invariant from "tiny-invariant";
import { fileStorage } from "~/your-file-storage.server";

export async function loader() {
  const storageKey = "some-file.png";
  const file = await fileStorage.get(storageKey);
  invariant(file, "invalid file");

  return new Response(file, {
    headers: {
      "Content-Type": file.type,
      "Content-Disposition": `attachment; filename=${storageKey}`,
    },
  });
}

Categories: Remix