I just recently wrote a guide on Remix file uploads and asked the Remix folks for some feedback. Turns out Michael, from Remix, has been cooking up some new apis for this very thing that he encouraged me to use.

Here we go again 😅.

<Form />

To use the Remix <Form /> component with files you must use the encType prop and update the method to "post" or "put":

<Form method='post' encType='multipart/form-data'>
    <input type='file' name='my-file' />
    <button>Submit</button>
</Form>

Parse the form

Next, we need to receive the form data in a route action. We could just use a native function:

export async function action ({request}: ActionFunctionArgs) {
  const formData = await request.formData()
  const file = formData.get('my-file')  
  // process the file
}

However, this loads the entire uploaded file contents into server memory. A much more efficient way to handle files would be to stream the uploads to their destination. This way our server never holds the entire contents of large files in memory at one time.

This is exactly what the @mjackson/form-data-parser package is for. It creates intermediary FileUpload instances that extend the File class that can be streamed to their destination. Clever stuff.

Here's what it looks like.

import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'

export async function action ({request}: ActionFunctionArgs) {

    const uploadHandler = async (fileUpload: FileUpload) => {
        if (fileUpload.fieldName === 'my-file') {
            // process the upload and return a File
        }

        // ignore any unrecognized files
    }

    const formData = await parseFormData(request, uploadHandler)

    // 'my-file' has already been processed at this point 
    const file = formData.get('my-file')  
}

Adapters

The @mjackson/form-data-parser package is designed in a way that allows for implementing various storage destinations. It uses another package that defines a FileStorage interface. Your storage destination could be anything, you just need to create a class that implements this interface and you're good to go. However, you'll be dealing with streams which can be tricky. I found this helpful to understand the basics.

Luckily, this package comes with two basic adapters out of the box: MemoryFileStorage and LocalFileStorage.

Local file storage

Want to store uploads directly on your server? That's what LocalFileStorage is for.

import { LocalFileStorage } from '@mjackson/file-storage/local';
import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'

const fileStorage = new LocalFileStorage('/uploads/user-avatars');

export async function action ({request}: ActionFunctionArgs) {
    // however you authenticate users
    const user = await getUser(request)

    const uploadHandler = async (fileUpload: FileUpload) => {
        if (fileUpload.fieldName === 'my-file') {
            let storageKey = `user-${user.id}-avatar`;

            await fileStorage.set(storageKey, fileUpload);

            return fileStorage.get(storageKey);
        }

        // Ignore any files we don't recognize the name of...
    }
    const formData = await parseFormData(request, uploadHandler)
    const file = formData.get('my-file')  
    // file has been processed
}

Memory File Storage

This stores the uploads in an in-memory Map but it doesn't work the way I expect it to. I'm going to hold off on my description until I get some clarification.

S3

I have opened a pull request to merge in an S3 adapter. I don't know if/when that will be merged but you can grab the implementation here.

const fileStorage = new S3FileStorage(s3Client, 'bucket-name')

R2

Sergio Xalambrí just published a package that implements the FileStorage interface for R2. Check it out here. Be aware that it does not take advantage of streaming files as far as I can tell. It will load them into memory and then push them to R2.

SFTP

After lots of trial and error I was also able to come up with an SFTP implementation if you want to upload and read files to and from a remote server. You can check it out here.

const fileStorage = new SftpFileStorage(
    {
        host: "1.23.456.789",
        username: "some-user",
        privateKey: fs.readFileSync("/home/some-user/.ssh/id_rsa"),
    },
    "/somewhere/on/your/server",
);

Gotchas

There are a couple of gotchas you need to be aware of.

Limiting file size

You definitely don't want to give an individual the power to take up all of your disk space. You can limit file size with the 3rd argument of parseFormData. The _unstable_parseMultipartFormData function defaults to have a max file size of 3MB. I don't believe parseFormData has a default max which means it will allow everything.

Here's how you would do it with parseFormData.

const formData = await parseFormData(request, uploadHandler, {
    maxFileSize: 3000000, // 3MB
});

Accessing other form data

Due to the nature of processing the uploads this way, you don't have access to the other form values because the FormData instance hasn't been created yet. For example, say you have another input in the same form for the user to select a category for the file that you need to do some validation on.

const formData = await parseFormData(request, uploadHandler)
const category = formData.get('category')

// Now I just need to valida....oh wait, my file has already been uploaded...

To work around this, you can use the useSubmit hook and append values as search params to be read in the action.

const submit = useSubmit()

const handleSubmit = (e: React.FormEvent<HTMLFormElement>) => {
    const formData = new FormData(e.currentTarget)
    const params = new URLSearchParams({category: formData.get('category')})
    const options = {
        action: `/app/upload?${params}`,
        method: "post",
        encType: "multipart/form-data"
    }
    submit(formData, options)
}

return (
    <Form
        method="post"
        action="/app/upload"
        encType="multipart/form-data"
        onSubmit={handleSubmit}
    >
        ...
    </Form>
)

And then in your action:

export async function action({request}: ActionFuntionArgs) {
    const url = new URL(request.url)
    const cateogry = url.searchParams.get('category')

    // continue on to use parseFormData()
}

Streaming files to the user

To complete the circle, here's how you can stream files to your users from a loader.

import invariant from "tiny-invariant";
import { fileStorage } from "~/your-file-storage.server";

export async function loader() {
  const storageKey = "some-file.png";
  const file = await fileStorage.get(storageKey);
  invariant(file, "invalid file");

  return new Response(file, {
    headers: {
      "Content-Type": file.type,
      "Content-Disposition": `attachment; filename=${storageKey}`,
    },
  });
}

As of writing this there is an official guide on file uploads in Remix but it's basically a placeholder until they iron out some APIs and have time to revisit the guide.

This is my unofficial guide.

<Form />

To use the Remix <Form /> component with files you must use the encType prop and update the method to "post" or "put":

<Form method='post' encType='multipart/form-data'>
    <input type='file' name='my-file' />
    <button>Submit</button>
</Form>

Now, in your route's action you can pull that File out of the form data.

export async function action({request}: ActionFuntionArgs) {
    const formData = await request.formData()
    const file = formData.get('my-file')
    if (!(file instanceof File)) throw new Error('Invalid file')

    // process file

    return {
        message: "File received"
    }
}

The uploaded file(s) will be stored in memory and you'll need to decide what to do with them.

Base64

You could convert the upload into a base64 value:

const formData = await request.formData()
const file = formData.get('my-file')
if (!(file instanceof File)) throw new Error('Invalid file')
const arrayBuffer = await file.arrayBuffer()
const buffer = Buffer.from(arrayBuffer)
const base64 = buffer.toString('base64')

But most likely you'll want to store them somewhere...

Local file system

Luckily, Remix has a handy utility for storing uploads locally. Here's what that looks like:

import {
    unstable_createFileUploadHandler,
    unstable_parseMultipartFormData
} from '@remix-run/node'

export async function action({request}: ActionFuntionArgs) {
    const uploadHandler = unstable_createFileUploadHandler({
        // where to save the file
        directory: '/app/storage',
        // use the incoming filename instead of creating a new name
        file: (args) => args.filename
    })
    const formData = await unstable_parseMultipartFormData(request, uploadHandler)

    return {
        message: "File received"
    }
}

unstable_parseMultipartFormData

You might have noticed that request.formData() went away and was replaced with this unstable_parseMultipartFormData function. This is a Remix utility for iteratively deciding where uploads go.

• request.formData() - read everything into memory and then decide what to do with file data
• unstable_parseMultipartFormData - decide what to do with file data as it's being read

This allows users to upload big files without the application storing those entirely in memory all at once. Instead, you can stream those files into their destinations.

The unstable_parseMultipartFormData function will iterate over the form data keys and if the key contains a file, it will hand that off to be processed by the upload handler.

Gotcha

One "gotcha" with parsing the form data this way is that you don't have access to the other form values because the FormData instance hasn't been created yet. For example, say you had another input in the same form for the user to select a category for the file that determines where that file lives on disk of if you need to do some validation on the provided category.

const formData = await unstable_parseMultipartFormData(request, uploadHandler)
const category = formData.get('category')

// Now I just need to valida....oh wait, my file has already been uploaded...

To get around this, you can use the useSubmit hook and append values as search params to be read in the action.

const submit = useSubmit()

return (
    <Form method="post" onSubmit={(e) => {
        const formData = new FormData(e.currentTarget)
        const params = new URLSearchParams({category: formData.get('category')})
        const options = {
            action: `/app/upload?${params}`,
            method: "post",
            encType: "multipart/form-data"
        }
        submit(formData, options)
    }}>
)

And then in your action:

export async function action({request}: ActionFuntionArgs) {
    const url = new URL(request.url)
    const cateogry = url.searchParams.get('category')

    // continue on to use unstable_parseMultipartFormData()
}

Custom upload handlers

Remix allows you to create your very own upload handlers.

const uploadHandler: UploadHandler = async ({ data, filename }) => {

    // handle the data and filename however you need to

    // return File | string | null | undefined
    return "https://example.com/assets/newly-uploaded-file.png";
}
const formData = await unstable_parseMultipartFormData(request, uploadHandler);

// Remix swaps in whatever you return from your upload handler
const uploadUrl = formData.get('my-file')

Composing upload handlers

Remix also comes with a utility for using multiple upload handlers to handle certain uploads differently. The following is lifted directly from the Remix docs.

  const uploadHandler = unstable_composeUploadHandlers(
    // custom upload handler
    async ({ name, contentType, data, filename }) => {
        if (name !== "img") {
            return undefined;
        }
        const uploadedImage = await uploadImageToCloudinary(data);
        return uploadedImage.secure_url;
    },
    // fallback to memory for everything else
    unstable_createMemoryUploadHandler()
);

const formData = await unstable_parseMultipartFormData(
    request,
    uploadHandler
);

In this example, the first handler only applies to the upload coming from the input with the name of "img":

<input type='file' name='img' />

By returning undefined, Remix will try the next upload handler. In this case, it's the unstable_createMemoryUploadHandler.

unstable_createMemoryUploadHandler

Wait, why is there a unstable_createMemoryUploadHandler? Wouldn't that be the same as request.formData()? Yep, almost. This utility does come with the ability to limit the upload size. It defaults to 3MB.

const uploadHandler = unstable_createMemoryUploadHandler({maxPartSize: 5000000}) // 5MB
const formData = await unstable_parseMultipartFormData(
    request,
    uploadHandler
);
const file = formData.get('my-file')

// up to you to handle now

Upload to S3

Here's an example that illustrates efficiently streaming the upload to S3.

import {
  type ActionFunctionArgs,
  json,
  unstable_parseMultipartFormData,
  type UploadHandler,
} from "@remix-run/node";
import { S3Client } from "@aws-sdk/client-s3";
import { env } from "~/env.server";
import { Upload } from "@aws-sdk/lib-storage";
import { Readable } from "stream";

// you probably wouldn't actually create the s3 client here...
const s3 = new S3Client({
  region: env.AWS_REGION,
  credentials: {
    accessKeyId: env.AWS_ACCESS_KEY_ID,
    secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
  },
});

export async function action({ request }: ActionFunctionArgs) {
  const uploadHandler: UploadHandler = async ({
    data,
    filename,
  }) => {
    const readableStream = Readable.from(data);
    const upload = new Upload({
      client: s3,
      params: {
        Bucket: env.AWS_S3_BUCKET,
        Key: filename,
        Body: readableStream,
      },
    });
    await upload.done();
    return filename;
  };

  await unstable_parseMultipartFormData(request, uploadHandler);

  return json({
    success: true,
  });
}

Hope this helps!

MSW and cypress/playwright are awesome testing tools. However, things can get tricky when you combine them. The crux of the problem is that your test code is not in the same process that your app code is. They are two discreet processes so you don't have direct access to the msw server instance.

With msw, you typically define the "happy path" responses in a mocks/handlers.ts file and this is used in your mock server. But...what if you want to test an error scenario?

You have a few options.

Dynamic mock scenarios

Basically split up the handlers.ts into a scenarios object and add a query parameter to "activate" a certain scenario. You can see this in the msw docs here.

There are two less-than-desirable traits to this approach though:

• They're separate from the test code
• You have to come up with your own arbitrary scenario naming convention

Use a development branch of msw

There is work being done to bridge the gap between msw server and e2e test code. It's just not ready for prime time yet. The setupRemoteServer API allows you to communicate to your server over a web socket connection. The idea is very promising and I'm excited for it to get merged! You can help by funding the maintainer's development efforts.

I'd love to ship the remote server API but, sadly, have no time for it right now. If your team would benefit from it, consider funding the effort! You can DM me for more details. Thanks.

— Artem Zakharchenko (@kettanaito) August 27, 2024

Development only app endpoint

I came up with a low-tech setupRemoteServer alternative to using network behavior overrides in a dedicated test endpoint. I'm using Remix so I simply created a development-only action route that I can make requests to that will interact with the msw server for me.

Now, I can set up a scenario directly from the e2e test.

The loader of my /app/contacts route makes an api call to fetch data from a http://localhost:6000/people api. The above code allows me to mock that request directly in my test code, keeping it all together.

I can further clean this up by abstracting this into a mock() function.

Hope this helps!