I just recently wrote a guide on Remix file uploads and asked the Remix folks for some feedback. Turns out Michael, from Remix, has been cooking up some new apis for this very thing that he encouraged me to use.
Here we go again 😅.
<Form />
To use the Remix <Form /> component with files you must use the encType prop and update the method to "post" or "put":
<Form method='post' encType='multipart/form-data'>
<input type='file' name='my-file' />
<button>Submit</button>
</Form>
Next, we need to receive the form data in a route action. We could just use a native function:
export async function action ({request}: ActionFunctionArgs) {
const formData = await request.formData()
const file = formData.get('my-file')
// process the file
}
However, this loads the entire uploaded file contents into server memory. A much more efficient way to handle files would be to stream the uploads to their destination. This way our server never holds
the entire contents of large files in memory at one time.
This is exactly what the @mjackson/form-data-parser package is for. It creates intermediary FileUpload instances that extend the File class that can be streamed to their destination. Clever stuff.
Here's what it looks like.
import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'
export async function action ({request}: ActionFunctionArgs) {
const uploadHandler = async (fileUpload: FileUpload) => {
if (fileUpload.fieldName === 'my-file') {
// process the upload and return a File
}
// ignore any unrecognized files
}
const formData = await parseFormData(request, uploadHandler)
// 'my-file' has already been processed at this point
const file = formData.get('my-file')
}
Adapters
The @mjackson/form-data-parser package is designed in a way that allows for implementing various storage destinations. It uses another package that defines a FileStorage interface. Your storage destination could be anything, you just need to create a class that implements this interface and you're good to go. However, you'll be dealing with streams which can be tricky. I found this helpful to understand the basics.
Luckily, this package comes with two basic adapters out of the box: MemoryFileStorage and LocalFileStorage.
Local file storage
Want to store uploads directly on your server? That's what LocalFileStorage is for.
import { LocalFileStorage } from '@mjackson/file-storage/local';
import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'
const fileStorage = new LocalFileStorage('/uploads/user-avatars');
export async function action ({request}: ActionFunctionArgs) {
// however you authenticate users
const user = await getUser(request)
const uploadHandler = async (fileUpload: FileUpload) => {
if (fileUpload.fieldName === 'my-file') {
let storageKey = `user-${user.id}-avatar`;
await fileStorage.set(storageKey, fileUpload);
return fileStorage.get(storageKey);
}
// Ignore any files we don't recognize the name of...
}
const formData = await parseFormData(request, uploadHandler)
const file = formData.get('my-file')
// file has been processed
}
Memory File Storage
This stores the uploads in an in-memory Map but it doesn't work the way I expect it to. I'm going to hold off on my description until I get some clarification.
```typescript
import { MemoryFileStorage } from '@mjackson/file-storage/memory';
import { type FileUpload, parseFormData } from '@mjackson/form-data-parser'
const fileStorage = new MemoryFileStorage();
export async function action ({request}: ActionFunctionArgs) {
const user = await getUser(request)
const uploadHandler = async (fileUpload: FileUpload) => {
if (fileUpload.fieldName === 'my-file') {
let storageKey = `user-${user.id}-avatar`;
await fileStorage.set(storageKey, fileUpload);
return fileStorage.get(storageKey);
}
// Ignore any files we don't recognize the name of...
}
const formData = await parseFormData(request, uploadHandler)
const file = formData.get('my-file')
if (!(file instanceof File)) throw new Error('Invalid file')
// the entire content of the file is not currently stored in memory
const arrayBuffer = await file.arrayBuffer()
// the entire content of the file is now stored in memory at this point
// because we actually read it
const buffer = Buffer.from(arrayBuffer)
const base64 = buffer.toString('base64')
return json({
base64
})
}
```
This stores the upload _references_ in-memory and the file contents only take up memory when they're actually read from; `file.arrayBuffer()` in this case.
If you omit a `uploadHandler`, the parser behaves just like `request.formData()`. The contents of the files are immediately stored in memory.
```typescript
const formData = await parseFormData(request)
// the entire contents of the file is immediately stored in memory
const file = formData.get('my-file')
```
S3
I have opened a pull request to merge in an S3 adapter. I don't know if/when that will be merged but you can grab the implementation here.
const fileStorage = new S3FileStorage(s3Client, 'bucket-name')
R2
Sergio Xalambrí just published a package that implements the FileStorage interface for R2. Check it out here. Be aware that it does not take advantage of streaming files as far as I can tell. It will load them into memory and then push them to R2.
SFTP
After lots of trial and error I was also able to come up with an SFTP implementation if you want to upload and read files to and from a remote server. You can check it out here.
const fileStorage = new SftpFileStorage(
{
host: "1.23.456.789",
username: "some-user",
privateKey: fs.readFileSync("/home/some-user/.ssh/id_rsa"),
},
"/somewhere/on/your/server",
);
Gotchas
There are a couple of gotchas you need to be aware of.
Limiting file size
You definitely don't want to give an individual the power to take up all of your disk space. You can limit file size with the 3rd argument of parseFormData. The _unstable_parseMultipartFormData function defaults to have a max file size of 3MB. I don't believe parseFormData has a default max which means it will allow everything.
Here's how you would do it with parseFormData.
const formData = await parseFormData(request, uploadHandler, {
maxFileSize: 3000000, // 3MB
});
Due to the nature of processing the uploads this way, you don't have access to the other form values because the FormData instance hasn't been created yet. For example, say you have another input in the same form for the user to select a category for the file that you need to do some validation on.
const formData = await parseFormData(request, uploadHandler)
const category = formData.get('category')
// Now I just need to valida....oh wait, my file has already been uploaded...
To work around this, you can use the useSubmit hook and append values as search params to be read in the action.
const submit = useSubmit()
const handleSubmit = (e: React.FormEvent<HTMLFormElement>) => {
const formData = new FormData(e.currentTarget)
const params = new URLSearchParams({category: formData.get('category')})
const options = {
action: `/app/upload?${params}`,
method: "post",
encType: "multipart/form-data"
}
submit(formData, options)
}
return (
<Form
method="post"
action="/app/upload"
encType="multipart/form-data"
onSubmit={handleSubmit}
>
...
</Form>
)
And then in your action:
export async function action({request}: ActionFuntionArgs) {
const url = new URL(request.url)
const cateogry = url.searchParams.get('category')
// continue on to use parseFormData()
}
Streaming files to the user
To complete the circle, here's how you can stream files to your users from a loader.
import invariant from "tiny-invariant";
import { fileStorage } from "~/your-file-storage.server";
export async function loader() {
const storageKey = "some-file.png";
const file = await fileStorage.get(storageKey);
invariant(file, "invalid file");
return new Response(file, {
headers: {
"Content-Type": file.type,
"Content-Disposition": `attachment; filename=${storageKey}`,
},
});
}
If you found this helpful or find something that needs a correction, reach out to me! -> dadamssg
