How many GB do you need? How fast is it growing? How much data transfer do you need? These are the crucial questions.
It might be worth asking:
- People using S3, how much storage did you need when you first chose to do that?
I think it will come down to cost and flexibility - I wouldn’t expect any performance problem. Using local storage on the instance will, I think, be more expensive and only comes in certain fixed sizes: you’ll always have some unused space which you’re paying for. But you can’t predict the future so you can’t really model the costs either way.
For cases with modest storage needs it’s surely fine to use local storage, and it’s certainly simpler. Note that, as far as I understand it, migrating from local to block storage is straightforward, but migrating back from block storage to local storage is not. See MJK’s excellent opinionated guide:
Note that there are various storage providers each with their own pricing. Cloudflare also has an offering (without egress fees) but it’s not quite ready:
Configure an S3 compatible object storage provider for uploads
And of course, the various hosting companies will be competing on price, so shop around even if using local storage.