Hmmm it does seem to be working… though I ran into a serious problem this morning… suddenly my disk usage spiked 25% and filled the drive, crashing the forums completely.
Right now, when I’m doing the rake, it seems like it’s downloading/uploading the images right during the batch run. I enter the rake command, it processes 300 images exactly (I’m running them in batches of 300) and then it ends.
So the important question is: could just moving those download from S3 to the local disk be queued? Could that have built up any kind of batch that then happened at 5am and brought down my forums? 