So a while ago I wanted to migrate from using the local disk of a server to using Amazon S3 for a site running Rails 6 with Active Storage. I couldn't find any info on how to do so in the docs and the mapping from how the file storage worked and how the cloud storage works isn't a 1-to-1 solution.
I stumbled across a good (but a little outdated) blog post from Stefan Wienert after some extensive Googling. So I used his solution and fixed it to work in a newer setup with Rails 6 - see below
To use it yourself you can just run rake storage:migrate
on the production server after you've setup your credentials and also activated the amazon storage alternative in storage.yml
. Once you have executed the migration script you can then switch to the Amazon storage alternative in environments/production.rb
# frozen_string_literal: true
module ActiveStorage
class Downloader
def initialize(blob, tempdir: nil)
@blob = blob
@tempdir = tempdir
end
def download_blob_to_tempfile
open_tempfile do |file|
download_blob_to file
verify_integrity_of file
yield file
end
end
private
attr_reader :blob, :tempdir
def open_tempfile
file = Tempfile.open(["ActiveStorage-#{blob.id}-", blob.filename.extension_with_delimiter], tempdir)
begin
yield file
ensure
file.close!
end
end
def download_blob_to(file)
file.binmode
blob.download { |chunk| file.write(chunk) }
file.flush
file.rewind
end
def verify_integrity_of(file)
raise ActiveStorage::IntegrityError unless Digest::MD5.file(file).base64digest == blob.checksum
end
end
end
module AsDownloadPatch
def open(tempdir: nil, &block)
ActiveStorage::Downloader.new(self, tempdir: tempdir).download_blob_to_tempfile(&block)
end
end
Rails.application.config.to_prepare do
ActiveStorage::Blob.send(:include, AsDownloadPatch)
end
def migrate(from, to)
configs = Rails.configuration.active_storage.service_configurations
from_service = ActiveStorage::Service.configure from, configs
to_service = ActiveStorage::Service.configure to, configs
ActiveStorage::Blob.service = from_service
puts "#{ActiveStorage::Blob.count} Blobs to go..."
ActiveStorage::Blob.find_each do |blob|
print '.'
file = Tempfile.new("file#{Time.now}")
file.binmode
file << blob.download
file.rewind
checksum = blob.checksum
to_service.upload(blob.key, file, checksum: checksum)
rescue Errno::ENOENT
puts "Rescued by Errno::ENOENT statement. ID: #{blob.id} / Key: #{blob.key}"
next
rescue ActiveStorage::FileNotFoundError
puts "Rescued by FileNotFoundError. ID: #{blob.id} / Key: #{blob.key}"
next
end
end
namespace :storage do
desc 'Migrate ActiveStorage files from local to Amazon S3'
task migrate: :environment do
migrate(:local, :amazon)
end
end