Amazon S3 is cloud storage from Amazon that provides unlimited storage capacity at relatively low prices. I am currently using it to keep a backup of my local NAS (network attached storage). However, Amazon S3 is not the best option for storing large amounts of data that you rarely access.
Amazon Glacier is a solution from Amazon that dramatically reduces the cost of storing large amounts of data in the cloud. For example, storing 2500 GB of data on S3 costs about $ 215 per month. That’s a lot of money just for backing up your data. However, storage of 2500 GB on Amazon Glacier will cost you only $ 25 per month. That’s almost 1/10 the cost of S3.
So how do you migrate data from Amazon S3 to Glacier? Using lifecycle policies. These policies are basically just rules that you can configure to move data from S3 to Glacier at specific times. Let’s find out how to create a lifecycle policy.
Create a Lifecycle Policy on Amazon S3
To get started, log into Amazon Web Services (aws.amazon.com) and click “My Account / Console” at the top. Then click AWS Management Console.
Now from the list of listed Amazon web services, click S3.
Then click on the name of the bucket containing the data you want to transfer to Glacier. Please note that you will be able to transfer the entire Trash, only folders, or even individual files.
When you open the bucket, you will see its contents on the left side. Click “Properties” in the upper right corner to open the settings for this segment.
Below you will see the Life Cycle. Go ahead and expand the Lifecycle to see your current rules, if any. I already have one setup that brings all the contents of the bucket to Glacier after it’s loaded into S3.
To configure a new rule, click Add Rule. The New Lifecycle Rule dialog box will appear as shown below.
Now let’s look at the different options. First, you can give it a name, which can be whatever you live for. The “Apply to entire recycle bin” checkbox will apply the rule to all files and folders inside the recycle bin. If you want to move only a certain portion of your data to Glacier and leave the rest in S3, leave this check box unchecked.
Instead, you can enter a prefix, which is the name of the file or folder that you want to move to Glacier. For example, if I wanted to just move the music folder in my bucket to Glacier, I would type music / in the Prefix field. To specify the file, enter a path such as music / mymusic.mp3.
Next comes the time period format. You can select Days from Creation Date or Effective Date. This option allows you to choose when you want the objects you specify to be moved into Glacier. If you select Days from Creation Date, you can say, for example, that you want to move data to Glacier in 10 days. This means that the first time you upload a data file to S3, it will be moved to Glacier 10 days after it was created.
The effective date allows you to simply specify a date in the future when data will be transferred to Glacier. You must click the Add Transition button to specify the time period. My screenshot says Go to the Glacier, but that’s because I’ve already created a rule. When you click Add Transition, you can enter a number of days or a date. Please note that if you enter 0 for the number of days, the data will be moved immediately the next time you run the rule.
There is also an expiration button, but be careful with that. Again, depending on what you chose from the time period format, you can specify a number of days or a specific date in the future. Adding an expiration date means that the data will be deleted after the time you specified. It is important to remember that it will be removed from S3, RRS and Glacier. This way, if you add an expiration date, it disappears completely.
If you don’t add an expiration date, the data will always remain in Glacier and will not be deleted. That’s all. After you save the rule, it will run once a day. If your rule matches the criteria you specified, the data will be migrated.
Several points should be noted about this process. First, you can tell that your data has been moved to Glacier by checking the storage class. If the standard, then this is S3. If it is an RRS, it is reduced redundancy. The third class is Glacier, which means it is currently stored there.
It should also be noted that when you move data from S3 to Glacier, you still need to access it from S3. If you upload data directly to Glacier, it will appear in the Glacier console when you log into AWS. However, moving data using lifecycle rules means that the data will be stored in Glacier and you will be charged for Glacier, but you will need to access the data from the S3 console. A bit confusing, but here’s how it works.
Retrieve data from Glacier
Getting data from Glacier is also pretty straightforward. Keep in mind about Glacier that data is not immediately available as in S3. With S3, you can download any file at any time. With Glacier, you need to wait 3 to 5 hours for the data to be fetched and put back into S3. This is why it is so much cheaper.
To start recovery, select the file or folder you want to recover and then right-click it. You will see the “Start Recovery” option.
If the option is disabled, it means that the file is not stored in Glacier. When restoring, you will need to choose how long the data will be available in S3.
Please note that files are restored to S3 RRS (Reduced Redundancy) storage class, which is slightly cheaper than S3 Standard. Also note that you cannot recover data irrevocably; over time, it will be deleted. I’m not sure what the largest value can be entered within a few days in order to save the data, but this is not permanent. In addition, the longer the data is stored in the RRS storage class, the higher the charges for it, so it is better to reduce the duration.
To see the status of the recovery, just click on the recovered file or folder and click Properties. “Recovery in Progress†will be written. When the restore is complete and you click Properties again, you will see the date until which the restore will be saved.
In general, migrating data from S3 to Glacier is very easy. Just create one rule and you’re done. Moving data to Glacier can mean big savings if you have a lot of data on S3. If you have any questions, please leave a comment. Enjoy!
–