Young Implementing Cloud Design Patterns for AWS

Create highly efficient design patterns for scalability, redundancy, and high availability in the AWS Cloud
1. Auflage 2025
ISBN: 978-1-78217-735-7
Verlag: De Gruyter
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 228 Seiten

This book is aimed at architects, solution providers, and those of the DevOps community who are looking to implement repeatable patterns for deploying and maintaining services in the Amazon cloud infrastructure. Prior experience using AWS is required as the book focuses more on the patterns and not on the basics of using AWS.

Autoren/Hrsg.

Young, Marcus

Fachgebiete

• Mathematik | Informatik EDV | Informatik EDV & Informatik Allgemein

Leseproben

On-demand disk pattern

The on-demand disk pattern is similar in nature to the scale up pattern, in that it operates on an already running instance and requires some downtime. This pattern is a manual process and might be avoided in automated processes such as the scale out pattern, as the downtime involved might cause alarm or scaling policies to trigger, causing some unwanted side effects.

The benefit of on-demand disk size is that you do not need to plan ahead for disk resources. Once the instance is running, you can simply resize its volume when it gets to its maximum capacity. Another example would be in the event where your application is very I/O heavy. At first your instance performs very well, but over time, with increased usage, the I/O operations take longer. An easy way to get more performance out of the base volume might be to enable RAID and stripe the volume.

First we will launch the AWS Linux AMI with an additional 20 GB EBS volume. Ensure that the Volume Type is and it is set to Delete on Termination as shown in the following screenshot:

Once it is running, we will need to format and mount the drive for usage. We will mount it to :

[ec2-user@ip-10-203-10-237 ~]$ sudo su - Last login: Sun Nov 9 13:32:35 UTC 2014 on pts/0 [root@ip-10-203-10-99 ~]# lsblk # view block devices attached NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk +-xvda1 202:1 0 8G 0 part / xvdb 202:16 0 20G 0 disk [root@ip-10-203-10-237 ~]# mkfs -t ext4 /dev/xvdb #format the secondary drive as ext4 filesystem mke2fs 1.42.9 (28-Dec-2013) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 1310720 inodes, 5242880 blocks 262144 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=2153775104 160 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done [root@ip-10-203-10-237 ~]# echo $'/dev/xvdb\t/opt/data\text4\tdefaults,nofail\t0\t2' >> /etc/fstab [root@ip-10-203-10-237 ~]# mkdir /opt/data && mount /opt/data

Now that we have the EBS volume mounted to , we will write a file to the disk to ensure that a resize retains the data:

[root@ip-10-203-10-99 ~]# time dd if=/dev/zero of=/opt/data/test bs=512k count=200 200+0 records in 200+0 records out 104857600 bytes (105 MB) copied, 0.0667198 s, 1.6 GB/s real 0m0.068s user 0m0.000s sys 0m0.064s [root@ip-10-203-10-99 ~]# time dd if=/dev/zero of=/opt/data/test bs=512k count=200 200+0 records in 200+0 records out 104857600 bytes (105 MB) copied, 2.63703 s, 39.8 MB/s real 0m2.648s user 0m0.000s sys 0m0.076s

Volume resize

First we will unmount the volume from the running instance with the command:

[root@ip-10-203-10-99 ~]# umount /opt/data

After the volume is unmounted, select the instance from the console, and select the 20 GiB drive attached as . This will take you to the corresponding volume in the Volumes section of the console. Click the Actions drop-down, and select Create Snapshot, as described in the snapshot pattern. Once this snapshot is 100 percent available for use, detach the volume from the instance as shown in the following screenshot:

To increase the size of the volume, select the snapshot, and select Create Volume from the Actions menu. Here, select a larger size such as 30 GB, and click Create. When the new volume is available, click Attach Volume from the Actions menu, and select the instance that we detached, ensuring you attach it back to the device we detached it from, or in this case . Back inside the virtual machine, you will remount the device, check it for errors, resize the partition, check the new size, and ensure that we didn't lose any data:

[root@ip-10-203-10-99 ~]# lsblk # view block devices attached NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk +-xvda1 202:1 0 8G 0 part / xvdb 202:16 0 20G 0 disk [root@ip-10-203-10-99 ~]# e2fsck -f /dev/xvdb e2fsck 1.42.9 (28-Dec-2013) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/xvdb: 13/1310720 files (7.7% non-contiguous), 215923/5242880 blocks [root@ip-10-203-10-99 ~]# resize2fs /dev/xvdb resize2fs 1.42.9 (28-Dec-2013) Resizing the filesystem on /dev/xvdb to 7864320 (4k) blocks. The filesystem on /dev/xvdb is now 7864320 blocks long. [root@ip-10-203-10-99 ~]# mount -a [root@ip-10-203-10-99 ~]# df -mh /opt/data Filesystem Size Used Avail Use% Mounted on /dev/xvdb 30G 394M 28G 2% /opt/data [root@ip-10-203-10-99 ~]# ls -alh /opt/data total 351M drwxr-xr-x 3 root root 4.0K Nov 9 21:06 . drwxr-xr-x 4 root root 4.0K Nov 9 21:03 .. -rw-r--r-- 1 root root 250M Nov 9 21:05 test

Change volume from magnetic to SSD

One method to get some increased I/O performance is to change the volume from magnetic to a general purpose SSD, which has a throughput of about 128 MB/s. To do this, select the snapshot that we created for the volume, and click Create Volume from the Actions drop-down menu. Once this volume is available, attach it as to the instance. From the running instance we will mount it back, ensure that no loss of data has occurred, and check for an increase in throughput:

[root@ip-10-203-10-99 ~]# mount -a [root@ip-10-203-10-99 ~]# ls /opt/data lost+found test [root@ip-10-203-10-99 ~]# time dd if=/dev/zero of=/opt/data/test1 bs=512k count=200 200+0 records in 200+0 records out 104857600 bytes (105 MB) copied, 1.5918 s, 65.9 MB/s real 0m1.600s user 0m0.008s sys 0m0.068s

Here, we can see that we nearly doubled our throughput.

Increase I/O through software RAID

Tip

If you wish to read more about the different software RAID levels and how they perform in the AWS virtualized environment, more information can be found at their User Guide at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/raid-config.html.

As seen, the throughput for the new file was about 1.6 GB/s for creation and about 40 MB/s for a rewrite. Magnetic EBS volumes are estimated between 40 and 90 MB/s. This is a very rough...