Amazon Web Services Notes

What is Cloud Computing?

The Five Characteristics of Cloud Computing

·        On-demand Self Service

·        Broad Network Access

·        Multi – tenancy and resource pooling

·        Rapid elasticity and Scalability

·        Measure Service

 

Types of Cloud Computing

·        Infrastructure as a Service (Iaas)

·        Platform as a Service (PaaS)

·        Software as a Service (SaaS)

 

IAM: Identity and Access Management

·        IAM: Permissions

o   Users or Groups can be assigned JSON documents called Policies

o   These policies define the permissions of the users.

o   In AWS you apply the Least Privilege Principle

·        IAM: Password Policy

o   Multi Factor Authentication (MFA)

·        How Can users access AWS?

o   To access AWS, you have three options:

§  AWS Management Console

§  AWS Command Line Interface (CLI)

§  AWS Software Developer Kit (SDK)

o   Access keys are generated through the AWS Console

o   Users manage their own access keys

o   Access Key ID = name

o   Secret Access Key = password

·        IAM > Users > stephane > Security credentials > Access Keys > Create Access Key >

·        AWS CloudShell

·        IAM Roles

o   Identity and Access Management (IAM) > Roles > create rule > Trusted entity type: ü AWS service, Use case: ü EC2, Next > Add Permissions: Permissions Policies: ü IAMReadOnlyAccess, Next > Role name: DemoRoleForEC2, Create role

·        IAM Security Tools

o   IAM Credentials Report (account-level)
Identity and Access Management (IAM) > Credential report > Download Report (.csv file)

o   IAM Access Advisor (user-level)
Identity and Access Management (IAM) > Users > Last Accessed >

·        IAM Best Practices

o   Don’t use the root account except for AWS account setup

o   One physical user = One AWS user

o   Assign users to groups and assign permissions to groups

o   Create a strong password policy

o   Use and enforce the use of Multi Factor Authentication (MFA)

o   Create and use Roles for giving permissions to AWS services

o   Use Access Keys for Programmatic Access (CLI / SDK)

o   Audit permissions of your account using IAM Credentials Report & IAM Access Advisor

·        Shared Responsibility Model for IAM

·        IAM Section – Summary

o   Users: mapped to a physical user; has a password for AWS Console

o   Groups: contains users only

o   Policies: JSON document that outlines permissions for users or groups

o   Roles: for EC2 instances or AWS services

o   Security: MFA + Password Policy

o   AWS CLI: manage your AWS services using the command-line

o   AWS SDK: manage your AWS services using a Programming language.

o   Access Keys: access AWS using the CLI or SDK

o   Audit: IAM Credential Reports & IAM Access Advisor

EC2 Elastic Compute Cloud

·        AWS Budget Setup

o   Root user: Billing and Cost Management > Account > Billing and Payments > IAM user and role access to Billing information: ü Activate IAM Access > update

o   IAM user:  
Billing and Cost Management > Bills > Charges by service >
Billing and Cost Management > Free tier
Billing and Cost Management > Budgets > Create budget > Choose budget type: Budget setup: use a template (simplified), Templates – new: zero spend budget, Zero spend budget – Template: Budget name: My zero-spend budget, Email recilpients: mohammedimransharif@proton.me, create budget

·        EC2 Basics:

o   Amazon EC2

§  EC2 = Elastic Compute Cloud = Infrastructure as a Service

§  It mainly consists in the capability of:
Renting virtual machines (EC2)
Storing data on Virtual drives (EBS)
Distributing load across machines (ELB)
Scaling the services using an auto-scaling group (ASG)

o   EC2 sizing & configuration options:

§  Operating System (OS): Linux, Windows or Mac OS.

§  How much compute power & cores (CPU)

§  How much random-access memory (RAM)

§  How much storage space:
Network-attached (EBS & EFS)
hardware (EC2 Instance Store)

§  Network card: speed of the card, Public IP address

§  Firewall rules: security group

§  Bootstrap script (configure at first launch): EC2 User Data

o   EC2 User Data:

§  It is possible to bootstrap our instances using an EC2 User Data script.

§  Bootstrapping means launching commands when a machine starts.

§  That script is only run once at the instance first start.

§  EC2 user data is used to automate boot tasks such as:
     Installing updates
     Installing software
     Downloading common files from the internet
     Anything you can think of

§  The EC2 user data script runs with the root user

o   EC2 instance types: example:

Instance	vCPU	Mem (GiB)	Storage	Network Performance	EBS Bandwidth
t2.micro	1	1	EBS-Only	Low to Moderate
t2.xlarge	4	16	EBS-Only	Moderate
c5d.4xlarge	16	32	1 x 400 NVMe SSD	Up to 10 Gbps	4750
r5.16xlarge	64	512	EBS-Only	20 Gbps	13600
m5.8xlarge	32	128	EBS-Only	10 Gbps	6800

 

·        Create an EC2 Instance with EC2 User Data to have a Website Hands On

o   Hands-On: Launching an EC2 Instance running Linux

§  We’ll be launching our first virtual server using the AWS Console.

§  We’ll get a first high-level approach to the various parameters.

§  We’ll see that our web server is launched using EC2 user data

§  We’ll learn how to start / stop / terminate our instance.

o   EC2 > Instances > Launch an instance >

§  Name and tags: Name: My First Instance

§  Key pair (login): Create New key pair: Key pair name: EC2 tutorial, Create Key pair (file downloaded)

§  Network settings: ü Allow HTTP traffic from the internet

§  User data
#!/bin/bash
# use this for your user data (script from top to bottom)
# install httpd (Linux 2 version)
yum update -y
yum install -y httpd
systemctl start httpd
systemctl enable httpd
echo "<h1>Hello World from $(hostname -f)</h1>"> /var/www/html/index.html

§  Launch instance

o   EC2 > Instance > ü Select Instance > Details > Public IPv4 address > Open Address (make sure you are using http:// )

o   Public IP will change after restart but Private IP remains the same.

·        EC2 Instance Types Basics

o   EC2 Instance Types – Overview:
You can use different types of EC2 instances that are optimised for different use cases (https://aws.amazon.com/ec2/instance-types/)

§  General Purpose

·        Great for a diversity of workloads such as web servers or code repositories.

·        Balance between: Compute, Memory, Networking

·        In the course, we will be using the t2.micro which is a General purpose EC2 instance

§  Compute Optimized

·        Great for compute-intensive tasks that require high performance processors:

o   Batch processing workloads

o   Media transcoding

o   High performance web servers

o   High performance computing (HPC)

o   Scientific modelling & machine learning

o   Dedicated gaming servers

·        Ex: C8g

§  Memory Optimized

·        Fast performance for workloads that process large data sets in memory.

·        Use cases:

o   High performance, relational/non-relational databases

o   Distributed web scale cache stores

o   In-memory databases optimized for BI (business intelligence)

o   Application performing real-time processing of big structured data

·        Ex: R7g, R7i, R7iz …

§  Accelerated Computing

§  Storage Optimized

·        Great for storage-intensive tasks that require high, sequential read and write access to large data sets on local storage.

·        Use cases:

o   High frequency online transaction processing (OLTP) systems

o   Relational & NoSQL databases

o   Cache for in-memory databases (ex: Redis)

o   Data warehousing applications

o   Distributed file systems

·        Ex: i8g

§  Instance Features

§  Measuring Instance Performance

o   m: instance class
5: generation
2xlarge: size within the instance class

o   https://www.ec2instances.info/

·        Security Groups Classic Ports Overview

o   Introduction to Security Groups

§  Security groups are the fundamental of network security in AWS

§  They control how traffic is allowed into or out of our EC2 instances.

§  Security groups only contain allow rules

§  Security groups rules can reference by IP or by security group.

o   Security Groups Deeper Dive

§  Security groups are acting as a “firewall” on EC2 instances

§  They regulate:

·        Access to Ports

·        Authorised IP ranges – IPV4 and IPV6

·        Control of inbound network (from other to the instance)

·        Control of outbound network (from the instance to other)

o   Security Groups Good to know

§  Can be attached to multiple instances

§  Locked down to a region / VPC combination

§  Does live “outside” the EC2 – if traffic is blocked the EC2 instance won’t see it.

§  It’s good to maintain one separate security group for SSH access

§  If your application is not accessible (time out), then it’s a security group issue.

§  If your application gives a “connection refused” error, then it’s an application error or it’s not launched

§  All inbound traffic is blocked by default.

§  All outbound traffic is authorised by default.

o   Classic Ports to know

§  22 = SSH (Secure Shell) – log into a Linux instance

§  21 = FTP (File Transfer Protocol) – upload files into a file share

§  22 = SFTP (Secure File Transfer Protocol) – upload files using SSH

§  80 = HTTP – access unsecured websites

§  443 = HTTPS – access secured websites

§  3389 = RDP (Remote Desktop Protocol) – log into a Windows instance

·        Security Groups Hands On

o   EC2 > Network & Security > Security Groups > ü select security group > Inbound rules, Outbound rules (delete, add.. etc)

·        SSH Overview

o    

·        How to SSH using Linux or Mac

o  

o   Make sure the .pem file name has no spaces in it. If there are space in the name rename it.

o   Make sure you are accessing ssh from the same location as the file location.

o   Copy Public IP of the EC2 Instance

o   In case of “WARNING: UNPROTECTED PRIVATE KEY FILE!”
use chmod 0400 Ec2tuto.pem

o   ssh -i  Ec2tuto.pem ec2-user@13.233.133.250

·        How to SSH using Windows

o   Download putty: 64-bit x86: putty-64bit-0.82-installer.msi (signature)

o   Install putty

o   Open PuTTYgen (putty use ppk format)

§  Incase you have downloaded .pem file then

§  PuTTYgen > Load > Browse .pem file > save as private key > save somewhere

o   Open PuTTY

§  Session >

·        Hostname: ec2-user@13.233.133.250

§  Connection > SSH > Auth > Credentials >

·        Private-key: Ec2tuto.ppk (as u save after convertion)

·        Open

o   connected

·        How to SSH using Windows 10

o   Use .pem file to log in.

o   Use power shell or command promt.

·        EC2 Instance Connect

o   Can’t connect to instance console. FAILED

o   EC2 > Instance > i-0c8973e019f3fd350 > Connect to instance

o   Even though you setup your inbound settings, can’t able to connect. It need for an upgrade, which gonna cost money. 螺

·        EC2 Instance Roles Demo

·        EC2 Instance Purchasing Options

o   On-Demand Instances – short workload, predictable pricing, pay by second

o   Reserved (1 & 3 years)

§  Reserved Instances – Long workloads

§  Convertible Reserved instances – long workloads with flexible instaces

o   Savings Plans (1 & 3 years) – commitment to an amount of usage, long workload

o   Spot Instances – short workloads, cheap, can lose instances (less reliable).

o   Dedicated Hosts – book an entire physical server, control instance placement.

o   Dedicated Instances – no other customers will share your hardware

o   EC2 on Demand

§  Pay for what you use:

·        Linux or Windows – billing per second, after the first minute

·        All other operating systems – billing per hour.

§  Has the highest cost but no upfront payment

§  No long-term commitment.

§  Recommended for short-term and un-interrupted workloads, where you can’t predict how the application will behave.

o   EC2 Reserved Instances

§  Up to 72% discount compared to On-demand

§  You reserve a specific instance attribute (instance type, Region, Tenancy, OS)

§  Reservation period – 1 year (+ discount) or 3 years (+++discount)

§  Payment Options – No Upfront (+), Partial Upfront (++), All Upfront (+++)

§  Reserved Instance’s Scope – Regional or Zonal (reserve capacity in an AZ)

§  Recommended for steady-state usage applications (think database)

§  You can buy and sell in the Reserved Instance Marketplace.

§  Convertible Reserved Instance

·        Can change the EC2 instance type, instance family, OS, scope and tenancy.

·        Up to 66% discount

o   EC2 Savings Plans

§  Get a discount based on long-term usage (up to 72% - same as Rls)

§  Commit to a certain type of usage ($10/hour for 1 or 3 years)

§  Usage beyond EC2 Savings Plans is billed at the On-Demand price.

§  Locked to a specific instance family & AWS region (e.g. M5 in us-east-1)

§  Flexible across:

·        Instance Size (eg: m5.xlarge, m5.2xlarge)

·        OS (eg: Linux, Windows)

·        Tenancy (Host, Dedicated, Default)

o   EC2 Spot Instances

§  Can get a discount of up to 90% compared to On-demand.

§  Instances that you can “lose” at any point of time if your max price is less than the current spot price.

§  The MOST cost-efficient instances in AWS

§  Useful for workloads that are resilient to failure.

·        Batch jobs

·        Data analysis

·        Image processing

·        Any distributed workloads

·        Workloads with a flexible start and end time.

§  Not suitable for Crital jobs or databases

o   EC2 Dedicated Hosts

§  A physical server with EC2 instance capacity fully dedicated to your use

§  Allows you address compliance requirements and use your existing serverbound software licenses (per-socket, per-core, pe-VM software licenses)

§  Purchasing Options:

·        On-demand – pay per second for active Dedicated Host.

·        Reserved – 1 or 3 years (No Upfront, Partial Upfront, All Upfront)

§  The most expensive option.

§  Useful for software that have complicated licensing model (BYOL – Bring Your Own License)

§  Or for companies that have strong regulatory or compliance needs

o   EC2 Dedicated Instances

§  Instances run on hardware that’s dedicated to you.

§  May share hardware with other instances in same account

§  No control over instance placement (can move hardware after Stop/Start)

o   EC2 Capacity Reservations

§  Reserve On-Demand instances capacity in a specific AZ for any duration.

§  You always have access to EC2 capacity when you need it.

§  No time commitment (create/cancel anytime) no billing discounts.

§  Combine with regional reserved Instances and Saving Plans to benefit from billing discounts.

§  You’re charged at On-Demand rate whether you run instances or not.

§  Suitable for short-term, uninterrupted workloads that needs to be in a specific AZ.

o   Which purchasing option is right for me?

§  On demand: coming and staying in resort whenever we like, we pay the full price.

§  Reserved: Like planning ahead and if we plan to stay for a long time, we may get a good discount.

§  Savings Plans: pay a certain amount per hour for certain period and stay in any room type (e.g: King, Suite, sea view,..)

§  Spot instances: the hotel allows people to bid for the empty rooms and the highest bidder keeps the rooms. You can get kicked out at any time.

§  Dedicated Hosts: We book an entire building of the resort.

§  Capacity Reservations: you book a room for a period with full price even you don’t stay in it.

·        Shared Responsibility Model for EC2

o   Infrastructure (global network security)

o   Isolation on physical hosts

o   Replacing faulty hardware

o   Compliance validation

o   Security Groups rules

o   Operating-system patches and updates

o   Software and utilities installed on the EC2 instance

o   IAM Roles assigned to EC2 & IAM user access management.

o   Data security on your instance

·        EC2 Summary

o   EC2 Section – Summary

§  EC2 Instance: AMI (OS) + Instance Size (CPU + RAM) + Storage + security groups + EC2 User Data.

§  Security Groups: Firewall attached to the EC2 instance

§  EC2 User Data: Script launched at the first start of an instance.

§  SSH: start a terminal into our EC2 Instance (port 22)

§  EC2 Instance Role: Link to IAM roles.

§  Purchasing Options: On-Demand, Spot, Reserved (Standard + Convertible + Scheduled), Dedicated Host, Dedicated Instance

6 - EC2 Instance Storage

·        EBS Overview

o   What’s an EBS Volume?

§  An EBS (Elastic Block Store) Volume is a network drive you can attach to your instances while they run.

§  It allows your instances to persist data, even after their termination.

§  They can only be mounted to one instance at a time (at the CCP level)

§  They are bound to a specific availability zone.

 

§  Analogy: Think of them as a “network USB stick”

§  Free tier: 30 GB of free EBS storage of type General Purpose (SSD) or Magnetic per month

o   EBS Volume:

§  It’s a network drive (i.e. not a physical drive)

·        It uses the network to communicate the instance, which means there might be a bit of latency.

·        It can be detached from an EC2 instance and attached to another one quickly.

§  It’s locked to an Availability Zone (AZ)

·        An EBS Volume in us-east-I a cannot be attached to us-east-I b.

·        To move a volume across, you first need to snapshot it.

§  Have a provisioned capacity (size in GBs, and IOPS)

·        You get billed for all the provisioned capacity.

·        You can increase the capacity of the drive over time.

o   EBS – Delete on Termination attribute

§  Controls the EBS behaviour when an EC2 instance terminates.

·        By default, the root EBS volume is deleted (attribute enabled)

·        By default, any other attached EBS volume is not deleted (attribute disabled)

§  This can be controlled by the AWS console / AWS CLI

§  Use case: preserve root volume when instance is terminated.

§   

·        EBS Hands On

EC2 > Volumes > Create volume > Volume settings > size: 2, Availability zone: ap-south-1b (same location as instance location), Create Volume select created volume > Action > Attact volume > instance, Device name > Attach Volume

·        EBS snapshot overview

EBS snapshots - Make a backup (snapshot) of your EBS volume at a point in time - Not necessary to detach volume to do snapshot, but recommended - Can copy snapshots across AZ or Region EBS Snapshots features - EBS Snapshot Archive -- Move a Snapshot to an "archive tier" that is 75% cheaper -- Takes within 24 to 72 hours for restoring the archive - Recycle Bin for EBS snapshots -- Setup rules to retain deleted snapshots so you can recover them after an accidental deletion. -- Specify retention (from 1 day to 1 year)

·        EBS snapshots Hands On

·        AMI Overview

·        AMI Hands on

·        EC2 Image Builder Overview

·        EC2 Instance Store

·        EFS Overview

·        Shared Responsibility Model for EC2 Storage

·        Amazon FSx Overview

·        EC2 Instance Storage Summary

·        Section Cleanup

7 - ELB ASG Elastic Load Balancing Auto Scaling Groups

63 - High Availability Scalability Elasticity

Scalability & Hight Availability

64 - Elastic Load Balancing ELB Overview

65 - Application Load Balancer ALB Hands On

66 - Auto Scaling Groups ASG Overview

67 - Auto Scaling Groups ASG Hands On

68 - Auto Scaling Groups ASG Strategies

69 - Section Cleanup

70 - ELB ASG Summary

8 - Amazon S3

9 - Databases Analytics

10 - Other Compute Services ECS Lambda Batch Lightsail

11 - Deployments Managing Infrastructure at Scale

12 - Leveraging the AWS Global Infrastructure

13 - Cloud Integrations

14 - Cloud Monitoring

15 - VPC Networking

16 - Security Compliance

17 - Machine Learning

18 - Account Management Billing Support

19 - Advanced Identity

20 - Other Services

21 - AWS Architecting Ecosystem

22 - Preparing for the Exam Practice Exam AWS Certified Cloud Practitioner

23 - Congratulations AWS Certified Cloud Practitioner

 

https://796973492212.signin.aws.amazon.com/console

User: 796973492212

Name: sharif

HTTP status code: [400]

Context: [User: arn:aws:iam::796973492212:user/sharif is not authorized to perform: ce:GetCostAndUsage on resource: arn:aws:ce:us-east-1:796973492212:/GetCostAndUsage because no identity-based policy allows the ce:GetCostAndUsage action]

Request ID: [71126d48-aed2-44d4-9f55-a1c39f75e2ff]

 

Note:

Ø  Amazon Simple Storage Service (Amazon S3) is a highly scalable, secure, and cost-effective object storage service provided by AWS.

### Key Features:
- Scalability: Amazon S3 can store virtually unlimited amounts of data, automatically scaling up or down as needed.
- Durability and Availability: It offers 99.999999999% (11 nines) data durability and 99.99% availability.
- Security: Data stored in Amazon S3 is secure by default, with features like encryption, access control, and compliance capabilities.
- Cost-Effective: You only pay for what you use, with various storage classes to optimize costs based on data access frequency.
- Performance: Amazon S3 provides high performance for any workload, with low latency and high throughput.

### Use Cases:
- Data Lakes: Centralized repositories for storing structured and unstructured data for analytics, AI, and machine learning.
- Backup and Restore: Reliable storage for backup and disaster recovery solutions.
- Archiving: Cost-effective storage for long-term data retention.
- Web and Mobile Applications: Hosting static websites and serving content for mobile apps.
- Big Data Analytics: Storing and analyzing large datasets for business intelligence and insights.

 

Ø  Amazon Route 53 is a highly available and scalable Domain Name System (DNS) web service. You can use Route 53 to perform three main functions in any combination: domain registration, DNS routing, and health checking.
If you choose to use Route 53 for all three functions:

1.      Register domain names
Your website needs a name, such as example.com. Route 53 lets you register a name for your website or web application, known as a domain name.

2.      Route internet traffic to the resources for your domain
When a user opens a web browser and enters your domain name (example.com) or subdomain name (acme.example.com) in the address bar, Route 53 helps connect the browser with your website or web application.

3.      Check the health of your resources
Route 53 sends automated requests over the internet to a resource, such as a web server, to verify that it's reachable, available, and functional. You also can choose to receive notifications when a resource becomes unavailable and choose to route internet traffic away from unhealthy resources.

 

Ø  AWS EC2, S3, RDS, S3 vs EBS, IAM, AWS Cloud Watch.

Ø  Which of the following storage solutions would better fit for distributed application spanning a large number of EC2 instance & perml

Ø  EBS, EFS, Instance store, AMI, Hypervisor Image, Snapshot, Ghost image.

Ø