Sruthi Palle's blog
Real Time Production grade AWS VPC project

Real Time Production grade AWS VPC project

Sruthi Palle's photo
Sruthi Palle
·

15 min read

Table of contents

This is the most recommended approach by AWS for secure deployment of applications in EC2 instance

About the Project

This example demonstrates how to create a VPC that you can use for servers in a production environment. To improve resiliency, you deploy the servers in two Availability Zones, by using an Auto Scaling group and an Application Load Balancer. For additional security, you deploy the servers in private subnets. The servers receive requests through the load balancer. The servers can connect to the internet by using a NAT gateway. To improve resiliency, you deploy the NAT gateway in both Availability Zones.

Overview

The following diagram provides an overview of the resources included in this example.

  1. The VPC has public subnets and private subnets in two Availability Zones.

  2. Each public subnet contains a NAT gateway and a load balancer node.

  3. The servers run in the private subnets, are launched and terminated by using an Auto Scaling group, and receive traffic from the load balancer.

  4. The servers can connect to the internet by using the NAT gateway.

  5. The servers can connect to Amazon S3 by using a gateway VPC endpoint.

A VPC with subnets in two Availability Zones.

While creating VPC, you see below options, you have to choose either of one

1) Option VPC (Basic VPC):

  • Creates a minimal VPC setup with only the essential components:

    • A single VPC with a default CIDR block.

    • No additional subnets, route tables, or gateways are created.

    • Suitable if you want a clean slate to configure all components manually.

2) Option VPC and More (With additional resources):

  • Creates a VPC with several predefined components:

    • Default subnets in multiple Availability Zones.

    • An internet gateway attached to the VPC.

    • Route tables with routes for internet access.

    • Network ACLs and security groups set up with default rules.

    • This option is ideal for users who want a VPC ready for typical workloads with minimal setup effort.

Create the VPC

Use the following procedure to create a VPC with a public subnet and a private subnet in two Availability Zones, and a NAT gateway in each Availability Zone.

To create the VPC

  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. On the dashboard, choose Create VPC.

  3. For Resources to create, choose VPC and more.

  4. Configure the VPC

    1. For Name tag auto-generation, enter a name for the VPC.

    2. For IPv4 CIDR block, you can keep the default suggestion, or alternatively you can enter the CIDR block required by your application or network. (*use IPV4 for this project)

    3. If your application communicates by using IPv6 addresses, choose IPv6 CIDR block, Amazon-provided IPv6 CIDR block.(* not needed for this project)

  5. Configure the subnets

    1. For Number of Availability Zones, choose 2, so that you can launch instances in multiple Availability Zones to improve resiliency.

    2. For Number of public subnets, choose 2.

    3. For Number of private subnets, choose 2.

    4. You can keep the default CIDR block for the public subnet, or alternatively you can expand Customize subnet CIDR blocks and enter a CIDR block. For more information, see Subnet CIDR blocks.( *keep as default)

  6. For NAT gateways, choose 1 per AZ to improve resiliency.

  7. For VPC endpoints, if your instances must access an S3 bucket, keep the S3 Gateway default. Otherwise, instances in your private subnet can't access Amazon S3. There is no cost for this option, so you can keep the default if you might use an S3 bucket in the future. If you choose None, you can always add a gateway VPC endpoint later on.( *for this project we are not using S3 bucket as we are focusing on creating VPC, so select none)

  8. For DNS options, keep DNS options as default

  9. Choose Create VPC.

  10. PREVIEW of VPC

Deploy your application

Ideally, you've finished testing your servers in a development or test environment, and created the scripts or images that you'll use to deploy your application in production.

You can use Amazon EC2 Auto Scaling to deploy servers in multiple Availability Zones and maintain the minimum server capacity required by your application.

To launch instances by using an Auto Scaling group

  1. Create a launch template to specify the configuration information needed to launch your EC2 instances by using Amazon EC2 Auto Scaling. For step-by-step directions, see Create a launch template for your Auto Scaling group in the Amazon EC2 Auto Scaling User Guide.

  2. Create an Auto Scaling group, which is a collection of EC2 instances with a minimum, maximum, and desired size. For step-by-step directions, see Create an Auto Scaling group using a launch template in the Amazon EC2 Auto Scaling User Guide.

  3. Create a load balancer, which distributes traffic evenly across the instances in your Auto Scaling group, and attach the load balancer to your Auto Scaling group. For more information, see the Elastic Load Balancing User Guide and Use Elastic Load Balancing in the Amazon EC2 Auto Scaling User Guide.

Detailed Explanation :

1) Create launch template:

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. On the navigation pane, under Instances, choose Launch Templates.

  3. Choose Create launch template.

  4. Enter a Name(eg:prod-example) and provide a description(eg: proof of concept for app deploying in AWS private subnet) for the initial version of the launch template.

  5. (Optional) Under Auto Scaling guidance, select the check box to have Amazon EC2 provide guidance to help create a template to use with Amazon EC2 Auto Scaling. (*enable)

  6. Under Launch template contents, fill out each required field and any optional fields as needed.

    a) Application and OS Images (Amazon Machine Image): (Required) Choose the ID of the AMI for your instances. You can search through all available AMIs, or select an AMI from the Recents or Quick Start list.
    b) For Instance type, choose a single instance type that's compatible with the AMI that you specified.(*select t2.micro which is a free tier)
    c) Key pair (login): For Key pair name, choose an existing key pair, or choose Create new key pair to create a new one.
    d) Network settings:
    → Subnet :Don’t change anything, keep default(* Don’t include in launch template)
    → For Firewall (security groups), use one or more security groups if exists, else create Security group

    Create Security group:
    1. Enter Name and Description of security group.
    2. VPC : Select VPC in which you want to launch ASG instances( the one which you created above)
    3.Inbound Security Group Rules: you can allow “All Traffic”, but its not best practice. you should open ports that are required.
    a) SSH port(22) : open this port for SSH access and select “Source type” : Anywhere
    b) Custom TCP : this port is for the application that are deployed in your instances. select “Port r range” according to your needs (eg : 8000)and select “Source type” : Anywhere
    Note: If you don't specify any security groups in your launch template, Amazon EC2 uses the default security group for the VPC that your Auto Scaling group will launch instances into. By default, this security group doesn't allow inbound traffic from external networks

  7. EBS Volumes: if you don’t want to add (*keep the settings as default)

  8. Resource Tags and Advanced details: (*keep the settings as default)

  9. Click on Create launch template

2) Create ASG using launch Template

When you create an Auto Scaling group, you must specify the necessary information to configure the Amazon EC2 instances, the Availability Zones and VPC subnets for the instances, the desired capacity, and the minimum and maximum capacity limits.

Prerequisites

  • You must have created a launch template.

Steps to create ASG using launch template:

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/, and choose Auto Scaling Groups from the navigation pane.

  2. On the navigation bar at the top of the screen, choose the same AWS Region that you used when you created the launch template.

  3. Choose Create an Auto Scaling group.

  4. On the Choose launch template or configuration page, do the following:

    1. For Auto Scaling group name, enter a name for your Auto Scaling group.

    2. For Launch template, choose an existing launch template.(*select the one which you have created above)

    3. For Launch template version, choose whether the Auto Scaling group uses the default, the latest, or a specific version of the launch template when scaling out.(*select default)

    4. Verify that your launch template supports all of the options that you are planning to use, and then choose Next.

  5. On the Choose instance launch options page,

    1. Don’t override instance type requirements, keep the same instance attributes or instance type from your launch template

    2. Under Network, for VPC, choose a VPC. The Auto Scaling group must be created in the same VPC as the security group you specified in your launch template.

    3. For Availability Zones and subnets, choose one or more subnets( *select private subnets in two availability zone because we launch our application in private subnet) in the specified VPC. Use subnets in multiple Availability Zones for high availability .

    4. Choose Next to continue to the next step.

  6. On the Configure advanced options page,

    1. for now select No load balancer , for this project we are not attaching load balancer in ASG. we will create Application Load Balancer (ALB) in public subnet.

    2. VPC Lattice integration options ; Health checks ; Additional settings (* Dont configure anything ,keep as default) and click on Next

  7. (Optional) On the Configure group size and scaling policies page, configure the following options, and then choose Next:

    1. Under Group size, for Desired capacity, enter the initial number of instances to launch.(*select 2)

    2. In the Scaling section, under Scaling limits, if your new value for Desired capacity is greater than Min desired capacity and Max desired capacity, the Max desired capacity is automatically increased to the new desired capacity value. You can change these limits as needed

  8. Scaling policies ; Instance scale-in protection (* Dont configure anything ,keep as default ) and

  9. Notification and Tags (optional )

  10. click on Next and Launch your ASG

  11. After launching ASG, verify whether ASG has created instances in private subnets of each availablity zone

-> Note : Before creating ALB, you have to install the application inside the server (instances)

Install Application in EC2 instances:

  • EC2 instances in a private subnet do not have public IPs, so they cannot be accessed directly. To access these instances, a Bastion host is used. The Bastion host is placed in a public subnet, allowing secure access to the private subnet.

  • First we will SSH into bastion host, from bastion host we will SSH into private subnet

1) Launch the Bastion Host

  1. Go to the EC2 Dashboard:

    • Click on Launch Instance.

  2. Choose AMI:

    • Select an Amazon Linux or any other Linux/Windows AMI.

  3. Choose Instance Type:

    • Select an appropriate instance type (e.g., t2.micro for testing).

  4. Add Key Pair:

    • Use an existing key pair or create a new one for SSH/RDP access.

  5. Configure Network Settings:

    • Ensure Bastion Host is created in same VPC (*select required VPC in which you need to launch this instance otherwise you cannot access private subnets, if you launch in different VPC)

    • Assign the instance to the public subnet.

    • Enable Auto-assign Public IP.

  6. Add Security Group:

    • Create a security group to allow SSH (port 22) or RDP (if Windows) from your trusted IP range (e.g., your office/home IP).

  7. Configure Storage and Additional settings (* keep as default)

  8. Launch the instance

Why Copy a .pem File to a Bastion Host?

  1. No Direct Access to Private Instances:

    • If private instances are in a VPC and do not have public IPs, they can only be accessed through the Bastion host, which serves as a gateway to the private network.

  2. SSH Key Required for Access:

    • Private instances may require the .pem key for SSH authentication. Without the .pem file on the Bastion host, it can be difficult to connect to the private instances from there.

2)To copy a .pem file from your personal laptop to an AWS Bastion host, follow these detailed steps:

Step 1: Prerequisites

  1. SSH Access to the Bastion Host:

    • Ensure you have the .pem file (key pair) for the Bastion host. This is required to SSH into it. If you don't have the private key for the Bastion host, refer to key recovery or access alternatives (e.g., AWS Systems Manager).

  2. Public IP of the Bastion Host:

    • Obtain the public IP address of your Bastion host from the AWS EC2 console.

  3. Trusted IP Configuration:

    • Ensure the security group for the Bastion host allows SSH (port 22) from your personal laptop's public IP. Use services like https://whatismyipaddress.com to find your IP.

  4. .pem File to Transfer:

    • Identify the .pem file on your laptop that you need to transfer to the Bastion host. Ensure this file has proper permissions:

        chmod 400 /path/to/private-key.pem

Step 2:Use scp to Transfer the .pem File

The scp (Secure Copy Protocol) command allows you to transfer files securely to the Bastion host over SSH.

  1. Run the scp Command:

     scp -i /path/to/bastion-key.pem /path/to/private-key.pem ec2-user@<bastion-public-ip>:/home/ec2-user/

    Replace:

    • /path/to/bastion-key.pem: Path to the Bastion host's key pair.

    • /path/to/private-key.pem: Path to the .pem file on your laptop that you need to copy.

    • <bastion-public-ip>: Public IP of the Bastion host.

    • /home/ec2-user/: Destination directory on the Bastion host.

Step 3: Test SSH Connectivity to the Bastion Host

  • Use the ssh command to connect to the Bastion host:

      ssh -i /path/to/bastion-key.pem ec2-user@<bastion-public-ip>

    Replace:

    • /path/to/bastion-key.pem: Path to the key pair for the Bastion host.

    • <bastion-public-ip>: Public IP of the Bastion host.

    • After running the command, verify the file is copied:

        ls

Step 4: Set Correct Permissions on the .pem File

Set Permissions for the Copied File:

    • Make the .pem file readable only by the owner:

         chmod 400 /home/ec2-user/private-key.pem

Step 5: Use the .pem File to Access Private Instances

  1. SSH from the Bastion Host to the Private Instance:

    • Use the .pem file to SSH into a private EC2 instance:

        ssh -i /home/ec2-user/private-key.pem ec2-user@<private-instance-private-ip>

      Replace <private-instance-private-ip> with the private IP address of the EC2 instance in the private subnet.

  2. Confirm Access:

    • Verify that you are now connected to the private instance.

Step 6: Clean Up

  1. Delete the .pem File from the Bastion Host:

    • Once you're done, delete the .pem file from the Bastion host to minimize security risks:

        rm /home/ec2-user/private-key.pem

  2. Restrict Access:

    • Update the security group for the Bastion host to allow SSH access only from trusted IP addresses.

Troubleshooting

  1. Permission Denied:

    • If you see a "Permission denied" error while SSHing, check the permissions of your .pem files on your laptop and Bastion host:

        chmod 400 /path/to/private-key.pem
  chmod 400 /home/ec2-user/private-key.pem

  2. Incorrect IP:

    • Ensure your laptop’s public IP matches the allowed IP in the Bastion host's security group.

  3. Firewall/Route Issues:

    • Confirm that the private instance’s security group allows SSH from the Bastion host's private IP.

Security Best Practices

  • Avoid Persistent Keys:

    • Use AWS Systems Manager Session Manager to manage instances without needing to copy .pem files.

  • Logging:

    • Enable logging on the Bastion host to monitor SSH activities.

  • Secure .pem Files:

    • Never share .pem files publicly and always store them securely.

3)Install the python application in private EC2 instances

3) Create ALB and attach to ASG

→ ALB should be in Public Subnet and should be internet facing

Steps to Create an Application Load Balancer

1. Prerequisites

  • AWS Account: Ensure you have an active AWS account.

  • VPC Configuration: Identify the VPC and subnets where you want to deploy the ALB. The subnets should be in different availability zones for high availability.

  • Security Groups: Prepare security groups for the ALB and the backend target instances. The ALB security group should allow incoming HTTP/HTTPS traffic.

  • Target Instances: Ensure you have EC2 instances or a service running to register as targets in the load balancer.

2. Create the Application Load Balancer

  1. Log in to AWS Management Console

    • Navigate to the EC2 Dashboard.

  2. Go to Load Balancers

    • In the left-hand menu, select Load Balancers under the "Load Balancing" section.

    • Click Create Load Balancer.

  3. Choose Load Balancer Type

    • Select Application Load Balancer.

  4. Configure Basic Settings

    • Name: Enter a name for your ALB (e.g., my-app-alb).

    • Scheme: Choose Internet-facing: To route requests from the internet.

    • IP Address Type: Choose IPv4

  5. Select Network Mappings

    • Select the VPC where the ALB will be deployed (*select VPC that you have created)

    • Choose at least two subnets in different availability zones for high availability (*choose public subnet in two availability zones)

  6. Security Groups

    • Assign a security group to the ALB. Ensure the group allows incoming traffic on the listener ports (e.g., 80 or 443).

  7. Listeners and routing

    • Add one or more listeners (e.g., HTTP (port 80) or HTTPS (port 443)).

    • select HTTP→port 80

      1. Create Target Group( where you will define which instances should be accessible)

        • Define a target group that the ALB will forward traffic to:

          • Target Type: Select Instances

          • Target group Name

          • Protocol: HTTP

          • VPC: (select VPC that you have created)

          • Protocol version: (*keep default)

          • Health Check Settings: Configure health checks to monitor target health (e.g., /health endpoint). (*keep default)

          • Set the thresholds for health checks (e.g., response timeout, success codes).

      2. Register Targets

        • Add the targets (e.g., EC2 instances) to the target group.( Do not choose bastion Host)

        • click Include as pending below

      3. Click on Create Target group

  8. Select target group for port 80

  9. Add on services and Load balancer tags (*keep as default)

  10. Click on Create Load balancer

Confirm that the instances are marked as healthy in the target group.

Get ALB DNS Name

  1. In the AWS Management Console, navigate to EC2 > Load Balancers.

  2. Select your ALB and copy the DNS name (e.g., my-alb-1234567890.us-west-2.elb.amazonaws.com).

Access the Website

  • Open a browser and visit the ALB DNS name:

      http://my-alb-1234567890.us-west-2.elb.amazonaws.com

  • If you're using HTTPS, ensure the ALB has a valid SSL certificate configured in a listener.

Debugging Tips

  • ALB Health Check Failure: Review the health check path and port in the target group settings.

  • Error in Browser: Check logs on the Python webserver for errors and confirm the ALB is forwarding traffic correctly.

Test your configuration

After you've finished deploying your application, you can test it. If your application can't send or receive the traffic that you expect, you can use Reachability Analyzer to help you troubleshoot. For example, Reachability Analyzer can identify configuration issues with your route tables or security groups.

Clean up

When you are finished with this configuration, you can delete it. Before you can delete the VPC, you must delete the Auto Scaling group, terminate your instances, delete the NAT gateways, and delete the load balancer.

"Thank you for reading! I hope this blog sparked new ideas and insights. If you have questions or thoughts, drop a comment below. Until next time, keep learning and growing!"
Reach out to me at linkedin.com/in/sruthipalle
Happy Coding😊

ALB-AWSAWSaws productionvpcauto-scaling-aws