How to create custom CloudWatch memory metrics for ubuntu EC2 instance?

by XDK 28. September 2019 01:39

Explanation:

Launch an EC2 instance for which the custom CloudWatch memory metrics need to be generated.

Create an IAM role named "CustomMetricsRole" with a custom policy name "CustomEC2CloudWatch" to establish the interoperability between EC2 instance and CloudWatch as below
Policy : CustomEC2CloudWatch
Permissions:
{
"Version": "2012-10-17",
   "Statement": [
     {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [

          "cloudwatch:GetMetricStatistics",
          "cloudwatch:PutMetricData",
          "cloudwatch:GetMetricData",
          "ec2:DescribeTags",
          "cloudwatch:ListMetrics"

       ],
       "Resource": "*"
     }
  ]
}
Attach IAM role to the EC2 instance

SSH to EC2 instance
$ sudo apt update
$ sudo apt install unzip
$ sudo apt install perl
$ sudo apt install liblwp-protocol-https-perl libdatetime-perl

Download the CloudWatch Monitoring perl scripts for Linux
$ wget http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
$ unzip CloudWatchMonitoringScripts-1.2.1.zip
$ cd aws-scripts-mon

mon-put-instance-data.pl - Collects system metrics on an EC2 instance and sends them to CloudWatch
mon-get-instance-stats.pl - Queries CloudWatch and displays the most resent metrics
AwsSignatureV4.pm - File template for AWS credentials

Push custom metrics from EC2 instance to CloudWatch
$ ./mon-put-instance-data.pl --mem-util --mem-used --mem-avail

Create crontab and add the script to run at every minute intervals.
$ crontab -e
* * * * * /home/ubuntu/aws-scripts-mon/mon-put-instance-data.pl --mem-util --mem-used --mem-avail

 

To retrive the most recent custom metrics from CloudWatch to EC2 instance
$
./mon-get-instance-stats.pl

Tags: ,

Amazon Web Service | CloudWatch | Perl

Setup and Configure Master-Slaves Architecture in Jenkins

by XDK 26. September 2019 23:05

 

 

 

 

Setup Auto SSH Login

SSH to Jenkins Master
    $ sudo -iu jenkins
Generate Public & Private RSA Key.
    $ ssh-keygen -t rsa
       public key location : /var/lib/jenkins/.ssh/id_rsa.pub
       private key location : /var/lib/jenkins/.ssh/id_rsa
Create .ssh directory on Jenkins Slave 1 from Jenkins Master
    $ ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.91.65.12 mkdir -p .ssh
Attach master public key with slave authorized directory
    $ cat .ssh/id_rsa.pub | ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.91.65.12 'cat >> .ssh/authorized_keys'
Verify connection
    $ ssh ubuntu@3.91.65.12
Create .ssh directory on Jenkins Slave 2 from Jenkins Master
    $ ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.95.20.115 mkdir -p .ssh
Attach master public key with slave authorized directory
    $ cat .ssh/id_rsa.pub | ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.95.20.115 'cat >> .ssh/authorized_keys'

Download Jenkins Slave 1 Agent Program on Jenkins Slave 1 Machine

SSH to Jenkins Slave 1
Create bin directory.
    $ mkdir ~/bin
Goto bin and download slave.jar from master
    $ cd bin 
    $ wget http://54.81.205.161:8080/jnlpJars/slave.jar

Add Jenkins Slave 1 Machine to Jenkins Master
Login to Jenkins site

 

Launch Command
    ssh ubuntu@3.91.65.12 java -jar ./bin/slave.jar

Download Jenkins Slave 2 Agent Program on Jenkins Slave 2 Machine

SSH to Jenkins Slave 2
Create bin directory.
    $ mkdir ~/bin
Goto bin and download slave.jar from master
    $ cd bin 
    $ wget http://54.81.205.161:8080/jnlpJars/slave.jar

Add Jenkins Slave 2 Machine to Jenkins Master
Login to Jenkins site

Launch Command
    $ ssh ubuntu@3.95.20.115 java -jar ./bin/slave.jar

Tags: , ,

Amazon Web Service | Jenkins

Docker Cheat Sheet for Ubuntu

by XDK 19. September 2019 18:37

Tags: ,

Amazon Web Service | Docker

EC2 instance user data is not executed.

by XDK 18. September 2019 22:27
Explanation:
 
The user data is not executed due to the leading whitespaces in the following format
 
resource "aws_instance" "Terraform-Demo" {
  ami = "${lookup(var.RegionToAmazonAMI, var.Region)}"
  instance_type = "t2.micro"
  key_name = "myaccountkey"
  vpc_security_group_ids =  ["${aws_security_group.webserver-sg-2.id}"]
  user_data = <<EOF
    #cloud-boothook
    #! /bin/bash
    sudo apt update
    sudo apt install -y apache2
    IP_ADDR=$(curl http://169.254.169.254/latest/meta-data/public-ipv4)
    sudo chmod 777  /var/www/html
    sudo echo "Terraform instance 2 with IP IP_ADDR" > /var/www/html/index.html
    sudo echo "Terraform instance 2 with IP IP_ADDR -- OK" > /var/www/html/health.html
  EOF
  provisioner "local-exec" {
    command = "echo ${aws_instance.Terraform-Demo.public_ip} > testP.txt"
    }
  tags = {
    Name = "TerraformDemo"
  }
}
 
Solution:
 
Either add hyphen in <<-EOF to trim the leading whitespaces or remove the whitespaces manually 

Tags:

Amazon Web Service | Terraform

EC2 instances launched in the subnet missing public IPv4 address

by XDK 18. September 2019 22:09

Explanation:
EC2 instances launched in the subnet using Terraform or Cloudformation is missing public IPv4 address

Solution:
Add the following argument/property

CloudFormation:

MapPublicIpOnLaunch - Indicates whether instances launched in this subnet receive a public IPv4 address.

PubSubnetZoneA:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone:
Fn::Select:
- '0'
- Fn::GetAZs:
Ref: 'AWS::Region'
CidrBlock: '10.0.10.0/24'
MapPublicIpOnLaunch: 'True'
VpcId:
Ref: 'VPC'
Tags:
- Key: 'Name'
Value:
'Fn::Join': [ ':', [ 'Public', 'Zone A', !Ref 'AWS::StackName' ] ]

Terraform:

map_public_ip_on_launch - (Optional) Specify true to indicate that instances launched into the subnet should be assigned a public IP address. Default is false.

resource "aws_subnet" "public-subnet-a" {
vpc_id = "${aws_vpc.vpc.id}"
map_public_ip_on_launch = true
cidr_block = "10.0.10.0/24"
availability_zone = "${data.aws_availability_zones.available.names[0]}"
tags = {
Name = "public-subnet-a:${var.labname}"
}
}

Tags:

Amazon Web Service | Terraform | CloudFormation

About the author

My name is Xavier Dilip Kumar Jayaraj having 13+ years of IT experience which includes solid experience and depth Knowledge in Application Life Cycle Management, Configuration Management, Implementation and Support using TFS on-premises and Azure DevOps. I have invested in gaining DevOps knowledege to expertise with Cloud Computing providers namely Microsoft Azure and Amazon Web Services in recent years. I am very positive to learn and adapt emerging technologies to client’s environment.

DevOps Engineer Certificate Program | Transcript | ePortfolio

Microsoft Certified: Azure Administrator Associate

[PCEP-30-01] PCEP – Certified Entry-Level Python Programmer

Quotes I Like

"Failure will never overtake me if my determination to succeed is strong enough."  - Dr. APJ. Abdul Kalam

"Always be yourself, express yourself, have faith in yourself, do not go out and look for a successful personality and duplicate it." - Bruce Lee

"Technology is just a tool. In terms of getting the kids working together and motivating them, the teacher is the most important." - Bill Gates

"Innovation distinguishes between a leader and a follower." - Steve Jobs

CategoryList

Disclaimer

The information provided here is based on my expreriences, troubleshooting and online/offline findings. It can be used as is on your own risk without any warranties and I impose no rights.