How to create custom CloudWatch memory metrics for ubuntu EC2 instance?

by XDK 28. September 2019 01:39

Explanation:

Launch an EC2 instance for which the custom CloudWatch memory metrics need to be generated.

Create an IAM role named "CustomMetricsRole" with a custom policy name "CustomEC2CloudWatch" to establish the interoperability between EC2 instance and CloudWatch as below
Policy : CustomEC2CloudWatch
Permissions:
{
"Version": "2012-10-17",
   "Statement": [
     {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [

          "cloudwatch:GetMetricStatistics",
          "cloudwatch:PutMetricData",
          "cloudwatch:GetMetricData",
          "ec2:DescribeTags",
          "cloudwatch:ListMetrics"

       ],
       "Resource": "*"
     }
  ]
}
Attach IAM role to the EC2 instance

SSH to EC2 instance
$ sudo apt update
$ sudo apt install unzip
$ sudo apt install perl
$ sudo apt install liblwp-protocol-https-perl libdatetime-perl

Download the CloudWatch Monitoring perl scripts for Linux
$ wget http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
$ unzip CloudWatchMonitoringScripts-1.2.1.zip
$ cd aws-scripts-mon

mon-put-instance-data.pl - Collects system metrics on an EC2 instance and sends them to CloudWatch
mon-get-instance-stats.pl - Queries CloudWatch and displays the most resent metrics
AwsSignatureV4.pm - File template for AWS credentials

Push custom metrics from EC2 instance to CloudWatch
$ ./mon-put-instance-data.pl --mem-util --mem-used --mem-avail

Create crontab and add the script to run at every minute intervals.
$ crontab -e
* * * * * /home/ubuntu/aws-scripts-mon/mon-put-instance-data.pl --mem-util --mem-used --mem-avail

 

To retrive the most recent custom metrics from CloudWatch to EC2 instance
$
./mon-get-instance-stats.pl

Tags: ,

Amazon Web Service | CloudWatch | Perl

Linux Cron Job Examples

by XDK 27. September 2019 22:38

Crontab Commands
export EDITOR=vi ;to specify a editor to open crontab file.
crontab -e Edit crontab file, or create one if it doesn’t already exist.
crontab -l crontab list of cronjobs , display crontab file contents.
crontab -r Remove your crontab file.
crontab -v Display the last time you edited your crontab file. (This option is only available on a few systems.)

Linux Crontab Format
MIN HOUR DOM MON DOW CMD

Schedule a cron to execute at 2am daily.
This will be useful for scheduling database backup on a daily basis.
0 2 * * * CMD
Asterisk (*) is used for matching all the records.

Schedule a cron to execute twice a day.
Below example command will execute at 5 AM and 5 PM daily. You can specify multiple time stamp by comma separated.
0 5,17 * * * CMD

Schedule a cron to execute on every minutes.
Generally, we don’t require any script to execute on every minute but in some case, you may need to configure it.
* * * * * CMD

Schedule a cron to execute on every Sunday at 5 PM.
This type of cron is useful for doing weekly tasks, like log rotation, etc.
0 17 * * sun CMD

Schedule a cron to execute on every 10 minutes.
If you want to run your script on 10 minutes interval, can configure like below. These type of crons are useful for monitoring.
*/10 * * * * /scripts/monitor.sh
*/10: means to run on every 10 minutes. Same as if you want to execute on every 5 minutes use */5.

Schedule a cron to execute on selected months.
Sometimes we required scheduling a task to be executed for selected months only. Below example script will run in January, May and August months.
* * * jan,may,aug * /script/script.sh

Schedule a cron to execute on selected days.
If you required scheduling a task to be executed for selected days only. Below example will run on each Sunday and Friday at 5 PM.
0 17 * * sun,fri /script/script.sh

Schedule a cron to execute on first sunday of every month.
To schedule a script to execute a script on first Sunday only is not possible by time parameter, But we can use the condition in command fields to do it.
0 2 * * sun [ $(date +%d) -le 07 ] && /script/script.sh

Schedule a cron to execute on every four hours.
If you want to run a script on 4 hours interval. It can be configured like below.
0 */4 * * * CMD

Schedule a cron to execute twice on every Sunday and Monday.
To schedule a task to execute twice on Sunday and Monday only. Use the following settings to do it.
0 4,17 * * sun,mon CMD

Schedule a cron to execute on every 30 Seconds.
To schedule a task to execute on every 30 seconds is not possible by time parameters, But it can be done by schedule same cron twice like below.
* * * * * CMD
* * * * * sleep 30; CMD

Schedule a multiple tasks in single cron.
To configure multiple tasks with single cron, Can be done by separating tasks by the semicolon ( ; ).
* * * * * CMD; /scripts/scrit2.sh

Schedule tasks to execute on yearly ( @yearly ).
@yearly timestamp is similar to “0 0 1 1 *”. It will execute task on the first minute of every year.
@yearly CMD

Schedule tasks to execute on monthly ( @monthly ).
@monthly timestamp is similar to “0 0 1 * *”. It will execute a task in the first minute of the month.
@monthly CMD

Schedule tasks to execute on Weekly ( @weekly ).
@weekly timestamp is similar to “0 0 1 * mon”. It will execute a task in the first minute of the week.
@weekly /bin/script.sh

Schedule tasks to execute on daily ( @daily ).
@daily timestamp is similar to “0 0 * * *”. It will execute a task in the first minute of every day.
@daily CMD

Schedule tasks to execute on hourly ( @hourly ).
@hourly timestamp is similar to “0 * * * *”. It will execute a task in the first minute of every hour.
@hourly CMD

Schedule tasks to execute on system reboot ( @reboot ).
@reboot is useful for those tasks which you want to run on your system startup. It is useful for starting tasks in the background automatically.
@reboot CMD

Tags: , ,

Linux

Setup and Configure Master-Slaves Architecture in Jenkins

by XDK 26. September 2019 23:05

 

 

 

 

Setup Auto SSH Login

SSH to Jenkins Master
    $ sudo -iu jenkins
Generate Public & Private RSA Key.
    $ ssh-keygen -t rsa
       public key location : /var/lib/jenkins/.ssh/id_rsa.pub
       private key location : /var/lib/jenkins/.ssh/id_rsa
Create .ssh directory on Jenkins Slave 1 from Jenkins Master
    $ ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.91.65.12 mkdir -p .ssh
Attach master public key with slave authorized directory
    $ cat .ssh/id_rsa.pub | ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.91.65.12 'cat >> .ssh/authorized_keys'
Verify connection
    $ ssh ubuntu@3.91.65.12
Create .ssh directory on Jenkins Slave 2 from Jenkins Master
    $ ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.95.20.115 mkdir -p .ssh
Attach master public key with slave authorized directory
    $ cat .ssh/id_rsa.pub | ssh -i /home/ubuntu/bin/myaccountkey.pem ubuntu@3.95.20.115 'cat >> .ssh/authorized_keys'

Download Jenkins Slave 1 Agent Program on Jenkins Slave 1 Machine

SSH to Jenkins Slave 1
Create bin directory.
    $ mkdir ~/bin
Goto bin and download slave.jar from master
    $ cd bin 
    $ wget http://54.81.205.161:8080/jnlpJars/slave.jar

Add Jenkins Slave 1 Machine to Jenkins Master
Login to Jenkins site

 

Launch Command
    ssh ubuntu@3.91.65.12 java -jar ./bin/slave.jar

Download Jenkins Slave 2 Agent Program on Jenkins Slave 2 Machine

SSH to Jenkins Slave 2
Create bin directory.
    $ mkdir ~/bin
Goto bin and download slave.jar from master
    $ cd bin 
    $ wget http://54.81.205.161:8080/jnlpJars/slave.jar

Add Jenkins Slave 2 Machine to Jenkins Master
Login to Jenkins site

Launch Command
    $ ssh ubuntu@3.95.20.115 java -jar ./bin/slave.jar

Tags: , ,

Amazon Web Service | Jenkins

Docker Cheat Sheet for Ubuntu

by XDK 19. September 2019 18:37

Tags: ,

Amazon Web Service | Docker

How to install & configure AWS CLI?

by XDK 30. August 2019 21:28

Explanation:

Command to Install AWS CLI
ubuntu@ip-172-31-43-31:~$ sudo apt install awscli

Create user for AWS CLI configuration

Command to configure AWS CLI
ubuntu@ip-172-31-43-31:~$ aws configure
AWS Access Key ID [None]: XXXXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXXXX
Default region name [None]: us-east-1
Default output format [None]:

Tags: ,

Amazon Web Service

About the author

My name is Xavier Dilip Kumar Jayaraj having 16+ years of IT experience which includes solid experience and depth Knowledge in Application Life Cycle Management, Configuration Management, Implementation and Support using TFS on-premises and Azure DevOps. I have invested in gaining DevOps knowledege to expertise with Cloud Computing providers namely Microsoft Azure and Amazon Web Services in recent years. I am very positive to learn and adapt emerging technologies to client’s environment.

Microsoft Certified: Azure Administrator Associate

Microsoft Certified: Azure DevOps Engineer Expert

DevOps Engineer Certificate Program | Transcript 

OTP-AWSD5: AWS IoT: Developing and Deploying an Internet of Things

[PCEP-30-01] PCEP – Certified Entry-Level Python Programmer

Quotes I Like

"Failure will never overtake me if my determination to succeed is strong enough."  - Dr. APJ. Abdul Kalam

"Always be yourself, express yourself, have faith in yourself, do not go out and look for a successful personality and duplicate it." - Bruce Lee

"Technology is just a tool. In terms of getting the kids working together and motivating them, the teacher is the most important." - Bill Gates

"Innovation distinguishes between a leader and a follower." - Steve Jobs

CategoryList

Disclaimer

The information provided here is based on my expreriences, troubleshooting and online/offline findings. It can be used as is on your own risk without any warranties and I impose no rights.