tag:blogger.com,1999:blog-32676036724902664052024-02-07T14:34:26.307-08:00TECH OUTREACHsuramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.comBlogger50125tag:blogger.com,1999:blog-3267603672490266405.post-73940917517448549972020-04-14T11:26:00.008-07:002020-04-14T11:26:55.445-07:00AWS Project<div dir="ltr" style="text-align: left;" trbidi="on">
Follow the posts step by step to create your own aws project from scratch:<br />
1) Create public private subnet internet gateway and NAT:<br />
<a href="https://tech-outreach.blogspot.com/2020/04/create-vpc-public-private-subnets-nat.html">https://tech-outreach.blogspot.com/2020/04/create-vpc-public-private-subnets-nat.html</a><br />
2) Creating EC2 instance with bootstrap script<br />
<a href="https://tech-outreach.blogspot.com/2020/01/creating-ec2-instance-with-bootstrap.html">https://tech-outreach.blogspot.com/2020/01/creating-ec2-instance-with-bootstrap.html</a><br />
3)Create RDS security group and RDS subnet group with private subnets<br />
<a href="https://tech-outreach.blogspot.com/2020/01/rds.html">https://tech-outreach.blogspot.com/2020/01/rds.html</a><br />
4)Create ELB launch configuration and AutoScaling.<br />
<a href="https://tech-outreach.blogspot.com/2020/01/create-elb-and-auto-scaling.html">https://tech-outreach.blogspot.com/2020/01/create-elb-and-auto-scaling.html</a><br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-71200986233047378232020-04-14T10:54:00.000-07:002020-04-14T10:54:54.785-07:00kubernetes: copy secret from one namespace to another<div dir="ltr" style="text-align: left;" trbidi="on">
kubectl get secret SECRET-NAME --namespace=NS --export -o yaml | kubectl apply --namespace=NS -f -<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-15583661044268850372020-04-01T04:05:00.002-07:002020-04-01T04:07:46.767-07:00Create VPC , Public, Private subnets, NAT Gateway, IGW<div dir="ltr" style="text-align: left;" trbidi="on">
Following are the steps:<br />
<i><b>1) Create VPC</b></i><br />
<i><b>2) Create subnets</b></i><br />
<i><b>3) Create and attach Internet Gateway to VPC</b></i><br />
<i><b>4) Attach Internet Gateway to default route table</b></i><br />
<b><i>5) Creating an additional route table for private subnets and public subnets.</i> </b><br />
Let us get started:<br />
<b>1)</b>Open AWS Console select a region from top left , and on services search VPC,<br />
click on your vpcs on the left and the create vpc<br />
Name tag<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"> : project-vpc CIDR: 10.0.0.0/16 (range </span><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">can only be between 16 to 28, 16 being the max) </span></span><br />
<div class="GD0KV33CGTF" style="width: 230px;">
<div class="GD0KV33CFTF">
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><b>2)</b>For subnet</span></span>: 4 subnets -<br />
2 private and 2 public </div>
</div>
give proper names and <span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">associate 251 ips by giving CIDR block like </span></span><br />
<i><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">public-subnet-us-west-2a - > </span></span><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">10.0.1.0/24</span></span></span></span></i><br />
<i><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">public-subnet-1b -> 10.0.2.0/24</span></span></i><br />
<i><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">private-subnet-1a -> </span></span><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">10.0.3.0/24</span></span></span></span></i><br />
<i><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">private-subnet-1b -> </span></span><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">10.0.4.0/24</span></span></span></span></i><br />
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">Till now the configuration for public and private subnets will be same attaching a route table will make a difference!! </span></span></span></span>
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><br /></span></span>
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><b>3)</b> For IGW</span></span><br />
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">Create an Internet Gateway and attach it to our vpc named "project-vpc"</span></span><br />
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">note: 1 Internet Gateway can only be attached to 1 VPC</span></span>
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><b>4) </b>Route Table</span></span><br />
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">Gets created whenever we create a vpc.</span></span><br />
<span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">Make sure it accepts our traffic from outside, for that under route table add our Internet Gateway.</span></span><br />
<i><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">on destination: 0.0.0.0/0</span></span></i><br />
<i><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0"><span aria-haspopup="true" class="GD0KV33CNTF GD0KV33CLB GD0KV33CCD GD0KV33CED" tabindex="0">on IGW name add your Internet gateway ID</span></span></i><br />
<b>5)</b> Create 2 route tables named public and private route table for your VPC.<br />
Private route table: To private route table attach a NAT gateway or NAT Instance.<br />
1) NAT Gateway: On NAT Gateway choose a public subnet of the VPC, as NAT Gateway must be under a public subnet.<br />
2) Elastic IP: It is mandatory to create an Elastic IP.<br />
After this click on create NAT Gateway.<br />
3) Open Route Table again and add the created NAT Gateway to Private subnet Route Table:<br />
Destination: 0.0.0.0/0, Target: NAT-gateway-id<br />
Add the private subnets to the private route table by subnet association.<br />
Hola! now you have 2 route table:<br />
a- private route table with a NAT gateway attached with subnet association of private subnets.<br />
b- main route table which we renamed as public route table which now by default have 2 public subnets (2 private subnets were detached automatically after the subnet association we used in step a), this public route table also have an IGW attachedto it.</div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-76728288396933054972020-03-31T04:11:00.001-07:002020-03-31T04:27:14.740-07:00Build an aws project<div dir="ltr" style="text-align: left;" trbidi="on">
Following are the aws services we will be using:<br />
<br />
<b>1) Route 53: </b><br />
hosted domain and record set <br />
<b>2) VPC</b><br />
subnets, internet gateways, route table, security groups<br />
<b>3) SNS </b><br />
SES<br />
<b>4) ELB</b><br />
ACM<br />
<b>5) EC2</b><br />
Elastic IP, AutoScaling<br />
<b>6) RDS</b><br />
RDS Master, RDS read replica, Snapshot, RDS Subnets<br />
<b>7) S3</b><br />
Buckets and IAM access<br />
<b>8) Lambda</b><br />
Lambda Snaps<b>Lambda Snapshots and Event Processors</b>hots and Event Processors<br />
<b>9) CloudWatch</b><br />
Monitor Events and Alerts to SNS<br />
<b>10) Open VPN</b><br />
Admin access<br />
<br />
<br />
<b>Create VPC </b><br />
<br />
</div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-54754495886947274342020-03-30T07:27:00.003-07:002020-03-30T07:28:07.169-07:00centralised logs in kubernetes<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<br />
kubectl get po -n glp2-qa | awk '{print $1}' | grep -v environment-operator| grep -v eviews| grep -v kafkasvc-0| grep -v kafkasvc-1| grep -v kafkasvc-2| grep -v kong-admin|grep -v kong-api-gw|grep -v kong-api-gw|grep -v stub|grep -v stub-gms|grep -v zipkin |grep -v zkex-0|grep -v zkex-1| grep -v zkex-2| xargs -I {} kubectl -n glp2-qa logs {} | grep -i 5df09196e4b0b9222b58fcd3<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-29479052974382834942020-03-30T07:24:00.002-07:002020-03-30T07:24:56.397-07:00xclip<div dir="ltr" style="text-align: left;" trbidi="on">
xclip -selection clipboard < ~/.ssh/id_rsa.pub</div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-48676500985198188832020-03-30T05:10:00.005-07:002020-03-30T05:10:55.651-07:00Vi text Editor<div dir="ltr" style="text-align: left;" trbidi="on">
<br />--------------------------------------------------<br /><br />VI Text editor<br />---------------------------------------------------<br /><br />search /<br /><br /> Arrow keys - move the cursor around<br /> j, k, h, l - move the cursor down, up, left and right (similar to the arrow keys)<br /> ^ (caret) - move cursor to beginning of current line<br /> $ - move cursor to end of the current line<br /> nG - move to the nth line (eg 5G moves to 5th line)<br /> G - move to the last line<br /> w - move to the beginning of the next word<br /> nw - move forward n word (eg 2w moves two words forwards)<br /> b - move to the beginning of the previous word<br /> nb - move back n word<br /> { - move backward one paragraph<br /> } - move forward one paragraph<br /><br /><br /> :set nu - enable line numbers<br /><br /><br /> x - delete a single character<br /> nx - delete n characters (eg 5x deletes five characters)<br /> dd - delete the current line<br />
dn - d followed by a movement command. Delete to where the movement
command would have taken you. (eg d5w means delete 5 words)<br /><br /><br /><br /> u - Undo the last action (you may keep pressing u to keep undoing)<br /> U (Note: capital) - Undo all changes to the current line<br /><br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-6875317813954630892020-03-30T05:10:00.001-07:002020-03-30T05:11:23.040-07:00SHELL<div dir="ltr" style="text-align: left;" trbidi="on">
https://ryanstutorials.net/linuxtutorial/filters.php<br />
<br />
<b>commit only if some changes are there :</b><br />
<br />
if [ -n "$(git status --porcelain)" ]; then<br />
git add/git commit<br />
fi<br />
<br />
<b>cut command usage:</b><br />
v=$(echo "$file" | cut -d '/' -f2)<br />
<br />
using jquery with curl for json output:<br />
<br />
file=`curl -u "$bbUser":"$bbPass" "https://bitbucket.pearson.com/rest/api/1.0/projects/GLP/repos/glp-application-config-common-bitesize/pull-requests/$key/diff" | jq '.diffs[].source.toString'`<br />
<br />
<b>looping in shell:</b><br />
for i in `cat test.txt`;<br />
do<br />
echo $i<br />
done<br />
grep/sort/awk:<br />
grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}<br />
or cd 'Holiday Photos'<br />
<br />
Another method is to use what is called an escape character, which is a backslash ( \ ). What the backslash does is escape (or nullify) the special meaning of the next character.<br />
<br />
cd Holiday\ Photos<br />
pwd<br />
/home/ryan/Documents/Holiday Photos<br />
<br />
to know what a command does type man:<br />
e.g. man ls<br />
man -k to get what a manual command does<br />
<br />
touch: create a blank file<br />
<br />
----------------------------------------------------<br />
----------------------------------------------------<br />
<br />
Wildcards<br />
<br />
----------------------------------------------------<br />
----------------------------------------------------<br />
<br />
* - represents zero or more characters<br />
? - represents a single character<br />
[] - represents a range of characters<br />
<br />
<br />
<br />
----------------------------------------------------------------------------------------------------------------<br />
<br />
Filters<br />
--------------------------------------------------------<br />
--------------------------------------------------------<br />
<br />
head [-number of lines to print] [path]<br />
tail [-number of lines to print] [path]<br />
sort [options] filename<br />
sed 's/search/replace/g' sample.txt<br />
uniq file.txt<br />
tac- print last lines first<br />
<br />
<br />
top<br />
View real-time data about processes running on the system.<br />
ps<br />
Get a listing of processes running on the system.<br />
kill<br />
End the running of a process.<br />
jobs<br />
Display a list of current jobs running in the background.<br />
fg<br />
Move a background process into the foreground.<br />
ctrl + z<br />
Pause the current foreground process and move it into the background.<br />
<br />
<br />
----------------------------------------------------------------------------------------------------------------<br />
<br />vi<br /> Edit a file.<br />cat<br /> View a file.<br />less<br /> Convenient for viewing large files.<br />-----------------------------------------------------------------------------------------------------------------<br />
gitIDTag=`echo ${GIT_COMMIT} | cut -c 1-10` <br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-66431195537940717602020-03-30T05:06:00.002-07:002020-04-13T06:43:56.499-07:00AWS limits and defaults<div dir="ltr" style="text-align: left;" trbidi="on">
- Default no. of EC2 instances/region = 20<br />
- <span class="ng-scope"><span class="ng-scope"><span class="ng-binding ng-scope">IAM: A group can have no more than 10 policies attached.</span></span></span><br />
<span class="ng-scope"><span class="ng-scope"><span class="ng-binding ng-scope">- CIDR Range used in VPC 16 to 28 (e.g 10.0.0.0/15 not allowed)</span></span></span><br />
<span class="ng-scope"><span class="ng-scope"><span class="ng-binding ng-scope">- 1 IGW attached to only 1 VPC, 1 VPC cannot </span></span></span><br />
<span class="ng-scope"><span class="ng-scope"><span class="ng-binding ng-scope">- </span></span></span>You can ensure a public IP address is assigned to instances launched
with this configuration by selecting only default subnets of your
default VPC.<br />
- </div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-76741639439851976102020-03-30T04:55:00.000-07:002020-03-30T04:55:10.036-07:00AWS DBS<div dir="ltr" style="text-align: left;" trbidi="on">
- AWS no sequel DB: Neptune: NoSQL graph database<br />64 TB of datastore<br />- can be accessed through opensource APIs like gremlin and rdf4j console<br /><br />Elastic cache<br />- used to offload other DB such as rds and dynamodb<br />elastic cache works on 2 cluster engines=<br />1. Memcached<br />2.redis<br /><br /><br />Redshift<br />- data warehouse - the only service optimised for data analytics.<br /><br />Hybrid Environments:<br />1- <br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-46019118808570108512020-03-30T04:52:00.001-07:002020-03-30T07:26:56.464-07:00Taking java heap dump of application using jattach and s3<div dir="ltr" style="text-align: left;" trbidi="on">
exec app.<br />
CD BIN<br />
jattach 57 dumpheap /tmp/java_pid$java_pid-$(date +%Y-%m-%d_%H-%M-%S).hprof<br />
<br />
kubectl cp glp2-nf2/rbassessment-549967bb85-5d79l:/tmp/57-2019-09-16_11-59-21-nf2.hprof /home/suramyashah/heapdump.hprof<br />
<br />
<br />
exec app.<br />
CD BIN<br />
jattach 57 dumpheap /tmp/java_pid$java_pid-$(date +%Y-%m-%d_%H-%M-%S).hprof<br />
<br />
<br />
kubectl cp glp2-nf2/rbassessment-549967bb85-5d79l:/tmp/57-2019-09-16_11-59-21-nf2.hprof /home/suramyashah/heapdump.hprof<br />
<br />
<br />
//make bucket: aws s3 mb s3://heapdump-bucket<br />
<br />
aws s3 cp /home/suramyashah/heapdump.hprof s3://heapdump-bucket/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers<br />
<br />
<br />
</div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-38115177499635052052020-03-30T04:49:00.005-07:002020-03-30T04:50:11.155-07:00AWS networking<div dir="ltr" style="text-align: left;" trbidi="on">
services not bound to any region:<br />
route 53 <br />
<br />
bound on region level:<br />
s3, cdn,vpc<br />
<br />
VPC level:<br />
ELB,ec2,rds<br />
<br />
inside vpc:<br />
rds database, application load balancer<br />
<br />
AZ availability zones level:<br />
EC2, RDS<br />
<br />
CIDR- Classless Interdomain routing<br />
<br />
VPC Addressing : CIDR<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-2700323907311229452020-03-30T02:14:00.001-07:002020-04-14T11:39:11.083-07:00k8<div dir="ltr" style="text-align: left;" trbidi="on">
https://github.com/wardviaene/advanced-kubernetes-course/blob/master/authorization/role.yml<br />
<br />
<b>fluentd -</b> log forwarding<br />
<b>elastic search -</b> log indexing<br />
<b>kibana -</b> log visualisation<br />
<b>Logtrail -</b> easy to use kibana to show logs<br />
<br />
mount efs and ebs in kubernetes using storageClass<br />
<br />
stateful set needed- when static hostname<br />
<br />
some volume attached to sts efs or ebs<br />
<br />
<b>list all resources in kubernetes:</b><br />
kubectl get all -n glp2-int<br />
<b>Persistent volumes and PVC:</b><br />
PV is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes<br />
PVC <br />
kubectl get storage class<br />
for i in `kubectl get node | cut -d ' ' - f 1 | grep -i internal`; do kubectl label nodes ${i} NAME; done<br />
<br />
<b>Create everythin on the current directory:</b><br />
kubectl create -f .<br />
kubectl get role -n namespace<br />
kubectl get roleBinding -n namespace<br />
kubectl get ClusterRole <br />
kubectl get clusterRole binding<br />
<br />
<b>JOBS:</b><br />
3 kinds:<br />
<b>1- parallel</b><br />
<b>2- non parallel</b> with fixed completion count<br />
<b>3- non parallel</b> with work queue<br />
<br />
CRONJOBS:<br />
<br />
<br />
kubectl get pods --field-selector=status.phase!=Running </div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-82292572888795519842020-03-30T02:10:00.000-07:002020-03-30T02:10:13.671-07:00Linux Important commands<div dir="ltr" style="text-align: left;" trbidi="on">
check disk space in human readable format:<br />
<blockquote class="tr_bq">
<i>sudo df -h</i></blockquote>
subdirectory using the most space:<br />
<blockquote class="tr_bq">
<i>sudo du -h -d 1 /var/ </i></blockquote>
how your process are consuming memory, cpu and which cores are under pressure:<br />
<blockquote class="tr_bq">
<i>sudo htop</i></blockquote>
see process id :<br />
<blockquote class="tr_bq">
<i>sudo ps aux</i></blockquote>
a = show processes for all users<br /> u = display the process's user/owner<br /> x = also show processes not attached to a terminal<br />
<br />
Use of sed:<br />
<blockquote class="tr_bq">
<i>sed 's/mikesh/mukesh/' test.txt</i></blockquote>
netstat | wc -l<br />
<br />
<code><em></em></code><br /><br /><br /><br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-551042764397805962020-02-19T07:31:00.000-08:002020-04-13T06:44:19.490-07:00DevOps resources <div dir="ltr" style="text-align: left;" trbidi="on">
Some of the DevOps learning link I could find:<br />
<br />
https://github.com/100daysofdevops/100daysofdevops<br />
https://stripe.com/blog/operating-kubernetes<br />
https://blog.scottlowe.org/2016/10/23/managing-aws-infrastructure-ansible/<br />
https://kubedex.com/kubernetes-courses/<br />
https://ramitsurana.gitbooks.io/awesome-kubernetes/content/<br />
https://landing.google.com/sre/sre-book/toc/index.html<br />
https://github.com/dastergon/awesome-sre<br />
<br />
https://www.youtube.com/watch?v=E-WdMk7_IQo <br />
<br />
https://www.youtube.com/watch?v=Z_HmWT-VA5w&feature=youtu.be<br />
<br />
<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-42102939440458446852020-01-15T06:47:00.000-08:002020-04-13T06:47:30.981-07:00Useful terraform commands<div dir="ltr" style="text-align: left;" trbidi="on">
<h4>
Useful terraform Commands</h4>
$ terraform plan # plan<br />
$ terraform apply # shortcut for plan & apply - avoid this in production<br />
$ terraform plan -out out.terraform # terraform plan and write the plan to out file<br />
$ terraform apply out.terraform # apply terraform plan using out file<br />
$ terraform show # show current state<br />
$ cat terraform.tfstate # show state in JSON format<br />
<br />
<br />
What is terraform?<br />
<br />
An IaaC which provides automation of our infrastructure, by ensuring our infrastructure is kept on a state.<br />
Infrastructure change is managed in VCS.<br />
Terraform (HCL hashicorp configuration language)<br />
<br />
<br />
create a seperate terraform work directory:<br />
<br />
mkdir terraform-test<br />
<br />
cd terraform-test<br />
<br />
create a main.tf file:<br />
<br />
vi main.tf<br />
<br />
<br />
----------------------------------------------------------------------------------------------------------------------<br />
<br />
variable "myvar" {<br />
type = "string"<br />
default = "hello world"<br />
<br />
} <br />
<br />
}<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-6202827169308156782020-01-10T06:49:00.000-08:002020-04-13T06:49:55.795-07:00Create ELB and Auto Scaling<div dir="ltr" style="text-align: left;" trbidi="on">
1) create ELB<br />
2) create Launch configuration and AutoScaling <br />
3) Enable ACM in ELB<br />
4) <br />
<br />
Goto EC2 console > ELB > <br />
Give load_balancer_name > give your vpc > open http, port 80 > Choose only the public subnets (do not choose the private ones as it connot access the traffic whenever in private) > create a new security group for ELB, open port 80 only > configure health chack: TCP: port 80> add ec2 instances > add tags> ec2-elb > create<br />
<br />
On the created load balancer > instance (you can see as out of service)<br />
<br />
On EC2 let us install Wordpress application<br />
<br />
Now let's take AMI of EC2 Instance (AMI is input for autoscaling's launch configuration) > create<br />
<br />
Go to certificate manager > register your domain name(for security)<br />
Go to AMIs to check if AMIs are created or not<br />
Go to autoscaling > Launch Configurations (make sure to select same security group)><br />
<b>configure auto-scaling through launch configurations</b>><br />
create autoscaling group > select group size as the minimum amount of ec2s launched> make sure to select right vpc> add multiple subnets where you want these instances to be launched in our case these are public subnets add all those > on advanced settings attach your classic load balancer with the auto-scaling configuration<br />
<br />
on creating ASG I selected '<br />
<div class="CDB" style="width: 670px;">
<div>
<span class="gwt-InlineLabel">Scale between</span> 2 <span class="gwt-InlineLabel">and</span> 4 <span class="gwt-InlineLabel">instances. These will be the minimum and maximum size of your group.</span></div>
</div>
'<br />
you can add notification to the SNS that you have configured> configure and create an auto-scaling group<br />
Once you are done go to ELB > ELB's instances section > look for the status you the instances<br />
One of them is showing OutOfservice, after clicking on the related instance I get the instance status as running.<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-21064174562903598472020-01-08T23:48:00.000-08:002020-04-13T06:49:33.069-07:00Install Wordpress application on EC2<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
wget https://wordpress.org/latest.tar.gz<br />
<br />
<pre>tar -zxf package-name</pre>
<br />
mkdir /var/www/html<br />
cd wordpress<br />
mv . /var/www/html<br />
<br />
<br />
yum install httpd <br />
<br />
cd /etc/httpd/conf<br />
edit httpd.conf<br />
search for "AllowOverride None" change none to All<br />
save and exit<br />
chown -R apache:apache /var/www/html<br />
<br />
<br />
cd /var/www/html directory<br />
<br />
<code>yum update </code><br />
<code>yum install php-fpm php-gd php-pdo php-mbstring php-pear -y<br />
systemctl enable php-fpm<br />
systemctl start php-fpm</code> <br />
<code>service https restart</code><br />
<code>yum install phpphp-mysqlmysql</code><br />
<code><br /></code>
doing this I am getting: <br />
Your server is running PHP version 5.4.16 but WordPress 5.4 requires at least 5.6.20 as I was previously getting this error "<code>ou cannot update because WordPress 5.2 requires PHP version 5.6.20 or higher. You are running version 5.6.3.</code>"<br />
<br />
<br />
so for this I tried this with my aws linux instance:<br />
<pre><code>#!/bin/bash
yum install httpd php-mysql -y
amazon-linux-extras install -y php7.3
cd /var/www/html
wget https://wordpress.org/latest.tar.gz
tar -xzf latest.tar.gz
cp -r wordpress/* /var/www/html/
rm -rf wordpress
rm -rf latest.tar.gz
chmod -R 755 wp-content
chown -R apache:apache wp-content
service httpd start
chkconfig httpd on</code></pre>
<br />
<br />
<br />
<br />
------------------------------------------------------------------------------------------------------------------------<br />
once you get to see WordPress admin page from the Apache server page you are ready to go further:<br />
<br />
<br />
on the UI fill the asked details<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-80830245520864000812020-01-05T06:48:00.000-08:002020-04-13T06:49:12.467-07:00Creating EC2 instance with bootstrap script<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<br />
1) Launch an EC2 instance<br />
2) On advanced details > user data as text, write:<br />
#!/bin/bash<br />
yum install httpd -y<br />
service httpd start<br />
chkconfig httpd on<br />
yum install wget -y<br />
yum install php php-msql mysql -y\<br />
3) Launch the instance<br />
4) Goto actions> Instance settings > View Change User Data<br />
<br />
Creating role for EC2 instance:<br />
<br />
Goto IAM> Roles > Create role > (select ec2 service, ec2 use case)<br />
On permissions > Give S3 full access <span class="ng-scope"><span class="templated-header"><a href="https://console.aws.amazon.com/iam/home?region=us-west-2#/roles/ec2-s3-full-access-project"><b class="ng-binding"></b></a></span></span>> give-a-name <br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-88051577066016828152020-01-04T23:31:00.000-08:002020-04-13T06:48:03.301-07:00RDS<div dir="ltr" style="text-align: left;" trbidi="on">
In this we will create RDS Security Group and RDS subnet group with private subnets<br />
RDS Subnet: Also called custom subnet, database subnet, private subnet<br />
Firstly for RDS you have to create<b> RDS security group </b><br />
<b><br /></b>
<b>In Inbound </b>add mysql <br />
type: mysql,protocol: tcp,port range: 3306, source: custom with value of the security group id of the web servers (so that all the web servers could access the RDS instance)<br />
<b></b><br />
<br />
<b></b>
<b>Parameter Groups:</b> On RDS>click on create parameter groups>select mysql > write a group name > click create (can edit parameter groups for parameters like timezones etc)<br />
You can modify the parameters on the edit parameters option.<br />
<b>Subnet Groups : </b>Goto create DB subnet group > add the public subnets (minimum 2 req from different azs) to the <br />
<b>Create RDS Instance</b><br />
<b>Connect RDS instance with EC2 instance(a public ec2 instance)</b><br />
<b>For connection use</b><br />
$ ssh -i project.pem ec2-user@-public-ip-address<b></b><br />
<b>Setup DB client</b><br />
$ yum -y install mariadb (install MySQL/MariaDB client)<br />
<b>rep1) </b>$ mysql -h <RDS Endpoint> -P 3306 -u <userneme> -p(will prompt)<br />
mine endpoint : project.cuhcuc7a50ji.us-west-2.rds.amazonaws.com<br />
ec2: <span id="detailsPublicIp">34.222.130.122</span><br />
<span id="detailsPublicIp"> mysql -h project.cuhcuc7a50ji.us-west-2.rds.amazonaws.com -P 3306 -u admin -p </span> <b></b><br />
<b>result:</b> not able to connect to my rds instance ,not able to telnet -><br />
added port 3306 on inbound sg of ec2 instance now able to telnet, lets go to rep1 and try connecting... and horray we are connected.<br />
once you are connected let us start playing:<br />
Mysql> show databases;<br />
Mysql> create database project;<br />
Mysql> use project;<br />
Mysql><br />
<pre><span class="pl-k">CREATE</span> <span class="pl-k">TABLE</span> <span class="pl-en">Students</span> ( StudentID <span class="pl-k">int</span>, LastName <span class="pl-k">varchar</span>(<span class="pl-c1">255</span>), FirstName <span class="pl-k">varchar</span>(<span class="pl-c1">255</span>), City <span class="pl-k">varchar</span>(<span class="pl-c1">255</span>) );</pre>
Mysql><br />
<div class="highlight highlight-source-sql">
<pre><span class="pl-k">INSERT INTO</span> Students ( StudentID, LastName, FirstName, City) <span class="pl-k">VALUES</span> ( <span class="pl-s"><span class="pl-pds">"</span>001<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Kumar<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Anil<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Singapore<span class="pl-pds">"</span></span> );
<span class="pl-k">INSERT INTO</span> Students ( StudentID, LastName, FirstName, City) <span class="pl-k">VALUES</span> ( <span class="pl-s"><span class="pl-pds">"</span>002<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Reddy<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>M<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Hyderabad<span class="pl-pds">"</span></span> );
<span class="pl-k">INSERT INTO</span> Students ( StudentID, LastName, FirstName, City) <span class="pl-k">VALUES</span> ( <span class="pl-s"><span class="pl-pds">"</span>003<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Reddy<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>N<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Hyderabad<span class="pl-pds">"</span></span> );
<span class="pl-k">INSERT INTO</span> Students ( StudentID, LastName, FirstName, City) <span class="pl-k">VALUES</span> ( <span class="pl-s"><span class="pl-pds">"</span>004<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Vel<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>D<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Chennai<span class="pl-pds">"</span></span> );
<span class="pl-k">INSERT INTO</span> Students ( StudentID, LastName, FirstName, City) <span class="pl-k">VALUES</span> ( <span class="pl-s"><span class="pl-pds">"</span>005<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Student<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Martian<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>Mars<span class="pl-pds">"</span></span> );</pre>
</div>
Mysql> <br />
<pre><span class="pl-k">select</span> <span class="pl-k">*</span> <span class="pl-k">from</span> Students;</pre>
<br />
Now let us <b>create snapshot for the RDS Instance </b><br />
via the create snapshot button<br />
Create new RDS Instance with snapshot.<br />
https://youtu.be/TdLhNA9cDuc<br />
<br />
<br />
<br />
<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-78673593003400425682020-01-02T06:28:00.000-08:002020-04-13T06:45:33.748-07:00SNS<div dir="ltr" style="text-align: left;" trbidi="on">
Go to service > SNS<br />
<br />
create a topic TOPIC_NAME<br />
<br />
after successfully creating the topic create subscription of the topic<br />
on subscriptions> create subscription >choose your created topic name> protocol: email > Endpoint: enter email id for email type protocol > create subscription<br />
Once the email id has confirmed the subscription you are ready to go!<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-65131228023739408902020-01-01T06:44:00.000-08:002020-04-13T06:45:13.675-07:00Creating a CloudWatch alarm<div dir="ltr" style="text-align: left;" trbidi="on">
We will create a cloudwatch alarm that sends an email using SNS notification when CPU Utilization is more than 70%<br />
<br />
1. Using console<br />
<br /></div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-31055071353009728422018-06-12T03:44:00.001-07:002018-06-12T03:44:10.773-07:00push a branch to your fork<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="tr_bq">
git push origin branch-name</blockquote>
</div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-82069816702861143252018-06-12T00:28:00.000-07:002018-06-12T00:28:04.407-07:00recursively copy folder with files to another folder<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<pre><code>cp -R source destination/</code></pre>
</blockquote>
</blockquote>
</blockquote>
</div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0tag:blogger.com,1999:blog-3267603672490266405.post-78669177082518682242018-06-03T07:43:00.001-07:002018-06-03T07:43:10.657-07:00How to to discard all changed files on git<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
<blockquote class="tr_bq">
Git checkout -- .</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</div>
suramyahttp://www.blogger.com/profile/07903794610681172069noreply@blogger.com0