How to remotely manage an Amazon EC2 instance using Systems Manager Run Command by Raj Gupta

Step 1:-

Attach an IAM role with the AmazonEC2RoleforSSM managed policy to an Amazon EC2 instance.

Step 2 :-

If you are using IAM user account then Grant Your User Account Access to Systems Manager by attaching Policy AmazonSSMFullAccess to user account.

Step 3:-

Install the SSM Agent

The agent is installed by default on Windows AMIs starting in November 2016 and later, Amazon Linux AMIs starting with 2017.09, and all Amazon Linux 2 AMIs

Step 4:-

To execute a command using Run Command from the console

 EC2 console --->> Run Command Tab

For Command document, choose AWS-RunPowerShellScript for Windows instances, and AWSRunShellScript for Linux instances.

Name : AWS-RunShellScript

For Target instances, choose the instance you created

For Commands, type Get-Service for Windows, or ps aux for Linux

Choose Run to execute the command. Run Command displays a status screen. Choose View result

How To Add Volumes to an AMI of EC2 Server By Raj Gupta

Open the Amazon EC2 console --->> In the navigation pane, choose Instances --->> Select an instance and choose Actions, Image, Create Image

In the Create Image dialog box, choose Add New Volume -->> Select a volume type from the Type list and a device name from the Device list. For an EBS volume, you can optionally specify a snapshot, volume size, and volume type --->> Choose Create Image.

Viewing the EBS Volumes in an AMI Block Device Mapping

Select the desired AMI, and look at the Details tab. At a minimum, the following information is available for the root device:

• Root Device Type (ebs) 
• Root Device Name (for example, /dev/sda1) 
• Block Devices (for example, /dev/sda1=snap-1234567890abcdef0:8:true)

If the AMI was created with additional EBS volumes using a block device mapping, the Block Devices field displays the mapping for those additional volumes as well

How do I resolve the error when trying to release an Elastic IP address from my Amazon EC2 instance by Raj Gupta

Issue:-

when we are getting error "The address with allocation id cannot be released because it is locked to your account" when trying to release an Elastic IP address from my Amazon EC2 instance?

Root Cause:-  

This error message is generated when a reverse Domain Name System (rDNS) record is created for your Elastic IP address. The Elastic IP address is locked to your account for as long as the rDNS record exists.

Reverse DNS (rDNS) is a method of resolving an IP address into a domain name, just as the domain name system (DNS) resolves domain names into associated IP addresses.


Solution:- 

For this we need to Request removal of the rDNS entry from AWS  by creating ticket.

Open the link Request to Remove Email Sending Limitations form. 

Then fill the below details:- 

• Email Address: Your email address
• Use Case Description: Leave blank
• Elastic IP Address 1: The first IP address with an rDNS
• Elastic IP Address 2: Leave blank
• Reverse DNS Record for EIP 1: Enter please remove rDNS
• Reverse DNS Record for EIP 2: Leave blank

Choose Submit.

Note: Removing the rDNS might take a few days to propagate through the system.

After you receive notice that the rDNS is removed then you are able to remove the Elastic IP from EC2 Server.

We can also able to check rDNS is removed or not by running the below command

If it is removed then it will give output like below
[root@ip-172-31-81-127 ~]# host 3.212.44.7

7.44.212.3.in-addr.arpa domain name pointer ec2-3-212-44-7.compute-1.amazonaws.com.

If it is not removed then it will give output like below
[root@ip-172-31-81-127 ~]# host 3.212.44.7
3.212.44.7.in-addr.arpa. domain-name-pointer mail.domain.com

What are all Tag Restrictions for AWS services by Raj Gupta

The following basic restrictions apply to tags for any AWS services:

• Maximum number of tags per resource – 50
• For each resource, each tag key must be unique, and each tag key can have only one value.
• Maximum key length – 128 Unicode characters in UTF-8
• Maximum value length – 256 Unicode characters in UTF-8
• Although EC2 allows for any character in its tags, other services may be more restrictive. Generally allowed characters are: letters, numbers, and spaces re-presentable in UTF-8, and the following characters: + - = . _ : / @. These characters may not be allowed by more restrictive services.
• Tag keys and values are case-sensitive.
• Don't use the aws: prefix for either keys or values; it's reserved for AWS use. You can't edit or delete tag keys or values with this prefix. Tags with this prefix do not count against your tags per resource limit.
• You can tag public or shared resources, but the tags you assign are available only to your AWS account and not to the other accounts sharing the resource.
• You can't tag all resources.

How to Work with Tags Using the AWS Console by Raj Gupta

1. Displaying Tags for Individual Resources

• On the Tags tab, select Show Column. A new column is added to the console.

2. Displaying Tags for All Resources

EC2 console  --> Tags Tab

3. To add a tag to an individual resource


EC2 console  -->Select EC2 server --> Tags Tab  ---> Add/Edit Tags ---> Create Tag ---> Enter the Key and Value ---> Save

4. To delete a tag from an individual resource

EC2 console  -->Select EC2 server --> Tags Tab  ---> Add/Edit Tags ---> Select the Delete icon  ---> Save

5. To add a tag to a group of resources

EC2 console --> Tags tab ---> Manage Tags ---> Form Filter, select the type of resource to which to add tags   ---> In the resources list, select the check box next to each resource to which to add tags ---> Under Add Tag, for Key and Value, type the tag key and values, and then choose Add Tag.

Note:- If you add a new tag with the same tag key as an existing tag, the new tag overwrites the existing tag.

6. To remove a tag from a group of resources

EC2 console --> Tags tab ---> Manage Tags ---> Form Filter, select the type of resource to which to remove tags   ---> In the resources list, select the check box next to each resource to which to remove tags ---> Under Remove Tag, for Key, type the tag's name and choose Remove Tag

7. Adding a Tag When You Launch an Instance

On the Add Tags page, specify tags for the instance, the volumes, or both. Choose Add another tag to add more than one tag to your instance.

8. To filter a list of resources by tag

Choose the filter icon in the top right corner of the column for the tag to display the filter list.Select the tag values, and then choose Apply Filter to filter the results list.

How to work with Tags Using the AWS CLI by Raj Gupta

1. The following command describes the instances with a Env tag, regardless of the value of the tag

[root@ip-172-31-89-253 ~]# aws ec2 describe-instances --filters Name=tag-key,Values=Env

2. The following command describes the instances with the tag Env=Production.

[root@ip-172-31-89-253 ~]# aws ec2 describe-instances --filters Name=tag:Env,Values=Production

3. The following command describes the instances with a tag with the value production, regardless of the tag key

[root@ip-172-31-89-253 ~]# aws ec2 describe-instances --filters Name=tag-value,Values=Production

4. Add a tag to a resource

[root@ip-172-31-89-253 ~]# aws ec2 create-tags --resources i-06c71b2bc3588adb8 --tags Key=Raj,Value=Gupta

5. Add tags with special characters

This example adds the tag [Group]=test to an instance. The square brackets ([ and ]) are special characters, and must be escaped with enclose the entire key and value structure with single quotes ('), and then enclose the element with the special character with double quotes (")..

[root@ip-172-31-89-253 ~]# aws ec2 create-tags --resources i-06c71b2bc3588adb8 --tags 'Key="[Group]",Value=test'

6. Add tags to multiple resources

[root@ip-172-31-89-253 ~]# aws ec2 create-tags --resources i-070e26baf98bd7575 i-06c71b2bc3588adb8 --tags Key=Raj1,Value=Gupta1 Key=Raj2,Value=Gupta2

7. Create a volume and apply a tag

The following command creates a volume and applies two tags: purpose = production, and costcenter = cc123.

aws ec2 create-volume --availability-zone us-east-1a --volume-type gp2 --size 80 --tag-specifications 'ResourceType=volume,Tags=[{Key=purpose,Value=production},{Key=costcenter,Value=cc123}]'

8. Launch an instance and apply tags to the instance and volume

aws ec2 run-instances --image-id ami-abc12345 --count 1 --instancetype t2.micro --key-name MyKeyPair --subnet-id subnet-6e7f829e --tagspecifications 'ResourceType=instance,Tags=[{Key=webserver,Value=production}]' 'ResourceType=volume,Tags=[{Key=cost-center,Value=cc123}]' 

How to use Amazon EC2 Usage Reports by Raj Gupta

If you want to know the answer for below question regarding about your EC2 server

• How much am I spending on instances of each instance type?
• How many instance hours are being used by a particular department?
• How is my instance usage distributed across Availability Zones?
• How is my instance usage distributed across AWS accounts?
• How well am I using my Reserved Instances?
• Are my Reserved Instances helping me save money?


Then you need to use Cost Explorer serveses of AWS

EC2 console ---> Reports Tab ---> Then click on  EC2 Instance Usage Report or EC2 Reserved Instance Utilization Report as per your requirement  ---> Then it will redirect to Cost Explorer Tab


Cost Explorer provides a preconfigured view, based on fixed filter settings, that displays information about your usage and cost trends.

How to View Your Current Limits of AWS services by Raj Gupta

To view Your Current Limits of AWS services on a per-region basis.

 EC2 console ---> Limit Tab

How to reslove Could not load /lib/modules" or "BusyBox" (Missing kernel modules) of EC2 server

If our EC2 server are giving the below error

FATAL: Could not load /lib/modules/2

Root Cause:-

• Missing ramdisk
• Missing correct modules from ramdisk
• Amazon EBS root volume not correctly attached as /dev/sda1

Solution:- 

A.  If server is Amazon EBS-backed:-

Use the following procedure: 

1. Select corrected ramdisk for the Amazon EBS volume. 

2. Stop the instance. 

3. Detach the volume and repair it. 

4. Attach the volume to the instance. 

5. Start the instance. 

6. Modify the AMI to use the corrected ramdisk.

B. If server is Instance store-backed:-

Use the following procedure: 

1. Terminate the instance and launch a new instance with the correct ramdisk. 

2. Create a new AMI with the correct ramdisk.

How to resolve FATAL: kernel too old of EC2 server by Raj Gupta

If our EC2 server are giving below error

FATAL: kernel too old

Root Caused:-

Incompatible kernel and userland

Solution:- 

A.  If server is Amazon EBS-backed:-

Use the following procedure: 

1. Stop the instance. 

2. Modify the configuration to use a newer kernel. 

3. Start the instance.

B. If server is Instance store-backed:-

Use the following procedure: 

1. Create an AMI that uses a newer kernel. 

2. Terminate the instance. 

3. Start a new instance from the AMI you created. 

How to resolve ERROR Invalid kernel (EC2 incompatible kernel) by Raj Gupta

If our EC2 server are giving below error

ERROR Invalid kernel: elf_xen_note_check: ERROR: Will only load images built for the generic loader or Linux images xc_dom_parse_image returned -1 


Root Cause:-

• Supplied kernel is not supported by GRUB
• Fallback kernel does not exist

Solution:- 

A.  If server is Amazon EBS-backed:-

Use the following procedure: 

1. Stop the instance. 

2. Replace with working kernel. 

3. Install a fallback kernel. 

4. Modify the AMI by correcting the kernel.

B. If server is Instance store-backed:-

Use the following procedure: 

1. Terminate the instance and launch a new instance with the correct kernel. 

2. Create an AMI with the correct kernel.

How to resolve Looping legacy kernel modprobe on older Linux versions of EC2 sever by Raj Gupta

If our EC2 server are giving the below error

request_module: runaway loop modprobe binfmt-464c

Root Cause:-

Using an unstable or old Linux kernel (for example, 2.6.16-xenU) can cause an interminable loop condition at startup.

Solution:- 

A.  If server is Amazon EBS-backed:-

Use a newer kernel, either GRUB-based or static, using one of the following options: 

Option 1: 

Terminate the instance and launch a new instance, specifying the –kernel and – ramdisk parameters. 

Option 2: 

1. Stop the instance. 

2. Modify the kernel and ramdisk attributes to use a newer kernel. 

3. Start the instance.

B. If server is Instance store-backed:-

Terminate the instance and launch a new instance, specifying the –kernel and –ramdisk parameters.

How to resolve fsck: No such file or directory while trying to open issue of EC2 server by Raj Gupta

If our EC2 server are giving below error:-

fsck.ext3: No such file or directory while trying to open /dev/sdh


Root caused:-

• A bug exists in ramdisk filesystem definitions /etc/fstab
• Misconfigured filesystem definitions in /etc/fstab
• Missing/failed drive

Solution:- 

A.  If server is Amazon EBS-backed:-

Use the following procedure: 

1. Stop the instance, detach the root volume, repair/modify /etc/fstab the volume, attach the volume to the instance, and start the instance. 

2. Fix ramdisk to include modified /etc/fstab (if applicable). 

3. Modify the AMI to use a newer ramdisk. 

The sixth field in the fstab defines availability requirements of the mount – a nonzero value implies that an fsck will be done on that volume and must succeed. Using this field can be problematic in Amazon EC2 because a failure typically results in an interactive console prompt that is not currently available in Amazon EC2. Use care with this feature and read the Linux man page for fstab. 

B. If server is Instance store-backed:-

Use the following procedure: 

1. Terminate the instance and launch a new instance. 

2. Detach any errant Amazon EBS volumes and the reboot instance.

How to resolve General error mounting filesystems (Failed mount) issue of EC2 by Raj Gupta

If our EC2 server are giving error like below :-

General error mounting filesystems. 

Root caused:-

Amazon EBS-backed

• Detached or failed Amazon EBS volume.
• Corrupted filesystem.
• Mismatched ramdisk and AMI combination (such as Debian ramdisk with a SUSE AMI).

Instance store-backed:-

• A failed drive.
• A corrupted file system.
• A mismatched ramdisk and combination (for example, a Debian ramdisk with a SUSE AMI).

Solution:- 

A.  If server is Amazon EBS-backed:-

Use the following procedure: 

1. Stop the instance. 

2. Detach the root volume. 

3. Attach the root volume to a known working instance.

4. Run filesystem check (fsck –a /dev/...). 

5. Fix any errors. 

6. Detach the volume from the known working instance. 

7. Attach the volume to the stopped instance. 

8. Start the instance. 

9. Recheck the instance status. 

B. If server is Instance store-backed:-

• Start a new instance.

How to resolve Unable to mount root fs on unknown-block issue of EC2 server by Raj Gupta

If our EC2 server are giving the below error:-

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,1)


Root caused:-

Amazon EBS-backed

• Device not attached correctly.
• Root device not attached at correct device point.
• Filesystem not in expected format.
• Use of legacy kernel (such as 2.6.16-XenU).
• A recent kernel update on your instance (faulty update, or an update bug)


Instance store-backed

• Hardware device failure.

Solution:- 

A.  If server is Amazon EBS-backed:-

• Stop and then restart the instance. 

• Modify root volume to attach at the correct device point, possible /dev/sda1 instead of / dev/sda. 

• Stop and modify to use modern kernel. 

• Refer to the documentation for your Linux distribution to check for known update bugs. Change or reinstall the kernel. 

B. If server is Instance store-backed:-

Terminate the instance and launch a new instance using a modern kernel. 

How to resolve Error: Unable to determine major/minor number of root device... (Root file system/device mismatch) Error of EC2 server by Raj Gupta

If our EC2 server are giving the below error:-

ERROR: Unable to determine major/minor number of root device '/dev/xvda1'.


Root caused:-

• Missing or incorrectly configured virtual block device driver
• Device enumeration clash (sda versus xvda or sda instead of sda1)
• Incorrect choice of instance kernel

Solution:- 

A.  If server is Amazon EBS-backed:-

Use the following procedure: 

1. Stop the instance. 

2. Detach the volume. 

3. Fix the device mapping problem. 

4. Start the instance. 

5. Modify the AMI to address device mapping issues.

B. If server is Instance store-backed:-

Use the following procedure: 

1. Create a new AMI with the appropriate fix (map block device correctly). 

2. Terminate the instance and launch a new instance from the AMI you created.

How to resolve the XENBUS: Device with no driver error of EC2 server by Raj Gupta

If we get the error like below by EC2 server

Waiting 10 seconds for device /dev/xvda1 ... Root device '/dev/xvda1' doesn't exist. Attempting to create it.
ERROR: Unable to determine major/minor number of root device '/dev/xvda1'.
You are being dropped to a recovery shell Type 'exit' to try and continue booting sh: can't access tty; job control turned off [ramfs /]#


Root caused :-

• Missing or incorrectly configured virtual block device driver
• Device enumeration clash (sda versus xvda)
• Incorrect choice of instance kernel


Solution:-

A.  If server is Amazon EBS-backed:-

Use the following procedure:

1. Stop the instance.
2. Detach the volume.
3. Fix the device mapping problem.
4. Start the instance.
5. Modify the AMI to address device mapping issues.

B. If server is Instance store-backed:-

Use the following procedure:

1. Create an AMI with the appropriate fix (map block device correctly).
2. Terminate the instance and launch a new instance using the AMI you created.

How to resolve days without being checked, check forced error of EC2 server by Raj Gupta

If we are getting below error by EC2 server

... Checking filesystems Checking all file systems. [/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/sda1 /dev/sda1 has gone 361
days without being checked, check forced

Root Caused:-

Filesystem check time passed; a filesystem check is being forced.

Solution:- 

• Wait until the filesystem check completes. A filesystem check can take a long time depending on the size of the root filesystem.
• Modify your filesystems to remove the filesystem check (fsck) enforcement using tune2fs or tools appropriate for your filesystem. 

How to resolve fsck died with exit status error of EC2 by Raj Gupta

If our EC2 server are giving error

Cleaning up ifupdown.... Loading kernel modules...done. ... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.16.2 /sbin/fsck.xfs: /dev/sdh does not exist
fsck died with exit status 8
[31mfailed (code 8).[39;49m


Root caused:-

• Ramdisk looking for missing drive 
• Filesystem consistency check forced 
• Drive failed or detached 

Solution:- 

A.  If server is Amazon EBS-backed:-

Try one or more of the following to resolve the issue: 

• Stop the instance, attach the volume to an existing running instance. 

• Manually run consistency checks. 

• Fix ramdisk to include relevant utilities. 

• Modify filesystem tuning parameters to remove consistency requirements (not recommended).

B. If server is Instance store-backed:-

Try one or more of the following to resolve the issue: 

• Rebundle ramdisk with correct tooling. 

• Modify file system tuning parameters to remove consistency requirements (not recommended). 

• Terminate the instance and launch a new instance. 

• (Optional) Seek technical assistance for data recovery using AWS Support. 

How to reslove GRUB prompt error of EC2 server by Raj Gupat

If our EC2 server giving the below error:-

 GNU GRUB version 0.97 (629760K lower / 0K upper memory) [ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename. ]
grubdom> 

Root Caused:-

If server is Amazon EBS-backed

• Missing GRUB configuration file.
• Incorrect GRUB image used, expecting GRUB configuration file at a different location.
• Unsupported filesystem used to store your GRUB configuration file (for example, converting your root file system to a type that is not supported by an earlier version of GRUB).

If server is Instance store-backed

• Missing GRUB configuration file.
• Incorrect GRUB image used, expecting GRUB configuration file at a different location.
• Unsupported filesystem used to store your GRUB configuration file (for example, converting your root file system to a type that is not supported by an earlier version of GRUB).

Solution:-

A.  If server is Amazon EBS-backed:-

Option 1: Modify the AMI and relaunch the instance: 

1. Modify the source AMI to create a GRUB configuration file at the standard location (/ boot/grub/menu.lst). 

2. Verify that your version of GRUB supports the underlying file system type and upgrade GRUB if necessary. 

3. Pick the appropriate GRUB image, (hd0-1st drive or hd00 – 1st drive, 1st partition). 

4. Terminate the instance and launch a new one using the AMI that you created.

Option 2: Fix the existing instance: 

1. Stop the instance. 

2. Detach the root filesystem. 

3. Attach the root filesystem to a known working instance. 

4. Mount filesystem. 

5. Create a GRUB configuration file. 

6. Verify that your version of GRUB supports the underlying file system type and upgrade GRUB if necessary. 

7. Detach filesystem. 

8. Attach to the original instance. 

9. Modify kernel attribute to use the appropriate GRUB image (1st disk or 1st partition on 1st disk). 10.Start the instance

B. If server is Instance store-backed:-

Option 1: Modify the AMI and relaunch the instance:

1. Create the new AMI with a GRUB configuration file at the standard location (/boot/grub/ menu.lst). 

2. Pick the appropriate GRUB image, (hd0-1st drive or hd00 – 1st drive, 1st partition). 

3. Verify that your version of GRUB supports the underlying file system type and upgrade GRUB if necessary. 

4. Terminate the instance and launch a new instance using the AMI you created.  

Option 2: Terminate the instance and launch a new instance, specifying the correct kernel.

Note :- To recover data from the existing instance, contact AWS Support.