Monitoring Memory Usage: Cloudwatch Strategies And Insights

Monitoring memory usage in AWS CloudWatch is a crucial aspect of managing cloud resources. While the AWS dashboard provides various metrics such as CPU load logs and network latency, it does not offer visibility into EC2 memory usage by default. To address this, users can employ the CloudWatch agent, a background task that collects specified metrics from EC2 instances.

The process involves creating an EC2 role with CloudWatchFullAccess permission and attaching it to the desired EC2 instance. Then, by downloading and installing the CloudWatch agent, users can define the metrics they want to monitor, such as memory usage and disk usage. The agent supports both Windows and Linux systems.

Additionally, users can configure the agent by specifying the metric to be monitored and the monitoring interval in a configuration file. This allows for customisation of the data collection process. With the agent running, users can visualise their EC2 memory usage on the CloudWatch dashboard, providing valuable insights for performance optimisation and resource management.

Installing a CloudWatch agent

To install a CloudWatch agent, you must first ensure that you have an AWS account and access to the EC2 dashboard.

Then, follow these steps:

• Create an IAM role: Create an IAM role with permissions to run the CloudWatch Agent on your EC2 instance.
• Launch an EC2 instance: Launch a new EC2 instance with Amazon Linux 2 as the operating system. Assign the IAM role you created in the previous step.
• Connect to the EC2 instance: Use SSH to connect to your EC2 instance. Open a terminal and run the following command:

`ssh -i ec2-user@`

Replace `` with the path to your private key file and `` with your instance's public DNS.

Download the CloudWatch Agent package: On your EC2 instance, download the CloudWatch Agent package using the following command:

`wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm`

Install the Agent: Install the CloudWatch Agent package using the package manager appropriate for your Linux distribution. For Amazon Linux 2, use the following command:

`sudo rpm -U ./amazon-cloudwatch-agent.rpm`

Configure the Agent: After installation, configure the CloudWatch Agent by running the configuration wizard:

`sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard`

Start the Agent: Once configured, start the CloudWatch Agent service:

`sudo systemctl start amazon-cloudwatch-agent`

Verify Installation: Check the status of the CloudWatch Agent to ensure it's running without errors:

`sudo systemctl status amazon-cloudwatch-agent`

After installing the CloudWatch Agent, you can configure it to collect metrics and logs from your EC2 instance. Use the CloudWatch Agent configuration file (`amazon-cloudwatch-agent.json`) to specify the metrics, logs, and additional configurations.

If you are using Amazon Linux 2023 or Amazon Linux 2 as your operating system, you can install the package by entering the following command:

`sudo yum install amazon-cloudwatch-agent`

You must also ensure that the IAM role attached to the instance has the `CloudWatchAgentServerPolicy` attached.

On all supported operating systems, including Linux and Windows Server, you can download and install the CloudWatch agent using either the command line with an Amazon S3 download link, using Amazon EC2 Systems Manager, or using an AWS CloudFormation template.

Creating an IAM role

To create an IAM role, you need to sign in to the AWS Management Console and open the IAM console. In the navigation pane, choose 'Roles' and then 'Create role'.

Make sure that 'AWS service' is selected under 'Trusted entity type'. For 'Use case', choose 'EC2' under 'Common use cases'.

In the list of policies, select the check box next to 'CloudWatchAgentServerPolicy'. If the agent is sending traces to X-Ray, you also need to give the role the 'AWSXRayDaemonWriteAccess' policy.

Enter a name for the role, such as 'CloudWatchAgentServerRole', and optionally give it a description. Then choose 'Create role'.

If the agent is going to send logs to CloudWatch Logs and you want it to be able to set retention policies for these log groups, you need to add the 'logs:PutRetentionPolicy' permission to the role.

You can also add any additional permissions that you would like your EC2 machine with that role assigned to have. For example, you can allow the EC2 machine to access S3 without entering a password.

Once you are done with the permissions, you can add some tags and proceed to the summary. In the last tab, give your role a name and you can add a description to it.

Configuring the agent

With our agent now installed, let’s configure it. The configuration file specifies two key things:

• The metric to be monitored
• The monitoring interval or frequency

To specify those, create a file at /opt/aws/amazon-cloudwatch-agent/bin/config.json using your preferred method and paste in the following configuration:

Json

{

"metrics": {

"metrics_collected": {

"mem": {

"measurement": [

"mem_used_percent"

],

"metrics_collection_interval": 30

}

}

}

}

If you’re on a Windows server, use this path instead: $Env:ProgramDataAmazonAmazonCloudWatchAgentamazon-cloudwatch-agent.json.)

The file is quite self-explanatory. Here, we’re telling the agent to collect memory usage data in percentage format every 30 seconds. Of course, you can specify whatever interval you prefer.

Now, start the CloudWatch agent with this command:

Bash

Sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

And voila! We’re up and running.

Visualizing Your EC2 Memory Usage

Everything we’ve done so far will be for nothing if we can’t visualize it. So, let’s see what we have to do next. Oh, wait. We have absolutely nothing to do.

Go to the CloudWatch dashboard in your AWS console and click on Metrics. You’ll see a new CWAgent card. After you click on this card, you should see your EC2 memory usage in a graph. Be sure to give it some time for the collection process to start properly.

If, like myself, you’re a big fan of great presentation, try out DataSet. Their integration with AWS EC2 allows for even better organization and presentation of metric graphs all in one place.

Visualising memory usage

If you are a fan of great presentation, you can try out third-party tools like Scalyr or DataSet, which includes an EC2 monitoring widget.

Improving EC2 performance

Monitoring Memory Usage:

• By installing and configuring a CloudWatch agent on EC2 instances, you can gain visibility into memory usage metrics. This is crucial as high memory consumption can lead to application freezes.
• The CloudWatch agent collects specified metrics, such as memory usage data in percentage format, and sends them to CloudWatch for visualization and analysis.
• Visualizing memory usage data can help identify potential issues and optimize performance.

Manually Killing Processes:

If specific processes are consuming excessive memory, manually terminating them can free up resources and prevent application crashes.

Autoscaling:

Implementing autoscaling ensures that redundant servers are automatically started when one fails, maintaining application uptime.

Comprehensive Monitoring:

• In addition to memory usage, monitor other metrics such as CPU load logs, network latency, and request volume.
• Utilize tools like CloudWatch alarms, Amazon EventBridge events, AWS CloudTrail logs, and the CloudWatch agent to collect data from various sources within your AWS solutions.
• This comprehensive monitoring approach facilitates easier debugging in the event of a multi-point failure.

Frequently asked questions