Active worker nodes usage in HDinsight cluster for the period of last one year

UDAYA SRINIVASARAO KOTHAMASU 1 Reputation point
2022-01-24T12:50:17.057+00:00

Hi Team,

Can someone help us to get the Active worker nodes usage in HDinsight cluster for the period of last one year.

Thanks & Regards,
Udaya

Azure AI Metrics Advisor
Azure AI Metrics Advisor
An Azure artificial intelligence analytics service that proactively monitors metrics and diagnoses issues.
82 questions
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
217 questions
{count} votes

1 answer

Sort by: Most helpful
  1. MartinJaffer-MSFT 26,206 Reputation points
    2022-01-26T01:02:29.063+00:00

    Hello and welcome to Microsoft Q&A @UDAYA SRINIVASARAO KOTHAMASU .

    If I understand correctly, you want to get historical data for active nodes. With the appropriate preparations this would be easy to do.

    Have you configured Azure Monitor to save the logs from your HDInsight cluster, or set up the Diagnostic Settings to save your logs? If so, this should be easy to do. If not, I am uncertain whether the entire year's data can be retrieved. Generally, there is a limit to historical data.

    Some data can be retrieved from the portal without setting up Diagnostic Settings or Azure Monitor:

    Under Monitoring: Metrics, the number of active worker nodes is available. This data extends to the past 30 days.

    When autoscale is enabled, the scale-up, scale-down is recorded in the activity log. This records the number of nodes. The activity logs can be downloaded/exported.
    168542-image.png

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.