What is the best way to monitor memory and cpu usage of worker nodes in a cluster

Question

Hi Team,

What is the best way to monitor the CPU and memory of the databricks cluster ? Is it supporting any tools ( paid or free) ?

Answer 1

Answer by rsmith54 · Mar 09, 2018 at 02:28 PM

Hi, was there any answer to this?

Add comment · Share

Answer 2

Answer by jules · Mar 14, 2018 at 01:36 AM

We internally use Prometheus, but you can use Ganglia too and other commercial tools such as DataDog.

We did a meetup on monitoring at Databricks, and the talks are here:

Cheers
Jules

Add comment · Share