Get Graphite run for a Django project

This post is intended first of all to memorize my experience of getting the Graphite system work.

Graphite system

Graphite is designed to install itself in /opt/graphite. There is no way to change it without getting a pain in the ass. I chose to install it in Ubuntu in VM without touching my system. Here we go to install Graphite.

Getting sources

mkdir -p ~/tmp/graphite-install/
cd ~/tmp/graphite-install/

wget http://launchpad.net/graphite/0.9/0.9.9/+download/graphite-web-0.9.9.tar.gz
wget http://launchpad.net/graphite/0.9/0.9.9/+download/carbon-0.9.9.tar.gz
wget http://launchpad.net/graphite/0.9/0.9.9/+download/whisper-0.9.9.tar.gz

tar -zxvf graphite-web-0.9.9.tar.gz
tar -zxvf carbon-0.9.9.tar.gz
tar -zxvf whisper-0.9.9.tar.gz

mv graphite-web-0.9.9 graphite
mv carbon-0.9.9 carbon
mv whisper-0.9.9 whisper

rm carbon-0.9.9.tar.gz
rm graphite-web-0.9.9.tar.gz
rm whisper-0.9.9.tar.gz

Installing dependencies (only packages required to get it working)

sudo apt-get install python-setuptools
sudo easy_install django django_tagging

Install Whisper

cd whisper
sudo python setup.py install

Install Carbon

cd ../carbon
sudo python setup.py install
cd /opt/graphite/conf
sudo cp carbon.conf.example carbon.conf
sudo cp storage-schemas.conf.example storage-schemas.conf
sudo vim storage-schemas.conf
### edited storage-schemas.conf to be the following
[stats]
priority = 110
pattern = .*
retentions = 10:2160,60:10080,600:262974
# note, it does not allow comments in values and causes errors

install Graphite

cd ../graphite
sudo python check-dependencies.py
sudo python setup.py install

# initial database creation
cd /opt/graphite/webapp/graphite/
sudo python manage.py syncdb
### follow prompts to setup django admin user
sudo chown -R <your_user>:>your_user> /opt/graphite/storage/
sudo cp local_settings.py.example local_settings.py

Run

# carbon
cd ../../
sudo ./bin/carbon-cache.py start
## it will listen on 2003 port

# graphite web ui
cd webapp/graphite
python manage.py runserver 0.0.0.0:8000

That's all with Graphite. Now let's organize data sending from your project to the Graphite instance we've made.

Statsd

I launched the daemon locally, not in a virtual machine. To do it follow the steps:

• download (or git clone) sources from https://github.com/etsy/statsd
• install Node.js
• copy exampleConfig.js
• run node stats.js /path/to/config

Project

First, activate a virtual environment (hope you use it)

pip install django-statsd-mozilla statsd

Configure django_statsd client. In settings.py:

STATSD_CLIENT = 'django_statsd.clients.normal'
STATSD_HOST = 'localhost'
STATSD_PORT = 8125
STATSD_PREFIX = 'your_prefix'
# Note, port number must be an integer (not '8125')
# as it causes an error deep inside.

If you're going to use django_statsd's middlewares and/or patches

pip install django-debug-toolbar

Collecting metrics

from django_statsd.clients import statsd

statsd.timing('performance.page.index', 123)  # in milliseconds
statsd.incr('performance.page.index')
statsd.decr('performance.page.index')

P.S. Thanks to Rob Goldings for inspiration.