How to view & interpret disk space usage of your ElasticSearch cluster

In order to find out how much disk space every node of your elasticsearch cluster is using and how much disk space is remaining, use

es_allocation_curl.sh

curl -XGET "http://localhost:9200/_cat/allocation?v&pretty"

curl -XGET "http://localhost:9200/_cat/allocation?v&pretty"

Example output for one node without any data:

example.txt

shards disk.indices disk.used disk.avail disk.total disk.percent host       ip         node
     0           0b     2.4gb    200.9gb    203.3gb            1 172.18.0.2 172.18.0.2 TxYuHLF

shards disk.indices disk.used disk.avail disk.total disk.percent host       ip         node
     0           0b     2.4gb    200.9gb    203.3gb            1 172.18.0.2 172.18.0.2 TxYuHLF

es_allocation_example.txt

This example output tells you:
* `shards: 0` The cluster currently has no shards. This means there is no data in the cluster
* `disk.indices: 0b` The cluster currently uses 0 bytes of disk space for indexes.
* `disk.used: 2.4gb` The disk ElasticSearch will store its data on has 2.4 Gigabytes used spaced. This does not mean that ElasticSearch uses 2.4 Gigabytes, any other application (including the operating system) might also use (part of) that space.
* `disk.avail: 200.9gb` The disk ElasticSearch will store its data on has 200.9 Gigabytes of free space. Remember that this will not shrink only if ElasticSearch is using data on said disk, other applications might also consume some of the disk space depending on how you set up ElasticSearch.
* `disk.total: 203.3gb` The disk ElasticSearch will store its data on has a total size of 203.3 gigabytes (total size as in *available space on the filesystem without any files*)
* `disk.percent: 1` Currently 1 % of the total disk space available (`disk.total`) is used. This value is always rounded to full percents.
* `host, ip, node`: Which node this line is referring to.


Example with one node with some test data (see [this TechOverflow post](/2019/04/06/how-to-insert-test-data-into-elasticsearch-6-x/) on how to generate test data):

```plaintext
shards disk.indices disk.used disk.avail disk.total disk.percent host       ip         node
     5        6.8kb     2.4gb     21.6gb       24gb           10 172.18.0.2 172.18.0.2 J3W5zqj
     5                                                                                 UNASSIGNED

This example output tells you:
* `shards: 0` The cluster currently has no shards. This means there is no data in the cluster
* `disk.indices: 0b` The cluster currently uses 0 bytes of disk space for indexes.
* `disk.used: 2.4gb` The disk ElasticSearch will store its data on has 2.4 Gigabytes used spaced. This does not mean that ElasticSearch uses 2.4 Gigabytes, any other application (including the operating system) might also use (part of) that space.
* `disk.avail: 200.9gb` The disk ElasticSearch will store its data on has 200.9 Gigabytes of free space. Remember that this will not shrink only if ElasticSearch is using data on said disk, other applications might also consume some of the disk space depending on how you set up ElasticSearch.
* `disk.total: 203.3gb` The disk ElasticSearch will store its data on has a total size of 203.3 gigabytes (total size as in *available space on the filesystem without any files*)
* `disk.percent: 1` Currently 1 % of the total disk space available (`disk.total`) is used. This value is always rounded to full percents.
* `host, ip, node`: Which node this line is referring to.


Example with one node with some test data (see [this TechOverflow post](/2019/04/06/how-to-insert-test-data-into-elasticsearch-6-x/) on how to generate test data):

```plaintext
shards disk.indices disk.used disk.avail disk.total disk.percent host       ip         node
     5        6.8kb     2.4gb     21.6gb       24gb           10 172.18.0.2 172.18.0.2 J3W5zqj
     5                                                                                 UNASSIGNED

es_allocation_example_with_data.txt

As we can see, ElasticSearch now has 5 shards. Note that the second line tells us that 5 shards are `UNASSIGNED`. This is because ElasticSearch has been configured to make one replica for each shard and there is no second node where it can put the replica. For development configurations this is usually OK, but production configurations should usually have at least two nodes. See our [docker-compose and systemd service generator](/2019/03/16/elasticsearch-docker-compose-yml-and-systemd-service-generator/) for ElasticSearch for instructions on how to configure a local multi-node cluster using docker.

As we can see, ElasticSearch now has 5 shards. Note that the second line tells us that 5 shards are `UNASSIGNED`. This is because ElasticSearch has been configured to make one replica for each shard and there is no second node where it can put the replica. For development configurations this is usually OK, but production configurations should usually have at least two nodes. See our [docker-compose and systemd service generator](/2019/03/16/elasticsearch-docker-compose-yml-and-systemd-service-generator/) for ElasticSearch for instructions on how to configure a local multi-node cluster using docker.

Check out similar posts by category: ElasticSearch

If this post helped you, please consider buying me a coffee or donating via PayPal to support research & publishing of new posts on TechOverflow

Buy me a coffee