Notes:

When the disk passes about 95% full, Elasticsearch flips indices to read-only and Mastodon indexing quietly stops. so make sure you alert at 85% for disk space.
also consider putting /var/lib/elasticsearch on SSD/NVME if you have the option.

Prepare system

echo "vm.max_map_count=262144" | sudo tee /etc/sysctl.d/99-elasticsearch.conf
echo "vm.swappiness=1" | sudo tee /etc/sysctl.d/99-elasticsearch.conf
sudo sysctl --system

Install package

sudo apt-get update
sudo apt-get install -y apt-transport-https gnupg wget # apt-transport-https is usually already present on 26.04

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

gpg --show-keys --with-fingerprint /usr/share/keyrings/elasticsearch-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/9.x/apt stable main" \
| sudo tee /etc/apt/sources.list.d/elastic-9.x.list

sudo apt-get update
sudo apt-get install elasticsearch

Notice output

The generated password for the elastic built-in superuser is : xxxxxxxxxxxxxx

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token '
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

Edit /etc/elasticsearch/jvm.options.d/heap.options

Add about 50% of the memory for the JVM and the other 50% for the Apache Lucene (full text search) and OS cache.
Both values should be the same… last time I checked, having a lower minimum caused a hard error starting ES.

-Xms5g
-Xmx5g

Edit /etc/elasticsearch/elasticsearch.yml

Update or Add the following values

# Important - Comment out the cluster.initial_master_nodes line
# or the sever won't boot in conflict with the single-node discovery.type

cluster.name: es-mastodon
node.name: node-1
bootstrap.memory_lock: true

network.host: [<private-IP>, _local_]
http.port: 9200
discovery.type: single-node

http.host: [<private-IP>, _local_]
transport.host: [<private-IP>, _local_]

xpack.license.self_generated.type: basic
xpack.ml.enabled: false
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.profiling.enabled: false

Consider bootstrap checks (memory lock, file descriptors, max_map_count etc)

Add to /etc/elasticsearch/jvm.options.d/bootstrap.options as these are disabled in prod apparently?

-Des.enforce.bootstrap.checks=true

sudo systemctl edit elasticsearch

Add the following if required. Apparently the new packages might set these by default now.

[Service]
LimitNOFILE=65535
LimitMEMLOCK=infinity
LimitNPROC=4096

Start the service using systemctl

sudo systemctl daemon-reload
sudo systemctl enable --now elasticsearch.service

Test the connecting including the self signed cert

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://<IP>:9200

Create dedicated user for mastodon (least privilege user)

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic \
  -X POST "https://<IP>:9200/_security/role/mastodon_full_access" \
  -H 'Content-Type: application/json' -d'
{ "cluster": ["monitor"],
  "indices": [{ "names": ["*"], "privileges": ["read","monitor","write","manage"] }] }'


# Set password
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic \
  -X POST "https://<IP>:9200/_security/user/mastodon" \
  -H 'Content-Type: application/json' -d'
{ "password": "<long random>", "roles": ["mastodon_full_access"] }'

Update mastodon .env.production

Copy the http_ca.crt from the ES server to all of the mastodon instances

ES_ENABLED=true
ES_HOST=<IP>
ES_PORT=9200
ES_USER=mastodon
ES_PASS=<password>
ES_CA_FILE=/path/to/http_ca.crt
ES_PRESET=single_node_cluster

Categories: Uncategorised

shlee