ElasticSearch SSL

Filebeat 使用默认初始化生成 SSL 证书

ES v8.x 默认初始化会启用 SSL 相关加密特性，配置文件类似以下：

---
#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 14-01-9999 01:58:24
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
# cluster.initial_master_nodes: ["my-hostname"]
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

其中，certs/http.p12 作为 keystore（密钥库），是一个安全的存储容器，一个 PKCS#12 格式的 keystore，它包含：

• 服务器私钥（用于签署数据）
• 服务器 SSL 证书（公钥部分）
• 可信 CA 证书（如果有的话）

查看 HTTP SSL keystore 的 Import Password 密码

bin/elasticsearch-keystore list
# 通过上面命令得到 HTTP SSL keystore key 值为 "xpack.security.http.ssl.keystore.secure_password"

bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password

查看 keystore 内容

openssl pkcs12 -info -in http.p12 -password pass:<密码>

# or
keytool -list -keystore http.p12 -storetype PKCS12

另 PEM pass phrase 默认为 elastic 。

提取私钥 openssl pkcs12 -in http.p12 -nocerts -out http.key -nodes -password pass:<密码>

提取 SSL 证书 openssl pkcs12 -in http.p12 -nokeys -out http.crt -password pass:<密码>

提取 CA 证书（如果存在） openssl pkcs12 -in http.p12 -cacerts -out http_ca.crt -password pass:<密码> -passin pass:elastic -passout pass:elastic

crt 转换 pem openssl x509 -in http_ca.crt -outform PEM -out http_ca.pem

ca.pem http.pem http.key 三个文件通过前面提到步骤生成。

Filebeat 配置文件示例

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: "elastic"
  password: "elastic"
  ssl.verification_mode: "full"

output.elasticsearch.ssl.certificate_authorities: ["path/to/etc/http_ca.pem"]
output.elasticsearch.ssl.certificate: "path/to/etc/http.pem"
output.elasticsearch.ssl.key: "path/to/etc/http.key"

Filebeat 使用 openssl 命令行工具生成自定义 SSL 证书

创建和进入相关目录

cd path/to/ES_HOME
mkdir config/certs
cd config/certs

生成 CA 证书:

openssl req -new -x509 -keyout cacert.key -out cacert.pem \
-days 3650 \
-passin pass:secret -passout pass:secret \
-subj "/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd" \
-addext "keyUsage=critical,digitalSignature,keyCertSign"

参数说明

• 3650 天有效期
• 设置 PEM pass phrase 密码为 secret，不是必须，如果不需设置密码增加参数 -nodes Don't encrypt private keys
• 设置 C/Country Name 为 AU, ST/State or Province Name 为 Some-State, O/Organization Name 为 Internet Widgits Pty Ltd
• 设置 key usage extension 为 critical,digitalSignature,keyCertSign , 使用 Python 3.13 版本和客户端库访问 ES 时必须设置

查看证书内容 openssl x509 -noout -text -in cacert.pem

生成客户端证书

openssl genrsa -out client2.key 2048

# 设置 FQDN 为 localhost
openssl req -new -key client2.key -out client2.csr -subj "/CN=localhost"

openssl x509 -req -in client2.csr -CA cacert.pem -CAkey cacert.key -CAcreateserial -out client2.crt -days 3650 -sha256 -passin pass:secret

打包为 JKS 格式

openssl pkcs12 -export -out bundle.p12 -in client2.crt -inkey client2.key -password pass:elastic

keytool -keystore truststore.jks -import -file cacert.pem -alias cacert -storepass elastic -noprompt

keytool -destkeystore keystore.jks -importkeystore -srckeystore bundle.p12 -srcstoretype PKCS12 -srcstorepass elastic -deststorepass elastic -noprompt

cURL 测试使用证书访问 curl -u elastic:elastic --key client2.key --cert client2.crt --cacert cacert.pem https://localhost:9200

Filebeat 配置文件示例

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: "elastic"
  password: "elastic"
  ssl.verification_mode: "full"

output.elasticsearch.ssl.certificate_authorities: ["path/to/etc/cacert.pem"]
output.elasticsearch.ssl.certificate: "path/to/etc/client2.crt"
output.elasticsearch.ssl.key: "path/to/etc/client2.key"

参考

• https://docs.confluent.io/kafka-connectors/elasticsearch/current/security.html
• https://github.com/canonical/microk8s/issues/4864

Python demo

依赖

• ElasticSearch v8.18.0 or newer
• Python elasticsearch package pip install elasticsearch==8.18

ES 配置参考

xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.client_authentication: required
xpack.security.http.ssl.key: certs/client2.key
xpack.security.http.ssl.certificate: certs/client2.crt
xpack.security.http.ssl.certificate_authorities: ["certs/cacert.pem"]

xpack.security.transport.ssl.enabled: false

Python 示例代码

import os.path
import elasticsearch

ca_certs = os.path.join(os.getenv("ES_HOME"), "config/certs/cacert.pem")
client_cert = os.path.join(os.getenv("ES_HOME"), "config/certs/client2.crt")
client_key = os.path.join(os.getenv("ES_HOME"), "config/certs/client2.key")

client = elasticsearch.Elasticsearch(
    "https://localhost:9200",
    basic_auth=("elastic", "elastic"),

    # set option `verify_certs=True` requires ElasticSearch v8.18.0 or newer.
    # see also https://github.com/elastic/elasticsearch/issues/117769
    verify_certs=True,
    ca_certs=ca_certs,

    # OPTIONAL: Client TLS certificate authentication
    client_cert=client_cert,
    client_key=client_key,
)

print(client.info())

See also

• https://github.com/elastic/elasticsearch/issues/117769
• https://www.elastic.co/guide/en/elasticsearch/reference/8.18/release-notes-8.18.0.html
• https://www.elastic.co/docs/reference/elasticsearch/clients/python

CRT 和 PEM 对比

文件类型	特点	常见用途	应用示例
.crt	主要存储 X.509 SSL 证书，通常是 DER 或 PEM 格式	服务器证书（如 server.crt），用于 HTTPS	ElasticSearch
.pem	采用 Base64 ASCII 编码，可以存储 证书、密钥或 CA 证书	私钥 (key.pem)、证书 (cert.pem)、CA 证书 (ca.pem)	Filebeat

CRT 和 PEM 两者都用于存储 SSL/TLS 证书。都可以包含 公钥、私钥 或 CA 证书。在大多数情况下，它们可以互相转换，并用于相同的用途（如服务器认证）。

参考

• https://www.elastic.co/docs/deploy-manage/security/different-ca

参考

• https://www.elastic.co/docs/reference/beats/filebeat/configuration-ssl#ssl-common-config
• https://support.azure.cn/docs/azure-operations-guide/application-gateway/aog-application-gateway-howto-create-self-signed-cert-via-openssl.html