Elastic Search
Elastic Search connections give you access to your company’s indexed documents. Data for a specific defined Type is fetched from all Indices. This integration is best suited for time-series data that is partitioned by immutable Elastic Search Indices.
Source Setup
Step 1. Get the Host Address and Port of the Cluster
Etleap requires the host address and port that the Elastic Search cluster is running behind. SSL can be enabled during connection setup if the cluster is configured for TLS .
Step 2. Authenticate with the Cluster
If the Elastic Search cluster has an authentication service like Kibana configured, a username and password can be supplied during connection creation. If no authentication is required to connect to the cluster, this can be omitted when creating the connection in Etleap.
Password-based user authentication is currently the only authentication method that Etleap supports.
Prefer using an API? Go here and select Elastic Search under the Body header to start creating a connection via API.
What Data is Available?
Etleap extracts documents by their Type.
The file picker will show a list of Types that exist in the cluster, across all Indices.
Extracted data will contain all documents for a specific Type, the name of the Index where a document exists is specified in the _index column of the extracted record.
All Elastic Search pipelines operate in append mode. Each Index is only extracted once. To prevent partial indexes from being extracted, only indexes that were created at least 24 hours before the time of the extraction are accessed. Indexes are accessed in creation order.
Key Considerations
Supported Versions
Etleap only supports Elastic Search sources up to version 6.X.
Version 7 removes Mapping Types, which this integration relies on.
Immutable Indexes
All Elastic Search Indexes are assumed to be immutable 24 hours after the index was created. Therefore, pipelines only read from any given index once. See What Data is Available? for more information.