Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/index.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
[[elastic-stack]]
= Elastic Stack

:version: 5.0.0
:major-version: 5.x
:branch: 5.0
:beatsref: https://www.elastic.co/guide/en/beats/libbeat/5.0
:cloudref: https://www.elastic.co/guide/en/cloud/current
:esref: https://www.elastic.co/guide/en/elasticsearch/reference/5.0
:hadoopref: https://www.elastic.co/guide/en/elasticsearch/hadoop/5.0
:kibanaref: https://www.elastic.co/guide/en/kibana/5.0
:logstashref: https://www.elastic.co/guide/en/logstash/5.0
:xpackref: https://www.elastic.co/guide/en/x-pack/5.0

include::installing-stack.asciidoc[]

include::upgrading-stack.asciidoc[]

The Elastic Stack is made up of multiple components developed by Elastic. In no particular
order, these are:

* {beatsref}/index.html[Beats {branch}]
* {esref}/index.html[Elasticsearch {branch}]
* {hadoopref}/index.html[Elasticsearch Hadoop {branch}]
* {kibanaref}/index.html[Kibana {branch}]
* {logstashref}/index.html[Logstash {branch}]
* {xpackref}/index.html[X-Pack {branch}]

76 changes: 76 additions & 0 deletions docs/installing-stack.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
[[installing-elastic-stack]]
== Installing the Elastic Stack

This guide is intended for users seeking to install all or part of the Elastic Stack as part of
clean or fresh installation. If you are upgrading, then please refer to <<upgrading-elastic-stack>>.

Beginning with 5.0, installing the Elastic Stack and understanding the compatibility between products
has been dramatically simplified. All products are now part of a unified release process, so when
any product is released, then all products are released with it. This helps to guarantee compatibility as
well as avoid confusion surrounding version numbers.

As a result, when performing any Elastic Stack installation, you should always start with the same
version across the entire stack. For example, if you select Elasticsearch 5.1.2, then you should also
select Kibana 5.1.2, Logstash 5.1.2, and Beats 5.1.2.

[cols="2", options="header"]
|===
|Component |Version
|Beats
|{branch}
|Elasticsearch
|{branch}
|Elasticsearch Hadoop
|{branch}
|Kibana
|{branch}
|Logstash
|{branch}
|X-Pack
|{branch}
|===

[[install-order-elastic-stack]]
=== Installation Order

We recommend that you install the Elastic Stack in the following order.

1. Elasticsearch
* X-Pack for Elasticsearch
2. Kibana
* X-Pack for Kibana
3. Logstash
4. Beats
5. Elasticsearch Hadoop

This helps to ensure that the right parts of your infrastructure are running before other parts
attempt to use them (e.g., Logstash sending data to Elasticsearch).

The table below lists the installation instructions, per component:

[cols="2", options="header"]
|===
|Product |Installation
|Elasticsearch
|{esref}/install-elasticsearch.html[Installing Elasticsearch 5.0]
|X-Pack
|{xpackref}/installing-xpack[Installing X-Pack 5.0]
|Kibana
|{kibanaref}/setup.html[Installing Kibana 5.0]
|Logstash
|{logstashref}/installing-logstash.html[Installing Logstash 5.0]
|Beats
|{beatsref}/installing-beats.html[Installing Beats 5.0]
|Elasticsearch Hadoop
|{hadoopref}/float.html[Installing Elasticsearch Hadoop 5.0]
|===

[[install-elastic-stack-for-elastic-cloud]]
=== Installing on the Elastic Cloud

The Elastic Cloud supports installing Elasticsearch 5.0, Kibana 5.0, and X-Pack 5.0. Unlike the
above instructions, the Elastic Cloud handles the actual installation process as part of its
managed service.

If you would like to trial Elasticsearch 5.0 using a fresh installation, then you can easily
do so by {cloudref}/getting-started.html[running Elasticsearch in the Elastic Cloud].
135 changes: 135 additions & 0 deletions docs/upgrading-stack.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
[[upgrading-elastic-stack]]
== Upgrading the Elastic Stack

Before upgrading any component of the Elastic Stack, you should read through this guide to ensure
that you upgrade in the right order and in the right way. When upgrading any component of the
Elastic Stack, such as Beats, you should refer to the instructions for that component, including
the breaking changes section.

Each component serves a particular role in the Elastic Stack. For some use cases, it is normal to
not use every component in the Elastic Stack, for instance only using Elasticsearch, Logstash, and
Kibana; Elasticsearch and Kibana; Elasticsearch and Beats; or just Elasticsearch.

[[upgrading-elastic-stack-audience]]
=== Intended Audience

This guide is intended for existing users of the Elastic Stack, running specific version ranges of
each component:

[cols="2", options="header"]
|===
|Component |Version
|Beats
|1.0 or later
|Elasticsearch
|2.0 or later
|Kibana
|4.2 or later
|Logstash
|2.0 or later
|Hadoop
|2.2 or later
|Marvel, Shield, Watcher, Graph, Reporting <1>
|2.0 or later
|===
1. Marvel, Shield, Watcher, Graph, and Reporting have all been combined into a new, unified plugin called
X-Pack. Unlike before, the same X-Pack distribution works for both Elasticsearch and Kibana.

NOTE: Kibana 4.2 and Elasticsearch Hadoop 2.2 were the first versions that were compatible with
Elasticsearch 2.x!

The spread of version numbers in the above table is the reason that the Elastic Stack is moving to
a unified release number: 5.0. Beginning with 5.0, all of the components above will be released at the
same time with the same version number. As such, you can choose a version with confidence across the
entire stack.

It is critical to note that you cannot upgrade data that was written using Elasticsearch 1.x to
Elasticsearch 2.x, then upgrade directly to Elasticsearch 5.x. Elasticsearch uses Lucene to store its
data and Lucene is only compatible with the current version of Lucene, and one major release behind
it. The Elasticsearch upgrade instructions do cover that path.

WARNING: This includes system indices, such as the `.kibana` index that is created by Kibana, and
{esref}/modules-snapshots.html#modules-snapshots[snapshots of indices from Elasticsearch 1.x]!

[cols="3", options="header"]
|===
|Elasticsearch Version |Lucene Version| Path
|5.x |6.x | {esref}/rolling-upgrades.html[Rolling Upgrade]
|2.x |5.x | {esref}/restart-upgrade.html[Full Cluster Restart Upgrade] <1>
|1.x |4.x | {esref}/reindex-upgrade.html[Reindexing Required]
|===
1. Some features require reindexing to take advantage of them, such as Lucene's new Block KD tree support.

When upgrading from two major releases ago, it is important to read the breaking changes from
1.x to 2.x, as well as from 2.x to 5.x!

[[upgrade-order-elastic-stack]]
=== Upgrade Order

As noted above, this only applies for users of Beats 1.x, Elasticsearch 2.x, Logstash 2.x, and
Kibana 4.2+. For Elasticsearch in particular, it is critical to run the
https://github.com/elastic/elasticsearch-migration/[Elasticsearch Migration Plugin] prior to
any upgrade to verify upgrade compatibility!

To maintain the most compatibility, you must upgrade the stack in the recommended order. You
may skip any components that you do not use in your own system. The upgrade requires a full
cluster shutdown for both Elasicsearch and Kibana because neither Elasticsearch 5.0, nor Kibana 5.0,
can communicate with earlier versions of Elasticsearch.

1. Elasticsearch Hadoop (can talk to Elasticsearch 5.0 and 2.x)
2. Elasticsearch
* X-Pack for Elasticsearch (combines Marvel Agent, Shield, Watcher, and Graph)
3. Kibana (now includes Timelion and Console, formerly known as Sense)
* X-Pack for Kibana (combines Marvel, Shield, Graph, and Reporting)
4. Logstash
5. Beats

Elasticsearch Hadoop versions prior to 5.0 are not compatible with Elasticsearch 5.0, but Elasticsearch
Hadoop 5.0 is compatible with Elasticsearch 2.0 and Elasticsearch 5.0.

Logstash 2.0+ and Beats 1.0+ are compatible with both Elasticsearch 2.0+ and Elasticsearch 5.0. This
provides flexibility in when you schedule the upgrades for each Logstash instance and Beats agent.

By following the above order, you should upgrade Elasticsearch Hadoop first; then Elasticsearch
by performing a full cluster restart and upgrade; then install X-Pack; then upgrade Kibana immediately
afterward by restarting and upgrading all instances of Kibana; then installing X-Pack there as well.
Afterward, you can choose when it makes the most sense to upgrade Logstash and Beats for your architecture. It is worth
upgrading Logstash and Beats as soon as possible to take advantage of performance improvements
and other enhancements.

The following table lists the upgrade instructions and breaking changes for each component. Before
upgrading, make sure you read through the upgrade guide and breaking changes list for every component
that you are upgrading.

[cols="3", options="header"]
|===
|component |Breaking Changes |Upgrade
|Elasticsearch
|{esref}/breaking-changes.html[Breaking Changes from 2.x]
|{esref}/setup-upgrade.html[Upgrading to Elasticsearch 5.0]
|X-Pack
|{xpackref}/breaking-changes.html[Breaking Changes from 2.x]
|{xpackref}/setup-upgrade.html[Upgrading to X-Pack 5.0]
|Kibana
|{kibanaref}/breaking-changes.html[Breaking Changes from 4.x]
|{kibanaref}/upgrading-kibana.html[Upgrading to Kibana 5.0]
|Logstash
|{logstashref}/breaking-changes.html[Breaking Changes from 2.x]
|{logstashref}/upgrading-logstash.html[Upgrading to Logstash 5.0]
|Beats
|{beatsref}/breaking-changes.html[Breaking Changes from 1.x]
|{beatsref}/upgrading.html[Upgrading to Beats 5.0]
|Elasticsearch Hadoop
|{hadoopref}/breaking-changes.html[Breaking Changes from 2.x]
|{hadoopref}/install.html[Upgrading to Elasticsearch Hadoop 5.0]
|===

[[upgrade-elastic-stack-for-elastic-cloud]]
=== Upgrading on the Elastic Cloud

The Elastic Cloud supports upgrading to Elasticsearch 5.0, Kibana 5.0, and X-Pack 5.0. Unlike the above
instructions, the Elastic Cloud handles the actual upgrade process as part of its managed service,
which means that you only need to be worried about breaking changes.

You can read more about the
{cloudref}/_upgrading_to_elasticsearch_5_0.html[Elastic Cloud upgrade process here].