diff --git a/source/_static/images/deployment-planner-diagram.png b/source/_static/images/deployment-planner-diagram.png new file mode 100644 index 0000000000..a041ff0b26 Binary files /dev/null and b/source/_static/images/deployment-planner-diagram.png differ diff --git a/source/_static/images/import-vm-from-vmware-to-kvm-options.png b/source/_static/images/import-vm-from-vmware-to-kvm-options.png index dd54249dc1..147776de0f 100644 Binary files a/source/_static/images/import-vm-from-vmware-to-kvm-options.png and b/source/_static/images/import-vm-from-vmware-to-kvm-options.png differ diff --git a/source/_static/images/import-vm-from-vmware-to-kvm-tasks.png b/source/_static/images/import-vm-from-vmware-to-kvm-tasks.png new file mode 100644 index 0000000000..2e14b9e2a0 Binary files /dev/null and b/source/_static/images/import-vm-from-vmware-to-kvm-tasks.png differ diff --git a/source/adminguide/accounts.rst b/source/adminguide/accounts.rst index e23d9fe6e1..f57f67064b 100644 --- a/source/adminguide/accounts.rst +++ b/source/adminguide/accounts.rst @@ -868,7 +868,7 @@ global settings are available to configure SMTP for password recovery. `You have requested to reset your password. Please click the following link to reset your password:`` - `http://{{{resetLink}}}` + `{{{resetLink}}}` `If you did not request a password reset, please ignore this email.` @@ -879,6 +879,12 @@ global settings are available to configure SMTP for password recovery. - Template of mail sent to the user to reset ACS user's password. This uses mustache template engine. Available variables are: `username`, `firstName`, `lastName`, `resetLink`, `token`. + * - ``user.password.reset.mail.domain.url`` + - `null` + - Domain URL (along with scheme - `http://` or `https://` and port as applicable) + for reset password links sent to the user via email. If this is not set, CloudStack + would determine the domain url based on the first management server from 'host' setting + and http scheme based on the https.enabled flag from server.properties file in the management server. Once the global settings are configured, follow the below steps to reset the diff --git a/source/adminguide/deployment_planners.rst b/source/adminguide/deployment_planners.rst new file mode 100644 index 0000000000..64cee50d90 --- /dev/null +++ b/source/adminguide/deployment_planners.rst @@ -0,0 +1,99 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information# + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + + +Deployment Planners +====================== + + +Deployment planners determine *how and where instances* are placed across clusters within a zone. +A planner builds and orders a *list of candidate clusters* based on a placement strategy such as available capacity, user dispersion, or pod concentration. +This ordered list is then passed to the *host allocator*, which attempts to deploy the instance following the planner’s priority order. + +Administrators can configure the global setting ``vm.deployment.planner`` to define the default deployment planner for the environment. +This can also be overridden per *Compute Offering*, allowing flexible control over how instances are distributed across the infrastructure. + +Available Planners +------------------ + +FirstFitPlanner +~~~~~~~~~~~~~~~ + +The ``FirstFitPlanner`` ranks all clusters in a zone by their *available (free) capacity*, placing clusters with the most available resources at the top of the list. +This approach prioritizes capacity-driven placement, ensuring efficient utilization of resources across the zone. + +UserDispersingPlanner +~~~~~~~~~~~~~~~~~~~~~ + +The ``UserDispersingPlanner`` aims to *spread a user’s instances across multiple clusters*, reducing the impact of any single cluster failure on that user. + +#. The planner counts the number of instances in the *Running* or *Starting* state for the user’s account in each cluster. +#. Clusters are sorted in **ascending order** based on this count, so clusters with fewer instances from the user are preferred. +#. The global setting ``vm.user.dispersion.weight`` (default: ``1``) controls how strongly dispersion affects ordering: + * ``1``: Ranking is based entirely on dispersion. + * ``< 1``: Available capacity has more influence in placement decisions. + +Lowering the dispersion weight allows a balance between *even distribution* and *efficient capacity usage*. + +UserConcentratedPodPlanner +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``UserConcentratedPodPlanner`` focuses on *pod-level affinity*, preferring pods where the user already has active instances. + +#. The planner identifies all pods in the zone that contain *Running* instances for the user’s account. +#. Pods are sorted in **descending order** by the number of user instances — pods with more user instances come first. +#. Clusters from these pods are then added to the top of the list in that order, so deployment is biased toward pods where the user is already active. +#. Clusters within each pod are *not* further sorted by capacity or instance count. +#. If no pods contain user instances, the cluster order remains unchanged. + +Summary of Planner Behavior +--------------------------- + +.. list-table:: + :header-rows: 1 + + * - Planner + - Placement Focus + - Ordering Criteria + - Typical Use Case + * - FirstFitPlanner + - Capacity + - Descending by available resources + - Capacity-optimized or general-purpose placement + * - UserDispersingPlanner + - Dispersion + - Ascending by user instance count (optionally weighted with capacity) + - Distribute user instances evenly across clusters + * - UserConcentratedPodPlanner + - Pod Affinity + - Descending by user instance count per pod + - Keep user instances within the same pod for locality or data proximity + +Pod-Level vs Cluster-Level Allocation +------------------------------------ + +When ``apply.allocation.algorithm.to.pods = true``: +The allocation algorithm (for example, *FirstFit*) is applied at *pod granularity* first. +The planner will evaluate or rank pods according to the allocation heuristics — for *FirstFit*, that means prioritizing pods with more available capacity according to the FirstFit capacity checks. +After pods are ordered, the planner then considers clusters *inside each pod* — typically evaluating clusters within the selected pod in order (or applying cluster-level heuristics only within that pod). +In other words, *pod-level ordering happens before cluster selection*. + +When ``apply.allocation.algorithm.to.pods = false`` (the default in many deployments): +The allocation algorithm operates at the *cluster level* across the entire zone. + +|deployment-planner-diagram.png| + +.. |deployment-planner-diagram.png| image:: /_static/images/deployment-planner-diagram.png + :alt: Deployment Planner Diagram diff --git a/source/adminguide/index.rst b/source/adminguide/index.rst index 040ad1cd67..1c12ffa9a1 100644 --- a/source/adminguide/index.rst +++ b/source/adminguide/index.rst @@ -138,6 +138,7 @@ Managing VM and Volume Allocation .. toctree:: :maxdepth: 4 + deployment_planners host_and_storage_tags arch_types vm_volume_allocators diff --git a/source/adminguide/virtual_machines/importing_vmware_vms_into_kvm.rst b/source/adminguide/virtual_machines/importing_vmware_vms_into_kvm.rst index 84848fed26..84843962a8 100644 --- a/source/adminguide/virtual_machines/importing_vmware_vms_into_kvm.rst +++ b/source/adminguide/virtual_machines/importing_vmware_vms_into_kvm.rst @@ -154,10 +154,29 @@ Converting and importing a VMware VM .. note:: You can configure the parallel import of VM disk files on KVM host and management server, using the global settings: threads.on.kvm.host.to.import.vmware.vm.files and threads.on.ms.to.import.vmware.vm.files respectively. -In the UI import wizard, you can optionally select a KVM host and temporary destination storage (default is Secondary Storage, but if using Primary Storage - only NFS pools are supported) for the conversion, where VM files (OVF) will be copied to. This can be done by a random (or explicitly chosen) KVM host (if the ovftools are installed), otherwise, the management server will export/copy the VM files (optionally, you can force this action to be done by the management server even the KVM hosts have the ovftools installed in it). Irrelevant if the KVM host or the management server performs the copy of the VM files (OVF), you can further either let CloudStack choose which KVM host should do the conversion of the VM files using virt-v2v and which host will import the files to the destination Primary Storage Pool, or you can explicitly choose these KVM hosts for each of the 2 mentioned operations. - |import-vm-from-vmware-to-kvm-options.png| +In the UI import wizard, administrators can select: + + - (Optional) A KVM host to perform the conversion (must have virt-v2v installed). In case it is not set, then a KVM host is randomly selected. + - (Optional) A KVM host to import the converted files into CloudStack (this host must have access to the temporary conversion storage in order to move the converted files to the destination storage). In case it is not set, then a KVM host is randomly selected. + - (Optional) Extra parameters: in case the global setting **convert.vmware.instance.to.kvm.extra.params.allowed** (disabled by default) is enabled, then administrators are allowed to pass extra parameters for the virt-v2v conversion command. This setting must be set along with the setting **convert.vmware.instance.to.kvm.extra.params.allowed.list** (empty by default) which indicates the list of parameters that CloudStack will accept for passing to the virt-v2v conversion command on the KVM hosts. + +Since version 4.22 it is possible to indicate converting to storage pool directly (not using temporary storage for conversion). This is set to false by default, in which case a temporary conversion storage is used. + + - When set to false (temporary storage used), then administrators must select a Temporary destination storage. The default is Secondary Storage, but if using Primary Storage - only NFS pools are supported. + - When set to true (not using temporary storage), then administrators must select the destination storage pool. The supported storage pools are: NFS, Local Storage and SharedMountPoint storage pools, under the following assumptions: + - The KVM host for conversion must be selected and must have access to the destination storage + - The KVM host for importing must be selected and must have access to the destination storage + - In case of Local storage, the selected KVM host for conversion must be the same as the KVM host for importing + +Since version 4.22.1 it is possible to select the Guest OS for the VM to be imported, based on the source VMware VM Guest OS. + + - When CloudStack has Guest OS mappings for the source VMware Guest OS VM, then the list of supported Guest OS is displayed and administratos can select one of them. + - In case there are no Guest OS mappings for the source VMware Guest OS VM, then the default import template Guest OS will be used. + +The conversion is performed on a random (or explicitly chosen) KVM host (if the ovftools are installed), otherwise, the management server will export/copy the VM files (optionally, you can force this action to be done by the management server even the KVM hosts have the ovftools installed in it). Irrelevant if the KVM host or the management server performs the copy of the VM files (OVF), you can further either let CloudStack choose which KVM host should do the conversion of the VM files using virt-v2v and which host will import the files to the destination Primary Storage Pool, or you can explicitly choose these KVM hosts for each of the 2 mentioned operations. + When importing an instance from VMware to KVM, CloudStack performs the following actions: - Export the VM files (OVF) of the instance to a temporary storage location @@ -195,6 +214,15 @@ When importing an instance from VMware to KVM, CloudStack performs the following .. note:: The resulting imported VM uses the default Guest OS type: **CentOS 4.5 (32-bit)**. After importing the VM, please Edit the Instance to change the Guest OS Type accordingly. +VM Import Tasks +--------------- + +Since version 4.22 administrators can monitor the VMware to KVM migration jobs. The new section is displayed on *Tools > Import-Export Instances > Migrate existing instances to KVM > VM Import Tasks*: + +|import-vm-from-vmware-to-kvm-tasks.png| + +The tasks can be filtered by state: Completed, Running or Failed, or listing all the tasks. + .. |import-vm-from-vmware-to-kvm.png| image:: /_static/images/import-vm-from-vmware-to-kvm.png :alt: Import VMware Virtual Machines into KVM. :width: 800 px @@ -202,3 +230,7 @@ When importing an instance from VMware to KVM, CloudStack performs the following .. |import-vm-from-vmware-to-kvm-options.png| image:: /_static/images/import-vm-from-vmware-to-kvm-options.png :alt: Import VMware Virtual Machines into KVM Options. :width: 800 px + +.. |import-vm-from-vmware-to-kvm-tasks.png| image:: /_static/images/import-vm-from-vmware-to-kvm-tasks.png + :alt: Listing Importing Tasks to Migrate VMware Virtual Machines into KVM. + :width: 800 px diff --git a/source/adminguide/vm_volume_allocators.rst b/source/adminguide/vm_volume_allocators.rst index c15ebd8796..dded93ec03 100644 --- a/source/adminguide/vm_volume_allocators.rst +++ b/source/adminguide/vm_volume_allocators.rst @@ -37,14 +37,14 @@ VM allocator supports following algorithms to select a host in the cluster: .. cssclass:: table-striped table-bordered table-hover ============================= ======================== -Algorithm Description +Algorithm Description ============================= ======================== -random Selects a host in the cluster randomly. -firstfit Selects the first available host in the cluster. +random Selects a host in the cluster randomly. +firstfit Selects the first available host in the cluster. userdispersing Selects the host running least instances for the account, aims to spread out the instances belonging to a single user account. -userconcentratedpod_random Selects the host randomly aiming to keep all instances belonging to single user account in same pod. -userconcentratedpod_firstfit Selects the first suitable host from a pod running most instances for the user. -firstfitleastconsumed Selects the first host after sorting eligible hosts by least allocated resources (such as CPU or RAM). +userconcentratedpod_random Behaves same as random algorithm. +userconcentratedpod_firstfit Behaves same as firstfit algorithm. +firstfitleastconsumed Selects the first host after sorting eligible hosts by least allocated resources (such as CPU or RAM). ============================= ======================== Use global configuration parameter: @@ -62,14 +62,14 @@ Volume allocator supports following algorithms to select a host in the cluster: .. cssclass:: table-striped table-bordered table-hover ============================= ======================== -Algorithm Description +Algorithm Description ============================= ======================== -random Selects a storage pool in the cluster randomly. -firstfit Selects the first available storage pool in the cluster. -userdispersing Selects the storage pool running least instances for the account, aims to spread out the instances belonging to a single user account. -userconcentratedpod_random Selects the storage pool randomly aiming to keep all instances belonging to single user account in same pod. -userconcentratedpod_firstfit Selects the first suitable pool from a pod running most instances for the user. -firstfitleastconsumed Selects the first storage pool after sorting eligible pools by least allocated resources. +random Selects a storage pool in the cluster randomly. +firstfit Selects the first available storage pool in the cluster. +userdispersing Selects the storage pool running least instances for the account, aims to spread out the instances belonging to a single user account. +userconcentratedpod_random Behaves same as random algorithm. +userconcentratedpod_firstfit Behaves same as firstfit algorithm. +firstfitleastconsumed Selects the first storage pool after sorting eligible pools by least allocated resources. ============================= ======================== .. note:: @@ -98,11 +98,11 @@ Key: `host.capacityType.to.order.clusters` .. cssclass:: table-striped table-bordered table-hover ========= ======================== -Value Behavior +Value Behavior ========= ======================== -CPU Prioritizes resources with the most available CPU. -RAM Prioritizes resources with the most available memory. -COMBINED Uses a weighted formula to balance CPU and RAM in prioritization. +CPU Prioritizes resources with the most available CPU. +RAM Prioritizes resources with the most available memory. +COMBINED Uses a weighted formula to balance CPU and RAM in prioritization. ========= ======================== **Additional Configuration for COMBINED** @@ -132,8 +132,9 @@ Example Configuration Above config prioritizes CPU at 70% weight and RAM at 30% when ranking pods, clusters, and hosts. .. note:: - - `host.capacityType.to.order.clusters` is only respected for host ordering when: + - `host.capacityType.to.order.clusters` is only respected for cluster/host ordering when: .. code:: bash + vm.deployment.planner: FirstFitPlanner, UserDispersingPlanner (when vm.user.dispersion.weight is < 1) vm.allocation.algorithm: firstfitleastconsumed - When using COMBINED, make sure to tune cpu.to.memory.capacity.weight to reflect your environment’s resource constraints and workload profiles. diff --git a/source/conceptsandterminology/choosing_deployment_architecture.rst b/source/conceptsandterminology/choosing_deployment_architecture.rst index 55f673a05f..0f847dc897 100644 --- a/source/conceptsandterminology/choosing_deployment_architecture.rst +++ b/source/conceptsandterminology/choosing_deployment_architecture.rst @@ -299,7 +299,8 @@ Setup Best Practices - Use multiple clusters per pod if you need to achieve a certain switch density. -- Primary storage mountpoints or LUNs should not exceed 6 TB in size. +- Primary storage mountpoints or LUNs should be limited to a manageable + capacity as CloudStack does not impose any limit on them. It is better to have multiple smaller primary storage elements per cluster than one large one. diff --git a/source/releasenotes/about.rst b/source/releasenotes/about.rst index 89c68b9ea9..d521ad4708 100644 --- a/source/releasenotes/about.rst +++ b/source/releasenotes/about.rst @@ -17,26 +17,26 @@ What's New in |release| ======================= -Apache CloudStack |release| is a 4.22 LTS release with 10 new features, +Apache CloudStack |release| is a LTS release with 10 new features, around 15 improvements and more than 140 bug fixes since the 4.21.0.0 release. Some of the highlights include: -Enhanced Backup and Disaster Recovery -SSL Offloading for Load Balancers -Baremetal/MaaS Extension -CSI Driver for CKS -Console Access for Proxmox in Extensions Framework -VMware-to-KVM Migration Enhancements -Snapshot/Backup Schedule Listing -Per-Zone Console Proxy Configuration -Direct Volume Migration within Cluster -Persistent KVM Domains -Support for userdata on System VMs -EL10 & OpenSUSE 15.6 Platform Support -Stronger Checksum Algorithm (SHA-512) -Enable KVM volume and VM snapshot by default -Support xz format for template registration -Support for shared Filesystem on Config Drive Networks +• Enhanced Backup and Disaster Recovery +• SSL Offloading for Load Balancers +• Baremetal/MaaS Extension +• CSI Driver for CKS +• Console Access for Proxmox in Extensions Framework +• VMware-to-KVM Migration Enhancements +• Snapshot/Backup Schedule Listing +• Per-Zone Console Proxy Configuration +• Direct Volume Migrations by-passing Secondary Storage +• Persistent domain for unmanaged KVM instances from CS +• Support for User Data on System VMs +• EL10 & OpenSUSE 15.6 Platform Support +• Stronger Checksum Algorithm (SHA-512) +• Enable KVM Volume and VM snapshot by default +• Support XZ compression format for template registration with KVM +• Support for Shared Filesystem on Networks with Config Drive Known Issues ------------ diff --git a/source/upgrading/upgrade/_log4j_file_check.rst b/source/upgrading/upgrade/_log4j_file_check.rst new file mode 100644 index 0000000000..c8aa002165 --- /dev/null +++ b/source/upgrading/upgrade/_log4j_file_check.rst @@ -0,0 +1,26 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information# + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. sub-section included in upgrade notes. + +.. note:: + + During upgrades from versions prior to 4.20, the logging configuration file may not be migrated automatically to the new Log4j2 format - especially if the original log4j configuration file was manually customized or modified. + + It is strongly recommended to verify **before starting the Management Server and the Usage Server** that the configuration file (e.g. `log4j-cloud.xml`) under `/etc/cloudstack/management` and `/etc/cloudstack/usage` respectively uses the Log4j2 format. + + If the file still uses legacy Log4j (version 1) syntax or structure, **manually replace or update** the configuration using the default Log4j2 configuration supplied with the latest package. + + Failure to update may result in missing or incomplete log generation after upgrade. \ No newline at end of file diff --git a/source/upgrading/upgrade/upgrade-4.20.rst b/source/upgrading/upgrade/upgrade-4.20.rst index 083ac2e81e..dacf9a1ca6 100644 --- a/source/upgrading/upgrade/upgrade-4.20.rst +++ b/source/upgrading/upgrade/upgrade-4.20.rst @@ -207,6 +207,8 @@ Setup the GPG public key if you wish to enable ``gpgcheck=1``: $ sudo yum upgrade cloudstack-usage +.. include:: _log4j_file_check.rst + .. _upg_hyp_414: Upgrade Hypervisors diff --git a/source/upgrading/upgrade/upgrade-4.21.rst b/source/upgrading/upgrade/upgrade-4.21.rst index 0449de1645..441990c851 100644 --- a/source/upgrading/upgrade/upgrade-4.21.rst +++ b/source/upgrading/upgrade/upgrade-4.21.rst @@ -207,6 +207,8 @@ Setup the GPG public key if you wish to enable ``gpgcheck=1``: $ sudo yum upgrade cloudstack-usage +.. include:: _log4j_file_check.rst + .. _upg_hyp_414: Upgrade Hypervisors diff --git a/source/upgrading/upgrade/upgrade-4.22.rst b/source/upgrading/upgrade/upgrade-4.22.rst index 38ad410ecb..333fa2aaca 100644 --- a/source/upgrading/upgrade/upgrade-4.22.rst +++ b/source/upgrading/upgrade/upgrade-4.22.rst @@ -216,6 +216,8 @@ Setup the GPG public key if you wish to enable ``gpgcheck=1``: $ sudo yum upgrade cloudstack-usage +.. include:: _log4j_file_check.rst + .. _upg_hyp_414: Upgrade Hypervisors