Hi,
For apigee hybrid 1.9.x do nodes requires taints ? document did not indicate them, let me know if missing anything.
We can remove taints and tolerations for apigee nodes (in a cluster) installation either for dev or higher environments ?
Listed link does not indicates about same https://cloud.google.com/apigee/docs/hybrid/v1.9/configure-dedicated-nodes
Do see taints for nodes attached for apigee nodes from cluster, hence when installing apigee get listed error.
1)kubectl get nodes
ip-xx.yy.dd.mm.ec2. Ready <none> 2d2h v1.23.x-eks-xyz
ip-xx.yy.dd.mm1.ec2.Ready <none> 2d2h v1.23.x-eks-xyz
ip-xx.yy.dd.mm3.ec2. Ready <none> 2d2h v1.23.x-eks-xyz
kubectl describe nodes <node name> -0 yaml shows taints attached to it
2) kubectl get pod -n apigee
apigee-ingressgateway-manager-xyz 0/3 Pending 0 20h
apigee-ingressgateway-manager-xyz1 0/3 Pending 0 20h
3) describing each pod resulted in following
cluster-autoscaler pod didn't trigger scale-up: 1 node(s) had taint {xyz.com/usage: my-services}, that the pod didn't tolerate, 4 node(s) didn't match Pod's node affinity/selector
Warning FailedScheduling default-scheduler 0/3 nodes are available: 1 node(s) didn't match Pod's node affinity/selector, 2 node(s) had taint {xyz.com/usage: my-services}, that the pod didn't tolerate.
Normal NotTriggerScaleUp cluster-autoscaler pod didn't trigger scale-up: 4 node(s) didn't match Pod's node affinity/selector, 1 node(s) had taint {xyz.com/usage: my-services}, that the pod didn't tolerate
The affinity settings ensure pods are scheduled to the correct node pool. It's recommended not to remove these settings.
Agree with affinity settings but does same applies to taints and tolerations recommendations for apigee hybrid node cluster ?
It's not recommended to remove the taints and tolerations settings.
@dhtx Listed doc gives importance of scheduling using listed information.
1) The nodeSelector config section has a property called requiredForScheduling: If set to false (the default), underlying pods will be scheduled whether or not node pools are defined with the required names.
This list best practice as
Note: The best practice is to set this value to requiredForScheduling:true for a production environment.
2) Docs referenced for creation of cluster (either GKE cluster or EKS cluster, which references to Amazon link) covers required fundamental cluster requirements and configuration,
https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
Recommendation for Apigee hybrid with taint-and-toleration with cluster with respect to apigee hybrid cluster was not clear.