Hi
I am trying to upgrade APIGEE hybrid system from 1.7.3 to the 1.8.5 version. Components of apigee system always get stuck at the "creating" state except for cassandra . Status of the pods looks healthy and always starts up as expected.
NAME STATE AG
apigeedatastore.apigee.cloud.google.com/default running 13d
NAME STATE AGE
apigeetelemetry.apigee.cloud.google.com/apigee-telemetry creating 19m
After further investigation. I found the following error (listed below). Did anyone come across this issue before? is so please share any insight on how you managed to fix this issue. Thank you in advance
Output of kubectl -n apigee-system logs apigee-controller-manager-90905 manager.
{"level":"error","ts":1679375329.0994973,"caller":"common/error.go:91","msg":"error retrieving replica status from ReplicaSet, error updating ApigeeDeployment status, Operation cannot be fulfilled on apigeedeployments.apigee.cloud.google.com \"apigee-metrics-apigee-telemetry-app\": the object has been modified; please apply your changes to the latest version and try againerror retrieving replica status from ReplicaSet, error updating ApigeeDeployment status, Operation cannot be fulfilled on apigeedeployments.apigee.cloud.google.com \"apigee-metrics-apigee-telemetry-app\": the object has been modified; please apply your changes to the latest version and try again","controller":"ApigeeDeployment","kind":"ApigeeDeployment","apiVersion":"apigee.cloud.google.com/v1alpha3","Namespace":"apigee","Name":"apigee-metrics-apigee-telemetry-app","resourceVersion":"386782475","uid":"4d83e971-b1a2-45e6-882a-735ceeaf8355","stacktrace":"edge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers/common.HandleError\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/common/error.go:91\nedge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers.(*ApigeeDeploymentReconciler).reconcileReplicaSet\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/apigeedeployment_controller.go:117\nedge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers.(*ApigeeDeploymentReconciler).Reconcile\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/apigeedeployment_controller.go:109\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:121\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:320\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:234"}
{"level":"info","ts":1679375329.0996196,"caller":"controllers/apigeedeployment_controller.go:99","msg":"reconciling apigee deployment apigee-metrics-apigee-telemetry-app ...","controller":"ApigeeDeployment","kind":"ApigeeDeployment","apiVersion":"apigee.cloud.google.com/v1alpha3","Namespace":"apigee","Name":"apigee-metrics-apigee-telemetry-app","resourceVersion":"386785885","uid":"4d83e971-b1a2-45e6-882a-735ceeaf8355"}
{"level":"error","ts":1679375329.2091553,"caller":"common/error.go:91","msg":"error creating K8S HorizontalPodAutoscaler, the server could not find the requested resourceerror creating K8S HorizontalPodAutoscaler, the server could not find the requested resource","controller":"ApigeeDeployment","kind":"ApigeeDeployment","apiVersion":"apigee.cloud.google.com/v1alpha3","Namespace":"apigee","Name":"apigee-metrics-apigee-telemetry-app","resourceVersion":"386785885","uid":"4d83e971-b1a2-45e6-882a-735ceeaf8355","stacktrace":"edge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers/common.HandleError\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/common/error.go:91\nedge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers.(*ApigeeDeploymentReconciler).syncK8SHPA\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/apigeedeployment_controller.go:600\nedge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers.(*ApigeeDeploymentReconciler).syncADReplicaSet\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/apigeedeployment_controller.go:418\nedge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers.(*ApigeeDeploymentReconciler).stateCreatingReplicaSet\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/apigeedeployment_controller.go:189\nedge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers.(*ApigeeDeploymentReconciler).reconcileReplicaSet\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/apigeedeployment_controller.go:126\nedge-internal.git.corp.google.com/k8s-controllers.git/provisioning/controllers.(*ApigeeDeploymentReconciler).Reconcile\n\t/go/src/edge-internal/k8s-controllers/provisioning/controllers/apigeedeployment_controller.go:109\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:121\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:320\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:234"}
LinkedIn
Twitter
