Introduction

Argo CD synchronizes Kubernetes applications with Git repositories, ensuring declarative configuration management. However, when resources are removed from the Git repository but remain in the Kubernetes cluster, sync failures, namespace drift, or conflicts with new deployments may occur. These orphaned resources can lead to security risks, increased resource consumption, and unintended application behavior. This article explores the causes, debugging techniques, and solutions to prevent stale or orphaned resources from disrupting Argo CD sync.

Common Causes of Stale or Orphaned Resources

1. Argo CD Not Pruning Deleted Resources

By default, Argo CD does not automatically delete resources removed from Git unless explicitly configured.

Solution: Enable Automatic Pruning

argocd app set my-app --auto-prune

Alternatively, enable pruning in `Application` manifests:

syncPolicy:
  automated:
    prune: true

2. Resources Created Outside Argo CD

Manually created Kubernetes resources may conflict with Argo CD-managed applications.

Solution: Enable `selfHeal` Mode

syncPolicy:
  automated:
    selfHeal: true

3. Argo CD Losing Track of Resource Ownership

Changes in resource annotations or namespace scope can cause Argo CD to no longer manage certain resources.

Solution: Manually Delete Orphaned Resources

kubectl delete all -l app.kubernetes.io/instance=my-app

4. Namespace-Level Resource Drift

Argo CD syncs application resources but may not clean up entire namespaces.

Solution: Use Namespace Scoping

argocd app set my-app --dest-namespace my-namespace

5. Application Sync Failing Due to Finalizers

Some Kubernetes resources (e.g., CRDs, PVCs) use finalizers that prevent deletion.

Solution: Remove Finalizers Manually

kubectl patch resource my-resource -p '[{"op": "remove", "path": "/metadata/finalizers"}]' --type=json

Debugging Stale or Orphaned Resources

1. Checking Argo CD Application Status

argocd app get my-app

2. Identifying Orphaned Resources in the Cluster

kubectl get all -n my-namespace | grep -v argo

3. Viewing Argo CD Logs for Sync Errors

kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller

4. Checking Resource Annotations for Argo CD Ownership

kubectl get deployment my-app -o jsonpath='{.metadata.annotations}')

5. Listing Resources Pending Deletion

kubectl get all -o json | jq '.items[] | select(.metadata.deletionTimestamp != null)'

Preventative Measures

1. Always Enable Pruning to Remove Stale Resources

argocd app set my-app --auto-prune

2. Use Labels to Identify Argo CD-Managed Resources

metadata:
  labels:
    argocd-managed: "true"

3. Regularly Audit Kubernetes Namespaces

kubectl get namespaces --show-labels

4. Ensure Argo CD Permissions Include Namespace Cleanup

kubectl apply -f rbac.yaml

5. Set Up Argo CD Sync Alerts

argocd notifications create my-app --alert-type sync-failure

Conclusion

Stale or orphaned resources in Argo CD can cause sync failures, namespace drift, and security risks. By enabling auto-pruning, self-healing, and auditing cluster resources, developers can maintain a clean and predictable GitOps workflow. Debugging tools like `argocd app get`, Kubernetes logs, and resource annotations help identify and resolve orphaned resources effectively.

Frequently Asked Questions

1. Why are my deleted resources still present in the cluster?

Argo CD does not automatically prune resources unless `prune: true` is enabled.

2. How do I clean up orphaned resources in Argo CD?

Use `kubectl delete` to manually remove resources not tracked by Argo CD.

3. Can Argo CD detect manually created Kubernetes resources?

Yes, enabling `selfHeal: true` ensures Argo CD reconciles unintended changes.

4. How do I resolve Argo CD sync failures due to finalizers?

Manually remove finalizers using `kubectl patch` to force deletion.

5. What’s the best way to monitor stale resources?

Regularly audit namespaces and enable Argo CD sync alerts to track uncleaned resources.