Grouping your resources when you have more than "just a few" is critical to maintaining understanding of your system, as well as allowing you to think about the resources by their responsibility rather than individual names or IDs.
You should consider making and maintaining a living document with labels you use and their meaning and intentions. I prefer to do this in a README.md in the deploy directory where I keep the Kubernetes declarations, and find that whatever conventions you set are critical to understand, especially when you are working as part of a team. Even if you work alone, this is an excellent practice: what is obvious to you today may be completely obscure to future you in six months or even longer.
