Since my Apple Distribution signing certificate had expired I recently got a new one via https://developer.apple.com/account/resources/certificates/list and installed in on my login keychain.
Since I had some issues with signing I suspected that codesign might still be trying to use an old expired certificate (as they have the same name "Apple Distribution: <Name> (<ID>)"). So to fix this I figured I could just delete the old expired certificate from Keychain Access so there was only the valid new certificate there with the same name.
However, after doing this and trying to use it with codesign I get the following error
error: The specified item is no longer valid. It may have been deleted from the keychain.
In other words it seems it's not finding the new valid certificate and somehow still linking the name to the old certificate that it rightly guesses is removed.
Following the tips from https://developer.apple.com/forums/thread/701514 I used
security find-identity -p codesigning -v to check for installed codesigning certificates, and this listed the new certificate as expected.
Using another tip in the same post I saw that you can also use the certificate hash as an identifier beside the name, and using this I can use it with codesign to sign. However, it's still not finding it via the name (or rather, it's still finding the old now removed one).
What could be the reason for codesign not finding the correct new valid certificate based on the name and instead still finding the old one? Maybe there's some reference set somewhere to point the name towards specifically the old certificate?
I figured out the cause for the problem, and in case anyone else has the same issue I'll post it here:
The problem was something called an Identity Preference that you can create in Keychain Access. To simplify things basically it can be used as an alias where a specific identity name will point to a specific certificate. And it turns out that when codesign selects an identity based on a name it will first look for matching identity preferences.
This can be useful (I think this was my case) where you have two valid certificates with the same name. In this case using the name will not be possible in codesign as it will find both and not know which one to pick. So you can create an identity preference with the certificate name and point it to the certificate you want to use. This is likely what I did a couple of year ago.
The problem is that these identity preferences persist even when the certificate they point to become invalid or are removed from the keychain. So like described above the certificate name will become unusable and not update if you renew your certificates. To complicate things these identity preferences are not easy to find as they're not shown in the certificate list, or in any other separate list to indicate they exist.
But once I figured out this might be the issue I was able to use the search feature in Keychain Access to search the all items tab using the certificate name and found that there was indeed an identity preference there with the same name that I could then remove.
It would have saved a lot of work if Keychain Access more clearly showed these identity preferences existed, or even better removed or did not use ones that pointed to removed certificates.