I can’t speak for all sites, but when I asked Mermet Springs about it they aren’t "Holding your card hostage" but verifying that you didn’t drown. :-)
When they have 150-200 divers in the water they know everyone is out of the water safely when they have no cards left to hand back out which makes sense to me.
I guess your car sitting in the parking lot could be a clue as well, but this way they know your name, and the name of your dive buddy b/c their card should be there as well.
It’s a good security precaution and unless they have a horrible check-in/check-out process shouldn’t be that big of an issue.
P.S. I’ve been to many bars that also hold either your CC or DL if you want to run a tab so that does happen as well.