Re: Am I being unreasonable?
To me, the issues start when someone feels they need to hide something. Before that point, it's innocent; but people know when they're doing something wrong (otherwise it wouldn't be wrong). Intention is everything, because it defines the respect he has for you.
For example, I've had plenty of male friends through the course of my marriage. And that is okay because I have always been happy to tell my husband where I was headed and who with, and haven't said or done anything I felt a need to hide.
It's important to note that this means anything I might want to hide, because I've realized a few times that I've told someone something my husband doesn't know yet (even something as innocuous as how I feel about spanakopita or something), and there are a few possible responses to that situation. One is to keep it between you and that person, making them a new confidante. And the other is to make a point to bring it up to your spouse so it's not something you're hiding. If I honestly don't realize my husband doesn't know how I feel about spanakopita, it doesn't matter. But if I do realize, I feel a need to tell him. The details might not matter (who really cares how much I love spinach pastries?), but the fact that I share more with him than anyone else does matter.
In other words, I think the concerning thing here is that your husband is hiding his phone and not openly sharing all the details of the time he spends with this woman, or what they talk about.
It is possible that you spurred this dynamic by bothering him about spending time with her, but even if that's the case, he should have stopped to talk to you about it back then. The fact that he's hiding things means he knows you'll be upset by something he's saying or doing, and that's not how you treat someone you care about.