I’ve always resented Father’s Day. Although my father is, as far as I know, alive and well, his relationship with me ended abruptly when I was six and my parents divorced. I am now in my forties, and have seen him once since then, spoken to him maybe a dozen times on the phone and exchanged a few letters over the years. As a child, I interpreted his absence as an active dislike of me, his first-born child. As an adult, I have tried to be more rational about why he did what he did, but still my gut feeling is that he never cared about me.

My mum is English and my father Pakistani. They met while studying in Yorkshire — he had come over to the UK. When they both graduated, they moved to London, where I was born, and my sister two years later. When I was three, we moved to Pakistan. My mum found the culture shock of life in hectic, poverty-stricken Karachi, compared to life in the Yorkshire village where she grew up, too much to bear. She begged him to bring us all back to the UK, but he was happy to be home and refused to budge. She then fell in love with a friend of the family, so it was with him that my mum, my sister and I returned to London, not my dad.

The silence from my father throughout my growing-up years was deafening. My life has been full of sadness at being cheated of what is the rightful inheritance of every child — the unconditional love of both parents. The problem with never seeing my dad is that I idealised — and idolised — him. Every time I had a falling-out with my mum or stepfather, I would think, daddy wouldn’t let this happen to me, daddy would be nicer to me than they are. He became, in my childish imagination, a glamorised amalgam of movie-star looks (a young Omar Sharif) and Santa Claus kindliness, with the uncanny ability to know me even better than I knew myself.

I hoarded the few photos we had of him and stared at them intently, seeing my face reflected in his. I loved it when relatives told me I looked like him. I’d pore over maps, tracing the distance between London and Karachi, between my idol and me. But my dad quickly remarried, this time to a much younger Pakistani woman, who obligingly produced three sons for him in rapid succession. I was just a girl, far away on the other side of the world, and he had no time for me. I’d lie in bed at night, filled with a horrible loneliness, besotted with this vague image based on my idea of the perfect dad.

He did not know when my birthday was. I received the occasional card in celebration of Eid, in which he wrote “To my dear daughter, from your father”, and that was it. His rare letters told blandly of family news, of people who meant nothing to me. I’d open them in heart-stopping excitement, thinking this would be the one in which he declared his love. I was convinced he wanted to come to the UK to take care of me. But why was he taking so long?

He never asked about my life and, on the rare occasions when he rang, conversation was forced and stilted. My hopes were shattered every time by the silence, interspersed with the toe-curlingly embarrassing phone conversations and stiff, formal letters.

My 16th birthday passed without acknowledgment from him, as did my 18th and my 21st. I grew angry: why didn’t he try harder to keep in touch? I would have gone to the ends of the earth for my own flesh and blood. But, as I grew older, a cold realisation settled in my heart. My fallible father stopped writing and phoning, and I stopped caring about him. My heart turned to stone. As I grew up, I told people that he was a selfish man who had forgotten all about my existence.

I married my first boyfriend, whom I met at university. He was emotionally distant — just like my dad. After my divorce from him, I was attracted to men who treated me shabbily. I wanted men who did not want me, repeating the pattern I had learnt growing up. Love meant suffering. Love meant never having a deep intimacy, which I secretly craved, but which I was also terrified of, because that meant allowing myself to trust a man who might end up leaving me, opening the raw wound again.

Then I fell in love with a divorcé with two daughters he adored, and saw at first hand what a father’s love can do for a girl. Both of them are self-confidently making their way in the world. My boyfriend is interested in their lives, and they have daily phone conversations. They tell him everything and he listens closely, giving considered opinions, financial assistance, fun, praise and, most of all, loving attention. Confusingly, while I admire their relationship, I also have to fight bad feelings. Why, when I had felt like the most unloved daughter in the world growing up, have I ended up with a man who treats his own so well, as if to rub my nose in the loss I have carried around all my life?

I have a thing about father figures, and was attracted to him because he provides what I crave, which is to be looked after almost paternally by a loving and loyal man. Yet I constantly test the boundaries, seeing how far I can go before he leaves, too. This has caused heartbreak for both of us, and we see a counsellor to try to make sense of my need to push him away in order to protect myself.

I saw my father again five years ago, when he was briefly in London. I felt as if my heart would burst with everything I wanted to say. He looked so old, with hunched shoulders and a shuffling walk. But he still had those sparkling dark-brown eyes, like Omar Sharif. I wanted to trace my fingers over his face, to make up for all the years I had wanted to be with him. I wanted to hug him, to know the smell of him. I craved the simple things, to be able to introduce him to my friends and say proudly: “This is my dad.” I’ve never had that opportunity and, in the end, I was so scared that if I did cuddle him, I’d never be able to stop myself crying. So, instead, I treated him like a stranger, with polite distance, my face a mask. He responded in kind. I let the opportunity I had dreamt of all my life slip through my fingers.

I think I must have been born the archetypal daddy’s girl, instinctively drawn more to him than to my mother. My tragedy was that I had a father in name only. Yet, stubbornly and idiotically, I still crave that simple but vital connection to the man who made me. Dad must be coming up to 80 now, and my thoughts have been turning to seeing him again, maybe for the last time.

The older I get, the more I try to come to terms with why he acted as he did. The geographical distance between us was great; my step dad actively discouraged him from staying in touch; and society believed that “out of sight, out of mind” was the best way to deal with absent parents. We were doomed from the start.

I think a lot about going over to Karachi, but tell myself not to expect too much from him if I do. The temptation is to pour out all my years of locked-in grief and to ask how he could have abandoned me. But I sense that if I were emotional, he would retreat even further into platitudes and clichés. Making a scene would only make things more uncomfortable between us, and I would be left wondering why I had bothered going all that way for nothing. The trick is to expect little, not a dramatic resolution, and to make do with small gestures and quiet words.

My father was a man who found fatherhood difficult, and probably did the best he was able to, in difficult circumstances. The only way I can let go of my pain now is to forgive him by acknowledging that fact. I have spent my life as the little girl whose daddy never loved her. Perhaps it is time for me to grow up and move on.