Pooja_ i had to go to their office. They didn't come to my home.
It depends on district though. Each DM/collector office has freedom to choose how they do verification (but they can't make you undress or have you do something to proove transness etc, so don't worry. Your affidavit is for that.). It's just checking you are real person behind application.
I'd recommend paying them a visit directly if you're worried, so they don't even have to think about coming to see you. Or giving them a call to ask procedure, they'll help.