I've played some indie horror games and they always have the standard tropes of running away and hiding from enemies, finding clues and solving puzles, resource management, atmosphere and audio etc.

I would like to find out more about creative gimmicks that could be implemented in a horror game which would captivate the feeling of horror, helplessness and tension.

There is the old Lovecraftian “losing sanity” instead of hit points theme. The scarier and longer you are exposed to horror, the lower your sanity score goes until you reach zero, go insane and its game over.

One interesting idea could be messing with the player's controls. Imagine if movement or vision suddenly got reversed or slowed down during tense moments. It could create confusion and fear without needing combat.

Non euclidean geometry should work well. Like that infinite hotel corridor trope. Or like in that John Carpenter movie: Guy drives out of the cursed village finally, but then he realizes all that's in front of him is the same village again.