There are several factors that will prevent an intruder from stealing your gem:

1. Lock

Upgrade the lock on your door to increase the number of keyholes in it. This will make it harder for attackers to break into your dungeon. To make an upgrade, tap the castle icon in your dungeon (or simply swipe down). 

You'll be taken to the upgrade screen. Locate the lock icon and tap "Upgrade". All upgrades take time and require gold coins. 

2. Traps

Use Edit Mode to design the best defense for your dungeon, or buy the pre-made design with the best defense right out of the box. You can change the type of traps you have at any time by tapping the "Change Traps" icon in the Edit Mode.
Upgrade your traps to increase their attack damage by tapping on them. Attackers will lose more health each time they die, and may even leave your dungeon empty-handed eventually.

3. Gem Defense Chance

If an intruder reaches your totem, the game is not over! They need to spin a spinner and land the pointer in the green zone to steal a gem.
Upgrade your Gem Defense Chance to lower the chances of the attacker of stealing a gem from your totem. This can be done in the same menu as the lock upgrade, but you will need to unlock this perk first.