There are two ways to get a gem:

1. Steal it from another player

From your dungeon, tap the "Attack" button, and then tap "Find a Match". You will then be able to attack a random player's dungeon.

 

You can skip the current dungeon by tapping the "Skip" button. Note that each skip will cost you some gold. The price depends on your level, but can't be more than 300 coins per skip.

From your dungeon, tap the "Attack" button. On the right side of the screen, there is a map with single player levels. You can unlock mines by playing in single player. Mines are represented by a wagon icon. Each mine will give you a gem once in a while.

However, the gems that can be found in mines are not very valuable (under 5000), so stealing gems is still the most efficient way to increase your rating.