Part 5 in a series by Mel Gorman describes how to measure the potential benefit from hugepages. The results match up reasonably well with CPU-intensive synthetic benchmarks on linux, which tend to show 10-15% improvement over ordinary pages.
The larger impact may be to application environments under heavy memory pressure. The OS can swap everything else out, but hugepage allocations are pinned. This is a double-edged sword. Preventing swapping may benefit some environments (cf vm.swappiness). But imagine a situation where you have 8-GB RAM and designate 4-GB for huge pages, but only use 2-GB. Now the OS has only 4-GB to manage, and the free 2-GB in huge pages are effectively wasted. If the system comes under memory pressure, that could lead to swapping or activate the OOM killer.