Why does Boron have a smaller ionization energy than Beryllium?

I know that the atoms in the top right of the periodic table have higher ionization energy and those in the lower left have smaller ionization energy, so you would think B would have a larger ionization energy when following this rule. Why is this an exception for these two atoms?