Back bonding is a type of resonance. I can explain it through an example.
See, in compounds like BF3, the boron atom has an incomplete octet. The fluorine atom on its side has a lone pair which it can donate to boron. But, flurorine is also a very electronegative element. So, it also has a tendency to take back the electrons that it had donated to boron. This way, the lone pair of electrons keep jumping between fluorine and boron. This is called back bonding. This provides the lone pair of electrons more number of exchange positions (which simply means more space). As a result, the molecule becomes more stable.
However, back bonding is effective only when the size of the valence shell matches. In the case of BF3, both boron and fluorine have their valence electrons in 2p. But in BBr3, lone pair electrons are in 4p while valence electrons of Boron are in 2p. So, the size does not match. Also, electronegativity of the halogen decreases down the group. Hence, effectiveness of back bonding with Boron decreases down the halogen group.