Die back on birch trees

I have five birch trees in my garden planted six years ago when they were about five foot tall.Two have developed really well, three less so. Over the last two years the lower branches failed to grow leaves, this year the smaller trees have grown no leaves and many branches have become dry and brittle. Now even the larger trees have apparently dead lower branches and the leaves are less numerous and smaller than they ought to be. Adjacent rowan,and white beam seem to be thriving.Any suggestions please, south facing site altitude 800 ft.