There are an estimated 2.4 million UK citizens who live elsewhere in the EU.
All the different sources acknowledge that the numbers in France and Spain are significantly understated probably by as much as 100%. That would add another 400-600,000 to that figure.
In addition there are in France alone 200,000 properties owned by UK citizens as second/holiday homes. Given that many of these holiday homes are owned by a couple/family and some are jointly owned by more than one family that probably represents another .5 million people. The figures for Spain are thought to be similar if not higher.
That adds up to 3 to 4 million people or 4-6% of the UK population.
To put it another way, if they all lived in one city it would be the size of Bristol and not far short of the size of Manchester.
In addition to the UK citizens who will be directly and personally affected there are another 2.4 million people from elsewhere in the EU living in the UK.
That adds up to 5 to 6 million people or somewhere around the population of Leeds.