I also have my own mail server, and I have the same problem.
I'm guessing this is machine learning algorithms confusing correlation with causation once again. Being a small private mail server is strongly correlated with being a spammer. So even if you are not relying any spam, you certainly look like someone that would. And even if you have not done it yet, you will probably in the near future, so they might as well treat you as a spammer right away to avoid potential problems. (analogy with racism intended).
I'm guessing this is machine learning algorithms confusing correlation with causation once again. Being a small private mail server is strongly correlated with being a spammer. So even if you are not relying any spam, you certainly look like someone that would. And even if you have not done it yet, you will probably in the near future, so they might as well treat you as a spammer right away to avoid potential problems. (analogy with racism intended).