Neighbour dispute discovered before exchange

I am due to exchange on a house and everything was on track to complete before stamp duty increase before the end of March. At the last minute I was told that the neighbour is stating that the garden fence is on his land and wants it moved. My solicitor advised that I do not complete until the dispute is resolved. I asked the seller for a £5k discount to cover the cost of dealing with the dispute post-sale but the seller does not want to accept this and instead he is only willing to offer £1500 to cover the cost of moving the fence. I think this is too low given I need to deal with a difficult neighbour and I may need legal advice. Should I move forward with the sale with the dispute or hold firm and ask for a discount?