I have noticed that everyone is generally offering topical applications and i hope something works for you, it sounds awful. I am thinking from the cause point of view, if the swelling is creating the pain it would be an idea to reduce the swelling. There are plenty of natural anti-inflammatory remedies the best one being fish oil, whether you want to eat lots of fish or get a good quality supplement (check with your doctor first as it can affect some medication). The aloe vera juice someone mentioned would also be good, ginger, tumeric, pineapple and plenty of fresh fruit and vegetables (home grown or organic preferably) - as much as possible, lots of juices, soups and smoothies. Reducing meat, dairy, sugar, and refined carbohydrates as these can agrivate the inflammation. Vitamin C is really good for healing also.
I am also doubtful about unnecessary antibiotics, I think they do more harm than good. If you are on them permanently this can really destroy your natural flora, which are really helpful in reducing inflammation. It is possible to take natural flora (probiotics) while taking antibiotics, it just needs to be seperated by at least 3 hours, but i don't know how much help it would be while you are still taking antibiotics as this is usually done to help healthy colonisation once you come off. You might need to get yourself healthy first.
As you mentioned it has happened before, I wondered if you would like to prevent the ulcers in the first place? It might be an idea to seek advice from a natural practitioner.