While 4GB is possible, it’s on the tight end of things. A Celeron processor I’d say is out of the question.
My advice would be at least an i5/2.0GHz+ CPU with 8GB of RAM and an SSD instead of a regular hard drive (and not a hybrid SSD/HD either).
Finding something in the price bracket might prove to be a challenge. Since you don’t need fancy graphics or sound facilities, when shopping don’t look for those features as you will pay for them at the expense of getting better components that you DO need.
The SSD is something you can swap later yourself if that helps spread the budget pain. All you usually need is a screwdriver and some patience or someone that can do it for you :-).
.
